Compare commits

4 Commits
Author SHA1 Message Date
clawbot 61802bfcf4 Add a Download button that exports a database target's archive as gzipped JSON (closes #374)
check / check (push) Successful in 3m36s
Each database target on the webhook page links to
/hook/ID/targets/TARGETID/download, which streams the target's archive
as archive-WEBHOOK-TARGET-YYYYMMDDTHHMMSSZ.json.gz: the webhook, the
target, exported_at, and archived_events, one object per row keyed by
column, a body that is not valid UTF-8 in base64 with body_encoding.
The export reads through one cursor inside a read-only transaction:
one snapshot, no write lock. A download runs for as long as the client
keeps reading, and one that fails after it has started aborts the
connection. The request limit answers 504 only when the handler has
not started its response, so a long download is logged as its 200.

Model: opus-5-5
2026-10-02 16:54:15 +00:00
clawbot 40f59ec4d2 Try every event database file on delete and say which was left (closes #275)
check / check (push) Successful in 3m20s
DeleteDB returned on the first event database file it failed to remove, so with WAL sidecars present a failure could be reported as "the file was left behind" after the events themselves were already gone. It now tries the database file and both sidecars, and its error says which case happened and names each file left: the database file is still on disk, or it is gone (the events are lost) and only a -wal or -shm sidecar remains. The webhook delete handler logs a different message for each and never says the events survive when they do not. Tests cover a normal delete with both sidecars, and a failed removal of the database file or a sidecar, through DeleteDB and the handler.

Model: opus-5-5
2026-10-02 18:42:29 +02:00
clawbot 806c95e305 Make every clickable control look clickable, in two shared styles (closes #375)
check / check (push) Successful in 3m16s
Many controls were plain coloured text with no sign they could be clicked: Edit, Activate, Deactivate, Delete, Replay, Resubmit, both Add controls and the copy control beside each entrypoint URL. Every clickable control is now a real button or link in one of two shared styles: the buttons in input.css, now with a pointer cursor, and btn-small, a small bordered secondary action in style.css, which the layout now loads. The site name, the footer links and each webhook list card show at rest that they open something, with focus states. The copy control shows "Copied" after a click. Rows on the webhook page and in the event log wrap at phone width. The browser test covers the copy control.

Model: opus-5-5
2026-10-02 18:32:19 +02:00
clawbot 45bd7e9b94 Pin the close before the reopen in the archive sweep (closes #103)
check / check (push) Successful in 3m35s
The archive sweep closes a target's archive connection before it reopens the file, and no test noticed if that close was removed, which would leak one SQLite connection per target per sweep. A test now keeps the connection from before a sweep that reopens the archive and requires it closed afterwards. The sweeper's query for database targets takes the sweep's context; a sweep whose context is done returns without an error line, so stopping is not logged as a failure, and a test pins that. The comment on the sweeper's cancel function gives the true reason it needs no lock: fx calls the stop hook only after the start hook has returned.

Model: opus-5-5
2026-10-02 18:16:31 +02:00
33 changed files with 1948 additions and 112 deletions
+36 -10
View File
@@ -1326,15 +1326,16 @@ A browser test in `internal/server` loads the webhook page and the event log
under the real policy and checks that: both add forms stay hidden until Add is
clicked; choosing Slack in the add target form leaves the HTTP fields out of
what it submits, also after leaving the page and going back to it, when the
browser restores the choice; an event expands and collapses, and so do a
delivery's attempts inside it; and at phone width the menu button opens and
closes the mobile menu. It also fails if the browser reports a console warning
or error, an uncaught exception, or anything the policy refused. `make check`
and the image build lint it but do not run it, and `make test` leaves it out
(its file is built only with the `browser` build tag). Run it with
`make test-browser` after changing `templates/` or `static/js/`: that builds
`Dockerfile.browser`, which runs the test in a digest-pinned headless browser
image, so the host needs no browser.
browser restores the choice; the Copy button beside an entrypoint URL reads
"Copied" once clicked; an event expands and collapses, and so do a delivery's
attempts inside it; and at phone width the menu button opens and closes the
mobile menu. It also fails if the browser reports a console warning or error,
an uncaught exception, or anything the policy refused. `make check` and the
image build lint it but do not run it, and `make test` leaves it out (its file
is built only with the `browser` build tag). Run it with `make test-browser`
after changing `templates/` or `static/js/`: that builds `Dockerfile.browser`,
which runs the test in a digest-pinned headless browser image, so the host
needs no browser.
The package's tarball is committed as `3p/alpinejs-csp-3.14.9.tgz`, byte for
byte as the npm registry publishes it. It is a dependency, not this repo's build
@@ -2030,6 +2031,29 @@ Because each `database` target has its own archive file, a target's
webhook with different expiries keep two archives, each pruned on its
own schedule.
Each `database` target on the webhook page has a **Download** button,
which returns its archive as one gzipped JSON file,
`archive-{webhook_name}-{target_name}-{YYYYMMDDTHHMMSSZ}.json.gz`, the
names made safe as above and the time in UTC. The file holds one
object: `webhook` and `target`, each an `id` and a `name`;
`exported_at`; and `archived_events`, one object per archived row with
every column, keyed by column name. A body that is not valid UTF-8 is
written in base64, with `"body_encoding": "base64"` beside it. An
archive that does not exist yet, or was moved away, downloads with an
empty `archived_events`; the download never creates the file.
The download streams: each row is read and written out compressed
before the next is read, so neither the archive nor the JSON is held in
memory. It reads on a connection of its own, inside one read-only
transaction, so the file holds the archive as it stood when the
download started, and archive writes go on meanwhile, since under WAL a
reader never blocks a writer. While it runs, the `-wal` cannot be
checkpointed past what it reads, so a long download lets the `-wal`
grow. It finds the file by the stored names under the lock that webhook
edits, target edits and target creation hold, and lets go once the file
is open: a rename during the download moves the file without affecting
it.
Deleting a webhook releases its archives: the delivery engine's cached
archive writers are dropped and their file handles closed, so nothing
lingers after the webhook is gone. The archive **files themselves are
@@ -2897,6 +2921,7 @@ returns to the page that was asked for.
| `POST` | `/hook/{id}/targets` | Add target to webhook |
| `GET` | `/hook/{id}/targets/{targetID}/edit` | Edit target form. The one page that renders a target's destination URL and header values in full, rather than masked |
| `POST` | `/hook/{id}/targets/{targetID}/edit` | Edit target submission |
| `GET` | `/hook/{id}/targets/{targetID}/download` | Download a `database` target's archive as one gzipped JSON file. See [Database Architecture](#database-architecture) |
| `POST` | `/hook/{id}/targets/{targetID}/delete` | Delete a target |
| `POST` | `/hook/{id}/targets/{targetID}/toggle` | Enable or disable a target |
@@ -2978,6 +3003,7 @@ webhooker/
│ │ ├── target_slack.go # Slack/Mattermost incoming-webhook target
│ │ ├── target_database.go # Database archive target
│ │ ├── target_database_archive.go # Archive file lifecycle and pruning
│ │ ├── target_database_export.go # Archive download as gzipped JSON
│ │ ├── target_log.go # Log target (stdout)
│ │ ├── target_config_view.go # Masked target config for templates
│ │ ├── archive_sweeper.go # Periodic pruning of idle archives
@@ -3026,7 +3052,7 @@ webhooker/
│ ├── static.go # //go:embed directive
│ ├── css/input.css # Tailwind input, source for tailwind.css (make css)
│ ├── css/tailwind.css # Generated stylesheet the pages load
│ ├── css/style.css # Older hand-written stylesheet, no longer loaded
│ ├── css/style.css # Hand-written, loaded after tailwind.css: btn-small, the pointer cursor for input.css's buttons, the webhook list cards' focus outline
│ ├── js/app.js # Copy-to-clipboard, and the Alpine.js components
│ └── js/alpine.min.js # Alpine.js CSP build, extracted from 3p/ by make assets, not committed
├── templates/ # Go HTML templates (base, login, sources, etc.)
+44 -10
View File
@@ -33,6 +33,19 @@ var errInvalidCachedDBType = errors.New(
"invalid cached database type",
)
// ErrEventDBNotRemoved is in DeleteDB's error when the event
// database file itself could not be removed: it is still on disk.
var ErrEventDBNotRemoved = errors.New(
"event database file not removed",
)
// ErrSidecarNotRemoved is in DeleteDB's error when the event
// database file was removed, so its events are gone, but its -wal
// or -shm sidecar could not be.
var ErrSidecarNotRemoved = errors.New(
"event database file removed, but a -wal or -shm sidecar was not",
)
// WebhookDBManager manages per-webhook SQLite database files
// for event storage. Each webhook gets its own dedicated
// database containing Events, Deliveries, DeliveryResults and the
@@ -151,7 +164,10 @@ func (m *WebhookDBManager) DBExists(
}
// DeleteDB closes the connection and deletes the database file
// for a webhook. The file is permanently removed.
// for a webhook, with its -wal and -shm sidecars. The files are
// permanently removed. Each file is tried even when another could
// not be removed, and the error wraps ErrEventDBNotRemoved or
// ErrSidecarNotRemoved to say which was left, naming each file.
func (m *WebhookDBManager) DeleteDB(
webhookID string,
) error {
@@ -170,16 +186,23 @@ func (m *WebhookDBManager) DeleteDB(
}
}
// Delete the main DB file and WAL/SHM files
path := m.dbPath(webhookID)
for _, suffix := range []string{"", "-wal", "-shm"} {
err := os.Remove(path + suffix)
if err != nil && !os.IsNotExist(err) {
return fmt.Errorf(
"deleting webhook database file %s%s: %w",
path, suffix, err,
)
}
dbErr := removeFile(path)
sidecarErr := errors.Join(
removeFile(path+"-wal"),
removeFile(path+"-shm"),
)
if dbErr != nil {
return fmt.Errorf(
"%w: %w",
ErrEventDBNotRemoved, errors.Join(dbErr, sidecarErr),
)
}
if sidecarErr != nil {
return fmt.Errorf("%w: %w", ErrSidecarNotRemoved, sidecarErr)
}
m.log.Info(
@@ -190,6 +213,17 @@ func (m *WebhookDBManager) DeleteDB(
return nil
}
// removeFile removes path. A file that is already gone counts as
// removed; the error from any other failure names the file.
func removeFile(path string) error {
err := os.Remove(path)
if errors.Is(err, os.ErrNotExist) {
return nil
}
return err
}
// CloseAll closes all open per-webhook database connections.
// Called during application shutdown.
func (m *WebhookDBManager) CloseAll() error {
+77 -3
View File
@@ -182,17 +182,91 @@ func TestWebhookDBManager_DeleteDB(t *testing.T) {
}
require.NoError(t, db.Create(event).Error)
// Under WAL, an open database that has been written to has both
// sidecars beside it.
dbPath := mgr.DBPath(webhookID)
require.FileExists(t, dbPath+"-wal")
require.FileExists(t, dbPath+"-shm")
// Delete the DB
require.NoError(t, mgr.DeleteDB(webhookID))
// File should no longer exist
assert.False(t, mgr.DBExists(webhookID))
// Verify the file is actually gone from disk
// Verify the files are actually gone from disk
assert.NoFileExists(t, dbPath)
assert.NoFileExists(t, dbPath+"-wal")
assert.NoFileExists(t, dbPath+"-shm")
}
// blockRemoval puts a non-empty directory at path, which os.Remove
// cannot remove whoever runs the test, root included.
func blockRemoval(t *testing.T, path string) {
t.Helper()
require.NoError(t, os.MkdirAll(filepath.Join(path, "keep"), 0o700))
}
// TestWebhookDBManager_DeleteDBKeepsDatabaseFile proves that when the
// event database file cannot be removed, the error says so, and both
// sidecars are still removed.
func TestWebhookDBManager_DeleteDBKeepsDatabaseFile(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhookID := uuid.New().String()
dbPath := mgr.DBPath(webhookID)
_, err = os.Stat(dbPath)
assert.True(t, os.IsNotExist(err))
blockRemoval(t, dbPath)
require.NoError(t, os.WriteFile(dbPath+"-wal", nil, 0o600))
require.NoError(t, os.WriteFile(dbPath+"-shm", nil, 0o600))
err := mgr.DeleteDB(webhookID)
require.ErrorIs(t, err, database.ErrEventDBNotRemoved)
require.NotErrorIs(t, err, database.ErrSidecarNotRemoved)
assert.Contains(t, err.Error(), dbPath)
assert.NoFileExists(t, dbPath+"-wal")
assert.NoFileExists(t, dbPath+"-shm")
}
// TestWebhookDBManager_DeleteDBKeepsSidecar proves that when the
// event database file is removed but a sidecar is not, the error
// says the database file is gone, and the other sidecar is still
// removed.
func TestWebhookDBManager_DeleteDBKeepsSidecar(t *testing.T) {
t.Parallel()
mgr, lc := setupTestWebhookDBManager(t)
ctx := context.Background()
require.NoError(t, lc.Start(ctx))
defer func() { require.NoError(t, lc.Stop(ctx)) }()
webhookID := uuid.New().String()
dbPath := mgr.DBPath(webhookID)
require.NoError(t, mgr.CreateDB(webhookID))
// Closing removes the sidecars, so the ones below are the only
// ones there.
require.NoError(t, mgr.CloseAll())
blockRemoval(t, dbPath+"-wal")
require.NoError(t, os.WriteFile(dbPath+"-shm", nil, 0o600))
err := mgr.DeleteDB(webhookID)
require.ErrorIs(t, err, database.ErrSidecarNotRemoved)
require.NotErrorIs(t, err, database.ErrEventDBNotRemoved)
assert.Contains(t, err.Error(), dbPath+"-wal")
assert.NoFileExists(t, dbPath)
assert.NoFileExists(t, dbPath+"-shm")
}
func TestWebhookDBManager_LazyCreation(t *testing.T) {
+15 -2
View File
@@ -45,8 +45,13 @@ type ArchiveSweeper struct {
eng *Engine
log *slog.Logger
interval time.Duration
cancel context.CancelFunc
wg sync.WaitGroup
// cancel needs no lock: fx calls the stop hook only after the
// start hook has returned, so stop never reads it while start
// is still setting it.
cancel context.CancelFunc
wg sync.WaitGroup
}
// NewArchiveSweeper creates the archive sweeper and registers
@@ -163,10 +168,18 @@ func (s *ArchiveSweeper) sweep(ctx context.Context) {
var targets []database.Target
err := s.db.DB().
WithContext(ctx).
Model(&database.Target{}).
Where("type = ?", database.TargetTypeDatabase).
Find(&targets).Error
if err != nil {
// The app stopping as a sweep starts cancels the listing.
// Stopping is not a failure, so it must not produce an
// error line.
if ctx.Err() != nil {
return
}
s.log.Error(
"archive sweep: failed to list database targets",
"error", err,
+60
View File
@@ -1,9 +1,11 @@
package delivery_test
import (
"bytes"
"context"
"database/sql"
"fmt"
"log/slog"
"net/http"
"os"
"path/filepath"
@@ -681,6 +683,64 @@ func TestArchiveSweep_ClosesHandleOfRegisteredWriter(
)
}
// TestArchiveSweep_ClosesHandleBeforeReopening proves the sweep
// closes the handle it finds open before it reopens the file.
// TestArchiveSweep_LeavesArchiveClosed cannot see this: without the
// close, the reopen replaces the handle without closing it, the
// sweep then closes only the new one, and one connection leaks per
// archive per sweep.
func TestArchiveSweep_ClosesHandleBeforeReopening(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive.db")
w := delivery.NewExportArchiveWriter(
path, archiveTestLogger(), 0,
)
require.NoError(t, w.Open(time.Hour))
before, err := w.DB().DB()
require.NoError(t, err)
require.NoError(t, w.SweepExpired(time.Hour))
assert.Error(
t, before.PingContext(t.Context()),
"the handle open before the sweep must be closed by it",
)
}
// TestArchiveSweep_CancelledSweepLogsNoError proves a sweep whose
// context is already cancelled, as when the app stops just as a
// sweep starts, returns without an error line: stopping is not a
// failure.
func TestArchiveSweep_CancelledSweepLogsNoError(t *testing.T) {
t.Parallel()
env := setupArchiveTest(t)
var errorLines bytes.Buffer
sweeper := delivery.NewTestArchiveSweeper(
env.mainDB, env.eng,
slog.New(slog.NewTextHandler(
&errorLines,
&slog.HandlerOptions{Level: slog.LevelError},
)),
)
ctx, cancel := context.WithCancel(context.Background())
cancel()
sweeper.ExportSweep(ctx)
assert.Empty(
t, errorLines.String(),
"a cancelled sweep must not log at error level",
)
}
// TestArchiveSweep_NeverExpiryUntouched proves the sweep is a
// no-op for the default retention policy, so archives with no
// expiry (or the literal "never") behave exactly as before.
+6 -10
View File
@@ -3,7 +3,6 @@ package delivery
import (
"context"
"fmt"
"path/filepath"
"strings"
"sync"
"time"
@@ -277,10 +276,9 @@ func (t *databaseTarget) releaseSweepWriter(
}
// newWriter builds the writer for a database target's archive. The
// file lives beside the webhook's event database in the data
// directory and is named for the webhook and the target as the main
// database has them now; from then on only rename changes the name
// the writer uses. It does not touch the archive file.
// file is the one ArchivePath gives for the webhook and the target as
// the main database names them now; from then on only rename changes
// the name the writer uses. It does not touch the archive file.
func (t *databaseTarget) newWriter(
targetID string,
) (*archiveWriter, error) {
@@ -299,12 +297,10 @@ func (t *databaseTarget) newWriter(
)
}
dir := filepath.Dir(t.eng.dbManager.DBPath(target.WebhookID))
name := ArchiveFileName(
target.Webhook.Name, target.Name, target.ID,
w := newArchiveWriter(
ArchivePath(t.eng.dbManager, &target.Webhook, &target),
t.eng.log,
)
w := newArchiveWriter(filepath.Join(dir, name), t.eng.log)
w.webhookID = target.WebhookID
return w, nil
+275
View File
@@ -0,0 +1,275 @@
package delivery
import (
"compress/gzip"
"context"
"database/sql"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"log/slog"
"path/filepath"
"time"
"unicode/utf8"
"gorm.io/driver/sqlite"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/gormlog"
)
// archiveTableQuery counts the archive's table: 0 when the archive
// writer has created the file but not yet the table in it.
const archiveTableQuery = "SELECT count(*) FROM sqlite_master " +
"WHERE type = 'table' AND name = 'archived_events'"
// ArchivePath returns where a database target's archive file is: in
// the data directory, beside the webhook's event database, under the
// name ArchiveFileName gives it.
func ArchivePath(
dbMgr *database.WebhookDBManager,
webhook *database.Webhook,
target *database.Target,
) string {
return filepath.Join(
filepath.Dir(dbMgr.DBPath(webhook.ID)),
ArchiveFileName(webhook.Name, target.Name, target.ID),
)
}
// ArchiveExportFileName returns the name a database target's archive
// downloads under:
// archive-WEBHOOKNAME-TARGETNAME-YYYYMMDDTHHMMSSZ.json.gz, the names
// made safe as in ArchiveFileName and the time in UTC.
func ArchiveExportFileName(
webhookName, targetName string, at time.Time,
) string {
return "archive-" + archiveNamePart(webhookName) + "-" +
archiveNamePart(targetName) + "-" +
at.UTC().Format("20060102T150405Z") + ".json.gz"
}
// ArchiveExport is a database target's archive opened for download.
// It reads the file on its own connection, inside one read-only
// transaction, so it writes out the archive as it stood when
// OpenArchiveExport returned.
//
// Archives are in WAL mode, where a reader works from a snapshot and
// never blocks a writer: archive writes go on while an export is open,
// and the export does not see them. SQLite cannot checkpoint the -wal
// past an open snapshot, so the -wal grows until the export is closed.
type ArchiveExport struct {
db *sql.DB
tx *gorm.DB
// empty is true when there is nothing to read: no file, or a file
// without the archive's table yet.
empty bool
}
// exportedName is how an export names its webhook and its target.
type exportedName struct {
ID string `json:"id"`
Name string `json:"name"`
}
// OpenArchiveExport opens the archive file at path for export and
// takes the snapshot the export reads. It never creates the file: with
// no file at path, the export has no rows.
//
// Once it has returned, the file is open, so a rename or a move of it
// does not affect the export, which reads the same file under its new
// name.
//
// The transaction lasts as long as ctx does, so ctx must last for the
// whole export.
func OpenArchiveExport(
ctx context.Context, path string, log *slog.Logger,
) (*ArchiveExport, error) {
if !fileExists(path) {
return &ArchiveExport{empty: true}, nil
}
db, err := database.OpenSQLite(path, archiveModeExisting)
if err != nil {
return nil, fmt.Errorf("opening archive %s: %w", path, err)
}
gdb, err := gorm.Open(
sqlite.Dialector{Conn: db}, &gorm.Config{
// Never leave this at GORM's default. See
// internal/gormlog.
Logger: gormlog.New(log),
},
)
if err != nil {
_ = db.Close()
return nil, fmt.Errorf("opening archive %s: %w", path, err)
}
// ReadOnly makes the driver begin a deferred transaction in place
// of the BEGIN IMMEDIATE the connection string asks for, so the
// export never takes the archive's write lock.
tx := gdb.WithContext(ctx).Begin(&sql.TxOptions{ReadOnly: true})
if tx.Error != nil {
_ = db.Close()
return nil, fmt.Errorf("reading archive %s: %w", path, tx.Error)
}
// The transaction's first read is what takes the snapshot.
var tables int
err = tx.Raw(archiveTableQuery).Row().Scan(&tables)
if err != nil {
_ = tx.Rollback()
_ = db.Close()
return nil, fmt.Errorf("reading archive %s: %w", path, err)
}
return &ArchiveExport{db: db, tx: tx, empty: tables == 0}, nil
}
// WriteGzipJSON writes the export to w as one gzipped JSON object:
// webhook and target, each an id and a name; exported_at; and
// archived_events, one object per archived row, keyed by column name.
// A body that is not valid UTF-8 cannot be a JSON string, so it is
// written in base64, with "body_encoding": "base64" beside it.
//
// Each row is written out before the next is read, so neither the
// archive nor its JSON is ever held in memory whole. After an error
// the gzip stream is left unfinished, so what was written does not
// decompress as a whole file.
func (x *ArchiveExport) WriteGzipJSON(
ctx context.Context,
w io.Writer,
webhook *database.Webhook,
target *database.Target,
exportedAt time.Time,
) error {
head, err := json.Marshal(map[string]any{
"webhook": exportedName{ID: webhook.ID, Name: webhook.Name},
"target": exportedName{ID: target.ID, Name: target.Name},
"exported_at": exportedAt.UTC(),
})
if err != nil {
return fmt.Errorf("encoding archive export: %w", err)
}
zw := gzip.NewWriter(w)
err = x.writeJSON(ctx, zw, head)
if err != nil {
return fmt.Errorf("writing archive export: %w", err)
}
return zw.Close()
}
// Close ends the export's transaction and closes its connection.
func (x *ArchiveExport) Close() error {
if x.db == nil {
return nil
}
_ = x.tx.Rollback()
return x.db.Close()
}
// writeJSON writes head with archived_events added as its last key,
// the rows going into it one at a time.
func (x *ArchiveExport) writeJSON(
ctx context.Context, w io.Writer, head []byte,
) error {
// head goes out without its closing brace, so that
// archived_events can follow it.
_, err := w.Write(head[:len(head)-1])
if err != nil {
return err
}
_, err = io.WriteString(w, `,"archived_events":[`)
if err != nil {
return err
}
err = x.writeRows(ctx, w)
if err != nil {
return err
}
_, err = io.WriteString(w, "\n]}\n")
return err
}
// writeRows writes each archived row to w, oldest first, one per line,
// separated by commas.
func (x *ArchiveExport) writeRows(ctx context.Context, w io.Writer) error {
if x.empty {
return nil
}
rows, err := x.tx.WithContext(ctx).
Model(&archivedEvent{}).Order("id").Rows()
if err != nil {
return err
}
defer func() { _ = rows.Close() }()
for sep := "\n"; rows.Next(); sep = ",\n" {
var ev archivedEvent
err = x.tx.ScanRows(rows, &ev)
if err != nil {
return err
}
_, err = io.WriteString(w, sep)
if err != nil {
return err
}
err = writeRow(w, &ev)
if err != nil {
return err
}
}
return rows.Err()
}
// writeRow writes an archived row to w as a JSON object keyed by
// column name, its body in base64 when it is not valid UTF-8.
func writeRow(w io.Writer, ev *archivedEvent) error {
row := map[string]any{
"id": ev.ID,
"event_id": ev.EventID,
"webhook_id": ev.WebhookID,
"entrypoint_id": ev.EntrypointID,
"method": ev.Method,
"headers": ev.Headers,
"body": ev.Body,
"content_type": ev.ContentType,
"archived_at": ev.ArchivedAt.UTC(),
}
if !utf8.ValidString(ev.Body) {
row["body"] = base64.StdEncoding.EncodeToString([]byte(ev.Body))
row["body_encoding"] = "base64"
}
line, err := json.Marshal(row)
if err != nil {
return err
}
_, err = w.Write(line)
return err
}
@@ -0,0 +1,412 @@
package delivery_test
import (
"bufio"
"bytes"
"compress/gzip"
"crypto/rand"
"encoding/base64"
"encoding/json"
"fmt"
"io"
"os"
"path/filepath"
"runtime"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
// The webhook and the target the export tests' archives belong to.
const (
exportWebhookID = "wh-export"
exportWebhookName = "Orders (EU)"
exportTargetID = "tgt-export"
exportTargetName = "Long-term archive"
)
const (
// binaryBody is a body that is not valid UTF-8.
binaryBody = "\xff\xfe\x00\x01binary\x80"
// openedEventID is the event the snapshot tests archive before
// they open the export.
openedEventID = "opened"
)
// writeExportTo writes export to w as the archive of the export tests'
// webhook and target, exported at 2026-10-02T12:03:04Z.
func writeExportTo(
t *testing.T, export *delivery.ArchiveExport, w io.Writer,
) error {
t.Helper()
return export.WriteGzipJSON(
t.Context(), w,
&database.Webhook{
BaseModel: database.BaseModel{ID: exportWebhookID},
Name: exportWebhookName,
},
&database.Target{
BaseModel: database.BaseModel{ID: exportTargetID},
Name: exportTargetName,
},
time.Date(2026, 10, 2, 12, 3, 4, 0, time.UTC),
)
}
// exportArchive runs a whole export of the archive at path and returns
// its JSON, decompressed and parsed.
func exportArchive(t *testing.T, path string) map[string]any {
t.Helper()
export, err := delivery.OpenArchiveExport(
t.Context(), path, archiveTestLogger(),
)
require.NoError(t, err)
defer func() { require.NoError(t, export.Close()) }()
return writeExport(t, export)
}
// writeExport writes an opened export and returns its JSON,
// decompressed and parsed. Reading to the end makes the gzip reader
// check that the stream was finished.
func writeExport(
t *testing.T, export *delivery.ArchiveExport,
) map[string]any {
t.Helper()
var buf bytes.Buffer
require.NoError(t, writeExportTo(t, export, &buf))
zr, err := gzip.NewReader(&buf)
require.NoError(t, err)
raw, err := io.ReadAll(zr)
require.NoError(t, err)
var got map[string]any
require.NoError(t, json.Unmarshal(raw, &got))
return got
}
// exportedEvents returns an export's archived_events.
func exportedEvents(t *testing.T, got map[string]any) []map[string]any {
t.Helper()
list, ok := got["archived_events"].([]any)
require.True(t, ok, "archived_events must be an array: %v", got)
events := make([]map[string]any, len(list))
for i, v := range list {
events[i], ok = v.(map[string]any)
require.True(t, ok, "an archived event must be an object: %v", v)
}
return events
}
// exportedEventIDs returns the event_id of each of an export's
// archived_events.
func exportedEventIDs(t *testing.T, got map[string]any) []string {
t.Helper()
events := exportedEvents(t, got)
ids := make([]string, 0, len(events))
for _, ev := range events {
ids = append(ids, fmt.Sprint(ev["event_id"]))
}
return ids
}
// TestArchiveExport_MatchesStoredRows proves an export holds the
// webhook, the target, the time, and every column of every stored
// row: a body that is valid UTF-8 as a string, and one that is not in
// base64, marked as such.
func TestArchiveExport_MatchesStoredRows(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive.db")
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0)
bodies := []string{`{"order":1}`, "plain text", "", binaryBody}
for i, body := range bodies {
require.NoError(t, w.Write(delivery.ExportArchivedEvent{
EventID: fmt.Sprintf("ev-%d", i),
WebhookID: exportWebhookID,
EntrypointID: "ep-1",
Method: "POST",
Headers: `{"X-Test":["yes"]}`,
Body: body,
ContentType: testContentType,
}, 0))
}
var stored []delivery.ExportArchivedEvent
require.NoError(t, openArchiveDBForRead(t, path).
Order("id").Find(&stored).Error)
got := exportArchive(t, path)
assert.Equal(t,
map[string]any{"id": exportWebhookID, "name": exportWebhookName},
got["webhook"],
)
assert.Equal(t,
map[string]any{"id": exportTargetID, "name": exportTargetName},
got["target"],
)
assert.Equal(t, "2026-10-02T12:03:04Z", got["exported_at"])
events := exportedEvents(t, got)
require.Len(t, events, len(bodies))
for i, row := range stored {
assertExportedRow(t, row, events[i])
}
}
// assertExportedRow checks that ev, from an export, holds every column
// of the stored row.
func assertExportedRow(
t *testing.T, row delivery.ExportArchivedEvent, ev map[string]any,
) {
t.Helper()
archivedAt, err := time.Parse(
time.RFC3339Nano, fmt.Sprint(ev["archived_at"]),
)
require.NoError(t, err)
assert.True(t, archivedAt.Equal(row.ArchivedAt))
assert.EqualValues(t, row.ID, ev["id"])
assert.Equal(t, row.EventID, ev["event_id"])
assert.Equal(t, row.WebhookID, ev["webhook_id"])
assert.Equal(t, row.EntrypointID, ev["entrypoint_id"])
assert.Equal(t, row.Method, ev["method"])
assert.Equal(t, row.Headers, ev["headers"])
assert.Equal(t, row.ContentType, ev["content_type"])
if row.Body != binaryBody {
assert.Equal(t, row.Body, ev["body"])
assert.Len(t, ev, 9, "the nine columns and nothing else: %v", ev)
return
}
body, err := base64.StdEncoding.DecodeString(fmt.Sprint(ev["body"]))
require.NoError(t, err)
assert.Equal(t, binaryBody, string(body))
assert.Equal(t, "base64", ev["body_encoding"])
assert.Len(t, ev, 10, "the nine columns and body_encoding: %v", ev)
}
// TestArchiveExport_Empty proves an archive with nothing in it exports
// as an empty archived_events: no file, which the export must not
// create; a file the archive writer has not yet put its table in; and
// a table with no rows.
func TestArchiveExport_Empty(t *testing.T) {
t.Parallel()
dir := t.TempDir()
missing := filepath.Join(dir, "missing.db")
noTable := filepath.Join(dir, "no-table.db")
noRows := filepath.Join(dir, "no-rows.db")
require.NoError(t, os.WriteFile(noTable, nil, 0o600))
require.NoError(t,
delivery.NewExportArchiveWriter(noRows, archiveTestLogger(), 0).
Open(0),
)
for _, path := range []string{missing, noTable, noRows} {
assert.Empty(t, exportedEvents(t, exportArchive(t, path)), path)
}
for _, suffix := range archiveFileSuffixes() {
assert.NoFileExists(t, missing+suffix)
}
}
// TestArchiveExport_ReadsOneSnapshot proves an export writes the
// archive as it was when it was opened, and holds up no archive
// write: a row written while the export is open is stored, and is not
// in the export. A write held up for the whole busy timeout would
// fail.
func TestArchiveExport_ReadsOneSnapshot(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive.db")
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0)
require.NoError(t, w.Write(delivery.ExportArchivedEvent{EventID: openedEventID}, 0))
export, err := delivery.OpenArchiveExport(
t.Context(), path, archiveTestLogger(),
)
require.NoError(t, err)
defer func() { require.NoError(t, export.Close()) }()
require.NoError(t, w.Write(delivery.ExportArchivedEvent{EventID: "during"}, 0))
assert.Equal(t,
[]string{openedEventID}, exportedEventIDs(t, writeExport(t, export)),
)
var stored int64
require.NoError(t, openArchiveDBForRead(t, path).
Model(&delivery.ExportArchivedEvent{}).Count(&stored).Error)
assert.Equal(t, int64(2), stored)
}
// TestArchiveExport_SurvivesRename proves that renaming the archive
// while an export of it is open, as renaming its webhook or target
// does, leaves the export reading the same file.
func TestArchiveExport_SurvivesRename(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "archive-old.db")
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0)
require.NoError(t, w.Write(delivery.ExportArchivedEvent{EventID: openedEventID}, 0))
export, err := delivery.OpenArchiveExport(
t.Context(), path, archiveTestLogger(),
)
require.NoError(t, err)
defer func() { require.NoError(t, export.Close()) }()
require.NoError(t, w.Rename("archive-new.db"))
require.NoError(t, w.Write(delivery.ExportArchivedEvent{EventID: "after"}, 0))
require.NoFileExists(t, path)
assert.Equal(t,
[]string{openedEventID}, exportedEventIDs(t, writeExport(t, export)),
)
}
// heapPeak is an io.Writer that discards what it is given and records
// the largest heap it saw at a write. It collects garbage before each
// reading, so the heap it reads is what is still held.
type heapPeak struct {
max uint64
}
func (p *heapPeak) Write(b []byte) (int, error) {
var m runtime.MemStats
runtime.GC()
runtime.ReadMemStats(&m)
p.max = max(p.max, m.HeapAlloc)
return len(b), nil
}
// exportHeapGrowth exports an archive of rows random bodies, each
// bodySize bytes of base64, and returns how far the heap rose above
// where it stood when the export began, at its highest.
func exportHeapGrowth(t *testing.T, rows, bodySize int) uint64 {
t.Helper()
path := filepath.Join(t.TempDir(), "archive.db")
w := delivery.NewExportArchiveWriter(path, archiveTestLogger(), 0)
// Base64 makes four characters of every three bytes.
random := make([]byte, bodySize/4*3)
for range rows {
_, _ = rand.Read(random)
require.NoError(t, w.Write(delivery.ExportArchivedEvent{
Body: base64.StdEncoding.EncodeToString(random),
}, 0))
}
export, err := delivery.OpenArchiveExport(
t.Context(), path, archiveTestLogger(),
)
require.NoError(t, err)
defer func() { require.NoError(t, export.Close()) }()
runtime.GC()
var start runtime.MemStats
runtime.ReadMemStats(&start)
// Through a buffer, the heap is read once per 8 KiB of output
// rather than at each of gzip's small writes, which takes far
// longer.
peak := &heapPeak{max: start.HeapAlloc}
buffered := bufio.NewWriterSize(peak, 8<<10)
require.NoError(t, writeExportTo(t, export, buffered))
require.NoError(t, buffered.Flush())
return peak.max - start.HeapAlloc
}
// TestArchiveExport_Streams proves an export holds neither the archive
// nor its output in memory whole: exporting 384 KiB more of archive
// raises the heap's peak by less than half of that. The export's own
// memory, mostly gzip's compressor, is the same for both archives, so
// it cancels out. The bodies are random bytes in base64, which gzip
// shrinks by only a quarter, so an export that read every row before
// writing, or built the JSON or the gzipped file before writing it,
// would raise the peak by at least three quarters of the difference.
//
// The smaller archive has two rows so that its export, too, writes
// out more than the 8 KiB buffer in exportHeapGrowth before it ends:
// the heap must be read while the export's own memory is held.
//
//nolint:paralleltest // It measures the heap, which tests share.
func TestArchiveExport_Streams(t *testing.T) {
const (
bodySize = 16 << 10
smallRows = 2
largeRows = smallRows + 24
limit = (largeRows - smallRows) * bodySize / 2
)
small := exportHeapGrowth(t, smallRows, bodySize)
large := exportHeapGrowth(t, largeRows, bodySize)
assert.Less(t, large, small+limit,
"the heap rose by %d for %d rows and by %d for %d rows",
small, smallRows, large, largeRows,
)
}
// TestArchiveExportFileName proves the download is named for the
// webhook and the target, with the names made safe as for the archive
// file, and the export time in UTC.
func TestArchiveExportFileName(t *testing.T) {
t.Parallel()
cest := time.FixedZone("CEST", int((2 * time.Hour).Seconds()))
assert.Equal(t,
"archive-orders-eu-long-term-archive-20261002T120304Z.json.gz",
delivery.ArchiveExportFileName(
exportWebhookName, exportTargetName,
time.Date(2026, 10, 2, 14, 3, 4, 0, cest),
),
)
}
+9
View File
@@ -36,6 +36,15 @@ const MaxRenderedAttemptsForTest = maxRenderedAttempts
// the handlers enforce rather than a number copied beside it.
const MaxTargetRetriesForTest = maxTargetRetries
// EventDBLeftMsgForTest and SidecarLeftMsgForTest expose the two
// messages the webhook delete handler logs when a file of the event
// database is left on disk, so a test checking that one is absent
// checks for the handler's own wording.
const (
EventDBLeftMsgForTest = eventDBLeftMsg
SidecarLeftMsgForTest = sidecarLeftMsg
)
// PageOrFirstForTest exposes pageOrFirst for use in the handlers_test
// package.
func PageOrFirstForTest(s string) int {
+2 -1
View File
@@ -97,7 +97,8 @@ type Handlers struct {
// names through the archive rename, the save and any move back.
// Interleaved, one could rename an archive between another's
// rename and save, leaving the file named for one edit and the
// stored names from the other.
// stored names from the other. An archive download holds it while
// it reads the stored names and opens the file they give.
renameMu sync.Mutex
// dummyVerifications counts the equivalent-cost verifications
+117
View File
@@ -1,8 +1,10 @@
package handlers_test
import (
"bytes"
"context"
"errors"
"log/slog"
"net/http"
"net/http/httptest"
"os"
@@ -466,6 +468,121 @@ func TestHandleSourceDelete_RemovesConfigAndEventDatabase(
)
}
// TestHandleSourceDelete_LeftoverSidecar proves that when the event
// database file is removed but a sidecar beside it is not, the
// operator is told the events are gone, never that the event
// database file is still there.
func TestHandleSourceDelete_LeftoverSidecar(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
mgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &mgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
logs := new(bytes.Buffer)
h.SetLogForTest(slog.New(slog.NewTextHandler(logs, nil)))
wh := seedWebhook(t, db)
require.NoError(t, mgr.CreateDB(wh.ID))
// Closing removes the sidecars, so the -wal below is the only
// one there.
require.NoError(t, mgr.CloseAll())
// A non-empty directory in the -wal file's place, which
// os.Remove cannot remove whoever runs the test.
eventDBPath := mgr.DBPath(wh.ID)
require.NoError(t, os.MkdirAll(
filepath.Join(eventDBPath+"-wal", "keep"), 0o700,
))
cookies := authenticatedCookies(
t, sess, deleteTestUserID, deleteTestUsername,
)
req := postRequest(
"/hook/"+wh.ID+"/delete",
cookies,
map[string]string{paramSourceID: wh.ID},
)
w := httptest.NewRecorder()
h.HandleSourceDelete().ServeHTTP(w, req)
assert.Equal(t, http.StatusInternalServerError, w.Code)
assert.NoFileExists(t, eventDBPath)
assert.Contains(t, logs.String(), "its events are gone")
assert.Contains(t, logs.String(), eventDBPath+"-wal")
assert.NotContains(
t, logs.String(), handlers.EventDBLeftMsgForTest,
"the events are gone, so the operator must not be told "+
"the event database file survived",
)
}
// TestHandleSourceDelete_LeftoverDatabaseFile proves that when the
// event database file itself cannot be removed, the operator is told
// it is still on disk, never that its events are gone.
func TestHandleSourceDelete_LeftoverDatabaseFile(t *testing.T) {
t.Parallel()
var (
h *handlers.Handlers
sess *session.Session
db *database.Database
mgr *database.WebhookDBManager
)
app := newTestApp(t, &h, &sess, &db, &mgr)
app.RequireStart()
t.Cleanup(app.RequireStop)
logs := new(bytes.Buffer)
h.SetLogForTest(slog.New(slog.NewTextHandler(logs, nil)))
wh := seedWebhook(t, db)
// A non-empty directory in the database file's place, which
// os.Remove cannot remove whoever runs the test.
eventDBPath := mgr.DBPath(wh.ID)
require.NoError(t, os.MkdirAll(
filepath.Join(eventDBPath, "keep"), 0o700,
))
cookies := authenticatedCookies(
t, sess, deleteTestUserID, deleteTestUsername,
)
req := postRequest(
"/hook/"+wh.ID+"/delete",
cookies,
map[string]string{paramSourceID: wh.ID},
)
w := httptest.NewRecorder()
h.HandleSourceDelete().ServeHTTP(w, req)
assert.Equal(t, http.StatusInternalServerError, w.Code)
assert.Contains(
t, logs.String(), "event database file is still on disk",
)
assert.Contains(t, logs.String(), eventDBPath)
assert.NotContains(
t, logs.String(), handlers.SidecarLeftMsgForTest,
"the database file is still on disk, so the operator must "+
"not be told its events are gone",
)
}
// TestHandleTargetDelete_EvictsThatTarget proves that deleting a
// database target releases that target's archive writer and no
// other: the webhook's other database target keeps its own.
+20 -4
View File
@@ -723,6 +723,17 @@ func (h *Handlers) HandleSourceDelete() http.HandlerFunc {
}
}
// The messages deleteWebhookResources logs when a file of the event
// database cannot be removed: the database file itself, or only a
// sidecar once the database file is gone.
const (
eventDBLeftMsg = "webhook deleted, but its event database file is " +
"still on disk; remove it by hand"
sidecarLeftMsg = "webhook deleted and its events are gone, but a " +
"-wal or -shm sidecar of its event database is " +
"still on disk; remove it by hand"
)
// deleteWebhookResources soft-deletes config and hard-deletes
// the per-webhook event database.
func (h *Handlers) deleteWebhookResources(
@@ -762,13 +773,18 @@ func (h *Handlers) deleteWebhookResources(
err = h.dbMgr.DeleteDB(webhook.ID)
if err != nil {
// The configuration is committed, so the webhook is gone,
// but its event database file is still on disk with
// but a file of its event database is still on disk with
// nothing referencing it. Report the failure rather than
// redirecting as though everything succeeded: the file
// needs removing by hand, and the logged error names it.
h.serverError(
w, r, "failed to delete webhook event database", err,
)
// When only a sidecar is left, the events are already
// gone, and the message must not suggest they survive.
msg := eventDBLeftMsg
if errors.Is(err, database.ErrSidecarNotRemoved) {
msg = sidecarLeftMsg
}
h.serverError(w, r, msg, err)
return
}
+5 -1
View File
@@ -191,6 +191,7 @@ func storedRetentionDays(
type sourceTestEnv struct {
handlers *handlers.Handlers
db *database.Database
dbMgr *database.WebhookDBManager
archives *recordingArchives
cookies []*http.Cookie
}
@@ -204,9 +205,11 @@ func setupSourceTest(t *testing.T) *sourceTestEnv {
var db *database.Database
var dbMgr *database.WebhookDBManager
var archives *recordingArchives
app := newTestApp(t, &h, &sess, &db, &archives)
app := newTestApp(t, &h, &sess, &db, &dbMgr, &archives)
app.RequireStart()
t.Cleanup(app.RequireStop)
@@ -214,6 +217,7 @@ func setupSourceTest(t *testing.T) *sourceTestEnv {
return &sourceTestEnv{
handlers: h,
db: db,
dbMgr: dbMgr,
archives: archives,
cookies: authenticatedCookies(
t, sess, sourceTestUserID, "sourceuser",
+121
View File
@@ -0,0 +1,121 @@
package handlers
import (
"context"
"errors"
"net/http"
"time"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
)
// downloadWriteTimeout is how long one write of a download may wait
// for a client that has stopped reading.
const downloadWriteTimeout = 60 * time.Second
// HandleTargetDownload serves a database target's archive as one
// gzipped JSON file, named for the webhook, the target and the time;
// see delivery.ArchiveExport.WriteGzipJSON for what it holds. Other
// target types have no archive and are a 404.
//
// A download runs for as long as the client keeps reading: it reads
// under a context the request limit does not cancel, and gives each
// write its own deadline in place of the server's write timeout. It
// stops when a write fails.
func (h *Handlers) HandleTargetDownload() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
ctx := context.WithoutCancel(r.Context())
webhook, target, export, ok := h.openTargetArchive(ctx, w, r)
if !ok {
return
}
defer func() { _ = export.Close() }()
now := time.Now()
w.Header().Set("Content-Type", "application/gzip")
w.Header().Set(
"Content-Disposition",
`attachment; filename="`+delivery.ArchiveExportFileName(
webhook.Name, target.Name, now,
)+`"`,
)
err := export.WriteGzipJSON(
ctx,
downloadWriter{w: w, rc: http.NewResponseController(w)},
&webhook, target, now,
)
if err != nil {
h.log.Error(
"failed to export archive",
"target_id", target.ID,
"error", err,
)
// The 200 has gone out. Aborting the connection is what
// tells the client the file is incomplete.
panic(http.ErrAbortHandler)
}
}
}
// downloadWriter writes a download to the client, giving each write
// downloadWriteTimeout to finish.
type downloadWriter struct {
w http.ResponseWriter
rc *http.ResponseController
}
func (d downloadWriter) Write(b []byte) (int, error) {
// A writer that has no write deadline, such as a test's recorder,
// answers http.ErrNotSupported and needs none extended.
err := d.rc.SetWriteDeadline(time.Now().Add(downloadWriteTimeout))
if err != nil && !errors.Is(err, http.ErrNotSupported) {
return 0, err
}
return d.w.Write(b)
}
// openTargetArchive opens the archive of the request's database target
// for export, with its reads under ctx. It reports false once it has
// written the response.
//
// It holds renameMu, which every archive rename runs under, while it
// reads the stored names and opens the file, so the file it opens is
// the one those names give. It lets go before the export is streamed:
// once the file is open, a rename does not affect the export.
func (h *Handlers) openTargetArchive(
ctx context.Context,
w http.ResponseWriter,
r *http.Request,
) (database.Webhook, *database.Target, *delivery.ArchiveExport, bool) {
h.renameMu.Lock()
defer h.renameMu.Unlock()
webhook, target, ok := h.ownedTarget(w, r)
if !ok {
return database.Webhook{}, nil, nil, false
}
if target.Type != database.TargetTypeDatabase {
h.renderError(w, r, http.StatusNotFound)
return database.Webhook{}, nil, nil, false
}
export, err := delivery.OpenArchiveExport(
ctx, delivery.ArchivePath(h.dbMgr, &webhook, target), h.log,
)
if err != nil {
h.serverError(w, r, "failed to open archive for export", err)
return database.Webhook{}, nil, nil, false
}
return webhook, target, export, true
}
+379
View File
@@ -0,0 +1,379 @@
package handlers_test
import (
"bytes"
"compress/gzip"
"context"
"crypto/rand"
"encoding/json"
"errors"
"io"
"log/slog"
"net"
"net/http"
"net/http/httptest"
"net/url"
"sync"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/webhooker/internal/config"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/delivery"
"sneak.berlin/go/webhooker/internal/middleware"
)
// errClientGone is the write failure of a client that has gone away.
var errClientGone = errors.New("client gone")
// downloadPath is the archive download route of a target.
func downloadPath(webhookID, targetID string) string {
return "/hook/" + webhookID + "/targets/" + targetID + "/download"
}
// renameTarget submits the edit form renaming a target to Renamed.
func renameTarget(
env *sourceTestEnv, webhookID, targetID string,
) *httptest.ResponseRecorder {
form := url.Values{}
form.Set("name", "Renamed")
return submitTargetEdit(env, webhookID, targetID, form)
}
// TestHandleTargetDownload proves a database target's archive
// downloads as a gzipped JSON attachment named for the webhook, the
// target and the time, here with no archive file yet, so with no
// rows; and that a target of another type has no download.
func TestHandleTargetDownload(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
wh := seedWebhookWithRetention(t, env.db, 7)
archive := seedTarget(t, env.db, wh.ID, database.TargetTypeDatabase)
logTarget := seedTarget(t, env.db, wh.ID, database.TargetTypeLog)
w := serveTarget(
env, http.MethodGet, downloadPath(wh.ID, archive.ID), nil,
)
require.Equal(t, http.StatusOK, w.Code, w.Body.String())
assert.Equal(t, "application/gzip", w.Header().Get("Content-Type"))
assert.Regexp(t,
`^attachment; filename="archive-seeded-t-database-`+
`\d{8}T\d{6}Z\.json\.gz"$`,
w.Header().Get("Content-Disposition"),
)
zr, err := gzip.NewReader(w.Body)
require.NoError(t, err)
var got map[string]json.RawMessage
require.NoError(t, json.NewDecoder(zr).Decode(&got))
assert.JSONEq(t,
`{"id":"`+archive.ID+`","name":"t-database"}`,
string(got["target"]),
)
assert.JSONEq(t, `[]`, string(got["archived_events"]))
w = serveTarget(
env, http.MethodGet, downloadPath(wh.ID, logTarget.ID), nil,
)
assert.Equal(t, http.StatusNotFound, w.Code)
}
// TestHandleTargetDownload_WaitsForRename proves a download reads the
// target's names and opens its archive under the lock a rename holds:
// started while an edit is renaming the archive, it waits, and is
// named for the target's new name.
func TestHandleTargetDownload_WaitsForRename(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
wh := seedWebhookWithRetention(t, env.db, 7)
archive := seedTarget(t, env.db, wh.ID, database.TargetTypeDatabase)
renaming, release := env.archives.BlockNextRename()
edited := make(chan *httptest.ResponseRecorder, 1)
go func() {
edited <- renameTarget(env, wh.ID, archive.ID)
}()
<-renaming
downloaded := make(chan *httptest.ResponseRecorder, 1)
go func() {
downloaded <- serveTarget(
env, http.MethodGet, downloadPath(wh.ID, archive.ID), nil,
)
}()
select {
case <-downloaded:
release()
t.Fatal("the download did not wait for the rename")
case <-time.After(100 * time.Millisecond):
}
release()
require.Equal(t, http.StatusSeeOther, (<-edited).Code)
w := <-downloaded
require.Equal(t, http.StatusOK, w.Code)
assert.Contains(t,
w.Header().Get("Content-Disposition"), "archive-seeded-renamed-",
)
}
// stalledWriter is a response writer whose first write waits until
// resume is closed, closing writing when it starts to wait.
type stalledWriter struct {
*httptest.ResponseRecorder
once sync.Once
writing chan struct{}
resume chan struct{}
}
func (s *stalledWriter) Write(b []byte) (int, error) {
s.once.Do(func() {
close(s.writing)
<-s.resume
})
return s.ResponseRecorder.Write(b)
}
// TestHandleTargetDownload_StreamsWithoutTheLock proves a download
// lets go of the rename lock once its archive is open: while the
// download is stalled writing, an edit can still rename the target.
func TestHandleTargetDownload_StreamsWithoutTheLock(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
wh := seedWebhookWithRetention(t, env.db, 7)
archive := seedTarget(t, env.db, wh.ID, database.TargetTypeDatabase)
req := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, downloadPath(wh.ID, archive.ID), nil,
)
for _, c := range env.cookies {
req.AddCookie(c)
}
sw := &stalledWriter{
ResponseRecorder: httptest.NewRecorder(),
writing: make(chan struct{}),
resume: make(chan struct{}),
}
downloaded := make(chan struct{})
go func() {
targetRouter(env).ServeHTTP(sw, req)
close(downloaded)
}()
<-sw.writing
edited := make(chan *httptest.ResponseRecorder, 1)
go func() {
edited <- renameTarget(env, wh.ID, archive.ID)
}()
select {
case w := <-edited:
assert.Equal(t, http.StatusSeeOther, w.Code)
case <-time.After(10 * time.Second):
t.Error("the rename waited for the download")
}
close(sw.resume)
<-downloaded
assert.Equal(t, http.StatusOK, sw.Code)
}
// seedArchive writes rows to the archive file at path, each with a
// body of bodySize random bytes, which do not compress. Its table has
// only the columns the test fills; an export writes the others empty.
func seedArchive(t *testing.T, path string, rows, bodySize int) {
t.Helper()
db, err := database.OpenSQLite(path, database.SQLiteModeCreate)
require.NoError(t, err)
defer func() { require.NoError(t, db.Close()) }()
_, err = db.ExecContext(t.Context(),
"CREATE TABLE archived_events (id INTEGER PRIMARY KEY, body TEXT)",
)
require.NoError(t, err)
body := make([]byte, bodySize)
for range rows {
_, _ = rand.Read(body)
_, err = db.ExecContext(t.Context(),
"INSERT INTO archived_events (body) VALUES (?)", string(body),
)
require.NoError(t, err)
}
}
// limitedServer serves the target routes as the server does, behind the
// access log, whose lines it returns, and the request limit, here
// limit, which is also its write timeout. Each connection's send buffer
// is a few KiB, so a larger response is still being written while its
// client is not reading.
func limitedServer(
t *testing.T, env *sourceTestEnv, limit time.Duration,
) (*httptest.Server, *bytes.Buffer) {
t.Helper()
const sendBuffer = 4 << 10
logBuf := new(bytes.Buffer)
mw := middleware.NewForTest(
slog.New(slog.NewJSONHandler(logBuf, nil)),
&config.Config{Environment: config.EnvironmentDev},
nil,
)
srv := httptest.NewUnstartedServer(
mw.Logging()(mw.Timeout(limit)(targetRouter(env))),
)
srv.Config.WriteTimeout = limit
srv.Config.ConnContext = func(
ctx context.Context, c net.Conn,
) context.Context {
tcp, ok := c.(*net.TCPConn)
if assert.True(t, ok) {
assert.NoError(t, tcp.SetWriteBuffer(sendBuffer))
}
return ctx
}
srv.Start()
t.Cleanup(srv.Close)
return srv, logBuf
}
// TestHandleTargetDownload_OutlastsTheRequestLimit proves a download
// runs for as long as the client keeps reading, and is logged as the
// 200 it was. Behind a request limit and a server write timeout of a
// tenth of a second, the client stops reading once the response has
// started, waits three times as long, and still gets the whole file.
// The archive is larger than the connection holds, so the download is
// still being written while the client waits.
func TestHandleTargetDownload_OutlastsTheRequestLimit(t *testing.T) {
t.Parallel()
const (
limit = 100 * time.Millisecond
rows = 8
bodySize = 64 << 10
)
env := setupSourceTest(t)
wh := seedWebhookWithRetention(t, env.db, 7)
archive := seedTarget(t, env.db, wh.ID, database.TargetTypeDatabase)
seedArchive(
t, delivery.ArchivePath(env.dbMgr, &wh, archive), rows, bodySize,
)
srv, accessLog := limitedServer(t, env, limit)
req, err := http.NewRequestWithContext(
t.Context(), http.MethodGet,
srv.URL+downloadPath(wh.ID, archive.ID), nil,
)
require.NoError(t, err)
for _, c := range env.cookies {
req.AddCookie(c)
}
resp, err := srv.Client().Do(req)
require.NoError(t, err)
defer func() { _ = resp.Body.Close() }()
require.Equal(t, http.StatusOK, resp.StatusCode)
time.Sleep(3 * limit)
zr, err := gzip.NewReader(resp.Body)
require.NoError(t, err)
var (
got map[string]json.RawMessage
events []json.RawMessage
)
require.NoError(t, json.NewDecoder(zr).Decode(&got))
require.NoError(t, json.Unmarshal(got["archived_events"], &events))
assert.Len(t, events, rows)
// Reading to the end makes the gzip reader check that the file was
// finished.
_, err = io.ReadAll(zr)
require.NoError(t, err)
// Close waits for the handler, so the access log line is written.
srv.Close()
var access map[string]any
require.NoError(t, json.Unmarshal(accessLog.Bytes(), &access))
assert.EqualValues(t, http.StatusOK, access["status"])
assert.GreaterOrEqual(t,
access["latency_ms"], float64(limit.Milliseconds()),
"the download must outlast the request limit",
)
}
// brokenWriter is a response writer whose writes fail once the
// response has started, as they do when the client goes away.
type brokenWriter struct {
*httptest.ResponseRecorder
}
func (b brokenWriter) Write(p []byte) (int, error) {
if b.Body.Len() > 0 {
return 0, errClientGone
}
return b.ResponseRecorder.Write(p)
}
// TestHandleTargetDownload_AbortsWhenItFails proves a download that
// fails after its response has started aborts the connection, so the
// client sees a failed download rather than a file that looks
// complete and does not decompress.
func TestHandleTargetDownload_AbortsWhenItFails(t *testing.T) {
t.Parallel()
env := setupSourceTest(t)
wh := seedWebhookWithRetention(t, env.db, 7)
archive := seedTarget(t, env.db, wh.ID, database.TargetTypeDatabase)
req := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, downloadPath(wh.ID, archive.ID), nil,
)
for _, c := range env.cookies {
req.AddCookie(c)
}
w := brokenWriter{ResponseRecorder: httptest.NewRecorder()}
assert.PanicsWithValue(t, http.ErrAbortHandler, func() {
targetRouter(env).ServeHTTP(w, req)
})
assert.Equal(t, http.StatusOK, w.Code)
}
+6 -2
View File
@@ -37,14 +37,18 @@ const (
editAuthHeader = "Authorization: Bearer " + editBearerSecret
)
// targetRouter mounts the target create and edit routes on a chi
// router so the handlers see the URL parameters they read.
// targetRouter mounts the target create, edit and download routes on
// a chi router so the handlers see the URL parameters they read.
func targetRouter(env *sourceTestEnv) *chi.Mux {
router := chi.NewRouter()
router.Post(
"/hook/{sourceID}/targets",
env.handlers.HandleTargetCreate(),
)
router.Get(
"/hook/{sourceID}/targets/{targetID}/download",
env.handlers.HandleTargetDownload(),
)
router.Get(
"/hook/{sourceID}/targets/{targetID}/edit",
env.handlers.HandleTargetEdit(),
+6 -5
View File
@@ -82,9 +82,9 @@ func TestNavbarUsesWebhookTerminology(t *testing.T) {
})
assert.Contains(t, body, "Retention: 14 days")
assert.Contains(t, body, `class="btn-text">Webhooks</a>`)
assert.Contains(t, body, `class="btn-secondary">Webhooks</a>`)
assert.Contains(
t, body, `class="btn-text w-full text-left">Webhooks</a>`,
t, body, `class="btn-secondary w-full">Webhooks</a>`,
)
assert.Contains(
t, body,
@@ -163,7 +163,7 @@ func TestEventLogPageIsCalledFullEventLog(t *testing.T) {
)
assert.Contains(
t, detailBody,
`<a href="/hook/wh-1/events" class="btn-text text-sm">Full Event Log</a>`,
`<a href="/hook/wh-1/events" class="btn-small">Full Event Log</a>`,
"the link under recent events",
)
@@ -331,8 +331,9 @@ func TestEntrypointCopyButtonIsProgressiveEnhancement(t *testing.T) {
assert.Contains(t, body, "https://hooks.example.com/h/abc123")
assert.Contains(
t, body,
`hidden data-copy-target="entrypoint-url-ep-1"`,
"the button must start hidden and be revealed by script",
`<button type="button" hidden data-copy-target="entrypoint-url-ep-1"`,
"the copy control must be a button, start hidden and be "+
"revealed by script",
)
// renderTemplate streams to the ResponseWriter, so an abort
+71
View File
@@ -0,0 +1,71 @@
package middleware
import (
"context"
"errors"
"net/http"
"time"
)
// Timeout returns middleware that gives each request limit to finish:
// it cancels the request's context once limit has passed, and answers
// 504 when the handler then returns without having started its
// response.
//
// It replaces chi's middleware.Timeout, which writes that 504 even
// after the handler has sent its own status. A download that outlasts
// the limit has already sent its 200 and the whole file, so the late
// 504 changes nothing for the client: the access log and the metrics
// would record it in place of the 200, and net/http would complain of
// a superfluous WriteHeader.
func (s *Middleware) Timeout(
limit time.Duration,
) func(http.Handler) http.Handler {
return func(next http.Handler) http.Handler {
return http.HandlerFunc(func(
w http.ResponseWriter,
r *http.Request,
) {
ctx, cancel := context.WithTimeout(r.Context(), limit)
defer cancel()
tw := &timeoutResponseWriter{ResponseWriter: w}
next.ServeHTTP(tw, r.WithContext(ctx))
if !tw.started &&
errors.Is(ctx.Err(), context.DeadlineExceeded) {
w.WriteHeader(http.StatusGatewayTimeout)
}
})
}
}
// timeoutResponseWriter records whether the handler has started its
// response.
type timeoutResponseWriter struct {
http.ResponseWriter
started bool
}
func (w *timeoutResponseWriter) WriteHeader(code int) {
w.started = true
w.ResponseWriter.WriteHeader(code)
}
func (w *timeoutResponseWriter) Write(b []byte) (int, error) {
// A Write without a WriteHeader starts the response too: net/http
// sends 200 in front of it.
w.started = true
//nolint:wrapcheck // Pass the writer's own error through unchanged.
return w.ResponseWriter.Write(b)
}
// Unwrap lets http.ResponseController reach the writer underneath, so
// a handler can still set a write deadline through this wrapper.
func (w *timeoutResponseWriter) Unwrap() http.ResponseWriter {
return w.ResponseWriter
}
+56
View File
@@ -0,0 +1,56 @@
package middleware_test
import (
"net/http"
"net/http/httptest"
"testing"
"time"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
)
// TestTimeout proves the request limit answers 504 to a handler that
// outlasts it without starting its response, and leaves a response the
// handler has started with the status it sent. Both are what the
// access log records.
func TestTimeout(t *testing.T) {
t.Parallel()
const limit = 10 * time.Millisecond
for _, tc := range []struct {
name string
sent int // the status the handler sends, or 0 for none
want int
}{
{name: "not started", sent: 0, want: http.StatusGatewayTimeout},
{name: "started", sent: http.StatusOK, want: http.StatusOK},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
m, buf := capturingMiddleware(t)
handler := m.Logging()(m.Timeout(limit)(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
if tc.sent != 0 {
w.WriteHeader(tc.sent)
}
<-r.Context().Done()
},
)))
w := httptest.NewRecorder()
handler.ServeHTTP(w, httptest.NewRequestWithContext(
t.Context(), http.MethodGet, "/", nil,
))
assert.Equal(t, tc.want, w.Code)
entries := accessLogEntries(t, buf)
require.Len(t, entries, 1)
assert.EqualValues(t, tc.want, entries[0]["status"])
})
}
}
+37 -1
View File
@@ -17,6 +17,7 @@ import (
"testing"
"time"
"github.com/chromedp/cdproto/browser"
"github.com/chromedp/cdproto/log"
"github.com/chromedp/cdproto/network"
"github.com/chromedp/cdproto/runtime"
@@ -43,7 +44,7 @@ const (
// TestAlpineRunsUnderTheSecurityPolicy loads the webhook page and the
// event log in a headless browser, served by the real router and so
// under the real Content-Security-Policy, and checks that the pages'
// Alpine.js directives work.
// Alpine.js directives and the copy control work.
func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
t.Parallel()
@@ -55,6 +56,13 @@ func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
userID, _ := env.seedUser(t, "browser", "browser-password")
webhook := env.seedWebhook(t, userID)
require.NoError(t, env.db.DB().Omit(clause.Associations).Create(
&database.Entrypoint{
WebhookID: webhook.ID,
Path: "3c9e1f7a-5b2d-4e8a-9f6c-2a7d1e4b8c05",
Active: true,
},
).Error)
event := env.seedEvent(t, webhook.ID, `{"hello":"browser"}`)
target := env.seedTarget(t, webhook.ID)
dlv := env.seedFailedDelivery(t, webhook.ID, event.ID, target.ID)
@@ -77,6 +85,7 @@ func TestAlpineRunsUnderTheSecurityPolicy(t *testing.T) {
checkAddForms(ctx, t, page)
checkTargetType(ctx, t, page+"/events")
checkCopy(ctx, t, page)
checkEventLog(ctx, t, page+"/events", event.ID, target.Name)
checkMobileMenu(ctx, t, page)
@@ -220,6 +229,8 @@ func click(ctx context.Context, t *testing.T, xpath string) {
// checkAddForms loads a webhook page and checks that each section's add
// form stays hidden until the Add button beside its heading is clicked.
// The click looks for a button element there, so it also checks that
// Add is one.
func checkAddForms(ctx context.Context, t *testing.T, url string) {
t.Helper()
@@ -327,6 +338,31 @@ func checkTargetType(ctx context.Context, t *testing.T, elsewhere string) {
)
}
// checkCopy loads a webhook page and checks that the Copy control beside
// its entrypoint's URL is a button, and that clicking it copies the URL
// and says so: the button reads "Copied" only once the copy succeeded.
func checkCopy(ctx context.Context, t *testing.T, url string) {
t.Helper()
copyButton := `//button[@data-copy-target]`
// A browser lets the page in its active tab write to the clipboard
// on a click. A headless browser refuses unless told to allow it.
require.NoError(t, chromedp.Run(
ctx,
browser.SetPermission(
&browser.PermissionDescriptor{Name: "clipboard-write"},
browser.PermissionSettingGranted,
),
loadPage(url),
))
click(ctx, t, copyButton)
assert.True(t, shown(ctx, copyButton+`[text()="Copied"]`),
`clicking Copy does not show "Copied"`)
}
// checkEventLog loads the event log and checks that clicking an event's
// row expands it, that in there clicking its delivery shows the
// delivery's attempts and clicking again hides them, and that clicking
+69
View File
@@ -0,0 +1,69 @@
package server_test
import (
"compress/gzip"
"encoding/json"
"net/http"
"regexp"
"testing"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"gorm.io/gorm/clause"
"sneak.berlin/go/webhooker/internal/database"
)
// TestHook_DownloadArchive follows the Download link the webhook page
// shows for a database target, and only for it, and gets the archive
// as a gzipped JSON file. Signed out, the link leads to the login page.
func TestHook_DownloadArchive(t *testing.T) {
t.Parallel()
env := newTestEnv(t)
userID, _ := env.seedUser(t, "archivist", "somepassword")
cookies := env.authCookies(t, userID, "archivist")
wh := env.seedWebhook(t, userID)
env.seedTarget(t, wh.ID)
archive := &database.Target{
WebhookID: wh.ID,
Name: "kept",
Type: database.TargetTypeDatabase,
Active: true,
}
require.NoError(t,
env.db.DB().Omit(clause.Associations).Create(archive).Error,
)
page := env.get("/hook/"+wh.ID, cookies)
require.Equal(t, http.StatusOK, page.Code)
links := regexp.MustCompile(
`href="(/hook/[^/"]+/targets/[^/"]+/download)"`,
).FindAllStringSubmatch(page.Body.String(), -1)
require.Len(t, links, 1, "only the database target has a Download")
link := links[0][1]
assert.Equal(t,
"/hook/"+wh.ID+"/targets/"+archive.ID+"/download", link,
)
w := env.get(link, cookies)
require.Equal(t, http.StatusOK, w.Code)
assert.Equal(t, "application/gzip", w.Header().Get("Content-Type"))
zr, err := gzip.NewReader(w.Body)
require.NoError(t, err)
var got map[string]json.RawMessage
require.NoError(t, json.NewDecoder(zr).Decode(&got))
assert.JSONEq(t,
`{"id":"`+wh.ID+`","name":"routed"}`, string(got["webhook"]),
)
w = env.get(link, nil)
assert.Equal(t, http.StatusSeeOther, w.Code)
assert.Contains(t, w.Header().Get("Location"), "/pages/login")
}
+5 -1
View File
@@ -73,7 +73,7 @@ func (s *Server) setupGlobalMiddleware() {
}
s.router.Use(s.mw.CORS())
s.router.Use(middleware.Timeout(requestTimeout))
s.router.Use(s.mw.Timeout(requestTimeout))
// Panic recovery, deliberately here rather than first. It has to
// run inside every middleware that observes the response, so the
@@ -312,6 +312,10 @@ func (s *Server) setupSourceRoutes() {
"/targets/{targetID}/edit",
s.h.HandleTargetEditSubmit(),
)
r.Get(
"/targets/{targetID}/download",
s.h.HandleTargetDownload(),
)
r.Post(
"/targets/{targetID}/delete",
s.h.HandleTargetDelete(),
+2 -2
View File
@@ -1220,12 +1220,12 @@ func TestHook_LinksBetweenPages(t *testing.T) {
// mobile menu link.
{
"/user/navigator/",
`href="([^"]+)" class="btn-text">Webhooks<`,
`href="([^"]+)" class="btn-secondary">Webhooks<`,
list,
},
{
"/user/navigator/",
`href="([^"]+)" class="btn-text w-full[^"]*">Webhooks<`,
`href="([^"]+)" class="btn-secondary w-full">Webhooks<`,
list,
},
{list, `href="(/hook/[^"]+)"`, page},
+48 -1
View File
@@ -1 +1,48 @@
/* Webhooker custom styles — see input.css for Tailwind theme */
/*
* The two shared styles for the controls a user clicks. Every page loads
* this file after tailwind.css. It is plain CSS: make css does not build
* it.
*
* A button is btn-primary, btn-secondary or btn-danger, from input.css.
* A secondary or inline action, such as Copy beside an entrypoint URL or
* Edit beside a target, is btn-small.
*
* Each card on the webhook list is a card-elevated link as a whole. It
* shows an Open label in btn-small and takes btn-small's focus outline.
*
* The rules join tailwind.css's components layer, where input.css puts
* its own, so a utility class on an element still overrides them.
*/
@layer components {
/* input.css gives its buttons no pointer cursor. */
.btn-primary,
.btn-secondary,
.btn-danger {
cursor: pointer;
}
.btn-small {
display: inline-flex;
align-items: center;
padding: 0.25rem 0.625rem;
border: 1px solid var(--color-gray-300);
border-radius: var(--radius-md);
background-color: var(--color-white);
color: var(--color-primary-700);
font-size: var(--text-xs);
line-height: 1rem;
font-weight: var(--font-weight-medium);
cursor: pointer;
}
.btn-small:hover {
border-color: var(--color-primary-500);
background-color: var(--color-primary-50);
}
.btn-small:focus-visible,
.card-elevated:focus-visible {
outline: 2px solid var(--color-primary-500);
outline-offset: 2px;
}
}
+2 -2
View File
@@ -22,9 +22,9 @@
<footer class="bg-gray-100 border-t border-gray-200 shadow-[0_-4px_6px_-1px_rgba(0,0,0,0.1)] mt-8">
<div class="max-w-6xl mx-auto px-8 py-6">
<div class="text-center text-sm text-gray-500 font-mono font-light">
<a href="https://git.eeqj.de/sneak/webhooker" class="hover:text-gray-700">Webhooker</a>
<a href="https://git.eeqj.de/sneak/webhooker" class="btn-small">Webhooker</a>
<span class="mx-1">by</span>
<a href="https://sneak.berlin" class="hover:text-gray-700">@sneak</a>
<a href="https://sneak.berlin" class="btn-small">@sneak</a>
<span class="mx-3">|</span>
<span>{{if .Version}}{{.Version}}{{else}}dev{{end}}</span>
</div>
+1
View File
@@ -3,6 +3,7 @@
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>{{block "title" .}}Webhooker{{end}}</title>
<link rel="stylesheet" href="/s/css/tailwind.css">
<link rel="stylesheet" href="/s/css/style.css">
<style>[x-cloak] { display: none !important; }</style>
{{block "head" .}}{{end}}
{{end}}
+10 -10
View File
@@ -2,12 +2,12 @@
<nav class="app-bar" x-data="collapsible">
<div class="max-w-6xl mx-auto flex justify-between items-center">
<div class="flex items-center gap-3">
<a href="/" class="text-xl font-medium text-gray-900 hover:text-primary-600 transition-colors">Webhooker</a>
<a href="/" class="btn-secondary text-xl">Webhooker</a>
</div>
<!-- Mobile menu button -->
{{if .User}}
<button @click="toggle" class="md:hidden p-2 rounded-md text-gray-500 hover:bg-gray-100">
<button type="button" @click="toggle" class="btn-secondary md:hidden p-2">
<svg class="w-6 h-6" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path x-show="closed" stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M4 6h16M4 12h16M4 18h16"/>
<path x-show="open" x-cloak stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M6 18L18 6M6 6l12 12"/>
@@ -18,9 +18,9 @@
<!-- Desktop navigation -->
<div class="hidden md:flex items-center gap-4">
{{if .User}}
<a href="/hooks" class="btn-text">Webhooks</a>
<a href="/settings" class="btn-text">Settings</a>
<a href="/user/{{.User.Username}}" class="btn-text">
<a href="/hooks" class="btn-secondary">Webhooks</a>
<a href="/settings" class="btn-secondary">Settings</a>
<a href="/user/{{.User.Username}}" class="btn-secondary">
<svg class="w-5 h-5 mr-1" fill="currentColor" viewBox="0 0 16 16">
<path d="M11 6a3 3 0 1 1-6 0 3 3 0 0 1 6 0z"/>
<path fill-rule="evenodd" d="M0 8a8 8 0 1 1 16 0A8 8 0 0 1 0 8zm8-7a7 7 0 0 0-5.468 11.37C3.242 11.226 4.805 10 8 10s4.757 1.225 5.468 2.37A7 7 0 0 0 8 1z"/>
@@ -32,7 +32,7 @@
{{if .CSRFToken}}
<form method="POST" action="/pages/logout" class="inline">
<input type="hidden" name="csrf_token" value="{{.CSRFToken}}">
<button type="submit" class="btn-text">Logout</button>
<button type="submit" class="btn-secondary">Logout</button>
</form>
{{end}}
{{end}}
@@ -43,13 +43,13 @@
<div x-show="open" x-cloak x-transition class="md:hidden mt-4 pt-4 border-t border-gray-200">
<div class="flex flex-col gap-2">
{{if .User}}
<a href="/hooks" class="btn-text w-full text-left">Webhooks</a>
<a href="/settings" class="btn-text w-full text-left">Settings</a>
<a href="/user/{{.User.Username}}" class="btn-text w-full text-left">Profile</a>
<a href="/hooks" class="btn-secondary w-full">Webhooks</a>
<a href="/settings" class="btn-secondary w-full">Settings</a>
<a href="/user/{{.User.Username}}" class="btn-secondary w-full">Profile</a>
{{if .CSRFToken}}
<form method="POST" action="/pages/logout">
<input type="hidden" name="csrf_token" value="{{.CSRFToken}}">
<button type="submit" class="btn-text w-full text-left">Logout</button>
<button type="submit" class="btn-secondary w-full">Logout</button>
</form>
{{end}}
{{end}}
+19 -16
View File
@@ -9,7 +9,7 @@
no class this wide. -->
<div class="mx-auto px-6 py-8" style="max-width: 108rem">
<div class="mb-6">
<a href="/hooks" class="text-sm text-primary-600 hover:text-primary-700">&larr; Back to webhooks</a>
<a href="/hooks" class="btn-small">&larr; Back to webhooks</a>
<div class="flex flex-wrap justify-between items-center gap-2 mt-2">
<div>
<h1 class="text-2xl font-medium text-gray-900">{{.Webhook.Name}}</h1>
@@ -35,8 +35,8 @@
<div class="card" x-data="collapsible">
<div class="p-4 border-b border-gray-200 flex justify-between items-center">
<h2 class="text-lg font-medium text-gray-900">Entrypoints</h2>
<button @click="toggle" class="btn-text text-sm">
<svg class="w-4 h-4 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<button type="button" @click="toggle" class="btn-small">
<svg class="w-3 h-3 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 4v16m8-8H4"/>
</svg>
Add
@@ -55,9 +55,9 @@
<div class="divide-y divide-gray-100">
{{range .Entrypoints}}
<div class="p-4">
<div class="flex items-center justify-between mb-1">
<div class="flex flex-wrap items-center justify-between gap-2 mb-1">
<span class="text-sm font-medium text-gray-900">{{if .Description}}{{.Description}}{{else}}Entrypoint{{end}}</span>
<div class="flex items-center gap-2">
<div class="flex flex-wrap items-center gap-2">
{{if .Active}}
<span class="badge-success">Active</span>
{{else}}
@@ -65,13 +65,13 @@
{{end}}
<form method="POST" action="/hook/{{$.Webhook.ID}}/entrypoints/{{.ID}}/toggle" class="inline">
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
<button type="submit" class="text-xs text-gray-500 hover:text-primary-600" title="{{if .Active}}Deactivate{{else}}Activate{{end}}">
<button type="submit" class="btn-small" title="{{if .Active}}Deactivate{{else}}Activate{{end}}">
{{if .Active}}Deactivate{{else}}Activate{{end}}
</button>
</form>
<form method="POST" action="/hook/{{$.Webhook.ID}}/entrypoints/{{.ID}}/delete" onsubmit="return confirm('Delete this entrypoint?')" class="inline">
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
<button type="submit" class="text-xs text-red-500 hover:text-red-700" title="Delete">Delete</button>
<button type="submit" class="btn-small text-red-600" title="Delete">Delete</button>
</form>
</div>
</div>
@@ -79,7 +79,7 @@
<code id="entrypoint-url-{{.ID}}" class="text-xs text-gray-500 break-all block flex-1">{{$.BaseURL}}/h/{{.Path}}</code>
<!-- Hidden until app.js reveals it; without the
script the URL above stays selectable. -->
<button type="button" hidden data-copy-target="entrypoint-url-{{.ID}}" class="text-xs text-gray-500 hover:text-primary-600">Copy</button>
<button type="button" hidden data-copy-target="entrypoint-url-{{.ID}}" class="btn-small">Copy</button>
</div>
<!-- The URL above is the entrypoint's credential:
anyone holding it can submit events. -->
@@ -94,8 +94,8 @@
<div class="card" x-data="collapsible">
<div class="p-4 border-b border-gray-200 flex justify-between items-center">
<h2 class="text-lg font-medium text-gray-900">Targets</h2>
<button @click="toggle" class="btn-text text-sm">
<svg class="w-4 h-4 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<button type="button" @click="toggle" class="btn-small">
<svg class="w-3 h-3 mr-1" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M12 4v16m8-8H4"/>
</svg>
Add
@@ -148,25 +148,28 @@
<div class="divide-y divide-gray-100">
{{range .Targets}}
<div class="p-4">
<div class="flex items-center justify-between mb-1">
<div class="flex flex-wrap items-center justify-between gap-2 mb-1">
<span class="text-sm font-medium text-gray-900">{{.Name}}</span>
<div class="flex items-center gap-2">
<div class="flex flex-wrap items-center gap-2">
<span class="badge-info">{{.Type}}</span>
{{if .Active}}
<span class="badge-success">Active</span>
{{else}}
<span class="badge-error">Inactive</span>
{{end}}
<a href="/hook/{{$.Webhook.ID}}/targets/{{.ID}}/edit" class="text-xs text-gray-500 hover:text-primary-600" title="Edit">Edit</a>
{{if eq .Type "database"}}
<a href="/hook/{{$.Webhook.ID}}/targets/{{.ID}}/download" class="btn-small" title="Download the archive as gzipped JSON">Download</a>
{{end}}
<a href="/hook/{{$.Webhook.ID}}/targets/{{.ID}}/edit" class="btn-small" title="Edit">Edit</a>
<form method="POST" action="/hook/{{$.Webhook.ID}}/targets/{{.ID}}/toggle" class="inline">
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
<button type="submit" class="text-xs text-gray-500 hover:text-primary-600" title="{{if .Active}}Deactivate{{else}}Activate{{end}}">
<button type="submit" class="btn-small" title="{{if .Active}}Deactivate{{else}}Activate{{end}}">
{{if .Active}}Deactivate{{else}}Activate{{end}}
</button>
</form>
<form method="POST" action="/hook/{{$.Webhook.ID}}/targets/{{.ID}}/delete" onsubmit="return confirm('Delete this target?')" class="inline">
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
<button type="submit" class="text-xs text-red-500 hover:text-red-700" title="Delete">Delete</button>
<button type="submit" class="btn-small text-red-600" title="Delete">Delete</button>
</form>
</div>
</div>
@@ -188,7 +191,7 @@
<div class="card mt-6">
<div class="p-4 border-b border-gray-200 flex justify-between items-center">
<h2 class="text-lg font-medium text-gray-900">50 Most Recent Events</h2>
<a href="/hook/{{.Webhook.ID}}/events" class="btn-text text-sm">Full Event Log</a>
<a href="/hook/{{.Webhook.ID}}/events" class="btn-small">Full Event Log</a>
</div>
<div class="divide-y divide-gray-100">
{{range .Events}}
+1 -1
View File
@@ -5,7 +5,7 @@
{{define "content"}}
<div class="max-w-2xl mx-auto px-6 py-8">
<div class="mb-6">
<a href="/hook/{{.Webhook.ID}}" class="text-sm text-primary-600 hover:text-primary-700">&larr; Back to {{.Webhook.Name}}</a>
<a href="/hook/{{.Webhook.ID}}" class="btn-small">&larr; Back to {{.Webhook.Name}}</a>
<h1 class="text-2xl font-medium text-gray-900 mt-2">Edit Webhook</h1>
</div>
+29 -27
View File
@@ -5,7 +5,7 @@
{{define "content"}}
<div class="max-w-6xl mx-auto px-6 py-8">
<div class="mb-6">
<a href="/hook/{{.Webhook.ID}}" class="text-sm text-primary-600 hover:text-primary-700">&larr; Back to {{.Webhook.Name}}</a>
<a href="/hook/{{.Webhook.ID}}" class="btn-small">&larr; Back to {{.Webhook.Name}}</a>
<div class="flex justify-between items-center mt-2">
<h1 class="text-2xl font-medium text-gray-900">Full Event Log</h1>
<span class="text-sm text-gray-500">{{.TotalEvents}} total event{{if ne .TotalEvents 1}}s{{end}}</span>
@@ -16,8 +16,8 @@
<div class="divide-y divide-gray-100">
{{range .Events}}
<div class="p-4" x-data="collapsible">
<div class="flex items-center justify-between cursor-pointer" @click="toggle">
<div class="flex items-center gap-3">
<button type="button" class="btn-small w-full flex flex-wrap justify-between gap-2 text-left" @click="toggle">
<span class="flex flex-wrap items-center gap-3">
<span class="badge-info">{{.Method}}</span>
<span class="text-sm font-mono text-gray-700">{{.ID}}</span>
<span class="text-sm text-gray-500">{{.ContentType}}</span>
@@ -27,8 +27,8 @@
{{if .ResubmitCount}}
<span class="text-xs text-gray-500">resubmitted {{.ResubmitCount}} time{{if ne .ResubmitCount 1}}s{{end}}</span>
{{end}}
</div>
<div class="flex items-center gap-4">
</span>
<span class="flex flex-wrap items-center gap-4">
{{range .Deliveries}}
<span class="text-xs {{if eq .Status "delivered"}}text-green-600{{else if eq .Status "failed"}}text-red-600{{else if eq .Status "retrying"}}text-yellow-600{{else}}text-gray-400{{end}}">
{{.Target.DisplayName}}: {{.Status}}
@@ -38,8 +38,8 @@
<svg class="w-4 h-4 text-gray-400 transition-transform" :class="caretClass" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M19 9l-7 7-7-7"/>
</svg>
</div>
</div>
</span>
</button>
<div x-show="open" x-cloak class="mt-3 p-3 bg-gray-50 rounded-md">
<div class="mb-3 flex flex-wrap items-center justify-between gap-2">
@@ -50,12 +50,12 @@
<form method="POST" action="/hook/{{$.Webhook.ID}}/events/{{.ID}}/resubmit" class="inline">
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
<input type="hidden" name="page" value="{{$.Page}}">
<button type="submit" class="text-xs text-primary-600 hover:text-primary-700" title="Submit this event again as a new event, to every currently active target">Resubmit</button>
<button type="submit" class="btn-small" title="Submit this event again as a new event, to every currently active target">Resubmit</button>
</form>
</div>
<pre class="text-xs text-gray-700 overflow-x-auto whitespace-pre-wrap break-all">{{.Body}}</pre>
{{if .BodyTruncated}}
<p class="mt-2 text-xs text-gray-500">Body truncated for display: showing {{.BodyShownBytes}} of {{.BodyBytes}} bytes. The stored body is unchanged &mdash; <a href="/hook/{{$.Webhook.ID}}/events/{{.ID}}/body" class="text-primary-600 hover:text-primary-700 underline">download the full body</a>.</p>
<p class="mt-2 text-xs text-gray-500">Body truncated for display: showing {{.BodyShownBytes}} of {{.BodyBytes}} bytes. The stored body is unchanged &mdash; <a href="/hook/{{$.Webhook.ID}}/events/{{.ID}}/body" class="btn-small">download the full body</a>.</p>
{{end}}
{{if .Deliveries}}
@@ -64,24 +64,26 @@
<div class="mt-2 divide-y divide-gray-200">
{{range .Deliveries}}
<div class="py-2" x-data="collapsible">
<div class="flex items-center justify-between cursor-pointer" @click="toggle">
<div class="flex items-center gap-3">
<span class="text-sm text-gray-700">{{.Target.DisplayName}}</span>
<span class="text-xs {{if eq .Status "delivered"}}text-green-600{{else if eq .Status "failed"}}text-red-600{{else if eq .Status "retrying"}}text-yellow-600{{else}}text-gray-400{{end}}">{{.Status}}</span>
</div>
<div class="flex items-center gap-3">
{{if .Status.Terminal}}
<form method="POST" action="/hook/{{$.Webhook.ID}}/deliveries/{{.ID}}/replay" class="inline" @click.stop>
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
<input type="hidden" name="page" value="{{$.Page}}">
<button type="submit" class="text-xs text-primary-600 hover:text-primary-700" title="Send this event to the target again">Replay</button>
</form>
{{end}}
<span class="text-xs text-gray-400">{{.AttemptCount}} attempt{{if ne .AttemptCount 1}}s{{end}}</span>
<svg class="w-3 h-3 text-gray-400 transition-transform" :class="caretClass" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M19 9l-7 7-7-7"/>
</svg>
</div>
<div class="flex items-center gap-3">
<button type="button" class="btn-small flex-1 flex-wrap justify-between gap-2 text-left" @click="toggle">
<span class="flex flex-wrap items-center gap-3">
<span class="text-sm text-gray-700">{{.Target.DisplayName}}</span>
<span class="text-xs {{if eq .Status "delivered"}}text-green-600{{else if eq .Status "failed"}}text-red-600{{else if eq .Status "retrying"}}text-yellow-600{{else}}text-gray-400{{end}}">{{.Status}}</span>
</span>
<span class="flex flex-wrap items-center gap-3">
<span class="text-xs text-gray-400">{{.AttemptCount}} attempt{{if ne .AttemptCount 1}}s{{end}}</span>
<svg class="w-3 h-3 text-gray-400 transition-transform" :class="caretClass" fill="none" stroke="currentColor" viewBox="0 0 24 24">
<path stroke-linecap="round" stroke-linejoin="round" stroke-width="2" d="M19 9l-7 7-7-7"/>
</svg>
</span>
</button>
{{if .Status.Terminal}}
<form method="POST" action="/hook/{{$.Webhook.ID}}/deliveries/{{.ID}}/replay" class="inline">
<input type="hidden" name="csrf_token" value="{{$.CSRFToken}}">
<input type="hidden" name="page" value="{{$.Page}}">
<button type="submit" class="btn-small" title="Send this event to the target again">Replay</button>
</form>
{{end}}
</div>
<div x-show="open" x-cloak class="mt-2 space-y-2">
+6 -1
View File
@@ -25,7 +25,12 @@
<p class="text-sm text-gray-500 mt-1">{{.Description}}</p>
{{end}}
</div>
<span class="badge-info">Retention: {{.RetentionLabel}}</span>
<div class="flex flex-wrap items-center gap-2">
<span class="badge-info">Retention: {{.RetentionLabel}}</span>
<!-- A label, not a control of its own: the whole card
is the link. -->
<span class="btn-small">Open &rarr;</span>
</div>
</div>
<div class="flex flex-wrap gap-6 mt-4 text-sm text-gray-500">
<span>{{.EntrypointCount}} entrypoint{{if ne .EntrypointCount 1}}s{{end}}{{if .InactiveEntrypointCount}}, {{.InactiveEntrypointCount}} inactive{{end}}</span>
+1 -1
View File
@@ -5,7 +5,7 @@
{{define "content"}}
<div class="max-w-2xl mx-auto px-6 py-8">
<div class="mb-6">
<a href="/hooks" class="text-sm text-primary-600 hover:text-primary-700">&larr; Back to webhooks</a>
<a href="/hooks" class="btn-small">&larr; Back to webhooks</a>
<h1 class="text-2xl font-medium text-gray-900 mt-2">Create Webhook</h1>
</div>
+1 -1
View File
@@ -5,7 +5,7 @@
{{define "content"}}
<div class="max-w-2xl mx-auto px-6 py-8">
<div class="mb-6">
<a href="/hook/{{.Webhook.ID}}" class="text-sm text-primary-600 hover:text-primary-700">&larr; Back to {{.Webhook.Name}}</a>
<a href="/hook/{{.Webhook.ID}}" class="btn-small">&larr; Back to {{.Webhook.Name}}</a>
<h1 class="text-2xl font-medium text-gray-900 mt-2">Edit Target</h1>
<p class="text-sm text-gray-500 mt-1">Type: {{.Target.Type}}. A target's type cannot be changed; create a new target to deliver a different way.</p>
</div>