Evict archive writers on deletion and sweep idle archives (closes #89)

The per-webhook archiveWriter registry in the database delivery target
was never evicted, so a deleted webhook's writer -- and any archive
file handle open within its debounce window -- lingered for the process
lifetime. Separately, expiry pruning ran only when an archive was
(re)opened, and reopens only happen on writes, so an archive belonging
to a webhook that stopped receiving events kept its expired rows
forever.

Eviction: a new one-method delivery.WebhookEvictor interface (kept
separate from Notifier: archiving lifecycle is not notification) is
implemented by the Engine and injected into the handlers. Deleting a
webhook, or deleting its last database target, drops the writer from
the registry and closes its handle under the writer's own mutex, so
eviction can never race an in-flight write. An evicted writer refuses
further writes rather than reopening a file nothing holds. The archive
file is deliberately left on disk: it is long-term storage an operator
may want to keep or move away, and destroying it as a side effect of
deleting a webhook would be unrecoverable.

Idle sweep: a new ArchiveSweeper, modelled on the event RetentionReaper
(fx lifecycle hooks, cancellable context, WaitGroup, ticker loop),
prunes archives whose database target declares a positive expiry. It
reuses the existing RETENTION_SWEEP_INTERVAL rather than adding a
config key. It never creates an archive -- a missing file is skipped,
and the reopen uses SQLite mode=rw so the file cannot be conjured even
if it disappears mid-sweep -- routes the prune through the per-webhook
writer so its mutex orders the sweep against concurrent writes, and
leaves the archive closed so the move-the-file-away workflow keeps
working. A failure for one webhook is logged and the sweep continues.
Archives with no expiry or the expiry "never" are untouched.

The sweep loop's context is rooted at context.Background(), not at the
fx OnStart hook context. The hook context carries fx's 15 second start
timeout, so a loop derived from it is cancelled three quarters of an
hour before the first tick under the default one-hour interval, giving
a sweeper that never sweeps. OnStop still cancels the loop and waits on
the WaitGroup, so shutdown is unchanged.

The sweep also never leaves a registry entry behind. Reaching the
writer through the ordinary create-and-cache accessor would let a sweep
that raced a webhook deletion re-insert a writer for a webhook that no
longer exists, which nothing would ever evict again -- the very leak
this change closes. An entry the sweep has to create is marked
sweep-owned and released when the prune finishes, unless a delivery
claimed it meanwhile, in which case it belongs to the registry and an
eviction can still reach it. A writer evicted underneath a sweep is an
ordinary interleaving and is logged at debug, not error.

Each of those guards is pinned by a test that fails when the guard is
removed. The adopt-during-sweep window -- a delivery claiming the
sweep's own registry entry while that sweep is still running -- is
driven directly against the registry, because a delivery placed between
two sweeps never reaches the release path at all. The requirement that
an idle archive ends the sweep closed is asserted both on a writer
proven to hold an open handle beforehand and, end to end, on a
delivery-owned entry the sweep keeps, rather than on an entry the sweep
has already released and which therefore reports "not open" either
way. The "never" expiry short circuit is checked against an archive
file that has never been migrated, so any open of it would be
observable as a created table.

internal/handlers/source_delete_test.go

@@ -0,0 +1,356 @@
+package handlers_test
+
+import (
+	"context"
+	"net/http"
+	"net/http/httptest"
+	"os"
+	"path/filepath"
+	"testing"
+
+	"github.com/go-chi/chi"
+	"github.com/stretchr/testify/assert"
+	"github.com/stretchr/testify/require"
+	"gorm.io/gorm/clause"
+	"sneak.berlin/go/webhooker/internal/database"
+	"sneak.berlin/go/webhooker/internal/handlers"
+	"sneak.berlin/go/webhooker/internal/session"
+)
+
+const (
+	deleteTestUserID   = "test-user-id"
+	deleteTestUsername = "testuser"
+
+	// paramSourceID and paramTargetID are the chi URL parameter
+	// names the deletion handlers read.
+	paramSourceID = "sourceID"
+	paramTargetID = "targetID"
+)
+
+// seedWebhook inserts a webhook owned by the test user and
+// returns it.
+func seedWebhook(
+	t *testing.T,
+	db *database.Database,
+) *database.Webhook {
+	t.Helper()
+
+	wh := &database.Webhook{
+		UserID: deleteTestUserID,
+		Name:   "delete-me",
+	}
+
+	require.NoError(
+		t,
+		db.DB().Omit(clause.Associations).Create(wh).Error,
+	)
+
+	return wh
+}
+
+// seedTarget inserts a target of the given type for a webhook
+// and returns it.
+func seedTarget(
+	t *testing.T,
+	db *database.Database,
+	webhookID string,
+	targetType database.TargetType,
+) *database.Target {
+	t.Helper()
+
+	tgt := &database.Target{
+		WebhookID: webhookID,
+		Name:      "t-" + string(targetType),
+		Type:      targetType,
+		Active:    true,
+	}
+
+	require.NoError(
+		t,
+		db.DB().Omit(clause.Associations).Create(tgt).Error,
+	)
+
+	return tgt
+}
+
+// archivePathFor returns the archive database path the
+// delivery engine would use for a webhook: beside the webhook's
+// event database in the data directory.
+func archivePathFor(
+	t *testing.T,
+	mgr *database.WebhookDBManager,
+	webhookID string,
+) string {
+	t.Helper()
+
+	return filepath.Join(
+		filepath.Dir(mgr.DBPath(webhookID)),
+		"archive-"+webhookID+".db",
+	)
+}
+
+// writeArchivePlaceholder creates a stand-in archive file so a
+// test can assert the file survives webhook deletion.
+func writeArchivePlaceholder(path string) error {
+	return os.WriteFile(path, []byte("archive"), 0o600)
+}
+
+// postRequest builds an authenticated POST request carrying the
+// given chi URL parameters.
+func postRequest(
+	path string,
+	cookies []*http.Cookie,
+	params map[string]string,
+) *http.Request {
+	req := httptest.NewRequestWithContext(
+		context.Background(), http.MethodPost, path, nil,
+	)
+
+	for _, c := range cookies {
+		req.AddCookie(c)
+	}
+
+	rctx := chi.NewRouteContext()
+	for k, v := range params {
+		rctx.URLParams.Add(k, v)
+	}
+
+	return req.WithContext(
+		context.WithValue(req.Context(), chi.RouteCtxKey, rctx),
+	)
+}
+
+// TestHandleSourceDelete_EvictsArchiveWriter proves that
+// deleting a webhook reaches the delivery engine and releases
+// the webhook's archive writer, exercised through the real
+// deletion handler rather than by calling the evictor directly.
+func TestHandleSourceDelete_EvictsArchiveWriter(t *testing.T) {
+	t.Parallel()
+
+	var (
+		h    *handlers.Handlers
+		sess *session.Session
+		db   *database.Database
+		ev   *recordingEvictor
+	)
+
+	app := newTestApp(t, &h, &sess, &db, &ev)
+	app.RequireStart()
+
+	t.Cleanup(app.RequireStop)
+
+	wh := seedWebhook(t, db)
+	seedTarget(t, db, wh.ID, database.TargetTypeDatabase)
+
+	cookies := authenticatedCookies(
+		t, sess, deleteTestUserID, deleteTestUsername,
+	)
+
+	req := postRequest(
+		"/source/"+wh.ID+"/delete",
+		cookies,
+		map[string]string{paramSourceID: wh.ID},
+	)
+	w := httptest.NewRecorder()
+
+	h.HandleSourceDelete().ServeHTTP(w, req)
+
+	require.Equal(t, http.StatusSeeOther, w.Code)
+	assert.Equal(
+		t, []string{wh.ID}, ev.Evicted(),
+		"deleting a webhook should evict its archive writer",
+	)
+}
+
+// TestHandleSourceDelete_KeepsArchiveFile proves that deleting
+// a webhook does not remove its archive database file: the
+// archive is long-term storage the operator owns.
+func TestHandleSourceDelete_KeepsArchiveFile(t *testing.T) {
+	t.Parallel()
+
+	var (
+		h    *handlers.Handlers
+		sess *session.Session
+		db   *database.Database
+		mgr  *database.WebhookDBManager
+	)
+
+	app := newTestApp(t, &h, &sess, &db, &mgr)
+	app.RequireStart()
+
+	t.Cleanup(app.RequireStop)
+
+	wh := seedWebhook(t, db)
+
+	// Place an archive file where the delivery engine would.
+	archivePath := archivePathFor(t, mgr, wh.ID)
+	require.NoError(
+		t,
+		writeArchivePlaceholder(archivePath),
+	)
+
+	cookies := authenticatedCookies(
+		t, sess, deleteTestUserID, deleteTestUsername,
+	)
+
+	req := postRequest(
+		"/source/"+wh.ID+"/delete",
+		cookies,
+		map[string]string{paramSourceID: wh.ID},
+	)
+	w := httptest.NewRecorder()
+
+	h.HandleSourceDelete().ServeHTTP(w, req)
+
+	require.Equal(t, http.StatusSeeOther, w.Code)
+	assert.FileExists(
+		t, archivePath,
+		"webhook deletion must not destroy the archive file",
+	)
+}
+
+// TestHandleTargetDelete_EvictsWhenLastDatabaseTargetGone
+// proves that removing the last database target releases the
+// archive writer.
+func TestHandleTargetDelete_EvictsWhenLastDatabaseTargetGone(
+	t *testing.T,
+) {
+	t.Parallel()
+
+	var (
+		h    *handlers.Handlers
+		sess *session.Session
+		db   *database.Database
+		ev   *recordingEvictor
+	)
+
+	app := newTestApp(t, &h, &sess, &db, &ev)
+	app.RequireStart()
+
+	t.Cleanup(app.RequireStop)
+
+	wh := seedWebhook(t, db)
+	tgt := seedTarget(
+		t, db, wh.ID, database.TargetTypeDatabase,
+	)
+
+	cookies := authenticatedCookies(
+		t, sess, deleteTestUserID, deleteTestUsername,
+	)
+
+	req := postRequest(
+		"/source/"+wh.ID+"/targets/"+tgt.ID+"/delete",
+		cookies,
+		map[string]string{
+			paramSourceID: wh.ID,
+			paramTargetID: tgt.ID,
+		},
+	)
+	w := httptest.NewRecorder()
+
+	h.HandleTargetDelete().ServeHTTP(w, req)
+
+	require.Equal(t, http.StatusSeeOther, w.Code)
+	assert.Equal(
+		t, []string{wh.ID}, ev.Evicted(),
+		"removing the last database target should evict",
+	)
+}
+
+// TestHandleTargetDelete_KeepsWriterWhenDatabaseTargetRemains
+// proves that deleting one of several database targets leaves
+// the still-needed archive writer alone: the surviving target
+// keeps archiving to the same file, so the writer must stay.
+func TestHandleTargetDelete_KeepsWriterWhenDatabaseTargetRemains(
+	t *testing.T,
+) {
+	t.Parallel()
+
+	var (
+		h    *handlers.Handlers
+		sess *session.Session
+		db   *database.Database
+		ev   *recordingEvictor
+	)
+
+	app := newTestApp(t, &h, &sess, &db, &ev)
+	app.RequireStart()
+
+	t.Cleanup(app.RequireStop)
+
+	wh := seedWebhook(t, db)
+	doomed := seedTarget(
+		t, db, wh.ID, database.TargetTypeDatabase,
+	)
+	seedTarget(t, db, wh.ID, database.TargetTypeDatabase)
+
+	cookies := authenticatedCookies(
+		t, sess, deleteTestUserID, deleteTestUsername,
+	)
+
+	req := postRequest(
+		"/source/"+wh.ID+"/targets/"+doomed.ID+"/delete",
+		cookies,
+		map[string]string{
+			paramSourceID: wh.ID,
+			paramTargetID: doomed.ID,
+		},
+	)
+	w := httptest.NewRecorder()
+
+	h.HandleTargetDelete().ServeHTTP(w, req)
+
+	require.Equal(t, http.StatusSeeOther, w.Code)
+	assert.Empty(
+		t, ev.Evicted(),
+		"a second database target still needs the writer",
+	)
+}
+
+// TestHandleTargetDelete_KeepsWriterWhenOtherTypeDeleted proves
+// that deleting a target of an unrelated type leaves a
+// still-needed archive writer alone: the webhook's database
+// target is untouched, so its writer must stay.
+func TestHandleTargetDelete_KeepsWriterWhenOtherTypeDeleted(
+	t *testing.T,
+) {
+	t.Parallel()
+
+	var (
+		h    *handlers.Handlers
+		sess *session.Session
+		db   *database.Database
+		ev   *recordingEvictor
+	)
+
+	app := newTestApp(t, &h, &sess, &db, &ev)
+	app.RequireStart()
+
+	t.Cleanup(app.RequireStop)
+
+	wh := seedWebhook(t, db)
+	seedTarget(t, db, wh.ID, database.TargetTypeDatabase)
+	other := seedTarget(t, db, wh.ID, database.TargetTypeLog)
+
+	cookies := authenticatedCookies(
+		t, sess, deleteTestUserID, deleteTestUsername,
+	)
+
+	req := postRequest(
+		"/source/"+wh.ID+"/targets/"+other.ID+"/delete",
+		cookies,
+		map[string]string{
+			paramSourceID: wh.ID,
+			paramTargetID: other.ID,
+		},
+	)
+	w := httptest.NewRecorder()
+
+	h.HandleTargetDelete().ServeHTTP(w, req)
+
+	require.Equal(t, http.StatusSeeOther, w.Code)
+	assert.Empty(
+		t, ev.Evicted(),
+		"a surviving database target must keep its writer",
+	)
+}