Evict archive writers on deletion and sweep idle archives (closes #89)

The per-webhook archiveWriter registry in the database delivery target
was never evicted, so a deleted webhook's writer -- and any archive
file handle open within its debounce window -- lingered for the process
lifetime. Separately, expiry pruning ran only when an archive was
(re)opened, and reopens only happen on writes, so an archive belonging
to a webhook that stopped receiving events kept its expired rows
forever.

Eviction: a new one-method delivery.WebhookEvictor interface (kept
separate from Notifier: archiving lifecycle is not notification) is
implemented by the Engine and injected into the handlers. Deleting a
webhook, or deleting its last database target, drops the writer from
the registry and closes its handle under the writer's own mutex, so
eviction can never race an in-flight write. An evicted writer refuses
further writes rather than reopening a file nothing holds. The archive
file is deliberately left on disk: it is long-term storage an operator
may want to keep or move away, and destroying it as a side effect of
deleting a webhook would be unrecoverable.

Idle sweep: a new ArchiveSweeper, modelled on the event RetentionReaper
(fx lifecycle hooks, cancellable context, WaitGroup, ticker loop),
prunes archives whose database target declares a positive expiry. It
reuses the existing RETENTION_SWEEP_INTERVAL rather than adding a
config key. It never creates an archive -- a missing file is skipped,
and the reopen uses SQLite mode=rw so the file cannot be conjured even
if it disappears mid-sweep -- routes the prune through the per-webhook
writer so its mutex orders the sweep against concurrent writes, and
leaves the archive closed so the move-the-file-away workflow keeps
working. A failure for one webhook is logged and the sweep continues.
Archives with no expiry or the expiry "never" are untouched.

The sweep loop's context is rooted at context.Background(), not at the
fx OnStart hook context. The hook context carries fx's 15 second start
timeout, so a loop derived from it is cancelled three quarters of an
hour before the first tick under the default one-hour interval, giving
a sweeper that never sweeps. OnStop still cancels the loop and waits on
the WaitGroup, so shutdown is unchanged.

The sweep also never leaves a registry entry behind. Reaching the
writer through the ordinary create-and-cache accessor would let a sweep
that raced a webhook deletion re-insert a writer for a webhook that no
longer exists, which nothing would ever evict again -- the very leak
this change closes. An entry the sweep has to create is marked
sweep-owned and released when the prune finishes, unless a delivery
claimed it meanwhile, in which case it belongs to the registry and an
eviction can still reach it. A writer evicted underneath a sweep is an
ordinary interleaving and is logged at debug, not error.

internal/delivery/target_database.go

@@ -5,6 +5,7 @@ import (
 	"fmt"
 	"path/filepath"
 	"sync"
+	"time"
 
 	"gorm.io/gorm"
 	"sneak.berlin/go/webhooker/internal/database"
@@ -111,15 +112,11 @@ func (t *databaseTarget) archive(d *database.Delivery) error {
 func (t *databaseTarget) writerFor(
 	webhookID string,
 ) (*archiveWriter, error) {
-	if t.eng.dbManager == nil {
-		return nil, errArchiveNoDataDir
+	path, err := t.archivePath(webhookID)
+	if err != nil {
+		return nil, err
 	}
 
-	dir := filepath.Dir(t.eng.dbManager.DBPath(webhookID))
-	path := filepath.Join(
-		dir, fmt.Sprintf("archive-%s.db", webhookID),
-	)
-
 	t.mu.Lock()
 	defer t.mu.Unlock()
 
@@ -133,5 +130,166 @@ func (t *databaseTarget) writerFor(
 		t.writers[webhookID] = w
 	}
 
+	// A delivery claims the entry: even if the idle sweep created
+	// it moments ago, it now belongs to the registry proper and
+	// the sweep must leave it in place when it finishes.
+	w.sweepOwned = false
+
 	return w, nil
 }
+
+// sweepWriterFor returns the archive writer the idle sweep should
+// prune a webhook through, together with whether the sweep itself
+// created the registry entry.
+//
+// The sweep must route its prune through the registered writer so
+// the writer's mutex orders it against concurrent writes, but it
+// must never leave a registry entry behind: a sweep that ran
+// concurrently with the webhook's deletion would otherwise
+// re-create an entry that nothing will ever evict again, which is
+// exactly the leak eviction exists to prevent. An entry the sweep
+// creates is therefore marked sweep-owned and handed back to
+// releaseSweepWriter when the sweep is done.
+func (t *databaseTarget) sweepWriterFor(
+	webhookID string,
+) (*archiveWriter, bool, error) {
+	path, err := t.archivePath(webhookID)
+	if err != nil {
+		return nil, false, err
+	}
+
+	t.mu.Lock()
+	defer t.mu.Unlock()
+
+	if t.writers == nil {
+		t.writers = make(map[string]*archiveWriter)
+	}
+
+	w, ok := t.writers[webhookID]
+	if ok {
+		return w, false, nil
+	}
+
+	w = newArchiveWriter(path, t.eng.log)
+	w.sweepOwned = true
+	t.writers[webhookID] = w
+
+	return w, true, nil
+}
+
+// releaseSweepWriter drops a registry entry that the idle sweep
+// created, so a sweep leaves the registry exactly as it found it.
+//
+// The entry is removed only if it is still the very writer the
+// sweep installed and no delivery has claimed it in the meantime
+// (writerFor clears sweepOwned when it hands a writer to the
+// write path). Both conditions are evaluated under the registry
+// lock, so an eviction that raced the sweep — which removes the
+// entry outright — simply finds nothing left to do here, and a
+// delivery that adopted the writer keeps a registered, evictable
+// one.
+func (t *databaseTarget) releaseSweepWriter(
+	webhookID string, w *archiveWriter,
+) {
+	t.mu.Lock()
+	defer t.mu.Unlock()
+
+	cur, ok := t.writers[webhookID]
+	if !ok || cur != w || !cur.sweepOwned {
+		return
+	}
+
+	delete(t.writers, webhookID)
+}
+
+// archivePath returns the archive file path for a webhook: it
+// lives beside the per-webhook event database in the data
+// directory. It does not touch the filesystem.
+func (t *databaseTarget) archivePath(
+	webhookID string,
+) (string, error) {
+	if t.eng.dbManager == nil {
+		return "", errArchiveNoDataDir
+	}
+
+	dir := filepath.Dir(t.eng.dbManager.DBPath(webhookID))
+
+	return filepath.Join(
+		dir, fmt.Sprintf("archive-%s.db", webhookID),
+	), nil
+}
+
+// evict drops a webhook's archive writer from the registry and
+// closes its handle, so a deleted webhook does not leave a
+// writer (and an open archive handle within its debounce
+// window) alive for the process lifetime.
+//
+// The map entry is removed under the registry lock, which is
+// then released before the handle is closed under the writer's
+// own lock: that ordering keeps the registry available to other
+// webhooks while an in-flight write on this one drains, and
+// closing under the writer's lock means eviction can never race
+// a write.
+//
+// Eviction is idempotent and silent for a webhook with no
+// writer, which is the common case: a webhook with no database
+// target never creates one. It never deletes the archive file.
+func (t *databaseTarget) evict(webhookID string) {
+	t.mu.Lock()
+
+	w, ok := t.writers[webhookID]
+	if ok {
+		delete(t.writers, webhookID)
+	}
+
+	t.mu.Unlock()
+
+	if !ok {
+		return
+	}
+
+	w.evict()
+
+	t.eng.log.Info(
+		"evicted archive writer",
+		"webhook_id", webhookID,
+		"path", w.path,
+	)
+}
+
+// sweepWebhook prunes one webhook's archive of rows older than
+// expiry, without requiring a write. It returns nil (nothing to
+// do) when the archive file does not exist, so a sweep never
+// creates an archive for a webhook that has a database target
+// but has never received an event.
+//
+// It also never leaves a registry entry behind: an entry it had
+// to create to reach the writer's mutex is released again once
+// the prune is done, so a sweep racing a webhook deletion cannot
+// resurrect the writer the eviction just dropped.
+func (t *databaseTarget) sweepWebhook(
+	webhookID string, expiry time.Duration,
+) error {
+	path, err := t.archivePath(webhookID)
+	if err != nil {
+		return err
+	}
+
+	// Check before taking a writer at all: a webhook whose
+	// archive has never been created gets no writer, no handle,
+	// and no file.
+	if !fileExists(path) {
+		return nil
+	}
+
+	w, created, err := t.sweepWriterFor(webhookID)
+	if err != nil {
+		return err
+	}
+
+	if created {
+		defer t.releaseSweepWriter(webhookID, w)
+	}
+
+	return w.sweepExpired(expiry)
+}