Evict archive writers on deletion and sweep idle archives (closes #89)

The per-webhook archiveWriter registry in the database delivery target
was never evicted, so a deleted webhook's writer -- and any archive
file handle open within its debounce window -- lingered for the process
lifetime. Separately, expiry pruning ran only when an archive was
(re)opened, and reopens only happen on writes, so an archive belonging
to a webhook that stopped receiving events kept its expired rows
forever.

Eviction: a new one-method delivery.WebhookEvictor interface (kept
separate from Notifier: archiving lifecycle is not notification) is
implemented by the Engine and injected into the handlers. Deleting a
webhook, or deleting its last database target, drops the writer from
the registry and closes its handle under the writer's own mutex, so
eviction can never race an in-flight write. An evicted writer refuses
further writes rather than reopening a file nothing holds. The archive
file is deliberately left on disk: it is long-term storage an operator
may want to keep or move away, and destroying it as a side effect of
deleting a webhook would be unrecoverable.

Idle sweep: a new ArchiveSweeper, modelled on the event RetentionReaper
(fx lifecycle hooks, cancellable context, WaitGroup, ticker loop),
prunes archives whose database target declares a positive expiry. It
reuses the existing RETENTION_SWEEP_INTERVAL rather than adding a
config key. It never creates an archive -- a missing file is skipped,
and the reopen uses SQLite mode=rw so the file cannot be conjured even
if it disappears mid-sweep -- routes the prune through the per-webhook
writer so its mutex orders the sweep against concurrent writes, and
leaves the archive closed so the move-the-file-away workflow keeps
working. A failure for one webhook is logged and the sweep continues.
Archives with no expiry or the expiry "never" are untouched.

The sweep loop's context is rooted at context.Background(), not at the
fx OnStart hook context. The hook context carries fx's 15 second start
timeout, so a loop derived from it is cancelled three quarters of an
hour before the first tick under the default one-hour interval, giving
a sweeper that never sweeps. OnStop still cancels the loop and waits on
the WaitGroup, so shutdown is unchanged.

The sweep also never leaves a registry entry behind. Reaching the
writer through the ordinary create-and-cache accessor would let a sweep
that raced a webhook deletion re-insert a writer for a webhook that no
longer exists, which nothing would ever evict again -- the very leak
this change closes. An entry the sweep has to create is marked
sweep-owned and released when the prune finishes, unless a delivery
claimed it meanwhile, in which case it belongs to the registry and an
eviction can still reach it. A writer evicted underneath a sweep is an
ordinary interleaving and is logged at debug, not error.

internal/delivery/target_database_archive.go

@@ -24,6 +24,20 @@ const archiveExpiryNever = "never"
 // offline archiving, but never more than once per this window.
 const archiveReopenDebounce = time.Second
 
+const (
+	// archiveModeCreate is the SQLite URI mode used by the write
+	// path: open the archive file, creating it if missing, so a
+	// first write (or a write after the operator moved the file
+	// away) recreates it.
+	archiveModeCreate = "rwc"
+
+	// archiveModeExisting is the SQLite URI mode used by the idle
+	// sweep: open read-write but never create. A sweep must never
+	// conjure an empty archive file for a webhook that has a
+	// database target but has never received an event.
+	archiveModeExisting = "rw"
+)
+
 var (
 	// errArchiveMissingWebhookID is returned when an event to
 	// archive has no webhook id to key its archive file on.
@@ -44,6 +58,15 @@ var (
 	errArchiveExpiryNotPositive = errors.New(
 		"expiry must be a positive duration or \"never\"",
 	)
+
+	// errArchiveWriterEvicted is returned when a writer that has
+	// been evicted (its webhook was deleted, or its last database
+	// target was removed) is used again. An evicted writer is no
+	// longer in the registry, so reopening its file would leak a
+	// handle nothing owns.
+	errArchiveWriterEvicted = errors.New(
+		"archive writer has been evicted",
+	)
 )
 
 // databaseTargetConfig is the optional per-target JSON config
@@ -161,6 +184,25 @@ type archiveWriter struct {
 	db         *gorm.DB
 	lastReopen time.Time
 	reopens    int
+
+	// evicted marks a writer that has been removed from the
+	// per-webhook registry. Its handle is closed and it must
+	// never open the file again: nothing holds it any more, so a
+	// reopen would leak the handle for the process lifetime.
+	evicted bool
+
+	// sweepOwned marks a registry entry that the idle sweep
+	// created because no writer was cached for the webhook. The
+	// sweep removes such an entry again when it is done, so a
+	// sweep can never leave — or resurrect — a registry entry
+	// for a webhook that has been deleted. A delivery that adopts
+	// the writer clears the flag, handing the entry to the
+	// registry proper.
+	//
+	// Unlike every other field here it is guarded by
+	// databaseTarget.mu, not by this writer's mu: it describes the
+	// registry entry rather than the file.
+	sweepOwned bool
 }
 
 // newArchiveWriter builds an archiveWriter for a file path with
@@ -185,6 +227,12 @@ func (w *archiveWriter) write(
 	w.mu.Lock()
 	defer w.mu.Unlock()
 
+	if w.evicted {
+		return fmt.Errorf(
+			"%w: %s", errArchiveWriterEvicted, w.path,
+		)
+	}
+
 	if w.db == nil || !fileExists(w.path) {
 		err := w.reopen(expiry)
 		if err != nil {
@@ -212,7 +260,19 @@ func (w *archiveWriter) write(
 // its schema, records the reopen time, and prunes expired rows
 // when expiry is positive.
 func (w *archiveWriter) open(expiry time.Duration) error {
-	dbURL := fmt.Sprintf("file:%s?mode=rwc", w.path)
+	return w.openMode(archiveModeCreate, expiry)
+}
+
+// openMode opens the archive file with the given SQLite URI
+// mode, migrates its schema, records the reopen time, and
+// prunes expired rows when expiry is positive. The write path
+// passes archiveModeCreate so a missing file is recreated; the
+// idle sweep passes archiveModeExisting so a missing file is an
+// error rather than a newly conjured empty archive.
+func (w *archiveWriter) openMode(
+	mode string, expiry time.Duration,
+) error {
+	dbURL := fmt.Sprintf("file:%s?mode=%s", w.path, mode)
 
 	sqlDB, err := sql.Open("sqlite", dbURL)
 	if err != nil {
@@ -275,6 +335,63 @@ func (w *archiveWriter) close() {
 	w.db = nil
 }
 
+// sweepExpired prunes an archive that may have gone idle, with
+// no write to trigger the usual on-reopen prune. It takes the
+// writer's own mutex for the whole operation, so a sweep is
+// ordered against concurrent writes rather than reaching around
+// them to the file.
+//
+// It never creates the archive file: a missing file is skipped,
+// and the reopen uses archiveModeExisting so SQLite itself
+// refuses to create one if the file disappears between the
+// check and the open.
+//
+// The archive is left CLOSED afterwards. An idle archive holding
+// no handle is what keeps the operator's move-the-file-away
+// workflow working; the next write reopens (and recreates) the
+// file as it always has.
+func (w *archiveWriter) sweepExpired(expiry time.Duration) error {
+	w.mu.Lock()
+	defer w.mu.Unlock()
+
+	if w.evicted {
+		return fmt.Errorf(
+			"%w: %s", errArchiveWriterEvicted, w.path,
+		)
+	}
+
+	if !fileExists(w.path) {
+		return nil
+	}
+
+	// Drop any live handle first so the prune runs against a
+	// freshly opened file, matching the write path's semantics.
+	w.close()
+
+	err := w.openMode(archiveModeExisting, expiry)
+	if err != nil {
+		return err
+	}
+
+	w.close()
+
+	return nil
+}
+
+// evict closes the writer's handle and marks it unusable. It is
+// called when the writer leaves the registry, either because the
+// webhook was deleted or because its last database target was
+// removed. The archive FILE is deliberately left on disk: it is
+// long-term storage an operator may still want.
+func (w *archiveWriter) evict() {
+	w.mu.Lock()
+	defer w.mu.Unlock()
+
+	w.evicted = true
+
+	w.close()
+}
+
 // prune deletes archived rows older than expiry, measured from
 // each row's archived time. It runs on every (re)open, and
 // because the file is reopened after writes this keeps the