Name each database target's archive for its webhook and target (closes #376)
check / check (push) Waiting to run

Each database target now has its own archive file,
archive-WEBHOOKNAME-TARGETNAME-TARGETID.db, instead of one
archive-WEBHOOKID.db per webhook. delivery.ArchiveFileName builds the
name: each name is lowercased, keeps ASCII letters and digits, turns
every other run of characters into one dash, and is cut to 40
characters.

A change of webhook or target name renames its archive files under the
archive writer's lock, before the new name is saved, and back again if
the save fails. A rename never replaces a file: if one already has the
new name, the edit is refused. Deleting a target evicts only that
target's writer. Archive files are never deleted, and nothing looks for
files under the old name.

Model: opus-5-5
This commit is contained in:
2026-10-02 06:41:33 +00:00
committed by sneak
parent cb7bafab17
commit 17e6be34ab
21 changed files with 1438 additions and 663 deletions
+81 -17
View File
@@ -4,8 +4,10 @@ import (
"encoding/json"
"errors"
"fmt"
"io/fs"
"log/slog"
"os"
"path/filepath"
"sync"
"time"
@@ -41,7 +43,7 @@ const (
var (
// errArchiveMissingWebhookID is returned when an event to
// archive has no webhook id to key its archive file on.
// archive has no webhook id to record in its archive row.
errArchiveMissingWebhookID = errors.New(
"cannot archive event without a webhook id",
)
@@ -61,13 +63,19 @@ var (
)
// errArchiveWriterEvicted is returned when a writer that has
// been evicted (its webhook was deleted, or its last database
// target was removed) is used again. An evicted writer is no
// longer in the registry, so reopening its file would leak a
// handle nothing owns.
// been evicted (its target or its webhook was deleted) is used
// again. An evicted writer is no longer in the registry, so
// reopening its file would leak a handle nothing owns.
errArchiveWriterEvicted = errors.New(
"archive writer has been evicted",
)
// ErrArchiveNameTaken is returned when an archive cannot be
// renamed because a file already has the new name. That file may
// be an archive with rows of its own, so it is never replaced.
ErrArchiveNameTaken = errors.New(
"a file already has the archive's new name",
)
)
// databaseTargetConfig is the optional per-target JSON config
@@ -80,7 +88,7 @@ type databaseTargetConfig struct {
}
// archivedEvent is one fully captured webhook event stored in a
// per-webhook archive database for long-term retention. It is a
// database target's archive for long-term retention. It is a
// self-contained copy — independent of the per-webhook event
// database, which may prune events under its own retention.
type archivedEvent struct {
@@ -170,8 +178,8 @@ func ValidateArchiveExpiry(expiry string) error {
return nil
}
// archiveWriter owns one per-webhook archive SQLite file. It
// serialises writes, and after each write closes and reopens
// archiveWriter owns one database target's archive SQLite file.
// It serialises writes, and after each write closes and reopens
// the file (debounced to at most once per debounce window) so
// an operator can move the file away for offline archiving. The
// next write recreates a moved or removed file, because the
@@ -187,16 +195,21 @@ type archiveWriter struct {
reopens int
// evicted marks a writer that has been removed from the
// per-webhook registry. Its handle is closed and it must
// never open the file again: nothing holds it any more, so a
// reopen would leak the handle for the process lifetime.
// registry. Its handle is closed and it must never open the
// file again: nothing holds it any more, so a reopen would
// leak the handle for the process lifetime.
evicted bool
// webhookID is the webhook the archive's target belongs to,
// so deleting the webhook can find its writers. It is set
// when the writer is created and never changes.
webhookID string
// sweepOwned marks a registry entry that the idle sweep
// created because no writer was cached for the webhook. The
// created because no writer was cached for the target. The
// sweep removes such an entry again when it is done, so a
// sweep can never leave — or resurrect — a registry entry
// for a webhook that has been deleted. A delivery that adopts
// for a target that has been deleted. A delivery that adopts
// the writer clears the flag, handing the entry to the
// registry proper.
//
@@ -385,11 +398,62 @@ func (w *archiveWriter) sweepExpired(expiry time.Duration) error {
return nil
}
// rename gives the archive file a new name in the same directory,
// and the writer uses the file under that name from now on. The
// handle is closed first, which folds the -wal into the .db; any
// -wal or -shm still beside the file (left by a crash) is moved with
// it, because SQLite finds them by name. A missing file is not an
// error: the operator may have moved it away, and the next write
// creates it under the new name.
//
// If a file already has the new name, nothing is moved and the
// error is ErrArchiveNameTaken.
func (w *archiveWriter) rename(name string) error {
w.mu.Lock()
defer w.mu.Unlock()
if w.evicted {
return fmt.Errorf(
"%w: %s", errArchiveWriterEvicted, w.path,
)
}
path := filepath.Join(filepath.Dir(w.path), name)
if path == w.path {
return nil
}
suffixes := []string{"", "-wal", "-shm"}
for _, suffix := range suffixes {
if fileExists(path + suffix) {
return fmt.Errorf(
"%w: %s", ErrArchiveNameTaken, name+suffix,
)
}
}
w.close()
for _, suffix := range suffixes {
err := os.Rename(w.path+suffix, path+suffix)
if err != nil && !errors.Is(err, fs.ErrNotExist) {
return fmt.Errorf(
"renaming archive %s to %s: %w", w.path, path, err,
)
}
}
w.path = path
return nil
}
// evict closes the writer's handle and marks it unusable. It is
// called when the writer leaves the registry, either because the
// webhook was deleted or because its last database target was
// removed. The archive FILE is deliberately left on disk: it is
// long-term storage an operator may still want.
// called when the writer leaves the registry, because its target
// or its webhook was deleted, or at shutdown. The archive FILE is
// deliberately left on disk: it is long-term storage an operator
// may still want.
func (w *archiveWriter) evict() {
w.mu.Lock()
defer w.mu.Unlock()