Move the archive files into their own package (closes #541)
check / check (push) Waiting to run

The SQLite archive files the database target writes are now managed by
internal/archive: file names, rotation, the writer of each target,
pruning, renames, the gzipped JSON export, the idle sweep, and the
rename lock, which was Handlers.renameMu. The database target stays in
internal/delivery and writes through archive.Writers; the engine still
closes the writers when it stops, so the stop order is unchanged. The
handlers take the rename lock from archive.Archives.RenameLock, and
rename and evict through archive.Archives.

Exported names drop their Archive prefix in the new package. The tests
moved with the code; the two that check a delivery's recorded attempt
stay in delivery.

Model: opus-5-5
This commit was merged in pull request #552.
This commit is contained in:
2026-10-15 21:03:43 +02:00
parent f10440b7cc
commit 0cdda43e6c
42 changed files with 2178 additions and 1772 deletions
+11 -68
View File
@@ -13,6 +13,7 @@ import (
"go.uber.org/fx"
"gorm.io/gorm"
"sneak.berlin/go/webhooker/internal/archive"
"sneak.berlin/go/webhooker/internal/database"
"sneak.berlin/go/webhooker/internal/globals"
"sneak.berlin/go/webhooker/internal/lifecycle"
@@ -124,31 +125,11 @@ type Notifier interface {
Notify(tasks []Task)
}
// Archives is how the handlers keep the database targets' archive
// files in step with the configuration. Deleting a webhook or a
// target releases the cached archive writers, whose open file
// handles would otherwise outlive them; renaming one renames the
// archive files, which are named for the webhook and the target
// (see ArchiveFileName).
//
// It is deliberately separate from Notifier: archiving lifecycle
// is not notification, and a small interface keeps the handlers
// package free of any dependency on the engine's internals while
// staying trivially fakeable in tests.
//
// Neither eviction deletes an archive file. Both are idempotent
// and are no-ops for a webhook or target with no engine state.
type Archives interface {
EvictWebhook(webhookID string)
EvictTarget(targetID string)
Rename(targetID, webhookName, targetName string) error
}
// CircuitBreakers is how the handlers read a target's circuit
// breaker, so the webhook page and the event log can say that
// deliveries to the target are paused and until when. Like Archives,
// it keeps the handlers free of the engine's internals and is
// trivially faked in tests.
// deliveries to the target are paused and until when. It keeps the
// handlers free of the engine's internals and is trivially faked in
// tests.
type CircuitBreakers interface {
StateAndCooldown(targetID string) (CircuitState, time.Duration)
}
@@ -164,6 +145,7 @@ type EngineParams struct {
Logger *logger.Logger
SSRFGuard *Guard
Metrics *metrics.Set
Archives *archive.Writers
}
// Engine processes queued deliveries in the background
@@ -202,9 +184,10 @@ type Engine struct {
httpTarget *httpTarget
slackTarget *slackTarget
// dbTarget is retained so the engine can reach the archive
// writer registry for eviction, renames and the idle sweep.
dbTarget *databaseTarget
// archives is the database targets' archive writers. The
// database target writes each event through them, and stop
// closes them.
archives *archive.Writers
// inflight is the set of deliveries this engine currently owns.
// Recovery and the sweeps re-dispatch only what it does not
@@ -228,6 +211,7 @@ func New(
workers: defaultWorkers,
version: params.Globals.Version,
mtr: params.Metrics,
archives: params.Archives,
}
e.initTargets(&http.Client{
@@ -272,47 +256,6 @@ func (e *Engine) Notify(tasks []Task) {
}
}
// EvictWebhook implements Archives. The cached archive writer of
// every database target of the webhook is dropped from the
// registry and its file handle closed. The archive files
// themselves are left on disk — they are long-term storage the
// operator owns.
func (e *Engine) EvictWebhook(webhookID string) {
if e.dbTarget == nil {
return
}
e.dbTarget.evictWebhook(webhookID)
}
// EvictTarget implements Archives. It is EvictWebhook for a single
// database target, and leaves the archive file on disk the same
// way.
func (e *Engine) EvictTarget(targetID string) {
if e.dbTarget == nil {
return
}
e.dbTarget.evict(targetID)
}
// Rename implements Archives. It renames every one of a database
// target's archive files to ArchiveFileName(webhookName, targetName,
// targetID), each keeping the period in its name, under the lock the
// target's archive writes and the idle sweep take. It never replaces
// a file: if one already has a new name, the error is
// ErrArchiveNameTaken. The caller renames before it saves the new
// name: see databaseTarget.rename.
func (e *Engine) Rename(
targetID, webhookName, targetName string,
) error {
if e.dbTarget == nil {
return nil
}
return e.dbTarget.rename(targetID, webhookName, targetName)
}
// StateAndCooldown implements CircuitBreakers. It returns the state of
// the target's circuit breaker and, while the breaker is open, what is
// left of its cooldown; the cooldown is zero once that has passed and
@@ -466,7 +409,7 @@ func (e *Engine) stop(ctx context.Context) error {
return err
}
e.dbTarget.evictAll()
e.archives.EvictAll()
e.log.Info("delivery engine stopped")