Move the archive files into their own package (closes #541)
check / check (push) Waiting to run
check / check (push) Waiting to run
The SQLite archive files the database target writes are now managed by internal/archive: file names, rotation, the writer of each target, pruning, renames, the gzipped JSON export, the idle sweep, and the rename lock, which was Handlers.renameMu. The database target stays in internal/delivery and writes through archive.Writers; the engine still closes the writers when it stops, so the stop order is unchanged. The handlers take the rename lock from archive.Archives.RenameLock, and rename and evict through archive.Archives. Exported names drop their Archive prefix in the new package. The tests moved with the code; the two that check a delivery's recorded attempt stay in delivery. Model: opus-5-5
This commit was merged in pull request #552.
This commit is contained in:
+11
-68
@@ -13,6 +13,7 @@ import (
|
||||
|
||||
"go.uber.org/fx"
|
||||
"gorm.io/gorm"
|
||||
"sneak.berlin/go/webhooker/internal/archive"
|
||||
"sneak.berlin/go/webhooker/internal/database"
|
||||
"sneak.berlin/go/webhooker/internal/globals"
|
||||
"sneak.berlin/go/webhooker/internal/lifecycle"
|
||||
@@ -124,31 +125,11 @@ type Notifier interface {
|
||||
Notify(tasks []Task)
|
||||
}
|
||||
|
||||
// Archives is how the handlers keep the database targets' archive
|
||||
// files in step with the configuration. Deleting a webhook or a
|
||||
// target releases the cached archive writers, whose open file
|
||||
// handles would otherwise outlive them; renaming one renames the
|
||||
// archive files, which are named for the webhook and the target
|
||||
// (see ArchiveFileName).
|
||||
//
|
||||
// It is deliberately separate from Notifier: archiving lifecycle
|
||||
// is not notification, and a small interface keeps the handlers
|
||||
// package free of any dependency on the engine's internals while
|
||||
// staying trivially fakeable in tests.
|
||||
//
|
||||
// Neither eviction deletes an archive file. Both are idempotent
|
||||
// and are no-ops for a webhook or target with no engine state.
|
||||
type Archives interface {
|
||||
EvictWebhook(webhookID string)
|
||||
EvictTarget(targetID string)
|
||||
Rename(targetID, webhookName, targetName string) error
|
||||
}
|
||||
|
||||
// CircuitBreakers is how the handlers read a target's circuit
|
||||
// breaker, so the webhook page and the event log can say that
|
||||
// deliveries to the target are paused and until when. Like Archives,
|
||||
// it keeps the handlers free of the engine's internals and is
|
||||
// trivially faked in tests.
|
||||
// deliveries to the target are paused and until when. It keeps the
|
||||
// handlers free of the engine's internals and is trivially faked in
|
||||
// tests.
|
||||
type CircuitBreakers interface {
|
||||
StateAndCooldown(targetID string) (CircuitState, time.Duration)
|
||||
}
|
||||
@@ -164,6 +145,7 @@ type EngineParams struct {
|
||||
Logger *logger.Logger
|
||||
SSRFGuard *Guard
|
||||
Metrics *metrics.Set
|
||||
Archives *archive.Writers
|
||||
}
|
||||
|
||||
// Engine processes queued deliveries in the background
|
||||
@@ -202,9 +184,10 @@ type Engine struct {
|
||||
httpTarget *httpTarget
|
||||
slackTarget *slackTarget
|
||||
|
||||
// dbTarget is retained so the engine can reach the archive
|
||||
// writer registry for eviction, renames and the idle sweep.
|
||||
dbTarget *databaseTarget
|
||||
// archives is the database targets' archive writers. The
|
||||
// database target writes each event through them, and stop
|
||||
// closes them.
|
||||
archives *archive.Writers
|
||||
|
||||
// inflight is the set of deliveries this engine currently owns.
|
||||
// Recovery and the sweeps re-dispatch only what it does not
|
||||
@@ -228,6 +211,7 @@ func New(
|
||||
workers: defaultWorkers,
|
||||
version: params.Globals.Version,
|
||||
mtr: params.Metrics,
|
||||
archives: params.Archives,
|
||||
}
|
||||
|
||||
e.initTargets(&http.Client{
|
||||
@@ -272,47 +256,6 @@ func (e *Engine) Notify(tasks []Task) {
|
||||
}
|
||||
}
|
||||
|
||||
// EvictWebhook implements Archives. The cached archive writer of
|
||||
// every database target of the webhook is dropped from the
|
||||
// registry and its file handle closed. The archive files
|
||||
// themselves are left on disk — they are long-term storage the
|
||||
// operator owns.
|
||||
func (e *Engine) EvictWebhook(webhookID string) {
|
||||
if e.dbTarget == nil {
|
||||
return
|
||||
}
|
||||
|
||||
e.dbTarget.evictWebhook(webhookID)
|
||||
}
|
||||
|
||||
// EvictTarget implements Archives. It is EvictWebhook for a single
|
||||
// database target, and leaves the archive file on disk the same
|
||||
// way.
|
||||
func (e *Engine) EvictTarget(targetID string) {
|
||||
if e.dbTarget == nil {
|
||||
return
|
||||
}
|
||||
|
||||
e.dbTarget.evict(targetID)
|
||||
}
|
||||
|
||||
// Rename implements Archives. It renames every one of a database
|
||||
// target's archive files to ArchiveFileName(webhookName, targetName,
|
||||
// targetID), each keeping the period in its name, under the lock the
|
||||
// target's archive writes and the idle sweep take. It never replaces
|
||||
// a file: if one already has a new name, the error is
|
||||
// ErrArchiveNameTaken. The caller renames before it saves the new
|
||||
// name: see databaseTarget.rename.
|
||||
func (e *Engine) Rename(
|
||||
targetID, webhookName, targetName string,
|
||||
) error {
|
||||
if e.dbTarget == nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
return e.dbTarget.rename(targetID, webhookName, targetName)
|
||||
}
|
||||
|
||||
// StateAndCooldown implements CircuitBreakers. It returns the state of
|
||||
// the target's circuit breaker and, while the breaker is open, what is
|
||||
// left of its cooldown; the cooldown is zero once that has passed and
|
||||
@@ -466,7 +409,7 @@ func (e *Engine) stop(ctx context.Context) error {
|
||||
return err
|
||||
}
|
||||
|
||||
e.dbTarget.evictAll()
|
||||
e.archives.EvictAll()
|
||||
|
||||
e.log.Info("delivery engine stopped")
|
||||
|
||||
|
||||
Reference in New Issue
Block a user