Move the archive files into their own package (closes #541)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
The SQLite archive files the database target writes are now managed by internal/archive: file names, rotation, the writer of each target, pruning, renames, the gzipped JSON export, the idle sweep, and the rename lock, which was Handlers.renameMu. The database target stays in internal/delivery and writes through archive.Writers; the engine still closes the writers when it stops, so the stop order is unchanged. The handlers take the rename lock from archive.Archives.RenameLock, and rename and evict through archive.Archives. Exported names drop their Archive prefix in the new package. The tests moved with the code; the two that check a delivery's recorded attempt stay in delivery. Model: opus-5-5
This commit is contained in:
@@ -2100,15 +2100,15 @@ writes go on meanwhile, since under WAL a reader never blocks a writer. Until
|
||||
the open file is closed its `-wal` cannot be checkpointed past what the download
|
||||
reads, so a long download lets that `-wal` grow.
|
||||
|
||||
Deleting a webhook releases its archives: the delivery engine's cached archive
|
||||
writers are dropped and their file handles closed, so nothing lingers after the
|
||||
webhook is gone. The archive **files themselves are deliberately left on disk**.
|
||||
Unlike the event database — per-webhook working storage that is hard-deleted
|
||||
with the webhook — an archive is long-term storage an operator may still want to
|
||||
keep or move away for offline retention, and destroying it as a side effect of
|
||||
deleting a webhook would be unrecoverable. Removing an `archive-….db` is the
|
||||
operator's call. Deleting a `database` target releases its writer the same way,
|
||||
and for the same reason leaves its file alone.
|
||||
Deleting a webhook releases its archives: its cached archive writers are dropped
|
||||
and their file handles closed, so nothing lingers after the webhook is gone. The
|
||||
archive **files themselves are deliberately left on disk**. Unlike the event
|
||||
database — per-webhook working storage that is hard-deleted with the webhook —
|
||||
an archive is long-term storage an operator may still want to keep or move away
|
||||
for offline retention, and destroying it as a side effect of deleting a webhook
|
||||
would be unrecoverable. Removing an `archive-….db` is the operator's call.
|
||||
Deleting a `database` target releases its writer the same way, and for the same
|
||||
reason leaves its file alone.
|
||||
|
||||
The **Slack target type** sends webhook events as formatted messages to any
|
||||
Slack-compatible incoming webhook URL (works with Slack, Mattermost, and other
|
||||
@@ -2997,6 +2997,12 @@ webhooker/
|
||||
├── cmd/webhooker/
|
||||
│ └── main.go # Entry point: subcommand dispatch; no args locks DATA_DIR and wires fx
|
||||
├── internal/
|
||||
│ ├── archive/
|
||||
│ │ ├── archive.go # The archive writer of each database target, eviction, rename, and the rename lock
|
||||
│ │ ├── writer.go # Archive file lifecycle and pruning
|
||||
│ │ ├── rotation.go # Archive rotation and file names
|
||||
│ │ ├── download.go # Archive download as gzipped JSON
|
||||
│ │ └── sweeper.go # Periodic pruning of idle archives
|
||||
│ ├── banner/
|
||||
│ │ └── banner.go # Ruled block for the one credential shown in the clear
|
||||
│ ├── resetpw/
|
||||
@@ -3040,13 +3046,9 @@ webhooker/
|
||||
│ │ ├── target.go # Target interface, Task, Scheduler
|
||||
│ │ ├── target_http.go # HTTP target (retries, circuit breaker)
|
||||
│ │ ├── target_slack.go # Slack/Mattermost incoming-webhook target
|
||||
│ │ ├── target_database.go # Database archive target
|
||||
│ │ ├── target_database_archive.go # Archive file lifecycle and pruning
|
||||
│ │ ├── target_database_rotation.go # Archive rotation and file names
|
||||
│ │ ├── target_database_export.go # Archive download as gzipped JSON
|
||||
│ │ ├── target_database.go # Database archive target, writing through internal/archive
|
||||
│ │ ├── target_log.go # Log target (stdout)
|
||||
│ │ ├── target_config_view.go # Masked target config for templates
|
||||
│ │ ├── archive_sweeper.go # Periodic pruning of idle archives
|
||||
│ │ ├── queue_depth.go # Periodic sampler behind the queue-depth gauges
|
||||
│ │ ├── url_mask.go # Strips credentials from *url.Error
|
||||
│ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport)
|
||||
@@ -3137,14 +3139,16 @@ Components are wired via Uber fx in this order:
|
||||
11. `metrics.New` — The delivery collectors, registered on that registry
|
||||
12. `middleware.New` — HTTP middleware
|
||||
13. `delivery.New` — Event-driven delivery engine
|
||||
14. `delivery.NewArchiveSweeper` — Periodic pruning of idle archives
|
||||
15. `delivery.Engine` → `delivery.Notifier` — interface bridge
|
||||
16. `delivery.Engine` → `delivery.Archives` — interface bridge so deleting or
|
||||
14. `archive.NewWriters` — The database targets' archive writers, and the lock
|
||||
every rename of their files runs under
|
||||
15. `archive.NewSweeper` — Periodic pruning of idle archives
|
||||
16. `delivery.Engine` → `delivery.Notifier` — interface bridge
|
||||
17. `archive.Writers` → `archive.Archives` — interface bridge so deleting or
|
||||
renaming a webhook or target reaches its archive files
|
||||
17. `server.New` — HTTP server and router
|
||||
18. `server.New` — HTTP server and router
|
||||
|
||||
The server starts via
|
||||
`fx.Invoke(func(*server.Server, *delivery.Engine, *database.RetentionReaper, *delivery.ArchiveSweeper) {})`,
|
||||
`fx.Invoke(func(*server.Server, *delivery.Engine, *database.RetentionReaper, *archive.Sweeper) {})`,
|
||||
which triggers the fx lifecycle hooks in dependency order. The
|
||||
`delivery.Notifier` interface allows the webhook handler to send self-contained
|
||||
`delivery.Task` slices to the engine without a direct package dependency. Each
|
||||
@@ -3313,7 +3317,7 @@ order under a **5 second budget** (`fx.StopTimeout` in `cmd/webhooker/main.go`).
|
||||
That budget covers the whole sequence, not each hook. The order, read off the fx
|
||||
stop-hook log:
|
||||
|
||||
1. `ArchiveSweeper`
|
||||
1. `archive.Sweeper`
|
||||
2. `RetentionReaper`
|
||||
3. `server` — the HTTP drain, bounded by `server.ShutdownTimeout` (**3
|
||||
seconds**) and by what the hooks before it left, then a Sentry flush if
|
||||
@@ -3356,7 +3360,7 @@ skipped when that leaves too little to be worth attempting — so a full-length
|
||||
drain means Sentry events are dropped rather than the database close being
|
||||
skipped.
|
||||
|
||||
This does not make the database close unconditional. A slow `ArchiveSweeper` or
|
||||
This does not make the database close unconditional. A slow `archive.Sweeper` or
|
||||
`RetentionReaper` is enough to cut the shutdown short, not only one that
|
||||
consumes the whole budget: what they spend comes out of the drain first, so
|
||||
after 2 seconds of theirs a request still in flight gets 1 second to finish, and
|
||||
|
||||
Reference in New Issue
Block a user