Move the archive files into their own package (closes #541)
check / check (push) Canceled after 0s

The SQLite archive files the database target writes are now managed by
internal/archive: file names, rotation, the writer of each target,
pruning, renames, the gzipped JSON export, the idle sweep, and the
rename lock, which was Handlers.renameMu. The database target stays in
internal/delivery and writes through archive.Writers; the engine still
closes the writers when it stops, so the stop order is unchanged. The
handlers take the rename lock from archive.Archives.RenameLock, and
rename and evict through archive.Archives.

Exported names drop their Archive prefix in the new package. The tests
moved with the code; the two that check a delivery's recorded attempt
stay in delivery.

Model: opus-5-5
This commit is contained in:
2026-10-15 18:42:29 +00:00
parent f10440b7cc
commit ab7eec4b3e
42 changed files with 2178 additions and 1772 deletions
+25 -21
View File
@@ -2100,15 +2100,15 @@ writes go on meanwhile, since under WAL a reader never blocks a writer. Until
the open file is closed its `-wal` cannot be checkpointed past what the download
reads, so a long download lets that `-wal` grow.
Deleting a webhook releases its archives: the delivery engine's cached archive
writers are dropped and their file handles closed, so nothing lingers after the
webhook is gone. The archive **files themselves are deliberately left on disk**.
Unlike the event database — per-webhook working storage that is hard-deleted
with the webhook — an archive is long-term storage an operator may still want to
keep or move away for offline retention, and destroying it as a side effect of
deleting a webhook would be unrecoverable. Removing an `archive-….db` is the
operator's call. Deleting a `database` target releases its writer the same way,
and for the same reason leaves its file alone.
Deleting a webhook releases its archives: its cached archive writers are dropped
and their file handles closed, so nothing lingers after the webhook is gone. The
archive **files themselves are deliberately left on disk**. Unlike the event
database — per-webhook working storage that is hard-deleted with the webhook —
an archive is long-term storage an operator may still want to keep or move away
for offline retention, and destroying it as a side effect of deleting a webhook
would be unrecoverable. Removing an `archive-….db` is the operator's call.
Deleting a `database` target releases its writer the same way, and for the same
reason leaves its file alone.
The **Slack target type** sends webhook events as formatted messages to any
Slack-compatible incoming webhook URL (works with Slack, Mattermost, and other
@@ -2997,6 +2997,12 @@ webhooker/
├── cmd/webhooker/
│ └── main.go # Entry point: subcommand dispatch; no args locks DATA_DIR and wires fx
├── internal/
│ ├── archive/
│ │ ├── archive.go # The archive writer of each database target, eviction, rename, and the rename lock
│ │ ├── writer.go # Archive file lifecycle and pruning
│ │ ├── rotation.go # Archive rotation and file names
│ │ ├── download.go # Archive download as gzipped JSON
│ │ └── sweeper.go # Periodic pruning of idle archives
│ ├── banner/
│ │ └── banner.go # Ruled block for the one credential shown in the clear
│ ├── resetpw/
@@ -3040,13 +3046,9 @@ webhooker/
│ │ ├── target.go # Target interface, Task, Scheduler
│ │ ├── target_http.go # HTTP target (retries, circuit breaker)
│ │ ├── target_slack.go # Slack/Mattermost incoming-webhook target
│ │ ├── target_database.go # Database archive target
│ │ ├── target_database_archive.go # Archive file lifecycle and pruning
│ │ ├── target_database_rotation.go # Archive rotation and file names
│ │ ├── target_database_export.go # Archive download as gzipped JSON
│ │ ├── target_database.go # Database archive target, writing through internal/archive
│ │ ├── target_log.go # Log target (stdout)
│ │ ├── target_config_view.go # Masked target config for templates
│ │ ├── archive_sweeper.go # Periodic pruning of idle archives
│ │ ├── queue_depth.go # Periodic sampler behind the queue-depth gauges
│ │ ├── url_mask.go # Strips credentials from *url.Error
│ │ └── ssrf.go # SSRF prevention (IP validation, safe HTTP transport)
@@ -3137,14 +3139,16 @@ Components are wired via Uber fx in this order:
11. `metrics.New` — The delivery collectors, registered on that registry
12. `middleware.New` — HTTP middleware
13. `delivery.New` — Event-driven delivery engine
14. `delivery.NewArchiveSweeper` — Periodic pruning of idle archives
15. `delivery.Engine` → `delivery.Notifier` — interface bridge
16. `delivery.Engine` → `delivery.Archives` — interface bridge so deleting or
14. `archive.NewWriters` — The database targets' archive writers, and the lock
every rename of their files runs under
15. `archive.NewSweeper` — Periodic pruning of idle archives
16. `delivery.Engine` → `delivery.Notifier` — interface bridge
17. `archive.Writers` → `archive.Archives` — interface bridge so deleting or
renaming a webhook or target reaches its archive files
17. `server.New` — HTTP server and router
18. `server.New` — HTTP server and router
The server starts via
`fx.Invoke(func(*server.Server, *delivery.Engine, *database.RetentionReaper, *delivery.ArchiveSweeper) {})`,
`fx.Invoke(func(*server.Server, *delivery.Engine, *database.RetentionReaper, *archive.Sweeper) {})`,
which triggers the fx lifecycle hooks in dependency order. The
`delivery.Notifier` interface allows the webhook handler to send self-contained
`delivery.Task` slices to the engine without a direct package dependency. Each
@@ -3313,7 +3317,7 @@ order under a **5 second budget** (`fx.StopTimeout` in `cmd/webhooker/main.go`).
That budget covers the whole sequence, not each hook. The order, read off the fx
stop-hook log:
1. `ArchiveSweeper`
1. `archive.Sweeper`
2. `RetentionReaper`
3. `server` — the HTTP drain, bounded by `server.ShutdownTimeout` (**3
seconds**) and by what the hooks before it left, then a Sentry flush if
@@ -3356,7 +3360,7 @@ skipped when that leaves too little to be worth attempting — so a full-length
drain means Sentry events are dropped rather than the database close being
skipped.
This does not make the database close unconditional. A slow `ArchiveSweeper` or
This does not make the database close unconditional. A slow `archive.Sweeper` or
`RetentionReaper` is enough to cut the shutdown short, not only one that
consumes the whole budget: what they spend comes out of the drain first, so
after 2 seconds of theirs a request still in flight gets 1 second to finish, and