Add a Download button that exports a database target's archive as gzipped JSON (closes #374)
check / check (push) Successful in 3m31s
check / check (push) Successful in 3m31s
Each database target on the webhook page links to /hook/ID/targets/TARGETID/download, which streams the target's archive as archive-WEBHOOK-TARGET-YYYYMMDDTHHMMSSZ.json.gz: the webhook, the target, exported_at, and archived_events, one object per row keyed by column, a body that is not valid UTF-8 in base64 with body_encoding. The export reads one row at a time on its own connection inside a read-only transaction: one snapshot, and no write lock. The handler holds the rename lock only while it reads the names and opens the file. A missing archive exports empty and is not created. Model: opus-5-5
This commit is contained in:
@@ -1992,6 +1992,30 @@ Because each `database` target has its own archive file, a target's
|
||||
webhook with different expiries keep two archives, each pruned on its
|
||||
own schedule.
|
||||
|
||||
Each `database` target on the webhook page has a **Download** button,
|
||||
which returns its archive as one gzipped JSON file,
|
||||
`archive-{webhook_name}-{target_name}-{YYYYMMDDTHHMMSSZ}.json.gz`, the
|
||||
names made safe as above and the time in UTC. The file holds one
|
||||
object: `webhook` and `target`, each an `id` and a `name`;
|
||||
`exported_at`; and `archived_events`, one object per archived row with
|
||||
every column, keyed by column name. A body that is not valid UTF-8 is
|
||||
written in base64, with `"body_encoding": "base64"` beside it. An
|
||||
archive that does not exist yet, or was moved away, downloads with an
|
||||
empty `archived_events`; the download never creates the file.
|
||||
|
||||
The download streams: each row is read and written out compressed
|
||||
before the next is read, so neither the archive nor the JSON is held in
|
||||
memory. It reads on a connection of its own, inside one read-only
|
||||
transaction, so the file holds the archive as it stood when the
|
||||
download started, and archive writes go on meanwhile, since under WAL a
|
||||
reader never blocks a writer. While it runs, the `-wal` cannot be
|
||||
checkpointed past what it reads, so a long download lets the `-wal`
|
||||
grow. It finds the file by the stored names under the lock that webhook
|
||||
edits, target edits and target creation hold, and lets go once the file
|
||||
is open: a rename during the download moves the file without affecting
|
||||
it. Like every request it is cut off after 60 seconds, which leaves the
|
||||
file incomplete and failing to decompress.
|
||||
|
||||
Deleting a webhook releases its archives: the delivery engine's cached
|
||||
archive writers are dropped and their file handles closed, so nothing
|
||||
lingers after the webhook is gone. The archive **files themselves are
|
||||
@@ -2854,6 +2878,7 @@ returns to the page that was asked for.
|
||||
| `POST` | `/hook/{id}/targets` | Add target to webhook |
|
||||
| `GET` | `/hook/{id}/targets/{targetID}/edit` | Edit target form. The one page that renders a target's destination URL and header values in full, rather than masked |
|
||||
| `POST` | `/hook/{id}/targets/{targetID}/edit` | Edit target submission |
|
||||
| `GET` | `/hook/{id}/targets/{targetID}/download` | Download a `database` target's archive as one gzipped JSON file. See [Database Architecture](#database-architecture) |
|
||||
| `POST` | `/hook/{id}/targets/{targetID}/delete` | Delete a target |
|
||||
| `POST` | `/hook/{id}/targets/{targetID}/toggle` | Enable or disable a target |
|
||||
|
||||
@@ -2934,6 +2959,7 @@ webhooker/
|
||||
│ │ ├── target_slack.go # Slack/Mattermost incoming-webhook target
|
||||
│ │ ├── target_database.go # Database archive target
|
||||
│ │ ├── target_database_archive.go # Archive file lifecycle and pruning
|
||||
│ │ ├── target_database_export.go # Archive download as gzipped JSON
|
||||
│ │ ├── target_log.go # Log target (stdout)
|
||||
│ │ ├── target_config_view.go # Masked target config for templates
|
||||
│ │ ├── archive_sweeper.go # Periodic pruning of idle archives
|
||||
|
||||
Reference in New Issue
Block a user