sneak/webhooker
1
0
Fork 0
Code Issues 66 Pull Requests 3 Actions Packages Projects Releases Wiki Activity

configure db target type for archiving #43

New Issue
Closed
opened 2026-03-12 23:06:29 +01:00 by sneak · 1 comment
sneak commented 2026-03-12 23:06:29 +01:00
Owner
Copy Link
Copy Source

The database target type leverages this architecture: since events are already stored in the per-webhook database by design, the database target simply marks the delivery as immediately successful. The per-webhook DB IS the dedicated event database — that's the whole point of the database target type.

no. the database target type should have a configurable expiry (default: never) and should log events to a separate per-webhook archive sqlite file for long term storage. the file should be created if it does not exist and should be closed and reopened after each write (unless the last reopen was less than one second ago) and should support auto-recreate for the case where the archive sqlite db file is moved away for archiving.

> The database target type leverages this architecture: since events are already stored in the per-webhook database by design, the database target simply marks the delivery as immediately successful. The per-webhook DB IS the dedicated event database — that's the whole point of the database target type. no. the database target type should have a configurable expiry (default: never) and should log events to a separate per-webhook archive sqlite file for long term storage. the file should be created if it does not exist and should be closed and reopened after each write (unless the last reopen was less than one second ago) and should support auto-recreate for the case where the archive sqlite db file is moved away for archiving.
clawbot was assigned by sneak 2026-03-25 02:17:08 +01:00
clawbot removed their assignment 2026-07-25 12:10:14 +02:00
clawbot added this to the 1.0.0 milestone 2026-08-07 13:11:41 +02:00
clawbot commented 2026-08-07 17:20:21 +02:00
Collaborator
Copy Link
Copy Source

Implementation instructions

The Target interface (#77) is now merged, so implement this as the databaseTarget implementation.

Authoritative spec (from @sneak on this issue): the database target should have a configurable expiry (default: never) and should log events to a SEPARATE per-webhook archive sqlite file for long-term storage. The file is created if it does not exist; it is closed and reopened after each write (unless the last reopen was less than one second ago); and it supports auto-recreate for the case where the archive sqlite file is moved away for archiving.

Scope: internal/delivery/target_database.go plus a new archive-writer file it needs (keep it in the delivery or database package). Touch internal/config/config.go only if a genuinely global setting is required — prefer per-target config. Do NOT change the Target interface or other targets.

Behaviour:

  • databaseTarget.Deliver writes the full event (body, headers, method, content type, webhook id, entrypoint id, timestamp) as a row into a per-webhook archive sqlite file that is DISTINCT from the per-webhook event DB — e.g. archive-{webhookID}.db under the data dir. Create the file and its schema if missing.
  • After each write, close and reopen the archive handle, UNLESS the last reopen was less than 1 second ago (debounce). This lets an operator move the archive file away for offline archiving.
  • Auto-recreate: if the archive file was moved/removed since the last open, the next write recreates it (create-if-missing on open covers this).
  • Expiry: read an optional expiry from the target's config JSON (Target.Config), default "never" (keep forever). When an expiry is set, prune archived rows older than it — implementer's choice of mechanism (on open, or reuse the retention-sweep pattern), documented in the code. Default "never" means no pruning.
  • It stays fire-and-forget: record one successful attempt and mark delivered (as the current stub does), but now it actually archives.

Definition of done:

  • delivering to a database target writes the full event into a separate per-webhook archive sqlite file, created if absent
  • the handle is closed and reopened after each write with a 1-second debounce; moving the file away and delivering again recreates it
  • an optional per-target expiry (default never) prunes older archived rows when set
  • tests cover: a row is archived; the file is recreated after removal; the debounce; and expiry pruning when configured

Gates and process:

  • make fmt; validate with docker build . (must exit 0)
  • branch from main named issue-43-database-archiving; commit subject ends with (closes #43)
  • open a PR (base main) and comment on it with the diff summary and the docker build . result; no AI/tooling references
  • Note: main was just updated with the delivery refactor, the retention reaper, and NoCache. If docker build fails on something unrelated to your change, check whether an unmodified origin/main builds, and if it does not, STOP and report that main itself is broken rather than trying to fix it.
## Implementation instructions The `Target` interface (#77) is now merged, so implement this as the `databaseTarget` implementation. Authoritative spec (from @sneak on this issue): the database target should have a configurable expiry (default: never) and should log events to a SEPARATE per-webhook archive sqlite file for long-term storage. The file is created if it does not exist; it is closed and reopened after each write (unless the last reopen was less than one second ago); and it supports auto-recreate for the case where the archive sqlite file is moved away for archiving. Scope: `internal/delivery/target_database.go` plus a new archive-writer file it needs (keep it in the `delivery` or `database` package). Touch `internal/config/config.go` only if a genuinely global setting is required — prefer per-target config. Do NOT change the `Target` interface or other targets. Behaviour: - `databaseTarget.Deliver` writes the full event (body, headers, method, content type, webhook id, entrypoint id, timestamp) as a row into a per-webhook archive sqlite file that is DISTINCT from the per-webhook event DB — e.g. `archive-{webhookID}.db` under the data dir. Create the file and its schema if missing. - After each write, close and reopen the archive handle, UNLESS the last reopen was less than 1 second ago (debounce). This lets an operator move the archive file away for offline archiving. - Auto-recreate: if the archive file was moved/removed since the last open, the next write recreates it (create-if-missing on open covers this). - Expiry: read an optional expiry from the target's config JSON (`Target.Config`), default "never" (keep forever). When an expiry is set, prune archived rows older than it — implementer's choice of mechanism (on open, or reuse the retention-sweep pattern), documented in the code. Default "never" means no pruning. - It stays fire-and-forget: record one successful attempt and mark delivered (as the current stub does), but now it actually archives. Definition of done: - delivering to a `database` target writes the full event into a separate per-webhook archive sqlite file, created if absent - the handle is closed and reopened after each write with a 1-second debounce; moving the file away and delivering again recreates it - an optional per-target expiry (default never) prunes older archived rows when set - tests cover: a row is archived; the file is recreated after removal; the debounce; and expiry pruning when configured Gates and process: - `make fmt`; validate with `docker build .` (must exit 0) - branch from `main` named `issue-43-database-archiving`; commit subject ends with ` (closes #43)` - open a PR (base `main`) and comment on it with the diff summary and the `docker build .` result; no AI/tooling references - Note: `main` was just updated with the delivery refactor, the retention reaper, and NoCache. If `docker build` fails on something unrelated to your change, check whether an unmodified `origin/main` builds, and if it does not, STOP and report that `main` itself is broken rather than trying to fix it.
sneak closed this issue 2026-08-07 22:50:08 +02:00
Sign in to join this conversation.
No Branch/Tag Specified
Branches Tags
Labels
Clear labels
merge-ready
needs-checks
needs-rebase
needs-review
needs-rework
No labels
Milestone
No items
No Milestone 1.0.0
Projects
Clear projects
No projects
Assignees
Clear assignees
clawbot sneak (Jeffrey Paul)
No Assignees
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/webhooker#43