4 Commits

This Branch

This Branch

All Branches

Author	SHA1	Message	Date
clawbot	13de7cd290	Allow retention_days of 0 to mean retain forever (closes #79) ... RetentionDays carried gorm:"default:30", so GORM substituted 30 for a zero value while building the insert. A webhook could therefore never be configured to keep its events indefinitely: the reaper's retain-forever branch existed but was unreachable from the normal create and edit flows. Introduce database.RetentionForeverDays = 365 * 1000 as the sentinel for "retain forever" and a Webhook.BeforeSave hook that rewrites any non-positive RetentionDays to it. The rewrite has to live in the hook rather than at the call sites: GORM applies the column default while converting the model to insert values, which happens after BeforeSave, so anything later loses that race. Putting it on the model also means a future call site, such as the planned REST API, cannot bypass it. The reaper now skips a webhook when Webhook.RetainsForever reports true, which recognises the sentinel and keeps honouring the old <= 0 values for rows written before it existed. Without this the sentinel, being positive, would have produced a cutoff a thousand years in the past and a DELETE matching nothing on every sweep. Bound the finite retention range, which was previously unbounded on the server. The reaper computes its cutoff as a time.Duration, an int64 nanosecond count, so a day count above 106751 overflows, wraps the span negative, and moves the cutoff into the far future — where it matches every row and the sweep deletes every event, delivery, and delivery result the webhook has, including ones created seconds ago. Nothing rejected such a value: parseRetention accepted any v > 0, and max="365" was a client-side attribute a direct POST ignored, so the wipe was already reachable on main and removing that attribute would have made it reachable by ordinary use. The bound is database.MaxFiniteRetentionDays, derived from the arithmetic itself as math.MaxInt64 / time.Hour / hoursPerDay rather than picked as a round number, and a finite value above it is now a 400 that names the ceiling. retentionCutoff additionally clamps the day count it is given and reports whether any cutoff applies at all, so a row written by an older version, a migration, or a future call site cannot reach the overflow either. A value at or above the retain-forever sentinel stays accepted, because that is what the edit form pre-fills for a retain-forever webhook. Form handling is shared by create and edit through parseRetentionDays so the two cannot drift: an empty field keeps the previous behaviour (default on create, unchanged on edit), 0 is honoured, and an unparseable, negative, or out-of-range value is a 400 that re-renders the form rather than a silently substituted default. The two rejection reasons are distinct sentinel errors so the message can name the ceiling, and the create form now carries the submitted name and description back into the re-rendered inputs, which the edit form already did. The retention inputs drop max="365". That cap was not cosmetic: the edit form pre-fills the stored value, so a retain-forever webhook rendered 365000 into an input capped at 365 and browser validation would have blocked saving any edit to it. min becomes 0 with a hint explaining what 0 does, and the list and detail views render a RetentionLabel of "forever" instead of a raw day count. All three Webhook methods take pointer receivers, so there is no receiver mix and no lint suppression: BeforeSave must take a pointer to mutate the record, and the handlers hand templates a *Webhook because html/template cannot call a pointer method on a value held in a map. The 30-day default is consolidated into database.DefaultRetentionDays, referenced from the handler and from the create form's pre-filled value, with a test asserting it agrees with the struct tag that cannot reference it.	2026-08-09 02:57:28 +00:00
clawbot	4dd4dfa5eb	chore: consolidate DBURL into DATA_DIR, codebase audit for 1.0.0 ... DBURL → DATA_DIR consolidation: - Remove DBURL env var entirely; main DB now lives at {DATA_DIR}/webhooker.db - database.go constructs DB path from config.DataDir, ensures dir exists - Update DATA_DIR prod default from /data/events to /data - Update all tests to use DataDir instead of DBURL - Update Dockerfile: /data (not /data/events) for all SQLite databases - Update README configuration table, Docker examples, architecture docs Dead code removal: - Remove unused IndexResponse struct (handlers/index.go) - Remove unused TemplateData struct (handlers/handlers.go) Stale comment cleanup: - Remove TODO in server.go (DB cleanup handled by fx lifecycle) - Fix nolint:golint → nolint:revive on ServerParams for consistency - Clean up verbose middleware/routing comments in routes.go - Fix TODO fan-out description (worker pool, not goroutine-per-target) .gitignore fixes: - Add data/ directory to gitignore - Remove stale config.yaml entry (env-only config since rework)	2026-03-01 23:33:20 -08:00
clawbot	32bd40b313	refactor: self-contained delivery tasks — engine delivers without DB reads in happy path ... The webhook handler now builds DeliveryTask structs carrying all target config and event data inline (for bodies ≤16KB) and sends them through the delivery channel. In the happy path, the engine delivers without reading from any database — it only writes to record delivery results. For large bodies (≥16KB), Body is nil and the engine fetches it from the per-webhook database on demand. Retry timers also carry the full DeliveryTask, so retries avoid unnecessary DB reads. The database is used for crash recovery only: on startup the engine scans for interrupted pending/retrying deliveries and re-queues them. Implements owner feedback from issue #15: > the message in the <=16KB case should have everything it needs to do > its delivery. it shouldn't touch the db until it has a success or > failure to record.	2026-03-01 22:09:41 -08:00
clawbot	43c22a9e9a	feat: implement per-webhook event databases ... Split data storage into main application DB (config only) and per-webhook event databases (one SQLite file per webhook). Architecture changes: - New WebhookDBManager component manages per-webhook DB lifecycle (create, open, cache, delete) with lazy connection pooling via sync.Map - Main DB (DBURL) stores only config: Users, Webhooks, Entrypoints, Targets, APIKeys - Per-webhook DBs (DATA_DIR) store Events, Deliveries, DeliveryResults in files named events-{webhook_uuid}.db - New DATA_DIR env var (default: ./data dev, /data/events prod) Behavioral changes: - Webhook creation creates per-webhook DB file - Webhook deletion hard-deletes per-webhook DB file (config soft-deleted) - Event ingestion writes to per-webhook DB, not main DB - Delivery engine polls all per-webhook DBs for pending deliveries - Database target type marks delivery as immediately successful (events are already in the dedicated per-webhook DB) - Event log UI reads from per-webhook DBs with targets from main DB - Existing webhooks without DB files get them created lazily Removed: - ArchivedEvent model (was a half-measure, replaced by per-webhook DBs) - Event/Delivery/DeliveryResult removed from main DB migrations Added: - Comprehensive tests for WebhookDBManager (create, delete, lazy creation, delivery workflow, multiple webhooks, close all) - Dockerfile creates /data/events directory README updates: - Per-webhook event databases documented as implemented (was Phase 2) - DATA_DIR added to configuration table - Docker instructions updated with data volume mount - Data model diagram updated - TODO updated (database separation moved to completed) Closes #15	2026-03-01 17:06:43 -08:00