feat: implement per-webhook event databases
All checks were successful
check / check (push) Successful in 1m50s

Split data storage into main application DB (config only) and
per-webhook event databases (one SQLite file per webhook).

Architecture changes:
- New WebhookDBManager component manages per-webhook DB lifecycle
  (create, open, cache, delete) with lazy connection pooling via sync.Map
- Main DB (DBURL) stores only config: Users, Webhooks, Entrypoints,
  Targets, APIKeys
- Per-webhook DBs (DATA_DIR) store Events, Deliveries, DeliveryResults
  in files named events-{webhook_uuid}.db
- New DATA_DIR env var (default: ./data dev, /data/events prod)

Behavioral changes:
- Webhook creation creates per-webhook DB file
- Webhook deletion hard-deletes per-webhook DB file (config soft-deleted)
- Event ingestion writes to per-webhook DB, not main DB
- Delivery engine polls all per-webhook DBs for pending deliveries
- Database target type marks delivery as immediately successful (events
  are already in the dedicated per-webhook DB)
- Event log UI reads from per-webhook DBs with targets from main DB
- Existing webhooks without DB files get them created lazily

Removed:
- ArchivedEvent model (was a half-measure, replaced by per-webhook DBs)
- Event/Delivery/DeliveryResult removed from main DB migrations

Added:
- Comprehensive tests for WebhookDBManager (create, delete, lazy
  creation, delivery workflow, multiple webhooks, close all)
- Dockerfile creates /data/events directory

README updates:
- Per-webhook event databases documented as implemented (was Phase 2)
- DATA_DIR added to configuration table
- Docker instructions updated with data volume mount
- Data model diagram updated
- TODO updated (database separation moved to completed)

Closes #15
This commit is contained in:
clawbot
2026-03-01 17:06:43 -08:00
parent 6c393ccb78
commit 43c22a9e9a
13 changed files with 814 additions and 198 deletions

View File

@@ -19,11 +19,12 @@ import (
// nolint:revive // HandlersParams is a standard fx naming convention
type HandlersParams struct {
fx.In
Logger *logger.Logger
Globals *globals.Globals
Database *database.Database
Healthcheck *healthcheck.Healthcheck
Session *session.Session
Logger *logger.Logger
Globals *globals.Globals
Database *database.Database
WebhookDBMgr *database.WebhookDBManager
Healthcheck *healthcheck.Healthcheck
Session *session.Session
}
type Handlers struct {
@@ -31,6 +32,7 @@ type Handlers struct {
log *slog.Logger
hc *healthcheck.Healthcheck
db *database.Database
dbMgr *database.WebhookDBManager
session *session.Session
templates map[string]*template.Template
}
@@ -53,6 +55,7 @@ func New(lc fx.Lifecycle, params HandlersParams) (*Handlers, error) {
s.log = params.Logger.Get()
s.hc = params.Healthcheck
s.db = params.Database
s.dbMgr = params.WebhookDBMgr
s.session = params.Session
// Parse all page templates once at startup