Four defects found by the integration review, each of which would have
made the README or the release notes untrue at the moment of tagging.
RETENTION_SWEEP_INTERVAL was absent from the README env table while two
other passages referred to it as documented. Enumerated every variable
read by internal/config from the source (12 in total) rather than by
eye; that was the only one missing.
TODO.md omitted five of the units landed in this milestone (#64, #79,
#90, #113, #118), two of them credential-exposure fixes, which are
precisely the entries a reader of the release notes wants to find. The
list is now derived from git log origin/main..origin/next.
The README sold Replay in the present tense as a core capability while
no redelivery code exists anywhere in the tree, and the roadmap entry
for it had been dropped without it being implemented. Every mention of
replay or redelivery in the README is now either marked planned or
already under a Planned heading: the Rationale item, the Use Cases
bullet, the Event model's field description, and the delivery-semantics
passage on target-type edits, which told an operator that a terminally
failed delivery was recoverable by hand when nothing can recover it.
The comment on failUnretryableRetry made the same claim and is
corrected with it. The roadmap entry is back in TODO.md.
The production JS asset shipped a console.log on load. The rest of the
file and every other shipped asset were checked; that was the only one
(alpine.min.js is vendored and untouched).
No behavioural change: the only non-comment, non-documentation edit is
the deleted console.log.
DBURL → DATA_DIR consolidation:
- Remove DBURL env var entirely; main DB now lives at {DATA_DIR}/webhooker.db
- database.go constructs DB path from config.DataDir, ensures dir exists
- Update DATA_DIR prod default from /data/events to /data
- Update all tests to use DataDir instead of DBURL
- Update Dockerfile: /data (not /data/events) for all SQLite databases
- Update README configuration table, Docker examples, architecture docs
Dead code removal:
- Remove unused IndexResponse struct (handlers/index.go)
- Remove unused TemplateData struct (handlers/handlers.go)
Stale comment cleanup:
- Remove TODO in server.go (DB cleanup handled by fx lifecycle)
- Fix nolint:golint → nolint:revive on ServerParams for consistency
- Clean up verbose middleware/routing comments in routes.go
- Fix TODO fan-out description (worker pool, not goroutine-per-target)
.gitignore fixes:
- Add data/ directory to gitignore
- Remove stale config.yaml entry (env-only config since rework)
The webhook handler now builds DeliveryTask structs carrying all target
config and event data inline (for bodies ≤16KB) and sends them through
the delivery channel. In the happy path, the engine delivers without
reading from any database — it only writes to record delivery results.
For large bodies (≥16KB), Body is nil and the engine fetches it from the
per-webhook database on demand. Retry timers also carry the full
DeliveryTask, so retries avoid unnecessary DB reads.
The database is used for crash recovery only: on startup the engine scans
for interrupted pending/retrying deliveries and re-queues them.
Implements owner feedback from issue #15:
> the message in the <=16KB case should have everything it needs to do
> its delivery. it shouldn't touch the db until it has a success or
> failure to record.
Split data storage into main application DB (config only) and
per-webhook event databases (one SQLite file per webhook).
Architecture changes:
- New WebhookDBManager component manages per-webhook DB lifecycle
(create, open, cache, delete) with lazy connection pooling via sync.Map
- Main DB (DBURL) stores only config: Users, Webhooks, Entrypoints,
Targets, APIKeys
- Per-webhook DBs (DATA_DIR) store Events, Deliveries, DeliveryResults
in files named events-{webhook_uuid}.db
- New DATA_DIR env var (default: ./data dev, /data/events prod)
Behavioral changes:
- Webhook creation creates per-webhook DB file
- Webhook deletion hard-deletes per-webhook DB file (config soft-deleted)
- Event ingestion writes to per-webhook DB, not main DB
- Delivery engine polls all per-webhook DBs for pending deliveries
- Database target type marks delivery as immediately successful (events
are already in the dedicated per-webhook DB)
- Event log UI reads from per-webhook DBs with targets from main DB
- Existing webhooks without DB files get them created lazily
Removed:
- ArchivedEvent model (was a half-measure, replaced by per-webhook DBs)
- Event/Delivery/DeliveryResult removed from main DB migrations
Added:
- Comprehensive tests for WebhookDBManager (create, delete, lazy
creation, delivery workflow, multiple webhooks, close all)
- Dockerfile creates /data/events directory
README updates:
- Per-webhook event databases documented as implemented (was Phase 2)
- DATA_DIR added to configuration table
- Docker instructions updated with data volume mount
- Data model diagram updated
- TODO updated (database separation moved to completed)
Closes#15
clawbot