Two places decided whether a request was TLS, by two different means,
and they disagreed.
The session cookie's Secure attribute was fixed at startup from
!Config.IsDev(). "dev" is the environment when WEBHOOKER_ENVIRONMENT is
unset, so a deployment terminating TLS at a proxy without also setting
the environment shipped the authentication cookie with no Secure
attribute -- on the same response as a CSRF cookie that had one. It
failed silently: everything kept working, so nothing prompted anyone to
look.
The CSRF middleware's per-request check compared X-Forwarded-Proto with
== "https" exactly, so "HTTPS", "https, http" and "https,https" all took
the plaintext path. Uppercase is legal for a case-insensitive token and
the comma forms are what a proxy chained behind another proxy emits by
appending rather than replacing. On that path gorilla/csrf stops
enforcing the strict Referer check on a site that genuinely is HTTPS.
Both now go through internal/reqtls.IsTLS, which folds case and takes
the leftmost comma-separated element -- the hop nearest the client, and
so the one a cookie's Secure attribute is about. A third package is
needed because internal/middleware already imports internal/session, so
session cannot import middleware back.
Per-request beat a startup warning for the session cookie because it
turned out to need no restructuring: gorilla/sessions gives every
session its own copy of the store's Options and renders the cookie from
that copy, and every session-cookie write here already goes through
Session.Save or Session.Regenerate, both of which hold the request. The
store's template Secure becomes true so that a write path added later
which forgets to track the transport fails visibly instead of silently
dropping Secure.
The flag tracks the transport in both directions rather than latching
on. Secure over plaintext is discarded by the browser without an error,
which would make a plain-HTTP local run impossible to log into -- and
would also void the deletion cookies in Destroy and Regenerate, leaving
a session the user just tried to end still live.
A third site that makes this decision, internal/handlers'
BaseURL construction, assigns the raw header straight into the URL
scheme. It is left alone here and filed separately.
The Status section claimed next held the completed 1.0.0 milestone with
every issue closed. Four are open (#176, #178, #186, #187), so a merge
of next to main would have shipped that claim to main.
Next Step still named #115 and #125 as open owner decisions; both
landed. It now names the real open items, #150 and #112, and the forced
merge order for the remaining four.
Completed Steps was seventeen units behind, back to 2026-08-12.
DBURL → DATA_DIR consolidation:
- Remove DBURL env var entirely; main DB now lives at {DATA_DIR}/webhooker.db
- database.go constructs DB path from config.DataDir, ensures dir exists
- Update DATA_DIR prod default from /data/events to /data
- Update all tests to use DataDir instead of DBURL
- Update Dockerfile: /data (not /data/events) for all SQLite databases
- Update README configuration table, Docker examples, architecture docs
Dead code removal:
- Remove unused IndexResponse struct (handlers/index.go)
- Remove unused TemplateData struct (handlers/handlers.go)
Stale comment cleanup:
- Remove TODO in server.go (DB cleanup handled by fx lifecycle)
- Fix nolint:golint → nolint:revive on ServerParams for consistency
- Clean up verbose middleware/routing comments in routes.go
- Fix TODO fan-out description (worker pool, not goroutine-per-target)
.gitignore fixes:
- Add data/ directory to gitignore
- Remove stale config.yaml entry (env-only config since rework)
The webhook handler now builds DeliveryTask structs carrying all target
config and event data inline (for bodies ≤16KB) and sends them through
the delivery channel. In the happy path, the engine delivers without
reading from any database — it only writes to record delivery results.
For large bodies (≥16KB), Body is nil and the engine fetches it from the
per-webhook database on demand. Retry timers also carry the full
DeliveryTask, so retries avoid unnecessary DB reads.
The database is used for crash recovery only: on startup the engine scans
for interrupted pending/retrying deliveries and re-queues them.
Implements owner feedback from issue #15:
> the message in the <=16KB case should have everything it needs to do
> its delivery. it shouldn't touch the db until it has a success or
> failure to record.
Split data storage into main application DB (config only) and
per-webhook event databases (one SQLite file per webhook).
Architecture changes:
- New WebhookDBManager component manages per-webhook DB lifecycle
(create, open, cache, delete) with lazy connection pooling via sync.Map
- Main DB (DBURL) stores only config: Users, Webhooks, Entrypoints,
Targets, APIKeys
- Per-webhook DBs (DATA_DIR) store Events, Deliveries, DeliveryResults
in files named events-{webhook_uuid}.db
- New DATA_DIR env var (default: ./data dev, /data/events prod)
Behavioral changes:
- Webhook creation creates per-webhook DB file
- Webhook deletion hard-deletes per-webhook DB file (config soft-deleted)
- Event ingestion writes to per-webhook DB, not main DB
- Delivery engine polls all per-webhook DBs for pending deliveries
- Database target type marks delivery as immediately successful (events
are already in the dedicated per-webhook DB)
- Event log UI reads from per-webhook DBs with targets from main DB
- Existing webhooks without DB files get them created lazily
Removed:
- ArchivedEvent model (was a half-measure, replaced by per-webhook DBs)
- Event/Delivery/DeliveryResult removed from main DB migrations
Added:
- Comprehensive tests for WebhookDBManager (create, delete, lazy
creation, delivery workflow, multiple webhooks, close all)
- Dockerfile creates /data/events directory
README updates:
- Per-webhook event databases documented as implemented (was Phase 2)
- DATA_DIR added to configuration table
- Docker instructions updated with data volume mount
- Data model diagram updated
- TODO updated (database separation moved to completed)
Closes#15
clawbot