The CSRF middleware previously tied its PlaintextHTTPRequest wrapping
and cookie Secure flag to the IsDev() environment check. This meant
production mode always assumed HTTPS, which broke login in two common
deployment scenarios:
1. Production behind a TLS-terminating reverse proxy: gorilla/csrf
assumed HTTPS but r.TLS was nil, causing Origin/Referer scheme
mismatches and 'referer not supplied' errors.
2. Production over direct HTTP (testing/development with prod config):
the Secure cookie flag prevented the browser from sending the CSRF
cookie back on POST, causing 'CSRF token invalid' errors.
The fix detects the actual transport protocol per-request using r.TLS
(direct TLS) and the X-Forwarded-Proto header (reverse proxy). Two
gorilla/csrf instances are maintained — one with Secure cookies for TLS
and one without for plaintext — since the csrf.Secure option is set at
creation time. Both instances share the same signing key, so cookies
are interchangeable between them.
Behavior after fix:
- Direct TLS: Secure cookies, strict Origin/Referer checks
- Behind TLS proxy (X-Forwarded-Proto: https): same as direct TLS
- Plaintext HTTP: non-Secure cookies, relaxed Origin/Referer checks
(csrf.PlaintextHTTPRequest), token validation still enforced
Closes#53
Closes#45.
## Problem
1. The README didn't clearly explain what `WEBHOOKER_ENVIRONMENT=dev` vs `prod` actually changes.
2. The dev-mode default for `DATA_DIR` was `./data` — a relative path whose meaning depends on the working directory. There's no reason to use a relative path even in development.
## Changes
### Code (`internal/config/config.go`)
- Replace the dev default `DATA_DIR` from `./data` to `$XDG_DATA_HOME/webhooker` (falling back to `$HOME/.local/share/webhooker`). This follows the XDG Base Directory Specification and ensures the data directory is always an absolute path regardless of the working directory.
- Add `devDataDir()` helper that resolves the XDG path, with a `/tmp/webhooker` last-resort fallback if `$HOME` can't be determined.
### Tests (`internal/config/config_test.go`)
- `TestDevDataDir`: verifies XDG_DATA_HOME is respected, HOME fallback works, and the result is always absolute.
- `TestDevDefaultDataDirIsAbsolute`: integration test that creates a full Config via fx and asserts the dev default DataDir is absolute.
### README
- Add a table documenting exactly what `dev` vs `prod` changes: DATA_DIR default, CORS policy, and session cookie Secure flag.
- Clarify that log format and security headers are independent of the environment setting.
- Update the DATA_DIR default in the configuration variable table.
Co-authored-by: clawbot <clawbot@eeqj.de>
Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #46
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
## Summary
Adds a new `slack` target type that sends webhook events as formatted messages to any Slack-compatible incoming webhook URL (Slack, Mattermost, and other compatible services).
closes#44
## What it does
When a webhook event is received, the Slack target:
1. Formats a human-readable message with event metadata (HTTP method, content type, timestamp, body size)
2. Pretty-prints the payload in a code block — JSON payloads get indented formatting, non-JSON payloads are shown as raw text
3. Truncates large payloads at 3500 characters to keep Slack messages reasonable
4. POSTs the message as a `{"text": "..."}` JSON payload to the configured webhook URL
## Changes
- **`internal/database/model_target.go`** — Add `TargetTypeSlack` constant
- **`internal/delivery/engine.go`** — Add `SlackTargetConfig` struct, `deliverSlack` method, `FormatSlackMessage` function (exported), `parseSlackConfig` helper. Route slack targets in `processDelivery` switch.
- **`internal/handlers/source_management.go`** — Handle `slack` type in `HandleTargetCreate`, building `webhook_url` config from the URL form field
- **`templates/source_detail.html`** — Add "Slack" option to target type dropdown with URL field and helper text
- **`README.md`** — Document the new target type, update roadmap
## Tests
- `TestParseSlackConfig_Valid` / `_Empty` / `_MissingWebhookURL` — Config parsing
- `TestFormatSlackMessage_JSONBody` / `_NonJSONBody` / `_EmptyBody` / `_LargeJSONTruncated` — Message formatting
- `TestDeliverSlack_Success` / `_Failure` / `_InvalidConfig` — End-to-end delivery
- `TestProcessDelivery_RoutesToSlack` — Routing from processDelivery switch
All existing tests continue to pass. `docker build .` (which runs `make check`) passes clean.
Co-authored-by: user <user@Mac.lan guest wan>
Reviewed-on: #47
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
## Summary
This PR implements three security hardening measures:
### Security Headers Middleware (closes #34)
Adds a `SecurityHeaders()` middleware applied globally to all routes. Every response now includes:
- `Strict-Transport-Security: max-age=63072000; includeSubDomains; preload`
- `X-Content-Type-Options: nosniff`
- `X-Frame-Options: DENY`
- `Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'`
- `Referrer-Policy: strict-origin-when-cross-origin`
- `Permissions-Policy: camera=(), microphone=(), geolocation=()`
### Session Fixation Prevention (closes #38)
Adds a `Regenerate()` method to the session manager that destroys the old session and creates a new one with a fresh ID, copying all session values. Called after successful login to prevent session fixation attacks.
### Request Body Size Limits (closes #39)
Adds a `MaxBodySize()` middleware using `http.MaxBytesReader` to limit POST/PUT/PATCH request bodies to 1 MB. Applied to all form endpoints (`/pages`, `/sources`, `/source/*`).
## Files Changed
- `internal/middleware/middleware.go` — Added `SecurityHeaders()` and `MaxBodySize()` middleware
- `internal/session/session.go` — Added `Regenerate()` method for session fixation prevention
- `internal/handlers/auth.go` — Updated login handler to regenerate session after authentication
- `internal/server/routes.go` — Added SecurityHeaders globally, MaxBodySize to form route groups
- `README.md` — Documented new middleware in stack, updated Security section, moved items to completed TODO
closes #34, closes #38, closes #39
Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #41
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
- README.md: remove 'in development mode' from admin user creation
description (admin user creation is unconditional)
- internal/delivery/engine.go: remove 'and retry' from HTTPTargetConfig
comment (retry was merged into http target type)
- internal/delivery/engine_test.go: remove '/retry' from
newHTTPTargetConfig comment for consistency
DBURL → DATA_DIR consolidation:
- Remove DBURL env var entirely; main DB now lives at {DATA_DIR}/webhooker.db
- database.go constructs DB path from config.DataDir, ensures dir exists
- Update DATA_DIR prod default from /data/events to /data
- Update all tests to use DataDir instead of DBURL
- Update Dockerfile: /data (not /data/events) for all SQLite databases
- Update README configuration table, Docker examples, architecture docs
Dead code removal:
- Remove unused IndexResponse struct (handlers/index.go)
- Remove unused TemplateData struct (handlers/handlers.go)
Stale comment cleanup:
- Remove TODO in server.go (DB cleanup handled by fx lifecycle)
- Fix nolint:golint → nolint:revive on ServerParams for consistency
- Clean up verbose middleware/routing comments in routes.go
- Fix TODO fan-out description (worker pool, not goroutine-per-target)
.gitignore fixes:
- Add data/ directory to gitignore
- Remove stale config.yaml entry (env-only config since rework)
Remove the entire pkg/config package (Viper-based YAML config file
loader) and simplify internal/config to read all settings directly from
environment variables via os.Getenv(). This eliminates the spurious
"Failed to load config" log messages that appeared when no config.yaml
file was present.
- Delete pkg/config/ (YAML loader, resolver, manager, tests)
- Delete configs/config.yaml.example
- Simplify internal/config helper functions to use os.Getenv() with
defaults instead of falling back to pkgconfig
- Update tests to set env vars directly instead of creating in-memory
YAML config files via afero
- Remove afero, cloud.google.com/*, aws-sdk-go dependencies from go.mod
- Update README: document env-var-only configuration, remove YAML/Viper
references
- Keep godotenv/autoload for .env file convenience in local development
closes #27
Replace unbounded goroutine-per-delivery fan-out with a fixed-size
worker pool (10 workers). Channels serve as bounded queues (10,000
buffer). Workers are the only goroutines doing HTTP delivery.
When retry channel overflows, timers are dropped instead of re-armed.
The delivery stays in 'retrying' status in the DB and a periodic sweep
(every 60s) recovers orphaned retries. The database is the durable
fallback — same path used on startup recovery.
Addresses owner feedback on circuit breaker recovery goroutine flood.
- Fan out all targets for an event in parallel goroutines (fire-and-forget)
- Add per-target circuit breaker for retry targets (closed/open/half-open)
- Circuit breaker trips after 5 consecutive failures, 30s cooldown
- Open circuit skips delivery and reschedules after cooldown
- Half-open allows one probe delivery to test recovery
- HTTP/database/log targets unaffected (no circuit breaker)
- Recovery path also fans out in parallel
- Update README with parallel delivery and circuit breaker docs
The webhook handler now builds DeliveryTask structs carrying all target
config and event data inline (for bodies ≤16KB) and sends them through
the delivery channel. In the happy path, the engine delivers without
reading from any database — it only writes to record delivery results.
For large bodies (≥16KB), Body is nil and the engine fetches it from the
per-webhook database on demand. Retry timers also carry the full
DeliveryTask, so retries avoid unnecessary DB reads.
The database is used for crash recovery only: on startup the engine scans
for interrupted pending/retrying deliveries and re-queues them.
Implements owner feedback from issue #15:
> the message in the <=16KB case should have everything it needs to do
> its delivery. it shouldn't touch the db until it has a success or
> failure to record.
Remove SESSION_KEY env var requirement. On first startup, a
cryptographically secure 32-byte key is generated and stored in a new
settings table. Subsequent startups load the key from the database.
- Add Setting model (key-value table) for application config
- Add Database.GetOrCreateSessionKey() method
- Session manager initializes in OnStart after database is connected
- Remove DevSessionKey constant and SESSION_KEY env var handling
- Remove prod validation requiring SESSION_KEY
- Update README: config table, Docker instructions, security notes
- Update config.yaml.example
- Update all tests to remove SessionKey references
Addresses owner feedback on issue #15.
Replace the polling-based delivery engine with a fully event-driven
architecture using Go channels and goroutines:
- Webhook handler notifies engine via buffered channel after creating
delivery records, with inline event data for payloads < 16KB
- Large payloads (>= 16KB) use pointer semantics (Body *string = nil)
and are fetched from DB on demand, keeping channel memory bounded
- Failed retry-target deliveries schedule Go timers with exponential
backoff; timers fire into a separate retry channel when ready
- On startup, engine scans DB once to recover interrupted deliveries
(pending processed immediately, retrying get timers for remaining
backoff)
- DB stores delivery status for crash recovery only, not for
inter-component communication during normal operation
- delivery.Notifier interface decouples handlers from engine; fx wires
*Engine as Notifier
No more periodic polling. No more wasted cycles when idle.
Split data storage into main application DB (config only) and
per-webhook event databases (one SQLite file per webhook).
Architecture changes:
- New WebhookDBManager component manages per-webhook DB lifecycle
(create, open, cache, delete) with lazy connection pooling via sync.Map
- Main DB (DBURL) stores only config: Users, Webhooks, Entrypoints,
Targets, APIKeys
- Per-webhook DBs (DATA_DIR) store Events, Deliveries, DeliveryResults
in files named events-{webhook_uuid}.db
- New DATA_DIR env var (default: ./data dev, /data/events prod)
Behavioral changes:
- Webhook creation creates per-webhook DB file
- Webhook deletion hard-deletes per-webhook DB file (config soft-deleted)
- Event ingestion writes to per-webhook DB, not main DB
- Delivery engine polls all per-webhook DBs for pending deliveries
- Database target type marks delivery as immediately successful (events
are already in the dedicated per-webhook DB)
- Event log UI reads from per-webhook DBs with targets from main DB
- Existing webhooks without DB files get them created lazily
Removed:
- ArchivedEvent model (was a half-measure, replaced by per-webhook DBs)
- Event/Delivery/DeliveryResult removed from main DB migrations
Added:
- Comprehensive tests for WebhookDBManager (create, delete, lazy
creation, delivery workflow, multiple webhooks, close all)
- Dockerfile creates /data/events directory
README updates:
- Per-webhook event databases documented as implemented (was Phase 2)
- DATA_DIR added to configuration table
- Docker instructions updated with data volume mount
- Data model diagram updated
- TODO updated (database separation moved to completed)
Closes#15
## Summary
Rewrites README.md from a basic scaffold into a comprehensive service description and specification that documents the entire webhooker application.
closes #3
## What Changed
### Naming Scheme
Proposes a clear naming scheme for the data model entities:
- **Processor → Webhook**: The top-level configuration entity that groups entrypoints and targets
- **Webhook → Entrypoint**: The receiver URL (`/hooks/<uuid>`) where external services POST events
- **Target**: Unchanged — delivery destinations for events
This is documented as the target architecture in the README. The actual code rename is tracked in [issue #12](#12).
### Data Model Documentation
Documents all 8 entities with:
- Complete field tables (name, type, description) for every entity
- Relationship descriptions (belongs-to, has-many)
- Enum values for TargetType and DeliveryStatus
- Entity relationship diagram (ASCII)
- Common fields from BaseModel
### Database Architecture
Documents the separate database architecture:
- Main application DB: users, webhook configs, entrypoints, targets, API keys
- Per-webhook event DBs: events, deliveries, delivery results
- Rationale for separation (isolation, lifecycle, clean deletion, per-webhook retention, performance)
### Other Sections
- Complete API endpoint tables (current + planned)
- Package layout with file descriptions
- Request flow diagram
- Middleware stack documentation
- Authentication design (web sessions + planned API keys)
- Security measures
- Rate limiting design
- Dependency injection order
- Docker build pipeline description
- Phased TODO roadmap with links to filed issues
- License set to MIT
### Code Style Divergence Issues Filed
As part of reviewing the code against sneak/prompts standards:
- [#7](#7) — Templates should use go:embed
- [#8](#8) — Logger should use slog.LevelVar
- [#9](#9) — Source management routes lack auth middleware
- [#10](#10) — Config should prefer environment variables
- [#11](#11) — Redundant godotenv/autoload import
- [#12](#12) — Rename Processor → Webhook, Webhook → Entrypoint
## Verification
- `make fmt` — ✅ passes
- `docker build .` — ✅ passes (README-only change, no code modifications)
Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #13
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
## Summary
This PR brings the webhooker repo into full REPO_POLICIES compliance, addressing both [issue #1](#1) and [issue #2](#2).
## Changes
### New files
- **`cmd/webhooker/main.go`** — The missing application entry point. Uses Uber fx to wire together all internal packages (config, database, logger, server, handlers, middleware, healthcheck, globals, session). Minimal glue code.
- **`REPO_POLICIES.md`** — Fetched from authoritative source (`sneak/prompts`)
- **`.editorconfig`** — Fetched from authoritative source
- **`.dockerignore`** — Sensible Go project exclusions
- **`.gitea/workflows/check.yml`** — CI workflow that runs `docker build .` on push to any branch (Gitea Actions format, actions/checkout pinned by sha256)
- **`configs/config.yaml.example`** — Moved from root `config.yaml`
### Modified files
- **`Makefile`** — Complete rewrite with all REPO_POLICIES required targets: `test`, `lint`, `fmt`, `fmt-check`, `check`, `build`, `hooks`, `docker`, `clean`, plus `dev`, `run`, `deps`
- **`Dockerfile`** — Complete rewrite:
- Builder: `golang:1.24` (Debian-based, pinned by `sha256:d2d2bc1c84f7...`). Debian needed because `gorm.io/driver/sqlite` pulls `mattn/go-sqlite3` (CGO) which fails on Alpine musl.
- golangci-lint v1.64.8 installed from GitHub release archive with sha256 verification (v1.x because `.golangci.yml` uses v1 config format)
- Runs `make check` (fmt-check + lint + test + build) as build step
- Final stage: `alpine:3.21` (pinned by `sha256:c3f8e73fdb79...`) with non-root user, healthcheck, port 8080
- **`README.md`** — Rewritten with all required REPO_POLICIES sections: description line with name/purpose/category/license/author, Getting Started, Rationale, Design, TODO (integrated from TODO.md), License, Author
- **`.gitignore`** — Fixed `webhooker` pattern to `/webhooker` (was blocking `cmd/webhooker/`), added `config.yaml` to prevent committing runtime config with secrets
- **`static/static.go`** — Removed `vendor` from embed directive (directory was empty/missing)
- **`internal/database/database_test.go`** — Fixed to use in-memory config via `afero.MemMapFs` instead of depending on `config.yaml` on disk. Test is now properly isolated.
- **`go.mod`/`go.sum`** — `go mod tidy`
### Removed files
- **`TODO.md`** — Content integrated into README.md TODO section
- **`config.yaml`** — Moved to `configs/config.yaml.example`
## Verification
- `docker build .` passes (lint ✅, test ✅, build ✅)
- All existing tests pass with no modifications to assertions or test logic
- `.golangci.yml` untouched
closes #1
closes #2
Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #6
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
