ee7c62607104a7dacba5082cc46098f57049271c
22
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
  clawbotandsneak
|
ee7c626071 |
Implement the database archiving target (closes #43) (#84)
check / check (push) Successful in 4s
Implements the `databaseTarget` as a real archiving target, replacing the always-successful stub. Delivering to a `database` target now writes the full event into a per-webhook archive SQLite file for long-term storage.
## Archive-writer semantics
- **Separate file:** each webhook's full events are written as rows into `archive-{webhookID}.db` under the data dir, distinct from the per-webhook event DB (`events-{webhookID}.db`). The file and its schema are created on first write if missing. Each row carries the full event: body, headers, method, content type, webhook id, entrypoint id, event id, and an archived-at timestamp.
- **Close/reopen with debounce:** after each write the archive handle is closed and reopened, unless the last (re)open was less than one second ago. This lets an operator move the archive file away for offline archiving while bounding file churn under load. A per-webhook `archiveWriter` owns this debounce state and serialises writes.
- **Auto-recreate:** the file is opened create-if-missing (`mode=rwc`) and its schema re-migrated on every open, so if the archive was moved or removed since the last open, the next write recreates it. The writer also detects a missing file before writing and reopens first, so a moved-away file is recreated rather than lost.
- **Optional expiry, validated at creation:** an optional `expiry` in the target's config JSON (e.g. `{"expiry":"720h"}`) is validated when the target is created (`ValidateArchiveExpiry`; bad values are rejected with a 400 at the add-target form, the Slack URL precedent). The default (missing, empty, or `"never"`) keeps rows forever with no pruning. When a positive duration is set, rows older than it (measured from each row's archived-at time) are pruned on every (re)open; because the file is reopened after writes, prune-on-open keeps the archive swept without a separate background sweeper. A set-but-invalid expiry in a stored config (unparseable, zero, or negative) is an error at delivery time too — never a silent default.
- **No-retry, fail-loud:** the target performs a single attempt with no retries. On success it records one successful attempt and marks the delivery delivered. If the archive write fails, the attempt is recorded as failed with the error and the delivery is marked failed — archiving errors never report success.
## Scope
- `internal/delivery/target_database.go` — the `databaseTarget` (no-retry) archives via a per-webhook writer registry; an archive error records a failed attempt and marks the delivery failed.
- `internal/delivery/target_database_archive.go` (new) — the `archiveWriter`, the archived-row model, config/expiry parsing (fail-loud on set-but-invalid values), `ValidateArchiveExpiry`, and prune-on-open.
- `internal/handlers/source_management.go` — database targets get a creation-validated `expiry` config (`buildDatabaseTargetConfig`); the expiry form value is read where the request body is bounded and bad values are rejected with a 400 at target creation.
- `templates/source_detail.html` — the add-target form shows an expiry field for database targets.
- `README.md` — the database-target documentation describes the archiving semantics.
- `internal/delivery/export_test.go`, `internal/delivery/target_database_test.go`, `internal/handlers` tests — tests and their exported shims.
No changes to the `Target` interface or other targets.
## Tests
- a row is archived (both at the writer level and end-to-end through `Deliver`)
- a forced archive failure (bad stored expiry config) yields a `Failed` delivery with a non-success `DeliveryResult` carrying the error and no archive file created
- the file is recreated after removal, with only the post-removal row
- the one-second reopen debounce (rapid writes reopen once; a write after the window reopens again)
- expiry pruning removes rows older than the configured expiry
- expiry config parsing (empty / `never` / duration accepted; unparseable, zero, and negative values error)
- expiry validation at target creation (`TestValidateArchiveExpiry`; valid values build the config, bad values get a 400)
## Validation
`docker build .` exits 0 (fmt-check, lint, test, build all pass).
Closes #43
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #84
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
|
||
|
clawbotandsneak
|
81413c56e9 |
Refactor delivery targets to a Target interface (closes #77) (#81)
check / check (push) Successful in 2m42s
Refactors the delivery engine so each target TYPE is an implementation of a `Target` interface, dispatched from a registry, with each target owning its full delivery including durable retries. Implements the authoritative design from issue #77 (the corrected "hand the DB + Scheduler to the target" design). ## The new interface ```go type Scheduler interface { ScheduleRetry(task Task, delay time.Duration) } type Target interface { Deliver(ctx context.Context, webhookDB *gorm.DB, d *database.Delivery, task *Task, sched Scheduler) } ``` `Deliver` receives everything a target needs to be autonomous and durable: the request context, the per-webhook `*gorm.DB`, the `*database.Delivery`, the attempt `*Task`, and a `Scheduler` (the engine) for durable re-enqueue. The target makes one attempt, writes the `DeliveryResult`, updates `DeliveryStatus`, and — for retry targets — decides whether to retry, computes its own backoff, gates with its own circuit breaker, and reschedules via the injected `Scheduler`. `processDelivery` collapses to a registry lookup (`map[database.TargetType]Target`) and a `Deliver` call; an unknown target type still fails the delivery as before. ## Per-target ownership - `httpTarget` and `slackTarget` share a retry core (`httpCore`) that owns retry, exponential backoff, and the per-target circuit breaker. The core is fire-and-forget when `MaxRetries == 0` and adds breaker-gated backed-off retries when `MaxRetries > 0`. The per-attempt request differs (HTTP forwards the body + filtered headers; Slack posts a formatted message) and is supplied as a closure, so each keeps its exact recording semantics (e.g. HTTP records no error string for a non-2xx, Slack records `HTTP <code>`). - `databaseTarget` and `logTarget` are fire-and-forget: they record a single successful attempt. Moved wholesale into the http/slack targets: `deliverHTTP*`, `handleHTTPRetry`, `circuitBreakerBlock`, `calcBackoff` / `calcRemainingBackoff` / `backoffElapsed`, the circuit-breaker `sync.Map` + `getCircuitBreaker`, `clientForConfig`, `doHTTPRequest`, `applyRequestHeaders`, and the config parsers. The engine keeps `recordResult`, `updateDeliveryStatus`, and `ScheduleRetry`. ## Slack MaxRetries gating Slack is now on the same shared core as HTTP, with retry + breaker gated on `MaxRetries`. A `MaxRetries` of 0 stays single-attempt fire-and-forget, so **every existing Slack target is unchanged**; a Slack target configured with retries gets backoff + circuit breaker. ## Log-target full content `logTarget` now logs the ENTIRE inbound webhook — full request body and full request headers, plus method, content type, and the webhook id and entrypoint id — rather than a summary line. This supersedes the smaller log-summary work (#70). ## `Task.EntrypointID` To carry the entrypoint id to the log target, `Task` gains an `EntrypointID` field, populated in the webhook handler's `buildDeliveryTasks`, the engine's recovery-task builder, and `buildEventFromTask`. ## Durability / recovery The crash-durable async retry model is preserved unchanged: one attempt per worker turn; on failure the status is set `retrying`, backoff is computed, and the task is re-enqueued via `ScheduleRetry` (a `time.AfterFunc` onto the retry channel). On restart, `recoverRetryingDeliveries` and the 60s sweep hand each orphaned `retrying` delivery back to its target to recompute the remaining backoff and reschedule (targets that own retries implement an internal `rescheduler`; fire-and-forget targets, which never produce `retrying` deliveries, are skipped). ## How behaviour is preserved No external behaviour changes except the two called out above (log target full content; Slack gaining `MaxRetries`-gated retries). All existing delivery tests pass with only their `export_test.go` wrappers re-pointed at the new structure — `ExportDeliverHTTP/Slack/Database/Log` now call the targets, `ExportGetCircuitBreaker` / `ExportClient` / `ExportClientForConfig` / `ExportDoHTTPRequest` resolve against the HTTP target's shared client and breaker map, and `ExportParseHTTPConfig` / `ExportParseSlackConfig` call the relocated free functions. Added: a `logTarget` test asserting the log line contains the full body, headers, and ids, and a Slack `MaxRetries`-gated retry test. `docker build .` is green (fmt-check, lint, test, static build all pass). Closes #77 Co-authored-by: sneak <sneak@sneak.berlin> Reviewed-on: #81 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandsneak
|
f6b929f2d7 |
Add per-webhook event retention reaper (closes #63) (#78)
check / check (push) Successful in 2m42s
Enforces each webhook's `RetentionDays` so per-webhook SQLite files no longer grow without bound. ## Reaper New `RetentionReaper` in `internal/database/retention.go`. A background ticker runs each sweep: it lists all webhooks from the main DB and, for each webhook with a positive `RetentionDays`, opens its per-webhook DB via `WebhookDBManager.GetDB` and deletes every `Event` (and its dependent `Delivery` and `DeliveryResult` rows) whose `CreatedAt` is older than `RetentionDays` days. - Deletions run in foreign-key-safe order: delivery results, then deliveries, then events. - Deletes are unscoped (hard deletes) so rows are physically removed and disk is reclaimed, rather than GORM soft-deleting them. - `RetentionDays <= 0` means retain forever; those webhooks are skipped. - Webhooks whose per-webhook DB does not yet exist are skipped. ## Config `internal/config/config.go` gains `RetentionSweepInterval` (env `RETENTION_SWEEP_INTERVAL`, parsed as a Go duration, default `1h`) via a new `envDuration` helper, following the existing env-helper conventions. ## Wiring `cmd/webhooker/main.go` registers `database.NewRetentionReaper` as an fx provider and forces its construction in `fx.Invoke`. The reaper starts its sweep loop on an fx `OnStart` hook and stops cleanly on `OnStop` via context cancellation, matching the existing lifecycle components. ## Test `internal/database/retention_test.go` seeds an old event chain (event + delivery + result, 40 days old) and a recent one (1 day old) in a real per-webhook DB and asserts a single sweep removes only the expired chain while keeping the recent one. A second test forces a non-positive `RetentionDays` and asserts an ancient event is retained. Note: the `Webhook.RetentionDays` column carries `gorm:"default:30"`, so a `0` passed to a GORM `Create` is replaced by the default; the test forces the value with an explicit column update to exercise the retain-forever path. No model changes were made. Validated with `docker build .` (fmt-check, lint, test, build) exit 0. Closes #63 Co-authored-by: sneak <sneak@sneak.berlin> Reviewed-on: #78 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandsneak
|
8ea7f76540 |
Add NoCache middleware for authenticated pages (closes #61) (#75)
check / check (push) Successful in 6s
Adds a `NoCache()` middleware that sets `Cache-Control: no-store` and `Pragma: no-cache`, and wires it onto the dynamic app route groups (`/pages`, `/user/{username}`, `/sources`, `/source/{sourceID}`) adjacent to their existing `CSRF()` call. The static `/s` mount, `/metrics`, `/webhook/{uuid}`, and `/.well-known/healthcheck` are intentionally left untouched (static assets are safe to cache; the others are not authenticated pages).
A middleware unit test asserts both headers are set.
Closes #61
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #75
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
|
||
|
clawbotandsneak
|
752d6beead |
Validate Slack target URLs at creation time (closes #68) (#73)
check / check (push) Successful in 4s
Slack delivery targets were only checked by the request-time dialer guard, not at creation, giving them a weaker SSRF gate than HTTP targets. This validates the Slack incoming-webhook URL with `delivery.ValidateTargetURL` in the Slack target creation path (`buildSlackTargetConfig`), before persisting, mirroring the existing HTTP-target path. On failure the create is rejected with the same clear, non-leaking user-facing error the HTTP path uses. Adds handlers-package tests covering both an accepted public URL and a rejected private/reserved URL. Confined to `internal/handlers/`; `internal/delivery/` is unchanged. Closes #68 Co-authored-by: sneak <sneak@sneak.berlin> Co-authored-by: Jeffrey Paul <sneak@noreply.example.org> Reviewed-on: #73 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandsneak
|
b1f43c9520 |
Keep the SSRF-safe transport in clientForConfig (closes #69) (#74)
check / check (push) Superseded by a newer commit; never tested
`clientForConfig()` in `internal/delivery/engine.go` built a fresh `http.Client` without a Transport when a per-target timeout was configured, dropping the request-time private-IP guard for that path. It now reuses the shared client's SSRF-safe transport (`e.client.Transport`, the same `NewSSRFSafeTransport` instance), overriding only the `Timeout`. Behaviour is unchanged when no per-target timeout is set (the shared client is returned as before), so no engine code path makes an outbound target request with a client lacking the SSRF-safe transport. Adds a delivery-package test proving a client from `clientForConfig()` with a per-target timeout still refuses private/reserved/link-local destinations, that the timeout is applied, that the SSRF-safe transport is reused (not duplicated), and that the no-timeout path returns the shared client unchanged. Confined to `internal/delivery/` only; handlers and server code untouched. Closes #69 Co-authored-by: sneak <sneak@sneak.berlin> Co-authored-by: Jeffrey Paul <sneak@noreply.example.org> Reviewed-on: #74 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandsneak
|
07fc63d9fa |
Wrap /user/{username} in RequireAuth middleware (closes #60) (#71)
check / check (push) Successful in 4s
Enforces authentication for the `/user/{username}` route group at the middleware layer, matching every other authenticated route group.
## Changes
- **`internal/server/routes.go`** (`setupUserRoutes`): added `r.Use(s.mw.RequireAuth())` immediately after the existing `r.Use(s.mw.CSRF())` on the `/user/{username}` group, so auth is enforced by design (CSRF first, then RequireAuth) — consistent with `/sources` and `/source/{sourceID}`.
- **`internal/handlers/profile.go`** (`HandleProfile`): removed the now-dead unauthenticated-redirect branch (RequireAuth guarantees an authenticated session before the handler runs). The handler still reads the username and user id from the session for the own-profile-only check; a request for another user's profile still returns 403. The session-retrieval error is now handled as a 500.
## Tests (`internal/handlers/profile_test.go`)
- own profile returns 200
- another user's profile returns 403
- an unauthenticated request to `/user/{username}` is redirected to `/pages/login` at the middleware layer and never reaches the endpoint handler (routing-level test replicating the CSRF + RequireAuth chain)
## Validation
`docker build .` (fmt-check, lint, test, build) passes.
Closes #60
Co-authored-by: sneak <sneak@sneak.berlin>
Co-authored-by: Jeffrey Paul <sneak@noreply.example.org>
Reviewed-on: #71
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
|
||
|
clawbotandsneak
|
0c9c885d51 |
Raise HTTP WriteTimeout above the request middleware timeout (closes #62) (#72)
check / check (push) Successful in 4s
Raise `httpWriteTimeout` in `internal/server/http.go` from 10s to `65 * time.Second` so it comfortably exceeds the router's 60s `requestTimeout`. This makes the `middleware.Timeout(60s)` the effective request limit — a slow response now returns a clean 503 from the middleware instead of being cut at the socket write deadline by the transport. `httpReadTimeout` stays at 10s. A comment on `httpWriteTimeout` documents that it must remain above the 60s request timeout. Change is confined to `internal/server/http.go`; `routes.go` is untouched. Closes #62 Co-authored-by: sneak <sneak@sneak.berlin> Reviewed-on: #72 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
 clawbotandclawbot
|
afe88c601a |
refactor: use pinned golangci-lint Docker image for linting (#55)
check / check (push) Successful in 5s
Closes [issue #50](#50) ## Summary Refactors the Dockerfile to use a separate lint stage with a pinned golangci-lint Docker image, following the pattern used by [sneak/pixa](https://git.eeqj.de/sneak/pixa). This replaces the previous approach of installing golangci-lint via curl in the builder stage. ## Changes ### Dockerfile - **New `lint` stage** using `golangci/golangci-lint:v2.11.3` (Debian-based, pinned by sha256 digest) as a separate build stage - **Builder stage** depends on lint via `COPY --from=lint /src/go.sum /dev/null` — build won't proceed unless linting passes - **Go bumped** from 1.24 to 1.26.1 (`golang:1.26.1-bookworm`, pinned by sha256) - **golangci-lint bumped** from v1.64.8 to v2.11.3 - All three Docker images (golangci-lint, golang, alpine) pinned by sha256 digest - Debian-based golangci-lint image used (not Alpine) because mattn/go-sqlite3 CGO does not compile on musl (off64_t) ### Linter Config (.golangci.yml) - Migrated from v1 to v2 format (`version: "2"` added) - Removed linters no longer available in v2: `gofmt` (handled by `make fmt-check`), `gosimple` (merged into `staticcheck`), `typecheck` (always-on in v2) - Same set of linters enabled — no rules weakened ### Code Fixes (all lint issues from v2 upgrade) - Added package comments to all packages - Added doc comments to all exported types, functions, and methods - Fixed unchecked errors flagged by `errcheck` (sqlDB.Close, os.Setenv in tests, resp.Body.Close, fmt.Fprint) - Fixed unused parameters flagged by `revive` (renamed to `_`) - Fixed `gosec` G120 warnings: added `http.MaxBytesReader` before `r.ParseForm()` calls - Fixed `staticcheck` QF1012: replaced `WriteString(fmt.Sprintf(...))` with `fmt.Fprintf` - Fixed `staticcheck` QF1003: converted if/else chain to tagged switch - Renamed `DeliveryTask` → `Task` to avoid package stutter (`delivery.Task` instead of `delivery.DeliveryTask`) - Renamed shadowed builtin `max` parameter to `upperBound` in `cryptoRandInt` - Used `t.Setenv` instead of `os.Setenv` in tests (auto-restores) ### README.md - Updated version requirements: Go 1.26+, golangci-lint v2.11+ - Updated Dockerfile description in project structure ## Verification `docker build .` passes cleanly — formatting check, linting, all tests, and build all succeed. Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Reviewed-on: #55 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotanduser
|
d771fe14df |
fix: detect TLS per-request in CSRF middleware to fix login (#54)
check / check (push) Successful in 1m55s
## Problem After the security hardening in PR #42, login fails with `Forbidden - invalid CSRF token` in production deployments. The CSRF middleware tied its `PlaintextHTTPRequest` wrapping and cookie `Secure` flag to the `IsDev()` environment check. This meant production mode always assumed HTTPS via gorilla/csrf's strict mode, which broke login in common deployment scenarios: 1. **Production behind a TLS-terminating reverse proxy**: gorilla/csrf assumed HTTPS but `r.TLS` was nil (the Go server receives HTTP from the proxy). Origin/Referer scheme mismatches caused `referer not supplied` or `origin invalid` errors. 2. **Production over direct HTTP** (testing/staging with prod config): the `Secure` cookie flag prevented the browser from sending the CSRF cookie back over HTTP, causing `CSRF token invalid` errors. ## Root Cause gorilla/csrf v1.7.3 defaults to HTTPS-strict mode unless `PlaintextHTTPRequest()` is called. In strict mode it: - Forces `requestURL.Scheme = "https"` for Origin/Referer comparisons - Requires a `Referer` header on POST and rejects `http://` Referer schemes - The `csrf.Secure(true)` option makes the browser refuse to send the CSRF cookie over HTTP The old code only called `PlaintextHTTPRequest()` in dev mode, leaving prod mode permanently stuck in HTTPS-strict mode regardless of the actual transport. ## Fix Detect the actual transport protocol **per-request** using: - `r.TLS != nil` — direct TLS connection to the Go server - `X-Forwarded-Proto: https` header — TLS-terminating reverse proxy Two gorilla/csrf middleware instances are maintained (one with `Secure: true`, one with `Secure: false`) since `csrf.Secure()` is a creation-time option. Both use the same signing key, so cookies are interchangeable. | Scenario | Cookie Secure | Origin/Referer Mode | |---|---|---| | Direct TLS (`r.TLS != nil`) | ✅ Secure | Strict (HTTPS scheme) | | Behind TLS proxy (`X-Forwarded-Proto: https`) | ✅ Secure | Strict (HTTPS scheme) | | Plaintext HTTP | ❌ Non-Secure | Relaxed (PlaintextHTTPRequest) | CSRF token validation (cookie + form double-submit) is always enforced regardless of mode. ## Testing - Added `TestCSRF_ProdMode_PlaintextHTTP_POSTWithValidToken` — prod mode over plaintext HTTP - Added `TestCSRF_ProdMode_BehindProxy_POSTWithValidToken` — prod mode behind TLS proxy - Added `TestCSRF_ProdMode_DirectTLS_POSTWithValidToken` — prod mode with direct TLS - Added `TestCSRF_ProdMode_PlaintextHTTP_POSTWithoutToken` — token still required - Added `TestIsClientTLS_*` — TLS detection unit tests - All existing CSRF tests pass unchanged - `docker build .` passes (includes `make check`) - Manual verification: built and ran the container in both `dev` and `prod` modes, confirmed login succeeds in both Closes #53 Co-authored-by: user <user@Mac.lan guest wan> Reviewed-on: #54 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
|
33e2140a5a |
feat: redirect root path based on auth state (#52)
check / check (push) Successful in 1m54s
Closes #51 The root path `/` now checks for an authenticated session and redirects accordingly: - **Authenticated users** → `303 See Other` redirect to `/sources` (the webhook dashboard) - **Unauthenticated users** → `303 See Other` redirect to `/pages/login` ### Changes - **`internal/handlers/index.go`** — Replaced the template-rendering `HandleIndex()` with a session-checking redirect handler. Removed `formatUptime` helper (dead code after this change). - **`internal/handlers/handlers.go`** — Removed `index.html` from the template map (no longer rendered). - **`internal/handlers/handlers_test.go`** — Replaced the old "handler is not nil" test with two proper redirect tests: - `unauthenticated redirects to login` — verifies 303 to `/pages/login` - `authenticated redirects to sources` — sets up an authenticated session cookie, verifies 303 to `/sources` - Removed `TestFormatUptime` (tested dead code). - **`README.md`** — Updated the API endpoints table to describe the new redirect behavior. ### How it works The handler calls `session.Get(r)` and `session.IsAuthenticated(sess)` — the same pattern used by the `RequireAuth` middleware and `HandleLoginPage`. No new dependencies or session logic introduced. The login flow is unaffected: `HandleLoginSubmit` redirects to `/` after successful login, which now forwards to `/sources` (one extra redirect hop, but correct and clean). Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Co-authored-by: clawbot <clawbot@eeqj.de> Reviewed-on: #52 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
|
f003ec7141 |
fix: use absolute path for dev DATA_DIR default, clarify env docs (#46)
check / check (push) Successful in 1m9s
Closes #45. ## Problem 1. The README didn't clearly explain what `WEBHOOKER_ENVIRONMENT=dev` vs `prod` actually changes. 2. The dev-mode default for `DATA_DIR` was `./data` — a relative path whose meaning depends on the working directory. There's no reason to use a relative path even in development. ## Changes ### Code (`internal/config/config.go`) - Replace the dev default `DATA_DIR` from `./data` to `$XDG_DATA_HOME/webhooker` (falling back to `$HOME/.local/share/webhooker`). This follows the XDG Base Directory Specification and ensures the data directory is always an absolute path regardless of the working directory. - Add `devDataDir()` helper that resolves the XDG path, with a `/tmp/webhooker` last-resort fallback if `$HOME` can't be determined. ### Tests (`internal/config/config_test.go`) - `TestDevDataDir`: verifies XDG_DATA_HOME is respected, HOME fallback works, and the result is always absolute. - `TestDevDefaultDataDirIsAbsolute`: integration test that creates a full Config via fx and asserts the dev default DataDir is absolute. ### README - Add a table documenting exactly what `dev` vs `prod` changes: DATA_DIR default, CORS policy, and session cookie Secure flag. - Clarify that log format and security headers are independent of the environment setting. - Update the DATA_DIR default in the configuration variable table. Co-authored-by: clawbot <clawbot@eeqj.de> Co-authored-by: user <user@Mac.lan guest wan> Reviewed-on: #46 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandclawbot
|
17e740a45f |
fix: use absolute paths and static linking in Dockerfile (#49)
check / check (push) Superseded by a newer commit; never tested
Closes #48 ## Problem The Docker container failed to start with: ``` exec ./webhooker: no such file or directory ``` Two root causes: 1. **Relative paths**: `COPY` destination and `CMD` used relative paths (`./webhooker`), depending on `WORKDIR` context. 2. **Dynamic linking** (the actual root cause): The binary was built with CGO enabled on Debian (glibc) via `make build`, but deployed to an Alpine runtime (musl). The kernel couldn't find the glibc dynamic linker (`/lib64/ld-linux-x86-64.so.2`), producing the misleading "no such file or directory" error — even though the file existed on disk. ## Fix - **Absolute paths throughout**: `COPY --from=builder /build/bin/webhooker /app/webhooker` and `CMD ["/app/webhooker"]` — no reliance on WORKDIR. - **Static rebuild for Alpine**: Added a `RUN CGO_ENABLED=1 go build -ldflags '-extldflags "-static"' -o bin/webhooker ./cmd/webhooker` step after `make check`. This rebuilds the binary with static linking so it runs on Alpine without glibc. The `make check` step still runs normally (formatting, linting, tests, dynamic build) — the static rebuild is only for the deployment binary. ## Verification - `docker build .` passes (all checks green) - Container starts successfully and initializes the Fx dependency graph - The README already stated "The runtime binary is statically linked and runs on Alpine" — this fix makes that claim actually true. Co-authored-by: clawbot <clawbot@users.noreply.git.eeqj.de> Reviewed-on: #49 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
|
60786c5019 |
feat: add CSRF protection, SSRF prevention, and login rate limiting (#42)
check / check (push) Successful in 4s
## Security Hardening This PR implements three security hardening issues: ### CSRF Protection (closes #35) - Session-based CSRF tokens with cryptographically random 256-bit generation - Constant-time token comparison to prevent timing attacks - CSRF middleware applied to `/pages`, `/sources`, `/source`, and `/user` routes - Hidden `csrf_token` field added to all 12+ POST forms in templates - Excluded from `/webhook` (inbound webhook POSTs) and `/api` (stateless API) ### SSRF Prevention (closes #36) - `ValidateTargetURL()` blocks private/reserved IP ranges at target creation time - Blocked ranges: `127.0.0.0/8`, `10.0.0.0/8`, `172.16.0.0/12`, `192.168.0.0/16`, `169.254.0.0/16`, `::1`, `fc00::/7`, `fe80::/10`, plus multicast, reserved, test-net, and CGN ranges - SSRF-safe HTTP transport with custom `DialContext` in the delivery engine for defense-in-depth (prevents DNS rebinding attacks) - Only `http` and `https` schemes allowed ### Login Rate Limiting (closes #37) - Per-IP rate limiter using `golang.org/x/time/rate` - 5 attempts per minute per IP on `POST /pages/login` - GET requests (form rendering) pass through unaffected - Automatic cleanup of stale per-IP limiter entries every 5 minutes - `X-Forwarded-For` and `X-Real-IP` header support for reverse proxies ### Files Changed **New files:** - `internal/middleware/csrf.go` + tests — CSRF middleware - `internal/middleware/ratelimit.go` + tests — Login rate limiter - `internal/delivery/ssrf.go` + tests — SSRF validation + safe transport **Modified files:** - `internal/server/routes.go` — Wire CSRF and rate limit middleware - `internal/handlers/handlers.go` — Inject CSRF token into template data - `internal/handlers/source_management.go` — SSRF validation on target creation - `internal/delivery/engine.go` — SSRF-safe HTTP transport for production - All form templates — Added hidden `csrf_token` fields - `README.md` — Updated Security section and TODO checklist `docker build .` passes (lint + tests + build). Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Co-authored-by: clawbot <clawbot@eeqj.de> Co-authored-by: Jeffrey Paul <sneak@noreply.example.org> Reviewed-on: #42 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotanduser
|
8d702a16c6 |
feat: add Slack target type for incoming webhook notifications (#47)
check / check (push) Successful in 4s
## Summary Adds a new `slack` target type that sends webhook events as formatted messages to any Slack-compatible incoming webhook URL (Slack, Mattermost, and other compatible services). closes #44 ## What it does When a webhook event is received, the Slack target: 1. Formats a human-readable message with event metadata (HTTP method, content type, timestamp, body size) 2. Pretty-prints the payload in a code block — JSON payloads get indented formatting, non-JSON payloads are shown as raw text 3. Truncates large payloads at 3500 characters to keep Slack messages reasonable 4. POSTs the message as a `{"text": "..."}` JSON payload to the configured webhook URL ## Changes - **`internal/database/model_target.go`** — Add `TargetTypeSlack` constant - **`internal/delivery/engine.go`** — Add `SlackTargetConfig` struct, `deliverSlack` method, `FormatSlackMessage` function (exported), `parseSlackConfig` helper. Route slack targets in `processDelivery` switch. - **`internal/handlers/source_management.go`** — Handle `slack` type in `HandleTargetCreate`, building `webhook_url` config from the URL form field - **`templates/source_detail.html`** — Add "Slack" option to target type dropdown with URL field and helper text - **`README.md`** — Document the new target type, update roadmap ## Tests - `TestParseSlackConfig_Valid` / `_Empty` / `_MissingWebhookURL` — Config parsing - `TestFormatSlackMessage_JSONBody` / `_NonJSONBody` / `_EmptyBody` / `_LargeJSONTruncated` — Message formatting - `TestDeliverSlack_Success` / `_Failure` / `_InvalidConfig` — End-to-end delivery - `TestProcessDelivery_RoutesToSlack` — Routing from processDelivery switch All existing tests continue to pass. `docker build .` (which runs `make check`) passes clean. Co-authored-by: user <user@Mac.lan guest wan> Reviewed-on: #47 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandclawbot
|
1fbcf96581 |
security: add headers middleware, session regeneration, and body size limits (#41)
check / check (push) Successful in 1m47s
## Summary This PR implements three security hardening measures: ### Security Headers Middleware (closes #34) Adds a `SecurityHeaders()` middleware applied globally to all routes. Every response now includes: - `Strict-Transport-Security: max-age=63072000; includeSubDomains; preload` - `X-Content-Type-Options: nosniff` - `X-Frame-Options: DENY` - `Content-Security-Policy: default-src 'self'; script-src 'self' 'unsafe-inline'; style-src 'self' 'unsafe-inline'` - `Referrer-Policy: strict-origin-when-cross-origin` - `Permissions-Policy: camera=(), microphone=(), geolocation=()` ### Session Fixation Prevention (closes #38) Adds a `Regenerate()` method to the session manager that destroys the old session and creates a new one with a fresh ID, copying all session values. Called after successful login to prevent session fixation attacks. ### Request Body Size Limits (closes #39) Adds a `MaxBodySize()` middleware using `http.MaxBytesReader` to limit POST/PUT/PATCH request bodies to 1 MB. Applied to all form endpoints (`/pages`, `/sources`, `/source/*`). ## Files Changed - `internal/middleware/middleware.go` — Added `SecurityHeaders()` and `MaxBodySize()` middleware - `internal/session/session.go` — Added `Regenerate()` method for session fixation prevention - `internal/handlers/auth.go` — Updated login handler to regenerate session after authentication - `internal/server/routes.go` — Added SecurityHeaders globally, MaxBodySize to form route groups - `README.md` — Documented new middleware in stack, updated Security section, moved items to completed TODO closes #34, closes #38, closes #39 Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Reviewed-on: #41 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandclawbot
|
a51e863017 |
Remove globals.Buildarch from codebase (#31)
check / check (push) Successful in 1m0s
Remove the `Buildarch` field from the globals package and all references throughout the codebase. **Changes:** - Removed `Buildarch` package-level var and struct field from `internal/globals/globals.go` - Removed `Buildarch` from the `New()` constructor - Removed `globals.Buildarch = runtime.GOARCH` and unused `runtime` import from `cmd/webhooker/main.go` - Removed `buildarch` from logger startup output in `internal/logger/logger.go` - Removed all `Buildarch` test setup and assertions from globals, logger, database, and webhook_db_manager tests All tests pass, `make check` passes, `docker build .` succeeds. closes [issue #30](#30) <!-- session: agent:sdlc-manager:subagent:5cae6803-6bdf-467d-9a56-43f135521e5f --> Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Reviewed-on: #31 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandclawbot
|
289f479772 |
test: add tests for delivery, middleware, and session packages (#32)
check / check (push) Has been cancelled
## Summary Add comprehensive test coverage for three previously-untested packages, addressing [issue #28](#28). ## Coverage Improvements | Package | Before | After | |---------|--------|-------| | `internal/delivery` | 37.1% | 74.5% | | `internal/middleware` | 0.0% | 70.2% | | `internal/session` | 0.0% | 51.5% | ## What's Tested ### delivery (37% → 75%) - `processNewTask` with inline and large (DB-fetched) bodies - `processRetryTask` success, skip non-retrying, large body fetch - Worker lifecycle start/stop, retry channel processing - `processDelivery` unknown target type handling - `recoverPendingDeliveries`, `recoverWebhookDeliveries`, `recoverInFlight` - HTTP delivery with custom headers, timeout, invalid config - `Notify` batching ### middleware (0% → 70%) - Logging middleware status code capture and pass-through - `LoggingResponseWriter` delegation - CORS dev mode (allow-all) and prod mode (no-op) - `RequireAuth` redirect for unauthenticated, pass-through for authenticated - `MetricsAuth` basic auth validation - `ipFromHostPort` helper ### session (0% → 52%) - `Get`/`Save` round-trip with real cookie store - `SetUser`, `GetUserID`, `GetUsername`, `IsAuthenticated` - `ClearUser` removes all keys - `Destroy` invalidates session (MaxAge -1) - Session persistence across requests - Edge cases: overwrite user, wrong type, constants ## Test Helpers Added - `database.NewTestDatabase` / `NewTestWebhookDBManager` — cross-package test helpers for delivery integration tests - `session.NewForTest` — creates session manager without fx lifecycle for middleware tests ## Notes - No production code modified - All tests use `httptest`, SQLite in-memory, and real cookie stores — no external network calls - Full test suite completes in ~3.5s within the 30s timeout - `docker build .` passes (lint + test + build) closes #28 Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Reviewed-on: #32 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbot
|
9b9ee1718a |
refactor: auto-generate session key and store in database
check / check (push) Successful in 57s
Remove SESSION_KEY env var requirement. On first startup, a cryptographically secure 32-byte key is generated and stored in a new settings table. Subsequent startups load the key from the database. - Add Setting model (key-value table) for application config - Add Database.GetOrCreateSessionKey() method - Session manager initializes in OnStart after database is connected - Remove DevSessionKey constant and SESSION_KEY env var handling - Remove prod validation requiring SESSION_KEY - Update README: config table, Docker instructions, security notes - Update config.yaml.example - Update all tests to remove SessionKey references Addresses owner feedback on issue #15. |
||
|
|
b5cf4c3d2f |
docs: comprehensive README rewrite with complete service specification (#13)
check / check (push) Successful in 4s
## Summary Rewrites README.md from a basic scaffold into a comprehensive service description and specification that documents the entire webhooker application. closes #3 ## What Changed ### Naming Scheme Proposes a clear naming scheme for the data model entities: - **Processor → Webhook**: The top-level configuration entity that groups entrypoints and targets - **Webhook → Entrypoint**: The receiver URL (`/hooks/<uuid>`) where external services POST events - **Target**: Unchanged — delivery destinations for events This is documented as the target architecture in the README. The actual code rename is tracked in [issue #12](#12). ### Data Model Documentation Documents all 8 entities with: - Complete field tables (name, type, description) for every entity - Relationship descriptions (belongs-to, has-many) - Enum values for TargetType and DeliveryStatus - Entity relationship diagram (ASCII) - Common fields from BaseModel ### Database Architecture Documents the separate database architecture: - Main application DB: users, webhook configs, entrypoints, targets, API keys - Per-webhook event DBs: events, deliveries, delivery results - Rationale for separation (isolation, lifecycle, clean deletion, per-webhook retention, performance) ### Other Sections - Complete API endpoint tables (current + planned) - Package layout with file descriptions - Request flow diagram - Middleware stack documentation - Authentication design (web sessions + planned API keys) - Security measures - Rate limiting design - Dependency injection order - Docker build pipeline description - Phased TODO roadmap with links to filed issues - License set to MIT ### Code Style Divergence Issues Filed As part of reviewing the code against sneak/prompts standards: - [#7](#7) — Templates should use go:embed - [#8](#8) — Logger should use slog.LevelVar - [#9](#9) — Source management routes lack auth middleware - [#10](#10) — Config should prefer environment variables - [#11](#11) — Redundant godotenv/autoload import - [#12](#12) — Rename Processor → Webhook, Webhook → Entrypoint ## Verification - `make fmt` — ✅ passes - `docker build .` — ✅ passes (README-only change, no code modifications) Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Co-authored-by: Jeffrey Paul <sneak@noreply.example.org> Reviewed-on: #13 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotanduser
|
011ec270c2 |
Replace Bootstrap with Tailwind CSS + Alpine.js (#14)
check / check (push) Has been cancelled
## Summary Replaces Bootstrap CSS/JS framework with Tailwind CSS v4 + Alpine.js, matching the µPaaS UI pattern. ## Changes - **Removed Bootstrap** — all Bootstrap CSS/JS references removed from templates - **Added Tailwind CSS v4** — `static/css/input.css` with Material Design inspired theme, compiled to `static/css/tailwind.css` - **Added Alpine.js 3.14.9** — vendored as `static/js/alpine.min.js` for reactive UI components - **Rewrote all templates** to use Tailwind utility classes: - `base.html` — new layout structure with footer, matches µPaaS pattern - `htmlheader.html` — Tailwind CSS link, `[x-cloak]` style - `navbar.html` — Alpine.js mobile menu toggle, responsive design - `index.html` — card-based dashboard with Tailwind classes - `login.html` — centered login form with Material Design styling - `profile.html` — clean profile layout - **Added `make css` target** — compiles Tailwind CSS using standalone CLI - **Component classes** in `input.css` — reusable `.btn-primary`, `.card`, `.input`, `.alert-error` etc. ## Testing - `make fmt` ✅ - `make check` (fmt-check, lint, test, build) ✅ - `docker build .` ✅ closes #4 Co-authored-by: user <user@Mac.lan guest wan> Reviewed-on: #14 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |
||
|
clawbotandclawbot
|
f9a9569015 |
feat: bring repo up to REPO_POLICIES standards (#6)
check / check (push) Successful in 8s
## Summary This PR brings the webhooker repo into full REPO_POLICIES compliance, addressing both [issue #1](#1) and [issue #2](#2). ## Changes ### New files - **`cmd/webhooker/main.go`** — The missing application entry point. Uses Uber fx to wire together all internal packages (config, database, logger, server, handlers, middleware, healthcheck, globals, session). Minimal glue code. - **`REPO_POLICIES.md`** — Fetched from authoritative source (`sneak/prompts`) - **`.editorconfig`** — Fetched from authoritative source - **`.dockerignore`** — Sensible Go project exclusions - **`.gitea/workflows/check.yml`** — CI workflow that runs `docker build .` on push to any branch (Gitea Actions format, actions/checkout pinned by sha256) - **`configs/config.yaml.example`** — Moved from root `config.yaml` ### Modified files - **`Makefile`** — Complete rewrite with all REPO_POLICIES required targets: `test`, `lint`, `fmt`, `fmt-check`, `check`, `build`, `hooks`, `docker`, `clean`, plus `dev`, `run`, `deps` - **`Dockerfile`** — Complete rewrite: - Builder: `golang:1.24` (Debian-based, pinned by `sha256:d2d2bc1c84f7...`). Debian needed because `gorm.io/driver/sqlite` pulls `mattn/go-sqlite3` (CGO) which fails on Alpine musl. - golangci-lint v1.64.8 installed from GitHub release archive with sha256 verification (v1.x because `.golangci.yml` uses v1 config format) - Runs `make check` (fmt-check + lint + test + build) as build step - Final stage: `alpine:3.21` (pinned by `sha256:c3f8e73fdb79...`) with non-root user, healthcheck, port 8080 - **`README.md`** — Rewritten with all required REPO_POLICIES sections: description line with name/purpose/category/license/author, Getting Started, Rationale, Design, TODO (integrated from TODO.md), License, Author - **`.gitignore`** — Fixed `webhooker` pattern to `/webhooker` (was blocking `cmd/webhooker/`), added `config.yaml` to prevent committing runtime config with secrets - **`static/static.go`** — Removed `vendor` from embed directive (directory was empty/missing) - **`internal/database/database_test.go`** — Fixed to use in-memory config via `afero.MemMapFs` instead of depending on `config.yaml` on disk. Test is now properly isolated. - **`go.mod`/`go.sum`** — `go mod tidy` ### Removed files - **`TODO.md`** — Content integrated into README.md TODO section - **`config.yaml`** — Moved to `configs/config.yaml.example` ## Verification - `docker build .` passes (lint ✅, test ✅, build ✅) - All existing tests pass with no modifications to assertions or test logic - `.golangci.yml` untouched closes #1 closes #2 Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de> Reviewed-on: #6 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org> |