Commit Graph

9 Commits

Author SHA1 Message Date
clawbot
43f72e0fd8 Make SQLite durable under concurrent readers and stop re-delivering stranded webhooks (closes #256)
All checks were successful
check / check (push) Successful in 3m33s
An operator running `sqlite3 <db> .dump` against their own per-webhook
database wedged it: inbound webhooks rejected with HTTP 500, delivered
webhooks stranded at `pending`, and every one of them POSTed a second
time on the next restart while the event log recorded a single attempt.

Durability. Every SQLite file — main, per-webhook, and archive — now
opens through one path, `internal/database/sqlite_open.go`, in WAL
journal mode with a 10-second busy timeout, `BEGIN IMMEDIATE`
transactions, and a bounded connection pool. WAL is what stops a reader
blocking writers at all. `_txlock=immediate` is what stops a `COMMIT`
failing while its transaction stays open on a pooled connection, which
is how four `database is locked` errors became 593 `cannot start a
transaction within a transaction`. `cache=shared` is gone, because
under it an in-process conflict is SQLITE_LOCKED, which the busy
handler does not retry. The busy timeout is applied before
journal_mode: the driver runs DSN pragmas in order on every new
connection, and `PRAGMA journal_mode` takes a lock, so the reverse
order leaves the one pragma that can block uncovered by the handler
meant to cover it.

Eligibility. `internal/delivery/inflight.go` holds the set of
deliveries the engine owns — taken when a task is queued, when a
target schedules a retry, and by every recovery path before it
re-dispatches; dropped when the worker that ran the task returns.
Recovery and both sweep arms re-dispatch only what the set does not
hold. Nothing decides that from a row's age: a delivery waiting in a
10000-deep channel is arbitrarily old and perfectly healthy, and
reasoning from age re-sends it. `takeForRedispatch` is the single gate
every re-dispatch goes through — ownership first, then a conditional
update confirming the row is still in the status the batch read.

Bookkeeping. `recordResult` and `updateDeliveryStatus` return their
errors instead of logging and dropping them, and a caller whose
bookkeeping write failed writes nothing at all: the delivery keeps
whichever non-terminal status it already held, and the sweeps recover
it. Every recovery path — pending and retrying alike — first settles
any delivery that already holds a successful `DeliveryResult` rather
than sending it again. Recovery continues each delivery's own attempt
numbering instead of restarting at 1. The sweep gains a
`pending`-with-age-bound arm, so a stranded delivery no longer waits
for a restart.

Docs. WAL produces `-wal`/`-shm` sidecars, so the backup and restore
procedures in README.md are corrected against measurement: both
documented procedures were re-run against a live instance, a `-wal`
left by a crash carries data the `.db` alone does not, and an archive
file normally holds its rows in a `-wal` rather than in the `.db`.
2026-08-24 00:55:39 +00:00
f0512f1c3c Render delivery attempt detail in the event log (closes #202) (#219)
All checks were successful
check / check (push) Successful in 2m55s
delivery_results stored status_code, response_body, error, duration and
attempt_num, and no template rendered any of it, so a failure read as
"target: failed" and diagnosing it meant opening the per-webhook SQLite
file by hand.

An expanded delivery now lists its attempts with attempt number, status
code, duration, error and response body. The body is bounded in the query
rather than read whole and truncated in Go (#135), and a body the engine
itself cut is no longer presented as complete.

The response body and error are untrusted remote content, so target
credentials are removed before rendering. Two cases needed care: a secret
severed by the 4096-byte cut matches nothing as a whole string, and the
engine's io.LimitReader cuts at the same constant the renderer uses, so
the guard keys on the body reaching the cap rather than on the stored size
exceeding it. Empty secrets are filtered where the secret list is built,
because an empty string passed to strings.ReplaceAll inserts the marker at
every byte boundary.

loadTargetMap builds the redactor half unscoped, so a soft-deleted
target's historical deliveries still render redacted.

Also regenerates static/css/tailwind.css, which had drifted from the
templates: hover:text-red-700, text-red-500, underline and w-28 were in
use but absent from the served stylesheet (#236).
2026-08-20 08:36:25 +02:00
4cc83b2326 Expose delivery metrics on /metrics (closes #209) (#224)
Some checks failed
check / check (push) Superseded by a newer commit; never tested
2026-08-20 07:19:04 +02:00
2ee720a9af Bound shutdown hooks by their stop context (closes #102)
All checks were successful
check / check (push) Successful in 3m49s
2026-08-14 06:18:33 +02:00
0ce8565f51 Terminally fail retrying deliveries with a non-retry target type (closes #82) (#104)
All checks were successful
check / check (push) Superseded by a newer commit; never tested
A delivery left in `retrying` whose target type was edited to a fire-and-forget
or unknown type was skipped forever by both restart recovery and the retry
sweep. Both paths now record a result row and mark it `failed`.
2026-08-10 16:00:03 +02:00
62481a6f1a Root background loops at context.Background() (closes #97) (#100)
All checks were successful
check / check (push) Successful in 5s
The delivery engine worker pool and the retention reaper both rooted their
goroutines in the fx OnStart hook context, which fx cancels 15s into startup.
Both now use context.WithCancel(context.Background()), bounded by OnStop.
2026-08-10 15:44:56 +02:00
734606b7af Update golangci-lint to v2.12.2 with canonical config (#86)
All checks were successful
check / check (push) Successful in 3s
Bumps golangci-lint from v2.11.3 to v2.12.2 and adopts the canonical lint config.

## Version pins

- `Dockerfile`: `golangci/golangci-lint:v2.12.2` Debian image, pinned by digest, dated `2026-08-07`
- `script/bootstrap`: `GOLANGCI_LINT_VERSION=2.12.2` with updated sha256 pins for the `linux-amd64` and `linux-arm64` release archives

## Config

`.golangci.yml` replaced with the canonical config. The previous file kept `lll`/`funlen`/`cyclop`/`dupl` settings under the top-level `linters-settings` key, which the v2 schema ignores; the canonical config nests them under `linters.settings`, so those thresholds now actually apply. The unsupported `issues.exclude-use-default` key was dropped.

## Lint fixes (32 findings)

- `lll` (7): wrapped or shortened over-length lines (struct tag comments moved above fields, test logger construction split, `session.NewForTest` signature wrapped, shortened a `#nosec` comment)
- `goconst` (17): replaced repeated `"POST"`/`"PUT"` literals with `http.MethodPost`/`http.MethodPut`, added shared test constants for `webhooker-test`/`test`/`application/json`, and added `tmplKeyError`/`tmplKeyWebhook` constants for template data keys in `internal/handlers`
- `dupl` (8): merged `buildHTTPTargetConfig` and `buildSlackTargetConfig` into a parameterized `buildURLTargetConfig`; removed the duplicate `iWebhookDB` test helper in favor of `testWebhookDB`; extracted shared helpers in middleware and session tests

No `//nolint` directives were added and behavior is unchanged. `make check` (fmt-check, tests, lint) passes.

Note: golangci-lint v2.12 deprecates the `gomodguard` linter in favor of `gomodguard_v2`; the canonical config change for that is left for a future coordinated update.
Co-authored-by: sneak <sneak@sneak.berlin>
Reviewed-on: #86
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-08-07 23:18:49 +02:00
afe88c601a refactor: use pinned golangci-lint Docker image for linting (#55)
All checks were successful
check / check (push) Successful in 5s
Closes [issue #50](#50)

## Summary

Refactors the Dockerfile to use a separate lint stage with a pinned golangci-lint Docker image, following the pattern used by [sneak/pixa](https://git.eeqj.de/sneak/pixa). This replaces the previous approach of installing golangci-lint via curl in the builder stage.

## Changes

### Dockerfile
- **New `lint` stage** using `golangci/golangci-lint:v2.11.3` (Debian-based, pinned by sha256 digest) as a separate build stage
- **Builder stage** depends on lint via `COPY --from=lint /src/go.sum /dev/null` — build won't proceed unless linting passes
- **Go bumped** from 1.24 to 1.26.1 (`golang:1.26.1-bookworm`, pinned by sha256)
- **golangci-lint bumped** from v1.64.8 to v2.11.3
- All three Docker images (golangci-lint, golang, alpine) pinned by sha256 digest
- Debian-based golangci-lint image used (not Alpine) because mattn/go-sqlite3 CGO does not compile on musl (off64_t)

### Linter Config (.golangci.yml)
- Migrated from v1 to v2 format (`version: "2"` added)
- Removed linters no longer available in v2: `gofmt` (handled by `make fmt-check`), `gosimple` (merged into `staticcheck`), `typecheck` (always-on in v2)
- Same set of linters enabled — no rules weakened

### Code Fixes (all lint issues from v2 upgrade)
- Added package comments to all packages
- Added doc comments to all exported types, functions, and methods
- Fixed unchecked errors flagged by `errcheck` (sqlDB.Close, os.Setenv in tests, resp.Body.Close, fmt.Fprint)
- Fixed unused parameters flagged by `revive` (renamed to `_`)
- Fixed `gosec` G120 warnings: added `http.MaxBytesReader` before `r.ParseForm()` calls
- Fixed `staticcheck` QF1012: replaced `WriteString(fmt.Sprintf(...))` with `fmt.Fprintf`
- Fixed `staticcheck` QF1003: converted if/else chain to tagged switch
- Renamed `DeliveryTask` → `Task` to avoid package stutter (`delivery.Task` instead of `delivery.DeliveryTask`)
- Renamed shadowed builtin `max` parameter to `upperBound` in `cryptoRandInt`
- Used `t.Setenv` instead of `os.Setenv` in tests (auto-restores)

### README.md
- Updated version requirements: Go 1.26+, golangci-lint v2.11+
- Updated Dockerfile description in project structure

## Verification

`docker build .` passes cleanly — formatting check, linting, all tests, and build all succeed.

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #55
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-25 02:16:38 +01:00
289f479772 test: add tests for delivery, middleware, and session packages (#32)
Some checks failed
check / check (push) Has been cancelled
## Summary

Add comprehensive test coverage for three previously-untested packages, addressing [issue #28](#28).

## Coverage Improvements

| Package | Before | After |
|---------|--------|-------|
| `internal/delivery` | 37.1% | 74.5% |
| `internal/middleware` | 0.0% | 70.2% |
| `internal/session` | 0.0% | 51.5% |

## What's Tested

### delivery (37% → 75%)
- `processNewTask` with inline and large (DB-fetched) bodies
- `processRetryTask` success, skip non-retrying, large body fetch
- Worker lifecycle start/stop, retry channel processing
- `processDelivery` unknown target type handling
- `recoverPendingDeliveries`, `recoverWebhookDeliveries`, `recoverInFlight`
- HTTP delivery with custom headers, timeout, invalid config
- `Notify` batching

### middleware (0% → 70%)
- Logging middleware status code capture and pass-through
- `LoggingResponseWriter` delegation
- CORS dev mode (allow-all) and prod mode (no-op)
- `RequireAuth` redirect for unauthenticated, pass-through for authenticated
- `MetricsAuth` basic auth validation
- `ipFromHostPort` helper

### session (0% → 52%)
- `Get`/`Save` round-trip with real cookie store
- `SetUser`, `GetUserID`, `GetUsername`, `IsAuthenticated`
- `ClearUser` removes all keys
- `Destroy` invalidates session (MaxAge -1)
- Session persistence across requests
- Edge cases: overwrite user, wrong type, constants

## Test Helpers Added
- `database.NewTestDatabase` / `NewTestWebhookDBManager` — cross-package test helpers for delivery integration tests
- `session.NewForTest` — creates session manager without fx lifecycle for middleware tests

## Notes
- No production code modified
- All tests use `httptest`, SQLite in-memory, and real cookie stores — no external network calls
- Full test suite completes in ~3.5s within the 30s timeout
- `docker build .` passes (lint + test + build)

closes #28

Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #32
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-03-04 12:07:23 +01:00