webhooker/TODO.md

# Workflow

One issue per unit of work, one branch and one PR per issue:

- ensure a tracked issue exists with a definition of done
- branch from `next` (never from `main`)
- do the work; open a PR based on `next` (never on `main`)
- pass an independent review, then the manager squash-merges into `next`
- push; nothing stays local-only

`next` is the branch for the next milestone and must stay green and mergeable to
`main` without notice. One `next` -> `main` PR accumulates the milestone;
releases are cut from `main` separately.

Issue branches do NOT touch this file — the manager maintains it on `next`.
Every branch editing `TODO.md` conflicts with every other (#112).

# Status

The milestone (https://git.eeqj.de/sneak/webhooker/milestone/9) is the
authoritative list, and the only place to read a count or a state of play from.
This file records where the project is, not what is in flight: a sentence whose
truth depends on a branch being unmerged is wrong the moment it merges, and this
file has been wrong that way before.

The durability defect that held the tag has landed
(https://git.eeqj.de/sneak/webhooker/issues/256, commit `8d64259`). Every SQLite
handle opens with WAL journaling and a busy timeout, a bookkeeping write that
fails leaves its delivery in a recoverable state rather than a lying one, and
recovery skips a delivery that already has a successful result row. Final
pre-tag verification exercised it and confirmed it holds. Whatever the milestone
still shows open is what remains before `v1.0.0`.

Delivery is at-least-once by design, not by accident: a send whose result row
does not land is attempted again, so a receiver can see a duplicate. That is
deliberate — the alternative is a silent lost delivery — and the README says so
under Rationale. It is not a defect to re-file.

# Next Step

Clear the rest of the open 1.0.0 milestone
(https://git.eeqj.de/sneak/webhooker/milestone/9) and tag `v1.0.0`. Merging
`next` into `main` is a separate act from tagging and waits on neither of those:
`next` is kept mergeable at all times, which is the point of the branch.

# Completed Steps

- 2026-08-24 Bind the plaintext HTTP listener deliberately, via `BIND_ADDRESS`
  defaulting to `127.0.0.1`, and document the reverse-proxy deployment. A
  hostname, an empty value or a value carrying a port is a startup error, and
  the `Dockerfile` sets `0.0.0.0` because a loopback bind inside a container is
  unreachable (https://git.eeqj.de/sneak/webhooker/issues/268). The same commit
  removed the shutdown race: `httpServer` is built in the constructor rather
  than assigned from the serving goroutine, which orders the write before every
  fx hook and rules out the nil dereference a SIGTERM arriving first would have
  caused, and `sentryEnabled` is an `atomic.Bool`
  (https://git.eeqj.de/sneak/webhooker/issues/226)
- 2026-08-24 Remove inbound request signature verification. The entrypoint UUID
  is the authentication secret, so the per-entrypoint shared secret, the
  `internal/signature` package, the receiver check, the model fields and the
  forms are all gone. This reverses the feature that landed earlier in the same
  milestone (https://git.eeqj.de/sneak/webhooker/issues/67,
  https://git.eeqj.de/sneak/webhooker/issues/279)
- 2026-08-24 Stamp the build version into the binary and render it in the UI
  footer. `script/version` is the single source — `$VERSION`, else
  `git describe --tags --always --dirty`, else `unknown` — so a `make build`
  binary and a `make docker` image from one checkout report the same thing, and
  nothing in it varies between two builds of the same commit, which the release
  gate's byte-identical assertion would catch
  (https://git.eeqj.de/sneak/webhooker/issues/253)
- 2026-08-24 Derive cookie `Secure` and CSRF strictness from the request
  transport rather than from `WEBHOOKER_ENVIRONMENT`. Behind a real TLS proxy
  with the environment left at its `dev` default, the session cookie silently
  lost `Secure` while the CSRF cookie on the same response kept it.
  `X-Forwarded-Proto` is now matched case-insensitively on its first
  comma-separated element, so `HTTPS` and `https, http` no longer fall to the
  relaxed CSRF path (https://git.eeqj.de/sneak/webhooker/issues/269)
- 2026-08-24 Roll back a failed webhook deletion instead of committing it. A
  failing delete committed whatever had already succeeded, hard-deleted the
  per-webhook event database anyway, and redirected as though it had worked —
  orphaned config plus permanently destroyed history, reported as success. All
  three delete positions now roll back with the event database intact
  (https://git.eeqj.de/sneak/webhooker/issues/262)
- 2026-08-24 Name a deleted target on its historical deliveries, marked
  `(deleted)`, rather than leaving the event log unable to say where a delivery
  went. A deleted target's credentials stay masked exactly as a live one's, and
  it cannot become deliverable again through the receiver, resubmit, replay, the
  edit form or the toggle (https://git.eeqj.de/sneak/webhooker/issues/211)
- 2026-08-24 Bound both request-controlled `/metrics` label dimensions, so the
  unauthenticated receiver is no longer a memory-exhaustion vector: `handler`
  carries the chi route pattern, and `method` folds anything chi cannot route
  onto a single `(unmatched)` sentinel. Both were reproduced before the fix —
  300 random method tokens took the series count from 106 to 7,631, and path
  flooding reached 62,532 — and a label audit across a live scrape found no
  third unbounded dimension (https://git.eeqj.de/sneak/webhooker/issues/254,
  https://git.eeqj.de/sneak/webhooker/issues/261)
- 2026-08-24 Validate `max_retries` on both target forms. `abc`, `2.7` and `-5`
  silently became 0 — fire-and-forget — including on the edit path, where it
  destroyed a working value, and `999999999` stored verbatim. The ceiling of 20
  is the `max` both templates already declared
  (https://git.eeqj.de/sneak/webhooker/issues/221)
- 2026-08-24 Resubmit a stored event as a new undelivered event, so a backend
  under development can be tested against real captured traffic. Per-delivery
  replay cannot serve that: it re-sends one finished delivery to its own
  original target, and a target created for a dev backend has no prior delivery
  to replay. Resubmit re-injects the stored event at the top of the receiver
  path and fans it out to whatever targets are active now
  (https://git.eeqj.de/sneak/webhooker/issues/250)
- 2026-08-20 Take an exclusive lock on `DATA_DIR` at startup, so two instances
  on one directory cannot both deliver
  (https://git.eeqj.de/sneak/webhooker/issues/201)
- 2026-08-20 Shut down the app when the HTTP listener fails. The `OnStart` hook
  returned as soon as the serving goroutine was spawned, so a failed listen left
  fx reporting RUNNING and a live process with nothing bound — invisible to
  systemd and Docker restart policies
  (https://git.eeqj.de/sneak/webhooker/issues/200)
- 2026-08-20 Stop target credentials leaking into the per-webhook event
  databases (https://git.eeqj.de/sneak/webhooker/issues/206), log SQL with
  placeholders rather than bound values
  (https://git.eeqj.de/sneak/webhooker/issues/207), and fail loudly on half-set
  metrics auth credentials (https://git.eeqj.de/sneak/webhooker/issues/205)
- 2026-08-20 Read queue depths with `Find`, not `Scan`. `Scan` swaps GORM's own
  trace recorder in for the logging adapter, and that recorder does not
  implement `gorm.ParamsFilter`, so those statements logged their bound values
  interpolated and bypassed the suppression above. The two units gated green
  against a `next` that lacked the other, and `next` went red when both landed
  (https://git.eeqj.de/sneak/webhooker/issues/234)
- 2026-08-20 Render per-attempt delivery detail in the event log
  (https://git.eeqj.de/sneak/webhooker/issues/202) and add replay of a
  terminally failed delivery (https://git.eeqj.de/sneak/webhooker/issues/203)
- 2026-08-20 Expose delivery metrics on `/metrics`
  (https://git.eeqj.de/sneak/webhooker/issues/209) and document the backup,
  restore and upgrade procedures
  (https://git.eeqj.de/sneak/webhooker/issues/210)
- 2026-08-20 Add a `webhooker resetpw` subcommand and a bootstrap banner. The
  admin bootstrap password was printed once among roughly 45 fx lines, and under
  `docker run -d` went to container logs subject to rotation; there was no reset
  path at all, so recovery meant hand-deleting the users row, documented
  nowhere. The password is read from stdin or generated, never from argv where
  `/proc` would publish it (https://git.eeqj.de/sneak/webhooker/issues/208)
- 2026-08-20 Add `ALLOWED_EGRESS_CIDRS`, an allowlist-only escape hatch for the
  SSRF guard, so a self-hosted proxy can forward into the operator's own
  network. The guard's always-blocked set cannot be reopened by configuration
  (https://git.eeqj.de/sneak/webhooker/issues/204)
- 2026-08-20 Harden operator-set target headers, which were carried unsafely
  across a redirect (https://git.eeqj.de/sneak/webhooker/issues/233)
- 2026-08-20 Add a target edit form with headers and timeout fields
  (https://git.eeqj.de/sneak/webhooker/issues/127)
- 2026-08-18 Raise `script/test`'s per-package timeout from 30s to 90s, matching
  the org-wide backstop. `go test` applies `-timeout` per package, and
  `internal/handlers` had grown past the old budget: a cache-defeated build
  failed outright at `GOMAXPROCS=4`, and every run under deliberate host load
  breached 30s. The measurement table lives in the script (#194)
- 2026-08-18 Re-sync `REPO_POLICIES.md` from `prompts`. The local copy was stale
  and still mandated a 20s test target with a 30s timeout, which the org
  replaced with a 60s cap and a 90s backstop. A synced copy is not a source;
  reading it as one nearly produced a PR against `prompts` proposing a change
  already merged there (#196)
- 2026-08-18 Report handler panics through the logger and answer 500. chi
  v1.5.5's `Recoverer` scans for a `panic(0x` frame the runtime no longer emits,
  then indexes `pkg[-1:]`, so it panicked inside its own stack printer before
  writing a byte: the recovery never ran, the client got a dropped connection
  instead of a 500, and the original panic was lost. A local middleware replaces
  it, bounded by `MaxPanicLogLineBytes` (#187)
- 2026-08-18 Route GORM's logger through `slog` and bound it. Every `gorm.Open`
  left `logger.Default` in place at `Warn` with `IgnoreRecordNotFoundError`
  false, so **every record-not-found printed the fully interpolated SQL to
  stdout** — including the client-chosen path on `/webhook/{uuid}` and the
  submitted username on the login form, at no level the operator set and outside
  `internal/logger` entirely. Three call sites, not the two the issue named
  (#178)
- 2026-08-18 Bound every `slog` line against client-chosen text. Eight sites
  reachable unauthenticated, found by reading every `slog` call in the tree
  rather than only the one reported; the budget moved to a shared
  `internal/logfield` so no second truncation exists. `DEBUG` being off by
  default is not a bound and is not treated as one (#176)
- 2026-08-18 Stop a slow host turning a login-guard test into a segfault. A
  non-fatal `assert` on an acquire result was dereferenced on the next line, so
  one timing miss killed the whole `internal/middleware` binary and reddened CI
  for unrelated PRs. The fix also removed a real production race — `acquire`
  could shed a request with a slot standing free, because Go picks uniformly
  among ready `select` cases (#186)
- 2026-08-18 Send the chi route pattern to Sentry rather than the concrete path.
  The receiver's path carries the entrypoint capability token, so every Sentry
  event from `/webhook/{uuid}` shipped a live credential to a third party.
  Request `Data`, `QueryString`, `Cookies` and `Env` are dropped and headers
  reduced to an allowlist (#179)
- 2026-08-18 Read form fields from the POST body only. `r.FormValue` merges the
  query string, so a login could be driven by URL parameters — putting the
  password somewhere that lands in access logs, proxy logs and browser history
  (#160)
- 2026-08-18 Verify login credentials before spending rate-limit budget, so a
  flood of wrong passwords cannot lock out the account it is guessing at. The
  manager took this decision rather than stall the queue; it is flagged on the
  issue for reversal (#150)
- 2026-08-18 Run all linting in Docker via `Dockerfile.lint`. Host lint was
  wrong in both directions from version skew and shared caches. `script/lint`
  asserts the summary line, because `--no-cache-filter` silently ignores a stage
  name it does not match — the flag that makes the gate meaningful fails open
  (#109)
- 2026-08-18 Serve an event's full stored body over HTTP. The list query
  truncates for rendering, and that truncated value was the only way to read a
  body, so the full payload was unreachable (#157)
- 2026-08-18 Bound the access log line against client-chosen text.
  `internal/logfield` budgets by _encoded_ bytes, not runes, so a handler's JSON
  escaping cannot multiply a field past its allowance (#146)
- 2026-08-18 Mark superseded CI commits `failure` rather than `skipped`. A
  skipped run rolls up green, so a commit that was never tested reported success
  (#152)
- 2026-08-18 Set `fx.StopTimeout` inside the container stop grace, so shutdown
  hooks are bounded by a deadline the orchestrator will actually honour rather
  than being killed mid-flush (#134)
- 2026-08-17 Bucket IPv6 rate-limit keys by `/64`. A single allocation hands out
  2^64 addresses, so per-address keying let one client mint unlimited buckets.
  Manager decision, recorded on the issue (#125)
- 2026-08-17 Correct release-blocking README and startup-warning inaccuracies,
  including claims about behaviour the code does not have (#151)
- 2026-08-17 Fetch and verify Alpine.js at build time against
  `static/vendor.sha256` instead of committing the minified blob, so the
  dependency is pinned by hash rather than by trust (#145)
- 2026-08-17 Bound the event log's rendered bodies in the query itself, so a
  large stored payload cannot be read into memory just to be truncated for
  display (#135)
- 2026-08-17 Mask the `http` target's destination URL in the UI: it can carry a
  bearer credential in its path or query, and was rendered verbatim. Manager
  decision to mask unconditionally (#115)
- 2026-08-14 Bound shutdown hooks by their stop context, so a hook that hangs
  cannot hold the process past its grace period (#102)
- 2026-08-14 Render templates via a buffer rather than the `ResponseWriter`, so
  a template error part-way through cannot commit a 200 and then fail — the
  response is written only once it is whole (#123)
- 2026-08-14 Align the session codec's max-age with the 7-day absolute cap. The
  codec accepted cookies the session layer considered expired, so the cap was
  enforced in one place and not the other (#108)
- 2026-08-12 Warn when `TRUSTED_PROXIES` is empty in production, where the safe
  default silently discards forwarded headers and every client rate-limits as
  the proxy's address (#149)
- 2026-08-12 Bound the receiver rate limit per client IP across the whole
  `/webhook/*` route. The existing limiter keyed on the request path and
  `/webhook/{uuid}` matches any single segment, so a client that invented a
  fresh path per request minted a fresh bucket per request: the limit on the
  only unauthenticated endpoint bounded nothing in aggregate, and every request
  still cost an entrypoint lookup before it 404ed. An outer limiter keyed on the
  client address alone now bounds that, chained in front of the unchanged
  per-entrypoint limiter (#139)
- 2026-08-12 Correct release-blocking documentation inaccuracies: the README
  promised manual redelivery in the present tense in three places when nothing
  implements it (the same false claim also sat in the doc comment that was its
  source text), the env table omitted `RETENTION_SWEEP_INTERVAL`, and `TODO.md`
  itself omitted five landed units (#141)
- 2026-08-12 Make the CI gate execute the checks it reports on. The workflow now
  writes a build-context fingerprint before calling `script/cibuild`, so a code
  commit invalidates the `COPY` layer of the lint and builder stages while a
  docs-only commit still replays from cache; a superseding run also rewrites the
  `failure` status Gitea leaves on commits it cancelled and never tested.
  Verified by pushing a deliberately broken test and watching CI go red (#119)
- 2026-08-12 Require a positive `RETENTION_SWEEP_INTERVAL`: a non-positive value
  reached `time.NewTicker` in both the retention reaper and the archive sweeper,
  panicking two goroutines with no recover after startup had already reported
  success (#140)
- 2026-08-12 Bound the `X-Forwarded-For` scan's allocation to the hop cap: the
  reverse walk cuts entries with `strings.LastIndexByte` instead of joining and
  splitting, so a 1 MB header allocates 16 bytes rather than 1.6 MB per request
  on the unauthenticated receiver. Semantics proven unchanged by differential
  testing against the previous implementation (#133)
- 2026-08-12 Cap the `X-Forwarded-For` hop walk at 64 entries, so an
  attacker-supplied chain cannot burn unbounded CPU in the rate-limit key
  function; running off the end falls back to the peer address (#124)
- 2026-08-12 Gate forwarded-header trust behind a `TRUSTED_PROXIES` CIDR list:
  all three rate limiters key on the connection's own address unless the direct
  peer is a configured proxy, in which case `X-Forwarded-For` is walked right to
  left for the first non-proxy hop. Default trusts nothing, and a
  set-but-unparseable value aborts startup. Before this, any client could mint a
  fresh bucket or drain another's by rotating a spoofed header (#88)
- 2026-08-11 Web UI cleanup: nav terminology unified on Webhooks, the Profile
  settings placeholder removed, a progressive-enhancement copy button for the
  entrypoint URL, and retention form copy that states the actual policy
  (deletion by the reaper, 0 retains forever) (#57)
- 2026-08-11 Mask the webhook credential in delivery errors and logs: Go embeds
  the request URL in `*url.Error`, so every transport failure persisted the full
  Slack webhook URL into the per-webhook event database via
  `DeliveryResult.Error`, a field a future REST API would have served.
  `maskURLError` drops path, query and userinfo while preserving the wrapped
  cause, so `errors.Is`/`As` and `Timeout()` still work and DNS, TLS and timeout
  failures still read differently (#118)
- 2026-08-11 Rate-limit the public webhook receiver endpoint
  (`RECEIVER_RATE_LIMIT`, default 120/min), keyed on client IP plus entrypoint
  path so one entrypoint cannot exhaust another's budget; over-limit requests
  get 429 with `Retry-After`. It was the one unauthenticated, internet-facing
  endpoint with no limit at all (#64)
- 2026-08-11 Enforce the body size limit before CSRF parses the form:
  `MaxBodySize` is now first in all four form-parsing route groups, so an
  oversized request is rejected with 413 instead of being read in full by the
  CSRF middleware before any cap applied (#90)
- 2026-08-11 Mask target config on the source detail page, which rendered the
  stored blob verbatim and so exposed the Slack incoming-webhook URL — a bearer
  credential that cannot be revoked per-holder. Config reaches the template only
  as a `TargetView` of labelled fields, and header values are rendered as a
  count (#113)
- 2026-08-11 Allow `retention_days` of 0 to mean retain forever, via a sentinel
  written in `BeforeSave` so the GORM column default cannot win the race. Also
  bounds the reaper's cutoff arithmetic: day counts above 106751 overflowed
  `time.Duration` and wrapped the cutoff into the future, where every row
  matched and the sweep deleted everything (#79)
- 2026-08-09 Inactivity-based session timeout: sliding idle expiry
  (`SESSION_IDLE_TIMEOUT`, default `24h`) refreshed on authenticated requests,
  with the 7-day absolute cap kept as an independent backstop that activity
  never extends (#66)
- 2026-08-09 Restart recovery and the 60s retry sweep terminally fail an
  orphaned `retrying` delivery whose target type no longer supports retries,
  recording a `DeliveryResult` with the reason instead of leaving the delivery
  stuck forever (#82)
- 2026-08-09 Root the delivery engine's worker pool and the retention reaper's
  sweep loop at `context.Background()` rather than the fx `OnStart` hook context
  (#97), which carries fx's 15s start timeout and killed both roughly fifteen
  seconds after boot: the proxy silently stopped delivering webhooks entirely,
  and the reaper never ran a single sweep under its default one-hour interval
- 2026-08-09 Archive writer lifecycle (#89): deleting a webhook (or its last
  `database` target) evicts the cached archive writer and closes its handle
  while deliberately leaving `archive-{webhookID}.db` on disk, and a new
  `ArchiveSweeper` prunes idle archives on the existing
  `RETENTION_SWEEP_INTERVAL` without ever creating an archive file
- 2026-08-09 Configuration parsing fails loudly on set-but-unparseable
  environment values: `envInt` removed in favour of `envPositiveInt` plus a
  `PORT` range check, `envBool` now parses with `strconv.ParseBool`, and
  defaults apply only to unset variables (#80)
- 2026-08-07 Automatic event retention cleanup based on `retention_days`,
  deleting expired events, deliveries, and delivery results from each
  per-webhook event database (#63)
- 2026-08-07 Update golangci-lint to v2.12.2 (Docker image digest in
  `Dockerfile`, release-archive sha256 pins in `script/bootstrap`), adopt the
  canonical `.golangci.yml` (v2 `linters.settings` layout so
  `lll`/`funlen`/`cyclop`/`dupl` thresholds actually apply), and fix all newly
  surfaced lint findings
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
  shims, README Entrypoints section
- 2026-03-25 pin golangci-lint Docker image for linting (#55)
- 2026-03-18 CSRF middleware detects TLS per-request, fixing login over plain
  HTTP and behind reverse proxies (#54)
- 2026-03-17 root path redirects based on auth state (#52)
- 2026-03-17 CSRF protection, SSRF prevention for HTTP delivery targets with DNS
  rebinding defense, and per-IP login rate limiting (#42)
- 2026-03-17 Slack target type for incoming webhook notifications (#47)
- 2026-03-17 Dockerfile absolute paths and static linking (#49); absolute dev
  DATA_DIR default and clarified env docs (#46)
- 2026-03-05 security headers middleware, session regeneration on login, request
  body size limits (#41)
- 2026-03-04 tests for delivery, middleware, and session packages (#32); removed
  the build-architecture global (#31)
- 2026-03-04 1.0 MVP merge: Webhook/Entrypoint/Target rename, core delivery
  engine with bounded worker pool and circuit breaker, parallel fan-out,
  per-webhook event databases, management UI (#16)
- 2026-03-01 repo brought to REPO_POLICIES standards; TODO.md folded into README
  (#6)

# Future Steps

- Delivery status and retry management UI. Replay of a terminally failed
  delivery and per-attempt detail already landed
  (https://git.eeqj.de/sneak/webhooker/issues/203,
  https://git.eeqj.de/sneak/webhooker/issues/202)
- Per-webhook rate limiting in the receiver handler (per-webhook config plus
  handler enforcement; global limits must not apply to receiver endpoints)
- API key authentication for programmatic access (APIKey model exists; Bearer
  token middleware does not)
- REST API v1
    - CRUD for webhooks, entrypoints, targets
    - event viewing and filtering endpoints
    - event redelivery endpoint
    - OpenAPI specification
- Analytics dashboard: success rates, response times, volume
- A remember-me option at login
- Password reset flow for a forgotten password over the web. The authenticated
  password _change_ flow already landed, and a lost password is recoverable from
  the console with `webhooker resetpw`
  (https://git.eeqj.de/sneak/webhooker/issues/208)
- Later, nice to have
    - email delivery target type
    - SNS and S3 delivery targets
    - data transformations (e.g. webhook to Slack message formatting)
    - JSONL file delivery with periodic S3 upload
    - webhook event search and filtering
    - multi-user with role-based access control