webhooker/TODO.md

# Workflow

One issue per unit of work, one branch and one PR per issue:

* ensure a tracked issue exists with a definition of done
* branch from `next` (never from `main`)
* do the work; open a PR based on `next` (never on `main`)
* pass an independent review, then the manager squash-merges into `next`
* push; nothing stays local-only

`next` is the branch for the next milestone and must stay green and
mergeable to `main` without notice. One `next` -> `main` PR accumulates
the milestone; releases are cut from `main` separately.

Issue branches do NOT touch this file — the manager maintains it on
`next`. Every branch editing `TODO.md` conflicts with every other
(#112).

# Status

1.0.0 is open, with work remaining. The milestone
(https://git.eeqj.de/sneak/webhooker/milestone/9) is the authoritative
list, and the only place to read a count or a state of play from. This
file records where the project is, not what is in flight: a sentence
whose truth depends on a branch being unmerged is wrong the moment it
merges, and this file has been wrong that way before.

The durability defect that held the tag has landed
(https://git.eeqj.de/sneak/webhooker/issues/256, commit `8d64259`).
Every SQLite handle opens with WAL journaling and a busy timeout, a
bookkeeping write that fails leaves its delivery in a recoverable
state rather than a lying one, and recovery skips a delivery that
already has a successful result row. Final pre-tag verification
exercised it and confirmed it holds. Whatever the milestone still
shows open is what remains before `v1.0.0`.

Delivery is at-least-once by design, not by accident: a send whose
result row does not land is attempted again, so a receiver can see a
duplicate. That is deliberate — the alternative is a silent lost
delivery — and the README says so under Rationale. It is not a defect
to re-file.

One caveat on reading a green check: a docs-only commit deliberately
replays from the layer cache
(https://git.eeqj.de/sneak/webhooker/issues/119), so a green status on
such a commit evidences a replay rather than an executed run. A code
commit invalidates the `COPY` layer and genuinely executes.

# Next Step

Clear the rest of the open 1.0.0 milestone
(https://git.eeqj.de/sneak/webhooker/milestone/9) and tag `v1.0.0`.
Merging `next` into `main` is a separate act from tagging and waits on
neither of those: `next` is kept mergeable at all times, which is the
point of the branch.

# Completed Steps

- 2026-08-24 Bind the plaintext HTTP listener deliberately, via
  `BIND_ADDRESS` defaulting to `127.0.0.1`, and document the
  reverse-proxy deployment. A hostname, an empty value or a value
  carrying a port is a startup error, and the `Dockerfile` sets
  `0.0.0.0` because a loopback bind inside a container is unreachable
  (https://git.eeqj.de/sneak/webhooker/issues/268). The same commit
  removed the shutdown race: `httpServer` is built in the constructor
  rather than assigned from the serving goroutine, which orders the
  write before every fx hook and rules out the nil dereference a
  SIGTERM arriving first would have caused, and `sentryEnabled` is an
  `atomic.Bool` (https://git.eeqj.de/sneak/webhooker/issues/226)
- 2026-08-24 Remove inbound request signature verification. The
  entrypoint UUID is the authentication secret, so the per-entrypoint
  shared secret, the `internal/signature` package, the receiver check,
  the model fields and the forms are all gone. This reverses the
  feature that landed earlier in the same milestone
  (https://git.eeqj.de/sneak/webhooker/issues/67,
  https://git.eeqj.de/sneak/webhooker/issues/279)
- 2026-08-24 Stamp the build version into the binary and render it in
  the UI footer. `script/version` is the single source — `$VERSION`,
  else `git describe --tags --always --dirty`, else `unknown` — so a
  `make build` binary and a `make docker` image from one checkout
  report the same thing, and nothing in it varies between two builds
  of the same commit, which the release gate's byte-identical
  assertion would catch
  (https://git.eeqj.de/sneak/webhooker/issues/253)
- 2026-08-24 Derive cookie `Secure` and CSRF strictness from the
  request transport rather than from `WEBHOOKER_ENVIRONMENT`. Behind a
  real TLS proxy with the environment left at its `dev` default, the
  session cookie silently lost `Secure` while the CSRF cookie on the
  same response kept it. `X-Forwarded-Proto` is now matched
  case-insensitively on its first comma-separated element, so `HTTPS`
  and `https, http` no longer fall to the relaxed CSRF path
  (https://git.eeqj.de/sneak/webhooker/issues/269)
- 2026-08-24 Roll back a failed webhook deletion instead of committing
  it. A failing delete committed whatever had already succeeded,
  hard-deleted the per-webhook event database anyway, and redirected as
  though it had worked — orphaned config plus permanently destroyed
  history, reported as success. All three delete positions now roll
  back with the event database intact
  (https://git.eeqj.de/sneak/webhooker/issues/262)
- 2026-08-24 Name a deleted target on its historical deliveries, marked
  `(deleted)`, rather than leaving the event log unable to say where a
  delivery went. A deleted target's credentials stay masked exactly as
  a live one's, and it cannot become deliverable again through the
  receiver, resubmit, replay, the edit form or the toggle
  (https://git.eeqj.de/sneak/webhooker/issues/211)
- 2026-08-24 Bound both request-controlled `/metrics` label dimensions,
  so the unauthenticated receiver is no longer a memory-exhaustion
  vector: `handler` carries the chi route pattern, and `method` folds
  anything chi cannot route onto a single `(unmatched)` sentinel. Both
  were reproduced before the fix — 300 random method tokens took the
  series count from 106 to 7,631, and path flooding reached 62,532 —
  and a label audit across a live scrape found no third unbounded
  dimension (https://git.eeqj.de/sneak/webhooker/issues/254,
  https://git.eeqj.de/sneak/webhooker/issues/261)
- 2026-08-24 Validate `max_retries` on both target forms. `abc`, `2.7`
  and `-5` silently became 0 — fire-and-forget — including on the edit
  path, where it destroyed a working value, and `999999999` stored
  verbatim. The ceiling of 20 is the `max` both templates already
  declared (https://git.eeqj.de/sneak/webhooker/issues/221)
- 2026-08-24 Resubmit a stored event as a new undelivered event, so a
  backend under development can be tested against real captured
  traffic. Per-delivery replay cannot serve that: it re-sends one
  finished delivery to its own original target, and a target created
  for a dev backend has no prior delivery to replay. Resubmit
  re-injects the stored event at the top of the receiver path and fans
  it out to whatever targets are active now
  (https://git.eeqj.de/sneak/webhooker/issues/250)
- 2026-08-20 Take an exclusive lock on `DATA_DIR` at startup, so two
  instances on one directory cannot both deliver
  (https://git.eeqj.de/sneak/webhooker/issues/201)
- 2026-08-20 Shut down the app when the HTTP listener fails. The
  `OnStart` hook returned as soon as the serving goroutine was
  spawned, so a failed listen left fx reporting RUNNING and a live
  process with nothing bound — invisible to systemd and Docker restart
  policies (https://git.eeqj.de/sneak/webhooker/issues/200)
- 2026-08-20 Stop target credentials leaking into the per-webhook event
  databases (https://git.eeqj.de/sneak/webhooker/issues/206), log SQL
  with placeholders rather than bound values
  (https://git.eeqj.de/sneak/webhooker/issues/207), and fail loudly on
  half-set metrics auth credentials
  (https://git.eeqj.de/sneak/webhooker/issues/205)
- 2026-08-20 Read queue depths with `Find`, not `Scan`. `Scan` swaps
  GORM's own trace recorder in for the logging adapter, and that
  recorder does not implement `gorm.ParamsFilter`, so those statements
  logged their bound values interpolated and bypassed the suppression
  above. The two units gated green against a `next` that lacked the
  other, and `next` went red when both landed
  (https://git.eeqj.de/sneak/webhooker/issues/234)
- 2026-08-20 Render per-attempt delivery detail in the event log
  (https://git.eeqj.de/sneak/webhooker/issues/202) and add replay of a
  terminally failed delivery
  (https://git.eeqj.de/sneak/webhooker/issues/203)
- 2026-08-20 Expose delivery metrics on `/metrics`
  (https://git.eeqj.de/sneak/webhooker/issues/209) and document the
  backup, restore and upgrade procedures
  (https://git.eeqj.de/sneak/webhooker/issues/210)
- 2026-08-20 Add a `webhooker resetpw` subcommand and a bootstrap
  banner. The admin bootstrap password was printed once among roughly
  45 fx lines, and under `docker run -d` went to container logs subject
  to rotation; there was no reset path at all, so recovery meant
  hand-deleting the users row, documented nowhere. The password is read
  from stdin or generated, never from argv where `/proc` would publish
  it (https://git.eeqj.de/sneak/webhooker/issues/208)
- 2026-08-20 Add `ALLOWED_EGRESS_CIDRS`, an allowlist-only escape hatch
  for the SSRF guard, so a self-hosted proxy can forward into the
  operator's own network. The guard's always-blocked set cannot be
  reopened by configuration
  (https://git.eeqj.de/sneak/webhooker/issues/204)
- 2026-08-20 Harden operator-set target headers, which were carried
  unsafely across a redirect
  (https://git.eeqj.de/sneak/webhooker/issues/233)
- 2026-08-20 Add a target edit form with headers and timeout fields
  (https://git.eeqj.de/sneak/webhooker/issues/127)
- 2026-08-18 Raise `script/test`'s per-package timeout from 30s to 90s,
  matching the org-wide backstop. `go test` applies `-timeout` per
  package, and `internal/handlers` had grown past the old budget: a
  cache-defeated build failed outright at `GOMAXPROCS=4`, and every run
  under deliberate host load breached 30s. The measurement table lives
  in the script (#194)
- 2026-08-18 Re-sync `REPO_POLICIES.md` from `prompts`. The local copy
  was stale and still mandated a 20s test target with a 30s timeout,
  which the org replaced with a 60s cap and a 90s backstop. A synced
  copy is not a source; reading it as one nearly produced a PR against
  `prompts` proposing a change already merged there (#196)
- 2026-08-18 Report handler panics through the logger and answer 500.
  chi v1.5.5's `Recoverer` scans for a `panic(0x` frame the runtime no
  longer emits, then indexes `pkg[-1:]`, so it panicked inside its own
  stack printer before writing a byte: the recovery never ran, the
  client got a dropped connection instead of a 500, and the original
  panic was lost. A local middleware replaces it, bounded by
  `MaxPanicLogLineBytes` (#187)
- 2026-08-18 Route GORM's logger through `slog` and bound it. Every
  `gorm.Open` left `logger.Default` in place at `Warn` with
  `IgnoreRecordNotFoundError` false, so **every record-not-found
  printed the fully interpolated SQL to stdout** — including the
  client-chosen path on `/webhook/{uuid}` and the submitted username on
  the login form, at no level the operator set and outside
  `internal/logger` entirely. Three call sites, not the two the issue
  named (#178)
- 2026-08-18 Bound every `slog` line against client-chosen text. Eight
  sites reachable unauthenticated, found by reading every `slog` call in
  the tree rather than only the one reported; the budget moved to a
  shared `internal/logfield` so no second truncation exists. `DEBUG`
  being off by default is not a bound and is not treated as one (#176)
- 2026-08-18 Stop a slow host turning a login-guard test into a
  segfault. A non-fatal `assert` on an acquire result was dereferenced
  on the next line, so one timing miss killed the whole
  `internal/middleware` binary and reddened CI for unrelated PRs. The
  fix also removed a real production race — `acquire` could shed a
  request with a slot standing free, because Go picks uniformly among
  ready `select` cases (#186)
- 2026-08-18 Send the chi route pattern to Sentry rather than the
  concrete path. The receiver's path carries the entrypoint capability
  token, so every Sentry event from `/webhook/{uuid}` shipped a live
  credential to a third party. Request `Data`, `QueryString`, `Cookies`
  and `Env` are dropped and headers reduced to an allowlist (#179)
- 2026-08-18 Read form fields from the POST body only. `r.FormValue`
  merges the query string, so a login could be driven by URL parameters
  — putting the password somewhere that lands in access logs, proxy
  logs and browser history (#160)
- 2026-08-18 Verify login credentials before spending rate-limit
  budget, so a flood of wrong passwords cannot lock out the account it
  is guessing at. The manager took this decision rather than stall the
  queue; it is flagged on the issue for reversal (#150)
- 2026-08-18 Run all linting in Docker via `Dockerfile.lint`. Host lint
  was wrong in both directions from version skew and shared caches.
  `script/lint` asserts the summary line, because `--no-cache-filter`
  silently ignores a stage name it does not match — the flag that makes
  the gate meaningful fails open (#109)
- 2026-08-18 Serve an event's full stored body over HTTP. The list
  query truncates for rendering, and that truncated value was the only
  way to read a body, so the full payload was unreachable (#157)
- 2026-08-18 Bound the access log line against client-chosen text.
  `internal/logfield` budgets by *encoded* bytes, not runes, so a
  handler's JSON escaping cannot multiply a field past its allowance
  (#146)
- 2026-08-18 Mark superseded CI commits `failure` rather than
  `skipped`. A skipped run rolls up green, so a commit that was never
  tested reported success (#152)
- 2026-08-18 Set `fx.StopTimeout` inside the container stop grace, so
  shutdown hooks are bounded by a deadline the orchestrator will
  actually honour rather than being killed mid-flush (#134)
- 2026-08-17 Bucket IPv6 rate-limit keys by `/64`. A single allocation
  hands out 2^64 addresses, so per-address keying let one client mint
  unlimited buckets. Manager decision, recorded on the issue (#125)
- 2026-08-17 Correct release-blocking README and startup-warning
  inaccuracies, including claims about behaviour the code does not have
  (#151)
- 2026-08-17 Fetch and verify Alpine.js at build time against
  `static/vendor.sha256` instead of committing the minified blob, so
  the dependency is pinned by hash rather than by trust (#145)
- 2026-08-17 Bound the event log's rendered bodies in the query itself,
  so a large stored payload cannot be read into memory just to be
  truncated for display (#135)
- 2026-08-17 Mask the `http` target's destination URL in the UI: it can
  carry a bearer credential in its path or query, and was rendered
  verbatim. Manager decision to mask unconditionally (#115)
- 2026-08-14 Bound shutdown hooks by their stop context, so a hook that
  hangs cannot hold the process past its grace period (#102)
- 2026-08-14 Render templates via a buffer rather than the
  `ResponseWriter`, so a template error part-way through cannot commit
  a 200 and then fail — the response is written only once it is whole
  (#123)
- 2026-08-14 Align the session codec's max-age with the 7-day absolute
  cap. The codec accepted cookies the session layer considered expired,
  so the cap was enforced in one place and not the other (#108)
- 2026-08-12 Warn when `TRUSTED_PROXIES` is empty in production, where
  the safe default silently discards forwarded headers and every client
  rate-limits as the proxy's address (#149)
- 2026-08-12 Bound the receiver rate limit per client IP across the
  whole `/webhook/*` route. The existing limiter keyed on the request
  path and `/webhook/{uuid}` matches any single segment, so a client
  that invented a fresh path per request minted a fresh bucket per
  request: the limit on the only unauthenticated endpoint bounded
  nothing in aggregate, and every request still cost an entrypoint
  lookup before it 404ed. An outer limiter keyed on the client address
  alone now bounds that, chained in front of the unchanged
  per-entrypoint limiter (#139)
- 2026-08-12 Correct release-blocking documentation inaccuracies: the
  README promised manual redelivery in the present tense in three
  places when nothing implements it (the same false claim also sat in
  the doc comment that was its source text), the env table omitted
  `RETENTION_SWEEP_INTERVAL`, and `TODO.md` itself omitted five landed
  units (#141)
- 2026-08-12 Make the CI gate execute the checks it reports on. The
  workflow now writes a build-context fingerprint before calling
  `script/cibuild`, so a code commit invalidates the `COPY` layer of
  the lint and builder stages while a docs-only commit still replays
  from cache; a superseding run also rewrites the `failure` status
  Gitea leaves on commits it cancelled and never tested. Verified by
  pushing a deliberately broken test and watching CI go red (#119)
- 2026-08-12 Require a positive `RETENTION_SWEEP_INTERVAL`: a
  non-positive value reached `time.NewTicker` in both the retention
  reaper and the archive sweeper, panicking two goroutines with no
  recover after startup had already reported success (#140)
- 2026-08-12 Bound the `X-Forwarded-For` scan's allocation to the hop
  cap: the reverse walk cuts entries with `strings.LastIndexByte`
  instead of joining and splitting, so a 1 MB header allocates 16 bytes
  rather than 1.6 MB per request on the unauthenticated receiver.
  Semantics proven unchanged by differential testing against the
  previous implementation (#133)
- 2026-08-12 Cap the `X-Forwarded-For` hop walk at 64 entries, so an
  attacker-supplied chain cannot burn unbounded CPU in the rate-limit
  key function; running off the end falls back to the peer address
  (#124)
- 2026-08-12 Gate forwarded-header trust behind a `TRUSTED_PROXIES` CIDR
  list: all three rate limiters key on the connection's own address
  unless the direct peer is a configured proxy, in which case
  `X-Forwarded-For` is walked right to left for the first non-proxy hop.
  Default trusts nothing, and a set-but-unparseable value aborts
  startup. Before this, any client could mint a fresh bucket or drain
  another's by rotating a spoofed header (#88)
- 2026-08-11 Web UI cleanup: nav terminology unified on Webhooks, the
  Profile settings placeholder removed, a progressive-enhancement copy
  button for the entrypoint URL, and retention form copy that states the
  actual policy (deletion by the reaper, 0 retains forever) (#57)
- 2026-08-11 Mask the webhook credential in delivery errors and logs:
  Go embeds the request URL in `*url.Error`, so every transport failure
  persisted the full Slack webhook URL into the per-webhook event
  database via `DeliveryResult.Error`, a field a future REST API would
  have served. `maskURLError` drops path, query and userinfo while
  preserving the wrapped cause, so `errors.Is`/`As` and `Timeout()`
  still work and DNS, TLS and timeout failures still read differently
  (#118)
- 2026-08-11 Rate-limit the public webhook receiver endpoint
  (`RECEIVER_RATE_LIMIT`, default 120/min), keyed on client IP plus
  entrypoint path so one entrypoint cannot exhaust another's budget;
  over-limit requests get 429 with `Retry-After`. It was the one
  unauthenticated, internet-facing endpoint with no limit at all (#64)
- 2026-08-11 Enforce the body size limit before CSRF parses the form:
  `MaxBodySize` is now first in all four form-parsing route groups, so
  an oversized request is rejected with 413 instead of being read in
  full by the CSRF middleware before any cap applied (#90)
- 2026-08-11 Mask target config on the source detail page, which
  rendered the stored blob verbatim and so exposed the Slack
  incoming-webhook URL — a bearer credential that cannot be revoked
  per-holder. Config reaches the template only as a `TargetView` of
  labelled fields, and header values are rendered as a count (#113)
- 2026-08-11 Allow `retention_days` of 0 to mean retain forever, via a
  sentinel written in `BeforeSave` so the GORM column default cannot
  win the race. Also bounds the reaper's cutoff arithmetic: day counts
  above 106751 overflowed `time.Duration` and wrapped the cutoff into
  the future, where every row matched and the sweep deleted everything
  (#79)
- 2026-08-09 Inactivity-based session timeout: sliding idle expiry
  (`SESSION_IDLE_TIMEOUT`, default `24h`) refreshed on authenticated
  requests, with the 7-day absolute cap kept as an independent
  backstop that activity never extends (#66)
- 2026-08-09 Restart recovery and the 60s retry sweep terminally fail an
  orphaned `retrying` delivery whose target type no longer supports
  retries, recording a `DeliveryResult` with the reason instead of
  leaving the delivery stuck forever (#82)
- 2026-08-09 Root the delivery engine's worker pool and the retention
  reaper's sweep loop at `context.Background()` rather than the fx
  `OnStart` hook context (#97), which carries fx's 15s start timeout and
  killed both roughly fifteen seconds after boot: the proxy silently
  stopped delivering webhooks entirely, and the reaper never ran a
  single sweep under its default one-hour interval
- 2026-08-09 Archive writer lifecycle (#89): deleting a webhook (or its
  last `database` target) evicts the cached archive writer and closes
  its handle while deliberately leaving `archive-{webhookID}.db` on
  disk, and a new `ArchiveSweeper` prunes idle archives on the existing
  `RETENTION_SWEEP_INTERVAL` without ever creating an archive file
- 2026-08-09 Configuration parsing fails loudly on set-but-unparseable
  environment values: `envInt` removed in favour of `envPositiveInt`
  plus a `PORT` range check, `envBool` now parses with
  `strconv.ParseBool`, and defaults apply only to unset variables (#80)
- 2026-08-07 Automatic event retention cleanup based on
  `retention_days`, deleting expired events, deliveries, and delivery
  results from each per-webhook event database (#63)
- 2026-08-07 Update golangci-lint to v2.12.2 (Docker image digest in
  `Dockerfile`, release-archive sha256 pins in `script/bootstrap`),
  adopt the canonical `.golangci.yml` (v2 `linters.settings` layout so
  `lll`/`funlen`/`cyclop`/`dupl` thresholds actually apply), and fix
  all newly surfaced lint findings
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
  Makefile shims, README Entrypoints section
- 2026-03-25 pin golangci-lint Docker image for linting (#55)
- 2026-03-18 CSRF middleware detects TLS per-request, fixing login over
  plain HTTP and behind reverse proxies (#54)
- 2026-03-17 root path redirects based on auth state (#52)
- 2026-03-17 CSRF protection, SSRF prevention for HTTP delivery targets
  with DNS rebinding defense, and per-IP login rate limiting (#42)
- 2026-03-17 Slack target type for incoming webhook notifications (#47)
- 2026-03-17 Dockerfile absolute paths and static linking (#49);
  absolute dev DATA_DIR default and clarified env docs (#46)
- 2026-03-05 security headers middleware, session regeneration on
  login, request body size limits (#41)
- 2026-03-04 tests for delivery, middleware, and session packages
  (#32); removed globals.Buildarch (#31)
- 2026-03-04 1.0 MVP merge: Webhook/Entrypoint/Target rename, core
  delivery engine with bounded worker pool and circuit breaker,
  parallel fan-out, per-webhook event databases, management UI (#16)
- 2026-03-01 repo brought to REPO_POLICIES standards; TODO.md folded
  into README (#6)

# Future Steps

- Delivery status and retry management UI. Replay of a terminally
  failed delivery and per-attempt detail already landed
  (https://git.eeqj.de/sneak/webhooker/issues/203,
  https://git.eeqj.de/sneak/webhooker/issues/202)
- Per-webhook rate limiting in the receiver handler (per-webhook config
  plus handler enforcement; global limits must not apply to receiver
  endpoints)
- API key authentication for programmatic access (APIKey model exists;
  Bearer token middleware does not)
- REST API v1
    - CRUD for webhooks, entrypoints, targets
    - event viewing and filtering endpoints
    - event redelivery endpoint
    - OpenAPI specification
- Analytics dashboard: success rates, response times, volume
- A remember-me option at login
- Password reset flow for a forgotten password over the web. The
  authenticated password *change* flow already landed, and a lost
  password is recoverable from the console with `webhooker resetpw`
  (https://git.eeqj.de/sneak/webhooker/issues/208)
- Later, nice to have
    - email delivery target type
    - SNS and S3 delivery targets
    - data transformations (e.g. webhook to Slack message formatting)
    - JSONL file delivery with periodic S3 upload
    - webhook event search and filtering
    - multi-user with role-based access control