sneak
92f3a016e1
All checks were successful
check / check (push) Successful in 2m59s
The empty-TRUSTED_PROXIES warning was gated on IsProd(), but WEBHOOKER_ENVIRONMENT defaults to dev, so an internet-exposed deployment whose operator never set it got no warning at all — the exact operator error the warning exists to catch. It now fires whenever the list is empty, in any environment, and its text is accurate both behind a reverse proxy (shared buckets, remotely deniable admin login) and with nothing in front of the process (harmless). The startup configuration summary also now logs sessionIdleTimeout, the one value where a valid setting silently disables a security control. The README documented a two-stage Docker build on golang:1.24 running "make check" (the tree has three stages: a golangci-lint lint stage running fmt-check and lint, a golang:1.26.1-bookworm builder running test and build, then the Alpine runtime), advertised the public receiver as accepting all methods (it answers 405 to everything but POST), claimed unqualified per-IP login rate limiting, and left the session-expiry prose orphaned inside the trusted-proxy subsection. The rest of the README was swept against the code rather than only the reported lines: every documented route checked method-by-method against internal/server/routes.go (adding the password-change, entrypoint and target routes that were missing), every environment variable checked against internal/config/config.go (MAINTENANCE_MODE serves no maintenance page — it only sets a healthcheck field), the fx wiring, package tree, prerequisites and dev commands brought back in line with the tree. Statements the sweep had waved through, each re-derived from the code this time: - Circuit breakers are not HTTP-only. target_slack.go embeds httpCore and hands its own MaxRetries to the same retry path, so a slack target with max_retries > 0 gets a breaker on the same defaults. - No WAL exists. Both DSNs are file:%s?cache=shared&mode=rwc and no journal_mode pragma is issued anywhere, so every database runs on SQLite's rollback journal. - Slack config is keyed webhookUrl, not webhook_url; the underscored spelling is only the error message. An operator copying the README wrote config that was silently ignored. - Setting carries no BaseModel, so neither the common-field list nor "soft deletes on all entities" held for it. - DeliveryTask names no type; it is delivery.Task. - script/lint runs golangci-lint on the host. Only script/cibuild and script/docker involve Docker; #109 tracks changing that, and until it lands the README says what the tree does. - An entrypoint's path column holds a bare UUID. The /webhook/ prefix is route only and never stored. - max_queue_size is stored and displayed but consulted by nothing; queue depth is the two fixed 10,000-entry channels. - Inline event bodies are cut at "< 16 KiB", not "≤". - retention_days 0 belongs to the retain-forever band — BeforeSave rewrites it to the sentinel — not the finite one, whose floor is 1. A negative value is a 400 in parseRetentionDays. - The receiver's per-client limit keys on the request path (httprate.KeyByEndpoint), not on the entrypoint; the paragraph eight lines below already said so and the two contradicted each other. - Shutdown goes through lifecycle.WaitForShutdown, which is bounded by fx's stop context and can return with goroutines still running, rather than a bare WaitGroup.Wait(). - Nine of the Makefile's fifteen targets shim script/; build, run, dev, deps, clean and css are inline. - Nine entities are documented and nine model files exist, not eight. - The metrics middleware is only registered when METRICS_USERNAME is set, so an over-limit request is not always counted. Two gaps closed while checking: the target-type list omitted slack entirely, and the package tree omitted event_log_view.go and the three testing.go files, which are ordinary compiled sources exporting NewTestDatabase and NewForTest rather than _test.go scaffolding. TestStaticServesEveryMethod settles what /s/* actually answers: chi's Mount registers every method and http.FileServer special-cases only HEAD, so POST, PUT and DELETE to an asset are served the file. The README claimed GET and HEAD. TODO.md drops the unsupported half of its CI claim, keeping the cache-defeated container runs, and splits the landed password change away from the unimplemented reset flow. Rebasing onto current next moved the tree under three of these sections, so they were re-derived against the rebased tree rather than the one the branch was cut from: alpine.min.js is no longer committed but fetched and hash-verified by script/fetch-assets, so the package tree entry for it was wrong and vendor.sha256 was absent; the builder stage runs that script before make test; the Makefile is now ten shims out of sixteen targets, and fmt-check was missing from the development command list. The Quick Start was also wrong in a way that costs a new contributor a red build: it said make deps, which only runs go mod download and tidy, and make check then fails two tests on the absent asset. It now says make bootstrap, which fetches it.
207 lines
11 KiB
Markdown
207 lines
11 KiB
Markdown
# Workflow
|
|
|
|
One issue per unit of work, one branch and one PR per issue:
|
|
|
|
* ensure a tracked issue exists with a definition of done
|
|
* branch from `next` (never from `main`)
|
|
* do the work; open a PR based on `next` (never on `main`)
|
|
* pass an independent review, then the manager squash-merges into `next`
|
|
* push; nothing stays local-only
|
|
|
|
`next` is the branch for the next milestone and must stay green and
|
|
mergeable to `main` without notice. One `next` -> `main` PR accumulates
|
|
the milestone; releases are cut from `main` separately.
|
|
|
|
Issue branches do NOT touch this file — the manager maintains it on
|
|
`next`. Every branch editing `TODO.md` conflicts with every other
|
|
(#112).
|
|
|
|
# Status
|
|
|
|
pre-1.0. No git tags exist. `main` (4f5ecb1) is a working webhook proxy
|
|
with auth, CSRF/SSRF protections, login rate limiting, Slack target,
|
|
event retention (#63), the database archiving target (#43), the admin
|
|
password change flow (#65), policy compliance (#6), pinned lint tooling
|
|
(#55), and fail-loud configuration parsing (#80).
|
|
|
|
`next` holds the completed 1.0.0 milestone: every issue in it is closed,
|
|
and it is verified green by cache-defeated container runs
|
|
(`docker build --no-cache-filter=lint --no-cache-filter=builder`). The
|
|
CI status is not independently claimed here: a superseded run is
|
|
recorded as `skipped` and still rolls up green, so a commit status on
|
|
`next` does not by itself evidence an executed check (#152). Before
|
|
#119, a warm layer cache also let the gate report success without
|
|
executing anything, and replayed the previous build's console log so
|
|
the lie looked like a real run. Note: `TODO.md` was deliberately
|
|
deleted from this repo in f9a9569 (2026-03-01, #6); its content was
|
|
folded into the README TODO section, which this draft reconstructs as
|
|
of 2026-07-06.
|
|
|
|
# Next Step
|
|
|
|
Merge the milestone PR to `main` and tag 1.0.0 from it.
|
|
|
|
Two decisions are open and belong to the owner, neither blocking the
|
|
tag: #115 (mask the `http` target's destination URL, implemented
|
|
speculatively and awaiting a yes or no) and #125 (whether IPv6
|
|
rate-limit keys should bucket by `/64`).
|
|
|
|
# Completed Steps
|
|
|
|
- 2026-08-12 Bound the receiver rate limit per client IP across the
|
|
whole `/webhook/*` route. The existing limiter keyed on the request
|
|
path and `/webhook/{uuid}` matches any single segment, so a client
|
|
that invented a fresh path per request minted a fresh bucket per
|
|
request: the limit on the only unauthenticated endpoint bounded
|
|
nothing in aggregate, and every request still cost an entrypoint
|
|
lookup before it 404ed. An outer limiter keyed on the client address
|
|
alone now bounds that, chained in front of the unchanged
|
|
per-entrypoint limiter (#139)
|
|
- 2026-08-12 Correct release-blocking documentation inaccuracies: the
|
|
README promised manual redelivery in the present tense in three
|
|
places when nothing implements it (the same false claim also sat in
|
|
the doc comment that was its source text), the env table omitted
|
|
`RETENTION_SWEEP_INTERVAL`, and `TODO.md` itself omitted five landed
|
|
units (#141)
|
|
- 2026-08-12 Make the CI gate execute the checks it reports on. The
|
|
workflow now writes a build-context fingerprint before calling
|
|
`script/cibuild`, so a code commit invalidates the `COPY` layer of
|
|
the lint and builder stages while a docs-only commit still replays
|
|
from cache; a superseding run also rewrites the `failure` status
|
|
Gitea leaves on commits it cancelled and never tested. Verified by
|
|
pushing a deliberately broken test and watching CI go red (#119)
|
|
- 2026-08-12 Require a positive `RETENTION_SWEEP_INTERVAL`: a
|
|
non-positive value reached `time.NewTicker` in both the retention
|
|
reaper and the archive sweeper, panicking two goroutines with no
|
|
recover after startup had already reported success (#140)
|
|
- 2026-08-12 Bound the `X-Forwarded-For` scan's allocation to the hop
|
|
cap: the reverse walk cuts entries with `strings.LastIndexByte`
|
|
instead of joining and splitting, so a 1 MB header allocates 16 bytes
|
|
rather than 1.6 MB per request on the unauthenticated receiver.
|
|
Semantics proven unchanged by differential testing against the
|
|
previous implementation (#133)
|
|
- 2026-08-12 Cap the `X-Forwarded-For` hop walk at 64 entries, so an
|
|
attacker-supplied chain cannot burn unbounded CPU in the rate-limit
|
|
key function; running off the end falls back to the peer address
|
|
(#124)
|
|
- 2026-08-12 Gate forwarded-header trust behind a `TRUSTED_PROXIES` CIDR
|
|
list: all three rate limiters key on the connection's own address
|
|
unless the direct peer is a configured proxy, in which case
|
|
`X-Forwarded-For` is walked right to left for the first non-proxy hop.
|
|
Default trusts nothing, and a set-but-unparseable value aborts
|
|
startup. Before this, any client could mint a fresh bucket or drain
|
|
another's by rotating a spoofed header (#88)
|
|
- 2026-08-11 Web UI cleanup: nav terminology unified on Webhooks, the
|
|
Profile settings placeholder removed, a progressive-enhancement copy
|
|
button for the entrypoint URL, and retention form copy that states the
|
|
actual policy (deletion by the reaper, 0 retains forever) (#57)
|
|
- 2026-08-11 Mask the webhook credential in delivery errors and logs:
|
|
Go embeds the request URL in `*url.Error`, so every transport failure
|
|
persisted the full Slack webhook URL into the per-webhook event
|
|
database via `DeliveryResult.Error`, a field a future REST API would
|
|
have served. `maskURLError` drops path, query and userinfo while
|
|
preserving the wrapped cause, so `errors.Is`/`As` and `Timeout()`
|
|
still work and DNS, TLS and timeout failures still read differently
|
|
(#118)
|
|
- 2026-08-11 Rate-limit the public webhook receiver endpoint
|
|
(`RECEIVER_RATE_LIMIT`, default 120/min), keyed on client IP plus
|
|
entrypoint path so one entrypoint cannot exhaust another's budget;
|
|
over-limit requests get 429 with `Retry-After`. It was the one
|
|
unauthenticated, internet-facing endpoint with no limit at all (#64)
|
|
- 2026-08-11 Enforce the body size limit before CSRF parses the form:
|
|
`MaxBodySize` is now first in all four form-parsing route groups, so
|
|
an oversized request is rejected with 413 instead of being read in
|
|
full by the CSRF middleware before any cap applied (#90)
|
|
- 2026-08-11 Mask target config on the source detail page, which
|
|
rendered the stored blob verbatim and so exposed the Slack
|
|
incoming-webhook URL — a bearer credential that cannot be revoked
|
|
per-holder. Config reaches the template only as a `TargetView` of
|
|
labelled fields, and header values are rendered as a count (#113)
|
|
- 2026-08-11 Allow `retention_days` of 0 to mean retain forever, via a
|
|
sentinel written in `BeforeSave` so the GORM column default cannot
|
|
win the race. Also bounds the reaper's cutoff arithmetic: day counts
|
|
above 106751 overflowed `time.Duration` and wrapped the cutoff into
|
|
the future, where every row matched and the sweep deleted everything
|
|
(#79)
|
|
- 2026-08-09 Inactivity-based session timeout: sliding idle expiry
|
|
(`SESSION_IDLE_TIMEOUT`, default `24h`) refreshed on authenticated
|
|
requests, with the 7-day absolute cap kept as an independent
|
|
backstop that activity never extends (#66)
|
|
- 2026-08-09 Restart recovery and the 60s retry sweep terminally fail an
|
|
orphaned `retrying` delivery whose target type no longer supports
|
|
retries, recording a `DeliveryResult` with the reason instead of
|
|
leaving the delivery stuck forever (#82)
|
|
- 2026-08-09 Root the delivery engine's worker pool and the retention
|
|
reaper's sweep loop at `context.Background()` rather than the fx
|
|
`OnStart` hook context (#97), which carries fx's 15s start timeout and
|
|
killed both roughly fifteen seconds after boot: the proxy silently
|
|
stopped delivering webhooks entirely, and the reaper never ran a
|
|
single sweep under its default one-hour interval
|
|
- 2026-08-09 Archive writer lifecycle (#89): deleting a webhook (or its
|
|
last `database` target) evicts the cached archive writer and closes
|
|
its handle while deliberately leaving `archive-{webhookID}.db` on
|
|
disk, and a new `ArchiveSweeper` prunes idle archives on the existing
|
|
`RETENTION_SWEEP_INTERVAL` without ever creating an archive file
|
|
- 2026-08-09 Configuration parsing fails loudly on set-but-unparseable
|
|
environment values: `envInt` removed in favour of `envPositiveInt`
|
|
plus a `PORT` range check, `envBool` now parses with
|
|
`strconv.ParseBool`, and defaults apply only to unset variables (#80)
|
|
- 2026-08-07 Automatic event retention cleanup based on
|
|
`retention_days`, deleting expired events, deliveries, and delivery
|
|
results from each per-webhook event database (#63)
|
|
- 2026-08-07 Update golangci-lint to v2.12.2 (Docker image digest in
|
|
`Dockerfile`, release-archive sha256 pins in `script/bootstrap`),
|
|
adopt the canonical `.golangci.yml` (v2 `linters.settings` layout so
|
|
`lll`/`funlen`/`cyclop`/`dupl` thresholds actually apply), and fix
|
|
all newly surfaced lint findings
|
|
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
|
|
Makefile shims, README Entrypoints section
|
|
- 2026-03-25 pin golangci-lint Docker image for linting (#55)
|
|
- 2026-03-18 CSRF middleware detects TLS per-request, fixing login over
|
|
plain HTTP and behind reverse proxies (#54)
|
|
- 2026-03-17 root path redirects based on auth state (#52)
|
|
- 2026-03-17 CSRF protection, SSRF prevention for HTTP delivery targets
|
|
with DNS rebinding defense, and per-IP login rate limiting (#42)
|
|
- 2026-03-17 Slack target type for incoming webhook notifications (#47)
|
|
- 2026-03-17 Dockerfile absolute paths and static linking (#49);
|
|
absolute dev DATA_DIR default and clarified env docs (#46)
|
|
- 2026-03-05 security headers middleware, session regeneration on
|
|
login, request body size limits (#41)
|
|
- 2026-03-04 tests for delivery, middleware, and session packages
|
|
(#32); removed globals.Buildarch (#31)
|
|
- 2026-03-04 1.0 MVP merge: Webhook/Entrypoint/Target rename, core
|
|
delivery engine with bounded worker pool and circuit breaker,
|
|
parallel fan-out, per-webhook event databases, management UI (#16)
|
|
- 2026-03-01 repo brought to REPO_POLICIES standards; TODO.md folded
|
|
into README (#6)
|
|
|
|
# Future Steps
|
|
|
|
- Manual event redelivery from the web UI — the "Replay" capability the
|
|
README describes as planned. No redelivery code exists anywhere in the
|
|
tree; events are stored in full, which is all it would be built on
|
|
- Delivery status and retry management UI
|
|
- Per-webhook rate limiting in the receiver handler (per-webhook config
|
|
plus handler enforcement; global limits must not apply to receiver
|
|
endpoints)
|
|
- Webhook signature verification for GitHub and Stripe HMAC formats
|
|
- API key authentication for programmatic access (APIKey model exists;
|
|
Bearer token middleware does not)
|
|
- REST API v1
|
|
- CRUD for webhooks, entrypoints, targets
|
|
- event viewing and filtering endpoints
|
|
- event redelivery endpoint
|
|
- OpenAPI specification
|
|
- Analytics dashboard: success rates, response times, volume
|
|
- A remember-me option at login
|
|
- Password reset flow for a forgotten password. The authenticated
|
|
password *change* flow already landed on `main` (#65); reset does not
|
|
exist
|
|
- Later, nice to have
|
|
- email delivery target type
|
|
- SNS and S3 delivery targets
|
|
- data transformations (e.g. webhook to Slack message formatting)
|
|
- JSONL file delivery with periodic S3 upload
|
|
- webhook event search and filtering
|
|
- multi-user with role-based access control
|