clawbot
339548d794
All checks were successful
check / check (push) Successful in 6s
Adds Completed Steps for the receiver aggregate rate limit, the documentation accuracy pass, the CI gate repair and the RETENTION_SWEEP_INTERVAL bound. Drops the commit hash that pinned the Status paragraph to a specific next head, and rewrites Next Step now that the gate repair it named has landed.
201 lines
11 KiB
Markdown
201 lines
11 KiB
Markdown
# Workflow
|
|
|
|
One issue per unit of work, one branch and one PR per issue:
|
|
|
|
* ensure a tracked issue exists with a definition of done
|
|
* branch from `next` (never from `main`)
|
|
* do the work; open a PR based on `next` (never on `main`)
|
|
* pass an independent review, then the manager squash-merges into `next`
|
|
* push; nothing stays local-only
|
|
|
|
`next` is the branch for the next milestone and must stay green and
|
|
mergeable to `main` without notice. One `next` -> `main` PR accumulates
|
|
the milestone; releases are cut from `main` separately.
|
|
|
|
Issue branches do NOT touch this file — the manager maintains it on
|
|
`next`. Every branch editing `TODO.md` conflicts with every other
|
|
(#112).
|
|
|
|
# Status
|
|
|
|
pre-1.0. No git tags exist. `main` (4f5ecb1) is a working webhook proxy
|
|
with auth, CSRF/SSRF protections, login rate limiting, Slack target,
|
|
event retention (#63), the database archiving target (#43), the admin
|
|
password change flow (#65), policy compliance (#6), pinned lint tooling
|
|
(#55), and fail-loud configuration parsing (#80).
|
|
|
|
`next` holds the completed 1.0.0 milestone: every issue in it is closed,
|
|
and it is verified green both by CI and by cache-defeated container
|
|
runs. The two were only made to mean the same thing this cycle — before
|
|
#119, a warm layer cache let the gate report success without executing
|
|
anything, and replayed the previous build's console log so the lie
|
|
looked like a real run. Note: TODO.md was deliberately deleted from this
|
|
repo in f9a9569 (2026-03-01, #6); its content was folded into the README
|
|
TODO section, which this draft reconstructs as of 2026-07-06.
|
|
|
|
# Next Step
|
|
|
|
Merge the milestone PR to `main` and tag 1.0.0 from it.
|
|
|
|
Two decisions are open and belong to the owner, neither blocking the
|
|
tag: #115 (mask the `http` target's destination URL, implemented
|
|
speculatively and awaiting a yes or no) and #125 (whether IPv6
|
|
rate-limit keys should bucket by `/64`).
|
|
|
|
# Completed Steps
|
|
|
|
- 2026-08-12 Bound the receiver rate limit per client IP across the
|
|
whole `/webhook/*` route. The existing limiter keyed on the request
|
|
path and `/webhook/{uuid}` matches any single segment, so a client
|
|
that invented a fresh path per request minted a fresh bucket per
|
|
request: the limit on the only unauthenticated endpoint bounded
|
|
nothing in aggregate, and every request still cost an entrypoint
|
|
lookup before it 404ed. An outer limiter keyed on the client address
|
|
alone now bounds that, chained in front of the unchanged
|
|
per-entrypoint limiter (#139)
|
|
- 2026-08-12 Correct release-blocking documentation inaccuracies: the
|
|
README promised manual redelivery in the present tense in three
|
|
places when nothing implements it (the same false claim also sat in
|
|
the doc comment that was its source text), the env table omitted
|
|
`RETENTION_SWEEP_INTERVAL`, and `TODO.md` itself omitted five landed
|
|
units (#141)
|
|
- 2026-08-12 Make the CI gate execute the checks it reports on. The
|
|
workflow now writes a build-context fingerprint before calling
|
|
`script/cibuild`, so a code commit invalidates the `COPY` layer of
|
|
the lint and builder stages while a docs-only commit still replays
|
|
from cache; a superseding run also rewrites the `failure` status
|
|
Gitea leaves on commits it cancelled and never tested. Verified by
|
|
pushing a deliberately broken test and watching CI go red (#119)
|
|
- 2026-08-12 Require a positive `RETENTION_SWEEP_INTERVAL`: a
|
|
non-positive value reached `time.NewTicker` in both the retention
|
|
reaper and the archive sweeper, panicking two goroutines with no
|
|
recover after startup had already reported success (#140)
|
|
- 2026-08-12 Bound the `X-Forwarded-For` scan's allocation to the hop
|
|
cap: the reverse walk cuts entries with `strings.LastIndexByte`
|
|
instead of joining and splitting, so a 1 MB header allocates 16 bytes
|
|
rather than 1.6 MB per request on the unauthenticated receiver.
|
|
Semantics proven unchanged by differential testing against the
|
|
previous implementation (#133)
|
|
- 2026-08-12 Cap the `X-Forwarded-For` hop walk at 64 entries, so an
|
|
attacker-supplied chain cannot burn unbounded CPU in the rate-limit
|
|
key function; running off the end falls back to the peer address
|
|
(#124)
|
|
- 2026-08-12 Gate forwarded-header trust behind a `TRUSTED_PROXIES` CIDR
|
|
list: all three rate limiters key on the connection's own address
|
|
unless the direct peer is a configured proxy, in which case
|
|
`X-Forwarded-For` is walked right to left for the first non-proxy hop.
|
|
Default trusts nothing, and a set-but-unparseable value aborts
|
|
startup. Before this, any client could mint a fresh bucket or drain
|
|
another's by rotating a spoofed header (#88)
|
|
- 2026-08-11 Web UI cleanup: nav terminology unified on Webhooks, the
|
|
Profile settings placeholder removed, a progressive-enhancement copy
|
|
button for the entrypoint URL, and retention form copy that states the
|
|
actual policy (deletion by the reaper, 0 retains forever) (#57)
|
|
- 2026-08-11 Mask the webhook credential in delivery errors and logs:
|
|
Go embeds the request URL in `*url.Error`, so every transport failure
|
|
persisted the full Slack webhook URL into the per-webhook event
|
|
database via `DeliveryResult.Error`, a field a future REST API would
|
|
have served. `maskURLError` drops path, query and userinfo while
|
|
preserving the wrapped cause, so `errors.Is`/`As` and `Timeout()`
|
|
still work and DNS, TLS and timeout failures still read differently
|
|
(#118)
|
|
- 2026-08-11 Rate-limit the public webhook receiver endpoint
|
|
(`RECEIVER_RATE_LIMIT`, default 120/min), keyed on client IP plus
|
|
entrypoint path so one entrypoint cannot exhaust another's budget;
|
|
over-limit requests get 429 with `Retry-After`. It was the one
|
|
unauthenticated, internet-facing endpoint with no limit at all (#64)
|
|
- 2026-08-11 Enforce the body size limit before CSRF parses the form:
|
|
`MaxBodySize` is now first in all four form-parsing route groups, so
|
|
an oversized request is rejected with 413 instead of being read in
|
|
full by the CSRF middleware before any cap applied (#90)
|
|
- 2026-08-11 Mask target config on the source detail page, which
|
|
rendered the stored blob verbatim and so exposed the Slack
|
|
incoming-webhook URL — a bearer credential that cannot be revoked
|
|
per-holder. Config reaches the template only as a `TargetView` of
|
|
labelled fields, and header values are rendered as a count (#113)
|
|
- 2026-08-11 Allow `retention_days` of 0 to mean retain forever, via a
|
|
sentinel written in `BeforeSave` so the GORM column default cannot
|
|
win the race. Also bounds the reaper's cutoff arithmetic: day counts
|
|
above 106751 overflowed `time.Duration` and wrapped the cutoff into
|
|
the future, where every row matched and the sweep deleted everything
|
|
(#79)
|
|
- 2026-08-09 Inactivity-based session timeout: sliding idle expiry
|
|
(`SESSION_IDLE_TIMEOUT`, default `24h`) refreshed on authenticated
|
|
requests, with the 7-day absolute cap kept as an independent
|
|
backstop that activity never extends (#66)
|
|
- 2026-08-09 Restart recovery and the 60s retry sweep terminally fail an
|
|
orphaned `retrying` delivery whose target type no longer supports
|
|
retries, recording a `DeliveryResult` with the reason instead of
|
|
leaving the delivery stuck forever (#82)
|
|
- 2026-08-09 Root the delivery engine's worker pool and the retention
|
|
reaper's sweep loop at `context.Background()` rather than the fx
|
|
`OnStart` hook context (#97), which carries fx's 15s start timeout and
|
|
killed both roughly fifteen seconds after boot: the proxy silently
|
|
stopped delivering webhooks entirely, and the reaper never ran a
|
|
single sweep under its default one-hour interval
|
|
- 2026-08-09 Archive writer lifecycle (#89): deleting a webhook (or its
|
|
last `database` target) evicts the cached archive writer and closes
|
|
its handle while deliberately leaving `archive-{webhookID}.db` on
|
|
disk, and a new `ArchiveSweeper` prunes idle archives on the existing
|
|
`RETENTION_SWEEP_INTERVAL` without ever creating an archive file
|
|
- 2026-08-09 Configuration parsing fails loudly on set-but-unparseable
|
|
environment values: `envInt` removed in favour of `envPositiveInt`
|
|
plus a `PORT` range check, `envBool` now parses with
|
|
`strconv.ParseBool`, and defaults apply only to unset variables (#80)
|
|
- 2026-08-07 Automatic event retention cleanup based on
|
|
`retention_days`, deleting expired events, deliveries, and delivery
|
|
results from each per-webhook event database (#63)
|
|
- 2026-08-07 Update golangci-lint to v2.12.2 (Docker image digest in
|
|
`Dockerfile`, release-archive sha256 pins in `script/bootstrap`),
|
|
adopt the canonical `.golangci.yml` (v2 `linters.settings` layout so
|
|
`lll`/`funlen`/`cyclop`/`dupl` thresholds actually apply), and fix
|
|
all newly surfaced lint findings
|
|
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
|
|
Makefile shims, README Entrypoints section
|
|
- 2026-03-25 pin golangci-lint Docker image for linting (#55)
|
|
- 2026-03-18 CSRF middleware detects TLS per-request, fixing login over
|
|
plain HTTP and behind reverse proxies (#54)
|
|
- 2026-03-17 root path redirects based on auth state (#52)
|
|
- 2026-03-17 CSRF protection, SSRF prevention for HTTP delivery targets
|
|
with DNS rebinding defense, and per-IP login rate limiting (#42)
|
|
- 2026-03-17 Slack target type for incoming webhook notifications (#47)
|
|
- 2026-03-17 Dockerfile absolute paths and static linking (#49);
|
|
absolute dev DATA_DIR default and clarified env docs (#46)
|
|
- 2026-03-05 security headers middleware, session regeneration on
|
|
login, request body size limits (#41)
|
|
- 2026-03-04 tests for delivery, middleware, and session packages
|
|
(#32); removed globals.Buildarch (#31)
|
|
- 2026-03-04 1.0 MVP merge: Webhook/Entrypoint/Target rename, core
|
|
delivery engine with bounded worker pool and circuit breaker,
|
|
parallel fan-out, per-webhook event databases, management UI (#16)
|
|
- 2026-03-01 repo brought to REPO_POLICIES standards; TODO.md folded
|
|
into README (#6)
|
|
|
|
# Future Steps
|
|
|
|
- Manual event redelivery from the web UI — the "Replay" capability the
|
|
README describes as planned. No redelivery code exists anywhere in the
|
|
tree; events are stored in full, which is all it would be built on
|
|
- Delivery status and retry management UI
|
|
- Per-webhook rate limiting in the receiver handler (per-webhook config
|
|
plus handler enforcement; global limits must not apply to receiver
|
|
endpoints)
|
|
- Webhook signature verification for GitHub and Stripe HMAC formats
|
|
- API key authentication for programmatic access (APIKey model exists;
|
|
Bearer token middleware does not)
|
|
- REST API v1
|
|
- CRUD for webhooks, entrypoints, targets
|
|
- event viewing and filtering endpoints
|
|
- event redelivery endpoint
|
|
- OpenAPI specification
|
|
- Analytics dashboard: success rates, response times, volume
|
|
- A remember-me option at login
|
|
- Password change and reset flow
|
|
- Later, nice to have
|
|
- email delivery target type
|
|
- SNS and S3 delivery targets
|
|
- data transformations (e.g. webhook to Slack message formatting)
|
|
- JSONL file delivery with periodic S3 upload
|
|
- webhook event search and filtering
|
|
- multi-user with role-based access control
|