Report handler panics through the logger and answer 500 (closes #187)
All checks were successful
check / check (push) Successful in 2m53s

chi v1.5.5's middleware.Recoverer neither logged a handler panic nor
answered 500. Its pretty-printer scans the stack for a frame beginning
"panic(0x", which the runtime no longer emits, so the scan never
terminates early and every line reaches decorateFuncCallLine, which
slices pkg[strings.Index(pkg, "."):] without checking for -1. That
second panic escaped chi's own deferred function, so its
WriteHeader(500) never ran: net/http closed the connection and reported
its own crash, losing the original panic value entirely.

Middleware.Recoverer replaces it. It writes one ERROR record through
internal/logger carrying the panic value, the stack and the request id,
and answers 500. http.ErrAbortHandler is re-panicked rather than
swallowed, and a response the handler already committed is left alone
rather than overwritten.

It is registered inside every middleware that observes the response, so
the 500 is the status the access log records and the metrics count, and
outside the sentryhttp handler, whose Repanic option needs something
further out to catch what it re-raises.

Both fields are bounded in encoded bytes, as the access log's are: 512
for the panic value, since a handler may build one out of the request,
and 8192 for the stack, cut at its far end so the panic site survives.
MaxPanicLogLineBytes states the resulting ceiling at 10240; measured,
the widest either handler produces is 8898, and the real case through
the shipped chain is 3959.
This commit is contained in:
2026-08-18 01:57:55 +00:00
parent b573959a26
commit f346625cad
8 changed files with 1202 additions and 20 deletions

View File

@@ -1140,18 +1140,42 @@ the figure has headroom. `internal/middleware/accesslog_test.go`
asserts it against 8 KB of client-chosen text in the path, in the
query, and in each of `User-Agent`, `Referer` and `X-Request-Id`,
including cases built from the characters the handlers escape, and
against the widest line the service can be made to write: a 5xx that
keeps its concrete path while all three header fields are also at their
budget. Every case runs through both handlers `internal/logger` can
select — the JSON one and the text one it installs on a tty — since the
two do not escape alike and the ceiling is quoted unqualified. Measured
over a real connection, the widest line is 1,972 bytes.
against the widest access log line: a 5xx that keeps its concrete path
while all three header fields are also at their budget. Every case runs
through both handlers `internal/logger` can select — the JSON one and
the text one it installs on a tty — since the two do not escape alike
and the ceiling is quoted unqualified. Measured over a real connection,
the widest access log line is 1,972 bytes.
Multiply that ceiling by the request rate to size log storage. Note
that the rate is not bounded by the limits above on every route:
`/.well-known/healthcheck` and `/s/*` sit behind no limiter, so there
the multiplier is whatever the deployment will serve.
One line is wider, and it is the widest this service writes: the record
a recovered panic produces. The recover middleware in
`internal/middleware` answers `500` and writes one `ERROR` record
carrying the panic value, the stack and the request id — the same
`request_id` the access log line for that request carries, which is how
the two are joined. It replaced chi's `middleware.Recoverer`, which on
a current Go release crashed inside its own stack pretty-printer: the
connection was dropped rather than answered, and what reached the
operator described that crash rather than the fault behind it.
That record is bounded the same way, in the same encoded bytes: 512 for
the panic value, because a handler is free to build one out of the
request, and 8,192 for the stack, cut at its far end so that the panic
site survives a cut and net/http's accept frames are what is lost. Net:
**at most 10,240 bytes, once per recovered panic** — 9,121 by the
arithmetic (523 + 8,203 + 139 + a 256-byte fixed portion), stated at
10,240 for headroom. `internal/middleware/recoverer_test.go` measures
8,898 with the stack and the panic value both driven past their
budgets, over both handlers. The real case is far below that: through
the shipped middleware chain the whole record is 3,959 bytes, which
`internal/server/recoverer_test.go` measures on the process's own file
descriptors, driving a panic through the production router over a real
server in a subprocess.
Every limiter here — receiver, login, and password change — identifies
the client the same way, through one shared key function: the
connection's own address, unless the peer is listed in
@@ -1485,19 +1509,32 @@ to record results.
Applied to all routes in this order:
1. **Recoverer**Panic recovery (chi built-in)
2. **RequestID** — Generate unique request IDs (chi built-in)
3. **SecurityHeaders** — Production security headers on every response
1. **RequestID**Generate unique request IDs (chi built-in)
2. **SecurityHeaders** — Production security headers on every response
(HSTS, X-Content-Type-Options, X-Frame-Options, CSP, Referrer-Policy,
Permissions-Policy)
4. **Logging** — Structured request logging (method, URL, status,
3. **Logging** — Structured request logging (method, URL, status,
latency, remote IP, user agent, request ID)
5. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set)
6. **CORS** — Cross-origin resource sharing headers
7. **Timeout** — 60-second request timeout
4. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set)
5. **CORS** — Cross-origin resource sharing headers
6. **Timeout** — 60-second request timeout
7. **Recoverer** — Panic recovery: one `ERROR` record through
`internal/logger` and a `500`
8. **Sentry** — Error reporting to Sentry (if `SENTRY_DSN` is set;
configured with `Repanic: true` so panics still reach Recoverer)
Recoverer sits seventh rather than first, and both neighbours are the
reason. It runs **inside** everything that observes the response, so
the `500` it writes for a panicking handler is the status the access
log records and the metrics count; registered first, as chi's own
`middleware.Recoverer` was, the same request was logged as a `200` that
the client never received. It runs **outside** the Sentry handler, so
`Repanic: true` has something to re-raise into: an operator with
`SENTRY_DSN` set keeps the report, and one without it now gets the
local record instead of nothing. What that placement gives up is
recovery of a panic in the six entries above it, none of which does
more than set a header or start a timer.
Additionally, form endpoints (`/pages`, `/user/*`, `/sources`,
`/source/*`) apply a **MaxBodySize** middleware that limits
POST/PUT/PATCH request bodies to 1 MB. It is registered ahead of the