Report handler panics through the logger and answer 500 (closes #187)
All checks were successful
check / check (push) Successful in 2m53s
All checks were successful
check / check (push) Successful in 2m53s
chi v1.5.5's middleware.Recoverer neither logged a handler panic nor answered 500. Its pretty-printer scans the stack for a frame beginning "panic(0x", which the runtime no longer emits, so the scan never terminates early and every line reaches decorateFuncCallLine, which slices pkg[strings.Index(pkg, "."):] without checking for -1. That second panic escaped chi's own deferred function, so its WriteHeader(500) never ran: net/http closed the connection and reported its own crash, losing the original panic value entirely. Middleware.Recoverer replaces it. It writes one ERROR record through internal/logger carrying the panic value, the stack and the request id, and answers 500. http.ErrAbortHandler is re-panicked rather than swallowed, and a response the handler already committed is left alone rather than overwritten. It is registered inside every middleware that observes the response, so the 500 is the status the access log records and the metrics count, and outside the sentryhttp handler, whose Repanic option needs something further out to catch what it re-raises. Both fields are bounded in encoded bytes, as the access log's are: 512 for the panic value, since a handler may build one out of the request, and 8192 for the stack, cut at its far end so the panic site survives. MaxPanicLogLineBytes states the resulting ceiling at 10240; measured, the widest either handler produces is 8898, and the real case through the shipped chain is 3959.
This commit is contained in:
63
README.md
63
README.md
@@ -1140,18 +1140,42 @@ the figure has headroom. `internal/middleware/accesslog_test.go`
|
||||
asserts it against 8 KB of client-chosen text in the path, in the
|
||||
query, and in each of `User-Agent`, `Referer` and `X-Request-Id`,
|
||||
including cases built from the characters the handlers escape, and
|
||||
against the widest line the service can be made to write: a 5xx that
|
||||
keeps its concrete path while all three header fields are also at their
|
||||
budget. Every case runs through both handlers `internal/logger` can
|
||||
select — the JSON one and the text one it installs on a tty — since the
|
||||
two do not escape alike and the ceiling is quoted unqualified. Measured
|
||||
over a real connection, the widest line is 1,972 bytes.
|
||||
against the widest access log line: a 5xx that keeps its concrete path
|
||||
while all three header fields are also at their budget. Every case runs
|
||||
through both handlers `internal/logger` can select — the JSON one and
|
||||
the text one it installs on a tty — since the two do not escape alike
|
||||
and the ceiling is quoted unqualified. Measured over a real connection,
|
||||
the widest access log line is 1,972 bytes.
|
||||
|
||||
Multiply that ceiling by the request rate to size log storage. Note
|
||||
that the rate is not bounded by the limits above on every route:
|
||||
`/.well-known/healthcheck` and `/s/*` sit behind no limiter, so there
|
||||
the multiplier is whatever the deployment will serve.
|
||||
|
||||
One line is wider, and it is the widest this service writes: the record
|
||||
a recovered panic produces. The recover middleware in
|
||||
`internal/middleware` answers `500` and writes one `ERROR` record
|
||||
carrying the panic value, the stack and the request id — the same
|
||||
`request_id` the access log line for that request carries, which is how
|
||||
the two are joined. It replaced chi's `middleware.Recoverer`, which on
|
||||
a current Go release crashed inside its own stack pretty-printer: the
|
||||
connection was dropped rather than answered, and what reached the
|
||||
operator described that crash rather than the fault behind it.
|
||||
|
||||
That record is bounded the same way, in the same encoded bytes: 512 for
|
||||
the panic value, because a handler is free to build one out of the
|
||||
request, and 8,192 for the stack, cut at its far end so that the panic
|
||||
site survives a cut and net/http's accept frames are what is lost. Net:
|
||||
**at most 10,240 bytes, once per recovered panic** — 9,121 by the
|
||||
arithmetic (523 + 8,203 + 139 + a 256-byte fixed portion), stated at
|
||||
10,240 for headroom. `internal/middleware/recoverer_test.go` measures
|
||||
8,898 with the stack and the panic value both driven past their
|
||||
budgets, over both handlers. The real case is far below that: through
|
||||
the shipped middleware chain the whole record is 3,959 bytes, which
|
||||
`internal/server/recoverer_test.go` measures on the process's own file
|
||||
descriptors, driving a panic through the production router over a real
|
||||
server in a subprocess.
|
||||
|
||||
Every limiter here — receiver, login, and password change — identifies
|
||||
the client the same way, through one shared key function: the
|
||||
connection's own address, unless the peer is listed in
|
||||
@@ -1485,19 +1509,32 @@ to record results.
|
||||
|
||||
Applied to all routes in this order:
|
||||
|
||||
1. **Recoverer** — Panic recovery (chi built-in)
|
||||
2. **RequestID** — Generate unique request IDs (chi built-in)
|
||||
3. **SecurityHeaders** — Production security headers on every response
|
||||
1. **RequestID** — Generate unique request IDs (chi built-in)
|
||||
2. **SecurityHeaders** — Production security headers on every response
|
||||
(HSTS, X-Content-Type-Options, X-Frame-Options, CSP, Referrer-Policy,
|
||||
Permissions-Policy)
|
||||
4. **Logging** — Structured request logging (method, URL, status,
|
||||
3. **Logging** — Structured request logging (method, URL, status,
|
||||
latency, remote IP, user agent, request ID)
|
||||
5. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set)
|
||||
6. **CORS** — Cross-origin resource sharing headers
|
||||
7. **Timeout** — 60-second request timeout
|
||||
4. **Metrics** — Prometheus HTTP metrics (if `METRICS_USERNAME` is set)
|
||||
5. **CORS** — Cross-origin resource sharing headers
|
||||
6. **Timeout** — 60-second request timeout
|
||||
7. **Recoverer** — Panic recovery: one `ERROR` record through
|
||||
`internal/logger` and a `500`
|
||||
8. **Sentry** — Error reporting to Sentry (if `SENTRY_DSN` is set;
|
||||
configured with `Repanic: true` so panics still reach Recoverer)
|
||||
|
||||
Recoverer sits seventh rather than first, and both neighbours are the
|
||||
reason. It runs **inside** everything that observes the response, so
|
||||
the `500` it writes for a panicking handler is the status the access
|
||||
log records and the metrics count; registered first, as chi's own
|
||||
`middleware.Recoverer` was, the same request was logged as a `200` that
|
||||
the client never received. It runs **outside** the Sentry handler, so
|
||||
`Repanic: true` has something to re-raise into: an operator with
|
||||
`SENTRY_DSN` set keeps the report, and one without it now gets the
|
||||
local record instead of nothing. What that placement gives up is
|
||||
recovery of a panic in the six entries above it, none of which does
|
||||
more than set a header or start a timer.
|
||||
|
||||
Additionally, form endpoints (`/pages`, `/user/*`, `/sources`,
|
||||
`/source/*`) apply a **MaxBodySize** middleware that limits
|
||||
POST/PUT/PATCH request bodies to 1 MB. It is registered ahead of the
|
||||
|
||||
Reference in New Issue
Block a user