check / check (push) Successful in 3m17s
The access log's response writer and the metrics middleware's writer hid the writer beneath them, so a handler's flush, hijack or write deadline set through http.ResponseController failed with "not supported" behind them. The access log's writer now has Unwrap. The metrics middleware calls the library's public Measure with a writer of our own that has Unwrap, in place of std.Handler's writer, so the middleware order and what metrics record are unchanged. A test over a real connection sets a write deadline and flushes, metrics on and off, on a global route and in an admin page group, and fails without either Unwrap. Model: opus-5-5
240 lines
8.4 KiB
Go
240 lines
8.4 KiB
Go
package middleware
|
|
|
|
import (
|
|
"context"
|
|
"net/http"
|
|
"time"
|
|
|
|
"github.com/go-chi/chi"
|
|
httpmetrics "github.com/slok/go-http-metrics/metrics"
|
|
ghmm "github.com/slok/go-http-metrics/middleware"
|
|
)
|
|
|
|
// inflightHandler is the fixed `handler` label on
|
|
// http_requests_inflight, the one HTTP metric here that cannot carry
|
|
// a route pattern.
|
|
//
|
|
// The gauge is incremented before the wrapped handler runs and
|
|
// decremented after it returns, and the pattern only exists between
|
|
// those two moments. Deriving the label from the route would
|
|
// therefore increment one series and decrement another, leaving every
|
|
// pattern permanently off by the number of requests it served — a
|
|
// broken gauge, on top of the per-path cardinality this file exists
|
|
// to remove. So the gauge is deliberately aggregate: one series,
|
|
// counting the requests in flight across the whole service.
|
|
const inflightHandler = "(all)"
|
|
|
|
// unmatchedMethod is the `method` label for a request whose method
|
|
// the router can never route.
|
|
//
|
|
// It is deliberately the same sentinel as unmatchedRoute rather than
|
|
// a spelling of its own: both stand for a client-chosen token that
|
|
// matched nothing this service registers, and giving one idea two
|
|
// spellings would read in a scrape as two different unmatched states.
|
|
const unmatchedMethod = unmatchedRoute
|
|
|
|
// routePatternID is the `handler` label for a request: the chi route
|
|
// pattern, never the concrete path.
|
|
//
|
|
// The pattern is what bounds the label's domain to the routes the
|
|
// service registers. The path does not bound it at all — every byte
|
|
// after /h/ is client-chosen, so labelling by path lets any
|
|
// unauthenticated client mint permanent series at will, and publishes
|
|
// the entrypoint UUID (the receiver's only credential) in the scrape
|
|
// while doing it.
|
|
//
|
|
// chi populates the route context during routeHTTP, so this is only
|
|
// valid once routing has run. Every caller below is on the recording
|
|
// side of the middleware, which go-http-metrics defers until after
|
|
// the wrapped handler returns.
|
|
func routePatternID(ctx context.Context) string {
|
|
if rc := chi.RouteContext(ctx); rc != nil {
|
|
if pattern := rc.RoutePattern(); pattern != "" {
|
|
return pattern
|
|
}
|
|
}
|
|
|
|
return unmatchedRoute
|
|
}
|
|
|
|
// methodID is the `method` label for a request: the request method
|
|
// when the router can route it, and the unmatched sentinel otherwise.
|
|
//
|
|
// net/http accepts any RFC 9110 token as a method and hands it
|
|
// through verbatim, so the raw method is client-chosen bytes and
|
|
// bounds the label at nothing — the same unauthenticated
|
|
// series-minting the handler label carried, reached through a second
|
|
// dimension. What bounds it is the set chi's router will match a
|
|
// route for: its methodMap, which is unexported, so it is restated
|
|
// here against the net/http constants it is built from. A token
|
|
// outside that set can only ever produce chi's 405, so folding every
|
|
// one of them onto a single series loses no information a scrape
|
|
// could have used, while the nine methods that can reach a handler
|
|
// stay distinguishable.
|
|
//
|
|
// chi.RegisterMethod would extend the router's set at runtime; this
|
|
// service never calls it, and a caller that started to would have to
|
|
// extend this switch with it.
|
|
func methodID(method string) string {
|
|
switch method {
|
|
case http.MethodConnect,
|
|
http.MethodDelete,
|
|
http.MethodGet,
|
|
http.MethodHead,
|
|
http.MethodOptions,
|
|
http.MethodPatch,
|
|
http.MethodPost,
|
|
http.MethodPut,
|
|
http.MethodTrace:
|
|
return method
|
|
default:
|
|
return unmatchedMethod
|
|
}
|
|
}
|
|
|
|
// boundedLabelRecorder wraps a go-http-metrics recorder and replaces
|
|
// the request-controlled labels on every observation with bounded
|
|
// ones: the handler id becomes the request's route pattern, and the
|
|
// method becomes one the router can route.
|
|
//
|
|
// This is the seam that makes the pattern usable at all. The metrics
|
|
// middleware is global (see Server.setupGlobalMiddleware), so it is
|
|
// entered before chi has matched anything, and go-http-metrics fixes
|
|
// its handler id up front — passing the pattern in as that id is not
|
|
// possible, and leaving the id empty makes the library substitute the
|
|
// raw URL path, which is the defect. What the library does hand over
|
|
// is the request context, unchanged, on each recorder call; that
|
|
// context carries the same *chi.Context pointer routing mutates in
|
|
// place, and the duration and size calls happen after the wrapped
|
|
// handler has returned. Reading the pattern there is what the access
|
|
// log already does in accessLogURL.
|
|
//
|
|
// Recording after the whole chain returns is also what makes this
|
|
// hold for requests the route-level receiver rate limiter rejects.
|
|
// Those never reach a handler, but chi has already matched the route
|
|
// by the time the limiter runs, so their 429s land on the pattern
|
|
// like any other response.
|
|
type boundedLabelRecorder struct {
|
|
inner httpmetrics.Recorder
|
|
}
|
|
|
|
func (r boundedLabelRecorder) ObserveHTTPRequestDuration(
|
|
ctx context.Context,
|
|
props httpmetrics.HTTPReqProperties,
|
|
duration time.Duration,
|
|
) {
|
|
props.ID = routePatternID(ctx)
|
|
props.Method = methodID(props.Method)
|
|
r.inner.ObserveHTTPRequestDuration(ctx, props, duration)
|
|
}
|
|
|
|
func (r boundedLabelRecorder) ObserveHTTPResponseSize(
|
|
ctx context.Context,
|
|
props httpmetrics.HTTPReqProperties,
|
|
sizeBytes int64,
|
|
) {
|
|
props.ID = routePatternID(ctx)
|
|
props.Method = methodID(props.Method)
|
|
r.inner.ObserveHTTPResponseSize(ctx, props, sizeBytes)
|
|
}
|
|
|
|
func (r boundedLabelRecorder) AddInflightRequests(
|
|
ctx context.Context,
|
|
props httpmetrics.HTTPProperties,
|
|
quantity int,
|
|
) {
|
|
props.ID = inflightHandler
|
|
r.inner.AddInflightRequests(ctx, props, quantity)
|
|
}
|
|
|
|
var _ httpmetrics.Recorder = boundedLabelRecorder{}
|
|
|
|
// Metrics returns middleware that records Prometheus HTTP metrics
|
|
// with the Middleware's one recorder, which New builds on the registry
|
|
// the /metrics route serves and NewForTest on a registry of its own.
|
|
// Every call reuses that recorder, so any number of routers can
|
|
// install it.
|
|
func (s *Middleware) Metrics() func(http.Handler) http.Handler {
|
|
return metricsMiddleware(s.metricsRecorder)
|
|
}
|
|
|
|
// metricsMiddleware builds the recording middleware against a given
|
|
// recorder, so tests can gather from a registry of their own.
|
|
func metricsMiddleware(
|
|
rec httpmetrics.Recorder,
|
|
) func(http.Handler) http.Handler {
|
|
mdlw := ghmm.New(ghmm.Config{
|
|
Recorder: boundedLabelRecorder{inner: rec},
|
|
})
|
|
|
|
return func(next http.Handler) http.Handler {
|
|
return http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
|
mw := &metricsResponseWriter{
|
|
ResponseWriter: w,
|
|
request: r,
|
|
statusCode: http.StatusOK,
|
|
}
|
|
|
|
// The handler id is unmatchedRoute rather than "" so
|
|
// that the client-chosen URL path never enters the
|
|
// metrics pipeline at all: an empty id is the library's
|
|
// signal to substitute it. boundedLabelRecorder
|
|
// overwrites this value on every observation, so it is
|
|
// reachable only if that decorator is removed — in which
|
|
// case the metrics collapse to one series instead of
|
|
// leaking again.
|
|
mdlw.Measure(unmatchedRoute, mw, func() {
|
|
next.ServeHTTP(mw, r)
|
|
})
|
|
})
|
|
}
|
|
}
|
|
|
|
// metricsResponseWriter records the status code and body size of a
|
|
// response, and hands them with the request to go-http-metrics'
|
|
// Measure as its Reporter.
|
|
//
|
|
// It stands in for the library's std.Handler, whose writer has no
|
|
// Unwrap: behind it, http.ResponseController cannot reach net/http's
|
|
// own writer, so a handler's write deadline fails with metrics on.
|
|
type metricsResponseWriter struct {
|
|
http.ResponseWriter
|
|
|
|
request *http.Request
|
|
statusCode int
|
|
bytesWritten int64
|
|
}
|
|
|
|
func (w *metricsResponseWriter) WriteHeader(code int) {
|
|
w.statusCode = code
|
|
|
|
w.ResponseWriter.WriteHeader(code)
|
|
}
|
|
|
|
func (w *metricsResponseWriter) Write(b []byte) (int, error) {
|
|
w.bytesWritten += int64(len(b))
|
|
|
|
//nolint:wrapcheck // Pass the writer's own error through unchanged.
|
|
return w.ResponseWriter.Write(b)
|
|
}
|
|
|
|
// Unwrap lets http.ResponseController reach the writer underneath, so
|
|
// a handler can still flush or set a write deadline with metrics on.
|
|
func (w *metricsResponseWriter) Unwrap() http.ResponseWriter {
|
|
return w.ResponseWriter
|
|
}
|
|
|
|
func (w *metricsResponseWriter) Method() string { return w.request.Method }
|
|
|
|
func (w *metricsResponseWriter) Context() context.Context {
|
|
return w.request.Context()
|
|
}
|
|
|
|
func (w *metricsResponseWriter) URLPath() string { return w.request.URL.Path }
|
|
|
|
func (w *metricsResponseWriter) StatusCode() int { return w.statusCode }
|
|
|
|
func (w *metricsResponseWriter) BytesWritten() int64 { return w.bytesWritten }
|
|
|
|
var _ ghmm.Reporter = (*metricsResponseWriter)(nil)
|