Bound the access log line against client-chosen text (closes #146)

The access log wrote one INFO line per request carrying
r.URL.String(). Registered with Use, it runs ahead of the route
limiter, so a client flooding the unauthenticated receiver with
invented paths wrote attacker-chosen text of attacker-chosen length
into the operator's log, one line per request.

3xx and 4xx responses now log the chi route pattern in place of the
concrete URL, and the fixed literal "(unmatched)" when routing matched
nothing at all. One line per request is retained, so real traffic
stays observable and rate accounting still works, but the line's
content is now bounded by the service's own route table. The pattern
is only populated after routing, so it is read in the deferred part of
the handler rather than before next.ServeHTTP.

The route pattern alone does not close the hole, because it leaves two
other ways for a request to choose the size of the line it writes.

The query string is one: /.well-known/healthcheck and /s/* answer 200
to anyone with no rate limiter in front of them, and /pages/login
behind only the login limiter, so appending 8 KB after the '?' bought
the same amplification as an invented 404 path. The branches that keep
the concrete URL now log the path only, with the query replaced by the
fixed marker "?(redacted)". Nothing debuggable is lost: `page`, on the
authenticated pagination links, is the only query parameter this
service reads.

The headers are the other: useragent and referer are logged on every
line, including the correctly redacted ones, so an 8 KB User-Agent
plus an 8 KB Referer produced a 24 KB line whose url field read
"(unmatched)". Each field a client supplies is now truncated rather
than dropped -- a truncated User-Agent is still worth reading -- to
512 bytes for url, useragent and referer, 128 for request_id (chi
passes an inbound X-Request-Id header straight through), and 32 for
method, which Go accepts as any token up to the header size limit.
Truncation also drops invalid UTF-8, which an encoder would otherwise
expand six-fold past the budget.

Each budget is spent in encoded bytes rather than in the bytes the
client sent, because the line an operator stores is the encoded one.
slog's JSON handler escapes a quotation mark, a backslash and a tab to
two bytes each and a non-printable rune to six; its text handler
spells any non-printable rune the same six-byte way; and Go's header
parser accepts all of them in a header value. Counted raw, a 512-byte
budget therefore bought a 1,024-byte field. Plain ASCII still encodes
one byte for one, so a real browser's User-Agent fits whole, while a
value built out of escapes keeps a proportionally shorter prefix.

A complete line is now at most 2,560 bytes: 3*(512+11) for url,
useragent and referer, 128+11 for request_id, 32+11 for method and a
336-byte fixed portion come to 2,087, stated with headroom. The tests
assert it against 8 KB in the path, in the query and in each of the
three headers, including values built from the characters the handler
escapes, and against a 5xx whose concrete url is at its own budget on
the same line. The README states it so an operator can size log
storage against it.

internal/middleware/middleware.go

@@ -6,9 +6,13 @@ import (
 	"log/slog"
 	"net"
 	"net/http"
+	"strings"
 	"time"
+	"unicode"
+	"unicode/utf8"
 
 	basicauth "github.com/99designs/basicauth-go"
+	"github.com/go-chi/chi"
 	"github.com/go-chi/chi/middleware"
 	"github.com/go-chi/cors"
 	metrics "github.com/slok/go-http-metrics/metrics/prometheus"
@@ -25,6 +29,71 @@ const (
 	// corsMaxAge is the maximum time (in seconds) that a
 	// preflight response can be cached.
 	corsMaxAge = 300
+
+	// unmatchedRoute is logged in the access log's url field when a
+	// redirected or rejected request matched no route pattern at
+	// all. Every byte of such a path is client-chosen, so none of it
+	// is logged.
+	unmatchedRoute = "(unmatched)"
+
+	// redactedQuery stands in for the query string on the access log
+	// branches that keep the concrete URL. The query is client-chosen
+	// on every route, including the ones that answer an
+	// unauthenticated 200, so logging it verbatim would let a client
+	// pick the size of the line it writes.
+	redactedQuery = "?(redacted)"
+
+	// maxLogFieldBytes bounds each access log field whose value the
+	// client supplies outright: the URL, the User-Agent and the
+	// Referer. The budget is spent in ENCODED bytes (see
+	// truncateLogField), so 512 still holds a real browser's User-Agent
+	// whole — those are plain ASCII, which encodes one byte for one —
+	// while a value built from characters the encoder escapes keeps a
+	// shorter prefix. That is the intended trade: 500 quotation marks
+	// are not a debugging asset.
+	maxLogFieldBytes = 512
+
+	// maxLogRequestIDBytes bounds the request id, which is also
+	// client-supplied: chi's RequestID middleware passes an inbound
+	// X-Request-Id header through verbatim. Its generated form is an
+	// order of magnitude shorter than this.
+	maxLogRequestIDBytes = 128
+
+	// maxLogMethodBytes bounds the method. Go accepts any RFC 7230
+	// token there, bounded only by the header size limit, so it is
+	// client-chosen text like the rest. The longest registered method
+	// is half this.
+	maxLogMethodBytes = 32
+
+	// truncationMarker is appended to any field the access log cut, so
+	// a short value and a truncated one cannot be confused. It is
+	// charged on top of the budget, not inside it.
+	truncationMarker = "[truncated]"
+
+	// MaxAccessLogLineBytes is the ceiling on one JSON access log line,
+	// and the number an operator multiplies by the request rate to size
+	// log storage. It is not an observation of a sample: it is the sum
+	// of the budgets above, each of which truncateLogField enforces in
+	// ENCODED bytes, plus the part of the line no client can influence.
+	//
+	//	url, useragent, referer   3*(512+11) = 1569
+	//	request_id                    128+11 =  139
+	//	method                         32+11 =   43
+	//	fixed portion                        =  336
+	//	                                        ----
+	//	                                        2087
+	//
+	// The fixed portion is the JSON punctuation, the field names, the
+	// level and the message, both timestamps at their longest, an IPv6
+	// remoteIP with a zone, a three-digit status and a full-width int64
+	// latency. Stated at 2560 so the figure carries headroom rather
+	// than sitting on the arithmetic.
+	//
+	// The tty text handler in internal/logger is covered by the same
+	// figure: encodedLogFieldBytes charges the worse of the two
+	// handlers' escapes, and the text handler's fixed portion is the
+	// smaller of the two.
+	MaxAccessLogLineBytes = 2560
 )
 
 //nolint:revive // MiddlewareParams is a standard fx naming convention.
@@ -94,6 +163,161 @@ func (lrw *loggingResponseWriter) WriteHeader(code int) {
 	lrw.ResponseWriter.WriteHeader(code)
 }
 
+// encodedLogFieldBytes is what r costs on the line once the log
+// handler has escaped it, taking the worse of the two handlers
+// internal/logger configures.
+//
+// slog's JSON handler escapes quote, backslash, newline, carriage
+// return and tab to two bytes each, and every other C0 control plus
+// LINE SEPARATOR and PARAGRAPH SEPARATOR to a six-byte \u escape. Its
+// text handler quotes with strconv.Quote, which spells any
+// non-printable rune the same six-byte way. Both pass printable runes
+// through as their own UTF-8, so unicode.IsPrint separates the two
+// cases for either handler. Go's header parser accepts quote,
+// backslash, tab and non-printable multi-byte runes in a header value,
+// so every one of these is reachable from a request.
+func encodedLogFieldBytes(r rune) int {
+	const (
+		// A backslash and the character itself.
+		shortEscapeBytes = 2
+		// \uXXXX, which is also the width of \u00XX.
+		escapedRuneBytes = 6
+	)
+
+	switch {
+	case r == '"' || r == '\\' || r == '\n' || r == '\r' || r == '\t':
+		return shortEscapeBytes
+	case !unicode.IsPrint(r):
+		return escapedRuneBytes
+	default:
+		return utf8.RuneLen(r)
+	}
+}
+
+// truncateLogField caps s at maxBytes of ENCODED output, marking the
+// value when it cuts.
+//
+// Budgeting raw bytes would not bound the line. Escaping only ever
+// grows a value, so a raw budget spent on characters the encoder
+// escapes buys a field several times its nominal size — and the line
+// is the thing an operator is told to multiply by their request rate.
+// Charging each rune what it will actually cost is what makes
+// MaxAccessLogLineBytes true rather than merely larger. The visible
+// consequence is that an escape-heavy value keeps a shorter prefix
+// than a plain one, which is the correct trade.
+//
+// The result is always valid UTF-8. A cut on a byte boundary can split
+// a multi-byte rune, and a header can carry bytes that were never
+// valid UTF-8 to begin with; both are dropped rather than kept, since
+// an encoder would otherwise spend six bytes replacing each one.
+func truncateLogField(s string, maxBytes int) string {
+	// No rune encodes to fewer bytes than it occupies, so nothing past
+	// maxBytes raw can fit the budget. Slicing first bounds the scan
+	// below to the budget rather than to the size of the header the
+	// client sent.
+	window, cut := s, false
+	if len(window) > maxBytes {
+		window, cut = window[:maxBytes], true
+	}
+
+	var (
+		kept  strings.Builder
+		spent int
+	)
+
+	for i := 0; i < len(window); {
+		r, size := utf8.DecodeRuneInString(window[i:])
+		if r == utf8.RuneError && size == 1 {
+			i += size
+
+			continue
+		}
+
+		cost := encodedLogFieldBytes(r)
+		if spent+cost > maxBytes {
+			cut = true
+
+			break
+		}
+
+		spent += cost
+
+		kept.WriteString(window[i : i+size])
+
+		i += size
+	}
+
+	if !cut {
+		return kept.String()
+	}
+
+	return kept.String() + truncationMarker
+}
+
+// concreteLogURL renders the request's own URL for the access log
+// branches that keep it, with the query string replaced by a fixed
+// marker.
+//
+// The path on those branches is bounded by the service's routes or by
+// the operator's data — a 2xx on the receiver means the UUID named a
+// stored entrypoint, a 2xx under /s means the file is in the embedded
+// tree. The query is not bounded by anything: /.well-known/healthcheck
+// and /s/* take no authentication and sit behind no rate limiter, and
+// /pages/login behind only the login limiter, so any of them will
+// answer 200 to a URL carrying an arbitrary number of arbitrary bytes
+// after the '?'. Keeping the path and dropping the query is what makes
+// this branch as bounded as the pattern branches below.
+//
+// Nothing debuggable is lost. One route in the service reads a query
+// parameter at all — `page`, on the authenticated pagination links in
+// internal/handlers/source_management.go — and the alternatives that
+// would preserve more (a key count, a key allowlist) all require
+// parsing an attacker-sized query on every request, which is work an
+// unauthenticated client would then be choosing for us.
+func concreteLogURL(r *http.Request) string {
+	path := r.URL.EscapedPath()
+
+	if r.URL.RawQuery == "" && !r.URL.ForceQuery {
+		return path
+	}
+
+	return path + redactedQuery
+}
+
+// accessLogURL returns the value for the access log's url field.
+//
+// 2xx and 5xx responses get the concrete path (see concreteLogURL). A
+// success resolved against a static route or against the operator's
+// own data — on the receiver, a 2xx means the UUID named a stored
+// entrypoint — and a server error is our own bug, where the exact URL
+// is the primary evidence and which no client can provoke at will.
+//
+// 3xx and 4xx responses get the chi route pattern instead. Those are
+// the outcomes an unauthenticated client drives for free: 404 or 429
+// on any invented /webhook/ path, 303 to the login page on any
+// invented /user/ path. Logging the concrete URL there lets a flood
+// write attacker-chosen text, of attacker-chosen length, into the
+// operator's log at one line per request. The pattern comes from the
+// router's own table, so it is bounded by the service's routes while
+// still naming which class of request was rejected.
+//
+// The pattern is only populated once routing has run, so this must be
+// called after the handler returns, not before.
+func accessLogURL(r *http.Request, status int) string {
+	if status < http.StatusMultipleChoices ||
+		status >= http.StatusInternalServerError {
+		return concreteLogURL(r)
+	}
+
+	if rc := chi.RouteContext(r.Context()); rc != nil {
+		if pattern := rc.RoutePattern(); pattern != "" {
+			return pattern
+		}
+	}
+
+	return unmatchedRoute
+}
+
 // Logging returns middleware that logs each HTTP request with
 // timing and metadata.
 func (s *Middleware) Logging() func(http.Handler) http.Handler {
@@ -118,13 +342,27 @@ func (s *Middleware) Logging() func(http.Handler) http.Handler {
 					}
 				}
 
+				// Every field below that a client can influence is
+				// truncated to a fixed budget, so the size of this
+				// line does not track the size of the request.
 				s.log.Info("http request",
 					"request_start", start,
-					"method", r.Method,
-					"url", r.URL.String(),
-					"useragent", r.UserAgent(),
-					"request_id", requestID,
-					"referer", r.Referer(),
+					"method", truncateLogField(
+						r.Method, maxLogMethodBytes,
+					),
+					"url", truncateLogField(
+						accessLogURL(r, lrw.statusCode),
+						maxLogFieldBytes,
+					),
+					"useragent", truncateLogField(
+						r.UserAgent(), maxLogFieldBytes,
+					),
+					"request_id", truncateLogField(
+						requestID, maxLogRequestIDBytes,
+					),
+					"referer", truncateLogField(
+						r.Referer(), maxLogFieldBytes,
+					),
 					"proto", r.Proto,
 					"remoteIP", ipFromHostPort(r.RemoteAddr),
 					"status", lrw.statusCode,