Log the rest of the request log's fields #82

Merged
clawbot merged 1 commits from issue-79-log-fields into next 2026-10-06 17:26:24 +02:00
Collaborator

Request log lines gain the fields "Request log" in SPEC.md lists whose features are built, described in README.md, for #79.

  • New settings SWWAF_INSTANCE_NAME (default: the host's name) and SWWAF_LOG_REQUEST_HEADERS. An entry of the latter that is not a header name, or is Host or Transfer-Encoding (Go's server takes both out of the request), stops the start.
  • request_id is a trusted proxy's X-Request-ID or a new random one, sent on to the app; an untrusted peer's is replaced. scheme believes X-Forwarded-Proto only from a trusted proxy.
  • Authorization, Cookie and Set-Cookie values are never logged; has_authorization and has_cookie say whether the first two were sent.
  • The health answer sets its own Content-Type, which Go's server would add after the log line.
  • Optional timings are pointers in requestlog.Line, so one under a microsecond is logged as zero.
  • ratelimit.Limiter.Count also returns the client's counts in each window.
  • requestlog.Line's fields follow SPEC.md's order.

Left for their features: asn, as_name, limit_percent, rule_ids, waf_rule_ids, waf_score, reputation, duration_waf.

Deviation: counts has request totals only; byte totals come with the byte limits.
Deviation: SWWAF_INSTANCE_NAME is on request lines only, not process lines or metrics.
Judgement call: the headers go under request_headers, a name SPEC.md does not give.
Judgement call: the issue's SWWAF_INSTANCE takes SPEC.md's name, SWWAF_INSTANCE_NAME.
Judgement call: request ids are 26 base32 characters from crypto/rand.Text; the standard library's uuid needs go 1.27.
Judgement call: header names are checked against RFC 9110's characters in a few lines, not with golang.org/x/net, which SPEC.md does not list.

Model: opus-5-5

Request log lines gain the fields "Request log" in `SPEC.md` lists whose features are built, described in `README.md`, for https://git.eeqj.de/sneak/smallwebwaf/issues/79. - New settings `SWWAF_INSTANCE_NAME` (default: the host's name) and `SWWAF_LOG_REQUEST_HEADERS`. An entry of the latter that is not a header name, or is `Host` or `Transfer-Encoding` (Go's server takes both out of the request), stops the start. - `request_id` is a trusted proxy's `X-Request-ID` or a new random one, sent on to the app; an untrusted peer's is replaced. `scheme` believes `X-Forwarded-Proto` only from a trusted proxy. - `Authorization`, `Cookie` and `Set-Cookie` values are never logged; `has_authorization` and `has_cookie` say whether the first two were sent. - The health answer sets its own `Content-Type`, which Go's server would add after the log line. - Optional timings are pointers in `requestlog.Line`, so one under a microsecond is logged as zero. - `ratelimit.Limiter.Count` also returns the client's counts in each window. - `requestlog.Line`'s fields follow `SPEC.md`'s order. Left for their features: `asn`, `as_name`, `limit_percent`, `rule_ids`, `waf_rule_ids`, `waf_score`, `reputation`, `duration_waf`. Deviation: `counts` has request totals only; byte totals come with the byte limits. Deviation: `SWWAF_INSTANCE_NAME` is on request lines only, not process lines or metrics. Judgement call: the headers go under `request_headers`, a name `SPEC.md` does not give. Judgement call: the issue's `SWWAF_INSTANCE` takes `SPEC.md`'s name, `SWWAF_INSTANCE_NAME`. Judgement call: request ids are 26 base32 characters from `crypto/rand.Text`; the standard library's `uuid` needs `go 1.27`. Judgement call: header names are checked against RFC 9110's characters in a few lines, not with `golang.org/x/net`, which `SPEC.md` does not list. Model: opus-5-5
clawbot added the needs-review label 2026-10-06 13:56:34 +02:00
clawbot self-assigned this 2026-10-06 13:56:34 +02:00
Author
Collaborator

Review failed.

  1. README.md conflicts with current next: the status paragraph and the TODO list were both changed by the commit for #68. Acceptable: rebased onto next with both changes kept, and the TODO no longer naming the rest of the request log's fields.
  2. internal/config/config.go, headerNames: SWWAF_LOG_REQUEST_HEADERS takes entries that are not header names, such as accept;origin, accept language or x-foo:. They can never match a request header, so the setting quietly logs nothing for them, where SPEC.md has invalid configuration stop the start. Acceptable: an entry that is not a valid header name stops the start with a message naming SWWAF_LOG_REQUEST_HEADERS, with such cases in TestInvalidValueStopsTheStart.
  3. internal/proxy/proxy.go, the health answer: its log line has no response_content_type, although the client receives Content-Type: text/plain; charset=utf-8, which Go's server adds after the handler has run. README.md says the field gives the Content-Type of smallwebwaf's own answers. Acceptable: the health answer sets its Content-Type itself, and a test checks the health line's response_content_type.

The PR's disclosed deviations and judgement calls are accepted.

Model: opus-5-5

Review failed. 1. `README.md` conflicts with current `next`: the status paragraph and the TODO list were both changed by the commit for https://git.eeqj.de/sneak/smallwebwaf/issues/68. Acceptable: rebased onto `next` with both changes kept, and the TODO no longer naming the rest of the request log's fields. 2. `internal/config/config.go`, `headerNames`: `SWWAF_LOG_REQUEST_HEADERS` takes entries that are not header names, such as `accept;origin`, `accept language` or `x-foo:`. They can never match a request header, so the setting quietly logs nothing for them, where `SPEC.md` has invalid configuration stop the start. Acceptable: an entry that is not a valid header name stops the start with a message naming `SWWAF_LOG_REQUEST_HEADERS`, with such cases in `TestInvalidValueStopsTheStart`. 3. `internal/proxy/proxy.go`, the health answer: its log line has no `response_content_type`, although the client receives `Content-Type: text/plain; charset=utf-8`, which Go's server adds after the handler has run. `README.md` says the field gives the Content-Type of `smallwebwaf`'s own answers. Acceptable: the health answer sets its `Content-Type` itself, and a test checks the health line's `response_content_type`. The PR's disclosed deviations and judgement calls are accepted. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 14:25:21 +02:00
clawbot force-pushed issue-79-log-fields from f61d821215 to 7e3aa6a5bb 2026-10-06 14:46:48 +02:00 Compare
Author
Collaborator
  1. Rebased onto next with both changes to README.md kept; the TODO now names only exemptions.
  2. An entry of SWWAF_LOG_REQUEST_HEADERS with a character RFC 9110 does not allow in a header name stops the start with a message naming the setting; TestInvalidValueStopsTheStart has accept;origin, accept language and x-foo:.
  3. The health answer sets Content-Type: text/plain; charset=utf-8 itself, and TestHealthEndpointIsAnsweredBeforeAnyCheck checks the health line's response_content_type.

Model: opus-5-5

1. Rebased onto `next` with both changes to `README.md` kept; the TODO now names only exemptions. 2. An entry of `SWWAF_LOG_REQUEST_HEADERS` with a character RFC 9110 does not allow in a header name stops the start with a message naming the setting; `TestInvalidValueStopsTheStart` has `accept;origin`, `accept language` and `x-foo:`. 3. The health answer sets `Content-Type: text/plain; charset=utf-8` itself, and `TestHealthEndpointIsAnsweredBeforeAnyCheck` checks the health line's `response_content_type`. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-06 14:47:03 +02:00
Author
Collaborator

Review failed.

  1. internal/proxy/request.go, requestHeaders, and README.md, "Request log": host or transfer-encoding in SWWAF_LOG_REQUEST_HEADERS is never logged, because Go's server takes those two headers out of the request before smallwebwaf sees it. The setting quietly logs nothing for them, while the README says request_headers holds the request's headers that the setting names. Acceptable: they are logged from where Go keeps them, or such an entry stops the start with a message naming the setting, either with a test; or the README says they are never among request_headers.
  2. The PR body has grown to about 280 words with the rework, over the limit of about 250. Acceptable: trimmed to about 250 words, its disclosures kept.

Model: opus-5-5

Review failed. 1. `internal/proxy/request.go`, `requestHeaders`, and `README.md`, "Request log": `host` or `transfer-encoding` in `SWWAF_LOG_REQUEST_HEADERS` is never logged, because Go's server takes those two headers out of the request before `smallwebwaf` sees it. The setting quietly logs nothing for them, while the README says `request_headers` holds the request's headers that the setting names. Acceptable: they are logged from where Go keeps them, or such an entry stops the start with a message naming the setting, either with a test; or the README says they are never among `request_headers`. 2. The PR body has grown to about 280 words with the rework, over the limit of about 250. Acceptable: trimmed to about 250 words, its disclosures kept. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 14:59:39 +02:00
clawbot force-pushed issue-79-log-fields from 7e3aa6a5bb to ea65587930 2026-10-06 16:01:31 +02:00 Compare
Author
Collaborator
  1. An entry of SWWAF_LOG_REQUEST_HEADERS naming Host or Transfer-Encoding, in any case, stops the start with a message naming the setting and saying why; TestInvalidValueStopsTheStart has such cases, and README.md says so.
  2. The PR body is trimmed to about 250 words, its disclosures kept.

Judgement call: the three SWWAF_LISTEN_ADDR cases of TestInvalidValueStopsTheStart now share one line, to keep the test within the lint's function length.

Model: opus-5-5

1. An entry of `SWWAF_LOG_REQUEST_HEADERS` naming `Host` or `Transfer-Encoding`, in any case, stops the start with a message naming the setting and saying why; `TestInvalidValueStopsTheStart` has such cases, and `README.md` says so. 2. The PR body is trimmed to about 250 words, its disclosures kept. Judgement call: the three `SWWAF_LISTEN_ADDR` cases of `TestInvalidValueStopsTheStart` now share one line, to keep the test within the lint's function length. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-06 16:01:52 +02:00
Author
Collaborator

Review failed.

  1. internal/proxy/requestlog_test.go, TestLogLineHasEachFieldWhereItApplies: its requests carry both Authorization and Cookie or neither, so has_authorization taken from Cookie and has_cookie taken from Authorization would pass every test. Acceptable: a request carrying only one of the two headers, whose line has that header's field and not the other's.
  2. internal/config/config.go, errHeaderTakenOut: transfer-encoding in SWWAF_LOG_REQUEST_HEADERS stops the start with a message ending "the request's host is the field host", which has nothing to do with that header. Acceptable: that hint given only for Host, or left out.

Judgement call accepted: the three SWWAF_LISTEN_ADDR cases sharing one line of TestInvalidValueStopsTheStart, as other cases in that table already do.

Model: opus-5-5

Review failed. 1. `internal/proxy/requestlog_test.go`, `TestLogLineHasEachFieldWhereItApplies`: its requests carry both `Authorization` and `Cookie` or neither, so `has_authorization` taken from `Cookie` and `has_cookie` taken from `Authorization` would pass every test. Acceptable: a request carrying only one of the two headers, whose line has that header's field and not the other's. 2. `internal/config/config.go`, `errHeaderTakenOut`: `transfer-encoding` in `SWWAF_LOG_REQUEST_HEADERS` stops the start with a message ending "the request's host is the field host", which has nothing to do with that header. Acceptable: that hint given only for `Host`, or left out. Judgement call accepted: the three `SWWAF_LISTEN_ADDR` cases sharing one line of `TestInvalidValueStopsTheStart`, as other cases in that table already do. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 16:14:47 +02:00
clawbot added 1 commit 2026-10-06 16:28:56 +02:00
Log the rest of the request log's fields (closes #79)
check / check (push) Successful in 2m57s
8582d7c5ed
Each request log line now has the fields "Request log" in SPEC.md lists
whose features are built: instance (SWWAF_INSTANCE_NAME), scheme,
request_id (a trusted proxy's X-Request-ID or a new one, sent on to the
app), forwarded_for, client_group, content_type, content_length, the
headers SWWAF_LOG_REQUEST_HEADERS names, has_authorization, has_cookie,
websocket, response_content_type, cache_control, location, counts and
the timings. Authorization, Cookie and Set-Cookie values are never
logged. An entry of SWWAF_LOG_REQUEST_HEADERS that is not a header name,
or is Host or Transfer-Encoding, stops the start.

Deviation: counts has request totals only.
Deviation: SWWAF_INSTANCE_NAME is on request lines only.

Model: opus-5-5
clawbot force-pushed issue-79-log-fields from ea65587930 to 8582d7c5ed 2026-10-06 16:28:56 +02:00 Compare
Author
Collaborator
  1. Fixed: a request carrying only Authorization and one carrying only Cookie are now tested, and the test fails with the two fields swapped.
  2. Fixed: only Host gets the hint about the field host, and both messages are tested.
    Same kind, also fixed: the log line test now sends a body without Content-Length, so content_length and request_bytes no longer always match; the counts test now has different minute, hour and day values; and one failure message printed a timing's memory address instead of its value.

Model: opus-5-5

1. Fixed: a request carrying only `Authorization` and one carrying only `Cookie` are now tested, and the test fails with the two fields swapped. 2. Fixed: only `Host` gets the hint about the field `host`, and both messages are tested. Same kind, also fixed: the log line test now sends a body without `Content-Length`, so `content_length` and `request_bytes` no longer always match; the counts test now has different minute, hour and day values; and one failure message printed a timing's memory address instead of its value. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-06 16:29:09 +02:00
Author
Collaborator

Review passed.

Judgement call accepted: counts has request totals only; the byte totals come with the byte limits.
Judgement call accepted: SWWAF_INSTANCE_NAME is on request lines only for now, not yet on process lines or metrics.
Judgement call accepted: the headers SWWAF_LOG_REQUEST_HEADERS names are logged under request_headers.
Judgement call accepted: the setting takes SPEC.md's name, SWWAF_INSTANCE_NAME, rather than the issue's SWWAF_INSTANCE.
Judgement call accepted: request ids come from crypto/rand.Text.
Judgement call accepted: header names are checked against RFC 9110's characters without golang.org/x/net.
Judgement call accepted: a trusted proxy's X-Request-ID is bounded only by SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES, as user_agent is; Go's server already refuses control characters in it.

Model: opus-5-5

Review passed. Judgement call accepted: `counts` has request totals only; the byte totals come with the byte limits. Judgement call accepted: `SWWAF_INSTANCE_NAME` is on request lines only for now, not yet on process lines or metrics. Judgement call accepted: the headers `SWWAF_LOG_REQUEST_HEADERS` names are logged under `request_headers`. Judgement call accepted: the setting takes `SPEC.md`'s name, `SWWAF_INSTANCE_NAME`, rather than the issue's `SWWAF_INSTANCE`. Judgement call accepted: request ids come from `crypto/rand.Text`. Judgement call accepted: header names are checked against RFC 9110's characters without `golang.org/x/net`. Judgement call accepted: a trusted proxy's `X-Request-ID` is bounded only by `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`, as `user_agent` is; Go's server already refuses control characters in it. Model: opus-5-5
clawbot merged commit 808e69f442 into next 2026-10-06 17:26:24 +02:00
clawbot deleted branch issue-79-log-fields 2026-10-06 17:26:24 +02:00
Sign in to join this conversation.