Files
smallwebwaf/docs/request-log.md
T
sneak fbb5158377
check / check (push) In progress
rproxy, the proxy in front of each app, asks servewaf and fails open (closes #171)
`smallwebwaf rproxy` reads the first 64 KiB of each body, asks servewaf, and
passes what servewaf allows to the app within its own size and time limits.
It reports each request servewaf answered. SWRP_DEFAULT_DISPOSITION decides
when servewaf gives no answer.

Duplication: internal/rproxy copies passing-on code from internal/proxy until
#172 removes the originals.
Rule suppressed: gosec G704 on servewaf's URL, a setting.
Rule suppressed: gosec G705 on the answer written to the visitor.
Judgement call: the SWRP_ size and time limits take off; SWRP_TIMEOUT_MS does not.

Model: opus-5-5
2026-11-05 00:08:53 +00:00

9.7 KiB

The request log's fields

The fields of the JSON object smallwebwaf writes for each request. "Request log" in SPEC.md says when a line is written and how the log is sent to a syslog server. A field that does not apply to a request is left out. smallwebwaf rproxy writes a shorter line, which "rproxy's line" below gives.

Example

{"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104}

Every line

Every line but rproxy's has:

  • type: request (the process's own messages have process);
  • the standard web log fields;
  • request_id, peer_ip and client_group;
  • asn, as_name and country;
  • action and duration_total.

Standard web log fields

  • time: when the request arrived, RFC 3339 with milliseconds, in UTC;
  • instance: SWWAF_INSTANCE_NAME;
  • client_ip, method, scheme, host, path, query, protocol;
  • status: what the client was sent, 0 if nothing;
  • request_bytes and response_bytes: body bytes each way;
  • referer and user_agent.

Request detail

  • request_id: the X-Request-ID a trusted proxy sent, or a new random one of 26 letters and digits.
  • peer_ip: the TCP peer, normally traefik.
  • forwarded_for: the X-Forwarded-For header as received.
  • client_group: the client as rate limiting counts it, an IPv4 address as a /32 or an IPv6 group.
  • asn, as_name and country: the client's, as looked up. Each is empty when:
    • lookups are off;
    • the client is not looked up;
    • the source cannot place the client;
    • GeoJS had not answered by the time the request went on.
  • content_type and content_length: the request's own.
  • request_headers: the headers listed in SWWAF_LOG_REQUEST_HEADERS, by name in lower case.
  • has_authorization and has_cookie: true when the request had those headers. Their values are never logged.
  • websocket: true when the app switched the connection to another protocol.

Response detail

  • upstream_status: what the app answered. It differs from status when smallwebwaf answered itself.
  • response_content_type, cache_control and location: the answer's headers.
  • aborted: true when the client went away early.

Decision

  • action: what was done:

    • forward: passed to the app;
    • denied: refused for SWWAF_DENY_NETS, or for a reputation source with the action deny;
    • banned: refused under a ban, or for a ban rule, a trap path or a CrowdSec decision, each of which bans the client;
    • country_denied, rate_limited, rule_blocked and waf_blocked: refused by that check;
    • too_large and timed_out: a size or time limit was passed;
    • upstream_error: the app could not be reached, or its answer broke off;
    • admin: answered at one of smallwebwaf's own endpoints.
  • would_action: in observe mode, what enforce mode would have done with a request it would have refused. action then names what was done, normally forward.

  • limit_percent and limit_percent_setting: the client's percentage of the rate limits and the setting that lowered it, when one did. bytes_percent and bytes_percent_setting do the same for the byte limits.

  • counts: the client's totals as rate limiting counts them, this request included. It is left out when rate limiting did not count the request. It holds:

    • minute, hour and day: requests;
    • minute_bytes, hour_bytes and day_bytes: bytes.

    A count can have a fraction (see "Counting method" in SPEC.md).

  • limit_hit: the limit broken:

    • minute, hour or day for a rate limit;
    • minute_bytes, hour_bytes or day_bytes for a byte limit;
    • error_burst.

    A request whose bytes broke a limit was passed on whole, so its action stays forward.

  • rule_ids: the rule file rules that matched, in order, up to the one that refused.

  • waf_score and waf_rule_ids: the Core Rule Set's score and the rules that matched, for a request it inspected.

  • reputation: the sources the client is on, in this order:

    • blocklist URLs;
    • the CrowdSec decision list's URL;
    • DNS blocklist zones;
    • abuseipdb.
  • offence: limit, for a request that broke a limit.

  • ban_expires: when the client's ban ends, for a request that made one or was refused under one: a time, or permanent.

Timings

All in milliseconds:

  • duration_total: from the request's headers being read to its log line;
  • duration_checks: to the end of the checks, before forwarding;
  • duration_waf: the part of that the Core Rule Set took;
  • duration_upstream_connect: from the hand-over to the app until there was a connection;
  • duration_upstream_first_byte: until the first byte of the answer;
  • duration_upstream_total: until the end of the answer.

servewaf's lines

smallwebwaf servewaf writes two lines for each question it answers, joined by request_id. The field servewaf_line says which of the two a line is:

  • decision: the line it writes once it has answered the question;
  • report: the line it writes when rproxy's report comes, with everything known. The repetition is intended.

A request to its own endpoints writes one line, without servewaf_line. A report writes no line but the report line of the request it names.

These questions write no line:

  • a question refused with 400 (see "The question" under "servewaf" in SPEC.md);
  • a question whose body stopped arriving, which gets no answer.

The decision line

It has the fields above, with these meanings where they differ:

  • client_ip: X-Smallwebwaf-Client.
  • peer_ip: X-Smallwebwaf-Peer.
  • host: the site the visitor asked for.
  • protocol: the question's own, HTTP/1.1.
  • content_length: X-Smallwebwaf-Content-Length.
  • request_id: the question's X-Request-ID.
  • status: what servewaf answered: 200 for allow, the status of deny, or 0 for close, as the visitor is then sent nothing.
  • request_bytes: how much of the question's body the Core Rule Set read.
  • response_bytes: the body of servewaf's answer.
  • action: forward for allow, and admin for the 404 for a path under /_smallwebwaf/.
  • aborted: true when rproxy went away before the answer.
  • duration_total: from servewaf having read the question's body to the line.
  • time: when servewaf had read the question's body.

The line leaves out upstream_status, websocket and the duration_upstream_... fields, since servewaf passes nothing to the app.

The bytes of an allowed request are counted when its report comes. So counts holds the client's bytes without this request's.

The report line

It has every field of the decision line, and time stays as the decision line gives it. These fields say what the report says:

  • status: what the visitor got, 0 if nothing;
  • request_bytes and response_bytes: the body bytes each way, with what a WebSocket carried each way;
  • duration_total: from the request's headers being read by rproxy to the end of the answer;
  • duration_upstream_total: from the hand-over to the app to the end of the answer, for an allowed request.

For an allowed request, the report's counting also changes these fields:

  • counts: the byte totals now include this request's;
  • limit_hit, offence and ban_expires: set when the report's bytes broke a byte limit.

rproxy's line

smallwebwaf rproxy writes one short line for each request. request_id joins it to servewaf's two. It has these fields and no others:

  • type: request.
  • time: when the request arrived, RFC 3339 with milliseconds, in UTC.
  • request_id: the id rproxy made for the request when it arrived.
  • client_ip: the visitor's address.
  • host and method.
  • path and query: as the visitor sent them.
  • status: what the visitor got, 0 if nothing.
  • request_bytes: the bytes of the visitor's body that rproxy read, with what a WebSocket carried from the visitor.
  • response_bytes: the bytes of the answer's body, with what a WebSocket carried from the app.
  • duration_request: milliseconds from the request's headers being read until the app had been sent the whole request, or the answer started if that was sooner. It includes the wait for servewaf.
  • duration_response: milliseconds from then until the end of the answer.
  • disposition: what was done with the request: allow, deny or close. It is servewaf's answer, or the value of SWRP_DEFAULT_DISPOSITION. It is left out when servewaf was not asked, as for the health check.
  • default_disposition: true when SWRP_DEFAULT_DISPOSITION gave disposition, and otherwise left out.

For example:

{"type":"request","time":"2026-10-03T12:00:00.123Z","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","client_ip":"203.0.113.9","host":"app.example","method":"GET","path":"/","query":"","status":200,"request_bytes":0,"response_bytes":5120,"duration_request":1.204,"duration_response":3.104,"disposition":"allow"}