Docs: rproxy runs behind any TLS-terminating reverse proxy (closes #388)

README, SPEC and docs no longer present traefik as the required front
proxy, and call it the TLS-terminating proxy. The README section of that
name lists what the proxy must do: forward plain HTTP to port 8080, keep
Host, set X-Forwarded-For and X-Forwarded-Proto, close an unused
connection in under 120 seconds, and connect from an address in
SWW_RPROXY_TRUSTED_PROXIES. traefik, Caddy and nginx appear as examples,
with the Caddy and nginx settings their defaults lack.

Model: opus-5-5

README.md

@@ -10,10 +10,10 @@ A web application firewall reads each request on its way to a web app and
 refuses the ones that look like abuse. `smallwebwaf` is one program, `sww`,
 shipped in one image, with two commands that do this together:
 
-- `sww rproxy` is a small reverse proxy in front of one app. traefik routes to
+- `sww rproxy` is a small reverse proxy in front of one app. It sits behind a
-  it. It asks `waf` about each request, passes the allowed ones to the app, and
+  TLS-terminating proxy, such as traefik, nginx or Caddy. It asks `waf` about
-  passes the app's answers back. It keeps nothing between requests and writes
+  each request, passes the allowed ones to the app, and passes the app's answers
-  nothing to disk.
+  back. It keeps nothing between requests and writes nothing to disk.
 - `sww waf` decides. It runs the checks and keeps the bans, the counters and
   each client's history. One `waf` serves many sites: a client banned through
   one site is refused on all of them, and its requests on all of them count
@@ -57,8 +57,7 @@ For the operator, it:
 
 Not built, and not planned:
 
-- TLS and routing by host name. Both stay with traefik, the reverse proxy in
+- TLS and routing by host name. Both stay with the TLS-terminating proxy.
-  front of `rproxy`.
 - Browser challenges such as captchas.
 - A web console, or a configuration file.
 - Defence against floods large enough to fill the host's network link.
@@ -90,20 +89,61 @@ accepts connections, whether or not `waf` answers; in `waf`'s container, once
 The usual setup is three containers on one private network:
 
 - the app;
-- `rproxy`, in front of the app, which traefik routes to:
+- `rproxy`, in front of the app, which the TLS-terminating proxy forwards to:
   `SWW_RPROXY_WAF_BACKEND` names `waf` and `SWW_RPROXY_UPSTREAM_URL` names the
   app;
 - `waf`, with `SWW_COMMAND=waf` and a volume at `/var/lib/sww` for its state
   files.
 
-Point traefik's labels at port 8080 of `rproxy`
+Publish no port of the app or of `waf`. Anything on that network can reach the
-(`traefik.http.services.<name>.loadbalancer.server.port=8080`). Publish no port
+app around `rproxy`, and can ask `waf` questions, so the network holds only what
-of the app or of `waf`. Anything on that network can reach the app around
+you trust. Each further app adds its own container and its own `rproxy`, whose
-`rproxy`, and can ask `waf` questions, so the network holds only what you trust.
-Each further app adds its own container and its own `rproxy`, whose
 `SWW_RPROXY_WAF_BACKEND` names the same `waf`. The "Smallest setup" example
 below shows this shape.
 
+### The TLS-terminating proxy
+
+`rproxy` serves plain HTTP. It runs behind any TLS-terminating proxy, such as
+traefik, nginx or Caddy. The TLS-terminating proxy must:
+
+- forward each request to port 8080 of `rproxy` over plain HTTP;
+- keep the visitor's `Host` header;
+- add the visitor's address to `X-Forwarded-For`;
+- set `X-Forwarded-Proto` to the scheme the visitor used;
+- close an unused connection to `rproxy` in under 120 seconds;
+- connect to `rproxy` from an address in `SWW_RPROXY_TRUSTED_PROXIES`, by
+  default `10.0.0.0/8`, `172.16.0.0/12` or `192.168.0.0/16`.
+
+From any other address, `rproxy` ignores the forwarded headers and takes the
+TLS-terminating proxy for the visitor. Every visitor then shares one address,
+and one ban.
+
+traefik and Caddy keep the `Host` header and set both forwarded headers by
+default. traefik's Docker label
+`traefik.http.services.<name>.loadbalancer.server.port=8080` names the port.
+Caddy keeps an unused connection for 120 seconds by default, so its site
+shortens that to 90 seconds:
+
+```caddyfile
+reverse_proxy rproxy:8080 {
+    transport http {
+        keepalive 90s
+    }
+}
+```
+
+By default nginx sends the host in `proxy_pass` as `Host` and sets neither
+forwarded header, so its site sets all three:
+
+```nginx
+location / {
+    proxy_pass http://rproxy:8080;
+    proxy_set_header Host $host;
+    proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
+    proxy_set_header X-Forwarded-Proto $scheme;
+}
+```
+
 ### `rproxy` in the app's container
 
 An app's image can instead be built on the `smallwebwaf` image, so that `rproxy`
@@ -154,13 +194,13 @@ entrypoint starts `rproxy` and the app side by side, each as its own user. Run
 the app's image with `SWW_RPROXY_WAF_BACKEND` set. `deploy/example-app` is such
 an app, and the "`rproxy` beside the app" example below runs one.
 
-The app sees the headers traefik sent, with traefik's address added to
+The app's TCP peer is `rproxy`. The app sees the headers the TLS-terminating
-`X-Forwarded-For`, from a TCP peer that is `rproxy`. So the app must trust
+proxy sent, with that proxy's address added to `X-Forwarded-For`. So the app
-`rproxy`'s address for forwarded headers. In a container of its own, `rproxy` is
+must trust `rproxy`'s address for forwarded headers. In a container of its own,
-on the private network, which the usual default of a trusted-proxy setting
+`rproxy` is on the private network, which the usual default of a trusted-proxy
-covers. Beside the app, its address is `127.0.0.1` or `::1`, which the private
+setting covers. Beside the app, its address is `127.0.0.1` or `::1`, which the
-ranges leave out, so the app's own setting must name them too, as `app.run`
+private ranges leave out, so the app's own setting must name them too, as
-does.
+`app.run` does.
 
 ### Checking that it works
 
@@ -168,7 +208,7 @@ does.
   `healthy` for each container of `smallwebwaf`.
 - A log line with `"msg":"starting"` gives the version, the address and every
   setting in effect, with secrets masked.
-- A request to the site through traefik gets the app's answer.
+- A request to the site through the TLS-terminating proxy gets the app's answer.
   `https://<site>/_sww/healthz` answers `ok`. `waf`'s log lines about the
   request have `"action":"forward"`.
 - To see a refusal without banning yourself, run `waf` with
@@ -260,7 +300,7 @@ Every setting the code reads, with its default:
 | `SWW_WAF_LOOKUP_SOURCE`                           | `geojs`                                                                                  | Where each client's AS number and country come from: `geojs` (a web service), `file` (a local database) or `off`.                                                                                                                                               |
 | `SWW_WAF_LOOKUP_DB_PATH`                          | empty                                                                                    | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`.                                                                                                                                                                            |
 | `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`       | `800`                                                                                    | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take, in milliseconds.                                                                                                                |
-| `SWW_WAF_ADD_LOOKUP_HEADERS`                      | `false`                                                                                  | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`.                                                                                                                                                              |
+| `SWW_WAF_ADD_LOOKUP_HEADERS`                      | `true`, or `false` while the lookup source is `off`                                      | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`.                                                                                                                                                              |
 | `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES`           | empty                                                                                    | When set, the only countries whose clients get through; a client the lookup cannot place is refused too.                                                                                                                                                        |
 | `SWW_WAF_TAG_LISTS`                               | empty                                                                                    | Address lists that each add a tag to their clients' requests, as `name=URL` pairs, such as `torexit=https://example.com/tor.txt`.                                                                                                                               |
 | `SWW_WAF_TAG_DENY`                                | empty                                                                                    | Tags whose requests are refused, such as `torexit,country=CN`.                                                                                                                                                                                                  |

SPEC.md

@@ -3,7 +3,7 @@
 `smallwebwaf` is a web application firewall for self-hosted apps. It is one
 binary, `sww`, and two of its commands do the work:
 
-- `sww rproxy` is a reverse proxy between traefik and one app.
+- `sww rproxy` is a reverse proxy between a TLS-terminating proxy and one app.
 - `sww waf` decides what happens to each request, for many sites.
 
 This document says what it must do and why, each requirement once. Everything in
@@ -45,7 +45,8 @@ never as a time.
 
 ## Non-goals
 
-- TLS, certificates and routing by host name. Those are traefik's job.
+- TLS, certificates and routing by host name. Those are the job of the
+  TLS-terminating proxy, such as traefik, nginx or Caddy.
 - More than one app behind one `rproxy`. Each app has its own.
 - Browser challenges such as captchas or proof of work. If wanted, chain Anubis
   between `rproxy` and the app.
@@ -161,16 +162,16 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
    (`deny_nets`); a client in both lists is let through.
 5. Bans. A client under a ban is refused (`active_ban`; see "Bans").
 6. The lookup, unless `SWW_WAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
-   and country"). The request goes on at once. It waits for the first GeoJS
+   and country"). The request waits for the first GeoJS answer, up to
-   answer, up to `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, only when one of
+   `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`, when one of these needs it:
-   these needs it:
+    - `SWW_WAF_ADD_LOOKUP_HEADERS`, which is `true` by default;
     - the country list;
     - a biased threshold;
     - `SWW_WAF_TAG_DENY` or `SWW_WAF_TAG_LIMIT_PERCENT` naming an `asn=` or
-      `country=` tag;
+      `country=` tag.
-    - `SWW_WAF_ADD_LOOKUP_HEADERS`.
 
-    A request that ends before the answer comes is logged without it.
+    Otherwise the request goes on at once. A request that ends before the answer
+    comes is logged without it.
 
 7. The country list (`countries`; see "Country list").
 8. The reputation sources (see "Reputation"), in this order:
@@ -197,10 +198,10 @@ field gives that name. `refusal_limit` and `byte_limits` give theirs in
 16. A path under `/_sww/` is answered `404`, so that the app never sees that
     prefix.
 17. `rproxy` passes the request to `SWW_RPROXY_UPSTREAM_URL`, streaming (see
-    "Passing a request to the app"). With `SWW_WAF_ADD_LOOKUP_HEADERS`,
+    "Passing a request to the app"). While `SWW_WAF_ADD_LOOKUP_HEADERS` is
-    `X-Client-ASN` and `X-Client-Country` are added for the app's own logs; a
+    `true`, as by default, `X-Client-ASN` and `X-Client-Country` are added for
-    header whose value is unknown is left out. Whatever that setting says, any
+    the app's own logs; a header whose value is unknown is left out. Whatever
-    such header the client sent is removed.
+    that setting says, any such header the client sent is removed.
 18. After the visitor's answer has ended, `rproxy` sends the report (see "The
     report"). `waf` then counts:
     - For a request it allowed, the body bytes of the response, of the request
@@ -1943,10 +1944,10 @@ leave out a cookie only when its name matches exactly, case included.
 ### Ban lengths, response and scope
 
 `SWW_WAF_BAN_RESPONSE` is `403` by default, which makes a ban easy to recognise
-when debugging; `429` and `close` are the alternatives. Behind traefik, `close`
+when debugging; `429` and `close` are the alternatives. `close` does not leave
-does not leave the client unanswered: traefik answers `502`, as it does whenever
+the client unanswered: the TLS-terminating proxy answers `502`, as it does
-its backend drops a connection. `SWW_WAF_BAN_SCOPE_V4_PREFIX` widens an IPv4 ban
+whenever its backend drops a connection. `SWW_WAF_BAN_SCOPE_V4_PREFIX` widens an
-to the surrounding netblock. The four ban lengths are under "Bans".
+IPv4 ban to the surrounding netblock. The four ban lengths are under "Bans".
 
 ### Reputation
 
@@ -2373,9 +2374,9 @@ runs no check, writes no request log line and counts nothing in the metrics.
 answer but the last below carries `X-Sww-Disposition`.
 
 - `X-Sww-Disposition: allow`, status `200`, no body. `rproxy` passes the request
-  to the app. With `SWW_WAF_ADD_LOOKUP_HEADERS` the answer also carries
+  to the app. While `SWW_WAF_ADD_LOOKUP_HEADERS` is `true`, the answer also
-  `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the request it
+  carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the
-  passes on. A header whose value is unknown is left out.
+  request it passes on. A header whose value is unknown is left out.
 - `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to
   get. `rproxy` gives them to the visitor, less this header. The status is one
   of:
@@ -2771,9 +2772,9 @@ its whole body. The visitor gets that answer at once, and the connection closes
 after it.
 
 The largest request line and headers is 32 KiB. A kept-open connection is closed
-after 120 seconds idle. Neither has a setting. traefik, by default, closes an
+after 120 seconds idle. Neither has a setting. The TLS-terminating proxy should
-unused connection after 90 seconds. So traefik closes first, and never sends a
+close an unused connection sooner, so that it never sends a request on a
-request on a connection `rproxy` is closing.
+connection `rproxy` is closing. traefik closes one after 90 seconds by default.
 
 What the visitor gets when a limit is passed:
 
@@ -2824,14 +2825,16 @@ chooses: `waf` runs `waf`, and with it unset the image runs `rproxy`.
 The usual setup is three containers on one private network:
 
 - the app;
-- `rproxy`, which traefik routes to, with `SWW_RPROXY_UPSTREAM_URL` naming the
+- `rproxy`, which the TLS-terminating proxy forwards to, with
-  app and `SWW_RPROXY_WAF_BACKEND` naming `waf`;
+  `SWW_RPROXY_UPSTREAM_URL` naming the app and `SWW_RPROXY_WAF_BACKEND` naming
+  `waf`;
 - `waf`, with `SWW_COMMAND=waf` and a volume for its state files.
 
 Each further app adds its own container and its own `rproxy`, whose
 `SWW_RPROXY_WAF_BACKEND` names the same `waf` (see "Sites"). Anything on that
 network can reach an app around `rproxy`, and can ask `waf` questions or send it
-reports. So the network holds only what you trust.
+reports. So the network holds only what you trust. `README.md` lists what the
+TLS-terminating proxy must do.
 
 ### `rproxy` in the app's container
 
@@ -2903,10 +2906,10 @@ process that stops at start is started again every second, not in a tight loop.
   container's output, which `docker logs` shows.
 - When a service exits, runit runs its `run` script again, and the container
   does not stop. Beside an app, the other process keeps running.
-- While `rproxy` is down, traefik answers `502`. While the app is down, `rproxy`
+- While `rproxy` is down, the TLS-terminating proxy answers `502`. While the app
-  answers `502`, which is not a client offence. While `waf` is down,
+  is down, `rproxy` answers `502`, which is not a client offence. While `waf` is
-  `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides (see "Without an answer
+  down, `SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION` decides (see "Without an
-  from `waf`").
+  answer from `waf`").
 - Only faults found at start stop `waf`:
     - invalid configuration;
     - a lookup database that is missing or unreadable;
@@ -2939,23 +2942,26 @@ checks:
 - With `SWW_COMMAND` unset, empty or `rproxy`, it passes while `rproxy` answers
   that request on `127.0.0.1`, at the port in `SWW_RPROXY_LISTEN_ADDR`, and the
   app accepts connections at `SWW_RPROXY_UPSTREAM_URL`. It does not ask `waf`,
-  so that the site stays in traefik's routes while `waf` is down.
+  so that the site stays in the TLS-terminating proxy's routes while `waf` is
+  down.
 - With any other value, it fails with a message naming `SWW_COMMAND` and its
   value.
 
-traefik sends a container no requests until it shows as healthy. So the check
+A TLS-terminating proxy that reads Docker's health status, as traefik does,
-runs every second from the container's start until it first passes, for up to a
+sends a container no requests until it shows as healthy. So the check runs every
-minute, and every 30 seconds after that. An app with a health check of its own
+second from the container's start until it first passes, for up to a minute, and
-can replace the image's `HEALTHCHECK` with one that checks both.
+every 30 seconds after that. An app with a health check of its own can replace
+the image's `HEALTHCHECK` with one that checks both.
 
 ### Ports
 
 `rproxy` listens on no port but the one in `SWW_RPROXY_LISTEN_ADDR`, 8080 by
-default, which traefik routes to. It passes requests to the app at
+default, which the TLS-terminating proxy forwards to. It passes requests to the
-`SWW_RPROXY_UPSTREAM_URL`, `http://127.0.0.1:8081` by default. Another port may
+app at `SWW_RPROXY_UPSTREAM_URL`, `http://127.0.0.1:8081` by default. Another
-be set: the health check follows the setting, and traefik's port label
+port may be set: the health check follows the setting, and the TLS-terminating
-(`traefik.http.services.<name>.loadbalancer.server.port`) must name it. An app
+proxy must name it, as traefik's port label
-beside `rproxy` must leave that port free.
+(`traefik.http.services.<name>.loadbalancer.server.port`) does. An app beside
+`rproxy` must leave that port free.
 
 `waf` listens on no port but the one in `SWW_WAF_LISTEN_ADDR`, 8080 by default.
 The questions, the reports and its own endpoints all come there.
@@ -2996,9 +3002,9 @@ in `docs/settings.md` shows the commands.
 ### Forwarded headers
 
 The app's TCP peer is `rproxy`, and the `X-Forwarded-For` the app receives ends
-with traefik's address, which `rproxy` adds after the visitor's. So the app must
+with the TLS-terminating proxy's address, which `rproxy` adds after the
-trust `rproxy`'s address for forwarded headers, besides the private address
+visitor's. So the app must trust `rproxy`'s address for forwarded headers,
-ranges traefik is on.
+besides the private address ranges the TLS-terminating proxy is on.
 
 - In a container of its own, `rproxy`'s address is on the private network. The
   usual default for a trusted-proxy setting, the private ranges, covers it.

docs/request-log.md

@@ -68,7 +68,8 @@ Every one of these lines has:
 ### Request detail
 
 - `request_id`: the id `rproxy` made for the request when it arrived.
-- `peer_ip`: the address of `rproxy`'s TCP peer, normally traefik.
+- `peer_ip`: the address of `rproxy`'s TCP peer, normally the TLS-terminating
+  proxy.
 - `forwarded_for`: the `X-Forwarded-For` header the visitor's request came with.
 - `client_group`: the client as rate limiting counts it, an IPv4 address as a
   /32 or an IPv6 group.

docs/settings.md

@@ -150,7 +150,7 @@ cut off; the client is banned.
 - `SWW_WAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off`
   stops the start while a setting that needs a lookup is set:
     - `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES`;
-    - `SWW_WAF_ADD_LOOKUP_HEADERS`;
+    - `SWW_WAF_ADD_LOOKUP_HEADERS` set to `true`;
     - `SWW_WAF_TAG_DENY` or `SWW_WAF_TAG_LIMIT_PERCENT` naming a `country=` tag,
       or an `asn=` tag while `SWW_WAF_ROUTEWATCH_URL` is unset;
     - the two byte percentage lists and `SWW_WAF_ASN_LIMIT_PERCENT_URL`;
@@ -161,9 +161,11 @@ cut off; the client is banned.
 - `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS` (default `800`): how long a
   request waits for its client's first answer from GeoJS, and how long a request
   to GeoJS may take. A time in milliseconds; never `off`.
-- `SWW_WAF_ADD_LOOKUP_HEADERS` (default `false`): `true` has `rproxy` pass the
+- `SWW_WAF_ADD_LOOKUP_HEADERS` (default `true`): `true` has `rproxy` pass the
   app the client's AS number in `X-Client-ASN` and its country in
-  `X-Client-Country`.
+  `X-Client-Country`. The default is `false` while `SWW_WAF_LOOKUP_SOURCE` is
+  `off`. With GeoJS, a new client's first request waits for its answer, up to
+  `SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS`.
 
 ## Country list
 
@@ -476,9 +478,9 @@ start, and an invalid value stops the start. Any can be given as a file, as a
   `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`): the netblocks whose
   `X-Forwarded-For` is believed. Set but empty, it trusts nothing. A client
   outside these netblocks cannot claim another address. Where other containers
-  reach `rproxy` directly, or visitors on a private network reach traefik, they
+  reach `rproxy` directly, or visitors on a private network reach the
-  can name another address in the header. Narrowing the setting to traefik's own
+  TLS-terminating proxy, they can name another address in the header. Narrowing
-  network closes that gap.
+  the setting to the TLS-terminating proxy's own network closes that gap.
 
 ## The image's command
 

internal/config/waf.go

@@ -84,7 +84,8 @@ type WAF struct {
 	// client's first answer from GeoJS while a setting needs it
 	// (SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS), which cannot be off.
 	// AddLookupHeaders is true when the app is passed the client's AS number
-	// and country in headers (SWW_WAF_ADD_LOOKUP_HEADERS).
+	// and country in headers (SWW_WAF_ADD_LOOKUP_HEADERS), by default unless
+	// LookupSource is off.
 	LookupSource               string
 	LookupDBPath               string
 	ASNAndCountryLookupTimeout time.Duration
@@ -549,7 +550,6 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
 			"SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS",
 			"800",
 		),
-		AddLookupHeaders: env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS", "false"),
 		ExclusivelyAllowedCountries: env.countries(
 			"SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES", ""),
 		TagLists:        env.tagLists("SWW_WAF_TAG_LISTS"),
@@ -724,6 +724,9 @@ func WAFFromEnvironment(v *viper.Viper) (*WAF, error) {
 
 	cfg.LogRemoteAppName = env.appName("SWW_WAF_LOG_REMOTE_APP_NAME",
 		cfg.InstanceName, cfg.LogRemoteURL != nil)
+	// The app gets the headers by default, unless no client is looked up.
+	cfg.AddLookupHeaders = env.boolean("SWW_WAF_ADD_LOOKUP_HEADERS",
+		strconv.FormatBool(cfg.LookupSource != off))
 	cfg.BlocklistAction, cfg.BlocklistLimitPercent = env.action(
 		"SWW_WAF_BLOCKLIST_ACTION", "deny")
 	cfg.DNSBLAbuseIPDBAction, cfg.DNSBLAbuseIPDBLimitPercent = env.action(

internal/config/waf_test.go

@@ -287,6 +287,7 @@ func TestDefaults(t *testing.T) {
 	})
 	wantLookupSettings(t, cfg, config.WAF{
 		LookupSource: defaultLookupSource, ASNAndCountryLookupTimeout: 800 * time.Millisecond,
+		AddLookupHeaders: true,
 	})
 	wantByteLimitSettings(t, cfg, config.WAF{
 		BytesLimitPerMinute: 0, BytesLimitPerHour: 50 << 30,
@@ -1395,13 +1396,13 @@ func TestLookupSettingsAsSet(t *testing.T) {
 	t.Parallel()
 
 	cfg := wafFromEnvironment(t, environment{
-		asnAndCountryLookupTimeout: "500", addLookupHeaders: enabled,
+		asnAndCountryLookupTimeout: "500", addLookupHeaders: disabled,
 	})
 	wantLookupSettings(t, cfg, config.WAF{
 		LookupSource: defaultLookupSource, ASNAndCountryLookupTimeout: 500 * time.Millisecond,
-		AddLookupHeaders: true,
 	})
 
+	// With no client looked up, the headers are off unless set.
 	cfg = wafFromEnvironment(t, environment{lookupSource: off})
 	wantLookupSettings(t, cfg, config.WAF{
 		LookupSource: off, ASNAndCountryLookupTimeout: 800 * time.Millisecond,
@@ -1422,6 +1423,10 @@ func TestLookupDBPathGoesWithTheFileSourceAlone(t *testing.T) {
 			cfg.LookupSource, cfg.LookupDBPath, path)
 	}
 
+	if !cfg.AddLookupHeaders {
+		t.Error("headers off by default with lookups from file, want on")
+	}
+
 	for _, tc := range []struct {
 		env  environment
 		want string
@@ -2927,7 +2932,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
 		lookupSource:                   defaultLookupSource,
 		lookupDBPath:                   "",
 		asnAndCountryLookupTimeout:     "800",
-		addLookupHeaders:               disabled,
+		addLookupHeaders:               enabled,
 		allowedCountries:               "",
 		tagLists:                       "",
 		tagDeny:                        "",

internal/requestlog/requestlog.go

@@ -21,8 +21,8 @@ const (
 	// the app.
 	ActionForward = "forward"
 
-	// ActionRequestLineTooLong is a request whose request line is over the
+	// ActionRequestLineTooLong is a request whose request line is too long for
-	// header limits' longest line.
+	// the header limits.
 	ActionRequestLineTooLong = "request_line_too_long"
 
 	// ActionRequestHeaderTooLarge is a request whose headers are over the

internal/waf/anomaly_test.go

@@ -349,6 +349,7 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
 		map[string]string{
 			trustedProxies:                 trustLocalhost,
 			asnAndCountryLookupTimeout:     "3600000",
+			addLookupHeaders:               falseValue,
 			anomalyASN + requestsPerMinute: aLot,
 		})
 	server.Store(started)

internal/waf/answertime_test.go

@@ -204,6 +204,10 @@ func TestAHeaderLineOverTheLimitIsRefusedWith431(t *testing.T) {
 		{"a Host line over the limit", func(req *http.Request) {
 			req.Host = strings.Repeat("a", lineLimit+1-len("Host: "))
 		}, true},
+		{"a header sent twice, each line within the limit", func(req *http.Request) {
+			req.Header.Add("X-Long", strings.Repeat("a", 8000))
+			req.Header.Add("X-Long", strings.Repeat("a", 8000))
+		}, false},
 	} {
 		t.Run(tc.name, func(t *testing.T) {
 			t.Parallel()

internal/waf/biased_test.go

@@ -387,12 +387,14 @@ func TestRequestWaitsForItsLookupWhileABiasedThresholdIsSet(t *testing.T) {
 
 			// In a synctest bubble, as
 			// TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays says,
-			// with a GeoJS that never answers.
+			// with a GeoJS that never answers. The headers, which need the
+			// answer too, are off.
 			synctest.Test(t, func(t *testing.T) {
 				server, out, _ := newWAF(t, silentGeoJSURL, time.Now,
 					map[string]string{
 						asnAndCountryLookupTimeout: strconv.FormatInt(
 							timeout.Milliseconds(), 10),
+						addLookupHeaders: falseValue,
 						tagLists:         "vpn=" + vpnList,
 						tc.setting:       tc.value,
 					})

internal/waf/countries_test.go

@@ -108,8 +108,11 @@ func TestCountryRefusalNamesItsHeuristicCountryAndSetting(t *testing.T) {
 	t.Parallel()
 
 	geojsURL, _ := startGeoJS(t)
+	// The headers are off, so that only the country list makes the request
+	// wait for its answer.
 	s, _, _ := startWithClock(t, geojsURL, map[string]string{
-		asnAndCountryLookupTimeout: "3600000", allowedCountries: "de",
+		asnAndCountryLookupTimeout: "3600000", addLookupHeaders: falseValue,
+		allowedCountries: "de",
 	})
 
 	line := s.get(
@@ -239,7 +242,10 @@ func TestPrivateAddressIsNeverLookedUp(t *testing.T) {
 		name string
 		env  map[string]string
 	}{
-		{"no setting needs the lookup", nil},
+		{
+			"no setting needs the lookup",
+			map[string]string{addLookupHeaders: falseValue},
+		},
 		{"a tag setting needs it", map[string]string{tagDeny: denyKP}},
 	} {
 		t.Run(tc.name, func(t *testing.T) {

internal/waf/lookup.go

@@ -12,7 +12,7 @@ import (
 )
 
 // The headers in which an answer of allow gives rproxy the client's AS number
-// and country, for the app, while SWW_WAF_ADD_LOOKUP_HEADERS is set. Go writes
+// and country, for the app, while SWW_WAF_ADD_LOOKUP_HEADERS is true. Go writes
 // every header name in this form, as it sends it and as it receives it, so
 // X-Client-ASN arrives as X-Client-Asn; header names are not case-sensitive.
 const (

internal/waf/lookup_test.go

@@ -36,6 +36,7 @@ func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(
 	geojsURL, asked, release := startHeldGeoJS(t)
 	s, _, server := startWithClock(t, geojsURL, map[string]string{
 		asnAndCountryLookupTimeout: "3600000",
+		addLookupHeaders:           falseValue,
 		rateLimitPerMinute:         "1",
 	})
 
@@ -418,7 +419,7 @@ func TestClientsOwnLookupHeadersAreRemovedWhileTheSettingIsOff(
 		geojsURL,
 		time.Now,
 		map[string]string{
-			trustedProxies: trustLocalhost,
+			trustedProxies: trustLocalhost, addLookupHeaders: falseValue,
 		},
 	)
 	s := &sender{t: t, addr: addr, out: out}
@@ -457,6 +458,8 @@ func TestRequestWaitsAsLongAsTheASNAndCountryLookupTimeoutSays(
 			map[string]string{
 				asnAndCountryLookupTimeout: strconv.FormatInt(
 					timeout.Milliseconds(), 10),
+				// Off, so that only the country list makes the request wait.
+				addLookupHeaders: falseValue,
 				allowedCountries: "DE",
 			})
 

internal/waf/proxy_test.go

@@ -93,6 +93,10 @@ const off = "off"
 // on is the value that switches a setting that takes true or false on.
 const on = "true"
 
+// falseValue is the value that switches a setting that takes true or false
+// off.
+const falseValue = "false"
+
 // output collects what sww writes on stdout.
 type output struct {
 	mu  sync.Mutex

internal/waf/waf.go

@@ -320,7 +320,7 @@ func asTheVisitorSentIt(r *http.Request, body []byte) *http.Request {
 }
 
 // allow answers a question with DispositionAllow and, while
-// SWW_WAF_ADD_LOOKUP_HEADERS is set, the client's AS number and country.
+// SWW_WAF_ADD_LOOKUP_HEADERS is true, the client's AS number and country.
 func (rq *request) allow() {
 	rq.allowed = true
 

internal/waf/waf_test.go

@@ -869,7 +869,7 @@ func TestAllowCarriesTheLookupHeadersWhileTheSettingIsSet(t *testing.T) {
 		wantASN, wantCountry string
 	}{
 		{on, "AS64496", "DE"},
-		{"false", "", ""},
+		{falseValue, "", ""},
 	} {
 		t.Run(tc.setting, func(t *testing.T) {
 			t.Parallel()