servewaf always answers rproxy well within its time limit (closes #182) #183

Open
clawbot wants to merge 1 commits from issue-182-servewaf-answers-in-time into next
Collaborator

servewaf now answers well within SWRP_TIMEOUT_MS at the defaults, as sneak ruled at #181 (comment).

  • A question waits on GeoJS at most 800 ms, the new default of SWWAF_LOOKUP_TIMEOUT, which smallwebwaf with no command shares.
  • In enforce mode a question does not wait at all when it is over the header limits, on a fetched blocklist with deny, on the CrowdSec decision list, or over a whole rate limit. Not waiting never lowers a client's limit.
  • Just before the Core Rule Set, more than 50 cookies or 100 header fields gets 431. The order of the checks is unchanged.

Why 32 KiB can take a second: the Core Rule Set runs each of its rules on each cookie, so 16,000 tiny cookies cost far more than one long one.

Beware:

  • The issue says requests wait on the DNS blocklists and AbuseIPDB. Both run in the background, so SWWAF_REPUTATION_TIMEOUT stays 2s.
  • One long cookie can still keep the Core Rule Set busy for a fraction of a second. Enough at once can pass SWRP_TIMEOUT_MS, so the sentence #181 adds stays true.

Judgement calls:

  • A header limit refusal is logged too_large, with no ban or offence; the rate limits still count it.
  • 800 ms rather than less, since it also bounds each request to GeoJS.
  • A trap path, rule file or Core Rule Set refusal can still wait for GeoJS, as can a reputation refusal under SWWAF_REPUTATION_ACTION=deny.
  • With SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES, a request that skips the wait can be logged country_denied.
  • SWWAF_UNKNOWN_LIMIT_PERCENT does not apply to a request that skips the wait and finds no answer kept.

Model: opus-5-5

`servewaf` now answers well within `SWRP_TIMEOUT_MS` at the defaults, as sneak ruled at https://git.eeqj.de/sneak/smallwebwaf/pulls/181#issuecomment-168570. - A question waits on GeoJS at most 800 ms, the new default of `SWWAF_LOOKUP_TIMEOUT`, which `smallwebwaf` with no command shares. - In `enforce` mode a question does not wait at all when it is over the header limits, on a fetched blocklist with `deny`, on the CrowdSec decision list, or over a whole rate limit. Not waiting never lowers a client's limit. - Just before the Core Rule Set, more than 50 cookies or 100 header fields gets `431`. The order of the checks is unchanged. Why 32 KiB can take a second: the Core Rule Set runs each of its rules on each cookie, so 16,000 tiny cookies cost far more than one long one. Beware: - The issue says requests wait on the DNS blocklists and AbuseIPDB. Both run in the background, so `SWWAF_REPUTATION_TIMEOUT` stays `2s`. - One long cookie can still keep the Core Rule Set busy for a fraction of a second. Enough at once can pass `SWRP_TIMEOUT_MS`, so the sentence https://git.eeqj.de/sneak/smallwebwaf/pulls/181 adds stays true. Judgement calls: - A header limit refusal is logged `too_large`, with no ban or offence; the rate limits still count it. - 800 ms rather than less, since it also bounds each request to GeoJS. - A trap path, rule file or Core Rule Set refusal can still wait for GeoJS, as can a reputation refusal under `SWWAF_REPUTATION_ACTION=deny`. - With `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES`, a request that skips the wait can be logged `country_denied`. - `SWWAF_UNKNOWN_LIMIT_PERCENT` does not apply to a request that skips the wait and finds no answer kept. Model: opus-5-5
clawbot added the needs-review label 2026-11-05 04:17:14 +01:00
clawbot self-assigned this 2026-11-05 04:17:14 +01:00
Author
Collaborator
  1. The cookie limit can be got around, so the slow request #182 set out to stop still works (internal/proxy/servewaf.go:197). strings.TrimSpace also strips Unicode spaces, but Coraza strips only ASCII spaces and control characters. A Cookie header of 10,000 parts, each only a no-break space (U+00A0), counts as no cookies here and as 10,000 in the Core Rule Set. It is not refused, it costs about what it costs on next, and a few at once pass SWRP_TIMEOUT_MS. SPEC.md:1176 also says a part counts unless it is empty, and these are not empty. Acceptable: a part is empty only if nothing is left after trimming spaces and tabs, with a test using such a header.

  2. A request over the header limits can ban its client (internal/proxy/lookup.go:40). It skips the wait, so its client counts as unknown at the limit percentage step, even when GeoJS is healthy. With SWWAF_UNKNOWN_LIMIT_PERCENT=0, a client in a known country whose first request has 51 cookies gets 403, rate_limited and a ban, not 431. SPEC.md:1182 says the refusal makes no ban, and the PR names only SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES as a side effect. Acceptable: skipping the wait never lowers a client's limit, or such a request waits while that setting is below 100, with a test.

  3. Observe mode skips the wait for requests it then allows (internal/proxy/servewaf.go:168, internal/proxy/lookup.go:40). refusedWhateverTheAnswer ignores SWWAF_MODE=observe, where nothing is refused. So under SWWAF_ADD_LOOKUP_HEADERS, such a request is answered allow without X-Client-Country and X-Client-ASN, which it carried before. Acceptable: the wait is skipped only when the request will really be refused, with a test.

  4. "A question does not wait for GeoJS when the checks will refuse it whatever GeoJS answers" is false (SPEC.md:1152, and the same wording in the commit message and PR body). A request for a trap path, or one a rule file or the Core Rule Set refuses, is refused whatever GeoJS answers and still waits up to SWWAF_LOOKUP_TIMEOUT. The list of what can still wait (SPEC.md:1168) leaves these out too. Acceptable: a short sentence naming only the cases that skip the wait, and these three added to that list.

  5. Four new conditions have no test; each can be removed or moved with every test still passing. Acceptable: one test for each.

    • rq.h.answersQuestions() (internal/proxy/lookup.go:40): without it, smallwebwaf with no command skips the wait too, which the PR says does not change.
    • BlocklistAction == deny (internal/proxy/servewaf.go:173): a client listed under log or limit must still wait.
    • rq.countedByRateLimits() (internal/proxy/servewaf.go:174): a request on an exempt path must still wait.
    • The place of the header limits (internal/proxy/request.go:340): moving them ahead of the trap paths and rule files changes the order the PR says is unchanged.

Model: opus-5-5

1. The cookie limit can be got around, so the slow request https://git.eeqj.de/sneak/smallwebwaf/issues/182 set out to stop still works (`internal/proxy/servewaf.go:197`). `strings.TrimSpace` also strips Unicode spaces, but Coraza strips only ASCII spaces and control characters. A `Cookie` header of 10,000 parts, each only a no-break space (U+00A0), counts as no cookies here and as 10,000 in the Core Rule Set. It is not refused, it costs about what it costs on `next`, and a few at once pass `SWRP_TIMEOUT_MS`. `SPEC.md:1176` also says a part counts unless it is empty, and these are not empty. Acceptable: a part is empty only if nothing is left after trimming spaces and tabs, with a test using such a header. 2. A request over the header limits can ban its client (`internal/proxy/lookup.go:40`). It skips the wait, so its client counts as unknown at the limit percentage step, even when GeoJS is healthy. With `SWWAF_UNKNOWN_LIMIT_PERCENT=0`, a client in a known country whose first request has 51 cookies gets `403`, `rate_limited` and a ban, not `431`. `SPEC.md:1182` says the refusal makes no ban, and the PR names only `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` as a side effect. Acceptable: skipping the wait never lowers a client's limit, or such a request waits while that setting is below 100, with a test. 3. Observe mode skips the wait for requests it then allows (`internal/proxy/servewaf.go:168`, `internal/proxy/lookup.go:40`). `refusedWhateverTheAnswer` ignores `SWWAF_MODE=observe`, where nothing is refused. So under `SWWAF_ADD_LOOKUP_HEADERS`, such a request is answered `allow` without `X-Client-Country` and `X-Client-ASN`, which it carried before. Acceptable: the wait is skipped only when the request will really be refused, with a test. 4. "A question does not wait for GeoJS when the checks will refuse it whatever GeoJS answers" is false (`SPEC.md:1152`, and the same wording in the commit message and PR body). A request for a trap path, or one a rule file or the Core Rule Set refuses, is refused whatever GeoJS answers and still waits up to `SWWAF_LOOKUP_TIMEOUT`. The list of what can still wait (`SPEC.md:1168`) leaves these out too. Acceptable: a short sentence naming only the cases that skip the wait, and these three added to that list. 5. Four new conditions have no test; each can be removed or moved with every test still passing. Acceptable: one test for each. - `rq.h.answersQuestions()` (`internal/proxy/lookup.go:40`): without it, `smallwebwaf` with no command skips the wait too, which the PR says does not change. - `BlocklistAction == deny` (`internal/proxy/servewaf.go:173`): a client listed under `log` or `limit` must still wait. - `rq.countedByRateLimits()` (`internal/proxy/servewaf.go:174`): a request on an exempt path must still wait. - The place of the header limits (`internal/proxy/request.go:340`): moving them ahead of the trap paths and rule files changes the order the PR says is unchanged. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-11-05 05:02:18 +01:00
clawbot added 1 commit 2026-11-05 05:14:32 +01:00
A question waits on GeoJS at most 800ms, the new default of
SWWAF_LOOKUP_TIMEOUT. In enforce mode it does not wait at all when it is
over the header limits, on a fetched blocklist with deny, on the CrowdSec
decision list, or over a whole rate limit. Not waiting never lowers a
client's limit. Just before the Core Rule Set, servewaf refuses more than
50 cookies or 100 header fields with 431.

Shared default changed: SWWAF_LOOKUP_TIMEOUT also applies to smallwebwaf
with no command.
Judgement call: SWWAF_REPUTATION_TIMEOUT stays 2s, since no request waits
on the DNS blocklists or AbuseIPDB.

Model: opus-5-5
clawbot force-pushed issue-182-servewaf-answers-in-time from a959df50ad to 5a278949d7 2026-11-05 05:14:32 +01:00 Compare
Author
Collaborator
  1. Fixed: a part now counts unless only spaces and tabs are left, with a test.
  2. Fixed: SWWAF_UNKNOWN_LIMIT_PERCENT does not apply to a request that skips the wait and finds no answer kept, with a test.
  3. Fixed: in observe mode no request skips the wait, with a test.
  4. Fixed: SPEC.md, the commit message and the PR body name only the cases, and the list adds the three.
  5. Fixed: one test for each of the four.

Deviation: two existing tests now use a new constant on, since the linter refused a third "true".

Model: opus-5-5

1. Fixed: a part now counts unless only spaces and tabs are left, with a test. 2. Fixed: `SWWAF_UNKNOWN_LIMIT_PERCENT` does not apply to a request that skips the wait and finds no answer kept, with a test. 3. Fixed: in `observe` mode no request skips the wait, with a test. 4. Fixed: `SPEC.md`, the commit message and the PR body name only the cases, and the list adds the three. 5. Fixed: one test for each of the four. Deviation: two existing tests now use a new constant `on`, since the linter refused a third `"true"`. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-11-05 05:14:36 +01:00
Some checks are pending
check / check (push) In progress
You are not authorized to merge this pull request.
This pull request can be merged automatically.
View command line instructions

Checkout

From your project repository, check out a new branch and test the changes.
git fetch -u origin issue-182-servewaf-answers-in-time:issue-182-servewaf-answers-in-time
git checkout issue-182-servewaf-answers-in-time
Sign in to join this conversation.