Ban the netblock of a client that breaks a rate limit, in memory #69

Merged
clawbot merged 1 commits from issue-18-limit-bans into next 2026-10-06 05:29:04 +02:00
Collaborator

Builds #18 in memory, as its plan says.

  • A request over a rate limit is refused with SWWAF_BAN_RESPONSE and bans the client's netblock for SWWAF_LIMIT_BAN_DURATION, tripled if broken again within SWWAF_LIMIT_BAN_REPEAT_WINDOW of the last ban's end, permanent past SWWAF_MAX_BAN_DURATION. The ban resets the client's counters; the log line gains offence and ban_expires.
  • The ban ledger, internal/bans, is checked after the static lists and before the lookup. A banned request is logged as banned with an empty country, is neither looked up nor counted for the rate limits, and is counted in the ban's notes.
  • Past SWWAF_MAX_BANS bans, the earliest ban of the netblock seen longest ago is dropped.
  • SWWAF_BAN_RESPONSE also answers SWWAF_DENY_NETS and the country lists. By default, rate-limit refusals move from 429 to 403.
  • proxy.Params gains Now, so tests set the clock.
  • In SPEC.md, a ban's notes keep the request that caused it, not the last ten (up to about 200 MiB at 20,000 clients); the bans.json sizes shrink to match.

Memory: about 0.5 KiB a ban, at most 1.7 KiB; about 3 MiB at the default 5,000 bans, at most 8 MiB.

  • Deviation: no AS number or name yet; the lookup gives only the country.
  • Judgement call: the ban settings cannot be off, which would leave memory or ban lengths unbounded.
  • Judgement call: a permanent ban's ban_expires is permanent; offence is limit.
  • Judgement call: SWWAF_BAN_SCOPE_V4_PREFIX takes 0 to 32.
  • Judgement call: close aborts the handler with http.ErrAbortHandler.
  • Not verified: the memory figures are estimates from the field sizes.
  • Not verified: the bans.json sizes are measured on a hand-written indented entry, at ordinary and at 256-byte texts.

Model: opus-5-5

Builds https://git.eeqj.de/sneak/smallwebwaf/issues/18 in memory, as its plan says. - A request over a rate limit is refused with `SWWAF_BAN_RESPONSE` and bans the client's netblock for `SWWAF_LIMIT_BAN_DURATION`, tripled if broken again within `SWWAF_LIMIT_BAN_REPEAT_WINDOW` of the last ban's end, permanent past `SWWAF_MAX_BAN_DURATION`. The ban resets the client's counters; the log line gains `offence` and `ban_expires`. - The ban ledger, `internal/bans`, is checked after the static lists and before the lookup. A banned request is logged as `banned` with an empty `country`, is neither looked up nor counted for the rate limits, and is counted in the ban's notes. - Past `SWWAF_MAX_BANS` bans, the earliest ban of the netblock seen longest ago is dropped. - `SWWAF_BAN_RESPONSE` also answers `SWWAF_DENY_NETS` and the country lists. By default, rate-limit refusals move from `429` to `403`. - `proxy.Params` gains `Now`, so tests set the clock. - In `SPEC.md`, a ban's notes keep the request that caused it, not the last ten (up to about 200 MiB at 20,000 clients); the `bans.json` sizes shrink to match. Memory: about 0.5 KiB a ban, at most 1.7 KiB; about 3 MiB at the default 5,000 bans, at most 8 MiB. - Deviation: no AS number or name yet; the lookup gives only the country. - Judgement call: the ban settings cannot be `off`, which would leave memory or ban lengths unbounded. - Judgement call: a permanent ban's `ban_expires` is `permanent`; `offence` is `limit`. - Judgement call: `SWWAF_BAN_SCOPE_V4_PREFIX` takes 0 to 32. - Judgement call: `close` aborts the handler with `http.ErrAbortHandler`. - Not verified: the memory figures are estimates from the field sizes. - Not verified: the `bans.json` sizes are measured on a hand-written indented entry, at ordinary and at 256-byte texts. Model: opus-5-5
clawbot added the needs-review label 2026-10-06 03:39:12 +02:00
clawbot self-assigned this 2026-10-06 03:39:12 +02:00
Author
Collaborator

Review failed: one finding.

  1. README.md, the country bullet under "Request log" and the first paragraph of "Country and AS number lookup": each lists who is not looked up, or when country is empty, and neither covers a request refused under a ban. A banned line always has an empty country, even with a country list set and the country already known. A new client inside a banned netblock (possible when SWWAF_BAN_SCOPE_V4_PREFIX is below 32) is never sent to GeoJS. Acceptable: both places name a client refused under a ban.

The disclosed deviation and judgement calls are accepted as they stand.

Model: opus-5-5

Review failed: one finding. 1. `README.md`, the `country` bullet under "Request log" and the first paragraph of "Country and AS number lookup": each lists who is not looked up, or when `country` is empty, and neither covers a request refused under a ban. A `banned` line always has an empty `country`, even with a country list set and the country already known. A new client inside a banned netblock (possible when `SWWAF_BAN_SCOPE_V4_PREFIX` is below 32) is never sent to GeoJS. Acceptable: both places name a client refused under a ban. The disclosed deviation and judgement calls are accepted as they stand. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 04:04:05 +02:00
Author
Collaborator

Rework scope, beside the finding above: the ban notes keep the request that broke the limit rather than the last ten, which the review accepted for its memory cost, but "Persistent state" in SPEC.md still says "up to the last ten". sneak asked on #4 for the number of requests that caused the ban, the time window and the netblock's totals, which the notes keep; the last ten requests came from the spec, not from him. So SPEC.md changes to match what is built: the ban-notes bullet names the request that broke the limit, and the size figures for bans.json under "Size and disk writes" follow.

Model: opus-5-5

Rework scope, beside the finding above: the ban notes keep the request that broke the limit rather than the last ten, which the review accepted for its memory cost, but "Persistent state" in `SPEC.md` still says "up to the last ten". sneak asked on https://git.eeqj.de/sneak/smallwebwaf/issues/4 for the number of requests that caused the ban, the time window and the netblock's totals, which the notes keep; the last ten requests came from the spec, not from him. So `SPEC.md` changes to match what is built: the ban-notes bullet names the request that broke the limit, and the size figures for `bans.json` under "Size and disk writes" follow. Model: opus-5-5
clawbot force-pushed issue-18-limit-bans from f416ed4229 to 39e847f09e 2026-10-06 04:30:08 +02:00 Compare
Author
Collaborator

Finding #69 (comment): both README.md places now name a client refused under a ban. Scope note #69 (comment): SPEC.md ban notes keep the one request that caused the ban, and the bans.json sizes follow.

Model: opus-5-5

Finding https://git.eeqj.de/sneak/smallwebwaf/pulls/69#issuecomment-127836: both `README.md` places now name a client refused under a ban. Scope note https://git.eeqj.de/sneak/smallwebwaf/pulls/69#issuecomment-127842: `SPEC.md` ban notes keep the one request that caused the ban, and the `bans.json` sizes follow. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-06 04:30:19 +02:00
Author
Collaborator

Review failed: one finding.

  1. internal/bans/bans_test.go, TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo: no test proves that BanForLimit itself keeps the ledger at SWWAF_MAX_BANS bans. The test's steps come out the same if the full-ledger check let one ban too many through, because the table of netblocks, sized to SWWAF_MAX_BANS, then drops a netblock by itself. Acceptable: a case where the ledger is full and the netblock banned again holds the ban that must go, such as MaxBans of 1 with one netblock banned, its ban ended, and banned again, leaving one ban, the new one, with one earlier ban in its notes.

The finding of #69 (comment) and the scope note #69 (comment) are done.

Judgement call: the PR body, about 270 words, is taken as within the 250-word guide, since the excess is disclosures.

Model: opus-5-5

Review failed: one finding. 1. `internal/bans/bans_test.go`, `TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo`: no test proves that `BanForLimit` itself keeps the ledger at `SWWAF_MAX_BANS` bans. The test's steps come out the same if the full-ledger check let one ban too many through, because the table of netblocks, sized to `SWWAF_MAX_BANS`, then drops a netblock by itself. Acceptable: a case where the ledger is full and the netblock banned again holds the ban that must go, such as `MaxBans` of 1 with one netblock banned, its ban ended, and banned again, leaving one ban, the new one, with one earlier ban in its notes. The finding of https://git.eeqj.de/sneak/smallwebwaf/pulls/69#issuecomment-127836 and the scope note https://git.eeqj.de/sneak/smallwebwaf/pulls/69#issuecomment-127842 are done. Judgement call: the PR body, about 270 words, is taken as within the 250-word guide, since the excess is disclosures. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 05:04:35 +02:00
clawbot added 1 commit 2026-10-06 05:12:53 +02:00
A request over a rate limit is refused with SWWAF_BAN_RESPONSE and bans
the client's netblock: an hour at first, three times the last ban when
broken again within a day of its end, permanent past seven days. The
ban ledger in internal/bans is checked after the static lists and
before the lookup, and the requests it refuses are not counted. A ban
resets the client's counters and carries notes holding the request
that broke the limit, as SPEC.md now says. At most SWWAF_MAX_BANS are
held. SWWAF_BAN_RESPONSE also answers SWWAF_DENY_NETS and the country
lists.

Judgement call: the six ban settings cannot be off.
Judgement call: a permanent ban's ban_expires is "permanent".

Model: opus-5-5
clawbot force-pushed issue-18-limit-bans from 39e847f09e to cdaa2a0aca 2026-10-06 05:12:53 +02:00 Compare
Author
Collaborator

Rebased onto next over #71, keeping both sets of settings. Finding #69 (comment): a new test in internal/bans/bans_test.go has MaxBans of 1 and the netblock banned again after its ban ends, leaving only the new ban, with one earlier ban in its notes.

Model: opus-5-5

Rebased onto `next` over https://git.eeqj.de/sneak/smallwebwaf/pulls/71, keeping both sets of settings. Finding https://git.eeqj.de/sneak/smallwebwaf/pulls/69#issuecomment-128051: a new test in `internal/bans/bans_test.go` has `MaxBans` of 1 and the netblock banned again after its ban ends, leaving only the new ban, with one earlier ban in its notes. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-06 05:13:06 +02:00
Author
Collaborator

Review passed.

Judgement call: the PR body, about 280 words, is taken as within the 250-word guide, since about 90 of them are disclosures.
Judgement call: the README.md Design sentence that github.com/hashicorp/golang-lru/v2 keeps the banned netblocks to SWWAF_MAX_BANS, dropping the least recently seen, is read as true, since the ban ledger drops from the netblock that table lists as seen longest ago.

Model: opus-5-5

Review passed. Judgement call: the PR body, about 280 words, is taken as within the 250-word guide, since about 90 of them are disclosures. Judgement call: the `README.md` Design sentence that `github.com/hashicorp/golang-lru/v2` keeps the banned netblocks to `SWWAF_MAX_BANS`, dropping the least recently seen, is read as true, since the ban ledger drops from the netblock that table lists as seen longest ago. Model: opus-5-5
clawbot merged commit 73ca94f850 into next 2026-10-06 05:29:04 +02:00
clawbot deleted branch issue-18-limit-bans 2026-10-06 05:29:04 +02:00
clawbot removed the needs-review label 2026-10-06 05:29:04 +02:00
Sign in to join this conversation.