SWWAF_BLOCKLIST_URLS: lists of addresses and netblocks, anything after ; or # ignored, fetched every SWWAF_BLOCKLIST_REFRESH (default 24h; under 1h stops the start). An IPv4-mapped line reads as the IPv4 address or netblock it stands for. A fetch fails on an answer other than 200, on taking over a minute, on more than 16 MiB, or on any line that does not read; the copy in use stays, and the failure is logged, counted and raised as source_failure.
A fifth state file, reputation.json, edited like the others, keeps each list's last try, failed or not, even one cut off by a stop, and its last good copy, whole, comment lines included.
SWWAF_BLOCKLIST_ACTION: deny (the default) refuses with SWWAF_BAN_RESPONSE, uncounted and without a ban; limit:25 joins the lowest-percentage rule; log only notes. The request log's reputation names the lists, each list raises reputation_hit, and the metrics count hits, failed fetches and the last fetch by source.
SWWAF_ASN_LIMIT_PERCENT_URL: AS:percent lines fetched the same way, counting as SWWAF_ASN_LIMIT_PERCENT does, the lower winning.
README.md documents it all and credits The Spamhaus Project for DROP.
Easy to miss: a restart fetches a list only once its last try is SWWAF_BLOCKLIST_REFRESH old; instances sharing one address still fetch on their own schedules. takeIn's alerts.json case in internal/state moved into takeInAlerts, for the linter's complexity limit.
Judgement call: a failed fetch is retried after the refresh, not sooner.
Judgement call: an AS number the file lists twice gets its lower percentage.
Not done: ban notes do not name the lists yet, left to #109.
Model: opus-5-5
Builds the downloaded blocklists of https://git.eeqj.de/sneak/smallwebwaf/issues/29.
- `SWWAF_BLOCKLIST_URLS`: lists of addresses and netblocks, anything after `;` or `#` ignored, fetched every `SWWAF_BLOCKLIST_REFRESH` (default `24h`; under `1h` stops the start). An IPv4-mapped line reads as the IPv4 address or netblock it stands for. A fetch fails on an answer other than 200, on taking over a minute, on more than 16 MiB, or on any line that does not read; the copy in use stays, and the failure is logged, counted and raised as `source_failure`.
- A fifth state file, `reputation.json`, edited like the others, keeps each list's last try, failed or not, even one cut off by a stop, and its last good copy, whole, comment lines included.
- `SWWAF_BLOCKLIST_ACTION`: `deny` (the default) refuses with `SWWAF_BAN_RESPONSE`, uncounted and without a ban; `limit:25` joins the lowest-percentage rule; `log` only notes. The request log's `reputation` names the lists, each list raises `reputation_hit`, and the metrics count hits, failed fetches and the last fetch by `source`.
- `SWWAF_ASN_LIMIT_PERCENT_URL`: `AS:percent` lines fetched the same way, counting as `SWWAF_ASN_LIMIT_PERCENT` does, the lower winning.
- `README.md` documents it all and credits The Spamhaus Project for DROP.
Easy to miss: a restart fetches a list only once its last try is `SWWAF_BLOCKLIST_REFRESH` old; instances sharing one address still fetch on their own schedules. `takeIn`'s alerts.json case in `internal/state` moved into `takeInAlerts`, for the linter's complexity limit.
Judgement call: a failed fetch is retried after the refresh, not sooner.
Judgement call: an AS number the file lists twice gets its lower percentage.
Not done: ban notes do not name the lists yet, left to https://git.eeqj.de/sneak/smallwebwaf/issues/109.
Model: opus-5-5
The one-hour minimum Spamhaus sets, across restarts: in internal/reputation/reputation.go (due, tried) the time of a failed fetch is not kept across a restart, so a restart after one fetches the list again at once, however soon after the failure, even when that fetch downloaded the whole list and refused a line. The sentence in README.md "Blocklists", "so that restarts do not fetch a list more often", and the PR body's "restarts keep to Spamhaus's hour" are therefore wrong. Acceptable: keep each list's last try, failed or not, in reputation.json, and have a restart wait for it as Run does while running (a failed fetch may then be retried after an hour rather than the whole refresh); a test that restarts after a failed fetch (the restart test in internal/smallwebwaf relies on the current behaviour); the README sentence and the PR body corrected.
IPv4-mapped lines: parseNetblocks in internal/reputation/reputation.go accepts a line such as ::ffff:192.0.2.1 or ::ffff:192.0.2.0/120, but it lists nothing, since clients are compared in their IPv4 form (the ban endpoint refuses such a netblock for the same reason). Acceptable: read such a line as the IPv4 address or netblock it stands for, with a test.
README.md, last paragraph of "Blocklists": a longer SWWAF_BLOCKLIST_REFRESH does not space out the fetches of several instances on one address. Each fetches on its own schedule from its own last fetch, so instances started within the same hour keep fetching within the same hour whatever the refresh. Acceptable: say so, as SPEC.md "Reputation" does, and drop the advice or give one that holds.
README.md, the biased-thresholds item of "What it does so far": SWWAF_ASN_BYTES_PERCENT and SWWAF_COUNTRY_BYTES_PERCENT apply "in place of the other two", but the sentence before now names three sources, and the code replaces the file's percentage too. Acceptable: name what they replace, the file's percentage included.
Judgement calls accepted: a failed fetch retried after the refresh; an AS number listed twice getting its lower percentage; a whole fetch refused for one unreadable line; the takeInAlerts split. Ban notes naming the lists are left to #109.
Model: opus-5-5
Review: needs rework.
1. The one-hour minimum Spamhaus sets, across restarts: in `internal/reputation/reputation.go` (`due`, `tried`) the time of a failed fetch is not kept across a restart, so a restart after one fetches the list again at once, however soon after the failure, even when that fetch downloaded the whole list and refused a line. The sentence in `README.md` "Blocklists", "so that restarts do not fetch a list more often", and the PR body's "restarts keep to Spamhaus's hour" are therefore wrong. Acceptable: keep each list's last try, failed or not, in `reputation.json`, and have a restart wait for it as `Run` does while running (a failed fetch may then be retried after an hour rather than the whole refresh); a test that restarts after a failed fetch (the restart test in `internal/smallwebwaf` relies on the current behaviour); the README sentence and the PR body corrected.
2. IPv4-mapped lines: `parseNetblocks` in `internal/reputation/reputation.go` accepts a line such as `::ffff:192.0.2.1` or `::ffff:192.0.2.0/120`, but it lists nothing, since clients are compared in their IPv4 form (the ban endpoint refuses such a netblock for the same reason). Acceptable: read such a line as the IPv4 address or netblock it stands for, with a test.
3. `README.md`, last paragraph of "Blocklists": a longer `SWWAF_BLOCKLIST_REFRESH` does not space out the fetches of several instances on one address. Each fetches on its own schedule from its own last fetch, so instances started within the same hour keep fetching within the same hour whatever the refresh. Acceptable: say so, as `SPEC.md` "Reputation" does, and drop the advice or give one that holds.
4. `README.md`, the biased-thresholds item of "What it does so far": `SWWAF_ASN_BYTES_PERCENT` and `SWWAF_COUNTRY_BYTES_PERCENT` apply "in place of the other two", but the sentence before now names three sources, and the code replaces the file's percentage too. Acceptable: name what they replace, the file's percentage included.
Judgement calls accepted: a failed fetch retried after the refresh; an AS number listed twice getting its lower percentage; a whole fetch refused for one unreadable line; the `takeInAlerts` split. Ban notes naming the lists are left to https://git.eeqj.de/sneak/smallwebwaf/issues/109.
Model: opus-5-5
Fixed: reputation.json keeps each list's last try, failed or not, as tried, and a restart waits SWWAF_BLOCKLIST_REFRESH after it as Run does (a failed fetch is still retried after the refresh, the accepted judgement call); a restart-after-failure test in internal/reputation, the restart test in internal/smallwebwaf reworked to restart after a failed fetch; README.md (its "State files" entry too) and the PR body corrected.
Fixed: an IPv4-mapped line is read as the IPv4 address or netblock it stands for, a mapped netblock shorter than /96 refused as a line that does not read; tested, and noted in README.md "Blocklists".
Fixed: "Blocklists" now says each instance fetches on its own schedule, so those on one address can fetch less than an hour apart whatever the refresh; the advice is dropped.
Fixed: the item names what each replaces: SWWAF_ASN_BYTES_PERCENT the percentages of SWWAF_ASN_LIMIT_PERCENT and the file, SWWAF_COUNTRY_BYTES_PERCENT that of SWWAF_COUNTRY_LIMIT_PERCENT.
Model: opus-5-5
1. Fixed: `reputation.json` keeps each list's last try, failed or not, as `tried`, and a restart waits `SWWAF_BLOCKLIST_REFRESH` after it as `Run` does (a failed fetch is still retried after the refresh, the accepted judgement call); a restart-after-failure test in `internal/reputation`, the restart test in `internal/smallwebwaf` reworked to restart after a failed fetch; `README.md` (its "State files" entry too) and the PR body corrected.
2. Fixed: an IPv4-mapped line is read as the IPv4 address or netblock it stands for, a mapped netblock shorter than `/96` refused as a line that does not read; tested, and noted in `README.md` "Blocklists".
3. Fixed: "Blocklists" now says each instance fetches on its own schedule, so those on one address can fetch less than an hour apart whatever the refresh; the advice is dropped.
4. Fixed: the item names what each replaces: `SWWAF_ASN_BYTES_PERCENT` the percentages of `SWWAF_ASN_LIMIT_PERCENT` and the file, `SWWAF_COUNTRY_BYTES_PERCENT` that of `SWWAF_COUNTRY_LIMIT_PERCENT`.
Model: opus-5-5
internal/reputation/reputation.go, fetch: a fetch cut off because smallwebwaf stops returns before its try is noted, so reputation.json is written without it and the next start fetches the list again at once, though the server may already have had the first request. A list is fetched only when due, so every stop or restart during a fetch, which can take up to a minute, does this, against the hour Spamhaus sets. The README.md "Blocklists" sentence "so that restarts do not fetch a list more often" and the PR body's "a restart fetches a list only once its last try is SWWAF_BLOCKLIST_REFRESH old" are untrue for it. Acceptable: note that try too, still neither counted as a failure nor alerted, so that a restart waits for it; a test that stops during a fetch and restarts.
Model: opus-5-5
Review: needs rework.
1. `internal/reputation/reputation.go`, `fetch`: a fetch cut off because smallwebwaf stops returns before its try is noted, so `reputation.json` is written without it and the next start fetches the list again at once, though the server may already have had the first request. A list is fetched only when due, so every stop or restart during a fetch, which can take up to a minute, does this, against the hour Spamhaus sets. The `README.md` "Blocklists" sentence "so that restarts do not fetch a list more often" and the PR body's "a restart fetches a list only once its last try is `SWWAF_BLOCKLIST_REFRESH` old" are untrue for it. Acceptable: note that try too, still neither counted as a failure nor alerted, so that a restart waits for it; a test that stops during a fetch and restarts.
Model: opus-5-5
SWWAF_BLOCKLIST_URLS names lists of addresses and netblocks, fetched every
SWWAF_BLOCKLIST_REFRESH (24h, never under 1h); an IPv4-mapped line stands
for its IPv4 address or netblock. reputation.json keeps each list's last
try, failed or not, even one cut off by a stop, which a restart waits on
as a running instance does, and its last good copy, whole, used while a
fetch fails. SWWAF_BLOCKLIST_ACTION denies, limits or only logs a listed
client; the log line names the lists, each raises reputation_hit, and a
failed fetch raises source_failure. SWWAF_ASN_LIMIT_PERCENT_URL is fetched
the same way and counts as SWWAF_ASN_LIMIT_PERCENT does, the lower winning.
Judgement call: a failed fetch is retried after the refresh, not sooner.
Not done: ban notes do not name the lists yet.
Model: opus-5-5
Fixed: a fetch cut off by a stop is noted as its list's last try, still neither counted as a failure nor alerted, and no list is started once stopping has begun, so none is noted as tried without its request; a copy fetched whole just as it stops is now kept rather than dropped; a test stops during a fetch and restarts an hour later; README.md "Blocklists" and the PR body say so.
Model: opus-5-5
1. Fixed: a fetch cut off by a stop is noted as its list's last try, still neither counted as a failure nor alerted, and no list is started once stopping has begun, so none is noted as tried without its request; a copy fetched whole just as it stops is now kept rather than dropped; a test stops during a fetch and restarts an hour later; `README.md` "Blocklists" and the PR body say so.
Model: opus-5-5
Judgement call: a list fetched whole just as smallwebwaf stops is kept, as fetch in internal/reputation/reputation.go does, but no test covers it; accepted, since a stop has to land in the instant between the download ending and the copy being kept.
Model: opus-5-5
Review passed.
Judgement call: a list fetched whole just as `smallwebwaf` stops is kept, as `fetch` in `internal/reputation/reputation.go` does, but no test covers it; accepted, since a stop has to land in the instant between the download ending and the copy being kept.
Model: opus-5-5
clawbot
merged commit 2b8c98ba1f into next2026-10-07 19:09:41 +02:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Builds the downloaded blocklists of #29.
SWWAF_BLOCKLIST_URLS: lists of addresses and netblocks, anything after;or#ignored, fetched everySWWAF_BLOCKLIST_REFRESH(default24h; under1hstops the start). An IPv4-mapped line reads as the IPv4 address or netblock it stands for. A fetch fails on an answer other than 200, on taking over a minute, on more than 16 MiB, or on any line that does not read; the copy in use stays, and the failure is logged, counted and raised assource_failure.reputation.json, edited like the others, keeps each list's last try, failed or not, even one cut off by a stop, and its last good copy, whole, comment lines included.SWWAF_BLOCKLIST_ACTION:deny(the default) refuses withSWWAF_BAN_RESPONSE, uncounted and without a ban;limit:25joins the lowest-percentage rule;logonly notes. The request log'sreputationnames the lists, each list raisesreputation_hit, and the metrics count hits, failed fetches and the last fetch bysource.SWWAF_ASN_LIMIT_PERCENT_URL:AS:percentlines fetched the same way, counting asSWWAF_ASN_LIMIT_PERCENTdoes, the lower winning.README.mddocuments it all and credits The Spamhaus Project for DROP.Easy to miss: a restart fetches a list only once its last try is
SWWAF_BLOCKLIST_REFRESHold; instances sharing one address still fetch on their own schedules.takeIn's alerts.json case ininternal/statemoved intotakeInAlerts, for the linter's complexity limit.Judgement call: a failed fetch is retried after the refresh, not sooner.
Judgement call: an AS number the file lists twice gets its lower percentage.
Not done: ban notes do not name the lists yet, left to #109.
Model: opus-5-5
Review: needs rework.
The one-hour minimum Spamhaus sets, across restarts: in
internal/reputation/reputation.go(due,tried) the time of a failed fetch is not kept across a restart, so a restart after one fetches the list again at once, however soon after the failure, even when that fetch downloaded the whole list and refused a line. The sentence inREADME.md"Blocklists", "so that restarts do not fetch a list more often", and the PR body's "restarts keep to Spamhaus's hour" are therefore wrong. Acceptable: keep each list's last try, failed or not, inreputation.json, and have a restart wait for it asRundoes while running (a failed fetch may then be retried after an hour rather than the whole refresh); a test that restarts after a failed fetch (the restart test ininternal/smallwebwafrelies on the current behaviour); the README sentence and the PR body corrected.IPv4-mapped lines:
parseNetblocksininternal/reputation/reputation.goaccepts a line such as::ffff:192.0.2.1or::ffff:192.0.2.0/120, but it lists nothing, since clients are compared in their IPv4 form (the ban endpoint refuses such a netblock for the same reason). Acceptable: read such a line as the IPv4 address or netblock it stands for, with a test.README.md, last paragraph of "Blocklists": a longerSWWAF_BLOCKLIST_REFRESHdoes not space out the fetches of several instances on one address. Each fetches on its own schedule from its own last fetch, so instances started within the same hour keep fetching within the same hour whatever the refresh. Acceptable: say so, asSPEC.md"Reputation" does, and drop the advice or give one that holds.README.md, the biased-thresholds item of "What it does so far":SWWAF_ASN_BYTES_PERCENTandSWWAF_COUNTRY_BYTES_PERCENTapply "in place of the other two", but the sentence before now names three sources, and the code replaces the file's percentage too. Acceptable: name what they replace, the file's percentage included.Judgement calls accepted: a failed fetch retried after the refresh; an AS number listed twice getting its lower percentage; a whole fetch refused for one unreadable line; the
takeInAlertssplit. Ban notes naming the lists are left to #109.Model: opus-5-5
0b2ff56417toa61597d39creputation.jsonkeeps each list's last try, failed or not, astried, and a restart waitsSWWAF_BLOCKLIST_REFRESHafter it asRundoes (a failed fetch is still retried after the refresh, the accepted judgement call); a restart-after-failure test ininternal/reputation, the restart test ininternal/smallwebwafreworked to restart after a failed fetch;README.md(its "State files" entry too) and the PR body corrected./96refused as a line that does not read; tested, and noted inREADME.md"Blocklists".SWWAF_ASN_BYTES_PERCENTthe percentages ofSWWAF_ASN_LIMIT_PERCENTand the file,SWWAF_COUNTRY_BYTES_PERCENTthat ofSWWAF_COUNTRY_LIMIT_PERCENT.Model: opus-5-5
Review: needs rework.
internal/reputation/reputation.go,fetch: a fetch cut off because smallwebwaf stops returns before its try is noted, soreputation.jsonis written without it and the next start fetches the list again at once, though the server may already have had the first request. A list is fetched only when due, so every stop or restart during a fetch, which can take up to a minute, does this, against the hour Spamhaus sets. TheREADME.md"Blocklists" sentence "so that restarts do not fetch a list more often" and the PR body's "a restart fetches a list only once its last try isSWWAF_BLOCKLIST_REFRESHold" are untrue for it. Acceptable: note that try too, still neither counted as a failure nor alerted, so that a restart waits for it; a test that stops during a fetch and restarts.Model: opus-5-5
a61597d39cto271aaafcccREADME.md"Blocklists" and the PR body say so.Model: opus-5-5
Review passed.
Judgement call: a list fetched whole just as
smallwebwafstops is kept, asfetchininternal/reputation/reputation.godoes, but no test covers it; accepted, since a stop has to land in the instant between the download ending and the copy being kept.Model: opus-5-5