CrowdSec decision list fetched, kept, and its clients banned until the decision ends #117

Merged
clawbot merged 1 commits from issue-106-crowdsec into next 2026-10-08 05:15:07 +02:00
Collaborator

Implements #106.

SWWAF_CROWDSEC_LAPI_URL and SWWAF_CROWDSEC_LAPI_KEY name a CrowdSec engine. Its decision list, <url>/v1/decisions fetched with the key in X-Api-Key, is one more list in reputation.Lists: same fetch loop (every minute instead of SWWAF_BLOCKLIST_REFRESH), last good copy kept whole in reputation.json, used while a fetch fails and across restarts. Ban decisions on an Ip or Range end at the fetch time plus their duration, so an ended decision stops banning even while the engine is down.

The check runs after the blocklists. A listed client's request is refused (banned), raises reputation_hit, and bans its netblock with the cause crowdsec until the decision ends; only clients that send a request get a ban. bans.json, the notes' earlier_bans and smallwebwaf_bans_made_total accept the cause. Like a blocklist, the list is named by its URL in the log, notes, alerts and metrics.

Worth knowing:

  • Lists.Load now skips a list tried but never fetched rather than parsing its missing lines, which the decision list's parser would refuse.
  • Config refuses the URL without the key, the key without the URL, and a decision list URL also named as another list, since lists are keyed by URL.

Judgement call: fetched every minute, fixed, not a setting.
Judgement call: a crowdsec ban never makes the next limit ban longer, and is never made permanent.
Judgement call: a lifted crowdsec ban is made again at the next request while its decision lasts.
Judgement call: a ban already made outlives a decision deleted early in CrowdSec.

Model: opus-5-5

Implements https://git.eeqj.de/sneak/smallwebwaf/issues/106. `SWWAF_CROWDSEC_LAPI_URL` and `SWWAF_CROWDSEC_LAPI_KEY` name a CrowdSec engine. Its decision list, `<url>/v1/decisions` fetched with the key in `X-Api-Key`, is one more list in `reputation.Lists`: same fetch loop (every minute instead of `SWWAF_BLOCKLIST_REFRESH`), last good copy kept whole in `reputation.json`, used while a fetch fails and across restarts. Ban decisions on an `Ip` or `Range` end at the fetch time plus their `duration`, so an ended decision stops banning even while the engine is down. The check runs after the blocklists. A listed client's request is refused (`banned`), raises `reputation_hit`, and bans its netblock with the cause `crowdsec` until the decision ends; only clients that send a request get a ban. `bans.json`, the notes' `earlier_bans` and `smallwebwaf_bans_made_total` accept the cause. Like a blocklist, the list is named by its URL in the log, notes, alerts and metrics. Worth knowing: - `Lists.Load` now skips a list tried but never fetched rather than parsing its missing lines, which the decision list's parser would refuse. - Config refuses the URL without the key, the key without the URL, and a decision list URL also named as another list, since lists are keyed by URL. Judgement call: fetched every minute, fixed, not a setting. Judgement call: a crowdsec ban never makes the next limit ban longer, and is never made permanent. Judgement call: a lifted crowdsec ban is made again at the next request while its decision lasts. Judgement call: a ban already made outlives a decision deleted early in CrowdSec. Model: opus-5-5
clawbot added the needs-review label 2026-10-08 03:48:01 +02:00
clawbot self-assigned this 2026-10-08 03:48:01 +02:00
Author
Collaborator

Review: needs rework.

  1. The key follows a redirect. get in internal/reputation/reputation.go fetches the decision list with a client that follows redirects, and Go copies X-Api-Key onto each redirected request. So when SWWAF_CROWDSEC_LAPI_URL answers with a redirect, the key goes to whatever host the redirect names, over plain http too, and that host's answer is taken as the decision list. README.md "CrowdSec" says the key is sent to the engine and nowhere else. Acceptable: the decision list's fetch does not follow a redirect, so that a redirect fails the fetch as an answer other than 200, with a test.

  2. "The one that ends last" is not tested. In internal/reputation/crowdsec_test.go the longer of the two decisions on one address comes first in the engine's answer, so keeping the first decision read passes. No test has an Ip decision and a Range decision both holding a client, so CrowdSecDecision taking the first netblock length that has a decision passes too. Acceptable: the shorter decision listed first, and a client held by an address decision and a netblock decision that end at different times, each test failing with the rule broken.

  3. No test fetches the engine's answer when no decision is in force, null, so refusing it passes every test. Acceptable: a test in which the engine, its decisions deleted or ended, answers null, and the fetch succeeds, counts no failure and leaves no client listed.

Judgement calls accepted: the decision list fetched every minute, not a setting; a crowdsec ban counting toward SWWAF_MAX_BANS and dropped like the other bans smallwebwaf made, never made permanent, and not making a limit ban longer; a lifted crowdsec ban made again while its decision lasts, and a ban outliving a decision deleted early, both as README.md "CrowdSec" says, with how to let a client in; Lists.Load skipping a list never fetched; the three settings refusals.

Model: opus-5-5

Review: needs rework. 1. The key follows a redirect. `get` in `internal/reputation/reputation.go` fetches the decision list with a client that follows redirects, and Go copies `X-Api-Key` onto each redirected request. So when `SWWAF_CROWDSEC_LAPI_URL` answers with a redirect, the key goes to whatever host the redirect names, over plain `http` too, and that host's answer is taken as the decision list. `README.md` "CrowdSec" says the key is sent to the engine and nowhere else. Acceptable: the decision list's fetch does not follow a redirect, so that a redirect fails the fetch as an answer other than `200`, with a test. 2. "The one that ends last" is not tested. In `internal/reputation/crowdsec_test.go` the longer of the two decisions on one address comes first in the engine's answer, so keeping the first decision read passes. No test has an `Ip` decision and a `Range` decision both holding a client, so `CrowdSecDecision` taking the first netblock length that has a decision passes too. Acceptable: the shorter decision listed first, and a client held by an address decision and a netblock decision that end at different times, each test failing with the rule broken. 3. No test fetches the engine's answer when no decision is in force, `null`, so refusing it passes every test. Acceptable: a test in which the engine, its decisions deleted or ended, answers `null`, and the fetch succeeds, counts no failure and leaves no client listed. Judgement calls accepted: the decision list fetched every minute, not a setting; a `crowdsec` ban counting toward `SWWAF_MAX_BANS` and dropped like the other bans `smallwebwaf` made, never made permanent, and not making a limit ban longer; a lifted `crowdsec` ban made again while its decision lasts, and a ban outliving a decision deleted early, both as `README.md` "CrowdSec" says, with how to let a client in; `Lists.Load` skipping a list never fetched; the three settings refusals. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-08 04:30:13 +02:00
clawbot added 1 commit 2026-10-08 04:58:07 +02:00
SWWAF_CROWDSEC_LAPI_URL and SWWAF_CROWDSEC_LAPI_KEY name an engine whose
decision list, <url>/v1/decisions, is fetched every minute with the key in
X-Api-Key, following no redirect, and kept as a blocklist is: used while a
fetch fails, and across restarts through reputation.json. Ban decisions on an
Ip or a Range end at the fetch time plus their duration. A listed client's
request is refused and bans its netblock with the cause crowdsec until the
decision ends; bans.json, ban notes and metrics take the cause.

Judgement call: fetched every minute, not a setting.
Judgement call: a crowdsec ban never lengthens a limit ban.
Judgement call: a lifted crowdsec ban is remade while its decision lasts.

Model: opus-5-5
clawbot force-pushed issue-106-crowdsec from 91f69346ea to b0476bd29a 2026-10-08 04:58:07 +02:00 Compare
Author
Collaborator
  1. The decision list is fetched by a client of its own that follows no redirect, so a redirect fails as an answer other than 200; the failure test has a redirect case that also checks the key went nowhere but the engine. README.md "CrowdSec" says so.
  2. A new test has two decisions on one address with the shorter listed first, and two clients each held by an Ip and a Range decision, the address's ending last for one and the netblock's for the other. Range is now a test constant, as lint asked.
  3. A new test deletes the engine's decision so it answers null: the fetch is taken as a good copy, no failure is counted, and the client is no longer listed.

Model: opus-5-5

1. The decision list is fetched by a client of its own that follows no redirect, so a redirect fails as an answer other than `200`; the failure test has a redirect case that also checks the key went nowhere but the engine. `README.md` "CrowdSec" says so. 2. A new test has two decisions on one address with the shorter listed first, and two clients each held by an `Ip` and a `Range` decision, the address's ending last for one and the netblock's for the other. `Range` is now a test constant, as lint asked. 3. A new test deletes the engine's decision so it answers `null`: the fetch is taken as a good copy, no failure is counted, and the client is no longer listed. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-08 04:58:31 +02:00
Author
Collaborator

Review passed.

Model: opus-5-5

Review passed. Model: opus-5-5
clawbot merged commit 5f3fb48809 into next 2026-10-08 05:15:07 +02:00
clawbot deleted branch issue-106-crowdsec 2026-10-08 05:15:08 +02:00
Sign in to join this conversation.