Compare commits
1
Commits
next
..
896ca69a82
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
896ca69a82 |
No files matched your search
+44
-56
@@ -12,8 +12,8 @@ configured only by environment variables, that provides:
|
|||||||
- R1: rate limits per minute, per hour and per day
|
- R1: rate limits per minute, per hour and per day
|
||||||
- R2: netblocks that bypass rate limiting
|
- R2: netblocks that bypass rate limiting
|
||||||
- R3: alerts on attacks through webhook, Slack or ntfy
|
- R3: alerts on attacks through webhook, Slack or ntfy
|
||||||
- R4: alerts on anomalies: requests made or bytes transferred per minute or
|
- R4: alerts on anomalies: requests or bytes per minute or hour, for given IPs
|
||||||
hour, for given IPs or netblocks, crossing a threshold
|
or netblocks, crossing a threshold
|
||||||
- R5: temporary blocks for abusers, permanent bans for repeat offenders
|
- R5: temporary blocks for abusers, permanent bans for repeat offenders
|
||||||
- R6: one or more RBLs or IP reputation APIs
|
- R6: one or more RBLs or IP reputation APIs
|
||||||
- R7: AS number lookup
|
- R7: AS number lookup
|
||||||
@@ -24,19 +24,14 @@ configured only by environment variables, that provides:
|
|||||||
|
|
||||||
## Verdict
|
## Verdict
|
||||||
|
|
||||||
Nothing existing is suitable. No surveyed tool meets R8, which asks for limits
|
Nothing existing is suitable. No surveyed tool meets R8 (scaling limits by AS
|
||||||
scaled by AS number or country. None meets R4's alerts on bytes transferred. The
|
number or country) or the bytes half of R4 at all, and the tools that come
|
||||||
tools that come closest on the rest fail R10: they need several containers, YAML
|
closest on the rest fail R10: they need several containers, YAML or a web UI.
|
||||||
or a web UI. The recommendation is to build a small single-binary sidecar;
|
The recommendation is to build a small single-binary sidecar; `SPEC.md` in this
|
||||||
`SPEC.md` in this directory begins that spec.
|
directory begins that spec. It reuses the mature parts of the surveyed field as
|
||||||
|
libraries and data sources (Coraza and the OWASP Core Rule Set for attack
|
||||||
The sidecar reuses the mature parts of the surveyed field instead of
|
detection, DNSBLs, AbuseIPDB, optionally a CrowdSec decision feed) instead of
|
||||||
reimplementing them. It takes these as libraries and data sources:
|
reimplementing them.
|
||||||
|
|
||||||
- Coraza and the OWASP Core Rule Set, for attack detection
|
|
||||||
- DNSBLs
|
|
||||||
- AbuseIPDB
|
|
||||||
- optionally, a CrowdSec decision feed
|
|
||||||
|
|
||||||
Closest existing options, and why each still falls short:
|
Closest existing options, and why each still falls short:
|
||||||
|
|
||||||
@@ -50,16 +45,13 @@ Closest existing options, and why each still falls short:
|
|||||||
external blacklist URLs, optional CrowdSec), R7 partly (AS number used for
|
external blacklist URLs, optional CrowdSec), R7 partly (AS number used for
|
||||||
blacklist and whitelist decisions), R9 (ModSecurity with the Core Rule
|
blacklist and whitelist decisions), R9 (ModSecurity with the Core Rule
|
||||||
Set, or Coraza plugin), env-var settings.
|
Set, or Coraza plugin), env-var settings.
|
||||||
- Fails:
|
- Fails: R8 (AS number and country can only allow or deny, never scale a
|
||||||
- R8: AS number and country can only allow or deny, never scale a limit.
|
limit), R4 (no volume or byte threshold alerts in the free edition;
|
||||||
- R4: no alerts on request-count or byte thresholds in the free edition;
|
reporting is a paid feature), R5 escalation, and R10 in spirit: since 1.6
|
||||||
reporting is a paid feature.
|
it needs a `bunkerweb` container plus a `bw-scheduler` container and a
|
||||||
- R5: no escalation for repeat offenders.
|
database, or the all-in-one image that bundles nginx, scheduler, UI and
|
||||||
- R10: since 1.6 it is not one plain sidecar. It needs a `bunkerweb`
|
Redis in one container. It is designed to be the front door for many
|
||||||
container, a `bw-scheduler` container and a database, or the
|
sites, not a per-app sidecar. AGPL-3.0.
|
||||||
all-in-one image that bundles nginx, the scheduler, the UI and Redis.
|
|
||||||
- It is designed as the proxy in front of many sites, not as a sidecar for
|
|
||||||
one app. AGPL-3.0.
|
|
||||||
- Unverified: whether several rates (minute, hour, day) can be stacked on
|
- Unverified: whether several rates (minute, hour, day) can be stacked on
|
||||||
the same URL pattern; the documented design is one rate per URL pattern.
|
the same URL pattern; the documented design is one rate per URL pattern.
|
||||||
- CrowdSec is the strongest detection and ban engine, but it is not a proxy.
|
- CrowdSec is the strongest detection and ban engine, but it is not a proxy.
|
||||||
@@ -74,20 +66,16 @@ Closest existing options, and why each still falls short:
|
|||||||
listed AS numbers is possible, but each is a hand-written YAML scenario,
|
listed AS numbers is possible, but each is a hand-written YAML scenario,
|
||||||
it reacts after the fact by banning, and it is not an inline limiter that
|
it reacts after the fact by banning, and it is not an inline limiter that
|
||||||
answers 429.
|
answers 429.
|
||||||
- Fails:
|
- Fails: R10 (needs the security engine container with persistent state, log
|
||||||
- R10, because it needs:
|
acquisition from traefik, a bouncer such as the traefik plugin, and YAML
|
||||||
- the security engine container with persistent state
|
for acquisition, profiles, scenarios and notifications), bytes half of R4.
|
||||||
- log acquisition from traefik
|
|
||||||
- a bouncer such as the traefik plugin
|
|
||||||
- YAML for acquisition, profiles, scenarios and notifications
|
|
||||||
- R4's alerts on bytes.
|
|
||||||
- CrowdSec plus traefik's own `rateLimit` middleware is the best combination
|
- CrowdSec plus traefik's own `rateLimit` middleware is the best combination
|
||||||
with no new code. It gives inline limiting (one window per middleware, keyed
|
with no new code. It gives inline limiting (one window per middleware, keyed
|
||||||
by IP, with `sourceCriterion` exclusions), bans with escalation, alerts and
|
by IP, with `sourceCriterion` exclusions), bans with escalation, alerts and
|
||||||
reputation lookups. It still fails R8, R4's alerts on bytes, and R10 entirely:
|
reputation. It still fails R8, the bytes half of R4, R1's three stacked
|
||||||
the configuration lives in traefik labels and CrowdSec YAML on each host, not
|
windows only by chaining three middlewares per router, and R10 entirely: the
|
||||||
in one sidecar's environment. It meets R1's three windows only by chaining
|
configuration lives in traefik labels and CrowdSec YAML on each host, not in
|
||||||
three middlewares per router.
|
one sidecar's environment.
|
||||||
|
|
||||||
## Candidate by candidate
|
## Candidate by candidate
|
||||||
|
|
||||||
@@ -99,10 +87,10 @@ Closest existing options, and why each still falls short:
|
|||||||
cached, streamed decision list (recommended), standalone against the
|
cached, streamed decision list (recommended), standalone against the
|
||||||
central API, or AppSec only. Supports captcha remediation.
|
central API, or AppSec only. Supports captcha remediation.
|
||||||
- Covers: R5, R6, R9, R3 and R7 through the engine (see Verdict).
|
- Covers: R5, R6, R9, R3 and R7 through the engine (see Verdict).
|
||||||
- Misses: the plugin itself does no rate limiting; R8; R4's alerts on bytes.
|
- Misses: the plugin itself does no rate limiting; R8; R4 bytes.
|
||||||
- Configuration: traefik static config to load the plugin, dynamic config or
|
- Configuration: traefik static config to load the plugin, dynamic config or
|
||||||
labels for the middleware, CrowdSec YAML for everything else.
|
labels for the middleware, CrowdSec YAML for everything else.
|
||||||
- Sidecar fit: no. It runs inside traefik and needs a separate engine
|
- Sidecar fit: no. It lives inside traefik, plus a separate engine
|
||||||
container.
|
container.
|
||||||
- Maturity: widely used (about 900 stars, listed in the traefik plugin
|
- Maturity: widely used (about 900 stars, listed in the traefik plugin
|
||||||
catalog, documented by CrowdSec itself), actively maintained. Traefik
|
catalog, documented by CrowdSec itself), actively maintained. Traefik
|
||||||
@@ -127,7 +115,7 @@ Closest existing options, and why each still falls short:
|
|||||||
- What it is: nginx-derived (Tengine) reverse proxy with a proprietary
|
- What it is: nginx-derived (Tengine) reverse proxy with a proprietary
|
||||||
semantic detection engine and a web console. About 22,000 stars.
|
semantic detection engine and a web console. About 22,000 stars.
|
||||||
- Covers: R9 well, basic per-IP rate limiting with block duration, bot
|
- Covers: R9 well, basic per-IP rate limiting with block duration, bot
|
||||||
challenge, authentication in front of the app.
|
challenge, authentication gate.
|
||||||
- Misses: R8, R7, R4; R1 has one window per rule. Country blocking, alert
|
- Misses: R8, R7, R4; R1 has one window per rule. Country blocking, alert
|
||||||
messages, syslog forwarding and the better reputation database are paid
|
messages, syslog forwarding and the better reputation database are paid
|
||||||
(Lite about 10 USD per month, Pro about 100 USD per month); the free
|
(Lite about 10 USD per month, Pro about 100 USD per month); the free
|
||||||
@@ -140,7 +128,7 @@ Closest existing options, and why each still falls short:
|
|||||||
- Coraza (`corazawaf/coraza`) and Coraza-based proxies
|
- Coraza (`corazawaf/coraza`) and Coraza-based proxies
|
||||||
- What it is: a Go library that implements the ModSecurity rule language and
|
- What it is: a Go library that implements the ModSecurity rule language and
|
||||||
runs the OWASP Core Rule Set. OWASP project, actively maintained, the
|
runs the OWASP Core Rule Set. OWASP project, actively maintained, the
|
||||||
successor to ModSecurity, which is now in maintenance only.
|
successor path now that ModSecurity is in maintenance only.
|
||||||
- Packagings: `coraza-caddy` (Caddy module), `coraza-spoa` (HAProxy),
|
- Packagings: `coraza-caddy` (Caddy module), `coraza-spoa` (HAProxy),
|
||||||
`coraza-proxy-wasm` (Envoy), a traefik WASM plugin, and
|
`coraza-proxy-wasm` (Envoy), a traefik WASM plugin, and
|
||||||
`coreruleset/coraza-crs-docker` (Caddy plus Coraza plus the Core Rule Set,
|
`coreruleset/coraza-crs-docker` (Caddy plus Coraza plus the Core Rule Set,
|
||||||
@@ -154,7 +142,7 @@ Closest existing options, and why each still falls short:
|
|||||||
- ModSecurity Core Rule Set containers (`owasp/modsecurity-crs`)
|
- ModSecurity Core Rule Set containers (`owasp/modsecurity-crs`)
|
||||||
- What it is: official images of Apache or nginx with ModSecurity and the
|
- What it is: official images of Apache or nginx with ModSecurity and the
|
||||||
Core Rule Set, as a reverse proxy configured by env vars (`BACKEND`,
|
Core Rule Set, as a reverse proxy configured by env vars (`BACKEND`,
|
||||||
paranoia level, anomaly thresholds, many more). Current images contain
|
paranoia level, anomaly thresholds, many more). Current images carry
|
||||||
ModSecurity 2.9.x on Apache and Core Rule Set 4.x.
|
ModSecurity 2.9.x on Apache and Core Rule Set 4.x.
|
||||||
- Covers: R9, and R10 (single container, env vars, one backend).
|
- Covers: R9, and R10 (single container, env vars, one backend).
|
||||||
- Misses: R1 to R8.
|
- Misses: R1 to R8.
|
||||||
@@ -164,8 +152,8 @@ Closest existing options, and why each still falls short:
|
|||||||
- Anubis (`TecharoHQ/anubis`), 1.27 current
|
- Anubis (`TecharoHQ/anubis`), 1.27 current
|
||||||
- What it is: a single-binary reverse proxy that makes browsers solve a
|
- What it is: a single-binary reverse proxy that makes browsers solve a
|
||||||
proof-of-work challenge before passing them to `TARGET`. Aimed at
|
proof-of-work challenge before passing them to `TARGET`. Aimed at
|
||||||
scrapers, which are likely a large share of the unwanted traffic on a
|
scrapers, which is likely a large share of unwanted traffic on a public
|
||||||
public code forge.
|
code forge.
|
||||||
- Covers: R10 well (one container, env vars for listener, target,
|
- Covers: R10 well (one container, env vars for listener, target,
|
||||||
difficulty, cookies). Policy rules can match on path, user agent, headers,
|
difficulty, cookies). Policy rules can match on path, user agent, headers,
|
||||||
IP ranges, and with the vendor's hosted data service also AS number and
|
IP ranges, and with the vendor's hosted data service also AS number and
|
||||||
@@ -192,13 +180,13 @@ Closest existing options, and why each still falls short:
|
|||||||
temporary ban, no escalation, state lost on traefik restart) and R2
|
temporary ban, no escalation, state lost on traefik restart) and R2
|
||||||
(allowlist). Nothing else.
|
(allowlist). Nothing else.
|
||||||
- `juitde/traefik-plugin-fail2ban`, fail2ban connector plugins, GeoBlock
|
- `juitde/traefik-plugin-fail2ban`, fail2ban connector plugins, GeoBlock
|
||||||
plugins (country allow or deny through an external lookup API): each adds
|
plugins (country allow or deny through an external lookup API): same
|
||||||
one small feature, is configured in traefik and keeps its state in memory.
|
shape, each one small feature, configured in traefik, state in memory.
|
||||||
- Sidecar fit: no; they live inside traefik.
|
- Sidecar fit: no; they live inside traefik.
|
||||||
- Traefik built-in middlewares
|
- Traefik built-in middlewares
|
||||||
- `rateLimit` (one average-and-burst window per middleware, optional Redis
|
- `rateLimit` (one average-and-burst window per middleware, optional Redis
|
||||||
in traefik 3.x), `inFlightReq`, `ipAllowList`. No bans, alerts, reputation
|
in traefik 3.x), `inFlightReq`, `ipAllowList`. No bans, alerts, reputation
|
||||||
lookups or AS number lookups.
|
or AS number awareness.
|
||||||
- caddy-waf (`fabriziosalmi/caddy-waf`), 0.4.x
|
- caddy-waf (`fabriziosalmi/caddy-waf`), 0.4.x
|
||||||
- What it is: a Caddy module with regex rules and anomaly scoring, per-IP
|
- What it is: a Caddy module with regex rules and anomaly scoring, per-IP
|
||||||
and per-path rate limiting with one configurable window, IP and DNS
|
and per-path rate limiting with one configurable window, IP and DNS
|
||||||
@@ -210,12 +198,12 @@ Closest existing options, and why each still falls short:
|
|||||||
- open-appsec (Check Point)
|
- open-appsec (Check Point)
|
||||||
- Machine-learning WAF agent attached to nginx, Kong, Envoy or similar, with
|
- Machine-learning WAF agent attached to nginx, Kong, Envoy or similar, with
|
||||||
a declarative policy file or the vendor's cloud console. Covers R9 only;
|
a declarative policy file or the vendor's cloud console. Covers R9 only;
|
||||||
rate limiting and richer features are in paid tiers. It attaches to an
|
rate limiting and richer features are in paid tiers. Not a sidecar in the
|
||||||
existing proxy, so it is not the sidecar R10 asks for.
|
required sense.
|
||||||
- iocaine
|
- iocaine
|
||||||
- Serves generated garbage pages to clients the fronting proxy classifies as
|
- Serves generated garbage pages to clients the fronting proxy classifies as
|
||||||
scrapers. Not a limiter, WAF or ban tool; out of scope, and only of
|
scrapers. Not a limiter, WAF or ban tool; out of scope except as a
|
||||||
passing interest for scraper traffic.
|
curiosity for scraper traffic.
|
||||||
- Pangolin
|
- Pangolin
|
||||||
- A tunnelled access platform that bundles traefik and optionally CrowdSec.
|
- A tunnelled access platform that bundles traefik and optionally CrowdSec.
|
||||||
Replaces the ingress rather than adding a sidecar; out of scope.
|
Replaces the ingress rather than adding a sidecar; out of scope.
|
||||||
@@ -226,17 +214,17 @@ Closest existing options, and why each still falls short:
|
|||||||
Everyone else offers one window per rule or none.
|
Everyone else offers one window per rule or none.
|
||||||
- R2 bypass netblocks: BunkerWeb, CrowdSec, traefik, fail2ban plugins.
|
- R2 bypass netblocks: BunkerWeb, CrowdSec, traefik, fail2ban plugins.
|
||||||
- R3 alerts: CrowdSec (best), BunkerWeb (denied requests only), SafeLine (paid).
|
- R3 alerts: CrowdSec (best), BunkerWeb (denied requests only), SafeLine (paid).
|
||||||
- R4 alerts on request counts and bytes: none. CrowdSec can approximate alerts
|
- R4 volume and byte anomalies: none. CrowdSec can approximate request-count
|
||||||
on request counts with a custom scenario; no surveyed tool alerts on bytes.
|
anomalies with a custom scenario; nothing handles bytes.
|
||||||
- R5 temporary then permanent: CrowdSec only. BunkerWeb's ban is either one
|
- R5 temporary then permanent: CrowdSec only. BunkerWeb has fixed or permanent,
|
||||||
fixed length or permanent, and does not change with a client's history.
|
not both by history.
|
||||||
- R6 reputation: CrowdSec, BunkerWeb, go-away, caddy-waf (static lists).
|
- R6 reputation: CrowdSec, BunkerWeb, go-away, caddy-waf (static lists).
|
||||||
- R7 AS number lookup: CrowdSec (enrichment), BunkerWeb and caddy-waf (allow or
|
- R7 AS number lookup: CrowdSec (enrichment), BunkerWeb and caddy-waf (allow or
|
||||||
deny), Anubis (hosted service).
|
deny), Anubis (hosted service).
|
||||||
- R8 biased thresholds: none.
|
- R8 biased thresholds: none.
|
||||||
- R9 attack detection: Coraza or ModSecurity with the Core Rule Set (open,
|
- R9 attack detection: Coraza or ModSecurity with the Core Rule Set (open,
|
||||||
standard), CrowdSec AppSec, SafeLine (closed engine), BunkerWeb (wraps Coraza
|
standard), CrowdSec AppSec, SafeLine (closed engine), BunkerWeb (wraps the
|
||||||
or ModSecurity).
|
first).
|
||||||
- R10 env-var sidecar: `owasp/modsecurity-crs`, `coraza-crs-docker`, Anubis.
|
- R10 env-var sidecar: `owasp/modsecurity-crs`, `coraza-crs-docker`, Anubis.
|
||||||
None of these covers anything beyond its one job.
|
None of these covers anything beyond its one job.
|
||||||
|
|
||||||
|
|||||||
@@ -25,11 +25,11 @@ without an answer in time, `rproxy` passes the request to the app.
|
|||||||
It protects against:
|
It protects against:
|
||||||
|
|
||||||
- Scrapers that fetch every page of a site, all day. Each client gets a limit on
|
- Scrapers that fetch every page of a site, all day. Each client gets a limit on
|
||||||
requests and on body bytes per minute, per hour and per day. The defaults
|
requests and on bytes per minute, per hour and per day. The defaults leave
|
||||||
leave room for a person reading a site for hours. They catch a scraper that
|
room for a person reading a site for hours. They catch a scraper that keeps up
|
||||||
keeps up one request every 2 seconds. A client that breaks a limit is banned:
|
one request every 2 seconds. A client over a limit is refused and banned: for
|
||||||
for a while at first, for three times as long at each repeat, and permanently
|
a while at first, for three times as long at each repeat, and permanently when
|
||||||
when it keeps at it.
|
it keeps at it.
|
||||||
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
|
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
|
||||||
default rule file bans such a client at its first probe, and its next request
|
default rule file bans such a client at its first probe, and its next request
|
||||||
during the ban makes the ban permanent. Your own rules, or a list of trap
|
during the ban makes the ban permanent. Your own rules, or a list of trap
|
||||||
@@ -173,7 +173,7 @@ does.
|
|||||||
the request have `"action":"forward"`.
|
the request have `"action":"forward"`.
|
||||||
- To see a refusal without banning yourself, run `discriminator` with
|
- To see a refusal without banning yourself, run `discriminator` with
|
||||||
`SWW_MODE=observe` and ask for `/.env`. The request reaches the app.
|
`SWW_MODE=observe` and ask for `/.env`. The request reaches the app.
|
||||||
`discriminator`'s log lines have `"would_action":"temporarily_banned"` and
|
`discriminator`'s log lines have `"would_action":"banned"` and
|
||||||
`"rule_ids":["env-file"]`. A ban you get in `enforce` mode is lifted as
|
`"rule_ids":["env-file"]`. A ban you get in `enforce` mode is lifted as
|
||||||
"Clients and bans" below describes.
|
"Clients and bans" below describes.
|
||||||
- With `SWW_METRICS_TOKEN` set, this answers with the metrics, from a host on
|
- With `SWW_METRICS_TOKEN` set, this answers with the metrics, from a host on
|
||||||
@@ -210,10 +210,6 @@ account, a key, a token or an alert destination stays off until you give one.
|
|||||||
percentage is a whole number from 0 to 100. A path prefix starts with `/`.
|
percentage is a whole number from 0 to 100. A path prefix starts with `/`.
|
||||||
- `off` switches a limit, a threshold or a timeout off where the table says a
|
- `off` switches a limit, a threshold or a timeout off where the table says a
|
||||||
setting takes it.
|
setting takes it.
|
||||||
- The byte limits and the byte anomaly thresholds count only body bytes: the
|
|
||||||
app's response bodies, the visitors' request bodies, or both, as
|
|
||||||
`SWW_BYTES_COUNT` says. Headers are never counted. What the visitor sent over
|
|
||||||
a WebSocket counts as request bytes, and what the app sent as response bytes.
|
|
||||||
- An invalid setting stops the process at start with one log line naming the
|
- An invalid setting stops the process at start with one log line naming the
|
||||||
variable and what is wrong with the value. The settings in effect are logged
|
variable and what is wrong with the value. The settings in effect are logged
|
||||||
on one line at start, with tokens, keys and webhook paths shown as `********`.
|
on one line at start, with tokens, keys and webhook paths shown as `********`.
|
||||||
@@ -221,7 +217,7 @@ account, a key, a token or an alert destination stays off until you give one.
|
|||||||
Every setting the code reads, with its default:
|
Every setting the code reads, with its default:
|
||||||
|
|
||||||
| Variable | Default | What it does |
|
| Variable | Default | What it does |
|
||||||
| ---------------------------------------- | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| ---------------------------------------- | ------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `SWW_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `discriminator`. |
|
| `SWW_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `discriminator`. |
|
||||||
| `SWRP_WAF_BACKEND` | required | `discriminator`'s URL, such as `http://discriminator:8080`. |
|
| `SWRP_WAF_BACKEND` | required | `discriminator`'s URL, such as `http://discriminator:8080`. |
|
||||||
| `SWRP_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. |
|
| `SWRP_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. |
|
||||||
@@ -230,8 +226,8 @@ Every setting the code reads, with its default:
|
|||||||
| `SWRP_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `discriminator`'s answer to a question and to a report, in milliseconds. |
|
| `SWRP_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `discriminator`'s answer to a question and to a report, in milliseconds. |
|
||||||
| `SWRP_DEFAULT_DISPOSITION` | `allow` | What `rproxy` does with a request when `discriminator` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. |
|
| `SWRP_DEFAULT_DISPOSITION` | `allow` | What `rproxy` does with a request when `discriminator` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. |
|
||||||
| `SWRP_MAX_REQUEST_SIZE` | `100M` | The largest request body, a size or `off`; a larger one is answered `413`. |
|
| `SWRP_MAX_REQUEST_SIZE` | `100M` | The largest request body, a size or `off`; a larger one is answered `413`. |
|
||||||
| `SWRP_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
|
| `SWRP_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`; a larger one is answered `502`, or cut off once it has started. |
|
||||||
| `SWRP_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
|
| `SWRP_MAX_REQUEST_TIME_MS` | `60000` | How long the visitor may take to send its request line and headers, and then the whole request to reach the app, in milliseconds or `off`. |
|
||||||
| `SWRP_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
| `SWRP_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
||||||
| `SWW_LISTEN_ADDR` | `:8080` | Where `discriminator` listens; leave the host part empty. |
|
| `SWW_LISTEN_ADDR` | `:8080` | Where `discriminator` listens; leave the host part empty. |
|
||||||
| `SWW_INSTANCE_NAME` | the container's host name | The name in every line `discriminator` logs, and in every alert and metric, such as `fsn1/discriminator`. |
|
| `SWW_INSTANCE_NAME` | the container's host name | The name in every line `discriminator` logs, and in every alert and metric, such as `fsn1/discriminator`. |
|
||||||
@@ -240,24 +236,24 @@ Every setting the code reads, with its default:
|
|||||||
| `SWW_MAX_TRACKED_CLIENTS` | `20000` | The most clients kept, in memory and in `clients.json`; the least recently seen is dropped first. |
|
| `SWW_MAX_TRACKED_CLIENTS` | `20000` | The most clients kept, in memory and in `clients.json`; the least recently seen is dropped first. |
|
||||||
| `SWW_MAX_BANS` | `5000` | The most bans `discriminator` keeps of its own; bans an admin made do not count. |
|
| `SWW_MAX_BANS` | `5000` | The most bans `discriminator` keeps of its own; bans an admin made do not count. |
|
||||||
| `SWW_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
|
| `SWW_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
|
||||||
| `SWW_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients are exempt from rate limiting and the byte limits. |
|
| `SWW_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients the request and byte limits neither count nor refuse. |
|
||||||
| `SWW_DENY_NETS` | empty | Netblocks whose clients are refused. |
|
| `SWW_DENY_NETS` | empty | Netblocks whose clients are refused. |
|
||||||
| `SWW_RATE_LIMIT_PER_MINUTE` | `120` | The most requests a client may make in a minute, a count or `off`; the next request is refused and bans the client. |
|
| `SWW_RATE_LIMIT_PER_MINUTE` | `120` | The most requests a client may make in a minute, a count or `off`; one more bans it. |
|
||||||
| `SWW_RATE_LIMIT_PER_HOUR` | `1500` | The most requests a client may make in an hour, a count or `off`; the next request is refused and bans the client. |
|
| `SWW_RATE_LIMIT_PER_HOUR` | `1500` | The most requests a client may make in an hour, a count or `off`; one more bans it. |
|
||||||
| `SWW_RATE_LIMIT_PER_DAY` | `6000` | The most requests a client may make in a day, a count or `off`; the next request is refused and bans the client. |
|
| `SWW_RATE_LIMIT_PER_DAY` | `6000` | The most requests a client may make in a day, a count or `off`; one more bans it. |
|
||||||
| `SWW_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes exempt from rate limiting and the byte limits, such as `/assets/`. |
|
| `SWW_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes whose requests the request and byte limits neither count nor refuse, such as `/assets/`. |
|
||||||
| `SWW_BYTES_LIMIT_PER_MINUTE` | `off` | The most bytes counted for a client in a minute, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
| `SWW_BYTES_LIMIT_PER_MINUTE` | `off` | The most bytes a client's requests may carry in a minute, a size or `off`; the request that takes a client past any byte limit bans it. |
|
||||||
| `SWW_BYTES_LIMIT_PER_HOUR` | `50G` | The most bytes counted for a client in an hour, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
| `SWW_BYTES_LIMIT_PER_HOUR` | `50G` | The most bytes a client's requests may carry in an hour, a size or `off`. |
|
||||||
| `SWW_BYTES_LIMIT_PER_DAY` | `100G` | The most bytes counted for a client in a day, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
| `SWW_BYTES_LIMIT_PER_DAY` | `100G` | The most bytes a client's requests may carry in a day, a size or `off`. |
|
||||||
| `SWW_BYTES_COUNT` | `both` | What the byte limits and the byte anomaly thresholds count: `response`, the app's response bodies; `request`, the visitors' request bodies; or `both`. What the visitor sent over a WebSocket counts as request bytes, and what the app sent as response bytes. |
|
| `SWW_BYTES_COUNT` | `both` | Which body bytes the byte limits count: `response`, `request` or `both`. |
|
||||||
| `SWW_ERROR_BURST_THRESHOLD` | `30` | The most refusals a client may get in a minute from a rule file, a trap path or the Core Rule Set, a count or `off`. The next one bans the client. |
|
| `SWW_ERROR_BURST_THRESHOLD` | `30` | The most refusals by a rule file, a trap path or the Core Rule Set a client may get in a minute, a count or `off`; one more bans it. |
|
||||||
| `SWW_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. |
|
| `SWW_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. |
|
||||||
| `SWW_LIMIT_BAN_DURATION` | `1h` | The ban for a first broken limit. |
|
| `SWW_LIMIT_BAN_DURATION` | `1h` | The ban for a first broken limit. |
|
||||||
| `SWW_LIMIT_BAN_REPEAT_WINDOW` | `24h` | A client that breaks a limit again within this time after its last such ban ended gets a ban three times as long. |
|
| `SWW_LIMIT_BAN_REPEAT_WINDOW` | `24h` | A limit broken again within this time after the last such ban ended gets a ban three times as long. |
|
||||||
| `SWW_MAX_BAN_DURATION` | `7d` | A ban for a broken limit that would be longer than this is permanent instead. |
|
| `SWW_MAX_BAN_DURATION` | `7d` | A ban for a broken limit that would be longer than this is permanent instead. |
|
||||||
| `SWW_ATTACK_BAN_DURATION` | `7d` | The ban for a first clear sign of attack: a `ban` rule or a trap path. |
|
| `SWW_ATTACK_BAN_DURATION` | `7d` | The ban for a first clear sign of attack: a `ban` rule or a trap path. |
|
||||||
| `SWW_BAN_SCOPE_V4_PREFIX` | `32` | The prefix length of the netblock a ban on an IPv4 client covers, from 0 to 32. |
|
| `SWW_BAN_SCOPE_V4_PREFIX` | `32` | The prefix length of the netblock a ban on an IPv4 client covers, from 0 to 32. |
|
||||||
| `SWW_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs` (a web service), `file` (a local database) or `off`. |
|
| `SWW_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs`, a web service, `file`, or `off`. |
|
||||||
| `SWW_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. |
|
| `SWW_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. |
|
||||||
| `SWW_LOOKUP_TIMEOUT` | `800ms` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take. |
|
| `SWW_LOOKUP_TIMEOUT` | `800ms` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take. |
|
||||||
| `SWW_ADD_LOOKUP_HEADERS` | `false` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
| `SWW_ADD_LOOKUP_HEADERS` | `false` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
||||||
@@ -316,27 +312,27 @@ Every setting the code reads, with its default:
|
|||||||
| `SWW_ALERT_MAX_PER_HOUR` | `60` | The most alerts sent in an hour, a number or `off`; the rest are counted in the hour's `summary`. |
|
| `SWW_ALERT_MAX_PER_HOUR` | `60` | The most alerts sent in an hour, a number or `off`; the rest are counted in the hour's `summary`. |
|
||||||
| `SWW_ANOMALY_CLIENT_REQUESTS_PER_MINUTE` | `off` | Alert when one client makes more requests than this in a minute, a count. |
|
| `SWW_ANOMALY_CLIENT_REQUESTS_PER_MINUTE` | `off` | Alert when one client makes more requests than this in a minute, a count. |
|
||||||
| `SWW_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Alert when one client makes more requests than this in an hour, a count. |
|
| `SWW_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Alert when one client makes more requests than this in an hour, a count. |
|
||||||
| `SWW_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one client in a minute, a size. |
|
| `SWW_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Alert when one client's requests carry more bytes than this in a minute, a size. |
|
||||||
| `SWW_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one client in an hour, a size. |
|
| `SWW_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Alert when one client's requests carry more bytes than this in an hour, a size. |
|
||||||
| `SWW_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Alert when the netblock around a client makes more requests than this in a minute, a count. |
|
| `SWW_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Alert when the netblock around a client makes more requests than this in a minute, a count. |
|
||||||
| `SWW_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Alert when the netblock around a client makes more requests than this in an hour, a count. |
|
| `SWW_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Alert when the netblock around a client makes more requests than this in an hour, a count. |
|
||||||
| `SWW_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the netblock around a client in a minute, a size. |
|
| `SWW_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Alert when the netblock around a client carries more bytes than this in a minute, a size. |
|
||||||
| `SWW_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the netblock around a client in an hour, a size. |
|
| `SWW_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Alert when the netblock around a client carries more bytes than this in an hour, a size. |
|
||||||
| `SWW_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_ANOMALY_NET_` thresholds count, from 0 to 32. |
|
| `SWW_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_ANOMALY_NET_` thresholds count, from 0 to 32. |
|
||||||
| `SWW_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_ANOMALY_NET_` thresholds count, from 0 to 128. |
|
| `SWW_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_ANOMALY_NET_` thresholds count, from 0 to 128. |
|
||||||
| `SWW_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Alert when one AS number makes more requests than this in a minute, a count. |
|
| `SWW_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Alert when one AS number makes more requests than this in a minute, a count. |
|
||||||
| `SWW_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Alert when one AS number makes more requests than this in an hour, a count. |
|
| `SWW_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Alert when one AS number makes more requests than this in an hour, a count. |
|
||||||
| `SWW_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one AS number in a minute, a size. |
|
| `SWW_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Alert when one AS number carries more bytes than this in a minute, a size. |
|
||||||
| `SWW_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one AS number in an hour, a size. |
|
| `SWW_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Alert when one AS number carries more bytes than this in an hour, a size. |
|
||||||
| `SWW_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Alert when the whole service gets more requests than this in a minute, a count. |
|
| `SWW_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Alert when the whole service gets more requests than this in a minute, a count. |
|
||||||
| `SWW_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Alert when the whole service gets more requests than this in an hour, a count. |
|
| `SWW_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Alert when the whole service gets more requests than this in an hour, a count. |
|
||||||
| `SWW_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the whole service in a minute, a size. |
|
| `SWW_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Alert when the whole service carries more bytes than this in a minute, a size. |
|
||||||
| `SWW_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the whole service in an hour, a size. |
|
| `SWW_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Alert when the whole service carries more bytes than this in an hour, a size. |
|
||||||
| `SWW_WATCH_NETS` | empty | Named netblocks the `SWW_WATCH_` thresholds count each as a whole, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. |
|
| `SWW_WATCH_NETS` | empty | Named netblocks the `SWW_WATCH_` thresholds count each as a whole, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. |
|
||||||
| `SWW_WATCH_REQUESTS_PER_MINUTE` | `off` | Alert when a watched netblock makes more requests than this in a minute, a count. |
|
| `SWW_WATCH_REQUESTS_PER_MINUTE` | `off` | Alert when a watched netblock makes more requests than this in a minute, a count. |
|
||||||
| `SWW_WATCH_REQUESTS_PER_HOUR` | `off` | Alert when a watched netblock makes more requests than this in an hour, a count. |
|
| `SWW_WATCH_REQUESTS_PER_HOUR` | `off` | Alert when a watched netblock makes more requests than this in an hour, a count. |
|
||||||
| `SWW_WATCH_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for a watched netblock in a minute, a size. |
|
| `SWW_WATCH_BYTES_PER_MINUTE` | `off` | Alert when a watched netblock carries more bytes than this in a minute, a size. |
|
||||||
| `SWW_WATCH_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for a watched netblock in an hour, a size. |
|
| `SWW_WATCH_BYTES_PER_HOUR` | `off` | Alert when a watched netblock carries more bytes than this in an hour, a size. |
|
||||||
|
|
||||||
[`docs/settings.md`](docs/settings.md) gives the values each setting takes in
|
[`docs/settings.md`](docs/settings.md) gives the values each setting takes in
|
||||||
full, and the limits that are fixed rather than settings.
|
full, and the limits that are fixed rather than settings.
|
||||||
@@ -532,9 +528,8 @@ of the request reaches the app.
|
|||||||
|
|
||||||
17. The request is passed to the app, streaming.
|
17. The request is passed to the app, streaming.
|
||||||
18. After the answer, `rproxy` sends `discriminator` a report with the status
|
18. After the answer, `rproxy` sends `discriminator` a report with the status
|
||||||
and the body bytes each way. For an allowed request, `discriminator` counts
|
and the bytes each way. For an allowed request, `discriminator` counts the
|
||||||
the bytes `SWW_BYTES_COUNT` names toward the byte limits, bans the client if
|
bytes toward the byte limits and adds the request to the client's history.
|
||||||
they take it past one, and adds the request to the client's history.
|
|
||||||
|
|
||||||
A refused visitor is answered with `SWW_BAN_RESPONSE`. A `block` rule or a Core
|
A refused visitor is answered with `SWW_BAN_RESPONSE`. A `block` rule or a Core
|
||||||
Rule Set match answers `403` and bans nobody, since the Core Rule Set has false
|
Rule Set match answers `403` and bans nobody, since the Core Rule Set has false
|
||||||
@@ -633,8 +628,8 @@ is set aside as `<name>.bad`, and never stops the running process (see
|
|||||||
- `internal/discriminator`: `discriminator`'s server, which runs the checks and
|
- `internal/discriminator`: `discriminator`'s server, which runs the checks and
|
||||||
answers its own endpoints.
|
answers its own endpoints.
|
||||||
- `internal/ratelimit`, `internal/bans`, `internal/rules`, `internal/waf`,
|
- `internal/ratelimit`, `internal/bans`, `internal/rules`, `internal/waf`,
|
||||||
`internal/lookup`, `internal/reputation`, `internal/routewatch`,
|
`internal/lookup`, `internal/reputation`, `internal/anomaly` and
|
||||||
`internal/anomaly` and `internal/alerts`: one check or source each.
|
`internal/alerts`: one check or source each.
|
||||||
- `internal/state`, `internal/requestlog`, `internal/remotelog` and
|
- `internal/state`, `internal/requestlog`, `internal/remotelog` and
|
||||||
`internal/metrics`: the state files, the log lines, the syslog sender and the
|
`internal/metrics`: the state files, the log lines, the syslog sender and the
|
||||||
metrics.
|
metrics.
|
||||||
|
|||||||
@@ -22,8 +22,8 @@ to them.
|
|||||||
key or a token stays off until the operator supplies it.
|
key or a token stays off until the operator supplies it.
|
||||||
- It keeps a site answering while the part that decides is down, slow or
|
- It keeps a site answering while the part that decides is down, slow or
|
||||||
restarting (see "Without an answer from `discriminator`").
|
restarting (see "Without an answer from `discriminator`").
|
||||||
- It limits each client's requests and body bytes per minute, hour and day. It
|
- It limits each client's requests and bytes per minute, hour and day. It bounds
|
||||||
bounds the size and duration of every request and answer. It detects attacks.
|
the size and duration of every request and answer. It detects attacks.
|
||||||
- It bans abusers: briefly at first, for seven days on a clear sign of attack,
|
- It bans abusers: briefly at first, for seven days on a clear sign of attack,
|
||||||
and permanently when they keep at it.
|
and permanently when they keep at it.
|
||||||
- It looks up each client's AS number and country. It can refuse whole countries
|
- It looks up each client's AS number and country. It can refuse whole countries
|
||||||
@@ -87,8 +87,6 @@ to them.
|
|||||||
database.
|
database.
|
||||||
- `internal/reputation`: fetches the lists, asks the DNS blocklists and
|
- `internal/reputation`: fetches the lists, asks the DNS blocklists and
|
||||||
AbuseIPDB in the background, and keeps their answers.
|
AbuseIPDB in the background, and keeps their answers.
|
||||||
- `internal/routewatch`: asks a routewatch instance about clients in the
|
|
||||||
background, and keeps its answers.
|
|
||||||
- `internal/anomaly`: the anomaly counters and their alerts.
|
- `internal/anomaly`: the anomaly counters and their alerts.
|
||||||
- `internal/alerts`: holds back repeats and alerts past the hourly limit,
|
- `internal/alerts`: holds back repeats and alerts past the hourly limit,
|
||||||
and sends the rest to the webhook, Slack and ntfy, each from a queue of
|
and sends the rest to the webhook, Slack and ntfy, each from a queue of
|
||||||
@@ -158,8 +156,8 @@ request log's `heuristic` field gives that name. `refusal_limit` and
|
|||||||
up to `SWW_LOOKUP_TIMEOUT`, only when one of these needs it:
|
up to `SWW_LOOKUP_TIMEOUT`, only when one of these needs it:
|
||||||
- the country list;
|
- the country list;
|
||||||
- a biased threshold;
|
- a biased threshold;
|
||||||
- `SWW_TAG_DENY` or `SWW_TAG_LIMIT_PERCENT` naming an `asn=` or `country=`
|
- `SWW_TAG_DENY` or `SWW_TAG_LIMIT_PERCENT` naming a `country=` tag, or an
|
||||||
tag;
|
`asn=` tag while `SWW_ROUTEWATCH_URL` is unset;
|
||||||
- `SWW_ADD_LOOKUP_HEADERS`.
|
- `SWW_ADD_LOOKUP_HEADERS`.
|
||||||
|
|
||||||
A request that ends before the answer comes is logged without it.
|
A request that ends before the answer comes is logged without it.
|
||||||
@@ -196,12 +194,11 @@ request log's `heuristic` field gives that name. `refusal_limit` and
|
|||||||
sent is removed.
|
sent is removed.
|
||||||
18. After the visitor's answer has ended, `rproxy` sends the report (see "The
|
18. After the visitor's answer has ended, `rproxy` sends the report (see "The
|
||||||
report"). `discriminator` then counts:
|
report"). `discriminator` then counts:
|
||||||
- For a request it allowed, the body bytes of the response, of the request
|
- For a request it allowed, the response bytes, the request bytes or both
|
||||||
or of both are added to the byte counters, as `SWW_BYTES_COUNT` says. For
|
are added to the byte counters, as `SWW_BYTES_COUNT` says. For a
|
||||||
a WebSocket, what it carried each way counts once it closes. A client
|
WebSocket, what it carried each way counts once it closes. A client whose
|
||||||
whose total passes a byte limit times the percentage has broken that limit
|
total passes a byte limit times the percentage has broken that limit and
|
||||||
and is banned (`byte_limits`). The request that broke it has already been
|
is banned (`byte_limits`). The response in progress is not cut off.
|
||||||
passed on whole; nothing is cut off.
|
|
||||||
- For a request it allowed, the client's history and the anomaly counters
|
- For a request it allowed, the client's history and the anomaly counters
|
||||||
are updated, and the alert thresholds are evaluated.
|
are updated, and the alert thresholds are evaluated.
|
||||||
|
|
||||||
@@ -264,8 +261,7 @@ to a ban, each by its own rule:
|
|||||||
clear sign of attack bans it permanently at once.
|
clear sign of attack bans it permanently at once.
|
||||||
- A broken limit, one of:
|
- A broken limit, one of:
|
||||||
- a request over a rate limit;
|
- a request over a rate limit;
|
||||||
- the bytes counted for a request and its response taking the client's total
|
- a response that takes the client's byte total past a byte limit;
|
||||||
past a byte limit;
|
|
||||||
- more than `SWW_ERROR_BURST_THRESHOLD` refusals within a minute.
|
- more than `SWW_ERROR_BURST_THRESHOLD` refusals within a minute.
|
||||||
|
|
||||||
The bans for a broken limit:
|
The bans for a broken limit:
|
||||||
@@ -321,13 +317,6 @@ ban is dropped alike. A scanner whose ban was dropped that way is refused and
|
|||||||
banned again by its next probe. A ban whose cause is `admin` is never dropped
|
banned again by its next probe. A ban whose cause is `admin` is never dropped
|
||||||
and does not count toward `SWW_MAX_BANS`, so such bans can never fill the table.
|
and does not count toward `SWW_MAX_BANS`, so such bans can never fill the table.
|
||||||
|
|
||||||
A request refused under a ban that runs out is logged with the action
|
|
||||||
`temporarily_banned`. Under a permanent ban the action is `permanently_banned`.
|
|
||||||
A request refused for a clear sign of attack or a CrowdSec decision takes the
|
|
||||||
action for the ban it made. A request that makes a ban permanent is logged as
|
|
||||||
`permanently_banned`. A request over a rate limit is logged as
|
|
||||||
`client_is_rate_limited`, whichever ban it makes.
|
|
||||||
|
|
||||||
## Settings
|
## Settings
|
||||||
|
|
||||||
`docs/settings.md` lists every setting with its default and the values it takes,
|
`docs/settings.md` lists every setting with its default and the values it takes,
|
||||||
@@ -461,15 +450,10 @@ the same way.
|
|||||||
|
|
||||||
### Byte limits, per client
|
### Byte limits, per client
|
||||||
|
|
||||||
The byte limits count body bytes: the app's response bodies, the visitors'
|
By default there is no minute limit, so a client can download in a burst at full
|
||||||
request bodies, or both, as `SWW_BYTES_COUNT` says. Headers are never counted.
|
speed. The hour limit defaults to 50 GiB and the day limit to 100 GiB. They
|
||||||
What the visitor sent over a WebSocket counts as request bytes, and what the app
|
allow several downloads of a few GiB each. A response's bytes are counted when
|
||||||
sent as response bytes. The bytes counted for a request and its response are
|
it ends (see step 18).
|
||||||
added to the client's total once the response has ended (see step 18). So the
|
|
||||||
request that takes a client past a limit is passed on whole, and the ban refuses
|
|
||||||
the client's requests from then on. By default there is no minute limit, so a
|
|
||||||
client can download in a burst at full speed. The hour limit defaults to 50 GiB
|
|
||||||
and the day limit to 100 GiB. They allow several downloads of a few GiB each.
|
|
||||||
|
|
||||||
### Lookup of AS number and country
|
### Lookup of AS number and country
|
||||||
|
|
||||||
@@ -569,11 +553,9 @@ Each tag and where it comes from:
|
|||||||
(`routewatch_tags`), while `SWW_ROUTEWATCH_URL` is set:
|
(`routewatch_tags`), while `SWW_ROUTEWATCH_URL` is set:
|
||||||
- The tag `prefix=` holds the netblock the client's address is announced in,
|
- The tag `prefix=` holds the netblock the client's address is announced in,
|
||||||
such as `prefix=192.0.2.0/24`.
|
such as `prefix=192.0.2.0/24`.
|
||||||
- An `asn=` tag has the same spelling as the lookup's. When routewatch and
|
- An `asn=` tag has the same spelling as the lookup's. A request carries it
|
||||||
the lookup give the same AS number, a request has one `asn=` tag. When
|
once, and carries both when the two sources disagree.
|
||||||
they differ, it has both.
|
- These tags come before `deny_nets`, so a line that check refused has them.
|
||||||
- They are added before `deny_nets`, so the log line of a request it refused
|
|
||||||
has them.
|
|
||||||
- `discriminator` asks routewatch in the background and never waits for it.
|
- `discriminator` asks routewatch in the background and never waits for it.
|
||||||
An answer is kept for 1 hour, in memory only.
|
An answer is kept for 1 hour, in memory only.
|
||||||
- After a query fails, routewatch is not asked for 1 minute.
|
- After a query fails, routewatch is not asked for 1 minute.
|
||||||
@@ -587,10 +569,9 @@ lower case, except a country code, which is in capitals. `SWW_TAG_DENY` and
|
|||||||
|
|
||||||
Two settings act on tags:
|
Two settings act on tags:
|
||||||
|
|
||||||
- `SWW_TAG_DENY` lists tags whose requests are refused with the action
|
- `SWW_TAG_DENY` lists tags whose requests are refused with the action `denied`
|
||||||
`client_denied` (`tag_deny`). The check runs twice: just before the rate
|
(`tag_deny`). The check runs twice: just before the rate limits, and again
|
||||||
limits, and again after the Core Rule Set, so that `exploitsignature` can
|
after the Core Rule Set, so that `exploitsignature` can refuse too.
|
||||||
refuse too.
|
|
||||||
- `SWW_TAG_LIMIT_PERCENT` lists tags with a percentage from 1 to 99. Their
|
- `SWW_TAG_LIMIT_PERCENT` lists tags with a percentage from 1 to 99. Their
|
||||||
clients get that percentage of every request and byte limit. The lowest
|
clients get that percentage of every request and byte limit. The lowest
|
||||||
percentage among a client's tags applies. `exploitsignature` cannot be listed,
|
percentage among a client's tags applies. `exploitsignature` cannot be listed,
|
||||||
@@ -598,9 +579,7 @@ Two settings act on tags:
|
|||||||
|
|
||||||
The tag settings are empty by default, for the same reason as the country list.
|
The tag settings are empty by default, for the same reason as the country list.
|
||||||
The lookup waits for GeoJS when either names an `asn=` or `country=` tag. A
|
The lookup waits for GeoJS when either names an `asn=` or `country=` tag. A
|
||||||
setting that names one stops the start while `SWW_LOOKUP_SOURCE` is `off`. The
|
setting that names one stops the start while `SWW_LOOKUP_SOURCE` is `off`.
|
||||||
exception is an `asn=` tag while `SWW_ROUTEWATCH_URL` is set, since routewatch
|
|
||||||
gives that tag.
|
|
||||||
|
|
||||||
### Biased thresholds
|
### Biased thresholds
|
||||||
|
|
||||||
@@ -989,8 +968,7 @@ which the metrics show. They reveal a widely distributed scraper, thousands of
|
|||||||
addresses at low rates each, which the per-client limits do not see. A low
|
addresses at low rates each, which the per-client limits do not see. A low
|
||||||
percentage for its `asn=` tag in `SWW_TAG_LIMIT_PERCENT` is the response.
|
percentage for its `asn=` tag in `SWW_TAG_LIMIT_PERCENT` is the response.
|
||||||
|
|
||||||
- Requests and body bytes, as `SWW_BYTES_COUNT` counts them, are counted per
|
- Requests and bytes are counted per minute and per hour, for each of:
|
||||||
minute and per hour, for each of:
|
|
||||||
- a client;
|
- a client;
|
||||||
- the surrounding netblock;
|
- the surrounding netblock;
|
||||||
- an AS number;
|
- an AS number;
|
||||||
@@ -1300,9 +1278,9 @@ short.
|
|||||||
### Answering in time
|
### Answering in time
|
||||||
|
|
||||||
`rproxy` stops waiting for `discriminator` after `SWRP_TIMEOUT_MS`, by default 2
|
`rproxy` stops waiting for `discriminator` after `SWRP_TIMEOUT_MS`, by default 2
|
||||||
seconds, and `SWRP_DEFAULT_DISPOSITION` then decides. The longest a question
|
seconds, and `SWRP_DEFAULT_DISPOSITION` then decides. So at the defaults,
|
||||||
waits on an outside source is `SWW_LOOKUP_TIMEOUT`, by default 800 milliseconds,
|
`discriminator` answers a request well within that time, whatever the request
|
||||||
well within that time.
|
and whatever its outside sources do.
|
||||||
|
|
||||||
GeoJS is the only outside source a question can wait on. A question waits only
|
GeoJS is the only outside source a question can wait on. A question waits only
|
||||||
when step 6 says so, and for at most `SWW_LOOKUP_TIMEOUT`, by default 800
|
when step 6 says so, and for at most `SWW_LOOKUP_TIMEOUT`, by default 800
|
||||||
@@ -1346,9 +1324,9 @@ over either one:
|
|||||||
`Transfer-Encoding` do not count. `X-Request-ID` and `X-Forwarded-Proto` do.
|
`Transfer-Encoding` do not count. `X-Request-ID` and `X-Forwarded-Proto` do.
|
||||||
|
|
||||||
Real clients never reach these limits. The answer is `deny` with `431`, and the
|
Real clients never reach these limits. The answer is `deny` with `431`, and the
|
||||||
log line's action is `request_header_too_large`. The refusal makes no ban and is
|
log line's action is `too_large`. The refusal makes no ban and is no offence.
|
||||||
no offence. The rate limits count the request as usual, so a client that keeps
|
The rate limits count the request as usual, so a client that keeps sending such
|
||||||
sending such requests breaks a limit.
|
requests breaks a limit.
|
||||||
|
|
||||||
The limits apply on every path and in every `SWW_WAF_MODE`. A client in
|
The limits apply on every path and in every `SWW_WAF_MODE`. A client in
|
||||||
`SWW_ALLOW_NETS` skips them, as it skips the Core Rule Set. In `observe` mode
|
`SWW_ALLOW_NETS` skips them, as it skips the Core Rule Set. In `observe` mode
|
||||||
@@ -1398,9 +1376,9 @@ answer.
|
|||||||
For a request it holds, `discriminator` writes the request's last log line (see
|
For a request it holds, `discriminator` writes the request's last log line (see
|
||||||
"The log"). For a request it allowed, it first:
|
"The log"). For a request it allowed, it first:
|
||||||
|
|
||||||
- counts the body bytes `SWW_BYTES_COUNT` names, toward the client's byte limits
|
- counts the bytes as `SWW_BYTES_COUNT` says, toward the client's byte limits at
|
||||||
at the percentages it gave that request;
|
the percentages it gave that request;
|
||||||
- bans a client whose total passes a byte limit;
|
- bans a client that passes a byte limit;
|
||||||
- adds the request with its status to the client's history, the anomaly counters
|
- adds the request with its status to the client's history, the anomaly counters
|
||||||
and the metrics.
|
and the metrics.
|
||||||
|
|
||||||
|
|||||||
@@ -5,9 +5,6 @@ What `GET /_sww/metrics` answers with, at `discriminator`'s own address.
|
|||||||
keeps the `instance` label as `exported_instance` unless the scrape sets
|
keeps the `instance` label as `exported_instance` unless the scrape sets
|
||||||
`honor_labels: true`. `rproxy` has no metrics.
|
`honor_labels: true`. `rproxy` has no metrics.
|
||||||
|
|
||||||
Every `sww_*request_bytes_total` and `sww_*response_bytes_total` series counts
|
|
||||||
body bytes, never headers, whatever `SWW_BYTES_COUNT` says.
|
|
||||||
|
|
||||||
## Traffic
|
## Traffic
|
||||||
|
|
||||||
- `sww_requests_total`, `sww_request_bytes_total` and
|
- `sww_requests_total`, `sww_request_bytes_total` and
|
||||||
|
|||||||
+13
-17
@@ -96,15 +96,13 @@ Every one of these lines has:
|
|||||||
|
|
||||||
- `action`: what was done:
|
- `action`: what was done:
|
||||||
- `forward`: answered `allow`, so that `rproxy` passes it to the app;
|
- `forward`: answered `allow`, so that `rproxy` passes it to the app;
|
||||||
- `client_denied`: refused for `SWW_DENY_NETS`, for a reputation source with
|
- `denied`: refused for `SWW_DENY_NETS`, for a reputation source with the
|
||||||
the action `deny`, or for a tag in `SWW_TAG_DENY`;
|
action `deny`, or for a tag in `SWW_TAG_DENY`;
|
||||||
- `temporarily_banned`: refused under a ban that runs out, or for a `ban`
|
- `banned`: refused under a ban, or for a `ban` rule, a trap path or a
|
||||||
rule, a trap path or a CrowdSec decision that made such a ban;
|
CrowdSec decision, each of which bans the client;
|
||||||
- `permanently_banned`: refused under a ban that never runs out, including a
|
- `country_denied`, `rate_limited`, `rule_blocked` and `waf_blocked`:
|
||||||
ban this request made or made permanent;
|
refused by that check;
|
||||||
- `country_denied`, `client_is_rate_limited`, `blocked_by_block_rule` and
|
- `too_large`: over a header limit;
|
||||||
`waf_blocked`: refused by that check;
|
|
||||||
- `request_header_too_large`: over a header limit;
|
|
||||||
- `admin`: answered at one of `discriminator`'s own endpoints, or `404` for
|
- `admin`: answered at one of `discriminator`'s own endpoints, or `404` for
|
||||||
a path under `/_sww/`.
|
a path under `/_sww/`.
|
||||||
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
|
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
|
||||||
@@ -147,22 +145,20 @@ Every one of these lines has:
|
|||||||
included. It is left out when rate limiting did not count the request. It
|
included. It is left out when rate limiting did not count the request. It
|
||||||
holds:
|
holds:
|
||||||
- `minute`, `hour` and `day`: requests;
|
- `minute`, `hour` and `day`: requests;
|
||||||
- `minute_bytes`, `hour_bytes` and `day_bytes`: the body bytes
|
- `minute_bytes`, `hour_bytes` and `day_bytes`: bytes.
|
||||||
`SWW_BYTES_COUNT` names, in that window.
|
|
||||||
|
|
||||||
A count can have a fraction (see "Counting method" in
|
A count can have a fraction (see "Counting method" in
|
||||||
[`SPEC.md`](../SPEC.md)). The bytes counted for an allowed request and its
|
[`SPEC.md`](../SPEC.md)). The bytes of an allowed request are counted when
|
||||||
response are added when its report comes. So on the `decision` line, the
|
its report comes. So on the `decision` line, the bytes are without this
|
||||||
three byte totals leave them out.
|
request's.
|
||||||
|
|
||||||
- `limit_hit`: the limit broken:
|
- `limit_hit`: the limit broken:
|
||||||
- `minute`, `hour` or `day` for a rate limit;
|
- `minute`, `hour` or `day` for a rate limit;
|
||||||
- `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit;
|
- `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit;
|
||||||
- `error_burst`.
|
- `error_burst`.
|
||||||
|
|
||||||
When the bytes counted for a request and its response break a limit, the
|
A request whose bytes broke a limit was passed on whole, so its `action`
|
||||||
request has been passed on whole, so its `action` stays `forward`. Only its
|
stays `forward`. Only its `report` line gives that limit.
|
||||||
`report` line gives that limit.
|
|
||||||
|
|
||||||
- `rule_ids`: the rule file rules that matched, in order, up to the one that
|
- `rule_ids`: the rule file rules that matched, in order, up to the one that
|
||||||
refused.
|
refused.
|
||||||
|
|||||||
+14
-30
@@ -98,8 +98,6 @@ between the two (see "The image's command" below).
|
|||||||
|
|
||||||
## Request rate limits, per client
|
## Request rate limits, per client
|
||||||
|
|
||||||
The request past one of these limits is refused, and the client is banned.
|
|
||||||
|
|
||||||
- `SWW_RATE_LIMIT_PER_MINUTE` (default `120`, 2 a second): the most requests a
|
- `SWW_RATE_LIMIT_PER_MINUTE` (default `120`, 2 a second): the most requests a
|
||||||
client may make in a minute. A count or `off`. The default leaves room for a
|
client may make in a minute. A count or `off`. The default leaves room for a
|
||||||
person opening two pages at once with an empty cache, up to 60 files each.
|
person opening two pages at once with an empty cache, up to 60 files each.
|
||||||
@@ -122,22 +120,11 @@ The request past one of these limits is refused, and the client is banned.
|
|||||||
|
|
||||||
## Byte limits, per client
|
## Byte limits, per client
|
||||||
|
|
||||||
These count body bytes, never headers. What the visitor sent over a WebSocket
|
|
||||||
counts as request bytes, and what the app sent as response bytes. The bytes
|
|
||||||
counted for a request and its response are added to the client's total once the
|
|
||||||
response has ended. The request that takes a client's total past a limit is not
|
|
||||||
cut off; the client is banned.
|
|
||||||
|
|
||||||
- `SWW_BYTES_LIMIT_PER_MINUTE` (default `off`), `SWW_BYTES_LIMIT_PER_HOUR`
|
- `SWW_BYTES_LIMIT_PER_MINUTE` (default `off`), `SWW_BYTES_LIMIT_PER_HOUR`
|
||||||
(default `50G`) and `SWW_BYTES_LIMIT_PER_DAY` (default `100G`): the most bytes
|
(default `50G`) and `SWW_BYTES_LIMIT_PER_DAY` (default `100G`): the most bytes
|
||||||
counted for a client in each window, a size or `off`. `SWW_BYTES_COUNT` says
|
counted for a client in each window. A size or `off`.
|
||||||
which bytes count: response bodies, request bodies or both. The request that
|
- `SWW_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
|
||||||
passes the limit is not cut off, and the client is banned when its response
|
`response`, `request` or `both`.
|
||||||
ends.
|
|
||||||
- `SWW_BYTES_COUNT` (default `both`): what the byte limits and the byte anomaly
|
|
||||||
thresholds count: `response`, the app's response bodies; `request`, the
|
|
||||||
visitors' request bodies; or `both`. What the visitor sent over a WebSocket
|
|
||||||
counts as request bytes, and what the app sent as response bytes.
|
|
||||||
|
|
||||||
## Lookup of AS number and country
|
## Lookup of AS number and country
|
||||||
|
|
||||||
@@ -145,8 +132,8 @@ cut off; the client is banned.
|
|||||||
the start while a setting that needs a lookup is set:
|
the start while a setting that needs a lookup is set:
|
||||||
- `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES`;
|
- `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES`;
|
||||||
- `SWW_ADD_LOOKUP_HEADERS`;
|
- `SWW_ADD_LOOKUP_HEADERS`;
|
||||||
- `SWW_TAG_DENY` or `SWW_TAG_LIMIT_PERCENT` naming a `country=` tag, or an
|
- `SWW_TAG_DENY` or `SWW_TAG_LIMIT_PERCENT` naming an `asn=` or `country=`
|
||||||
`asn=` tag while `SWW_ROUTEWATCH_URL` is unset;
|
tag;
|
||||||
- the two byte percentage lists and `SWW_ASN_LIMIT_PERCENT_URL`;
|
- the two byte percentage lists and `SWW_ASN_LIMIT_PERCENT_URL`;
|
||||||
- `SWW_UNKNOWN_LIMIT_PERCENT` below `100`;
|
- `SWW_UNKNOWN_LIMIT_PERCENT` below `100`;
|
||||||
- an anomaly threshold per AS number.
|
- an anomaly threshold per AS number.
|
||||||
@@ -187,11 +174,10 @@ Every request gets tags from what `discriminator` knows of its client. See
|
|||||||
it.
|
it.
|
||||||
|
|
||||||
- `SWW_ROUTEWATCH_URL` (default unset): the `http` or `https` URL of a
|
- `SWW_ROUTEWATCH_URL` (default unset): the `http` or `https` URL of a
|
||||||
routewatch instance. [routewatch](https://git.eeqj.de/sneak/routewatch) is a
|
routewatch instance. `discriminator` asks it about each client, in the
|
||||||
service that gives an address's announced netblock and origin AS.
|
background, and adds the tags `asn=<number>` and `prefix=<netblock>` from its
|
||||||
`discriminator` asks it about each client, in the background, and adds the
|
answer. While it is unset, nothing asks routewatch and no request gets a
|
||||||
tags `asn=<number>` and `prefix=<netblock>` from its answer. While it is
|
`prefix=` tag.
|
||||||
unset, nothing asks routewatch and no request gets a `prefix=` tag.
|
|
||||||
|
|
||||||
`SWW_TAG_DENY` and `SWW_TAG_LIMIT_PERCENT` accept tags in either case. A tag
|
`SWW_TAG_DENY` and `SWW_TAG_LIMIT_PERCENT` accept tags in either case. A tag
|
||||||
that no request can get stops the start. A request can get only these:
|
that no request can get stops the start. A request can get only these:
|
||||||
@@ -255,9 +241,8 @@ the request first.
|
|||||||
a range. So does a directive that removes or changes rules as the file loads,
|
a range. So does a directive that removes or changes rules as the file loads,
|
||||||
such as `SecRuleRemoveById`, since none of the Core Rule Set's rules is loaded
|
such as `SecRuleRemoveById`, since none of the Core Rule Set's rules is loaded
|
||||||
yet. It has no `_FILE` form.
|
yet. It has no `_FILE` form.
|
||||||
- `SWW_ERROR_BURST_THRESHOLD` (default `30`): the most refusals a client may get
|
- `SWW_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client in
|
||||||
in a minute from a rule file, a trap path or the Core Rule Set, a count or
|
a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
|
||||||
`off`. The next one bans the client (see "Attack detection" in `SPEC.md`).
|
|
||||||
|
|
||||||
## Ban lengths, response and scope
|
## Ban lengths, response and scope
|
||||||
|
|
||||||
@@ -334,10 +319,9 @@ the request first.
|
|||||||
|
|
||||||
## Anomaly thresholds
|
## Anomaly thresholds
|
||||||
|
|
||||||
Each scope has four thresholds: the most requests and the most body bytes
|
Each scope has four thresholds: the most requests and the most bytes counted in
|
||||||
counted in a minute and in an hour. The bytes are those `SWW_BYTES_COUNT` names.
|
a minute and in an hour. Requests are counts, bytes are sizes. Each is `off` by
|
||||||
Requests are counts, bytes are sizes. Each is `off` by default. The four
|
default. The four thresholds per client are:
|
||||||
thresholds per client are:
|
|
||||||
|
|
||||||
- `SWW_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
|
- `SWW_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
|
||||||
- `SWW_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
|
- `SWW_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
|
||||||
|
|||||||
+3
-4
@@ -61,11 +61,10 @@ since it was first seen:
|
|||||||
- the first and last time seen;
|
- the first and last time seen;
|
||||||
- the AS number, AS name and country last looked up, and when;
|
- the AS number, AS name and country last looked up, and when;
|
||||||
- total requests, and how many were forwarded and how many refused;
|
- total requests, and how many were forwarded and how many refused;
|
||||||
- body bytes of its requests, and of the responses it was sent;
|
- bytes in each direction;
|
||||||
- responses by status class;
|
- responses by status class;
|
||||||
- offences by kind: `limit`, `attack`, `blocked_by_block_rule` and
|
- offences by kind: `limit`, `attack`, `rule_blocked` and `waf_blocked`. A file
|
||||||
`waf_blocked`. A file may also hold `token_refused`, which is kept, and which
|
may also hold `token_refused`, which is kept, and which nothing adds to.
|
||||||
nothing adds to.
|
|
||||||
|
|
||||||
## `lookups.json`
|
## `lookups.json`
|
||||||
|
|
||||||
|
|||||||
@@ -341,48 +341,3 @@ func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
|
|||||||
|
|
||||||
wantChanged(t, ledger, false)
|
wantChanged(t, ledger, false)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPermanentBanOnAWiderNetblockIsReturnedOverATemporaryBan(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
client := netip.MustParseAddr("203.0.113.9")
|
|
||||||
ledger := bans.New(defaultRules())
|
|
||||||
|
|
||||||
// The temporary ban is made first, so that its netblock is looked at first.
|
|
||||||
ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"), midnight(), bans.Notes{})
|
|
||||||
ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.0/24"), midnight(), time.Time{},
|
|
||||||
"for good")
|
|
||||||
|
|
||||||
wantPermanentBan(t, ledger, client, midnight().Add(time.Minute))
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestOlderPermanentBanIsReturnedOverANewerTemporaryBan(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
netblock := netip.MustParsePrefix("203.0.113.0/24")
|
|
||||||
ledger := bans.New(defaultRules())
|
|
||||||
|
|
||||||
ledger.BanForAdmin(netblock, midnight(), time.Time{}, "for good")
|
|
||||||
ledger.BanForAdmin(netblock, midnight().Add(time.Minute), midnight().Add(time.Hour),
|
|
||||||
"for an hour")
|
|
||||||
|
|
||||||
wantPermanentBan(t, ledger, netblock.Addr(), midnight().Add(2*time.Minute))
|
|
||||||
}
|
|
||||||
|
|
||||||
// wantPermanentBan checks that Find and Check return a permanent ban for
|
|
||||||
// client at now.
|
|
||||||
func wantPermanentBan(
|
|
||||||
t *testing.T, ledger *bans.Ledger, client netip.Addr, now time.Time,
|
|
||||||
) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
for name, look := range map[string]func(netip.Addr, time.Time) (bans.Ban, bool, bool){
|
|
||||||
"Find": ledger.Find,
|
|
||||||
"Check": ledger.Check,
|
|
||||||
} {
|
|
||||||
ban, banned, _ := look(client, now)
|
|
||||||
if !banned || !ban.Permanent() {
|
|
||||||
t.Errorf("%s returned %+v and banned %v, want a permanent ban", name, ban, banned)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
+7
-29
@@ -274,27 +274,16 @@ func (l *Ledger) Find(client netip.Addr, now time.Time) (Ban, bool, bool) {
|
|||||||
|
|
||||||
// activeBan returns the ban in bans, a netblock's bans oldest first, that
|
// activeBan returns the ban in bans, a netblock's bans oldest first, that
|
||||||
// is active at now, or nil when none is. If several are, it returns the
|
// is active at now, or nil when none is. If several are, it returns the
|
||||||
// newest permanent one, or the newest when none is permanent. Every ban is
|
// one that started last. Every ban is looked at, since a ban an admin adds
|
||||||
// looked at, since a ban an admin adds to bans.json can start before the
|
// to bans.json can start before the netblock's others and outlast them.
|
||||||
// netblock's others and outlast them.
|
|
||||||
func activeBan(bans []Ban, now time.Time) *Ban {
|
func activeBan(bans []Ban, now time.Time) *Ban {
|
||||||
var newest *Ban
|
|
||||||
|
|
||||||
for i := len(bans) - 1; i >= 0; i-- {
|
for i := len(bans) - 1; i >= 0; i-- {
|
||||||
if !bans[i].ActiveAt(now) {
|
if bans[i].ActiveAt(now) {
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
if bans[i].Permanent() {
|
|
||||||
return &bans[i]
|
return &bans[i]
|
||||||
}
|
}
|
||||||
|
|
||||||
if newest == nil {
|
|
||||||
newest = &bans[i]
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return newest
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// BanForLimit bans netblock at now for a broken limit, with notes, and
|
// BanForLimit bans netblock at now for a broken limit, with notes, and
|
||||||
@@ -754,16 +743,13 @@ func (l *Ledger) markChanged() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// active returns the ban active at now on a netblock client is in, or
|
// active returns the ban active at now on a netblock client is in, or
|
||||||
// nil. If several netblocks have one, it returns a permanent ban when
|
// nil.
|
||||||
// there is one.
|
|
||||||
func (l *Ledger) active(client netip.Addr, now time.Time) *Ban {
|
func (l *Ledger) active(client netip.Addr, now time.Time) *Ban {
|
||||||
lengths := l.v6Lengths
|
lengths := l.v6Lengths
|
||||||
if client.Is4() {
|
if client.Is4() {
|
||||||
lengths = l.v4Lengths
|
lengths = l.v4Lengths
|
||||||
}
|
}
|
||||||
|
|
||||||
var first *Ban
|
|
||||||
|
|
||||||
for _, length := range lengths {
|
for _, length := range lengths {
|
||||||
bans, found := l.netblocks.Get(netip.PrefixFrom(client, length).Masked())
|
bans, found := l.netblocks.Get(netip.PrefixFrom(client, length).Masked())
|
||||||
if !found {
|
if !found {
|
||||||
@@ -771,20 +757,12 @@ func (l *Ledger) active(client netip.Addr, now time.Time) *Ban {
|
|||||||
}
|
}
|
||||||
|
|
||||||
ban := activeBan(*bans, now)
|
ban := activeBan(*bans, now)
|
||||||
if ban == nil {
|
if ban != nil {
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
if ban.Permanent() {
|
|
||||||
return ban
|
return ban
|
||||||
}
|
}
|
||||||
|
|
||||||
if first == nil {
|
|
||||||
first = ban
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return first
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// covering returns the bans of each netblock held that client is in,
|
// covering returns the bans of each netblock held that client is in,
|
||||||
|
|||||||
@@ -466,9 +466,6 @@ var (
|
|||||||
errNotLAPIURL = errors.New(
|
errNotLAPIURL = errors.New(
|
||||||
"is not an http or https URL without a user or a fragment, " +
|
"is not an http or https URL without a user or a fragment, " +
|
||||||
"such as http://172.17.0.1:8080")
|
"such as http://172.17.0.1:8080")
|
||||||
errNotRouteWatchURL = errors.New(
|
|
||||||
"is not an http or https URL without a user or a fragment, " +
|
|
||||||
"such as https://routewatch.example")
|
|
||||||
errNeedsLAPIKey = errors.New("the engine answers no request without it")
|
errNeedsLAPIKey = errors.New("the engine answers no request without it")
|
||||||
errLAPIKeyUnused = errors.New("it is sent only to the engine at that URL")
|
errLAPIKeyUnused = errors.New("it is sent only to the engine at that URL")
|
||||||
errAnotherList = errors.New(
|
errAnotherList = errors.New(
|
||||||
@@ -534,7 +531,7 @@ func FromEnvironment(
|
|||||||
TagLists: env.tagLists("SWW_TAG_LISTS"),
|
TagLists: env.tagLists("SWW_TAG_LISTS"),
|
||||||
TagDeny: env.tags("SWW_TAG_DENY"),
|
TagDeny: env.tags("SWW_TAG_DENY"),
|
||||||
TagLimitPercent: env.tagPercents("SWW_TAG_LIMIT_PERCENT"),
|
TagLimitPercent: env.tagPercents("SWW_TAG_LIMIT_PERCENT"),
|
||||||
RouteWatchURL: env.routeWatchURL("SWW_ROUTEWATCH_URL"),
|
RouteWatchURL: env.listURL("SWW_ROUTEWATCH_URL"),
|
||||||
ASNBytesPercent: env.percents("SWW_ASN_BYTES_PERCENT", ParseASN),
|
ASNBytesPercent: env.percents("SWW_ASN_BYTES_PERCENT", ParseASN),
|
||||||
CountryBytesPercent: env.percents("SWW_COUNTRY_BYTES_PERCENT", parseCountry),
|
CountryBytesPercent: env.percents("SWW_COUNTRY_BYTES_PERCENT", parseCountry),
|
||||||
UnknownLimitPercent: env.percent("SWW_UNKNOWN_LIMIT_PERCENT", "100"),
|
UnknownLimitPercent: env.percent("SWW_UNKNOWN_LIMIT_PERCENT", "100"),
|
||||||
@@ -1030,22 +1027,6 @@ func (e *environment) crowdSecDecisionsURL(name string) string {
|
|||||||
return lapi.JoinPath("v1", "decisions").String()
|
return lapi.JoinPath("v1", "decisions").String()
|
||||||
}
|
}
|
||||||
|
|
||||||
// routeWatchURL reads the setting that is the URL of a routewatch instance,
|
|
||||||
// "" while it is unset or empty.
|
|
||||||
func (e *environment) routeWatchURL(name string) string {
|
|
||||||
value := e.value(name, "")
|
|
||||||
if value == "" {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
instance, err := url.Parse(value)
|
|
||||||
if err != nil || !isHTTPURL(instance) {
|
|
||||||
e.check(name, fmt.Errorf("%q %w", value, errNotRouteWatchURL))
|
|
||||||
}
|
|
||||||
|
|
||||||
return value
|
|
||||||
}
|
|
||||||
|
|
||||||
// lookupSource reads the setting that is where clients are looked up:
|
// lookupSource reads the setting that is where clients are looked up:
|
||||||
// geojs, file, or off.
|
// geojs, file, or off.
|
||||||
func (e *environment) lookupSource(name, defaultValue string) string {
|
func (e *environment) lookupSource(name, defaultValue string) string {
|
||||||
@@ -1217,7 +1198,7 @@ func requestCanGetTag(cfg *Config, tag string) bool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// TagsNeedLookup reports whether SWW_TAG_DENY or SWW_TAG_LIMIT_PERCENT
|
// TagsNeedLookup reports whether SWW_TAG_DENY or SWW_TAG_LIMIT_PERCENT
|
||||||
// names an asn= or country= tag.
|
// names an asn= or country= tag, which only the lookup can give a request.
|
||||||
func (c *Config) TagsNeedLookup() bool {
|
func (c *Config) TagsNeedLookup() bool {
|
||||||
return slices.ContainsFunc(c.TagDeny, isLookupTag) || tagLimitNeedsLookups(c)
|
return slices.ContainsFunc(c.TagDeny, isLookupTag) || tagLimitNeedsLookups(c)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1450,15 +1450,6 @@ func TestRouteWatchURLAndPrefixTags(t *testing.T) {
|
|||||||
t.Errorf("%v gave the error %v, want one naming %s", stopped.env, err, stopped.name)
|
t.Errorf("%v gave the error %v, want one naming %s", stopped.env, err, stopped.name)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
badURL := environment{routeWatchURL: "ftp://routewatch.example"}
|
|
||||||
_, err := config.FromEnvironment(badURL.lookupEnv)
|
|
||||||
|
|
||||||
want := routeWatchURL + `: "ftp://routewatch.example" is not an http or https URL ` +
|
|
||||||
"without a user or a fragment, such as https://routewatch.example"
|
|
||||||
if err == nil || err.Error() != want {
|
|
||||||
t.Errorf("error %v, want %s", err, want)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestTagThatNoRequestCanGetIsToldApartFromOneItCan(t *testing.T) {
|
func TestTagThatNoRequestCanGetIsToldApartFromOneItCan(t *testing.T) {
|
||||||
|
|||||||
@@ -50,7 +50,7 @@ func (abuseIPDB) check(ctx context.Context, rq *request, _ time.Time) *verdict {
|
|||||||
}
|
}
|
||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionClientDenied,
|
action: requestlog.ActionDenied,
|
||||||
figures: map[string]any{
|
figures: map[string]any{
|
||||||
figureScore: score, "min_score": rq.h.config.AbuseIPDBMinScore,
|
figureScore: score, "min_score": rq.h.config.AbuseIPDBMinScore,
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -121,7 +121,7 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
|
|||||||
wantBans(t, s.admin(http.MethodPost, discriminator.BansPath, banOtherClient,
|
wantBans(t, s.admin(http.MethodPost, discriminator.BansPath, banOtherClient,
|
||||||
http.StatusOK), want)
|
http.StatusOK), want)
|
||||||
|
|
||||||
line := s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
line := s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
if line.BanExpires != requestlog.FormatTime(expires) {
|
if line.BanExpires != requestlog.FormatTime(expires) {
|
||||||
t.Errorf("the ban ends at %s, want %s", line.BanExpires, expires)
|
t.Errorf("the ban ends at %s, want %s", line.BanExpires, expires)
|
||||||
}
|
}
|
||||||
@@ -276,7 +276,7 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
|
|||||||
// limit of two a minute, and bans it.
|
// limit of two a minute, and bans it.
|
||||||
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(otherClient, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(otherClient, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
// Asked about by its address in IPv6 form too.
|
// Asked about by its address in IPv6 form too.
|
||||||
for _, addr := range []string{otherClient, "::ffff:" + otherClient} {
|
for _, addr := range []string{otherClient, "::ffff:" + otherClient} {
|
||||||
|
|||||||
@@ -31,14 +31,14 @@ func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
|
|||||||
start := clk.Now()
|
start := clk.Now()
|
||||||
|
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
netblock := netip.MustParsePrefix("203.0.113.0/24")
|
netblock := netip.MustParsePrefix("203.0.113.0/24")
|
||||||
ban := server.Ledger.Bans(netblock)[0]
|
ban := server.Ledger.Bans(netblock)[0]
|
||||||
|
|
||||||
// A request refused under the ban raises no other alert.
|
// A request refused under the ban raises no other alert.
|
||||||
clk.advance(time.Minute)
|
clk.advance(time.Minute)
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
|
||||||
wantAlerts(t, queue, banAlert(alerts.EventBan, start, client, bans.Ban{
|
wantAlerts(t, queue, banAlert(alerts.EventBan, start, client, bans.Ban{
|
||||||
Netblock: netblock, Cause: bans.CauseLimit,
|
Netblock: netblock, Cause: bans.CauseLimit,
|
||||||
@@ -62,24 +62,22 @@ func TestAttackBanRaisesABanAlertThenAPermanentBanAlert(t *testing.T) {
|
|||||||
|
|
||||||
// The probe bans the client for seven days, and its next request makes
|
// The probe bans the client for seven days, and its next request makes
|
||||||
// the ban permanent. The request after that changes nothing.
|
// the ban permanent. The request after that changes nothing.
|
||||||
s.request(client, "/.env", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.request(client, "/.env", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
|
||||||
attackBan := server.Ledger.Bans(netblock)[0]
|
attackBan := server.Ledger.Bans(netblock)[0]
|
||||||
|
|
||||||
clk.advance(time.Minute)
|
clk.advance(time.Minute)
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionPermanentlyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
|
||||||
permanentBan := server.Ledger.Bans(netblock)[0]
|
permanentBan := server.Ledger.Bans(netblock)[0]
|
||||||
|
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionPermanentlyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
|
||||||
// Another client's probe after its first ban has run out without a
|
// Another client's probe after its first ban has run out without a
|
||||||
// request makes a permanent ban at once.
|
// request makes a permanent ban at once.
|
||||||
s.request(otherClient, "/.env", http.StatusForbidden,
|
s.request(otherClient, "/.env", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
requestlog.ActionTemporarilyBanned)
|
|
||||||
clk.advance(7 * 24 * time.Hour)
|
clk.advance(7 * 24 * time.Hour)
|
||||||
s.request(otherClient, "/.env", http.StatusForbidden,
|
s.request(otherClient, "/.env", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
requestlog.ActionPermanentlyBanned)
|
|
||||||
|
|
||||||
otherBans := server.Ledger.Bans(other)
|
otherBans := server.Ledger.Bans(other)
|
||||||
|
|
||||||
|
|||||||
@@ -187,7 +187,7 @@ func TestEveryRequestIsCountedWhateverIsDoneWithIt(t *testing.T) {
|
|||||||
{allowed, "/", http.StatusOK, requestlog.ActionForward},
|
{allowed, "/", http.StatusOK, requestlog.ActionForward},
|
||||||
{exempt, "/", http.StatusOK, requestlog.ActionForward},
|
{exempt, "/", http.StatusOK, requestlog.ActionForward},
|
||||||
{client, "/static/app.js", http.StatusOK, requestlog.ActionForward},
|
{client, "/static/app.js", http.StatusOK, requestlog.ActionForward},
|
||||||
{denied, "/", http.StatusForbidden, requestlog.ActionClientDenied},
|
{denied, "/", http.StatusForbidden, requestlog.ActionDenied},
|
||||||
} {
|
} {
|
||||||
for range 3 {
|
for range 3 {
|
||||||
s.request(sent.from, sent.path, sent.status, sent.action)
|
s.request(sent.from, sent.path, sent.status, sent.action)
|
||||||
|
|||||||
@@ -84,7 +84,7 @@ func TestSixteenThousandCookiesAreRefusedWellWithinTheTimeLimit(t *testing.T) {
|
|||||||
wantDisposition(t, got, tooLarge, discriminator.DispositionDeny)
|
wantDisposition(t, got, tooLarge, discriminator.DispositionDeny)
|
||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
wantLine(t, line, tooLarge, requestlog.ActionRequestHeaderTooLarge)
|
wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
|
||||||
|
|
||||||
if line.DurationWAF != nil {
|
if line.DurationWAF != nil {
|
||||||
t.Errorf("the Core Rule Set read the request, for %f ms", *line.DurationWAF)
|
t.Errorf("the Core Rule Set read the request, for %f ms", *line.DurationWAF)
|
||||||
@@ -157,7 +157,7 @@ func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
|
|||||||
|
|
||||||
if tc.refused {
|
if tc.refused {
|
||||||
wantDisposition(t, got, tooLarge, discriminator.DispositionDeny)
|
wantDisposition(t, got, tooLarge, discriminator.DispositionDeny)
|
||||||
wantLine(t, line, tooLarge, requestlog.ActionRequestHeaderTooLarge)
|
wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -219,10 +219,9 @@ func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
|
|||||||
wantDisposition(t, got, http.StatusOK, discriminator.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, discriminator.DispositionAllow)
|
||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
if line.WouldAction != requestlog.ActionRequestHeaderTooLarge ||
|
if line.WouldAction != requestlog.ActionTooLarge || line.DurationWAF != nil {
|
||||||
line.DurationWAF != nil {
|
|
||||||
t.Errorf("would_action %q and duration_waf %v, want %q and none",
|
t.Errorf("would_action %q and duration_waf %v, want %q and none",
|
||||||
line.WouldAction, line.DurationWAF, requestlog.ActionRequestHeaderTooLarge)
|
line.WouldAction, line.DurationWAF, requestlog.ActionTooLarge)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -241,7 +240,7 @@ func TestRequestsOverTheHeaderLimitsCountTowardTheRateLimits(t *testing.T) {
|
|||||||
wantDisposition(t, ask(), http.StatusForbidden, discriminator.DispositionDeny)
|
wantDisposition(t, ask(), http.StatusForbidden, discriminator.DispositionDeny)
|
||||||
|
|
||||||
lines := s.out.requestLines(t, 2)
|
lines := s.out.requestLines(t, 2)
|
||||||
wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestTheHeaderLimitsComeAfterTheTrapPathsAndTheRuleFiles(t *testing.T) {
|
func TestTheHeaderLimitsComeAfterTheTrapPathsAndTheRuleFiles(t *testing.T) {
|
||||||
@@ -250,8 +249,8 @@ func TestTheHeaderLimitsComeAfterTheTrapPathsAndTheRuleFiles(t *testing.T) {
|
|||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
name, path, action string
|
name, path, action string
|
||||||
}{
|
}{
|
||||||
{"a trap path", "/wp-login.php", requestlog.ActionTemporarilyBanned},
|
{"a trap path", "/wp-login.php", requestlog.ActionBanned},
|
||||||
{"a block rule", "/blocked", requestlog.ActionBlockedByBlockRule},
|
{"a block rule", "/blocked", requestlog.ActionRuleBlocked},
|
||||||
} {
|
} {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
@@ -281,7 +280,7 @@ func TestNotWaitingForGeoJSNeverLowersAClientsLimit(t *testing.T) {
|
|||||||
wantDisposition(t, do(t, req), tooLarge, discriminator.DispositionDeny)
|
wantDisposition(t, do(t, req), tooLarge, discriminator.DispositionDeny)
|
||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
wantLine(t, line, tooLarge, requestlog.ActionRequestHeaderTooLarge)
|
wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
|
||||||
|
|
||||||
if line.LimitPercent != nil {
|
if line.LimitPercent != nil {
|
||||||
t.Errorf("limit percentage %d, want none", *line.LimitPercent)
|
t.Errorf("limit percentage %d, want none", *line.LimitPercent)
|
||||||
@@ -305,8 +304,7 @@ func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T
|
|||||||
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
||||||
|
|
||||||
wantDisposition(t, do(t, req), http.StatusForbidden, discriminator.DispositionDeny)
|
wantDisposition(t, do(t, req), http.StatusForbidden, discriminator.DispositionDeny)
|
||||||
wantLine(t, s.out.requestLine(t), http.StatusForbidden,
|
wantLine(t, s.out.requestLine(t), http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
requestlog.ActionClientIsRateLimited)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestObserveModeWaitsForGeoJSForARequestOverTheHeaderLimits(t *testing.T) {
|
func TestObserveModeWaitsForGeoJSForARequestOverTheHeaderLimits(t *testing.T) {
|
||||||
@@ -414,37 +412,33 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
|
|||||||
}{
|
}{
|
||||||
{
|
{
|
||||||
name: "SWW_DENY_NETS", env: map[string]string{denyNets: client},
|
name: "SWW_DENY_NETS", env: map[string]string{denyNets: client},
|
||||||
prepare: leaveAsItIs, status: forbidden, action: requestlog.ActionClientDenied,
|
prepare: leaveAsItIs, status: forbidden, action: requestlog.ActionDenied,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "a ban",
|
name: "a ban",
|
||||||
prepare: banClient, status: forbidden, action: requestlog.ActionTemporarilyBanned,
|
prepare: banClient, status: forbidden, action: requestlog.ActionBanned,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "the rate limits", env: map[string]string{rateLimitPerMinute: "1"},
|
name: "the rate limits", env: map[string]string{rateLimitPerMinute: "1"},
|
||||||
prepare: countClient, status: forbidden,
|
prepare: countClient, status: forbidden, action: requestlog.ActionRateLimited,
|
||||||
action: requestlog.ActionClientIsRateLimited,
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "the rate limits, with a biased threshold",
|
name: "the rate limits, with a biased threshold",
|
||||||
env: map[string]string{rateLimitPerMinute: "1", unknownLimitPercent: "50"},
|
env: map[string]string{rateLimitPerMinute: "1", unknownLimitPercent: "50"},
|
||||||
prepare: countClient, status: forbidden,
|
prepare: countClient, status: forbidden, action: requestlog.ActionRateLimited,
|
||||||
action: requestlog.ActionClientIsRateLimited,
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "a blocklist", env: map[string]string{blocklistURLs: dropURL},
|
name: "a blocklist", env: map[string]string{blocklistURLs: dropURL},
|
||||||
prepare: listClient, status: forbidden, action: requestlog.ActionClientDenied,
|
prepare: listClient, status: forbidden, action: requestlog.ActionDenied,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "the CrowdSec decision list",
|
name: "the CrowdSec decision list",
|
||||||
env: map[string]string{crowdSecURL: lapi, crowdSecKey: bouncerKey},
|
env: map[string]string{crowdSecURL: lapi, crowdSecKey: bouncerKey},
|
||||||
prepare: decideOnClient, status: forbidden,
|
prepare: decideOnClient, status: forbidden, action: requestlog.ActionBanned,
|
||||||
action: requestlog.ActionTemporarilyBanned,
|
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
name: "the header limits", cookie: cookies(cookieLimit + 1),
|
name: "the header limits", cookie: cookies(cookieLimit + 1),
|
||||||
prepare: leaveAsItIs, status: tooLarge,
|
prepare: leaveAsItIs, status: tooLarge, action: requestlog.ActionTooLarge,
|
||||||
action: requestlog.ActionRequestHeaderTooLarge,
|
|
||||||
},
|
},
|
||||||
} {
|
} {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
|||||||
@@ -50,23 +50,13 @@ func (activeBan) check(_ context.Context, rq *request, now time.Time) *verdict {
|
|||||||
}
|
}
|
||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: banAction(ban.Permanent()),
|
action: requestlog.ActionBanned,
|
||||||
figures: map[string]any{
|
figures: map[string]any{
|
||||||
figureNetblock: ban.Netblock.String(), "ban_expires": rq.line.BanExpires,
|
figureNetblock: ban.Netblock.String(), "ban_expires": rq.line.BanExpires,
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// banAction is the action of a request refused under a ban or for making one:
|
|
||||||
// permanent says the ban never runs out.
|
|
||||||
func banAction(permanent bool) string {
|
|
||||||
if permanent {
|
|
||||||
return requestlog.ActionPermanentlyBanned
|
|
||||||
}
|
|
||||||
|
|
||||||
return requestlog.ActionTemporarilyBanned
|
|
||||||
}
|
|
||||||
|
|
||||||
// limitFigures are the figures of a heuristic that broke hit, a limit: its
|
// limitFigures are the figures of a heuristic that broke hit, a limit: its
|
||||||
// window, the count and the limit.
|
// window, the count and the limit.
|
||||||
func limitFigures(hit ratelimit.Hit) map[string]any {
|
func limitFigures(hit ratelimit.Hit) map[string]any {
|
||||||
@@ -161,11 +151,11 @@ func (rq *request) banForLimit(now time.Time, hit ratelimit.Hit, status int) {
|
|||||||
// attack, which notes name: the ban rule that matched, or the trap path
|
// attack, which notes name: the ban rule that matched, or the trap path
|
||||||
// asked for. It fills in the rest of the notes. In observe mode it makes
|
// asked for. It fills in the rest of the notes. In observe mode it makes
|
||||||
// no ban, and raises the alert for the ban it would have made, if that
|
// no ban, and raises the alert for the ban it would have made, if that
|
||||||
// alert would be sent. It returns the action for that ban.
|
// alert would be sent.
|
||||||
func (rq *request) banForAttack(now time.Time, notes bans.Notes) string {
|
func (rq *request) banForAttack(now time.Time, notes bans.Notes) {
|
||||||
netblock := rq.h.netblock(rq.client)
|
netblock := rq.h.netblock(rq.client)
|
||||||
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseAttack) {
|
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseAttack) {
|
||||||
return banAction(rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseAttack))
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
notes.ASN = rq.line.ASN
|
notes.ASN = rq.line.ASN
|
||||||
@@ -181,7 +171,7 @@ func (rq *request) banForAttack(now time.Time, notes bans.Notes) string {
|
|||||||
rq.alertBan(ban)
|
rq.alertBan(ban)
|
||||||
}
|
}
|
||||||
|
|
||||||
return banAction(ban.Permanent())
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
ban, made := rq.h.ledger.BanForAttack(netblock, now, notes)
|
ban, made := rq.h.ledger.BanForAttack(netblock, now, notes)
|
||||||
@@ -190,18 +180,16 @@ func (rq *request) banForAttack(now time.Time, notes bans.Notes) string {
|
|||||||
if made {
|
if made {
|
||||||
rq.alertBan(ban)
|
rq.alertBan(ban)
|
||||||
}
|
}
|
||||||
|
|
||||||
return banAction(ban.Permanent())
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// banForCrowdSec bans the client's netblock at now until decision,
|
// banForCrowdSec bans the client's netblock at now until decision,
|
||||||
// CrowdSec's decision on the client, ends. In observe mode it makes no
|
// CrowdSec's decision on the client, ends. In observe mode it makes no
|
||||||
// ban, and raises the alert for the ban it would have made, if that alert
|
// ban, and raises the alert for the ban it would have made, if that alert
|
||||||
// would be sent. It returns the action for that ban.
|
// would be sent.
|
||||||
func (rq *request) banForCrowdSec(now time.Time, decision reputation.Decision) string {
|
func (rq *request) banForCrowdSec(now time.Time, decision reputation.Decision) {
|
||||||
netblock := rq.h.netblock(rq.client)
|
netblock := rq.h.netblock(rq.client)
|
||||||
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseCrowdSec) {
|
if rq.h.config.Observe && !rq.wouldAlertBan(netblock, now, bans.CauseCrowdSec) {
|
||||||
return banAction(rq.h.ledger.WouldBePermanent(netblock, now, bans.CauseCrowdSec))
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
notes := bans.Notes{
|
notes := bans.Notes{
|
||||||
@@ -220,7 +208,7 @@ func (rq *request) banForCrowdSec(now time.Time, decision reputation.Decision) s
|
|||||||
rq.alertBan(ban)
|
rq.alertBan(ban)
|
||||||
}
|
}
|
||||||
|
|
||||||
return banAction(ban.Permanent())
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
ban, made := rq.h.ledger.BanForCrowdSec(netblock, now, decision.Expires,
|
ban, made := rq.h.ledger.BanForCrowdSec(netblock, now, decision.Expires,
|
||||||
@@ -230,8 +218,6 @@ func (rq *request) banForCrowdSec(now time.Time, decision reputation.Decision) s
|
|||||||
if made {
|
if made {
|
||||||
rq.alertBan(ban)
|
rq.alertBan(ban)
|
||||||
}
|
}
|
||||||
|
|
||||||
return banAction(ban.Permanent())
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// wouldAlertBan reports whether the alert for a ban on netblock for cause
|
// wouldAlertBan reports whether the alert for a ban on netblock for cause
|
||||||
|
|||||||
@@ -40,7 +40,7 @@ func TestBrokenLimitBansTheClient(t *testing.T) {
|
|||||||
// client for an hour, the default.
|
// client for an hour, the default.
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
line := s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
line := s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
if line.LimitHit != minute || line.Offence != requestlog.OffenceLimit ||
|
if line.LimitHit != minute || line.Offence != requestlog.OffenceLimit ||
|
||||||
line.BanExpires != expires {
|
line.BanExpires != expires {
|
||||||
t.Errorf("log line has limit_hit %q, offence %q and ban_expires %q, "+
|
t.Errorf("log line has limit_hit %q, offence %q and ban_expires %q, "+
|
||||||
@@ -51,7 +51,7 @@ func TestBrokenLimitBansTheClient(t *testing.T) {
|
|||||||
// Every request while the ban lasts is refused.
|
// Every request while the ban lasts is refused.
|
||||||
clk.advance(time.Hour - time.Second)
|
clk.advance(time.Hour - time.Second)
|
||||||
|
|
||||||
line = s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
line = s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
if line.BanExpires != expires || line.Offence != "" || line.LimitHit != "" {
|
if line.BanExpires != expires || line.Offence != "" || line.LimitHit != "" {
|
||||||
t.Errorf("log line has ban_expires %q, offence %q and limit_hit %q, "+
|
t.Errorf("log line has ban_expires %q, offence %q and limit_hit %q, "+
|
||||||
"want %s and neither of the others", line.BanExpires, line.Offence,
|
"want %s and neither of the others", line.BanExpires, line.Offence,
|
||||||
@@ -78,9 +78,7 @@ func TestBanLengthsFollowTheSettings(t *testing.T) {
|
|||||||
breakLimit := func() string {
|
breakLimit := func() string {
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
line := s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
return s.get(client, http.StatusForbidden, requestlog.ActionRateLimited).BanExpires
|
||||||
|
|
||||||
return line.BanExpires
|
|
||||||
}
|
}
|
||||||
wantExpires := func(got string, length time.Duration) {
|
wantExpires := func(got string, length time.Duration) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
@@ -113,7 +111,7 @@ func TestBanLengthsFollowTheSettings(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
clk.advance(365 * 24 * time.Hour)
|
clk.advance(365 * 24 * time.Hour)
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionPermanentlyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBanIsNotCountedAndResetsTheCounters(t *testing.T) {
|
func TestBanIsNotCountedAndResetsTheCounters(t *testing.T) {
|
||||||
@@ -125,10 +123,10 @@ func TestBanIsNotCountedAndResetsTheCounters(t *testing.T) {
|
|||||||
// client for an hour.
|
// client for an hour.
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
for range 3 {
|
for range 3 {
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Later the same day the client has its whole allowance again: the
|
// Later the same day the client has its whole allowance again: the
|
||||||
@@ -137,7 +135,7 @@ func TestBanIsNotCountedAndResetsTheCounters(t *testing.T) {
|
|||||||
clk.advance(time.Hour)
|
clk.advance(time.Hour)
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
banned := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
|
banned := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
|
||||||
if len(banned) != 2 || banned[0].Notes.Refused != 3 {
|
if len(banned) != 2 || banned[0].Notes.Refused != 3 {
|
||||||
@@ -192,10 +190,10 @@ func TestBanCoversTheClientsNetblock(t *testing.T) {
|
|||||||
s, _, _ := startWithClock(t, "", env)
|
s, _, _ := startWithClock(t, "", env)
|
||||||
|
|
||||||
s.get(tc.breaker, http.StatusOK, requestlog.ActionForward)
|
s.get(tc.breaker, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(tc.breaker, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(tc.breaker, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
for _, sent := range tc.refused {
|
for _, sent := range tc.refused {
|
||||||
s.get(sent, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(sent, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, sent := range tc.let {
|
for _, sent := range tc.let {
|
||||||
@@ -218,9 +216,9 @@ func TestBannedClientIsRefusedBeforeItsCountryIsLookedUp(t *testing.T) {
|
|||||||
|
|
||||||
// fromDE's ban covers otherClient, which is refused unasked about.
|
// fromDE's ban covers otherClient, which is refused unasked about.
|
||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
line := s.get(otherClient, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
line := s.get(otherClient, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
if line.Country != "" {
|
if line.Country != "" {
|
||||||
t.Errorf("log line has country %q, want none", line.Country)
|
t.Errorf("log line has country %q, want none", line.Country)
|
||||||
}
|
}
|
||||||
@@ -239,9 +237,9 @@ func TestRefusalUnderABanNamesItsHeuristicNetblockAndEnd(t *testing.T) {
|
|||||||
expires := requestlog.FormatTime(clk.Now().Add(time.Hour))
|
expires := requestlog.FormatTime(clk.Now().Add(time.Hour))
|
||||||
|
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
line := s.get(otherClient, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
line := s.get(otherClient, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
wantHeuristic(t, line, "active_ban",
|
wantHeuristic(t, line, "active_ban",
|
||||||
`{"netblock": "`+clientsNet+`", "ban_expires": "`+expires+`"}`)
|
`{"netblock": "`+clientsNet+`", "ban_expires": "`+expires+`"}`)
|
||||||
}
|
}
|
||||||
@@ -277,11 +275,11 @@ func TestBanResponseAnswersEveryRefusalButTheSizeLimits(t *testing.T) {
|
|||||||
|
|
||||||
s, _, _ := startWithClock(t, geojsURL, env)
|
s, _, _ := startWithClock(t, geojsURL, env)
|
||||||
|
|
||||||
s.get(denied, tc.status, requestlog.ActionClientDenied)
|
s.get(denied, tc.status, requestlog.ActionDenied)
|
||||||
s.get(fromKP, tc.status, requestlog.ActionClientDenied)
|
s.get(fromKP, tc.status, requestlog.ActionDenied)
|
||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(fromDE, tc.status, requestlog.ActionClientIsRateLimited)
|
s.get(fromDE, tc.status, requestlog.ActionRateLimited)
|
||||||
s.get(fromDE, tc.status, requestlog.ActionTemporarilyBanned)
|
s.get(fromDE, tc.status, requestlog.ActionBanned)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -299,9 +297,9 @@ func TestBanNotes(t *testing.T) {
|
|||||||
|
|
||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
s.request(fromDE, "/repo/commits?page=2",
|
s.request(fromDE, "/repo/commits?page=2",
|
||||||
http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
|
||||||
netblock := netip.MustParsePrefix(fromDE + "/32")
|
netblock := netip.MustParsePrefix(fromDE + "/32")
|
||||||
want := bans.Ban{
|
want := bans.Ban{
|
||||||
@@ -344,7 +342,7 @@ func TestBanNotes(t *testing.T) {
|
|||||||
// The next ban counts this one among the earlier.
|
// The next ban counts this one among the earlier.
|
||||||
clk.advance(time.Hour)
|
clk.advance(time.Hour)
|
||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
got = ledger.Bans(netblock)
|
got = ledger.Bans(netblock)
|
||||||
if len(got) != 2 || got[1].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
if len(got) != 2 || got[1].Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
||||||
@@ -362,12 +360,12 @@ func TestMaxBansDropsTheBanOfTheNetblockSeenLongestAgo(t *testing.T) {
|
|||||||
|
|
||||||
// One ban is held, so otherClient's ban drops client's.
|
// One ban is held, so otherClient's ban drops client's.
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(otherClient, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(otherClient, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(otherClient, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(otherClient, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
// clock is the time a test sets, by which sww counts requests and
|
// clock is the time a test sets, by which sww counts requests and
|
||||||
|
|||||||
@@ -76,7 +76,7 @@ func TestEachBiasedThresholdLowersTheRateLimits(t *testing.T) {
|
|||||||
}{
|
}{
|
||||||
{http.StatusOK, requestlog.ActionForward},
|
{http.StatusOK, requestlog.ActionForward},
|
||||||
{http.StatusOK, requestlog.ActionForward},
|
{http.StatusOK, requestlog.ActionForward},
|
||||||
{http.StatusForbidden, requestlog.ActionClientIsRateLimited},
|
{http.StatusForbidden, requestlog.ActionRateLimited},
|
||||||
} {
|
} {
|
||||||
line := s.get(tc.from, sent.status, sent.action)
|
line := s.get(tc.from, sent.status, sent.action)
|
||||||
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
|
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
|
||||||
@@ -192,7 +192,7 @@ func TestZeroPercentIsAZeroAllowance(t *testing.T) {
|
|||||||
|
|
||||||
// The first request breaks the limit, and bans the client; the log line
|
// The first request breaks the limit, and bans the client; the log line
|
||||||
// gives the 0.
|
// gives the 0.
|
||||||
line := s.get(unplaced, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
line := s.get(unplaced, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
if line.fields["limit_percent"] != float64(0) ||
|
if line.fields["limit_percent"] != float64(0) ||
|
||||||
line.fields["limit_percent_setting"] != unknownLimitPercent {
|
line.fields["limit_percent_setting"] != unknownLimitPercent {
|
||||||
t.Errorf("log line has limit_percent %v from %v, want 0 from %s",
|
t.Errorf("log line has limit_percent %v from %v, want 0 from %s",
|
||||||
@@ -200,7 +200,7 @@ func TestZeroPercentIsAZeroAllowance(t *testing.T) {
|
|||||||
unknownLimitPercent)
|
unknownLimitPercent)
|
||||||
}
|
}
|
||||||
|
|
||||||
s.get(unplaced, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(unplaced, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestLowestPercentageApplies(t *testing.T) {
|
func TestLowestPercentageApplies(t *testing.T) {
|
||||||
@@ -250,7 +250,7 @@ func TestLowestPercentageApplies(t *testing.T) {
|
|||||||
// A quarter of 4 requests a minute: the second breaks the limit.
|
// A quarter of 4 requests a minute: the second breaks the limit.
|
||||||
s.get(tc.from, http.StatusOK, requestlog.ActionForward)
|
s.get(tc.from, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
line := s.get(tc.from, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
line := s.get(tc.from, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
|
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
|
||||||
tc.want)
|
tc.want)
|
||||||
})
|
})
|
||||||
@@ -269,7 +269,7 @@ func TestUnknownLimitPercentGivesEveryClientWithoutACountryItsPercentage(t *test
|
|||||||
for _, from := range []string{unplaced, "10.0.0.8"} {
|
for _, from := range []string{unplaced, "10.0.0.8"} {
|
||||||
s.get(from, http.StatusOK, requestlog.ActionForward)
|
s.get(from, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(from, http.StatusOK, requestlog.ActionForward)
|
s.get(from, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(from, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(from, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
}
|
}
|
||||||
|
|
||||||
// One in a country has the whole limit.
|
// One in a country has the whole limit.
|
||||||
@@ -293,7 +293,7 @@ func TestClientWithoutAnAnswerInTimeHasTheUnknownLimitPercent(t *testing.T) {
|
|||||||
serveFromDE(t, server, http.MethodGet, http.NoBody)
|
serveFromDE(t, server, http.MethodGet, http.NoBody)
|
||||||
|
|
||||||
line := out.requestLine(t)
|
line := out.requestLine(t)
|
||||||
wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
wantLine(t, line, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
|
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
|
||||||
"0 from "+unknownLimitPercent)
|
"0 from "+unknownLimitPercent)
|
||||||
})
|
})
|
||||||
@@ -374,7 +374,7 @@ func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(t *testing.T
|
|||||||
// A quarter of 12 requests a minute is 3: the fourth breaks it.
|
// A quarter of 12 requests a minute is 3: the fourth breaks it.
|
||||||
"a rate limit",
|
"a rate limit",
|
||||||
map[string]string{rateLimitPerMinute: "12", tagLimitPercent: asnTagDEQuarter},
|
map[string]string{rateLimitPerMinute: "12", tagLimitPercent: asnTagDEQuarter},
|
||||||
3, http.StatusForbidden, requestlog.ActionClientIsRateLimited,
|
3, http.StatusForbidden, requestlog.ActionRateLimited,
|
||||||
"requests per minute over the limit of 3", "25 from " + tagLimitPercent,
|
"requests per minute over the limit of 3", "25 from " + tagLimitPercent,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -28,7 +28,7 @@ func (blocklists) check(_ context.Context, rq *request, _ time.Time) *verdict {
|
|||||||
}
|
}
|
||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionClientDenied,
|
action: requestlog.ActionDenied,
|
||||||
figures: map[string]any{"lists": listedBy},
|
figures: map[string]any{"lists": listedBy},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -6,6 +6,8 @@ package discriminator
|
|||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
)
|
)
|
||||||
|
|
||||||
// byteLimits counts the request's bytes, as countedBytes gives them, for the
|
// byteLimits counts the request's bytes, as countedBytes gives them, for the
|
||||||
@@ -37,5 +39,5 @@ func (byteLimits) check(_ context.Context, rq *request, now time.Time) *verdict
|
|||||||
|
|
||||||
rq.banForLimit(now, hit, rq.out.status)
|
rq.banForLimit(now, hit, rq.out.status)
|
||||||
|
|
||||||
return &verdict{figures: limitFigures(hit)}
|
return &verdict{action: requestlog.ActionBanned, figures: limitFigures(hit)}
|
||||||
}
|
}
|
||||||
@@ -90,7 +90,7 @@ func TestEachByteLimitBansOnceTheResponseHasEnded(t *testing.T) {
|
|||||||
line.BanExpires, tc.window, expires)
|
line.BanExpires, tc.window, expires)
|
||||||
}
|
}
|
||||||
|
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
|
||||||
wantMetric(t, s.scrape(), `sww_rate_limit_hits_total{`+
|
wantMetric(t, s.scrape(), `sww_rate_limit_hits_total{`+
|
||||||
`instance="`+alertInstance+`",kind="bytes",window="`+tc.window+`"}`, 1)
|
`instance="`+alertInstance+`",kind="bytes",window="`+tc.window+`"}`, 1)
|
||||||
@@ -125,7 +125,7 @@ func TestResponseOverAByteLimitByItselfIsPassedOnWhole(t *testing.T) {
|
|||||||
len(got.body), got.err, line.LimitHit, answerBytes, minuteBytes)
|
len(got.body), got.err, line.LimitHit, answerBytes, minuteBytes)
|
||||||
}
|
}
|
||||||
|
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBytesOfAnAnswerThatBreaksOffAreCounted(t *testing.T) {
|
func TestBytesOfAnAnswerThatBreaksOffAreCounted(t *testing.T) {
|
||||||
@@ -147,7 +147,7 @@ func TestBytesOfAnAnswerThatBreaksOffAreCounted(t *testing.T) {
|
|||||||
answerBytes, minuteBytes, expires)
|
answerBytes, minuteBytes, expires)
|
||||||
}
|
}
|
||||||
|
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestWebSocketBytesAreCountedOnceItCloses(t *testing.T) {
|
func TestWebSocketBytesAreCountedOnceItCloses(t *testing.T) {
|
||||||
@@ -176,7 +176,7 @@ func TestWebSocketBytesAreCountedOnceItCloses(t *testing.T) {
|
|||||||
line.LimitHit, line.Counts.MinuteBytes, minuteBytes, tc.counted)
|
line.LimitHit, line.Counts.MinuteBytes, minuteBytes, tc.counted)
|
||||||
}
|
}
|
||||||
|
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -218,7 +218,7 @@ func TestBytesCountSaysWhichBytesCount(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -386,7 +386,7 @@ func TestObserveModeLeavesOutTheBytesOfARequestEnforceModeRefuses(t *testing.T)
|
|||||||
// counted, and the byte limit is not broken. Its line gives the bytes
|
// counted, and the byte limit is not broken. Its line gives the bytes
|
||||||
// counted before it.
|
// counted before it.
|
||||||
line := s.upload()
|
line := s.upload()
|
||||||
wantWouldAction(t, line, requestlog.ActionClientIsRateLimited)
|
wantWouldAction(t, line, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
if line.LimitHit != minute || line.Counts.MinuteBytes != bodyBytes+answerBytes {
|
if line.LimitHit != minute || line.Counts.MinuteBytes != bodyBytes+answerBytes {
|
||||||
t.Errorf("log line has limit_hit %q and minute_bytes %v, want minute and %d",
|
t.Errorf("log line has limit_hit %q and minute_bytes %v, want minute and %d",
|
||||||
|
|||||||
@@ -92,8 +92,7 @@ func TestDetectedAttackIsRefusedByTagDenyLast(t *testing.T) {
|
|||||||
wafMode: detect, tagDeny: "exploitsignature",
|
wafMode: detect, tagDeny: "exploitsignature",
|
||||||
})
|
})
|
||||||
|
|
||||||
line := s.request(client, sqlInjection, http.StatusForbidden,
|
line := s.request(client, sqlInjection, http.StatusForbidden, requestlog.ActionDenied)
|
||||||
requestlog.ActionClientDenied)
|
|
||||||
wantHeuristic(t, line, "tag_deny", `{"tag": "exploitsignature"}`)
|
wantHeuristic(t, line, "tag_deny", `{"tag": "exploitsignature"}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -196,7 +195,7 @@ func TestCoreRuleSetIsNotRunWhenOffOrForAnExemptClientPathOrRuleFileRefusal(
|
|||||||
|
|
||||||
// A request a rule file refuses is not inspected.
|
// A request a rule file refuses is not inspected.
|
||||||
line = s.request(otherClient, "/blocked?id=1'%20OR%20'1'='1", http.StatusForbidden,
|
line = s.request(otherClient, "/blocked?id=1'%20OR%20'1'='1", http.StatusForbidden,
|
||||||
requestlog.ActionBlockedByBlockRule)
|
requestlog.ActionRuleBlocked)
|
||||||
wantWAF(t, line, nil)
|
wantWAF(t, line, nil)
|
||||||
|
|
||||||
// With SWW_WAF_MODE off, no request is.
|
// With SWW_WAF_MODE off, no request is.
|
||||||
@@ -404,7 +403,7 @@ func TestCoreRuleSetRefusalIsAnOffenceAndCountsTowardTheErrorBurst(t *testing.T)
|
|||||||
line.LimitHit, line.Offence)
|
line.LimitHit, line.Offence)
|
||||||
}
|
}
|
||||||
|
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
|
||||||
want := ratelimit.Offences{Limit: 1, WAFBlocked: 3}
|
want := ratelimit.Offences{Limit: 1, WAFBlocked: 3}
|
||||||
if offences := historyOf(t, server, client).Offences; offences != want {
|
if offences := historyOf(t, server, client).Offences; offences != want {
|
||||||
|
|||||||
@@ -30,9 +30,10 @@ func (crowdSec) check(_ context.Context, rq *request, now time.Time) *verdict {
|
|||||||
rq.addTag(config.CrowdSecTag)
|
rq.addTag(config.CrowdSecTag)
|
||||||
rq.noteHit(bans.ReputationHit{Source: rq.h.config.CrowdSecDecisionsURL},
|
rq.noteHit(bans.ReputationHit{Source: rq.h.config.CrowdSecDecisionsURL},
|
||||||
"listed by the CrowdSec decision list")
|
"listed by the CrowdSec decision list")
|
||||||
|
rq.banForCrowdSec(now, decision)
|
||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: rq.banForCrowdSec(now, decision),
|
action: requestlog.ActionBanned,
|
||||||
figures: map[string]any{
|
figures: map[string]any{
|
||||||
"scenario": decision.Scenario,
|
"scenario": decision.Scenario,
|
||||||
"expires": requestlog.FormatTime(decision.Expires),
|
"expires": requestlog.FormatTime(decision.Expires),
|
||||||
|
|||||||
@@ -39,7 +39,7 @@ func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(t *testi
|
|||||||
expires := requestlog.FormatTime(fetched.Add(4 * time.Hour))
|
expires := requestlog.FormatTime(fetched.Add(4 * time.Hour))
|
||||||
|
|
||||||
// Its first request is refused, and bans it until the decision ends.
|
// Its first request is refused, and bans it until the decision ends.
|
||||||
line := s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
line := s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
wantReputation(t, line, decisionsURL)
|
wantReputation(t, line, decisionsURL)
|
||||||
|
|
||||||
if line.BanExpires != expires {
|
if line.BanExpires != expires {
|
||||||
@@ -72,7 +72,7 @@ func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(t *testi
|
|||||||
// ban, and once it has ended the client is let through.
|
// ban, and once it has ended the client is let through.
|
||||||
clk.advance(4*time.Hour - time.Second)
|
clk.advance(4*time.Hour - time.Second)
|
||||||
|
|
||||||
line = s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
line = s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
wantReputation(t, line)
|
wantReputation(t, line)
|
||||||
|
|
||||||
if line.BanExpires != expires {
|
if line.BanExpires != expires {
|
||||||
@@ -105,7 +105,7 @@ func TestCrowdSecBanNamesItsHeuristicScenarioAndEnd(t *testing.T) {
|
|||||||
`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
|
`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
|
||||||
`"value": "`+client+`"}]`)
|
`"value": "`+client+`"}]`)
|
||||||
|
|
||||||
line := s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
line := s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
wantHeuristic(t, line, "crowdsec", `{"scenario": "crowdsecurity/ssh-bf", `+
|
wantHeuristic(t, line, "crowdsec", `{"scenario": "crowdsecurity/ssh-bf", `+
|
||||||
`"expires": "`+requestlog.FormatTime(clk.Now().Add(4*time.Hour))+`"}`)
|
`"expires": "`+requestlog.FormatTime(clk.Now().Add(4*time.Hour))+`"}`)
|
||||||
wantTags(t, line, "crowdsec")
|
wantTags(t, line, "crowdsec")
|
||||||
@@ -129,9 +129,9 @@ func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt(t *testing.T
|
|||||||
// Just before its end, the decision bans a client in the netblock, and
|
// Just before its end, the decision bans a client in the netblock, and
|
||||||
// one on an IPv6 address bans the address's group, the /64.
|
// one on an IPv6 address bans the address's group, the /64.
|
||||||
clk.advance(time.Minute - time.Nanosecond)
|
clk.advance(time.Minute - time.Nanosecond)
|
||||||
s.get("198.51.100.7", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get("198.51.100.7", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
s.get("2001:db8::9", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get("2001:db8::9", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
s.get("2001:db8::5", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get("2001:db8::5", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
|
||||||
// Once it has ended, it bans no other client, and the bans it made end
|
// Once it has ended, it bans no other client, and the bans it made end
|
||||||
// with it.
|
// with it.
|
||||||
@@ -164,7 +164,7 @@ func TestObserveModeForwardsAClientTheCrowdSecDecisionListListsAndAlertsTheBan(
|
|||||||
`"value": "`+client+`"}]`)
|
`"value": "`+client+`"}]`)
|
||||||
|
|
||||||
line := s.get(client, http.StatusOK, requestlog.ActionForward)
|
line := s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionTemporarilyBanned)
|
wantWouldAction(t, line, requestlog.ActionBanned)
|
||||||
wantReputation(t, line, decisionsURL)
|
wantReputation(t, line, decisionsURL)
|
||||||
|
|
||||||
if held := server.Ledger.Snapshot(); len(held) != 0 {
|
if held := server.Ledger.Snapshot(); len(held) != 0 {
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ func (denyNets) check(_ context.Context, rq *request, _ time.Time) *verdict {
|
|||||||
for _, netblock := range rq.h.config.DenyNets {
|
for _, netblock := range rq.h.config.DenyNets {
|
||||||
if netblock.Contains(rq.client) {
|
if netblock.Contains(rq.client) {
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionClientDenied,
|
action: requestlog.ActionDenied,
|
||||||
figures: map[string]any{figureNetblock: netblock.String()},
|
figures: map[string]any{figureNetblock: netblock.String()},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -639,7 +639,7 @@ func TestQuestionDeniedGetsTheBanResponse(t *testing.T) {
|
|||||||
got := s.ask(client, "/")
|
got := s.ask(client, "/")
|
||||||
wantDisposition(t, got, status, discriminator.DispositionDeny)
|
wantDisposition(t, got, status, discriminator.DispositionDeny)
|
||||||
wantBody(t, got, http.StatusText(status)+"\n")
|
wantBody(t, got, http.StatusText(status)+"\n")
|
||||||
wantLine(t, s.out.requestLine(t), status, requestlog.ActionClientDenied)
|
wantLine(t, s.out.requestLine(t), status, requestlog.ActionDenied)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -655,8 +655,8 @@ func TestBanResponseCloseIsAnAnswerOfItsOwn(t *testing.T) {
|
|||||||
wantDisposition(t, got, http.StatusOK, discriminator.DispositionClose)
|
wantDisposition(t, got, http.StatusOK, discriminator.DispositionClose)
|
||||||
wantBody(t, got, "")
|
wantBody(t, got, "")
|
||||||
// The visitor is sent nothing.
|
// The visitor is sent nothing.
|
||||||
wantLine(t, s.out.requestLine(t), 0, requestlog.ActionClientDenied)
|
wantLine(t, s.out.requestLine(t), 0, requestlog.ActionDenied)
|
||||||
wantMetric(t, s.metrics(), `sww_requests_total{action="client_denied",instance="`+
|
wantMetric(t, s.metrics(), `sww_requests_total{action="denied",instance="`+
|
||||||
alertInstance+`",status_class="none"}`, 1)
|
alertInstance+`",status_class="none"}`, 1)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -671,12 +671,12 @@ func TestBlockRuleIsDeniedWith403AndABanRuleWithTheBanResponse(t *testing.T) {
|
|||||||
got := s.ask(client, "/blocked")
|
got := s.ask(client, "/blocked")
|
||||||
wantDisposition(t, got, http.StatusForbidden, discriminator.DispositionDeny)
|
wantDisposition(t, got, http.StatusForbidden, discriminator.DispositionDeny)
|
||||||
wantLine(t, s.out.requestLine(t), http.StatusForbidden,
|
wantLine(t, s.out.requestLine(t), http.StatusForbidden,
|
||||||
requestlog.ActionBlockedByBlockRule)
|
requestlog.ActionRuleBlocked)
|
||||||
|
|
||||||
got = s.ask(client, "/.env")
|
got = s.ask(client, "/.env")
|
||||||
wantDisposition(t, got, http.StatusTooManyRequests, discriminator.DispositionDeny)
|
wantDisposition(t, got, http.StatusTooManyRequests, discriminator.DispositionDeny)
|
||||||
wantLine(t, s.out.requestLines(t, 2)[1], http.StatusTooManyRequests,
|
wantLine(t, s.out.requestLines(t, 2)[1], http.StatusTooManyRequests,
|
||||||
requestlog.ActionTemporarilyBanned)
|
requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestQuestionUnderTheOwnPrefixIsDeniedWith404AfterTheChecks(t *testing.T) {
|
func TestQuestionUnderTheOwnPrefixIsDeniedWith404AfterTheChecks(t *testing.T) {
|
||||||
@@ -709,7 +709,7 @@ func TestObserveModeAllowsAndLogsWhatEnforceModeWouldHaveDone(t *testing.T) {
|
|||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
|
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionClientDenied)
|
wantWouldAction(t, line, requestlog.ActionDenied)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestObserveModeRefusalAnsweredAllowCountsForTheErrorBurst(t *testing.T) {
|
func TestObserveModeRefusalAnsweredAllowCountsForTheErrorBurst(t *testing.T) {
|
||||||
@@ -894,7 +894,7 @@ func TestDeniedRequestIsFinishedAtItsDecision(t *testing.T) {
|
|||||||
t.Errorf("history is %+v, want the refused request alone", history)
|
t.Errorf("history is %+v, want the refused request alone", history)
|
||||||
}
|
}
|
||||||
|
|
||||||
const series = `sww_requests_total{action="client_is_rate_limited",instance="` +
|
const series = `sww_requests_total{action="rate_limited",instance="` +
|
||||||
alertInstance + `",status_class="4xx"}`
|
alertInstance + `",status_class="4xx"}`
|
||||||
|
|
||||||
wantMetric(t, s.metrics(), series, 1)
|
wantMetric(t, s.metrics(), series, 1)
|
||||||
@@ -1298,11 +1298,11 @@ func TestEveryDeniedRequestsReportGetsItsLastLine(t *testing.T) {
|
|||||||
}{
|
}{
|
||||||
{
|
{
|
||||||
"deny", map[string]string{denyNets: client}, "/", http.StatusForbidden,
|
"deny", map[string]string{denyNets: client}, "/", http.StatusForbidden,
|
||||||
requestlog.ActionClientDenied,
|
requestlog.ActionDenied,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"close", map[string]string{denyNets: client, banResponse: banResponseClose}, "/",
|
"close", map[string]string{denyNets: client, banResponse: banResponseClose}, "/",
|
||||||
0, requestlog.ActionClientDenied,
|
0, requestlog.ActionDenied,
|
||||||
},
|
},
|
||||||
{"own prefix", nil, discriminator.HealthPath, http.StatusNotFound,
|
{"own prefix", nil, discriminator.HealthPath, http.StatusNotFound,
|
||||||
requestlog.ActionAdmin},
|
requestlog.ActionAdmin},
|
||||||
|
|||||||
@@ -37,7 +37,7 @@ func (dnsbl) check(ctx context.Context, rq *request, _ time.Time) *verdict {
|
|||||||
}
|
}
|
||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionClientDenied,
|
action: requestlog.ActionDenied,
|
||||||
figures: map[string]any{"zones": listedBy},
|
figures: map[string]any{"zones": listedBy},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -30,7 +30,7 @@ func (headerLimits) check(_ context.Context, rq *request, _ time.Time) *verdict
|
|||||||
}
|
}
|
||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionRequestHeaderTooLarge,
|
action: requestlog.ActionTooLarge,
|
||||||
figures: map[string]any{
|
figures: map[string]any{
|
||||||
"fields": fields, "fields_limit": maxHeaderFields,
|
"fields": fields, "fields_limit": maxHeaderFields,
|
||||||
"cookies": cookies, "cookies_limit": maxCookies,
|
"cookies": cookies, "cookies_limit": maxCookies,
|
||||||
|
|||||||
@@ -31,9 +31,9 @@ func TestHistoryKeepsEachRequestOfTheClient(t *testing.T) {
|
|||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
clk.advance(time.Second)
|
clk.advance(time.Second)
|
||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
clk.advance(time.Second)
|
clk.advance(time.Second)
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
|
||||||
want := ratelimit.History{
|
want := ratelimit.History{
|
||||||
FirstSeen: start,
|
FirstSeen: start,
|
||||||
|
|||||||
@@ -40,7 +40,7 @@ func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(t *testing.T)
|
|||||||
// comes too. None waits for GeoJS.
|
// comes too. None waits for GeoJS.
|
||||||
for _, line := range []logLine{
|
for _, line := range []logLine{
|
||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward),
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward),
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited),
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited),
|
||||||
s.get(fromKP, http.StatusOK, requestlog.ActionForward),
|
s.get(fromKP, http.StatusOK, requestlog.ActionForward),
|
||||||
} {
|
} {
|
||||||
got := asnAndCountry{line.ASN, line.ASName, line.Country}
|
got := asnAndCountry{line.ASN, line.ASName, line.Country}
|
||||||
@@ -179,7 +179,7 @@ func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert(t *testing.
|
|||||||
|
|
||||||
for _, line := range []logLine{
|
for _, line := range []logLine{
|
||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward),
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward),
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited),
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited),
|
||||||
} {
|
} {
|
||||||
if got := (asnAndCountry{line.ASN, line.ASName, line.Country}); got != want {
|
if got := (asnAndCountry{line.ASN, line.ASName, line.Country}); got != want {
|
||||||
t.Errorf("log line has %+v, want %+v", got, want)
|
t.Errorf("log line has %+v, want %+v", got, want)
|
||||||
@@ -264,7 +264,7 @@ func TestClientsAreLookedUpInTheLookupDatabaseAndGeoJSIsNotAsked(t *testing.T) {
|
|||||||
want asnAndCountry
|
want asnAndCountry
|
||||||
}{
|
}{
|
||||||
{s.get(fromDE, http.StatusOK, requestlog.ActionForward), de},
|
{s.get(fromDE, http.StatusOK, requestlog.ActionForward), de},
|
||||||
{s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited), de},
|
{s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited), de},
|
||||||
{
|
{
|
||||||
s.get(fromKP, http.StatusForbidden, requestlog.ActionCountryDenied),
|
s.get(fromKP, http.StatusForbidden, requestlog.ActionCountryDenied),
|
||||||
asnAndCountry{asnKP, asNameKP, "KP"},
|
asnAndCountry{asnKP, asNameKP, "KP"},
|
||||||
|
|||||||
@@ -125,15 +125,15 @@ func TestMetricsCountLimitsAndBans(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
// SWW_BAN_RESPONSE=close sends no status at all.
|
// SWW_BAN_RESPONSE=close sends no status at all.
|
||||||
s.get(denied, 0, requestlog.ActionClientDenied)
|
s.get(denied, 0, requestlog.ActionDenied)
|
||||||
|
|
||||||
// A first broken limit bans for an hour.
|
// A first broken limit bans for an hour.
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(client, 0, requestlog.ActionClientIsRateLimited)
|
s.get(client, 0, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
metrics := s.scrape()
|
metrics := s.scrape()
|
||||||
wantMetric(t, metrics,
|
wantMetric(t, metrics,
|
||||||
`sww_requests_total{action="client_denied",instance="app",status_class="none"}`, 1)
|
`sww_requests_total{action="denied",instance="app",status_class="none"}`, 1)
|
||||||
wantMetric(t, metrics, `sww_rate_limit_hits_total{instance="app",`+
|
wantMetric(t, metrics, `sww_rate_limit_hits_total{instance="app",`+
|
||||||
`kind="requests",window="minute"}`, 1)
|
`kind="requests",window="minute"}`, 1)
|
||||||
wantMetric(t, metrics, `sww_offences_total{instance="app",kind="limit"}`, 1)
|
wantMetric(t, metrics, `sww_offences_total{instance="app",kind="limit"}`, 1)
|
||||||
@@ -147,7 +147,7 @@ func TestMetricsCountLimitsAndBans(t *testing.T) {
|
|||||||
// A limit broken again right after would ban for three hours, longer
|
// A limit broken again right after would ban for three hours, longer
|
||||||
// than SWW_MAX_BAN_DURATION, so the ban is permanent.
|
// than SWW_MAX_BAN_DURATION, so the ban is permanent.
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(client, 0, requestlog.ActionClientIsRateLimited)
|
s.get(client, 0, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
metrics = s.scrape()
|
metrics = s.scrape()
|
||||||
wantMetric(t, metrics, `sww_rate_limit_hits_total{instance="app",`+
|
wantMetric(t, metrics, `sww_rate_limit_hits_total{instance="app",`+
|
||||||
|
|||||||
@@ -7,7 +7,6 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
)
|
)
|
||||||
@@ -15,93 +14,6 @@ import (
|
|||||||
// observe is the value of SWW_MODE for observe mode.
|
// observe is the value of SWW_MODE for observe mode.
|
||||||
const observe = "observe"
|
const observe = "observe"
|
||||||
|
|
||||||
func TestBanActionSaysWhetherTheBanRunsOut(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
const trapPath = "/wp-login.php"
|
|
||||||
|
|
||||||
for _, tc := range []struct {
|
|
||||||
name string
|
|
||||||
cause string
|
|
||||||
// endsIn is how long after the request the ban ends: negative for a
|
|
||||||
// ban that has ended, and zero for a permanent ban.
|
|
||||||
endsIn time.Duration
|
|
||||||
path string
|
|
||||||
want string
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
"temporary ban", bans.CauseAdmin, time.Hour, "/",
|
|
||||||
requestlog.ActionTemporarilyBanned,
|
|
||||||
},
|
|
||||||
{"permanent ban", bans.CauseAdmin, 0, "/", requestlog.ActionPermanentlyBanned},
|
|
||||||
{
|
|
||||||
"request that makes a ban permanent", bans.CauseAttack, time.Hour, "/",
|
|
||||||
requestlog.ActionPermanentlyBanned,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"trap path after an attack ban ended", bans.CauseAttack, -time.Hour,
|
|
||||||
trapPath, requestlog.ActionPermanentlyBanned,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"first trap path after another ban ended", bans.CauseAdmin, -time.Hour,
|
|
||||||
trapPath, requestlog.ActionTemporarilyBanned,
|
|
||||||
},
|
|
||||||
} {
|
|
||||||
for _, modeValue := range []string{"enforce", observe} {
|
|
||||||
t.Run(tc.name+" in "+modeValue, func(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
s, clk, server := startWithClock(t, "", map[string]string{
|
|
||||||
trapPaths: trapPathList, mode: modeValue,
|
|
||||||
})
|
|
||||||
|
|
||||||
ban := bans.Ban{
|
|
||||||
Netblock: netip.MustParsePrefix(client + "/32"),
|
|
||||||
Start: clk.Now().Add(-2 * time.Hour),
|
|
||||||
Cause: tc.cause,
|
|
||||||
}
|
|
||||||
if tc.endsIn != 0 {
|
|
||||||
ban.Expires = clk.Now().Add(tc.endsIn)
|
|
||||||
}
|
|
||||||
|
|
||||||
server.Ledger.Load([]bans.Ban{ban})
|
|
||||||
|
|
||||||
if modeValue == observe {
|
|
||||||
line := s.request(client, tc.path, http.StatusOK, requestlog.ActionForward)
|
|
||||||
wantWouldAction(t, line, tc.want)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
s.request(client, tc.path, http.StatusForbidden, tc.want)
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestObserveModeNamesAPermanentBanWhoseAlertWouldBeSent(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
s, clk, server, queue := startWithAlerts(t, map[string]string{
|
|
||||||
trapPaths: trapPathList, mode: observe,
|
|
||||||
})
|
|
||||||
|
|
||||||
server.Ledger.Load([]bans.Ban{{
|
|
||||||
Netblock: netip.MustParsePrefix(client + "/32"),
|
|
||||||
Start: clk.Now().Add(-2 * time.Hour),
|
|
||||||
Expires: clk.Now().Add(-time.Hour),
|
|
||||||
Cause: bans.CauseAttack,
|
|
||||||
}})
|
|
||||||
|
|
||||||
line := s.request(client, "/wp-login.php", http.StatusOK, requestlog.ActionForward)
|
|
||||||
wantWouldAction(t, line, requestlog.ActionPermanentlyBanned)
|
|
||||||
|
|
||||||
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
|
||||||
if len(waiting) != 1 || waiting[0].Event != alerts.EventPermanentBan {
|
|
||||||
t.Errorf("alerts waiting are %+v, want one %s", waiting, alerts.EventPermanentBan)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestObserveModeNamesTheHeuristicThatWouldHaveRefused(t *testing.T) {
|
func TestObserveModeNamesTheHeuristicThatWouldHaveRefused(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -110,7 +22,7 @@ func TestObserveModeNamesTheHeuristicThatWouldHaveRefused(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
line := s.get(client, http.StatusOK, requestlog.ActionForward)
|
line := s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionClientDenied)
|
wantWouldAction(t, line, requestlog.ActionDenied)
|
||||||
wantHeuristic(t, line, "deny_nets", `{"netblock": "`+client+`/32"}`)
|
wantHeuristic(t, line, "deny_nets", `{"netblock": "`+client+`/32"}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -157,10 +69,10 @@ func TestObserveModeForwardsWhatEnforceModeRefuses(t *testing.T) {
|
|||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
for _, sent := range []struct{ from, refusal string }{
|
for _, sent := range []struct{ from, refusal string }{
|
||||||
{denied, requestlog.ActionClientDenied},
|
{denied, requestlog.ActionDenied},
|
||||||
{banned, requestlog.ActionTemporarilyBanned},
|
{banned, requestlog.ActionBanned},
|
||||||
{fromKP, requestlog.ActionClientDenied},
|
{fromKP, requestlog.ActionDenied},
|
||||||
{fromDE, requestlog.ActionClientIsRateLimited},
|
{fromDE, requestlog.ActionRateLimited},
|
||||||
} {
|
} {
|
||||||
if !tc.observe {
|
if !tc.observe {
|
||||||
line := s.get(sent.from, http.StatusForbidden, sent.refusal)
|
line := s.get(sent.from, http.StatusForbidden, sent.refusal)
|
||||||
@@ -198,7 +110,7 @@ func TestObserveModeMakesNoBanAndKeepsTheBansItHas(t *testing.T) {
|
|||||||
|
|
||||||
for range 2 {
|
for range 2 {
|
||||||
line := s.get(client, http.StatusOK, requestlog.ActionForward)
|
line := s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionClientIsRateLimited)
|
wantWouldAction(t, line, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
if line.LimitHit != minute || line.Offence != requestlog.OffenceLimit ||
|
if line.LimitHit != minute || line.Offence != requestlog.OffenceLimit ||
|
||||||
line.BanExpires != "" {
|
line.BanExpires != "" {
|
||||||
@@ -211,7 +123,7 @@ func TestObserveModeMakesNoBanAndKeepsTheBansItHas(t *testing.T) {
|
|||||||
// The ban read from bans.json refuses nothing, and so counts no
|
// The ban read from bans.json refuses nothing, and so counts no
|
||||||
// refusal in its notes, but is kept.
|
// refusal in its notes, but is kept.
|
||||||
line := s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
line := s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionTemporarilyBanned)
|
wantWouldAction(t, line, requestlog.ActionBanned)
|
||||||
|
|
||||||
if line.BanExpires != requestlog.FormatTime(kept.Expires) {
|
if line.BanExpires != requestlog.FormatTime(kept.Expires) {
|
||||||
t.Errorf("log line has ban_expires %q, want %s", line.BanExpires,
|
t.Errorf("log line has ban_expires %q, want %s", line.BanExpires,
|
||||||
|
|||||||
@@ -42,9 +42,7 @@ func (rateLimits) check(_ context.Context, rq *request, now time.Time) *verdict
|
|||||||
|
|
||||||
rq.banForLimit(now, hit, rq.h.config.BanResponse)
|
rq.banForLimit(now, hit, rq.h.config.BanResponse)
|
||||||
|
|
||||||
return &verdict{
|
return &verdict{action: requestlog.ActionRateLimited, figures: limitFigures(hit)}
|
||||||
action: requestlog.ActionClientIsRateLimited, figures: limitFigures(hit),
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// overAWholeRateLimit reports whether the heuristic rate_limits will refuse the
|
// overAWholeRateLimit reports whether the heuristic rate_limits will refuse the
|
||||||
|
|||||||
@@ -59,7 +59,7 @@ func TestRateLimitRefusesBeforeTheApp(t *testing.T) {
|
|||||||
if sent.want == http.StatusOK {
|
if sent.want == http.StatusOK {
|
||||||
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
|
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
|
||||||
} else {
|
} else {
|
||||||
wantLine(t, line, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
wantLine(t, line, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
if line.LimitHit != minute {
|
if line.LimitHit != minute {
|
||||||
t.Errorf("log line has limit_hit %q, want minute", line.LimitHit)
|
t.Errorf("log line has limit_hit %q, want minute", line.LimitHit)
|
||||||
@@ -79,7 +79,7 @@ func TestRateLimitRefusalNamesItsHeuristicWindowCountAndLimit(t *testing.T) {
|
|||||||
|
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
line := s.get(client, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
line := s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
wantHeuristic(t, line, "rate_limits", `{"window": "minute", "count": 2, "limit": 1}`)
|
wantHeuristic(t, line, "rate_limits", `{"window": "minute", "count": 2, "limit": 1}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -102,7 +102,7 @@ func TestIPv6GroupPrefixSetsTheClientTheLimitsCount(t *testing.T) {
|
|||||||
status int
|
status int
|
||||||
action string
|
action string
|
||||||
}{
|
}{
|
||||||
{rateLimitPerMinute, "1", http.StatusForbidden, requestlog.ActionClientIsRateLimited},
|
{rateLimitPerMinute, "1", http.StatusForbidden, requestlog.ActionRateLimited},
|
||||||
{bytesLimitPerMinute, byteLimit, http.StatusOK, requestlog.ActionForward},
|
{bytesLimitPerMinute, byteLimit, http.StatusOK, requestlog.ActionForward},
|
||||||
} {
|
} {
|
||||||
t.Run(tc.setting, func(t *testing.T) {
|
t.Run(tc.setting, func(t *testing.T) {
|
||||||
@@ -160,17 +160,14 @@ func TestRateLimitExemptPathsAreNeitherCountedNorRefused(t *testing.T) {
|
|||||||
|
|
||||||
// A path outside every prefix is counted: /assets is not under
|
// A path outside every prefix is counted: /assets is not under
|
||||||
// /assets/, and breaks the limit.
|
// /assets/, and breaks the limit.
|
||||||
s.request(client, "/assets", http.StatusForbidden,
|
s.request(client, "/assets", http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
requestlog.ActionClientIsRateLimited)
|
|
||||||
|
|
||||||
// A ban, SWW_DENY_NETS and SWW_TAG_DENY still refuse a path under a
|
// A ban, SWW_DENY_NETS and SWW_TAG_DENY still refuse a path under a
|
||||||
// prefix.
|
// prefix.
|
||||||
s.request(client, "/assets/app.js", http.StatusForbidden,
|
s.request(client, "/assets/app.js", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
requestlog.ActionTemporarilyBanned)
|
s.request(denied, "/assets/app.js", http.StatusForbidden, requestlog.ActionDenied)
|
||||||
s.request(denied, "/assets/app.js", http.StatusForbidden,
|
|
||||||
requestlog.ActionClientDenied)
|
|
||||||
s.request(fromKP, "/assets/app.js",
|
s.request(fromKP, "/assets/app.js",
|
||||||
http.StatusForbidden, requestlog.ActionClientDenied)
|
http.StatusForbidden, requestlog.ActionDenied)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRateLimitCountsPathsThatAreNotExempt(t *testing.T) {
|
func TestRateLimitCountsPathsThatAreNotExempt(t *testing.T) {
|
||||||
@@ -214,7 +211,7 @@ func TestRateLimitCountsPathsThatAreNotExempt(t *testing.T) {
|
|||||||
// Counted, the second request breaks the limit of one request
|
// Counted, the second request breaks the limit of one request
|
||||||
// a minute.
|
// a minute.
|
||||||
s.request(client, sent, http.StatusOK, requestlog.ActionForward)
|
s.request(client, sent, http.StatusOK, requestlog.ActionForward)
|
||||||
s.request(client, sent, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.request(client, sent, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -28,7 +28,7 @@ func (refusalLimit) check(_ context.Context, rq *request, now time.Time) *verdic
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
if !rq.attack && !rq.blockedByBlockRule && !rq.wafBlocked {
|
if !rq.attack && !rq.ruleBlocked && !rq.wafBlocked {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -42,13 +42,13 @@ func (refusalLimit) check(_ context.Context, rq *request, now time.Time) *verdic
|
|||||||
status := rq.out.status
|
status := rq.out.status
|
||||||
|
|
||||||
switch rq.line.WouldAction {
|
switch rq.line.WouldAction {
|
||||||
case requestlog.ActionBlockedByBlockRule, requestlog.ActionWAFBlocked:
|
case requestlog.ActionRuleBlocked, requestlog.ActionWAFBlocked:
|
||||||
status = http.StatusForbidden
|
status = http.StatusForbidden
|
||||||
case requestlog.ActionTemporarilyBanned, requestlog.ActionPermanentlyBanned:
|
case requestlog.ActionBanned:
|
||||||
status = cfg.BanResponse
|
status = cfg.BanResponse
|
||||||
}
|
}
|
||||||
|
|
||||||
rq.banForLimit(now, hit, status)
|
rq.banForLimit(now, hit, status)
|
||||||
|
|
||||||
return &verdict{figures: limitFigures(hit)}
|
return &verdict{action: requestlog.ActionBanned, figures: limitFigures(hit)}
|
||||||
}
|
}
|
||||||
@@ -37,10 +37,9 @@ func (r refused) send(s *sender, from string) logLine {
|
|||||||
switch r {
|
switch r {
|
||||||
case blockRule:
|
case blockRule:
|
||||||
return s.request(from, blockedPath, http.StatusForbidden,
|
return s.request(from, blockedPath, http.StatusForbidden,
|
||||||
requestlog.ActionBlockedByBlockRule)
|
requestlog.ActionRuleBlocked)
|
||||||
case banRule:
|
case banRule:
|
||||||
return s.request(from, probePath, http.StatusForbidden,
|
return s.request(from, probePath, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
requestlog.ActionTemporarilyBanned)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
s.t.Fatalf("no request for the refusal %d", r)
|
s.t.Fatalf("no request for the refusal %d", r)
|
||||||
@@ -72,20 +71,9 @@ func TestErrorBurstBreaksAtOneOverTheThreshold(t *testing.T) {
|
|||||||
name string
|
name string
|
||||||
// refusals are four, one over the threshold of three.
|
// refusals are four, one over the threshold of three.
|
||||||
refusals []refused
|
refusals []refused
|
||||||
// nextAction is the action of the client's next request: the next
|
|
||||||
// request after a ban for a ban rule makes it permanent.
|
|
||||||
nextAction string
|
|
||||||
}{
|
}{
|
||||||
{
|
{"block rule", []refused{blockRule, blockRule, blockRule, blockRule}},
|
||||||
"block rule",
|
{"a ban rule last", []refused{blockRule, blockRule, blockRule, banRule}},
|
||||||
[]refused{blockRule, blockRule, blockRule, blockRule},
|
|
||||||
requestlog.ActionTemporarilyBanned,
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"a ban rule last",
|
|
||||||
[]refused{blockRule, blockRule, blockRule, banRule},
|
|
||||||
requestlog.ActionPermanentlyBanned,
|
|
||||||
},
|
|
||||||
} {
|
} {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
@@ -113,7 +101,7 @@ func TestErrorBurstBreaksAtOneOverTheThreshold(t *testing.T) {
|
|||||||
"and limit", line.LimitHit, line.Offence)
|
"and limit", line.LimitHit, line.Offence)
|
||||||
}
|
}
|
||||||
|
|
||||||
s.get(client, http.StatusForbidden, tc.nextAction)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -181,7 +169,7 @@ func TestErrorBurstBanNotesHistoryAndMetrics(t *testing.T) {
|
|||||||
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
|
t.Fatalf("bans\n%+v\nwant\n%+v", got, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantOffences := ratelimit.Offences{Limit: 1, BlockedByBlockRule: 3}
|
wantOffences := ratelimit.Offences{Limit: 1, RuleBlocked: 3}
|
||||||
if offences := historyOf(t, server, client).Offences; offences != wantOffences {
|
if offences := historyOf(t, server, client).Offences; offences != wantOffences {
|
||||||
t.Errorf("history counts the offences %+v, want %+v", offences, wantOffences)
|
t.Errorf("history counts the offences %+v, want %+v", offences, wantOffences)
|
||||||
}
|
}
|
||||||
@@ -191,7 +179,7 @@ func TestErrorBurstBanNotesHistoryAndMetrics(t *testing.T) {
|
|||||||
`kind="refusals",window="minute"}`, 1)
|
`kind="refusals",window="minute"}`, 1)
|
||||||
wantMetric(t, metrics, `sww_offences_total{instance="app",kind="limit"}`, 1)
|
wantMetric(t, metrics, `sww_offences_total{instance="app",kind="limit"}`, 1)
|
||||||
wantMetric(t, metrics,
|
wantMetric(t, metrics,
|
||||||
`sww_offences_total{instance="app",kind="blocked_by_block_rule"}`, 3)
|
`sww_offences_total{instance="app",kind="rule_blocked"}`, 3)
|
||||||
wantMetric(t, metrics, `sww_bans_made_total{cause="limit",instance="app"}`, 1)
|
wantMetric(t, metrics, `sww_bans_made_total{cause="limit",instance="app"}`, 1)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -310,7 +298,7 @@ func TestErrorBurstCountsEachClientTheChecksApplyTo(t *testing.T) {
|
|||||||
t.Errorf("log line has limit_hit %q, want error_burst", line.LimitHit)
|
t.Errorf("log line has limit_hit %q, want error_burst", line.LimitHit)
|
||||||
}
|
}
|
||||||
|
|
||||||
s.get(exempt, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(exempt, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestErrorBurstBanSetsTheRefusalsBackToZero(t *testing.T) {
|
func TestErrorBurstBanSetsTheRefusalsBackToZero(t *testing.T) {
|
||||||
@@ -353,7 +341,7 @@ func TestObserveModeLogsAndAlertsTheErrorBurst(t *testing.T) {
|
|||||||
for range 2 {
|
for range 2 {
|
||||||
line := s.request(otherClient, blockedPath, http.StatusOK,
|
line := s.request(otherClient, blockedPath, http.StatusOK,
|
||||||
requestlog.ActionForward)
|
requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionTemporarilyBanned)
|
wantWouldAction(t, line, requestlog.ActionBanned)
|
||||||
|
|
||||||
if line.LimitHit != "" {
|
if line.LimitHit != "" {
|
||||||
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
|
t.Errorf("log line has limit_hit %q, want none", line.LimitHit)
|
||||||
@@ -365,7 +353,7 @@ func TestObserveModeLogsAndAlertsTheErrorBurst(t *testing.T) {
|
|||||||
s.request(client, blockedPath, http.StatusOK, requestlog.ActionForward)
|
s.request(client, blockedPath, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
line := s.request(client, blockedPath, http.StatusOK, requestlog.ActionForward)
|
line := s.request(client, blockedPath, http.StatusOK, requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionBlockedByBlockRule)
|
wantWouldAction(t, line, requestlog.ActionRuleBlocked)
|
||||||
|
|
||||||
if line.LimitHit != requestlog.LimitHitErrorBurst || line.BanExpires != "" {
|
if line.LimitHit != requestlog.LimitHitErrorBurst || line.BanExpires != "" {
|
||||||
t.Errorf("log line has limit_hit %q and ban_expires %q, want error_burst "+
|
t.Errorf("log line has limit_hit %q and ban_expires %q, want error_burst "+
|
||||||
|
|||||||
@@ -49,7 +49,7 @@ const (
|
|||||||
func TestEachBlocklistActionForAListedAddressAndAListedNetblock(t *testing.T) {
|
func TestEachBlocklistActionForAListedAddressAndAListedNetblock(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
forward, denied := requestlog.ActionForward, requestlog.ActionClientDenied
|
forward, denied := requestlog.ActionForward, requestlog.ActionDenied
|
||||||
|
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
action string
|
action string
|
||||||
@@ -66,7 +66,7 @@ func TestEachBlocklistActionForAListedAddressAndAListedNetblock(t *testing.T) {
|
|||||||
{
|
{
|
||||||
// Half of 4 requests a minute: the third breaks the limit.
|
// Half of 4 requests a minute: the third breaks the limit.
|
||||||
limitHalf, []int{http.StatusOK, http.StatusOK, http.StatusForbidden},
|
limitHalf, []int{http.StatusOK, http.StatusOK, http.StatusForbidden},
|
||||||
[]string{forward, forward, requestlog.ActionClientIsRateLimited},
|
[]string{forward, forward, requestlog.ActionRateLimited},
|
||||||
"50 from " + blocklistAction,
|
"50 from " + blocklistAction,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -162,7 +162,7 @@ func TestReputationRefusalNamesItsHeuristicAndWhatListedTheClient(t *testing.T)
|
|||||||
s, server, _ := startWithLookups(t, tc.env)
|
s, server, _ := startWithLookups(t, tc.env)
|
||||||
tc.load(t, server)
|
tc.load(t, server)
|
||||||
|
|
||||||
line := s.get(client, http.StatusForbidden, requestlog.ActionClientDenied)
|
line := s.get(client, http.StatusForbidden, requestlog.ActionDenied)
|
||||||
wantHeuristic(t, line, tc.heuristic, tc.figures)
|
wantHeuristic(t, line, tc.heuristic, tc.figures)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -193,7 +193,7 @@ func TestObserveModeForwardsAClientABlocklistDeniesAndAlertsIt(t *testing.T) {
|
|||||||
loadLists(t, server, map[string][]string{dropURL: {fromDE}})
|
loadLists(t, server, map[string][]string{dropURL: {fromDE}})
|
||||||
|
|
||||||
line := s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
line := s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionClientDenied)
|
wantWouldAction(t, line, requestlog.ActionDenied)
|
||||||
wantReputation(t, line, dropURL)
|
wantReputation(t, line, dropURL)
|
||||||
|
|
||||||
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||||
@@ -380,7 +380,7 @@ const (
|
|||||||
func TestEachReputationActionForAClientADNSBLZoneLists(t *testing.T) {
|
func TestEachReputationActionForAClientADNSBLZoneLists(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
forward, denied := requestlog.ActionForward, requestlog.ActionClientDenied
|
forward, denied := requestlog.ActionForward, requestlog.ActionDenied
|
||||||
|
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
action string
|
action string
|
||||||
@@ -397,7 +397,7 @@ func TestEachReputationActionForAClientADNSBLZoneLists(t *testing.T) {
|
|||||||
{
|
{
|
||||||
// Half of 4 requests a minute: the third breaks the limit.
|
// Half of 4 requests a minute: the third breaks the limit.
|
||||||
limitHalf, []int{http.StatusOK, http.StatusOK, http.StatusForbidden},
|
limitHalf, []int{http.StatusOK, http.StatusOK, http.StatusForbidden},
|
||||||
[]string{forward, forward, requestlog.ActionClientIsRateLimited},
|
[]string{forward, forward, requestlog.ActionRateLimited},
|
||||||
"50 from " + reputationAction,
|
"50 from " + reputationAction,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -468,7 +468,7 @@ func TestDNSBLZonesComeAfterTheBlocklistsAndSkipAllowNets(t *testing.T) {
|
|||||||
// The blocklist refuses fromKP before its verdict is looked at, and
|
// The blocklist refuses fromKP before its verdict is looked at, and
|
||||||
// fromDE, in SWW_ALLOW_NETS, is not checked at all: neither is noted
|
// fromDE, in SWW_ALLOW_NETS, is not checked at all: neither is noted
|
||||||
// for the zone, nor alerted, nor asked about.
|
// for the zone, nor alerted, nor asked about.
|
||||||
wantReputation(t, s.get(fromKP, http.StatusForbidden, requestlog.ActionClientDenied),
|
wantReputation(t, s.get(fromKP, http.StatusForbidden, requestlog.ActionDenied),
|
||||||
dropURL)
|
dropURL)
|
||||||
wantReputation(t, s.get(fromDE, http.StatusOK, requestlog.ActionForward))
|
wantReputation(t, s.get(fromDE, http.StatusOK, requestlog.ActionForward))
|
||||||
|
|
||||||
@@ -532,7 +532,7 @@ func TestObserveModeForwardsAClientADNSBLZoneDeniesAndAlertsIt(t *testing.T) {
|
|||||||
loadVerdicts(server, map[string][]string{fromDE: {dnsblZone}})
|
loadVerdicts(server, map[string][]string{fromDE: {dnsblZone}})
|
||||||
|
|
||||||
line := s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
line := s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionClientDenied)
|
wantWouldAction(t, line, requestlog.ActionDenied)
|
||||||
wantReputation(t, line, dnsblZone)
|
wantReputation(t, line, dnsblZone)
|
||||||
|
|
||||||
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||||
@@ -741,8 +741,8 @@ func TestOnlyAClientThatHasCommittedAnOffenceIsCheckedWithAbuseIPDB(t *testing.T
|
|||||||
s.get(fromDE, http.StatusOK, forward)
|
s.get(fromDE, http.StatusOK, forward)
|
||||||
s.get(fromDE, http.StatusOK, forward)
|
s.get(fromDE, http.StatusOK, forward)
|
||||||
s.get(fromKP, http.StatusOK, forward)
|
s.get(fromKP, http.StatusOK, forward)
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
wantAbuseIPDBChecks(t, server, 0)
|
wantAbuseIPDBChecks(t, server, 0)
|
||||||
|
|
||||||
// Once the ban has ended, fromDE's first request has it checked in the
|
// Once the ban has ended, fromDE's first request has it checked in the
|
||||||
@@ -781,7 +781,7 @@ func TestIPv6ClientCostsOneAbuseIPDBCheckWhicheverOfItsAddressesSends(t *testing
|
|||||||
s.get(addresses[0], http.StatusOK, forward)
|
s.get(addresses[0], http.StatusOK, forward)
|
||||||
}
|
}
|
||||||
|
|
||||||
s.get(addresses[0], http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(addresses[0], http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
clk.advance(time.Hour)
|
clk.advance(time.Hour)
|
||||||
|
|
||||||
// Once the ban has ended, which set its counters back to zero, a
|
// Once the ban has ended, which set its counters back to zero, a
|
||||||
@@ -814,20 +814,20 @@ func TestClientRefusedForAnOffenceIsCheckedWithAbuseIPDBAtItsNextRequest(t *test
|
|||||||
want ratelimit.Offences
|
want ratelimit.Offences
|
||||||
}{
|
}{
|
||||||
{
|
{
|
||||||
"a block rule", blockedPath, http.StatusForbidden,
|
"a block rule", blockedPath, http.StatusForbidden, requestlog.ActionRuleBlocked,
|
||||||
requestlog.ActionBlockedByBlockRule, ratelimit.Offences{BlockedByBlockRule: 1},
|
ratelimit.Offences{RuleBlocked: 1},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"the Core Rule Set", sqlInjection, http.StatusForbidden,
|
"the Core Rule Set", sqlInjection, http.StatusForbidden,
|
||||||
requestlog.ActionWAFBlocked, ratelimit.Offences{WAFBlocked: 1},
|
requestlog.ActionWAFBlocked, ratelimit.Offences{WAFBlocked: 1},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"a ban rule", probePath, http.StatusForbidden, requestlog.ActionTemporarilyBanned,
|
"a ban rule", probePath, http.StatusForbidden, requestlog.ActionBanned,
|
||||||
ratelimit.Offences{Attack: 1},
|
ratelimit.Offences{Attack: 1},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"a trap path", "/xmlrpc.php", http.StatusForbidden,
|
"a trap path", "/xmlrpc.php", http.StatusForbidden, requestlog.ActionBanned,
|
||||||
requestlog.ActionTemporarilyBanned, ratelimit.Offences{Attack: 1},
|
ratelimit.Offences{Attack: 1},
|
||||||
},
|
},
|
||||||
} {
|
} {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
@@ -858,7 +858,7 @@ func TestClientRefusedForAnOffenceIsCheckedWithAbuseIPDBAtItsNextRequest(t *test
|
|||||||
func TestEachReputationActionForAClientAbuseIPDBScoresAtOrOverTheMinimum(t *testing.T) {
|
func TestEachReputationActionForAClientAbuseIPDBScoresAtOrOverTheMinimum(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
forward, denied := requestlog.ActionForward, requestlog.ActionClientDenied
|
forward, denied := requestlog.ActionForward, requestlog.ActionDenied
|
||||||
|
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
action string
|
action string
|
||||||
@@ -875,7 +875,7 @@ func TestEachReputationActionForAClientAbuseIPDBScoresAtOrOverTheMinimum(t *test
|
|||||||
{
|
{
|
||||||
// Half of 4 requests a minute: the third breaks the limit.
|
// Half of 4 requests a minute: the third breaks the limit.
|
||||||
limitHalf, []int{http.StatusOK, http.StatusOK, http.StatusForbidden},
|
limitHalf, []int{http.StatusOK, http.StatusOK, http.StatusForbidden},
|
||||||
[]string{forward, forward, requestlog.ActionClientIsRateLimited},
|
[]string{forward, forward, requestlog.ActionRateLimited},
|
||||||
"50 from " + reputationAction,
|
"50 from " + reputationAction,
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -972,7 +972,7 @@ func TestWithoutAnAbuseIPDBKeyNoClientIsCheckedNorAScoreUsed(t *testing.T) {
|
|||||||
// fromDE's score is not used, and once it has committed an offence it
|
// fromDE's score is not used, and once it has committed an offence it
|
||||||
// is not checked either.
|
// is not checked either.
|
||||||
wantReputation(t, s.get(fromDE, http.StatusOK, forward))
|
wantReputation(t, s.get(fromDE, http.StatusOK, forward))
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
clk.advance(time.Hour)
|
clk.advance(time.Hour)
|
||||||
wantReputation(t, s.get(fromDE, http.StatusOK, forward))
|
wantReputation(t, s.get(fromDE, http.StatusOK, forward))
|
||||||
wantAbuseIPDBChecks(t, server, 0)
|
wantAbuseIPDBChecks(t, server, 0)
|
||||||
@@ -996,10 +996,10 @@ func TestBanNotesNameEachReputationSourceThatListedTheClient(t *testing.T) {
|
|||||||
}{
|
}{
|
||||||
{"for a broken rate limit", func(s *sender, from string) {
|
{"for a broken rate limit", func(s *sender, from string) {
|
||||||
s.get(from, http.StatusOK, requestlog.ActionForward)
|
s.get(from, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(from, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(from, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
}},
|
}},
|
||||||
{"for a clear sign of attack", func(s *sender, from string) {
|
{"for a clear sign of attack", func(s *sender, from string) {
|
||||||
s.request(from, probePath, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.request(from, probePath, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
}},
|
}},
|
||||||
} {
|
} {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
|
|||||||
@@ -54,10 +54,10 @@ type decision struct {
|
|||||||
counted bool
|
counted bool
|
||||||
limitPercent, bytesPercent percentage
|
limitPercent, bytesPercent percentage
|
||||||
// attack is true for a request that matched a ban rule or asked for a
|
// attack is true for a request that matched a ban rule or asked for a
|
||||||
// trap path, blockedByBlockRule for one a block rule refused, and
|
// trap path, ruleBlocked for one a block rule refused, and wafBlocked for
|
||||||
// wafBlocked for one the Core Rule Set refused, each an offence its
|
// one the Core Rule Set refused, each an offence its client's history
|
||||||
// client's history counts.
|
// counts.
|
||||||
attack, blockedByBlockRule, wafBlocked bool
|
attack, ruleBlocked, wafBlocked bool
|
||||||
// reputation is the reputation sources that list the client, for the
|
// reputation is the reputation sources that list the client, for the
|
||||||
// notes of a ban the request makes.
|
// notes of a ban the request makes.
|
||||||
reputation []bans.ReputationHit
|
reputation []bans.ReputationHit
|
||||||
@@ -187,10 +187,9 @@ func (rq *request) check(ctx context.Context) *refusal {
|
|||||||
rq.line.WouldAction = action
|
rq.line.WouldAction = action
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
case action == requestlog.ActionBlockedByBlockRule ||
|
case action == requestlog.ActionRuleBlocked || action == requestlog.ActionWAFBlocked:
|
||||||
action == requestlog.ActionWAFBlocked:
|
|
||||||
return &refusal{status: http.StatusForbidden, action: action}
|
return &refusal{status: http.StatusForbidden, action: action}
|
||||||
case action == requestlog.ActionRequestHeaderTooLarge:
|
case action == requestlog.ActionTooLarge:
|
||||||
return &refusal{status: http.StatusRequestHeaderFieldsTooLarge, action: action}
|
return &refusal{status: http.StatusRequestHeaderFieldsTooLarge, action: action}
|
||||||
default:
|
default:
|
||||||
return rq.banResponse(action)
|
return rq.banResponse(action)
|
||||||
@@ -311,7 +310,7 @@ func (rq *request) addToHistory() {
|
|||||||
ResponseBytes: rq.out.bytes,
|
ResponseBytes: rq.out.bytes,
|
||||||
BrokeLimit: rq.line.Offence == requestlog.OffenceLimit,
|
BrokeLimit: rq.line.Offence == requestlog.OffenceLimit,
|
||||||
Attack: rq.attack,
|
Attack: rq.attack,
|
||||||
BlockedByBlockRule: rq.blockedByBlockRule,
|
RuleBlocked: rq.ruleBlocked,
|
||||||
WAFBlocked: rq.wafBlocked,
|
WAFBlocked: rq.wafBlocked,
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -112,25 +112,11 @@ func TestRouteWatchTagsAreOnALineDenyNetsRefused(t *testing.T) {
|
|||||||
|
|
||||||
waitForRouteWatch(t, server, fromKP, 32)
|
waitForRouteWatch(t, server, fromKP, 32)
|
||||||
|
|
||||||
line := s.get(fromKP, http.StatusForbidden, requestlog.ActionClientDenied)
|
line := s.get(fromKP, http.StatusForbidden, requestlog.ActionDenied)
|
||||||
wantHeuristic(t, line, "deny_nets", `{"netblock": "`+fromKP+`/32"}`)
|
wantHeuristic(t, line, "deny_nets", `{"netblock": "`+fromKP+`/32"}`)
|
||||||
wantTags(t, line, "asn=64511", "prefix="+prefixKP)
|
wantTags(t, line, "asn=64511", "prefix="+prefixKP)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRouteWithoutAnOriginASGivesOnlyThePrefixTag(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
url := startRouteWatch(t, map[string]standInRoute{fromDE: {prefixDE, 0}})
|
|
||||||
s, _, server := startWithClock(t, "", map[string]string{
|
|
||||||
lookupSource: off, routeWatchURL: url,
|
|
||||||
})
|
|
||||||
|
|
||||||
waitForRouteWatch(t, server, fromDE, 32)
|
|
||||||
|
|
||||||
wantTags(t, s.get(fromDE, http.StatusOK, requestlog.ActionForward),
|
|
||||||
"prefix="+prefixDE)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestUnsetRouteWatchChangesNothing(t *testing.T) {
|
func TestUnsetRouteWatchChangesNothing(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -222,7 +208,7 @@ func TestTagDenyRefusesOnARouteWatchPrefix(t *testing.T) {
|
|||||||
waitForRouteWatch(t, server, fromKP, 32)
|
waitForRouteWatch(t, server, fromKP, 32)
|
||||||
waitForRouteWatch(t, server, fromDE, 32)
|
waitForRouteWatch(t, server, fromDE, 32)
|
||||||
|
|
||||||
wantHeuristic(t, s.get(fromKP, http.StatusForbidden, requestlog.ActionClientDenied),
|
wantHeuristic(t, s.get(fromKP, http.StatusForbidden, requestlog.ActionDenied),
|
||||||
"tag_deny", `{"tag": "prefix=`+prefixKP+`"}`)
|
"tag_deny", `{"tag": "prefix=`+prefixKP+`"}`)
|
||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
}
|
}
|
||||||
@@ -241,5 +227,5 @@ func TestTagLimitPercentWorksWithAnIPv6Prefix(t *testing.T) {
|
|||||||
// Half of two is one: the second request breaks the limit.
|
// Half of two is one: the second request breaks the limit.
|
||||||
line := s.get(fromV6, http.StatusOK, requestlog.ActionForward)
|
line := s.get(fromV6, http.StatusOK, requestlog.ActionForward)
|
||||||
wantTags(t, line, "asn=64496", "prefix="+prefixV6)
|
wantTags(t, line, "asn=64496", "prefix="+prefixV6)
|
||||||
s.get(fromV6, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(fromV6, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
}
|
}
|
||||||
@@ -16,10 +16,9 @@ import (
|
|||||||
// ruleFiles checks the request against the rules of the rule files at now,
|
// ruleFiles checks the request against the rules of the rule files at now,
|
||||||
// notes the ids of those it matches in the log line, adds the tag
|
// notes the ids of those it matches in the log line, adds the tag
|
||||||
// exploitsignature when it matches any, and refuses the request
|
// exploitsignature when it matches any, and refuses the request
|
||||||
// when the last rule it matches refuses it: with ActionBlockedByBlockRule for a
|
// when the last rule it matches refuses it: with ActionRuleBlocked for a block
|
||||||
// block rule, and for a ban rule with the action for the ban it makes. A ban
|
// rule, and ActionBanned for a ban rule. A ban rule bans the client's netblock
|
||||||
// rule bans the client's netblock for a clear sign of attack, or in observe
|
// for a clear sign of attack, or in observe mode raises the alert for the ban
|
||||||
// mode raises the alert for the ban
|
|
||||||
// it would have made. Either rule's match is noted as an offence, for the
|
// it would have made. Either rule's match is noted as an offence, for the
|
||||||
// client's history. Its figures are the rule's id and action.
|
// client's history. Its figures are the rule's id and action.
|
||||||
type ruleFiles struct{}
|
type ruleFiles struct{}
|
||||||
@@ -46,16 +45,14 @@ func (ruleFiles) check(_ context.Context, rq *request, now time.Time) *verdict {
|
|||||||
|
|
||||||
switch last.Action {
|
switch last.Action {
|
||||||
case rules.ActionBlock:
|
case rules.ActionBlock:
|
||||||
rq.blockedByBlockRule = true
|
rq.ruleBlocked = true
|
||||||
|
|
||||||
return &verdict{action: requestlog.ActionBlockedByBlockRule, figures: figures}
|
return &verdict{action: requestlog.ActionRuleBlocked, figures: figures}
|
||||||
case rules.ActionBan:
|
case rules.ActionBan:
|
||||||
rq.attack = true
|
rq.attack = true
|
||||||
|
rq.banForAttack(now, bans.Notes{RuleID: last.ID, Target: last.Target})
|
||||||
|
|
||||||
return &verdict{
|
return &verdict{action: requestlog.ActionBanned, figures: figures}
|
||||||
action: rq.banForAttack(now, bans.Notes{RuleID: last.ID, Target: last.Target}),
|
|
||||||
figures: figures,
|
|
||||||
}
|
|
||||||
default:
|
default:
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -37,14 +37,13 @@ func TestEachRuleAction(t *testing.T) {
|
|||||||
// A block rule refuses with 403, whatever SWW_BAN_RESPONSE is, and
|
// A block rule refuses with 403, whatever SWW_BAN_RESPONSE is, and
|
||||||
// bans no one.
|
// bans no one.
|
||||||
line = s.request(client, "/blocked", http.StatusForbidden,
|
line = s.request(client, "/blocked", http.StatusForbidden,
|
||||||
requestlog.ActionBlockedByBlockRule)
|
requestlog.ActionRuleBlocked)
|
||||||
wantRuleIDs(t, line, "noted", "blocked")
|
wantRuleIDs(t, line, "noted", "blocked")
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
// A ban rule refuses with SWW_BAN_RESPONSE, and bans the client for
|
// A ban rule refuses with SWW_BAN_RESPONSE, and bans the client for
|
||||||
// seven days, the default.
|
// seven days, the default.
|
||||||
line = s.request(client, "/.env", http.StatusTooManyRequests,
|
line = s.request(client, "/.env", http.StatusTooManyRequests, requestlog.ActionBanned)
|
||||||
requestlog.ActionTemporarilyBanned)
|
|
||||||
wantRuleIDs(t, line, "noted", "probe")
|
wantRuleIDs(t, line, "noted", "probe")
|
||||||
|
|
||||||
if line.BanExpires != requestlog.FormatTime(start.Add(7*24*time.Hour)) {
|
if line.BanExpires != requestlog.FormatTime(start.Add(7*24*time.Hour)) {
|
||||||
@@ -83,7 +82,7 @@ func TestEachRuleAction(t *testing.T) {
|
|||||||
// against the rules, and makes the ban permanent.
|
// against the rules, and makes the ban permanent.
|
||||||
clk.advance(time.Hour)
|
clk.advance(time.Hour)
|
||||||
|
|
||||||
line = s.get(client, http.StatusTooManyRequests, requestlog.ActionPermanentlyBanned)
|
line = s.get(client, http.StatusTooManyRequests, requestlog.ActionBanned)
|
||||||
wantRuleIDs(t, line)
|
wantRuleIDs(t, line)
|
||||||
|
|
||||||
if line.BanExpires != permanent {
|
if line.BanExpires != permanent {
|
||||||
@@ -91,7 +90,7 @@ func TestEachRuleAction(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
clk.advance(365 * 24 * time.Hour)
|
clk.advance(365 * 24 * time.Hour)
|
||||||
s.get(client, http.StatusTooManyRequests, requestlog.ActionPermanentlyBanned)
|
s.get(client, http.StatusTooManyRequests, requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestEachRuleActionAddsTheExploitSignatureTag(t *testing.T) {
|
func TestEachRuleActionAddsTheExploitSignatureTag(t *testing.T) {
|
||||||
@@ -104,7 +103,7 @@ func TestEachRuleActionAddsTheExploitSignatureTag(t *testing.T) {
|
|||||||
wantTags(t, s.request(client, "/noted", http.StatusOK, requestlog.ActionForward),
|
wantTags(t, s.request(client, "/noted", http.StatusOK, requestlog.ActionForward),
|
||||||
"exploitsignature")
|
"exploitsignature")
|
||||||
wantTags(t, s.request(client, "/blocked", http.StatusForbidden,
|
wantTags(t, s.request(client, "/blocked", http.StatusForbidden,
|
||||||
requestlog.ActionBlockedByBlockRule), "exploitsignature")
|
requestlog.ActionRuleBlocked), "exploitsignature")
|
||||||
wantTags(t, s.get(client, http.StatusOK, requestlog.ActionForward))
|
wantTags(t, s.get(client, http.StatusOK, requestlog.ActionForward))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -116,11 +115,10 @@ func TestRuleRefusalNamesItsHeuristicAndTheRule(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
line := s.request(client, "/blocked", http.StatusForbidden,
|
line := s.request(client, "/blocked", http.StatusForbidden,
|
||||||
requestlog.ActionBlockedByBlockRule)
|
requestlog.ActionRuleBlocked)
|
||||||
wantHeuristic(t, line, "rule_files", `{"rule_id": "blocked", "rule_action": "block"}`)
|
wantHeuristic(t, line, "rule_files", `{"rule_id": "blocked", "rule_action": "block"}`)
|
||||||
|
|
||||||
line = s.request(client, "/.env", http.StatusForbidden,
|
line = s.request(client, "/.env", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
requestlog.ActionTemporarilyBanned)
|
|
||||||
wantHeuristic(t, line, "rule_files", `{"rule_id": "probe", "rule_action": "ban"}`)
|
wantHeuristic(t, line, "rule_files", `{"rule_id": "probe", "rule_action": "ban"}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -133,8 +131,7 @@ func TestNextClearSignOfAttackAfterABanBansPermanently(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
// The first probe bans for SWW_ATTACK_BAN_DURATION.
|
// The first probe bans for SWW_ATTACK_BAN_DURATION.
|
||||||
line := s.request(client, "/.env", http.StatusForbidden,
|
line := s.request(client, "/.env", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
requestlog.ActionTemporarilyBanned)
|
|
||||||
if line.BanExpires != requestlog.FormatTime(clk.Now().Add(time.Hour)) {
|
if line.BanExpires != requestlog.FormatTime(clk.Now().Add(time.Hour)) {
|
||||||
t.Errorf("log line has ban_expires %q, want an hour on", line.BanExpires)
|
t.Errorf("log line has ban_expires %q, want an hour on", line.BanExpires)
|
||||||
}
|
}
|
||||||
@@ -144,8 +141,7 @@ func TestNextClearSignOfAttackAfterABanBansPermanently(t *testing.T) {
|
|||||||
clk.advance(time.Hour)
|
clk.advance(time.Hour)
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
line = s.request(client, "/.env", http.StatusForbidden,
|
line = s.request(client, "/.env", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
requestlog.ActionPermanentlyBanned)
|
|
||||||
if line.BanExpires != permanent {
|
if line.BanExpires != permanent {
|
||||||
t.Errorf("log line has ban_expires %q, want permanent", line.BanExpires)
|
t.Errorf("log line has ban_expires %q, want permanent", line.BanExpires)
|
||||||
}
|
}
|
||||||
@@ -173,8 +169,7 @@ func TestRulesComeAfterTheOtherChecks(t *testing.T) {
|
|||||||
// A probe over the rate limit breaks the limit before any rule sees
|
// A probe over the rate limit breaks the limit before any rule sees
|
||||||
// it.
|
// it.
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
line = s.request(client, "/.env", http.StatusForbidden,
|
line = s.request(client, "/.env", http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
requestlog.ActionClientIsRateLimited)
|
|
||||||
wantRuleIDs(t, line)
|
wantRuleIDs(t, line)
|
||||||
|
|
||||||
limitBan := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
|
limitBan := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
|
||||||
@@ -184,7 +179,7 @@ func TestRulesComeAfterTheOtherChecks(t *testing.T) {
|
|||||||
|
|
||||||
// A client the rate limits do not apply to is still checked.
|
// A client the rate limits do not apply to is still checked.
|
||||||
s.get(exempt, http.StatusOK, requestlog.ActionForward)
|
s.get(exempt, http.StatusOK, requestlog.ActionForward)
|
||||||
s.request(exempt, "/.env", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.request(exempt, "/.env", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestObserveModeLogsWhatTheRulesWouldDo(t *testing.T) {
|
func TestObserveModeLogsWhatTheRulesWouldDo(t *testing.T) {
|
||||||
@@ -196,11 +191,11 @@ func TestObserveModeLogsWhatTheRulesWouldDo(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
line := s.request(client, "/blocked", http.StatusOK, requestlog.ActionForward)
|
line := s.request(client, "/blocked", http.StatusOK, requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionBlockedByBlockRule)
|
wantWouldAction(t, line, requestlog.ActionRuleBlocked)
|
||||||
wantRuleIDs(t, line, "blocked")
|
wantRuleIDs(t, line, "blocked")
|
||||||
|
|
||||||
line = s.request(client, "/.env", http.StatusOK, requestlog.ActionForward)
|
line = s.request(client, "/.env", http.StatusOK, requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionTemporarilyBanned)
|
wantWouldAction(t, line, requestlog.ActionBanned)
|
||||||
wantRuleIDs(t, line, "probe")
|
wantRuleIDs(t, line, "probe")
|
||||||
|
|
||||||
if line.BanExpires != "" {
|
if line.BanExpires != "" {
|
||||||
@@ -224,10 +219,9 @@ func TestMetricsCountRuleMatchesAndBansForAnAttack(t *testing.T) {
|
|||||||
metricsToken: token,
|
metricsToken: token,
|
||||||
})
|
})
|
||||||
|
|
||||||
s.request(client, "/blocked", http.StatusForbidden,
|
s.request(client, "/blocked", http.StatusForbidden, requestlog.ActionRuleBlocked)
|
||||||
requestlog.ActionBlockedByBlockRule)
|
s.request(client, "/.env", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
s.request(client, "/.env", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
s.get(client, http.StatusForbidden, requestlog.ActionPermanentlyBanned)
|
|
||||||
|
|
||||||
metrics := s.scrape()
|
metrics := s.scrape()
|
||||||
wantMetric(t, metrics,
|
wantMetric(t, metrics,
|
||||||
@@ -235,10 +229,10 @@ func TestMetricsCountRuleMatchesAndBansForAnAttack(t *testing.T) {
|
|||||||
wantMetric(t, metrics,
|
wantMetric(t, metrics,
|
||||||
`sww_rule_matches_total{action="ban",instance="app",rule_id="probe"}`, 1)
|
`sww_rule_matches_total{action="ban",instance="app",rule_id="probe"}`, 1)
|
||||||
wantMetric(t, metrics, `sww_rules_loaded{instance="app"}`, 2)
|
wantMetric(t, metrics, `sww_rules_loaded{instance="app"}`, 2)
|
||||||
wantMetric(t, metrics, `sww_requests_total{action="blocked_by_block_rule",`+
|
wantMetric(t, metrics, `sww_requests_total{action="rule_blocked",`+
|
||||||
`instance="app",status_class="4xx"}`, 1)
|
`instance="app",status_class="4xx"}`, 1)
|
||||||
wantMetric(t, metrics,
|
wantMetric(t, metrics,
|
||||||
`sww_offences_total{instance="app",kind="blocked_by_block_rule"}`, 1)
|
`sww_offences_total{instance="app",kind="rule_blocked"}`, 1)
|
||||||
wantMetric(t, metrics, `sww_offences_total{instance="app",kind="attack"}`, 1)
|
wantMetric(t, metrics, `sww_offences_total{instance="app",kind="attack"}`, 1)
|
||||||
wantMetric(t, metrics, `sww_bans_made_total{cause="attack",instance="app"}`, 1)
|
wantMetric(t, metrics, `sww_bans_made_total{cause="attack",instance="app"}`, 1)
|
||||||
wantMetric(t, metrics, `sww_bans_made_total{cause="limit",instance="app"}`, 0)
|
wantMetric(t, metrics, `sww_bans_made_total{cause="limit",instance="app"}`, 0)
|
||||||
|
|||||||
@@ -68,7 +68,7 @@ func TestRequestFromAllowNetsIsNotCounted(t *testing.T) {
|
|||||||
{listedAddr, http.StatusOK, requestlog.ActionForward},
|
{listedAddr, http.StatusOK, requestlog.ActionForward},
|
||||||
{listedAddr, http.StatusOK, requestlog.ActionForward},
|
{listedAddr, http.StatusOK, requestlog.ActionForward},
|
||||||
{unlistedAddr, http.StatusOK, requestlog.ActionForward},
|
{unlistedAddr, http.StatusOK, requestlog.ActionForward},
|
||||||
{unlistedAddr, http.StatusForbidden, requestlog.ActionClientIsRateLimited},
|
{unlistedAddr, http.StatusForbidden, requestlog.ActionRateLimited},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -92,7 +92,7 @@ func TestDenyNetsRefuseBeforeTheLookup(t *testing.T) {
|
|||||||
wantStatus(t, do(t, req), http.StatusForbidden)
|
wantStatus(t, do(t, req), http.StatusForbidden)
|
||||||
|
|
||||||
line := out.endedLine(t)
|
line := out.endedLine(t)
|
||||||
wantLine(t, line, http.StatusForbidden, requestlog.ActionClientDenied)
|
wantLine(t, line, http.StatusForbidden, requestlog.ActionDenied)
|
||||||
|
|
||||||
if len(asked()) != 0 {
|
if len(asked()) != 0 {
|
||||||
t.Errorf("GeoJS was asked about %v, want nothing", asked())
|
t.Errorf("GeoJS was asked about %v, want nothing", asked())
|
||||||
@@ -108,7 +108,7 @@ func TestDenyNetsRefusalNamesItsHeuristicAndNetblock(t *testing.T) {
|
|||||||
|
|
||||||
s, _, _ := startWithClock(t, "", map[string]string{denyNets: clientsNet})
|
s, _, _ := startWithClock(t, "", map[string]string{denyNets: clientsNet})
|
||||||
|
|
||||||
line := s.get(client, http.StatusForbidden, requestlog.ActionClientDenied)
|
line := s.get(client, http.StatusForbidden, requestlog.ActionDenied)
|
||||||
wantHeuristic(t, line, "deny_nets", `{"netblock": "`+clientsNet+`"}`)
|
wantHeuristic(t, line, "deny_nets", `{"netblock": "`+clientsNet+`"}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -125,10 +125,10 @@ func TestRequestRefusedByDenyNetsIsNotCounted(t *testing.T) {
|
|||||||
// listedAddr's refused requests are not counted, so the first request
|
// listedAddr's refused requests are not counted, so the first request
|
||||||
// from unlistedAddr is within the limit of one a minute.
|
// from unlistedAddr is within the limit of one a minute.
|
||||||
wantAnswers(t, addr, out, []sentRequest{
|
wantAnswers(t, addr, out, []sentRequest{
|
||||||
{listedAddr, http.StatusForbidden, requestlog.ActionClientDenied},
|
{listedAddr, http.StatusForbidden, requestlog.ActionDenied},
|
||||||
{listedAddr, http.StatusForbidden, requestlog.ActionClientDenied},
|
{listedAddr, http.StatusForbidden, requestlog.ActionDenied},
|
||||||
{unlistedAddr, http.StatusOK, requestlog.ActionForward},
|
{unlistedAddr, http.StatusOK, requestlog.ActionForward},
|
||||||
{unlistedAddr, http.StatusForbidden, requestlog.ActionClientIsRateLimited},
|
{unlistedAddr, http.StatusForbidden, requestlog.ActionRateLimited},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -152,8 +152,8 @@ func TestRateLimitExemptNetsAreNeitherCountedNorRefused(t *testing.T) {
|
|||||||
{listedAddr, http.StatusOK, requestlog.ActionForward},
|
{listedAddr, http.StatusOK, requestlog.ActionForward},
|
||||||
{listedAddr, http.StatusOK, requestlog.ActionForward},
|
{listedAddr, http.StatusOK, requestlog.ActionForward},
|
||||||
{unlistedAddr, http.StatusOK, requestlog.ActionForward},
|
{unlistedAddr, http.StatusOK, requestlog.ActionForward},
|
||||||
{unlistedAddr, http.StatusForbidden, requestlog.ActionClientIsRateLimited},
|
{unlistedAddr, http.StatusForbidden, requestlog.ActionRateLimited},
|
||||||
{fromKP, http.StatusForbidden, requestlog.ActionClientDenied},
|
{fromKP, http.StatusForbidden, requestlog.ActionDenied},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -20,7 +20,7 @@ func (tagDeny) check(_ context.Context, rq *request, _ time.Time) *verdict {
|
|||||||
for _, tag := range rq.line.Tags {
|
for _, tag := range rq.line.Tags {
|
||||||
if slices.Contains(rq.h.config.TagDeny, tag) {
|
if slices.Contains(rq.h.config.TagDeny, tag) {
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionClientDenied,
|
action: requestlog.ActionDenied,
|
||||||
figures: map[string]any{"tag": tag},
|
figures: map[string]any{"tag": tag},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ func TestTagDenyRefusesALookupTagAndNamesTheTag(t *testing.T) {
|
|||||||
// in capitals.
|
// in capitals.
|
||||||
s, _, _ := startWithLookups(t, map[string]string{tagDeny: "Country=kp"})
|
s, _, _ := startWithLookups(t, map[string]string{tagDeny: "Country=kp"})
|
||||||
|
|
||||||
line := s.get(fromKP, http.StatusForbidden, requestlog.ActionClientDenied)
|
line := s.get(fromKP, http.StatusForbidden, requestlog.ActionDenied)
|
||||||
wantHeuristic(t, line, "tag_deny", `{"tag": "country=KP"}`)
|
wantHeuristic(t, line, "tag_deny", `{"tag": "country=KP"}`)
|
||||||
wantTags(t, line, "asn=64511", "country=KP")
|
wantTags(t, line, "asn=64511", "country=KP")
|
||||||
|
|
||||||
@@ -26,7 +26,7 @@ func TestTagDenyRefusesAnASNTag(t *testing.T) {
|
|||||||
|
|
||||||
s, _, _ := startWithLookups(t, map[string]string{tagDeny: "asn=64511"})
|
s, _, _ := startWithLookups(t, map[string]string{tagDeny: "asn=64511"})
|
||||||
|
|
||||||
wantHeuristic(t, s.get(fromKP, http.StatusForbidden, requestlog.ActionClientDenied),
|
wantHeuristic(t, s.get(fromKP, http.StatusForbidden, requestlog.ActionDenied),
|
||||||
"tag_deny", `{"tag": "asn=64511"}`)
|
"tag_deny", `{"tag": "asn=64511"}`)
|
||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
}
|
}
|
||||||
@@ -41,7 +41,7 @@ func TestTagDenyRefusesAListTagBeforeTheRateLimitsCount(t *testing.T) {
|
|||||||
|
|
||||||
// Each is refused for its tag: none is counted to break the limit of one.
|
// Each is refused for its tag: none is counted to break the limit of one.
|
||||||
for range 3 {
|
for range 3 {
|
||||||
wantHeuristic(t, s.get(fromDE, http.StatusForbidden, requestlog.ActionClientDenied),
|
wantHeuristic(t, s.get(fromDE, http.StatusForbidden, requestlog.ActionDenied),
|
||||||
"tag_deny", `{"tag": "torexit"}`)
|
"tag_deny", `{"tag": "torexit"}`)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -58,7 +58,7 @@ func TestTagDenyRefusesAMixedCaseZoneByItsLowerCaseTag(t *testing.T) {
|
|||||||
})
|
})
|
||||||
loadVerdicts(server, map[string][]string{fromDE: {mixedCaseZone}})
|
loadVerdicts(server, map[string][]string{fromDE: {mixedCaseZone}})
|
||||||
|
|
||||||
wantHeuristic(t, s.get(fromDE, http.StatusForbidden, requestlog.ActionClientDenied),
|
wantHeuristic(t, s.get(fromDE, http.StatusForbidden, requestlog.ActionDenied),
|
||||||
"tag_deny", `{"tag": "dnsbl=`+dnsblZone+`"}`)
|
"tag_deny", `{"tag": "dnsbl=`+dnsblZone+`"}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -69,7 +69,7 @@ func TestTagDenyRefusesAnExploitSignatureOfARuleThatOnlyLogs(t *testing.T) {
|
|||||||
rulesDir: writeRules(t, "noted path log ^/\n"), tagDeny: "exploitsignature",
|
rulesDir: writeRules(t, "noted path log ^/\n"), tagDeny: "exploitsignature",
|
||||||
})
|
})
|
||||||
|
|
||||||
line := s.get(client, http.StatusForbidden, requestlog.ActionClientDenied)
|
line := s.get(client, http.StatusForbidden, requestlog.ActionDenied)
|
||||||
wantHeuristic(t, line, "tag_deny", `{"tag": "exploitsignature"}`)
|
wantHeuristic(t, line, "tag_deny", `{"tag": "exploitsignature"}`)
|
||||||
wantTags(t, line, "exploitsignature")
|
wantTags(t, line, "exploitsignature")
|
||||||
}
|
}
|
||||||
@@ -87,7 +87,7 @@ func TestTagLimitPercentLowersALimitTheLowestWinning(t *testing.T) {
|
|||||||
// A quarter of 4 requests a minute: the second breaks the limit.
|
// A quarter of 4 requests a minute: the second breaks the limit.
|
||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
line := s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
line := s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
|
wantPercent(t, "limit_percent", line.LimitPercent, line.LimitPercentSetting,
|
||||||
"25 from "+tagLimitPercent)
|
"25 from "+tagLimitPercent)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -80,7 +80,7 @@ func TestTagListsTagARequestRefusedBeforeTheLookup(t *testing.T) {
|
|||||||
})
|
})
|
||||||
loadLists(t, server, map[string][]string{vpnList: {fromDE, fromKP}})
|
loadLists(t, server, map[string][]string{vpnList: {fromDE, fromKP}})
|
||||||
|
|
||||||
line := s.get(fromKP, http.StatusForbidden, requestlog.ActionClientDenied)
|
line := s.get(fromKP, http.StatusForbidden, requestlog.ActionDenied)
|
||||||
if line.Heuristic != "deny_nets" {
|
if line.Heuristic != "deny_nets" {
|
||||||
t.Errorf("log line has heuristic %q, want deny_nets", line.Heuristic)
|
t.Errorf("log line has heuristic %q, want deny_nets", line.Heuristic)
|
||||||
}
|
}
|
||||||
@@ -89,9 +89,9 @@ func TestTagListsTagARequestRefusedBeforeTheLookup(t *testing.T) {
|
|||||||
|
|
||||||
// The second request breaks the limit of one a minute, which bans the client.
|
// The second request breaks the limit of one a minute, which bans the client.
|
||||||
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
s.get(fromDE, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(fromDE, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(fromDE, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
line = s.get(fromDE, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
line = s.get(fromDE, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
if line.Heuristic != "active_ban" {
|
if line.Heuristic != "active_ban" {
|
||||||
t.Errorf("log line has heuristic %q, want active_ban", line.Heuristic)
|
t.Errorf("log line has heuristic %q, want active_ban", line.Heuristic)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -8,6 +8,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -29,9 +30,10 @@ func (trapPaths) check(_ context.Context, rq *request, now time.Time) *verdict {
|
|||||||
}
|
}
|
||||||
|
|
||||||
rq.attack = true
|
rq.attack = true
|
||||||
|
rq.banForAttack(now, bans.Notes{TrapPath: path})
|
||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: rq.banForAttack(now, bans.Notes{TrapPath: path}),
|
action: requestlog.ActionBanned,
|
||||||
figures: map[string]any{"path": path},
|
figures: map[string]any{"path": path},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -43,7 +43,7 @@ func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
|
|||||||
|
|
||||||
// The query is not part of the path.
|
// The query is not part of the path.
|
||||||
line := s.request(client, "/wp-login.php?redirect_to=x", http.StatusTooManyRequests,
|
line := s.request(client, "/wp-login.php?redirect_to=x", http.StatusTooManyRequests,
|
||||||
requestlog.ActionTemporarilyBanned)
|
requestlog.ActionBanned)
|
||||||
wantRuleIDs(t, line)
|
wantRuleIDs(t, line)
|
||||||
|
|
||||||
if line.BanExpires != requestlog.FormatTime(start.Add(7*24*time.Hour)) {
|
if line.BanExpires != requestlog.FormatTime(start.Add(7*24*time.Hour)) {
|
||||||
@@ -77,7 +77,7 @@ func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// The next request is refused under the ban, and makes it permanent.
|
// The next request is refused under the ban, and makes it permanent.
|
||||||
line = s.get(client, http.StatusTooManyRequests, requestlog.ActionPermanentlyBanned)
|
line = s.get(client, http.StatusTooManyRequests, requestlog.ActionBanned)
|
||||||
if line.BanExpires != permanent {
|
if line.BanExpires != permanent {
|
||||||
t.Errorf("log line has ban_expires %q, want permanent", line.BanExpires)
|
t.Errorf("log line has ban_expires %q, want permanent", line.BanExpires)
|
||||||
}
|
}
|
||||||
@@ -89,7 +89,7 @@ func TestTrapPathRefusalNamesItsHeuristicAndPath(t *testing.T) {
|
|||||||
s, _, _ := startWithClock(t, "", map[string]string{trapPaths: trapPathList})
|
s, _, _ := startWithClock(t, "", map[string]string{trapPaths: trapPathList})
|
||||||
|
|
||||||
line := s.request(client, "/wp-login.php?redirect_to=x", http.StatusForbidden,
|
line := s.request(client, "/wp-login.php?redirect_to=x", http.StatusForbidden,
|
||||||
requestlog.ActionTemporarilyBanned)
|
requestlog.ActionBanned)
|
||||||
wantHeuristic(t, line, "trap_paths", `{"path": "/wp-login.php"}`)
|
wantHeuristic(t, line, "trap_paths", `{"path": "/wp-login.php"}`)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -100,8 +100,7 @@ func TestTrapPathMatchesThePathDecodedOnce(t *testing.T) {
|
|||||||
|
|
||||||
// %2577 decodes once to %77, not to w.
|
// %2577 decodes once to %77, not to w.
|
||||||
s.request(otherClient, "/%2577p-login.php", http.StatusOK, requestlog.ActionForward)
|
s.request(otherClient, "/%2577p-login.php", http.StatusOK, requestlog.ActionForward)
|
||||||
s.request(client, "/%77p-login.php", http.StatusForbidden,
|
s.request(client, "/%77p-login.php", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
requestlog.ActionTemporarilyBanned)
|
|
||||||
|
|
||||||
// The ban's notes give the path as the client sent it.
|
// The ban's notes give the path as the client sent it.
|
||||||
got := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
|
got := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
|
||||||
@@ -118,8 +117,7 @@ func TestTrapPathsNeedNoRuleFiles(t *testing.T) {
|
|||||||
"SWW_RULES_ENABLED": "false",
|
"SWW_RULES_ENABLED": "false",
|
||||||
})
|
})
|
||||||
|
|
||||||
s.request(client, "/xmlrpc.php", http.StatusForbidden,
|
s.request(client, "/xmlrpc.php", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
requestlog.ActionTemporarilyBanned)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestObserveModeLogsWhatATrapPathWouldDo(t *testing.T) {
|
func TestObserveModeLogsWhatATrapPathWouldDo(t *testing.T) {
|
||||||
@@ -131,7 +129,7 @@ func TestObserveModeLogsWhatATrapPathWouldDo(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
line := s.request(client, "/xmlrpc.php", http.StatusOK, requestlog.ActionForward)
|
line := s.request(client, "/xmlrpc.php", http.StatusOK, requestlog.ActionForward)
|
||||||
wantWouldAction(t, line, requestlog.ActionTemporarilyBanned)
|
wantWouldAction(t, line, requestlog.ActionBanned)
|
||||||
|
|
||||||
// No ban was made.
|
// No ban was made.
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|||||||
@@ -426,7 +426,7 @@ func (m *Metrics) Offences(r ratelimit.Request) {
|
|||||||
for kind, committed := range map[string]bool{
|
for kind, committed := range map[string]bool{
|
||||||
"limit": r.BrokeLimit,
|
"limit": r.BrokeLimit,
|
||||||
"attack": r.Attack,
|
"attack": r.Attack,
|
||||||
"blocked_by_block_rule": r.BlockedByBlockRule,
|
"rule_blocked": r.RuleBlocked,
|
||||||
"waf_blocked": r.WAFBlocked,
|
"waf_blocked": r.WAFBlocked,
|
||||||
} {
|
} {
|
||||||
if committed {
|
if committed {
|
||||||
|
|||||||
@@ -122,11 +122,11 @@ type Responses struct {
|
|||||||
type Offences struct {
|
type Offences struct {
|
||||||
// Limit is its requests that broke a rate limit, a byte limit or the
|
// Limit is its requests that broke a rate limit, a byte limit or the
|
||||||
// error burst, Attack those that were a clear sign of attack, a match
|
// error burst, Attack those that were a clear sign of attack, a match
|
||||||
// of a ban rule or a request for a trap path, BlockedByBlockRule those a
|
// of a ban rule or a request for a trap path, RuleBlocked those a block
|
||||||
// block rule refused, and WAFBlocked those the Core Rule Set refused.
|
// rule refused, and WAFBlocked those the Core Rule Set refused.
|
||||||
Limit int64 `json:"limit"`
|
Limit int64 `json:"limit"`
|
||||||
Attack int64 `json:"attack"`
|
Attack int64 `json:"attack"`
|
||||||
BlockedByBlockRule int64 `json:"blocked_by_block_rule"`
|
RuleBlocked int64 `json:"rule_blocked"`
|
||||||
WAFBlocked int64 `json:"waf_blocked"`
|
WAFBlocked int64 `json:"waf_blocked"`
|
||||||
// TokenRefused is what a clients.json written by an earlier version gives,
|
// TokenRefused is what a clients.json written by an earlier version gives,
|
||||||
// which must still load. Nothing adds to it.
|
// which must still load. Nothing adds to it.
|
||||||
@@ -147,11 +147,11 @@ type Request struct {
|
|||||||
ResponseBytes int64
|
ResponseBytes int64
|
||||||
// BrokeLimit is true for a request that broke a rate limit, a byte
|
// BrokeLimit is true for a request that broke a rate limit, a byte
|
||||||
// limit or the error burst, Attack for one that matched a ban rule or
|
// limit or the error burst, Attack for one that matched a ban rule or
|
||||||
// asked for a trap path, BlockedByBlockRule for one a block rule refused,
|
// asked for a trap path, RuleBlocked for one a block rule refused, and
|
||||||
// and WAFBlocked for one the Core Rule Set refused.
|
// WAFBlocked for one the Core Rule Set refused.
|
||||||
BrokeLimit bool
|
BrokeLimit bool
|
||||||
Attack bool
|
Attack bool
|
||||||
BlockedByBlockRule bool
|
RuleBlocked bool
|
||||||
WAFBlocked bool
|
WAFBlocked bool
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -323,8 +323,8 @@ func (l *Limiter) AddToHistory(client netip.Prefix, now time.Time, r Request) {
|
|||||||
h.Offences.Attack++
|
h.Offences.Attack++
|
||||||
}
|
}
|
||||||
|
|
||||||
if r.BlockedByBlockRule {
|
if r.RuleBlocked {
|
||||||
h.Offences.BlockedByBlockRule++
|
h.Offences.RuleBlocked++
|
||||||
}
|
}
|
||||||
|
|
||||||
if r.WAFBlocked {
|
if r.WAFBlocked {
|
||||||
|
|||||||
@@ -20,37 +20,32 @@ const (
|
|||||||
// the app.
|
// the app.
|
||||||
ActionForward = "forward"
|
ActionForward = "forward"
|
||||||
|
|
||||||
// ActionRequestHeaderTooLarge is a request over the header limits.
|
// ActionTooLarge is a request over the header limits.
|
||||||
ActionRequestHeaderTooLarge = "request_header_too_large"
|
ActionTooLarge = "too_large"
|
||||||
|
|
||||||
// ActionClientIsRateLimited is a request refused because it took its
|
// ActionRateLimited is a request refused because it took its client
|
||||||
// client over a rate limit, which bans the client.
|
// over a rate limit, which bans the client.
|
||||||
ActionClientIsRateLimited = "client_is_rate_limited"
|
ActionRateLimited = "rate_limited"
|
||||||
|
|
||||||
// ActionTemporarilyBanned is a request refused because a ban that runs
|
// ActionBanned is a request refused because a ban covers its client,
|
||||||
// out covers its client, or because it matched a ban rule, asked for a
|
// or because it matched a ban rule, asked for a trap path or the
|
||||||
// trap path or the CrowdSec decision list lists its client, and so made
|
// CrowdSec decision list lists its client, each of which bans the
|
||||||
// such a ban.
|
// client.
|
||||||
ActionTemporarilyBanned = "temporarily_banned"
|
ActionBanned = "banned"
|
||||||
|
|
||||||
// ActionPermanentlyBanned is a request refused because a ban that never
|
// ActionRuleBlocked is a request refused because it matched a block
|
||||||
// runs out covers its client, or because the request made such a ban or
|
// rule.
|
||||||
// made its client's ban permanent.
|
ActionRuleBlocked = "rule_blocked"
|
||||||
ActionPermanentlyBanned = "permanently_banned"
|
|
||||||
|
|
||||||
// ActionBlockedByBlockRule is a request refused because it matched a
|
|
||||||
// block rule.
|
|
||||||
ActionBlockedByBlockRule = "blocked_by_block_rule"
|
|
||||||
|
|
||||||
// ActionWAFBlocked is a request refused because the Core Rule Set
|
// ActionWAFBlocked is a request refused because the Core Rule Set
|
||||||
// scored it at or over SWW_WAF_ANOMALY_THRESHOLD.
|
// scored it at or over SWW_WAF_ANOMALY_THRESHOLD.
|
||||||
ActionWAFBlocked = "waf_blocked"
|
ActionWAFBlocked = "waf_blocked"
|
||||||
|
|
||||||
// ActionClientDenied is a request refused because its client is in
|
// ActionDenied is a request refused because its client is in
|
||||||
// SWW_DENY_NETS, in a blocklist while SWW_BLOCKLIST_ACTION is deny,
|
// SWW_DENY_NETS, in a blocklist while SWW_BLOCKLIST_ACTION is deny,
|
||||||
// or listed by a DNSBL zone, or scored a hit by AbuseIPDB, while
|
// or listed by a DNSBL zone, or scored a hit by AbuseIPDB, while
|
||||||
// SWW_REPUTATION_ACTION is deny, or that carries a tag in SWW_TAG_DENY.
|
// SWW_REPUTATION_ACTION is deny, or that carries a tag in SWW_TAG_DENY.
|
||||||
ActionClientDenied = "client_denied"
|
ActionDenied = "denied"
|
||||||
|
|
||||||
// ActionCountryDenied is a request refused because its client's country
|
// ActionCountryDenied is a request refused because its client's country
|
||||||
// is not in SWW_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
// is not in SWW_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
||||||
@@ -139,10 +134,10 @@ type Line struct {
|
|||||||
// The decision.
|
// The decision.
|
||||||
Action string `json:"action"`
|
Action string `json:"action"`
|
||||||
// WouldAction is, in observe mode, the action enforce mode would have
|
// WouldAction is, in observe mode, the action enforce mode would have
|
||||||
// taken with a request it would have refused: ActionClientDenied,
|
// taken with a request it would have refused: ActionDenied,
|
||||||
// ActionTemporarilyBanned, ActionPermanentlyBanned, ActionCountryDenied,
|
// ActionBanned, ActionCountryDenied, ActionRateLimited,
|
||||||
// ActionClientIsRateLimited, ActionBlockedByBlockRule, ActionWAFBlocked,
|
// ActionRuleBlocked, ActionWAFBlocked, or, in discriminator, ActionTooLarge
|
||||||
// or, in discriminator, ActionRequestHeaderTooLarge for the header limits.
|
// for the header limits.
|
||||||
WouldAction string `json:"would_action,omitempty"`
|
WouldAction string `json:"would_action,omitempty"`
|
||||||
// Heuristic is the name of the heuristic that refused the request, in
|
// Heuristic is the name of the heuristic that refused the request, in
|
||||||
// observe mode too, and HeuristicFigures are the figures it acted on, by
|
// observe mode too, and HeuristicFigures are the figures it acted on, by
|
||||||
|
|||||||
@@ -25,9 +25,8 @@ const (
|
|||||||
// failureKept is how long routewatch is not asked again after a query
|
// failureKept is how long routewatch is not asked again after a query
|
||||||
// fails, so that a dead routewatch is not asked on every request.
|
// fails, so that a dead routewatch is not asked on every request.
|
||||||
failureKept = time.Minute
|
failureKept = time.Minute
|
||||||
// queryTimeout is how long one query may take. routewatch answers only
|
// queryTimeout is how long one query may take.
|
||||||
// after a reverse DNS lookup it allows 2 seconds, so this is longer.
|
queryTimeout = 2 * time.Second
|
||||||
queryTimeout = 5 * time.Second
|
|
||||||
// maxAnswers is how many answers are kept. Past it, the one that came
|
// maxAnswers is how many answers are kept. Past it, the one that came
|
||||||
// longest ago is dropped.
|
// longest ago is dropped.
|
||||||
maxAnswers = 100000
|
maxAnswers = 100000
|
||||||
|
|||||||
@@ -21,10 +21,7 @@ const (
|
|||||||
routedAnswer = `{"status":"ok","data":{"results":[` +
|
routedAnswer = `{"status":"ok","data":{"results":[` +
|
||||||
`{"ip":"192.0.2.1","netblock":"192.0.2.0/24","asn":64496}]}}`
|
`{"ip":"192.0.2.1","netblock":"192.0.2.0/24","asn":64496}]}}`
|
||||||
noRouteAnswer = `{"status":"ok","data":{"results":[{"ip":"192.0.2.1"}]}}`
|
noRouteAnswer = `{"status":"ok","data":{"results":[{"ip":"192.0.2.1"}]}}`
|
||||||
// slowAnswer is longer than routewatch allows a reverse DNS lookup.
|
waitLimit = 5 * time.Second
|
||||||
slowAnswer = 2500 * time.Millisecond
|
|
||||||
// waitLimit is longer than the Client allows a query.
|
|
||||||
waitLimit = 10 * time.Second
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// clock is a time the tests move by hand.
|
// clock is a time the tests move by hand.
|
||||||
@@ -52,7 +49,6 @@ func (c *clock) advance(by time.Duration) {
|
|||||||
// the process log.
|
// the process log.
|
||||||
type fixture struct {
|
type fixture struct {
|
||||||
t *testing.T
|
t *testing.T
|
||||||
server *httptest.Server
|
|
||||||
client *routewatch.Client
|
client *routewatch.Client
|
||||||
clk *clock
|
clk *clock
|
||||||
queries *atomic.Int32
|
queries *atomic.Int32
|
||||||
@@ -91,16 +87,16 @@ func newFixture(t *testing.T, standIn http.HandlerFunc) *fixture {
|
|||||||
log: &syncBuffer{},
|
log: &syncBuffer{},
|
||||||
}
|
}
|
||||||
|
|
||||||
f.server = httptest.NewServer(http.HandlerFunc(
|
server := httptest.NewServer(http.HandlerFunc(
|
||||||
func(w http.ResponseWriter, r *http.Request) {
|
func(w http.ResponseWriter, r *http.Request) {
|
||||||
f.queries.Add(1)
|
f.queries.Add(1)
|
||||||
f.path.Store(r.URL.Path)
|
f.path.Store(r.URL.Path)
|
||||||
standIn(w, r)
|
standIn(w, r)
|
||||||
}))
|
}))
|
||||||
t.Cleanup(f.server.Close)
|
t.Cleanup(server.Close)
|
||||||
|
|
||||||
f.client = routewatch.New(routewatch.Params{
|
f.client = routewatch.New(routewatch.Params{
|
||||||
URL: f.server.URL + "/", Now: f.clk.Now,
|
URL: server.URL + "/", Now: f.clk.Now,
|
||||||
ProcessLog: slog.New(slog.NewTextHandler(f.log, nil)),
|
ProcessLog: slog.New(slog.NewTextHandler(f.log, nil)),
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -141,26 +137,6 @@ func (f *fixture) waitFor(why routewatch.Missing) routewatch.Route {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// wantFailureLoggedWithoutTheClient waits for the process log to name a failed
|
|
||||||
// query, and checks that it does not name the client.
|
|
||||||
func (f *fixture) wantFailureLoggedWithoutTheClient() {
|
|
||||||
f.t.Helper()
|
|
||||||
|
|
||||||
// The failure is noted before it is logged.
|
|
||||||
deadline := time.Now().Add(waitLimit)
|
|
||||||
for !strings.Contains(f.log.String(), "asking routewatch failed") {
|
|
||||||
if time.Now().After(deadline) {
|
|
||||||
f.t.Fatalf("process log %q names no failure", f.log.String())
|
|
||||||
}
|
|
||||||
|
|
||||||
time.Sleep(time.Millisecond)
|
|
||||||
}
|
|
||||||
|
|
||||||
if strings.Contains(f.log.String(), clientAddr) {
|
|
||||||
f.t.Errorf("process log %q names the client", f.log.String())
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRouteGivesRoutewatchsAnswerAndAsksByAddress(t *testing.T) {
|
func TestRouteGivesRoutewatchsAnswerAndAsksByAddress(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -182,25 +158,6 @@ func TestRouteGivesRoutewatchsAnswerAndAsksByAddress(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAnswerAfterASlowReverseDNSLookupIsKept(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
f := newFixture(t, func(w http.ResponseWriter, _ *http.Request) {
|
|
||||||
time.Sleep(slowAnswer)
|
|
||||||
|
|
||||||
_, _ = w.Write([]byte(routedAnswer))
|
|
||||||
})
|
|
||||||
|
|
||||||
f.route()
|
|
||||||
|
|
||||||
route := f.waitFor("")
|
|
||||||
|
|
||||||
want := routewatch.Route{Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496}
|
|
||||||
if route != want {
|
|
||||||
t.Errorf("route %+v, want %+v", route, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestAddressWithoutARouteIsKeptAsNoRoute(t *testing.T) {
|
func TestAddressWithoutARouteIsKeptAsNoRoute(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -234,23 +191,24 @@ func TestAnswerRoutewatchCannotBeTakenFromFails(t *testing.T) {
|
|||||||
|
|
||||||
f.route()
|
f.route()
|
||||||
f.waitFor(routewatch.RequestFailed)
|
f.waitFor(routewatch.RequestFailed)
|
||||||
f.wantFailureLoggedWithoutTheClient()
|
|
||||||
|
// The failure is noted before it is logged.
|
||||||
|
deadline := time.Now().Add(waitLimit)
|
||||||
|
for !strings.Contains(f.log.String(), "asking routewatch failed") {
|
||||||
|
if time.Now().After(deadline) {
|
||||||
|
t.Fatalf("process log %q names no failure", f.log.String())
|
||||||
|
}
|
||||||
|
|
||||||
|
time.Sleep(time.Millisecond)
|
||||||
|
}
|
||||||
|
|
||||||
|
if strings.Contains(f.log.String(), clientAddr) {
|
||||||
|
t.Errorf("process log %q names the client", f.log.String())
|
||||||
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRefusedConnectionFailsTheQueryAndTheLogOmitsTheClient(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
f := newFixture(t, answering(routedAnswer))
|
|
||||||
|
|
||||||
f.server.Close()
|
|
||||||
|
|
||||||
f.route()
|
|
||||||
f.waitFor(routewatch.RequestFailed)
|
|
||||||
f.wantFailureLoggedWithoutTheClient()
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest(t *testing.T) {
|
func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -295,7 +253,6 @@ func TestHangingRoutewatchNeverDelaysAndFailsTheQuery(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
f.waitFor(routewatch.RequestFailed)
|
f.waitFor(routewatch.RequestFailed)
|
||||||
f.wantFailureLoggedWithoutTheClient()
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRepeatedRequestsMakeOneQueryAndTheAnswerExpires(t *testing.T) {
|
func TestRepeatedRequestsMakeOneQueryAndTheAnswerExpires(t *testing.T) {
|
||||||
|
|||||||
@@ -373,13 +373,13 @@ func TestStateKeptAcrossRestarts(t *testing.T) {
|
|||||||
out := runUntilStopped(t, env, func(url string) {
|
out := runUntilStopped(t, env, func(url string) {
|
||||||
wantRefused(t, url)
|
wantRefused(t, url)
|
||||||
})
|
})
|
||||||
out.line(t, "action", "client_is_rate_limited")
|
out.line(t, "action", "rate_limited")
|
||||||
|
|
||||||
// After another, the ban still refuses it.
|
// After another, the ban still refuses it.
|
||||||
out = runUntilStopped(t, env, func(url string) {
|
out = runUntilStopped(t, env, func(url string) {
|
||||||
wantRefused(t, url)
|
wantRefused(t, url)
|
||||||
})
|
})
|
||||||
out.line(t, "action", "temporarily_banned")
|
out.line(t, "action", "banned")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBanRefusesItsNetblockAfterARestartWithAnotherScope(t *testing.T) {
|
func TestBanRefusesItsNetblockAfterARestartWithAnotherScope(t *testing.T) {
|
||||||
@@ -526,7 +526,7 @@ func TestRuleFileAddedWhileRunningTakesEffect(t *testing.T) {
|
|||||||
time.Sleep(pollInterval)
|
time.Sleep(pollInterval)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
out.line(t, "action", "blocked_by_block_rule")
|
out.line(t, "action", "rule_blocked")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRuleFileThatDoesNotParseStopsTheStart(t *testing.T) {
|
func TestRuleFileThatDoesNotParseStopsTheStart(t *testing.T) {
|
||||||
@@ -740,7 +740,7 @@ func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T)
|
|||||||
time.Sleep(pollInterval)
|
time.Sleep(pollInterval)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
wantDeniedByList(t, out.line(t, "action", "client_denied"), dropURL)
|
wantDeniedByList(t, out.line(t, "action", "denied"), dropURL)
|
||||||
|
|
||||||
if fetches := torFetches.Load(); fetches != 1 {
|
if fetches := torFetches.Load(); fetches != 1 {
|
||||||
t.Errorf("tor.txt fetched %d times, want once, before the restart", fetches)
|
t.Errorf("tor.txt fetched %d times, want once, before the restart", fetches)
|
||||||
@@ -843,8 +843,7 @@ func wantDeniedByList(t *testing.T, line map[string]any, listURL string) {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
reputation, _ := line["reputation"].([]any)
|
reputation, _ := line["reputation"].([]any)
|
||||||
if line["action"] != "client_denied" || len(reputation) != 1 ||
|
if line["action"] != "denied" || len(reputation) != 1 || reputation[0] != listURL {
|
||||||
reputation[0] != listURL {
|
|
||||||
t.Errorf("request log line %v, want one denied for %s", line, listURL)
|
t.Errorf("request log line %v, want one denied for %s", line, listURL)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+4
-5
@@ -187,16 +187,15 @@ main() {
|
|||||||
echo "example-app: rproxy passes a request to the app and its answer back"
|
echo "example-app: rproxy passes a request to the app and its answer back"
|
||||||
|
|
||||||
refused || fail "a second request in a minute was not refused"
|
refused || fail "a second request in a minute was not refused"
|
||||||
wait_for "discriminator logged no ban" \
|
wait_for "discriminator logged no ban" logged "$DISCRIMINATOR" '"action":"rate_limited"'
|
||||||
logged "$DISCRIMINATOR" '"action":"client_is_rate_limited"'
|
|
||||||
echo "example-app: a second request in a minute bans the client"
|
echo "example-app: a second request in a minute bans the client"
|
||||||
|
|
||||||
refused_from 203.0.113.9 /.env || fail "a probe for /.env was not refused"
|
refused_from 203.0.113.9 /.env || fail "a probe for /.env was not refused"
|
||||||
wait_for "discriminator logged no ban for the probe" \
|
wait_for "discriminator logged no ban for the probe" \
|
||||||
logged "$DISCRIMINATOR" '"action":"temporarily_banned"' '"rule_ids":["env-file"]'
|
logged "$DISCRIMINATOR" '"action":"banned"' '"rule_ids":["env-file"]'
|
||||||
refused_from 203.0.113.9 / || fail "the client of the probe was let through"
|
refused_from 203.0.113.9 / || fail "the client of the probe was let through"
|
||||||
wait_for "the client's next request did not make its ban permanent" \
|
wait_for "the client's next request did not make its ban permanent" \
|
||||||
logged "$DISCRIMINATOR" '"action":"permanently_banned"' '"ban_expires":"permanent"'
|
logged "$DISCRIMINATOR" '"ban_expires":"permanent"'
|
||||||
echo "example-app: a probe for /.env bans the client, its next request for good"
|
echo "example-app: a probe for /.env bans the client, its next request for good"
|
||||||
|
|
||||||
docker exec "$DISCRIMINATOR" sv stop sww >/dev/null ||
|
docker exec "$DISCRIMINATOR" sv stop sww >/dev/null ||
|
||||||
@@ -233,7 +232,7 @@ main() {
|
|||||||
start_app
|
start_app
|
||||||
refused || fail "the new discriminator container let the banned client through"
|
refused || fail "the new discriminator container let the banned client through"
|
||||||
wait_for "discriminator logged no request refused under the ban" \
|
wait_for "discriminator logged no request refused under the ban" \
|
||||||
logged "$DISCRIMINATOR" '"action":"temporarily_banned"'
|
logged "$DISCRIMINATOR" '"action":"banned"'
|
||||||
echo "example-app: a new discriminator container on the same volume keeps the ban"
|
echo "example-app: a new discriminator container on the same volume keeps the ban"
|
||||||
|
|
||||||
docker rm --force "$DISCRIMINATOR" >/dev/null
|
docker rm --force "$DISCRIMINATOR" >/dev/null
|
||||||
|
|||||||
Reference in new issue
Block a user