Compare commits
2
Commits
cc50b04f2a
..
next
| Author | SHA1 | Date | |
|---|---|---|---|
 clawbot
|
d7c62f2854 | ||
|
clawbot
|
c66277b111 |
No files matched your search
+56
-44
@@ -12,8 +12,8 @@ configured only by environment variables, that provides:
|
||||
- R1: rate limits per minute, per hour and per day
|
||||
- R2: netblocks that bypass rate limiting
|
||||
- R3: alerts on attacks through webhook, Slack or ntfy
|
||||
- R4: alerts on anomalies: requests or bytes per minute or hour, for given IPs
|
||||
or netblocks, crossing a threshold
|
||||
- R4: alerts on anomalies: requests made or bytes transferred per minute or
|
||||
hour, for given IPs or netblocks, crossing a threshold
|
||||
- R5: temporary blocks for abusers, permanent bans for repeat offenders
|
||||
- R6: one or more RBLs or IP reputation APIs
|
||||
- R7: AS number lookup
|
||||
@@ -24,14 +24,19 @@ configured only by environment variables, that provides:
|
||||
|
||||
## Verdict
|
||||
|
||||
Nothing existing is suitable. No surveyed tool meets R8 (scaling limits by AS
|
||||
number or country) or the bytes half of R4 at all, and the tools that come
|
||||
closest on the rest fail R10: they need several containers, YAML or a web UI.
|
||||
The recommendation is to build a small single-binary sidecar; `SPEC.md` in this
|
||||
directory begins that spec. It reuses the mature parts of the surveyed field as
|
||||
libraries and data sources (Coraza and the OWASP Core Rule Set for attack
|
||||
detection, DNSBLs, AbuseIPDB, optionally a CrowdSec decision feed) instead of
|
||||
reimplementing them.
|
||||
Nothing existing is suitable. No surveyed tool meets R8, which asks for limits
|
||||
scaled by AS number or country. None meets R4's alerts on bytes transferred. The
|
||||
tools that come closest on the rest fail R10: they need several containers, YAML
|
||||
or a web UI. The recommendation is to build a small single-binary sidecar;
|
||||
`SPEC.md` in this directory begins that spec.
|
||||
|
||||
The sidecar reuses the mature parts of the surveyed field instead of
|
||||
reimplementing them. It takes these as libraries and data sources:
|
||||
|
||||
- Coraza and the OWASP Core Rule Set, for attack detection
|
||||
- DNSBLs
|
||||
- AbuseIPDB
|
||||
- optionally, a CrowdSec decision feed
|
||||
|
||||
Closest existing options, and why each still falls short:
|
||||
|
||||
@@ -45,13 +50,16 @@ Closest existing options, and why each still falls short:
|
||||
external blacklist URLs, optional CrowdSec), R7 partly (AS number used for
|
||||
blacklist and whitelist decisions), R9 (ModSecurity with the Core Rule
|
||||
Set, or Coraza plugin), env-var settings.
|
||||
- Fails: R8 (AS number and country can only allow or deny, never scale a
|
||||
limit), R4 (no volume or byte threshold alerts in the free edition;
|
||||
reporting is a paid feature), R5 escalation, and R10 in spirit: since 1.6
|
||||
it needs a `bunkerweb` container plus a `bw-scheduler` container and a
|
||||
database, or the all-in-one image that bundles nginx, scheduler, UI and
|
||||
Redis in one container. It is designed to be the front door for many
|
||||
sites, not a per-app sidecar. AGPL-3.0.
|
||||
- Fails:
|
||||
- R8: AS number and country can only allow or deny, never scale a limit.
|
||||
- R4: no alerts on request-count or byte thresholds in the free edition;
|
||||
reporting is a paid feature.
|
||||
- R5: no escalation for repeat offenders.
|
||||
- R10: since 1.6 it is not one plain sidecar. It needs a `bunkerweb`
|
||||
container, a `bw-scheduler` container and a database, or the
|
||||
all-in-one image that bundles nginx, the scheduler, the UI and Redis.
|
||||
- It is designed as the proxy in front of many sites, not as a sidecar for
|
||||
one app. AGPL-3.0.
|
||||
- Unverified: whether several rates (minute, hour, day) can be stacked on
|
||||
the same URL pattern; the documented design is one rate per URL pattern.
|
||||
- CrowdSec is the strongest detection and ban engine, but it is not a proxy.
|
||||
@@ -66,16 +74,20 @@ Closest existing options, and why each still falls short:
|
||||
listed AS numbers is possible, but each is a hand-written YAML scenario,
|
||||
it reacts after the fact by banning, and it is not an inline limiter that
|
||||
answers 429.
|
||||
- Fails: R10 (needs the security engine container with persistent state, log
|
||||
acquisition from traefik, a bouncer such as the traefik plugin, and YAML
|
||||
for acquisition, profiles, scenarios and notifications), bytes half of R4.
|
||||
- Fails:
|
||||
- R10, because it needs:
|
||||
- the security engine container with persistent state
|
||||
- log acquisition from traefik
|
||||
- a bouncer such as the traefik plugin
|
||||
- YAML for acquisition, profiles, scenarios and notifications
|
||||
- R4's alerts on bytes.
|
||||
- CrowdSec plus traefik's own `rateLimit` middleware is the best combination
|
||||
with no new code. It gives inline limiting (one window per middleware, keyed
|
||||
by IP, with `sourceCriterion` exclusions), bans with escalation, alerts and
|
||||
reputation. It still fails R8, the bytes half of R4, R1's three stacked
|
||||
windows only by chaining three middlewares per router, and R10 entirely: the
|
||||
configuration lives in traefik labels and CrowdSec YAML on each host, not in
|
||||
one sidecar's environment.
|
||||
reputation lookups. It still fails R8, R4's alerts on bytes, and R10 entirely:
|
||||
the configuration lives in traefik labels and CrowdSec YAML on each host, not
|
||||
in one sidecar's environment. It meets R1's three windows only by chaining
|
||||
three middlewares per router.
|
||||
|
||||
## Candidate by candidate
|
||||
|
||||
@@ -87,10 +99,10 @@ Closest existing options, and why each still falls short:
|
||||
cached, streamed decision list (recommended), standalone against the
|
||||
central API, or AppSec only. Supports captcha remediation.
|
||||
- Covers: R5, R6, R9, R3 and R7 through the engine (see Verdict).
|
||||
- Misses: the plugin itself does no rate limiting; R8; R4 bytes.
|
||||
- Misses: the plugin itself does no rate limiting; R8; R4's alerts on bytes.
|
||||
- Configuration: traefik static config to load the plugin, dynamic config or
|
||||
labels for the middleware, CrowdSec YAML for everything else.
|
||||
- Sidecar fit: no. It lives inside traefik, plus a separate engine
|
||||
- Sidecar fit: no. It runs inside traefik and needs a separate engine
|
||||
container.
|
||||
- Maturity: widely used (about 900 stars, listed in the traefik plugin
|
||||
catalog, documented by CrowdSec itself), actively maintained. Traefik
|
||||
@@ -115,7 +127,7 @@ Closest existing options, and why each still falls short:
|
||||
- What it is: nginx-derived (Tengine) reverse proxy with a proprietary
|
||||
semantic detection engine and a web console. About 22,000 stars.
|
||||
- Covers: R9 well, basic per-IP rate limiting with block duration, bot
|
||||
challenge, authentication gate.
|
||||
challenge, authentication in front of the app.
|
||||
- Misses: R8, R7, R4; R1 has one window per rule. Country blocking, alert
|
||||
messages, syslog forwarding and the better reputation database are paid
|
||||
(Lite about 10 USD per month, Pro about 100 USD per month); the free
|
||||
@@ -128,7 +140,7 @@ Closest existing options, and why each still falls short:
|
||||
- Coraza (`corazawaf/coraza`) and Coraza-based proxies
|
||||
- What it is: a Go library that implements the ModSecurity rule language and
|
||||
runs the OWASP Core Rule Set. OWASP project, actively maintained, the
|
||||
successor path now that ModSecurity is in maintenance only.
|
||||
successor to ModSecurity, which is now in maintenance only.
|
||||
- Packagings: `coraza-caddy` (Caddy module), `coraza-spoa` (HAProxy),
|
||||
`coraza-proxy-wasm` (Envoy), a traefik WASM plugin, and
|
||||
`coreruleset/coraza-crs-docker` (Caddy plus Coraza plus the Core Rule Set,
|
||||
@@ -142,7 +154,7 @@ Closest existing options, and why each still falls short:
|
||||
- ModSecurity Core Rule Set containers (`owasp/modsecurity-crs`)
|
||||
- What it is: official images of Apache or nginx with ModSecurity and the
|
||||
Core Rule Set, as a reverse proxy configured by env vars (`BACKEND`,
|
||||
paranoia level, anomaly thresholds, many more). Current images carry
|
||||
paranoia level, anomaly thresholds, many more). Current images contain
|
||||
ModSecurity 2.9.x on Apache and Core Rule Set 4.x.
|
||||
- Covers: R9, and R10 (single container, env vars, one backend).
|
||||
- Misses: R1 to R8.
|
||||
@@ -152,8 +164,8 @@ Closest existing options, and why each still falls short:
|
||||
- Anubis (`TecharoHQ/anubis`), 1.27 current
|
||||
- What it is: a single-binary reverse proxy that makes browsers solve a
|
||||
proof-of-work challenge before passing them to `TARGET`. Aimed at
|
||||
scrapers, which is likely a large share of unwanted traffic on a public
|
||||
code forge.
|
||||
scrapers, which are likely a large share of the unwanted traffic on a
|
||||
public code forge.
|
||||
- Covers: R10 well (one container, env vars for listener, target,
|
||||
difficulty, cookies). Policy rules can match on path, user agent, headers,
|
||||
IP ranges, and with the vendor's hosted data service also AS number and
|
||||
@@ -180,13 +192,13 @@ Closest existing options, and why each still falls short:
|
||||
temporary ban, no escalation, state lost on traefik restart) and R2
|
||||
(allowlist). Nothing else.
|
||||
- `juitde/traefik-plugin-fail2ban`, fail2ban connector plugins, GeoBlock
|
||||
plugins (country allow or deny through an external lookup API): same
|
||||
shape, each one small feature, configured in traefik, state in memory.
|
||||
plugins (country allow or deny through an external lookup API): each adds
|
||||
one small feature, is configured in traefik and keeps its state in memory.
|
||||
- Sidecar fit: no; they live inside traefik.
|
||||
- Traefik built-in middlewares
|
||||
- `rateLimit` (one average-and-burst window per middleware, optional Redis
|
||||
in traefik 3.x), `inFlightReq`, `ipAllowList`. No bans, alerts, reputation
|
||||
or AS number awareness.
|
||||
lookups or AS number lookups.
|
||||
- caddy-waf (`fabriziosalmi/caddy-waf`), 0.4.x
|
||||
- What it is: a Caddy module with regex rules and anomaly scoring, per-IP
|
||||
and per-path rate limiting with one configurable window, IP and DNS
|
||||
@@ -198,12 +210,12 @@ Closest existing options, and why each still falls short:
|
||||
- open-appsec (Check Point)
|
||||
- Machine-learning WAF agent attached to nginx, Kong, Envoy or similar, with
|
||||
a declarative policy file or the vendor's cloud console. Covers R9 only;
|
||||
rate limiting and richer features are in paid tiers. Not a sidecar in the
|
||||
required sense.
|
||||
rate limiting and richer features are in paid tiers. It attaches to an
|
||||
existing proxy, so it is not the sidecar R10 asks for.
|
||||
- iocaine
|
||||
- Serves generated garbage pages to clients the fronting proxy classifies as
|
||||
scrapers. Not a limiter, WAF or ban tool; out of scope except as a
|
||||
curiosity for scraper traffic.
|
||||
scrapers. Not a limiter, WAF or ban tool; out of scope, and only of
|
||||
passing interest for scraper traffic.
|
||||
- Pangolin
|
||||
- A tunnelled access platform that bundles traefik and optionally CrowdSec.
|
||||
Replaces the ingress rather than adding a sidecar; out of scope.
|
||||
@@ -214,17 +226,17 @@ Closest existing options, and why each still falls short:
|
||||
Everyone else offers one window per rule or none.
|
||||
- R2 bypass netblocks: BunkerWeb, CrowdSec, traefik, fail2ban plugins.
|
||||
- R3 alerts: CrowdSec (best), BunkerWeb (denied requests only), SafeLine (paid).
|
||||
- R4 volume and byte anomalies: none. CrowdSec can approximate request-count
|
||||
anomalies with a custom scenario; nothing handles bytes.
|
||||
- R5 temporary then permanent: CrowdSec only. BunkerWeb has fixed or permanent,
|
||||
not both by history.
|
||||
- R4 alerts on request counts and bytes: none. CrowdSec can approximate alerts
|
||||
on request counts with a custom scenario; no surveyed tool alerts on bytes.
|
||||
- R5 temporary then permanent: CrowdSec only. BunkerWeb's ban is either one
|
||||
fixed length or permanent, and does not change with a client's history.
|
||||
- R6 reputation: CrowdSec, BunkerWeb, go-away, caddy-waf (static lists).
|
||||
- R7 AS number lookup: CrowdSec (enrichment), BunkerWeb and caddy-waf (allow or
|
||||
deny), Anubis (hosted service).
|
||||
- R8 biased thresholds: none.
|
||||
- R9 attack detection: Coraza or ModSecurity with the Core Rule Set (open,
|
||||
standard), CrowdSec AppSec, SafeLine (closed engine), BunkerWeb (wraps the
|
||||
first).
|
||||
standard), CrowdSec AppSec, SafeLine (closed engine), BunkerWeb (wraps Coraza
|
||||
or ModSecurity).
|
||||
- R10 env-var sidecar: `owasp/modsecurity-crs`, `coraza-crs-docker`, Anubis.
|
||||
None of these covers anything beyond its one job.
|
||||
|
||||
|
||||
@@ -25,11 +25,11 @@ without an answer in time, `rproxy` passes the request to the app.
|
||||
It protects against:
|
||||
|
||||
- Scrapers that fetch every page of a site, all day. Each client gets a limit on
|
||||
requests and on bytes per minute, per hour and per day. The defaults leave
|
||||
room for a person reading a site for hours. They catch a scraper that keeps up
|
||||
one request every 2 seconds. A client over a limit is refused and banned: for
|
||||
a while at first, for three times as long at each repeat, and permanently when
|
||||
it keeps at it.
|
||||
requests and on body bytes per minute, per hour and per day. The defaults
|
||||
leave room for a person reading a site for hours. They catch a scraper that
|
||||
keeps up one request every 2 seconds. A client that breaks a limit is banned:
|
||||
for a while at first, for three times as long at each repeat, and permanently
|
||||
when it keeps at it.
|
||||
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
|
||||
default rule file bans such a client at its first probe, and its next request
|
||||
during the ban makes the ban permanent. Your own rules, or a list of trap
|
||||
@@ -210,6 +210,10 @@ account, a key, a token or an alert destination stays off until you give one.
|
||||
percentage is a whole number from 0 to 100. A path prefix starts with `/`.
|
||||
- `off` switches a limit, a threshold or a timeout off where the table says a
|
||||
setting takes it.
|
||||
- The byte limits and the byte anomaly thresholds count only body bytes: the
|
||||
app's response bodies, the visitors' request bodies, or both, as
|
||||
`SWW_BYTES_COUNT` says. Headers are never counted. What the visitor sent over
|
||||
a WebSocket counts as request bytes, and what the app sent as response bytes.
|
||||
- An invalid setting stops the process at start with one log line naming the
|
||||
variable and what is wrong with the value. The settings in effect are logged
|
||||
on one line at start, with tokens, keys and webhook paths shown as `********`.
|
||||
@@ -217,7 +221,7 @@ account, a key, a token or an alert destination stays off until you give one.
|
||||
Every setting the code reads, with its default:
|
||||
|
||||
| Variable | Default | What it does |
|
||||
| ---------------------------------------- | ------------------------------------------------------------------------------ | ----------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| ---------------------------------------- | ------------------------------------------------------------------------------ | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `SWW_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `discriminator`. |
|
||||
| `SWRP_WAF_BACKEND` | required | `discriminator`'s URL, such as `http://discriminator:8080`. |
|
||||
| `SWRP_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. |
|
||||
@@ -226,8 +230,8 @@ Every setting the code reads, with its default:
|
||||
| `SWRP_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `discriminator`'s answer to a question and to a report, in milliseconds. |
|
||||
| `SWRP_DEFAULT_DISPOSITION` | `allow` | What `rproxy` does with a request when `discriminator` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. |
|
||||
| `SWRP_MAX_REQUEST_SIZE` | `100M` | The largest request body, a size or `off`; a larger one is answered `413`. |
|
||||
| `SWRP_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`; a larger one is answered `502`, or cut off once it has started. |
|
||||
| `SWRP_MAX_REQUEST_TIME_MS` | `60000` | How long the visitor may take to send its request line and headers, and then the whole request to reach the app, in milliseconds or `off`. |
|
||||
| `SWRP_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
|
||||
| `SWRP_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
|
||||
| `SWRP_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
||||
| `SWW_LISTEN_ADDR` | `:8080` | Where `discriminator` listens; leave the host part empty. |
|
||||
| `SWW_INSTANCE_NAME` | the container's host name | The name in every line `discriminator` logs, and in every alert and metric, such as `fsn1/discriminator`. |
|
||||
@@ -236,24 +240,24 @@ Every setting the code reads, with its default:
|
||||
| `SWW_MAX_TRACKED_CLIENTS` | `20000` | The most clients kept, in memory and in `clients.json`; the least recently seen is dropped first. |
|
||||
| `SWW_MAX_BANS` | `5000` | The most bans `discriminator` keeps of its own; bans an admin made do not count. |
|
||||
| `SWW_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
|
||||
| `SWW_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients the request and byte limits neither count nor refuse. |
|
||||
| `SWW_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients are exempt from rate limiting and the byte limits. |
|
||||
| `SWW_DENY_NETS` | empty | Netblocks whose clients are refused. |
|
||||
| `SWW_RATE_LIMIT_PER_MINUTE` | `120` | The most requests a client may make in a minute, a count or `off`; one more bans it. |
|
||||
| `SWW_RATE_LIMIT_PER_HOUR` | `1500` | The most requests a client may make in an hour, a count or `off`; one more bans it. |
|
||||
| `SWW_RATE_LIMIT_PER_DAY` | `6000` | The most requests a client may make in a day, a count or `off`; one more bans it. |
|
||||
| `SWW_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes whose requests the request and byte limits neither count nor refuse, such as `/assets/`. |
|
||||
| `SWW_BYTES_LIMIT_PER_MINUTE` | `off` | The most bytes a client's requests may carry in a minute, a size or `off`; the request that takes a client past any byte limit bans it. |
|
||||
| `SWW_BYTES_LIMIT_PER_HOUR` | `50G` | The most bytes a client's requests may carry in an hour, a size or `off`. |
|
||||
| `SWW_BYTES_LIMIT_PER_DAY` | `100G` | The most bytes a client's requests may carry in a day, a size or `off`. |
|
||||
| `SWW_BYTES_COUNT` | `both` | Which body bytes the byte limits count: `response`, `request` or `both`. |
|
||||
| `SWW_ERROR_BURST_THRESHOLD` | `30` | The most refusals by a rule file, a trap path or the Core Rule Set a client may get in a minute, a count or `off`; one more bans it. |
|
||||
| `SWW_RATE_LIMIT_PER_MINUTE` | `120` | The most requests a client may make in a minute, a count or `off`; the next request is refused and bans the client. |
|
||||
| `SWW_RATE_LIMIT_PER_HOUR` | `1500` | The most requests a client may make in an hour, a count or `off`; the next request is refused and bans the client. |
|
||||
| `SWW_RATE_LIMIT_PER_DAY` | `6000` | The most requests a client may make in a day, a count or `off`; the next request is refused and bans the client. |
|
||||
| `SWW_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes exempt from rate limiting and the byte limits, such as `/assets/`. |
|
||||
| `SWW_BYTES_LIMIT_PER_MINUTE` | `off` | The most bytes counted for a client in a minute, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||
| `SWW_BYTES_LIMIT_PER_HOUR` | `50G` | The most bytes counted for a client in an hour, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||
| `SWW_BYTES_LIMIT_PER_DAY` | `100G` | The most bytes counted for a client in a day, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||
| `SWW_BYTES_COUNT` | `both` | What the byte limits and the byte anomaly thresholds count: `response`, the app's response bodies; `request`, the visitors' request bodies; or `both`. What the visitor sent over a WebSocket counts as request bytes, and what the app sent as response bytes. |
|
||||
| `SWW_ERROR_BURST_THRESHOLD` | `30` | The most refusals a client may get in a minute from a rule file, a trap path or the Core Rule Set, a count or `off`. The next one bans the client. |
|
||||
| `SWW_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. |
|
||||
| `SWW_LIMIT_BAN_DURATION` | `1h` | The ban for a first broken limit. |
|
||||
| `SWW_LIMIT_BAN_REPEAT_WINDOW` | `24h` | A limit broken again within this time after the last such ban ended gets a ban three times as long. |
|
||||
| `SWW_LIMIT_BAN_REPEAT_WINDOW` | `24h` | A client that breaks a limit again within this time after its last such ban ended gets a ban three times as long. |
|
||||
| `SWW_MAX_BAN_DURATION` | `7d` | A ban for a broken limit that would be longer than this is permanent instead. |
|
||||
| `SWW_ATTACK_BAN_DURATION` | `7d` | The ban for a first clear sign of attack: a `ban` rule or a trap path. |
|
||||
| `SWW_BAN_SCOPE_V4_PREFIX` | `32` | The prefix length of the netblock a ban on an IPv4 client covers, from 0 to 32. |
|
||||
| `SWW_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs`, a web service, `file`, or `off`. |
|
||||
| `SWW_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs` (a web service), `file` (a local database) or `off`. |
|
||||
| `SWW_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. |
|
||||
| `SWW_LOOKUP_TIMEOUT` | `800ms` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take. |
|
||||
| `SWW_ADD_LOOKUP_HEADERS` | `false` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
||||
@@ -311,27 +315,27 @@ Every setting the code reads, with its default:
|
||||
| `SWW_ALERT_MAX_PER_HOUR` | `60` | The most alerts sent in an hour, a number or `off`; the rest are counted in the hour's `summary`. |
|
||||
| `SWW_ANOMALY_CLIENT_REQUESTS_PER_MINUTE` | `off` | Alert when one client makes more requests than this in a minute, a count. |
|
||||
| `SWW_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Alert when one client makes more requests than this in an hour, a count. |
|
||||
| `SWW_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Alert when one client's requests carry more bytes than this in a minute, a size. |
|
||||
| `SWW_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Alert when one client's requests carry more bytes than this in an hour, a size. |
|
||||
| `SWW_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one client in a minute, a size. |
|
||||
| `SWW_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one client in an hour, a size. |
|
||||
| `SWW_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Alert when the netblock around a client makes more requests than this in a minute, a count. |
|
||||
| `SWW_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Alert when the netblock around a client makes more requests than this in an hour, a count. |
|
||||
| `SWW_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Alert when the netblock around a client carries more bytes than this in a minute, a size. |
|
||||
| `SWW_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Alert when the netblock around a client carries more bytes than this in an hour, a size. |
|
||||
| `SWW_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the netblock around a client in a minute, a size. |
|
||||
| `SWW_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the netblock around a client in an hour, a size. |
|
||||
| `SWW_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_ANOMALY_NET_` thresholds count, from 0 to 32. |
|
||||
| `SWW_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_ANOMALY_NET_` thresholds count, from 0 to 128. |
|
||||
| `SWW_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Alert when one AS number makes more requests than this in a minute, a count. |
|
||||
| `SWW_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Alert when one AS number makes more requests than this in an hour, a count. |
|
||||
| `SWW_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Alert when one AS number carries more bytes than this in a minute, a size. |
|
||||
| `SWW_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Alert when one AS number carries more bytes than this in an hour, a size. |
|
||||
| `SWW_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one AS number in a minute, a size. |
|
||||
| `SWW_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one AS number in an hour, a size. |
|
||||
| `SWW_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Alert when the whole service gets more requests than this in a minute, a count. |
|
||||
| `SWW_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Alert when the whole service gets more requests than this in an hour, a count. |
|
||||
| `SWW_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Alert when the whole service carries more bytes than this in a minute, a size. |
|
||||
| `SWW_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Alert when the whole service carries more bytes than this in an hour, a size. |
|
||||
| `SWW_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the whole service in a minute, a size. |
|
||||
| `SWW_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the whole service in an hour, a size. |
|
||||
| `SWW_WATCH_NETS` | empty | Named netblocks the `SWW_WATCH_` thresholds count each as a whole, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. |
|
||||
| `SWW_WATCH_REQUESTS_PER_MINUTE` | `off` | Alert when a watched netblock makes more requests than this in a minute, a count. |
|
||||
| `SWW_WATCH_REQUESTS_PER_HOUR` | `off` | Alert when a watched netblock makes more requests than this in an hour, a count. |
|
||||
| `SWW_WATCH_BYTES_PER_MINUTE` | `off` | Alert when a watched netblock carries more bytes than this in a minute, a size. |
|
||||
| `SWW_WATCH_BYTES_PER_HOUR` | `off` | Alert when a watched netblock carries more bytes than this in an hour, a size. |
|
||||
| `SWW_WATCH_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for a watched netblock in a minute, a size. |
|
||||
| `SWW_WATCH_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for a watched netblock in an hour, a size. |
|
||||
|
||||
[`docs/settings.md`](docs/settings.md) gives the values each setting takes in
|
||||
full, and the limits that are fixed rather than settings.
|
||||
@@ -527,8 +531,9 @@ of the request reaches the app.
|
||||
|
||||
17. The request is passed to the app, streaming.
|
||||
18. After the answer, `rproxy` sends `discriminator` a report with the status
|
||||
and the bytes each way. For an allowed request, `discriminator` counts the
|
||||
bytes toward the byte limits and adds the request to the client's history.
|
||||
and the body bytes each way. For an allowed request, `discriminator` counts
|
||||
the bytes `SWW_BYTES_COUNT` names toward the byte limits, bans the client if
|
||||
they take it past one, and adds the request to the client's history.
|
||||
|
||||
A refused visitor is answered with `SWW_BAN_RESPONSE`. A `block` rule or a Core
|
||||
Rule Set match answers `403` and bans nobody, since the Core Rule Set has false
|
||||
|
||||
@@ -22,8 +22,8 @@ to them.
|
||||
key or a token stays off until the operator supplies it.
|
||||
- It keeps a site answering while the part that decides is down, slow or
|
||||
restarting (see "Without an answer from `discriminator`").
|
||||
- It limits each client's requests and bytes per minute, hour and day. It bounds
|
||||
the size and duration of every request and answer. It detects attacks.
|
||||
- It limits each client's requests and body bytes per minute, hour and day. It
|
||||
bounds the size and duration of every request and answer. It detects attacks.
|
||||
- It bans abusers: briefly at first, for seven days on a clear sign of attack,
|
||||
and permanently when they keep at it.
|
||||
- It looks up each client's AS number and country. It can refuse whole countries
|
||||
@@ -192,11 +192,12 @@ request log's `heuristic` field gives that name. `refusal_limit` and
|
||||
sent is removed.
|
||||
18. After the visitor's answer has ended, `rproxy` sends the report (see "The
|
||||
report"). `discriminator` then counts:
|
||||
- For a request it allowed, the response bytes, the request bytes or both
|
||||
are added to the byte counters, as `SWW_BYTES_COUNT` says. For a
|
||||
WebSocket, what it carried each way counts once it closes. A client whose
|
||||
total passes a byte limit times the percentage has broken that limit and
|
||||
is banned (`byte_limits`). The response in progress is not cut off.
|
||||
- For a request it allowed, the body bytes of the response, of the request
|
||||
or of both are added to the byte counters, as `SWW_BYTES_COUNT` says. For
|
||||
a WebSocket, what it carried each way counts once it closes. A client
|
||||
whose total passes a byte limit times the percentage has broken that limit
|
||||
and is banned (`byte_limits`). The request that broke it has already been
|
||||
passed on whole; nothing is cut off.
|
||||
- For a request it allowed, the client's history and the anomaly counters
|
||||
are updated, and the alert thresholds are evaluated.
|
||||
|
||||
@@ -259,7 +260,8 @@ to a ban, each by its own rule:
|
||||
clear sign of attack bans it permanently at once.
|
||||
- A broken limit, one of:
|
||||
- a request over a rate limit;
|
||||
- a response that takes the client's byte total past a byte limit;
|
||||
- the bytes counted for a request and its response taking the client's total
|
||||
past a byte limit;
|
||||
- more than `SWW_ERROR_BURST_THRESHOLD` refusals within a minute.
|
||||
|
||||
The bans for a broken limit:
|
||||
@@ -448,10 +450,15 @@ the same way.
|
||||
|
||||
### Byte limits, per client
|
||||
|
||||
By default there is no minute limit, so a client can download in a burst at full
|
||||
speed. The hour limit defaults to 50 GiB and the day limit to 100 GiB. They
|
||||
allow several downloads of a few GiB each. A response's bytes are counted when
|
||||
it ends (see step 18).
|
||||
The byte limits count body bytes: the app's response bodies, the visitors'
|
||||
request bodies, or both, as `SWW_BYTES_COUNT` says. Headers are never counted.
|
||||
What the visitor sent over a WebSocket counts as request bytes, and what the app
|
||||
sent as response bytes. The bytes counted for a request and its response are
|
||||
added to the client's total once the response has ended (see step 18). So the
|
||||
request that takes a client past a limit is passed on whole, and the ban refuses
|
||||
the client's requests from then on. By default there is no minute limit, so a
|
||||
client can download in a burst at full speed. The hour limit defaults to 50 GiB
|
||||
and the day limit to 100 GiB. They allow several downloads of a few GiB each.
|
||||
|
||||
### Lookup of AS number and country
|
||||
|
||||
@@ -953,7 +960,8 @@ which the metrics show. They reveal a widely distributed scraper, thousands of
|
||||
addresses at low rates each, which the per-client limits do not see. A low
|
||||
percentage for its `asn=` tag in `SWW_TAG_LIMIT_PERCENT` is the response.
|
||||
|
||||
- Requests and bytes are counted per minute and per hour, for each of:
|
||||
- Requests and body bytes, as `SWW_BYTES_COUNT` counts them, are counted per
|
||||
minute and per hour, for each of:
|
||||
- a client;
|
||||
- the surrounding netblock;
|
||||
- an AS number;
|
||||
@@ -1263,9 +1271,9 @@ short.
|
||||
### Answering in time
|
||||
|
||||
`rproxy` stops waiting for `discriminator` after `SWRP_TIMEOUT_MS`, by default 2
|
||||
seconds, and `SWRP_DEFAULT_DISPOSITION` then decides. So at the defaults,
|
||||
`discriminator` answers a request well within that time, whatever the request
|
||||
and whatever its outside sources do.
|
||||
seconds, and `SWRP_DEFAULT_DISPOSITION` then decides. The longest a question
|
||||
waits on an outside source is `SWW_LOOKUP_TIMEOUT`, by default 800 milliseconds,
|
||||
well within that time.
|
||||
|
||||
GeoJS is the only outside source a question can wait on. A question waits only
|
||||
when step 6 says so, and for at most `SWW_LOOKUP_TIMEOUT`, by default 800
|
||||
@@ -1361,9 +1369,9 @@ answer.
|
||||
For a request it holds, `discriminator` writes the request's last log line (see
|
||||
"The log"). For a request it allowed, it first:
|
||||
|
||||
- counts the bytes as `SWW_BYTES_COUNT` says, toward the client's byte limits at
|
||||
the percentages it gave that request;
|
||||
- bans a client that passes a byte limit;
|
||||
- counts the body bytes `SWW_BYTES_COUNT` names, toward the client's byte limits
|
||||
at the percentages it gave that request;
|
||||
- bans a client whose total passes a byte limit;
|
||||
- adds the request with its status to the client's history, the anomaly counters
|
||||
and the metrics.
|
||||
|
||||
|
||||
@@ -5,6 +5,9 @@ What `GET /_sww/metrics` answers with, at `discriminator`'s own address.
|
||||
keeps the `instance` label as `exported_instance` unless the scrape sets
|
||||
`honor_labels: true`. `rproxy` has no metrics.
|
||||
|
||||
Every `sww_*request_bytes_total` and `sww_*response_bytes_total` series counts
|
||||
body bytes, never headers, whatever `SWW_BYTES_COUNT` says.
|
||||
|
||||
## Traffic
|
||||
|
||||
- `sww_requests_total`, `sww_request_bytes_total` and
|
||||
|
||||
+8
-6
@@ -145,20 +145,22 @@ Every one of these lines has:
|
||||
included. It is left out when rate limiting did not count the request. It
|
||||
holds:
|
||||
- `minute`, `hour` and `day`: requests;
|
||||
- `minute_bytes`, `hour_bytes` and `day_bytes`: bytes.
|
||||
- `minute_bytes`, `hour_bytes` and `day_bytes`: the body bytes
|
||||
`SWW_BYTES_COUNT` names, in that window.
|
||||
|
||||
A count can have a fraction (see "Counting method" in
|
||||
[`SPEC.md`](../SPEC.md)). The bytes of an allowed request are counted when
|
||||
its report comes. So on the `decision` line, the bytes are without this
|
||||
request's.
|
||||
[`SPEC.md`](../SPEC.md)). The bytes counted for an allowed request and its
|
||||
response are added when its report comes. So on the `decision` line, the
|
||||
three byte totals leave them out.
|
||||
|
||||
- `limit_hit`: the limit broken:
|
||||
- `minute`, `hour` or `day` for a rate limit;
|
||||
- `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit;
|
||||
- `error_burst`.
|
||||
|
||||
A request whose bytes broke a limit was passed on whole, so its `action`
|
||||
stays `forward`. Only its `report` line gives that limit.
|
||||
When the bytes counted for a request and its response break a limit, the
|
||||
request has been passed on whole, so its `action` stays `forward`. Only its
|
||||
`report` line gives that limit.
|
||||
|
||||
- `rule_ids`: the rule file rules that matched, in order, up to the one that
|
||||
refused.
|
||||
|
||||
+27
-10
@@ -94,10 +94,12 @@ between the two (see "The image's command" below).
|
||||
- `SWW_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
|
||||
exempt from rate limiting and the byte limits.
|
||||
- `SWW_DENY_NETS` (default empty): netblocks whose clients are refused (see step
|
||||
3 in [`SPEC.md`](../SPEC.md)).
|
||||
4 in [`SPEC.md`](../SPEC.md)).
|
||||
|
||||
## Request rate limits, per client
|
||||
|
||||
The request past one of these limits is refused, and the client is banned.
|
||||
|
||||
- `SWW_RATE_LIMIT_PER_MINUTE` (default `120`, 2 a second): the most requests a
|
||||
client may make in a minute. A count or `off`. The default leaves room for a
|
||||
person opening two pages at once with an empty cache, up to 60 files each.
|
||||
@@ -120,11 +122,22 @@ between the two (see "The image's command" below).
|
||||
|
||||
## Byte limits, per client
|
||||
|
||||
These count body bytes, never headers. What the visitor sent over a WebSocket
|
||||
counts as request bytes, and what the app sent as response bytes. The bytes
|
||||
counted for a request and its response are added to the client's total once the
|
||||
response has ended. The request that takes a client's total past a limit is not
|
||||
cut off; the client is banned.
|
||||
|
||||
- `SWW_BYTES_LIMIT_PER_MINUTE` (default `off`), `SWW_BYTES_LIMIT_PER_HOUR`
|
||||
(default `50G`) and `SWW_BYTES_LIMIT_PER_DAY` (default `100G`): the most bytes
|
||||
counted for a client in each window. A size or `off`.
|
||||
- `SWW_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
|
||||
`response`, `request` or `both`.
|
||||
counted for a client in each window, a size or `off`. `SWW_BYTES_COUNT` says
|
||||
which bytes count: response bodies, request bodies or both. The request that
|
||||
passes the limit is not cut off, and the client is banned when its response
|
||||
ends.
|
||||
- `SWW_BYTES_COUNT` (default `both`): what the byte limits and the byte anomaly
|
||||
thresholds count: `response`, the app's response bodies; `request`, the
|
||||
visitors' request bodies; or `both`. What the visitor sent over a WebSocket
|
||||
counts as request bytes, and what the app sent as response bytes.
|
||||
|
||||
## Lookup of AS number and country
|
||||
|
||||
@@ -181,7 +194,9 @@ that no request can get stops the start. A request can get only these:
|
||||
- a `dnsbl=` tag of a zone in `SWW_DNSBL_ZONES`, in lower case, with a key
|
||||
masked as the `reputation` field shows it;
|
||||
- `abuseipdb`, while `SWW_ABUSEIPDB_KEY` is set;
|
||||
- `exploitsignature`, in `SWW_TAG_DENY` only.
|
||||
- `exploitsignature`, in `SWW_TAG_DENY` only, while `SWW_RULES_ENABLED` is
|
||||
`true`, or while neither `SWW_WAF_MODE` nor `SWW_WAF_ANOMALY_THRESHOLD` is
|
||||
`off`.
|
||||
|
||||
`crowdsec` stops the start in both settings, because a CrowdSec decision refuses
|
||||
the request first.
|
||||
@@ -231,8 +246,9 @@ the request first.
|
||||
a range. So does a directive that removes or changes rules as the file loads,
|
||||
such as `SecRuleRemoveById`, since none of the Core Rule Set's rules is loaded
|
||||
yet. It has no `_FILE` form.
|
||||
- `SWW_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client in
|
||||
a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
|
||||
- `SWW_ERROR_BURST_THRESHOLD` (default `30`): the most refusals a client may get
|
||||
in a minute from a rule file, a trap path or the Core Rule Set, a count or
|
||||
`off`. The next one bans the client (see "Attack detection" in `SPEC.md`).
|
||||
|
||||
## Ban lengths, response and scope
|
||||
|
||||
@@ -309,9 +325,10 @@ the request first.
|
||||
|
||||
## Anomaly thresholds
|
||||
|
||||
Each scope has four thresholds: the most requests and the most bytes counted in
|
||||
a minute and in an hour. Requests are counts, bytes are sizes. Each is `off` by
|
||||
default. The four thresholds per client are:
|
||||
Each scope has four thresholds: the most requests and the most body bytes
|
||||
counted in a minute and in an hour. The bytes are those `SWW_BYTES_COUNT` names.
|
||||
Requests are counts, bytes are sizes. Each is `off` by default. The four
|
||||
thresholds per client are:
|
||||
|
||||
- `SWW_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
|
||||
- `SWW_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
|
||||
|
||||
+1
-1
@@ -61,7 +61,7 @@ since it was first seen:
|
||||
- the first and last time seen;
|
||||
- the AS number, AS name and country last looked up, and when;
|
||||
- total requests, and how many were forwarded and how many refused;
|
||||
- bytes in each direction;
|
||||
- body bytes of its requests, and of the responses it was sent;
|
||||
- responses by status class;
|
||||
- offences by kind: `limit`, `attack`, `rule_blocked` and `waf_blocked`. A file
|
||||
may also hold `token_refused`, which is kept, and which nothing adds to.
|
||||
|
||||
+22
-10
@@ -483,7 +483,8 @@ var (
|
||||
errNoRequestGetsTag = errors.New(
|
||||
"is a tag no request can get: it is not a name in SWW_TAG_LISTS, an " +
|
||||
"asn= or country= tag, a dnsbl= tag of a zone in SWW_DNSBL_ZONES, " +
|
||||
"or abuseipdb with SWW_ABUSEIPDB_KEY set")
|
||||
"abuseipdb with SWW_ABUSEIPDB_KEY set, or exploitsignature with " +
|
||||
"rule files or the Core Rule Set on")
|
||||
)
|
||||
|
||||
// FromEnvironment reads the settings with lookupEnv, normally
|
||||
@@ -1156,11 +1157,16 @@ func checkTag(cfg *Config, tag string) error {
|
||||
// requestCanGetTag reports whether anything cfg sets up can give a request tag.
|
||||
func requestCanGetTag(cfg *Config, tag string) bool {
|
||||
isListName := func(list TagList) bool { return list.Name == tag }
|
||||
if isLookupTag(tag) || tag == ExploitSignatureTag ||
|
||||
slices.ContainsFunc(cfg.TagLists, isListName) {
|
||||
if isLookupTag(tag) || slices.ContainsFunc(cfg.TagLists, isListName) {
|
||||
return true
|
||||
}
|
||||
|
||||
if tag == ExploitSignatureTag {
|
||||
coreRuleSetOn := cfg.WAFMode != WAFModeOff && cfg.WAFAnomalyThreshold != 0
|
||||
|
||||
return cfg.RulesEnabled || coreRuleSetOn
|
||||
}
|
||||
|
||||
if tag == AbuseIPDBTag {
|
||||
return cfg.AbuseIPDBKey != ""
|
||||
}
|
||||
@@ -1964,11 +1970,13 @@ func parseTagLists(value string) ([]TagList, error) {
|
||||
}
|
||||
|
||||
// parseTags reads a comma-separated list of tags, which parseTag reads. A
|
||||
// tag listed twice is an error.
|
||||
// tag listed twice is an error, which shows the tag as MaskZoneKey does.
|
||||
func parseTags(value string) ([]string, error) {
|
||||
items, err := parseList(value)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
// parseList's error, for an empty item, shows the whole value, keys
|
||||
// included.
|
||||
return nil, errEmptyItem
|
||||
}
|
||||
|
||||
tags := make([]string, 0, len(items))
|
||||
@@ -1980,7 +1988,7 @@ func parseTags(value string) ([]string, error) {
|
||||
}
|
||||
|
||||
if slices.Contains(tags, tag) {
|
||||
return nil, fmt.Errorf("%q %w", item, errListedTwice)
|
||||
return nil, fmt.Errorf("%q %w", MaskZoneKey(item), errListedTwice)
|
||||
}
|
||||
|
||||
tags = append(tags, tag)
|
||||
@@ -2025,7 +2033,9 @@ func parseTag(value string) (string, error) {
|
||||
func parseTagPercents(value string) (map[string]int64, error) {
|
||||
items, err := parseList(value)
|
||||
if err != nil {
|
||||
return nil, err
|
||||
// parseList's error, for an empty item, shows the whole value, keys
|
||||
// included.
|
||||
return nil, errEmptyItem
|
||||
}
|
||||
|
||||
percents := make(map[string]int64, len(items))
|
||||
@@ -2033,7 +2043,7 @@ func parseTagPercents(value string) (map[string]int64, error) {
|
||||
for _, item := range items {
|
||||
colon := strings.LastIndex(item, ":")
|
||||
if colon <= 0 {
|
||||
return nil, fmt.Errorf("%q %w", item, errNotTagPercentItem)
|
||||
return nil, fmt.Errorf("%q %w", MaskZoneKey(item), errNotTagPercentItem)
|
||||
}
|
||||
|
||||
tag, err := parseTag(item[:colon])
|
||||
@@ -2043,7 +2053,7 @@ func parseTagPercents(value string) (map[string]int64, error) {
|
||||
|
||||
percent, err := strconv.ParseInt(item[colon+1:], 10, 64)
|
||||
if err != nil || percent < 1 || percent > 99 {
|
||||
return nil, fmt.Errorf("%q %w", item[colon+1:], errNotTagPercent)
|
||||
return nil, fmt.Errorf("%q %w", MaskZoneKey(item[colon+1:]), errNotTagPercent)
|
||||
}
|
||||
|
||||
if tag == ExploitSignatureTag {
|
||||
@@ -2051,7 +2061,9 @@ func parseTagPercents(value string) (map[string]int64, error) {
|
||||
}
|
||||
|
||||
if _, listed := percents[tag]; listed {
|
||||
return nil, fmt.Errorf("%q %w", item[:colon], errListedTwice)
|
||||
// tag, since item[:colon] may end in a space, which MaskZoneKey
|
||||
// would not mask.
|
||||
return nil, fmt.Errorf("%q %w", MaskZoneKey(tag), errListedTwice)
|
||||
}
|
||||
|
||||
percents[tag] = percent
|
||||
|
||||
@@ -208,6 +208,9 @@ const off = "off"
|
||||
// enabled is true, as a setting's value.
|
||||
const enabled = "true"
|
||||
|
||||
// disabled is false, as a setting's value.
|
||||
const disabled = "false"
|
||||
|
||||
// defaultLookupSource is the default of SWW_LOOKUP_SOURCE, and
|
||||
// fileSource the source that is the lookup database.
|
||||
const (
|
||||
@@ -345,7 +348,7 @@ func TestValuesAsSet(t *testing.T) {
|
||||
metricsToken: token,
|
||||
metricsTopN: "10",
|
||||
rulesDir: "/srv/waf-rules",
|
||||
rulesEnabled: "false",
|
||||
rulesEnabled: disabled,
|
||||
})
|
||||
|
||||
wantSettings(t, cfg, config.Config{
|
||||
@@ -1099,7 +1102,7 @@ func TestInstanceNameNotUTF8StopsTheStart(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestCodeOnBothCountryListsStopsTheStart(t *testing.T) {
|
||||
func TestCodeInAllowedCountriesAndTagDenyStopsTheStart(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
_, err := config.FromEnvironment(environment{
|
||||
@@ -1107,7 +1110,7 @@ func TestCodeOnBothCountryListsStopsTheStart(t *testing.T) {
|
||||
allowedCountries: "de,RU",
|
||||
}.lookupEnv)
|
||||
if err == nil {
|
||||
t.Fatal("ru on both country lists was accepted")
|
||||
t.Fatal("ru in SWW_EXCLUSIVELY_ALLOWED_COUNTRIES and SWW_TAG_DENY was accepted")
|
||||
}
|
||||
|
||||
if !strings.HasPrefix(err.Error(), allowedCountries+": ") ||
|
||||
@@ -1357,10 +1360,11 @@ func TestInvalidAnomalySettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// torExitTag and zenTag are tags these tests name.
|
||||
// torExitTag, zenTag and exploitTag are tags these tests name.
|
||||
const (
|
||||
torExitTag = "torexit"
|
||||
zenTag = "dnsbl=zen.example"
|
||||
exploitTag = "exploitsignature"
|
||||
)
|
||||
|
||||
func TestTagSettingsAsSet(t *testing.T) {
|
||||
@@ -1432,7 +1436,15 @@ func TestTagThatNoRequestCanGetIsToldApartFromOneItCan(t *testing.T) {
|
||||
{"crowdsec, though a list has its name", environment{
|
||||
tagLists: "crowdsec=https://example.com/c.txt", tagDeny: "crowdsec",
|
||||
}, `"crowdsec" cannot be used`},
|
||||
{"exploitsignature", environment{tagDeny: "exploitsignature"}, ""},
|
||||
{exploitTag, environment{tagDeny: exploitTag}, ""},
|
||||
{
|
||||
exploitTag + " with rule files off",
|
||||
environment{tagDeny: exploitTag, rulesEnabled: disabled}, "",
|
||||
},
|
||||
{
|
||||
exploitTag + " with the Core Rule Set off",
|
||||
environment{tagDeny: exploitTag, wafMode: off}, "",
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
@@ -1451,6 +1463,39 @@ func TestTagThatNoRequestCanGetIsToldApartFromOneItCan(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestErrorOfATagSettingLeavesOutAZoneKey(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const (
|
||||
tag = "dnsbl=" + spamhaus
|
||||
emptyItem = "has an empty item in its list"
|
||||
twice = `"` + spamhausMasked + `" is listed twice`
|
||||
)
|
||||
|
||||
for _, tc := range []struct{ name, value, want string }{
|
||||
{tagDeny, tag + "," + tag, twice},
|
||||
{tagDeny, tag + ",,", emptyItem},
|
||||
{
|
||||
tagLimitPercent, tag,
|
||||
`"` + spamhausMasked + `" is not a tag, : and a percentage, ` +
|
||||
"such as vpn:50 or asn=14061:25",
|
||||
},
|
||||
{
|
||||
tagLimitPercent, "50:" + tag,
|
||||
`"` + spamhausMasked + `" is not a percentage from 1 to 99`,
|
||||
},
|
||||
{tagLimitPercent, tag + ":50," + tag + " :25", twice},
|
||||
{tagLimitPercent, tag + ":50,,", emptyItem},
|
||||
} {
|
||||
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
|
||||
|
||||
want := tc.name + ": " + tc.want
|
||||
if err == nil || err.Error() != want {
|
||||
t.Errorf("%s=%s gave the error %v, want %s", tc.name, tc.value, err, want)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestTagListURLFetchedAsAnotherListStopsTheStart(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -2141,14 +2186,12 @@ func TestInvalidValueStopsTheStart(t *testing.T) {
|
||||
{countryBytesPercent, "cn"}, {countryBytesPercent, "cn:"},
|
||||
{countryBytesPercent, "cn:25:50"}, {countryBytesPercent, "china:25"},
|
||||
{tagDeny, "a,,b"}, {tagDeny, "country=nk"}, {tagDeny, "asn=abc"},
|
||||
{tagDeny, "asn="}, {tagDeny, "torexit,TOREXIT"},
|
||||
{tagDeny, "torexti"}, {tagDeny, zenTag},
|
||||
{tagDeny, "abuseipdb"}, {tagDeny, "crowdsec"},
|
||||
{tagDeny, "asn="}, {tagDeny, "torexit,TOREXIT"}, {tagDeny, "torexti"},
|
||||
{tagDeny, zenTag}, {tagDeny, "abuseipdb"}, {tagDeny, "crowdsec"},
|
||||
{tagLimitPercent, "torexti:50"}, {tagLimitPercent, "dnsbl=zen.example:50"},
|
||||
{tagLimitPercent, "abuseipdb:50"}, {tagLimitPercent, "crowdsec:50"},
|
||||
{tagLimitPercent, "vpn :50"},
|
||||
{tagLimitPercent, "vpn"}, {tagLimitPercent, ":50"}, {tagLimitPercent, "vpn:"},
|
||||
{tagLimitPercent, "vpn:0"}, {tagLimitPercent, "vpn:100"},
|
||||
{tagLimitPercent, "vpn :50"}, {tagLimitPercent, "vpn"}, {tagLimitPercent, ":50"},
|
||||
{tagLimitPercent, "vpn:"}, {tagLimitPercent, "vpn:0"}, {tagLimitPercent, "vpn:100"},
|
||||
{tagLimitPercent, "vpn:1.5"}, {tagLimitPercent, "vpn:50,VPN:25"},
|
||||
{tagLimitPercent, "exploitsignature:50"}, {tagLimitPercent, "asn=abc:50"},
|
||||
{tagLists, torExitTag}, {tagLists, "=https://example.com/a.txt"},
|
||||
@@ -2167,6 +2210,25 @@ func TestInvalidValueStopsTheStart(t *testing.T) {
|
||||
{logLevel, "INFO"}, {logLevel, "warning"}, {logLevel, "trace"}, {logLevel, ""},
|
||||
{rulesEnabled, "yes"}, {rulesEnabled, "True"},
|
||||
})
|
||||
|
||||
wantExploitTagStopsTheStartWithRulesAndCoreRuleSetOff(t)
|
||||
}
|
||||
|
||||
// wantExploitTagStopsTheStartWithRulesAndCoreRuleSetOff checks that with rule
|
||||
// files off, and either setting turning off the Core Rule Set, the tag
|
||||
// exploitsignature in SWW_TAG_DENY stops the start.
|
||||
func wantExploitTagStopsTheStartWithRulesAndCoreRuleSetOff(t *testing.T) {
|
||||
t.Helper()
|
||||
|
||||
for _, name := range []string{wafMode, wafAnomalyThreshold} {
|
||||
_, err := config.FromEnvironment(environment{
|
||||
tagDeny: exploitTag, rulesEnabled: disabled, name: off,
|
||||
}.lookupEnv)
|
||||
if err == nil || !strings.HasPrefix(err.Error(), tagDeny+": ") {
|
||||
t.Errorf("%s=off with rule files off gave the error %v, want one naming %s",
|
||||
name, err, tagDeny)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestInvalidBanOrStateValueStopsTheStart(t *testing.T) {
|
||||
@@ -2421,7 +2483,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
||||
lookupSource: defaultLookupSource,
|
||||
lookupDBPath: "",
|
||||
lookupTimeout: "800ms",
|
||||
addLookupHeaders: "false",
|
||||
addLookupHeaders: disabled,
|
||||
allowedCountries: "",
|
||||
tagLists: "",
|
||||
tagDeny: "",
|
||||
|
||||
@@ -168,7 +168,7 @@ func TestReputationRefusalNamesItsHeuristicAndWhatListedTheClient(t *testing.T)
|
||||
}
|
||||
}
|
||||
|
||||
func TestBlocklistsComeAfterTheCountryListsAndSkipAllowNets(t *testing.T) {
|
||||
func TestBlocklistsComeAfterAllowedCountriesAndSkipAllowNets(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, server, queue := startWithLookups(t, map[string]string{
|
||||
|
||||
@@ -46,6 +46,22 @@ func TestTagDenyRefusesAListTagBeforeTheRateLimitsCount(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTagDenyRefusesAMixedCaseZoneByItsLowerCaseTag(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const mixedCaseZone = "DNSBL.example"
|
||||
|
||||
// The zone's tag is in lower case, however SWW_DNSBL_ZONES writes it.
|
||||
s, server, _ := startWithLookups(t, map[string]string{
|
||||
dnsblZones: mixedCaseZone, dnsblResolver: noResolver,
|
||||
reputationAction: actionLog, tagDeny: "dnsbl=" + dnsblZone,
|
||||
})
|
||||
loadVerdicts(server, map[string][]string{fromDE: {mixedCaseZone}})
|
||||
|
||||
wantHeuristic(t, s.get(fromDE, http.StatusForbidden, requestlog.ActionDenied),
|
||||
"tag_deny", `{"tag": "dnsbl=`+dnsblZone+`"}`)
|
||||
}
|
||||
|
||||
func TestTagDenyRefusesAnExploitSignatureOfARuleThatOnlyLogs(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
Reference in new issue
Block a user