sneak/smallwebwaf
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
smallwebwaf/EVALUATION.md
T
sneak a0d2c21346 Add README, SPEC and tool evaluation, closes #1 
Initial documents: what smallwebwaf is and why, the proposed feature list, the design spec with the rule file format and the open design questions, and the survey of existing tools.

Model: fable-5-1
2026-09-21 07:42:45 +00:00

266 lines
15 KiB
Markdown
Raw Permalink Blame History

# Evaluation: existing tools for an env-var-configured protective sidecar
Date: 2026-09-21. Method: web survey of project documentation, repositories and
release pages. Nothing was installed or run. Claims marked "unverified" were not
confirmed against primary documentation.
## What was asked for
One container that sits between traefik and one application container,
configured only by environment variables, that provides:
- R1: rate limits per minute, per hour and per day
- R2: netblocks that bypass rate limiting
- R3: alerts on attacks through webhook, Slack or ntfy
- R4: alerts on anomalies: requests or bytes per minute or hour, for given IPs
or netblocks, crossing a threshold
- R5: temporary blocks for abusers, permanent bans for repeat offenders
- R6: one or more RBLs or IP reputation APIs
- R7: AS number lookup
- R8: thresholds biased by AS number or country (for example, listed AS
numbers get 50 percent of the normal limit)
- R9: WAF-style attack detection and prevention
- R10: runs as a plain env-var-configured sidecar between traefik and one app
## Verdict
Nothing existing is suitable. No surveyed tool meets R8 (scaling limits by AS
number or country) or the bytes half of R4 at all, and the tools that come
closest on the rest fail R10: they need several containers, YAML or a web UI.
The recommendation is to build a small single-binary sidecar; `SPEC.md` in this
directory begins that spec. It reuses the mature parts of the surveyed field as
libraries and data sources (Coraza and the OWASP Core Rule Set for attack
detection, DNSBLs, AbuseIPDB, optionally a CrowdSec decision feed) instead of
reimplementing them.
Closest existing options, and why each still falls short:
- BunkerWeb is the closest single product.
- Meets: R1 (rates in requests per second, minute, hour or day), R2
(`LIMIT_IGNORE_IP`, also by AS number and reverse DNS), R3 partly (webhook,
Slack, Discord, Matrix plugins, fired only on denied requests; ntfy only
through the generic webhook, payload format unverified), R5 partly
(`BAD_BEHAVIOR_BAN_TIME`, `0` means permanent; no escalation for repeat
offenders, the ban length is one fixed value), R6 (DNSBL plugin, external
blacklist URLs, optional CrowdSec), R7 partly (AS number used for
blacklist and whitelist decisions), R9 (ModSecurity with the Core Rule
Set, or Coraza plugin), env-var settings.
- Fails: R8 (AS number and country can only allow or deny, never scale a
limit), R4 (no volume or byte threshold alerts in the free edition;
reporting is a paid feature), R5 escalation, and R10 in spirit: since
1.6 it needs a `bunkerweb` container plus a `bw-scheduler` container and
a database, or the all-in-one image that bundles nginx, scheduler, UI and
Redis in one container. It is designed to be the front door for many
sites, not a per-app sidecar. AGPL-3.0.
- Unverified: whether several rates (minute, hour, day) can be stacked on
the same URL pattern; the documented design is one rate per URL pattern.
- CrowdSec is the strongest detection and ban engine, but it is not a proxy.
- Meets: R5 fully (profiles with `duration_expr`, for example four hours
times the number of previous decisions), R3 (notification plugins for
Slack, generic HTTP, email and others; ntfy through the HTTP plugin), R6
(community blocklist, further blocklists, reputation API), R7 (alerts are
enriched with AS number and country), R9 (AppSec component with virtual
patching and ModSecurity-syntax rules), R2 (allowlists).
- Partly: R1 and R8. Detection is by leaky-bucket scenarios over logs, and
a scenario can filter on AS number or country, so a stricter bucket for
listed AS numbers is possible, but each is a hand-written YAML scenario,
it reacts after the fact by banning, and it is not an inline limiter that
answers 429.
- Fails: R10 (needs the security engine container with persistent state,
log acquisition from traefik, a bouncer such as the traefik plugin, and
YAML for acquisition, profiles, scenarios and notifications), bytes half
of R4.
- CrowdSec plus traefik's own `rateLimit` middleware is the best combination
with no new code. It gives inline limiting (one window per middleware, keyed
by IP, with `sourceCriterion` exclusions), bans with escalation, alerts and
reputation. It still fails R8, the bytes half of R4, R1's three stacked
windows only by chaining three middlewares per router, and R10 entirely: the
configuration lives in traefik labels and CrowdSec YAML on each host, not in
one sidecar's environment.
## Candidate by candidate
- CrowdSec with the traefik bouncer plugin
(`maxlerebourg/crowdsec-bouncer-traefik-plugin`)
- What it is: a traefik middleware plugin that asks a CrowdSec local API
whether the client IP is banned, and can forward each request to the
CrowdSec AppSec component for a WAF verdict. Modes: query per request,
cached, streamed decision list (recommended), standalone against the
central API, or AppSec only. Supports captcha remediation.
- Covers: R5, R6, R9, R3 and R7 through the engine (see Verdict).
- Misses: the plugin itself does no rate limiting; R8; R4 bytes.
- Configuration: traefik static config to load the plugin, dynamic config
or labels for the middleware, CrowdSec YAML for everything else.
- Sidecar fit: no. It lives inside traefik, plus a separate engine
container.
- Maturity: widely used (about 900 stars, listed in the traefik plugin
catalog, documented by CrowdSec itself), actively maintained. Traefik
plugins run in an interpreter inside traefik, which costs some
per-request time.
- CrowdSec generic bouncers (nginx, Caddy, firewall)
- Same engine, different enforcement point. The firewall bouncer blocks at
nftables level on the host, which is cheap and covers every service on
the host at once; worth considering fleet-wide regardless of this
project. Not a sidecar, same misses as above.
- BunkerWeb 1.6.14 (`bunkerity/bunkerweb`)
- What it is: nginx with Lua plugins, ModSecurity and the Core Rule Set,
configured by settings that are passed as env vars to its scheduler
container. Documents running behind traefik with `USE_REAL_IP` and
`REAL_IP_FROM`.
- Coverage and misses: see Verdict. Also has bot challenges (cookie,
JavaScript, captcha), country allow and deny, Tor and user-agent lists.
- Sidecar fit: partial. Env vars yes; single small container no.
- Maturity: mature, frequent releases, commercial company behind it with a
paid edition; free plugins are AGPL-3.0.
- SafeLine (`chaitin/SafeLine`)
- What it is: nginx-derived (Tengine) reverse proxy with a proprietary
semantic detection engine and a web console. About 22,000 stars.
- Covers: R9 well, basic per-IP rate limiting with block duration, bot
challenge, authentication gate.
- Misses: R8, R7, R4; R1 has one window per rule. Country blocking, alert
messages, syslog forwarding and the better reputation database are paid
(Lite about 10 USD per month, Pro about 100 USD per month); the free
edition is capped at 10 applications.
- Configuration: web UI backed by PostgreSQL. No env-var configuration.
- Sidecar fit: no. Seven containers (postgres, management, detector,
tengine, and three helpers); the proxy container uses host networking.
The detection engine is closed source.
- Maturity: very active, vendor-driven.
- Coraza (`corazawaf/coraza`) and Coraza-based proxies
- What it is: a Go library that implements the ModSecurity rule language
and runs the OWASP Core Rule Set. OWASP project, actively maintained, the
successor path now that ModSecurity is in maintenance only.
- Packagings: `coraza-caddy` (Caddy module), `coraza-spoa` (HAProxy),
`coraza-proxy-wasm` (Envoy), a traefik WASM plugin, and
`coreruleset/coraza-crs-docker` (Caddy plus Coraza plus the Core Rule
Set, with env vars for backend address, engine mode and rule-set
tuning).
- Covers: R9 only. `coraza-crs-docker` fits R10 well: one container, env
vars, backend address.
- Misses: R1 to R8. ModSecurity-language rules can count requests per IP
in a persistent collection, but Coraza's support for persistent
collections is limited and this is not a practical rate limiter.
- Value here: the right library to embed for R9 in a purpose-built sidecar.
- ModSecurity Core Rule Set containers (`owasp/modsecurity-crs`)
- What it is: official images of Apache or nginx with ModSecurity and the
Core Rule Set, as a reverse proxy configured by env vars (`BACKEND`,
paranoia level, anomaly thresholds, many more). Current images carry
ModSecurity 2.9.x on Apache and Core Rule Set 4.x.
- Covers: R9, and R10 (single container, env vars, one backend).
- Misses: R1 to R8.
- Maturity: rule set is very actively maintained; the ModSecurity engine
itself is in maintenance under OWASP after Trustwave ended support in
2024.
- Anubis (`TecharoHQ/anubis`), 1.27 current
- What it is: a single-binary reverse proxy that makes browsers solve a
proof-of-work challenge before passing them to `TARGET`. Aimed at
scrapers, which is likely a large share of unwanted traffic on a public
gitea.
- Covers: R10 well (one container, env vars for listener, target,
difficulty, cookies). Policy rules can match on path, user agent,
headers, IP ranges, and with the vendor's hosted data service also AS
number and country, and can weigh a request toward a harder challenge.
- Misses: R1, R3, R4, R5, R6, R9. Bot policy needs a YAML file, not env
vars. AS number and country matching depend on the vendor's hosted
service. Breaks non-browser clients unless paths are exempted; for
gitea, git-over-HTTP and API paths must be allowed through by rule.
- Maturity: very active, widely deployed on code forges since 2025.
- Value here: complementary. It can be chained (traefik, then the sidecar,
then Anubis, then the app) if challenge pages are wanted.
- go-away (`git.gammaspectra.live/git/go-away`)
- What it is: a single Go reverse proxy with a rule language over request
properties, several challenge types (including ones that need no
JavaScript), built-in DNSBL check (default `dnsbl.dronebl.org`), network
range lists, and metrics by network range, AS and user agent.
- Covers: R6 partly, R7 partly (through range lists rather than a lookup
database; unverified), bot filtering.
- Misses: R1, R3, R4, R5, R8, R9. YAML policy configuration. Small
maintainer base.
- fail2ban-style traefik plugins
- `tomMoulard/fail2ban`: watches request rate or response status per IP
inside traefik and bans for a fixed time. Covers a slice of R5 (fixed
temporary ban, no escalation, state lost on traefik restart) and R2
(allowlist). Nothing else.
- `juitde/traefik-plugin-fail2ban`, fail2ban connector plugins, GeoBlock
plugins (country allow or deny through an external lookup API): same
shape, each one small feature, configured in traefik, state in memory.
- Sidecar fit: no; they live inside traefik.
- Traefik built-in middlewares
- `rateLimit` (one average-and-burst window per middleware, optional Redis
in traefik 3.x), `inFlightReq`, `ipAllowList`. No bans, alerts,
reputation or AS number awareness.
- caddy-waf (`fabriziosalmi/caddy-waf`), 0.4.x
- What it is: a Caddy module with regex rules and anomaly scoring, per-IP
and per-path rate limiting with one configurable window, IP and DNS
blacklists, Tor exit list fetch, country and AS number allow or deny
from MaxMind databases.
- Misses: R8 (allow or deny only), R3, R4, R5 (no documented ban state or
alerting), R1's three windows. Caddyfile configuration. One maintainer,
pre-1.0, AGPL-3.0.
- open-appsec (Check Point)
- Machine-learning WAF agent attached to nginx, Kong, Envoy or similar,
with a declarative policy file or the vendor's cloud console. Covers R9
only; rate limiting and richer features are in paid tiers. Not a sidecar
in the required sense.
- iocaine
- Serves generated garbage pages to clients the fronting proxy classifies
as scrapers. Not a limiter, WAF or ban tool; out of scope except as a
curiosity for scraper traffic.
- Pangolin
- A tunnelled access platform that bundles traefik and optionally
CrowdSec. Replaces the ingress rather than adding a sidecar; out of
scope.
## Requirement by requirement, across the field
- R1 three windows: BunkerWeb (one rate per URL pattern; stacking unverified).
Everyone else offers one window per rule or none.
- R2 bypass netblocks: BunkerWeb, CrowdSec, traefik, fail2ban plugins.
- R3 alerts: CrowdSec (best), BunkerWeb (denied requests only), SafeLine (paid).
- R4 volume and byte anomalies: none. CrowdSec can approximate request-count
anomalies with a custom scenario; nothing handles bytes.
- R5 temporary then permanent: CrowdSec only. BunkerWeb has fixed or permanent,
not both by history.
- R6 reputation: CrowdSec, BunkerWeb, go-away, caddy-waf (static lists).
- R7 AS number lookup: CrowdSec (enrichment), BunkerWeb and caddy-waf (allow or
deny), Anubis (hosted service).
- R8 biased thresholds: none.
- R9 attack detection: Coraza or ModSecurity with the Core Rule Set (open,
standard), CrowdSec AppSec, SafeLine (closed engine), BunkerWeb (wraps the
first).
- R10 env-var sidecar: `owasp/modsecurity-crs`, `coraza-crs-docker`, Anubis.
None of these covers anything beyond its one job.
## Not checked
- No tool was installed, load-tested or run against real fleet traffic.
- Licence terms of reputation data sources for this use (AbuseIPDB free tier,
Spamhaus query policy, IPinfo Lite attribution) were not read in full.
- How upaas describes a second container for a deployed service was not
examined; no repo was opened for this task.
- Paid editions (BunkerWeb PRO, SafeLine Pro, CrowdSec paid blocklists) were
assessed from public feature lists only.
- The owner's Go dependency defaults file was not found at the documented path,
so library choices in `SPEC.md` are proposals to be checked against it.
## Sources
- https://docs.bunkerweb.io/latest/features/
- https://docs.bunkerweb.io/latest/integrations/
- https://github.com/bunkerity/bunkerweb-plugins
- https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
- https://docs.crowdsec.net/u/bouncers/traefik/
- https://docs.crowdsec.net/docs/next/local_api/profiles/format/
- https://docs.crowdsec.net/u/user_guides/waf_rp_howto/
- https://www.crowdsec.net/blog/waf-traefik-crowdsec
- https://github.com/chaitin/SafeLine
- https://dev.to/carrie_luo1/differences-between-safeline-waf-free-and-safeline-waf-pro-2dak
- https://github.com/coreruleset/coraza-crs-docker
- https://github.com/coreruleset/modsecurity-crs-docker
- https://github.com/TecharoHQ/anubis
- https://git.gammaspectra.live/git/go-away
- https://github.com/fabriziosalmi/caddy-waf
- https://plugins.traefik.io/plugins/628c9ebcffc0cd18356a979f/fail2-ban
- https://github.com/juitde/traefik-plugin-fail2ban
- https://docs.openappsec.io/
Reference in New Issue View Git Blame Copy Permalink