smallwebwaf/EVALUATION.md

# Evaluation: existing tools for an env-var-configured protective sidecar

Date: 2026-09-21. Method: web survey of project documentation, repositories and
release pages. Nothing was installed or run. Claims marked "unverified" were not
confirmed against primary documentation.

## What was asked for

One container that sits between traefik and one application container,
configured only by environment variables, that provides:

- R1: rate limits per minute, per hour and per day
- R2: netblocks that bypass rate limiting
- R3: alerts on attacks through webhook, Slack or ntfy
- R4: alerts on anomalies: requests made or bytes transferred per minute or
  hour, for given IPs or netblocks, crossing a threshold
- R5: temporary blocks for abusers, permanent bans for repeat offenders
- R6: one or more RBLs or IP reputation APIs
- R7: AS number lookup
- R8: thresholds biased by AS number or country (for example, listed AS numbers
  get 50 percent of the normal limit)
- R9: WAF-style attack detection and prevention
- R10: runs as a plain env-var-configured sidecar between traefik and one app

## Verdict

Nothing existing is suitable. No surveyed tool meets R8 (scaling limits by AS
number or country) or R4's alerts on bytes transferred, and the tools that come
closest on the rest fail R10: they need several containers, YAML or a web UI.
The recommendation is to build a small single-binary sidecar; `SPEC.md` in this
directory begins that spec. The sidecar reuses the mature parts of the surveyed
field as libraries and data sources (Coraza and the OWASP Core Rule Set for
attack detection, DNSBLs, AbuseIPDB, optionally a CrowdSec decision feed)
instead of reimplementing them.

Closest existing options, and why each still falls short:

- BunkerWeb is the closest single product.
    - Meets: R1 (rates in requests per second, minute, hour or day), R2
      (`LIMIT_IGNORE_IP`, also by AS number and reverse DNS), R3 partly
      (webhook, Slack, Discord, Matrix plugins, fired only on denied requests;
      ntfy only through the generic webhook, payload format unverified), R5
      partly (`BAD_BEHAVIOR_BAN_TIME`, `0` means permanent; no escalation for
      repeat offenders, the ban length is one fixed value), R6 (DNSBL plugin,
      external blacklist URLs, optional CrowdSec), R7 partly (AS number used for
      blacklist and whitelist decisions), R9 (ModSecurity with the Core Rule
      Set, or Coraza plugin), env-var settings.
    - Fails: R8 (AS number and country can only allow or deny, never scale a
      limit), R4 (no alerts on request-count or byte thresholds in the free
      edition; reporting is a paid feature), R5 escalation, and R10's plain
      sidecar: since 1.6 it needs a `bunkerweb` container plus a `bw-scheduler`
      container and a database, or the all-in-one image that bundles nginx,
      scheduler, UI and Redis in one container. It is designed as the proxy in
      front of many sites, not as a sidecar for one app. AGPL-3.0.
    - Unverified: whether several rates (minute, hour, day) can be stacked on
      the same URL pattern; the documented design is one rate per URL pattern.
- CrowdSec is the strongest detection and ban engine, but it is not a proxy.
    - Meets: R5 fully (profiles with `duration_expr`, for example four hours
      times the number of previous decisions), R3 (notification plugins for
      Slack, generic HTTP, email and others; ntfy through the HTTP plugin), R6
      (community blocklist, further blocklists, reputation API), R7 (alerts are
      enriched with AS number and country), R9 (AppSec component with virtual
      patching and ModSecurity-syntax rules), R2 (allowlists).
    - Partly: R1 and R8. Detection is by leaky-bucket scenarios over logs, and a
      scenario can filter on AS number or country, so a stricter bucket for
      listed AS numbers is possible, but each is a hand-written YAML scenario,
      it reacts after the fact by banning, and it is not an inline limiter that
      answers 429.
    - Fails: R10 (needs the security engine container with persistent state, log
      acquisition from traefik, a bouncer such as the traefik plugin, and YAML
      for acquisition, profiles, scenarios and notifications), R4's alerts on
      bytes.
- CrowdSec plus traefik's own `rateLimit` middleware is the best combination
  with no new code. It gives inline limiting (one window per middleware, keyed
  by IP, with `sourceCriterion` exclusions), bans with escalation, alerts and
  reputation lookups. It still fails R8, R4's alerts on bytes, and R10 entirely:
  the configuration lives in traefik labels and CrowdSec YAML on each host, not
  in one sidecar's environment. It meets R1's three windows only by chaining
  three middlewares per router.

## Candidate by candidate

- CrowdSec with the traefik bouncer plugin
  (`maxlerebourg/crowdsec-bouncer-traefik-plugin`)
    - What it is: a traefik middleware plugin that asks a CrowdSec local API
      whether the client IP is banned, and can forward each request to the
      CrowdSec AppSec component for a WAF verdict. Modes: query per request,
      cached, streamed decision list (recommended), standalone against the
      central API, or AppSec only. Supports captcha remediation.
    - Covers: R5, R6, R9, R3 and R7 through the engine (see Verdict).
    - Misses: the plugin itself does no rate limiting; R8; R4's alerts on bytes.
    - Configuration: traefik static config to load the plugin, dynamic config or
      labels for the middleware, CrowdSec YAML for everything else.
    - Sidecar fit: no. It runs inside traefik and needs a separate engine
      container.
    - Maturity: widely used (about 900 stars, listed in the traefik plugin
      catalog, documented by CrowdSec itself), actively maintained. Traefik
      plugins run in an interpreter inside traefik, which costs some per-request
      time.
- CrowdSec generic bouncers (nginx, Caddy, firewall)
    - Same engine, different enforcement point. The firewall bouncer blocks at
      nftables level on the host, which is cheap and covers every service on the
      host at once; worth considering fleet-wide regardless of this project. Not
      a sidecar, same misses as above.
- BunkerWeb 1.6.14 (`bunkerity/bunkerweb`)
    - What it is: nginx with Lua plugins, ModSecurity and the Core Rule Set,
      configured by settings that are passed as env vars to its scheduler
      container. Documents running behind traefik with `USE_REAL_IP` and
      `REAL_IP_FROM`.
    - Coverage and misses: see Verdict. Also has bot challenges (cookie,
      JavaScript, captcha), country allow and deny, Tor and user-agent lists.
    - Sidecar fit: partial. Env vars yes; single small container no.
    - Maturity: mature, frequent releases, commercial company behind it with a
      paid edition; free plugins are AGPL-3.0.
- SafeLine (`chaitin/SafeLine`)
    - What it is: nginx-derived (Tengine) reverse proxy with a proprietary
      semantic detection engine and a web console. About 22,000 stars.
    - Covers: R9 well, basic per-IP rate limiting with block duration, bot
      challenge, authentication in front of the app.
    - Misses: R8, R7, R4; R1 has one window per rule. Country blocking, alert
      messages, syslog forwarding and the better reputation database are paid
      (Lite about 10 USD per month, Pro about 100 USD per month); the free
      edition is capped at 10 applications.
    - Configuration: web UI backed by PostgreSQL. No env-var configuration.
    - Sidecar fit: no. Seven containers (postgres, management, detector,
      tengine, and three helpers); the proxy container uses host networking. The
      detection engine is closed source.
    - Maturity: very active, vendor-driven.
- Coraza (`corazawaf/coraza`) and Coraza-based proxies
    - What it is: a Go library that implements the ModSecurity rule language and
      runs the OWASP Core Rule Set. OWASP project, actively maintained, the
      successor to ModSecurity, which is now in maintenance only.
    - Packagings: `coraza-caddy` (Caddy module), `coraza-spoa` (HAProxy),
      `coraza-proxy-wasm` (Envoy), a traefik WASM plugin, and
      `coreruleset/coraza-crs-docker` (Caddy plus Coraza plus the Core Rule Set,
      with env vars for backend address, engine mode and rule-set tuning).
    - Covers: R9 only. `coraza-crs-docker` fits R10 well: one container, env
      vars, backend address.
    - Misses: R1 to R8. ModSecurity-language rules can count requests per IP in
      a persistent collection, but Coraza's support for persistent collections
      is limited and this is not a practical rate limiter.
    - Value here: the right library to embed for R9 in a purpose-built sidecar.
- ModSecurity Core Rule Set containers (`owasp/modsecurity-crs`)
    - What it is: official images of Apache or nginx with ModSecurity and the
      Core Rule Set, as a reverse proxy configured by env vars (`BACKEND`,
      paranoia level, anomaly thresholds, many more). Current images contain
      ModSecurity 2.9.x on Apache and Core Rule Set 4.x.
    - Covers: R9, and R10 (single container, env vars, one backend).
    - Misses: R1 to R8.
    - Maturity: rule set is very actively maintained; the ModSecurity engine
      itself is in maintenance under OWASP after Trustwave ended support
      in 2024.
- Anubis (`TecharoHQ/anubis`), 1.27 current
    - What it is: a single-binary reverse proxy that makes browsers solve a
      proof-of-work challenge before passing them to `TARGET`. Aimed at
      scrapers, which are likely a large share of the unwanted traffic on a
      public code forge.
    - Covers: R10 well (one container, env vars for listener, target,
      difficulty, cookies). Policy rules can match on path, user agent, headers,
      IP ranges, and with the vendor's hosted data service also AS number and
      country, and can weigh a request toward a harder challenge.
    - Misses: R1, R3, R4, R5, R6, R9. Bot policy needs a YAML file, not env
      vars. AS number and country matching depend on the vendor's hosted
      service. Breaks non-browser clients unless paths are exempted; for a code
      forge, git-over-HTTP and API paths must be allowed through by rule.
    - Maturity: very active, widely deployed on code forges since 2025.
    - Value here: complementary. It can be chained (traefik, then the sidecar,
      then Anubis, then the app) if challenge pages are wanted.
- go-away (`git.gammaspectra.live/git/go-away`)
    - What it is: a single Go reverse proxy with a rule language over request
      properties, several challenge types (including ones that need no
      JavaScript), built-in DNSBL check (default `dnsbl.dronebl.org`), network
      range lists, and metrics by network range, AS and user agent.
    - Covers: R6 partly, R7 partly (through range lists rather than a lookup
      database; unverified), bot filtering.
    - Misses: R1, R3, R4, R5, R8, R9. YAML policy configuration. Small
      maintainer base.
- fail2ban-style traefik plugins
    - `tomMoulard/fail2ban`: watches request rate or response status per IP
      inside traefik and bans for a fixed time. Covers a slice of R5 (fixed
      temporary ban, no escalation, state lost on traefik restart) and R2
      (allowlist). Nothing else.
    - `juitde/traefik-plugin-fail2ban`, fail2ban connector plugins, GeoBlock
      plugins (country allow or deny through an external lookup API): each adds
      one small feature, is configured in traefik and keeps its state in memory.
    - Sidecar fit: no; they live inside traefik.
- Traefik built-in middlewares
    - `rateLimit` (one average-and-burst window per middleware, optional Redis
      in traefik 3.x), `inFlightReq`, `ipAllowList`. No bans, alerts, reputation
      lookups or AS number lookups.
- caddy-waf (`fabriziosalmi/caddy-waf`), 0.4.x
    - What it is: a Caddy module with regex rules and anomaly scoring, per-IP
      and per-path rate limiting with one configurable window, IP and DNS
      blacklists, Tor exit list fetch, country and AS number allow or deny from
      MaxMind databases.
    - Misses: R8 (allow or deny only), R3, R4, R5 (no documented ban state or
      alerting), R1's three windows. Caddyfile configuration. One maintainer,
      pre-1.0, AGPL-3.0.
- open-appsec (Check Point)
    - Machine-learning WAF agent attached to nginx, Kong, Envoy or similar, with
      a declarative policy file or the vendor's cloud console. Covers R9 only;
      rate limiting and richer features are in paid tiers. It attaches to an
      existing proxy, so it is not the sidecar R10 asks for.
- iocaine
    - Serves generated garbage pages to clients the fronting proxy classifies as
      scrapers. Not a limiter, WAF or ban tool; out of scope, and only of
      passing interest for scraper traffic.
- Pangolin
    - A tunnelled access platform that bundles traefik and optionally CrowdSec.
      Replaces the ingress rather than adding a sidecar; out of scope.

## Requirement by requirement, across the field

- R1 three windows: BunkerWeb (one rate per URL pattern; stacking unverified).
  Everyone else offers one window per rule or none.
- R2 bypass netblocks: BunkerWeb, CrowdSec, traefik, fail2ban plugins.
- R3 alerts: CrowdSec (best), BunkerWeb (denied requests only), SafeLine (paid).
- R4 alerts on request counts and bytes: none. CrowdSec can approximate alerts
  on request counts with a custom scenario; no surveyed tool alerts on bytes.
- R5 temporary then permanent: CrowdSec only. BunkerWeb's ban is either one
  fixed length or permanent, and does not change with a client's history.
- R6 reputation: CrowdSec, BunkerWeb, go-away, caddy-waf (static lists).
- R7 AS number lookup: CrowdSec (enrichment), BunkerWeb and caddy-waf (allow or
  deny), Anubis (hosted service).
- R8 biased thresholds: none.
- R9 attack detection: Coraza or ModSecurity with the Core Rule Set (open,
  standard), CrowdSec AppSec, SafeLine (closed engine), BunkerWeb (wraps Coraza
  or ModSecurity).
- R10 env-var sidecar: `owasp/modsecurity-crs`, `coraza-crs-docker`, Anubis.
  None of these covers anything beyond its one job.

## Not checked

- No tool was installed, load-tested or run against real fleet traffic.
- Licence terms of reputation data sources for this use (AbuseIPDB free tier,
  Spamhaus query policy, IPinfo Lite attribution) were not read in full.
- How upaas describes a second container for a deployed service was not
  examined; no repo was opened for this task.
- Paid editions (BunkerWeb PRO, SafeLine Pro, CrowdSec paid blocklists) were
  assessed from public feature lists only.
- The owner's Go dependency defaults file was not found at the documented path,
  so library choices in `SPEC.md` are proposals to be checked against it.

## Sources

- https://docs.bunkerweb.io/latest/features/
- https://docs.bunkerweb.io/latest/integrations/
- https://github.com/bunkerity/bunkerweb-plugins
- https://github.com/maxlerebourg/crowdsec-bouncer-traefik-plugin
- https://docs.crowdsec.net/u/bouncers/traefik/
- https://docs.crowdsec.net/docs/next/local_api/profiles/format/
- https://docs.crowdsec.net/u/user_guides/waf_rp_howto/
- https://www.crowdsec.net/blog/waf-traefik-crowdsec
- https://github.com/chaitin/SafeLine
- https://dev.to/carrie_luo1/differences-between-safeline-waf-free-and-safeline-waf-pro-2dak
- https://github.com/coreruleset/coraza-crs-docker
- https://github.com/coreruleset/modsecurity-crs-docker
- https://github.com/TecharoHQ/anubis
- https://git.gammaspectra.live/git/go-away
- https://github.com/fabriziosalmi/caddy-waf
- https://plugins.traefik.io/plugins/628c9ebcffc0cd18356a979f/fail2-ban
- https://github.com/juitde/traefik-plugin-fail2ban
- https://docs.openappsec.io/