sneak/smallwebwaf
1
0
Fork 0
Code Issues 4 Pull Requests Actions Packages Projects Releases Wiki Activity
Files
main
smallwebwaf/EVALUATION.md
T
sneak a0d2c21346 Add README, SPEC and tool evaluation, closes #1 
Initial documents: what smallwebwaf is and why, the proposed feature list, the design spec with the rule file format and the open design questions, and the survey of existing tools.

Model: fable-5-1
2026-09-21 07:42:45 +00:00

15 KiB
Raw Permalink Blame History

Evaluation: existing tools for an env-var-configured protective sidecar

Date: 2026-09-21. Method: web survey of project documentation, repositories and release pages. Nothing was installed or run. Claims marked "unverified" were not confirmed against primary documentation.

What was asked for

One container that sits between traefik and one application container, configured only by environment variables, that provides:

  • R1: rate limits per minute, per hour and per day
  • R2: netblocks that bypass rate limiting
  • R3: alerts on attacks through webhook, Slack or ntfy
  • R4: alerts on anomalies: requests or bytes per minute or hour, for given IPs or netblocks, crossing a threshold
  • R5: temporary blocks for abusers, permanent bans for repeat offenders
  • R6: one or more RBLs or IP reputation APIs
  • R7: AS number lookup
  • R8: thresholds biased by AS number or country (for example, listed AS numbers get 50 percent of the normal limit)
  • R9: WAF-style attack detection and prevention
  • R10: runs as a plain env-var-configured sidecar between traefik and one app

Verdict

Nothing existing is suitable. No surveyed tool meets R8 (scaling limits by AS number or country) or the bytes half of R4 at all, and the tools that come closest on the rest fail R10: they need several containers, YAML or a web UI. The recommendation is to build a small single-binary sidecar; SPEC.md in this directory begins that spec. It reuses the mature parts of the surveyed field as libraries and data sources (Coraza and the OWASP Core Rule Set for attack detection, DNSBLs, AbuseIPDB, optionally a CrowdSec decision feed) instead of reimplementing them.

Closest existing options, and why each still falls short:

  • BunkerWeb is the closest single product.
    • Meets: R1 (rates in requests per second, minute, hour or day), R2 (LIMIT_IGNORE_IP, also by AS number and reverse DNS), R3 partly (webhook, Slack, Discord, Matrix plugins, fired only on denied requests; ntfy only through the generic webhook, payload format unverified), R5 partly (BAD_BEHAVIOR_BAN_TIME, 0 means permanent; no escalation for repeat offenders, the ban length is one fixed value), R6 (DNSBL plugin, external blacklist URLs, optional CrowdSec), R7 partly (AS number used for blacklist and whitelist decisions), R9 (ModSecurity with the Core Rule Set, or Coraza plugin), env-var settings.
    • Fails: R8 (AS number and country can only allow or deny, never scale a limit), R4 (no volume or byte threshold alerts in the free edition; reporting is a paid feature), R5 escalation, and R10 in spirit: since 1.6 it needs a bunkerweb container plus a bw-scheduler container and a database, or the all-in-one image that bundles nginx, scheduler, UI and Redis in one container. It is designed to be the front door for many sites, not a per-app sidecar. AGPL-3.0.
    • Unverified: whether several rates (minute, hour, day) can be stacked on the same URL pattern; the documented design is one rate per URL pattern.
  • CrowdSec is the strongest detection and ban engine, but it is not a proxy.
    • Meets: R5 fully (profiles with duration_expr, for example four hours times the number of previous decisions), R3 (notification plugins for Slack, generic HTTP, email and others; ntfy through the HTTP plugin), R6 (community blocklist, further blocklists, reputation API), R7 (alerts are enriched with AS number and country), R9 (AppSec component with virtual patching and ModSecurity-syntax rules), R2 (allowlists).
    • Partly: R1 and R8. Detection is by leaky-bucket scenarios over logs, and a scenario can filter on AS number or country, so a stricter bucket for listed AS numbers is possible, but each is a hand-written YAML scenario, it reacts after the fact by banning, and it is not an inline limiter that answers 429.
    • Fails: R10 (needs the security engine container with persistent state, log acquisition from traefik, a bouncer such as the traefik plugin, and YAML for acquisition, profiles, scenarios and notifications), bytes half of R4.
  • CrowdSec plus traefik's own rateLimit middleware is the best combination with no new code. It gives inline limiting (one window per middleware, keyed by IP, with sourceCriterion exclusions), bans with escalation, alerts and reputation. It still fails R8, the bytes half of R4, R1's three stacked windows only by chaining three middlewares per router, and R10 entirely: the configuration lives in traefik labels and CrowdSec YAML on each host, not in one sidecar's environment.

Candidate by candidate

  • CrowdSec with the traefik bouncer plugin (maxlerebourg/crowdsec-bouncer-traefik-plugin)
    • What it is: a traefik middleware plugin that asks a CrowdSec local API whether the client IP is banned, and can forward each request to the CrowdSec AppSec component for a WAF verdict. Modes: query per request, cached, streamed decision list (recommended), standalone against the central API, or AppSec only. Supports captcha remediation.
    • Covers: R5, R6, R9, R3 and R7 through the engine (see Verdict).
    • Misses: the plugin itself does no rate limiting; R8; R4 bytes.
    • Configuration: traefik static config to load the plugin, dynamic config or labels for the middleware, CrowdSec YAML for everything else.
    • Sidecar fit: no. It lives inside traefik, plus a separate engine container.
    • Maturity: widely used (about 900 stars, listed in the traefik plugin catalog, documented by CrowdSec itself), actively maintained. Traefik plugins run in an interpreter inside traefik, which costs some per-request time.
  • CrowdSec generic bouncers (nginx, Caddy, firewall)
    • Same engine, different enforcement point. The firewall bouncer blocks at nftables level on the host, which is cheap and covers every service on the host at once; worth considering fleet-wide regardless of this project. Not a sidecar, same misses as above.
  • BunkerWeb 1.6.14 (bunkerity/bunkerweb)
    • What it is: nginx with Lua plugins, ModSecurity and the Core Rule Set, configured by settings that are passed as env vars to its scheduler container. Documents running behind traefik with USE_REAL_IP and REAL_IP_FROM.
    • Coverage and misses: see Verdict. Also has bot challenges (cookie, JavaScript, captcha), country allow and deny, Tor and user-agent lists.
    • Sidecar fit: partial. Env vars yes; single small container no.
    • Maturity: mature, frequent releases, commercial company behind it with a paid edition; free plugins are AGPL-3.0.
  • SafeLine (chaitin/SafeLine)
    • What it is: nginx-derived (Tengine) reverse proxy with a proprietary semantic detection engine and a web console. About 22,000 stars.
    • Covers: R9 well, basic per-IP rate limiting with block duration, bot challenge, authentication gate.
    • Misses: R8, R7, R4; R1 has one window per rule. Country blocking, alert messages, syslog forwarding and the better reputation database are paid (Lite about 10 USD per month, Pro about 100 USD per month); the free edition is capped at 10 applications.
    • Configuration: web UI backed by PostgreSQL. No env-var configuration.
    • Sidecar fit: no. Seven containers (postgres, management, detector, tengine, and three helpers); the proxy container uses host networking. The detection engine is closed source.
    • Maturity: very active, vendor-driven.
  • Coraza (corazawaf/coraza) and Coraza-based proxies
    • What it is: a Go library that implements the ModSecurity rule language and runs the OWASP Core Rule Set. OWASP project, actively maintained, the successor path now that ModSecurity is in maintenance only.
    • Packagings: coraza-caddy (Caddy module), coraza-spoa (HAProxy), coraza-proxy-wasm (Envoy), a traefik WASM plugin, and coreruleset/coraza-crs-docker (Caddy plus Coraza plus the Core Rule Set, with env vars for backend address, engine mode and rule-set tuning).
    • Covers: R9 only. coraza-crs-docker fits R10 well: one container, env vars, backend address.
    • Misses: R1 to R8. ModSecurity-language rules can count requests per IP in a persistent collection, but Coraza's support for persistent collections is limited and this is not a practical rate limiter.
    • Value here: the right library to embed for R9 in a purpose-built sidecar.
  • ModSecurity Core Rule Set containers (owasp/modsecurity-crs)
    • What it is: official images of Apache or nginx with ModSecurity and the Core Rule Set, as a reverse proxy configured by env vars (BACKEND, paranoia level, anomaly thresholds, many more). Current images carry ModSecurity 2.9.x on Apache and Core Rule Set 4.x.
    • Covers: R9, and R10 (single container, env vars, one backend).
    • Misses: R1 to R8.
    • Maturity: rule set is very actively maintained; the ModSecurity engine itself is in maintenance under OWASP after Trustwave ended support in 2024.
  • Anubis (TecharoHQ/anubis), 1.27 current
    • What it is: a single-binary reverse proxy that makes browsers solve a proof-of-work challenge before passing them to TARGET. Aimed at scrapers, which is likely a large share of unwanted traffic on a public gitea.
    • Covers: R10 well (one container, env vars for listener, target, difficulty, cookies). Policy rules can match on path, user agent, headers, IP ranges, and with the vendor's hosted data service also AS number and country, and can weigh a request toward a harder challenge.
    • Misses: R1, R3, R4, R5, R6, R9. Bot policy needs a YAML file, not env vars. AS number and country matching depend on the vendor's hosted service. Breaks non-browser clients unless paths are exempted; for gitea, git-over-HTTP and API paths must be allowed through by rule.
    • Maturity: very active, widely deployed on code forges since 2025.
    • Value here: complementary. It can be chained (traefik, then the sidecar, then Anubis, then the app) if challenge pages are wanted.
  • go-away (git.gammaspectra.live/git/go-away)
    • What it is: a single Go reverse proxy with a rule language over request properties, several challenge types (including ones that need no JavaScript), built-in DNSBL check (default dnsbl.dronebl.org), network range lists, and metrics by network range, AS and user agent.
    • Covers: R6 partly, R7 partly (through range lists rather than a lookup database; unverified), bot filtering.
    • Misses: R1, R3, R4, R5, R8, R9. YAML policy configuration. Small maintainer base.
  • fail2ban-style traefik plugins
    • tomMoulard/fail2ban: watches request rate or response status per IP inside traefik and bans for a fixed time. Covers a slice of R5 (fixed temporary ban, no escalation, state lost on traefik restart) and R2 (allowlist). Nothing else.
    • juitde/traefik-plugin-fail2ban, fail2ban connector plugins, GeoBlock plugins (country allow or deny through an external lookup API): same shape, each one small feature, configured in traefik, state in memory.
    • Sidecar fit: no; they live inside traefik.
  • Traefik built-in middlewares
    • rateLimit (one average-and-burst window per middleware, optional Redis in traefik 3.x), inFlightReq, ipAllowList. No bans, alerts, reputation or AS number awareness.
  • caddy-waf (fabriziosalmi/caddy-waf), 0.4.x
    • What it is: a Caddy module with regex rules and anomaly scoring, per-IP and per-path rate limiting with one configurable window, IP and DNS blacklists, Tor exit list fetch, country and AS number allow or deny from MaxMind databases.
    • Misses: R8 (allow or deny only), R3, R4, R5 (no documented ban state or alerting), R1's three windows. Caddyfile configuration. One maintainer, pre-1.0, AGPL-3.0.
  • open-appsec (Check Point)
    • Machine-learning WAF agent attached to nginx, Kong, Envoy or similar, with a declarative policy file or the vendor's cloud console. Covers R9 only; rate limiting and richer features are in paid tiers. Not a sidecar in the required sense.
  • iocaine
    • Serves generated garbage pages to clients the fronting proxy classifies as scrapers. Not a limiter, WAF or ban tool; out of scope except as a curiosity for scraper traffic.
  • Pangolin
    • A tunnelled access platform that bundles traefik and optionally CrowdSec. Replaces the ingress rather than adding a sidecar; out of scope.

Requirement by requirement, across the field

  • R1 three windows: BunkerWeb (one rate per URL pattern; stacking unverified). Everyone else offers one window per rule or none.
  • R2 bypass netblocks: BunkerWeb, CrowdSec, traefik, fail2ban plugins.
  • R3 alerts: CrowdSec (best), BunkerWeb (denied requests only), SafeLine (paid).
  • R4 volume and byte anomalies: none. CrowdSec can approximate request-count anomalies with a custom scenario; nothing handles bytes.
  • R5 temporary then permanent: CrowdSec only. BunkerWeb has fixed or permanent, not both by history.
  • R6 reputation: CrowdSec, BunkerWeb, go-away, caddy-waf (static lists).
  • R7 AS number lookup: CrowdSec (enrichment), BunkerWeb and caddy-waf (allow or deny), Anubis (hosted service).
  • R8 biased thresholds: none.
  • R9 attack detection: Coraza or ModSecurity with the Core Rule Set (open, standard), CrowdSec AppSec, SafeLine (closed engine), BunkerWeb (wraps the first).
  • R10 env-var sidecar: owasp/modsecurity-crs, coraza-crs-docker, Anubis. None of these covers anything beyond its one job.

Not checked

  • No tool was installed, load-tested or run against real fleet traffic.
  • Licence terms of reputation data sources for this use (AbuseIPDB free tier, Spamhaus query policy, IPinfo Lite attribution) were not read in full.
  • How upaas describes a second container for a deployed service was not examined; no repo was opened for this task.
  • Paid editions (BunkerWeb PRO, SafeLine Pro, CrowdSec paid blocklists) were assessed from public feature lists only.
  • The owner's Go dependency defaults file was not found at the documented path, so library choices in SPEC.md are proposals to be checked against it.

Sources

Reference in New Issue View Git Blame Copy Permalink