sneak/smallwebwaf
1
0 Fork 0
Code Issues 19 Pull Requests 6 Actions 1 Packages Projects Releases Wiki Activity
Files
f74f25a51df54d8bccabf389e6c165d320e0a822
smallwebwaf/EVALUATION.md
T
sneak f74f25a51d
check / check (push) In progress
Details
Say which bytes the byte limits count, and what passing one does (closes #203) 
Documents only. The byte limits and byte anomaly thresholds count body
bytes, as SWW_BYTES_COUNT chooses them, with WebSocket traffic each way,
once a response has ended; the request that passes a limit is not cut
off and the client is banned. README.md, SPEC.md and docs/ now say so,
and no longer call a response's bytes the request's. Also made plain:
rate limits and their exemptions, the refusal threshold,
SWRP_MAX_REQUEST_TIME_MS, SWRP_MAX_RESPONSE_SIZE, SWW_LOOKUP_SOURCE,
SWW_LIMIT_BAN_REPEAT_WINDOW, the metrics' byte series, and the SPEC.md
claim about answering within SWRP_TIMEOUT_MS. EVALUATION.md is audited
too; its survey facts and date are unchanged.

Model: fable-5-1 (first audit); opus-5-5 (rework)
2026-11-09 02:06:20 +00:00

16 KiB
Raw Blame History

Evaluation: existing tools for an env-var-configured protective sidecar

Date: 2026-09-21. Method: web survey of project documentation, repositories and release pages. Nothing was installed or run. Claims marked "unverified" were not confirmed against primary documentation.

What was asked for

One container that sits between traefik and one application container, configured only by environment variables, that provides:

  • R1: rate limits per minute, per hour and per day
  • R2: netblocks that bypass rate limiting
  • R3: alerts on attacks through webhook, Slack or ntfy
  • R4: alerts on anomalies: requests made or bytes transferred per minute or hour, for given IPs or netblocks, crossing a threshold
  • R5: temporary blocks for abusers, permanent bans for repeat offenders
  • R6: one or more RBLs or IP reputation APIs
  • R7: AS number lookup
  • R8: thresholds biased by AS number or country (for example, listed AS numbers get 50 percent of the normal limit)
  • R9: WAF-style attack detection and prevention
  • R10: runs as a plain env-var-configured sidecar between traefik and one app

Verdict

Nothing existing is suitable. No surveyed tool meets R8, which asks for limits scaled by AS number or country. None meets R4's alerts on bytes transferred. The tools that come closest on the rest fail R10: they need several containers, YAML or a web UI. The recommendation is to build a small single-binary sidecar; SPEC.md in this directory begins that spec.

The sidecar reuses the mature parts of the surveyed field instead of reimplementing them. It takes these as libraries and data sources:

  • Coraza and the OWASP Core Rule Set, for attack detection
  • DNSBLs
  • AbuseIPDB
  • optionally, a CrowdSec decision feed

Closest existing options, and why each still falls short:

  • BunkerWeb is the closest single product.
    • Meets: R1 (rates in requests per second, minute, hour or day), R2 (LIMIT_IGNORE_IP, also by AS number and reverse DNS), R3 partly (webhook, Slack, Discord, Matrix plugins, fired only on denied requests; ntfy only through the generic webhook, payload format unverified), R5 partly (BAD_BEHAVIOR_BAN_TIME, 0 means permanent; no escalation for repeat offenders, the ban length is one fixed value), R6 (DNSBL plugin, external blacklist URLs, optional CrowdSec), R7 partly (AS number used for blacklist and whitelist decisions), R9 (ModSecurity with the Core Rule Set, or Coraza plugin), env-var settings.
    • Fails:
      • R8: AS number and country can only allow or deny, never scale a limit.
      • R4: no alerts on request-count or byte thresholds in the free edition; reporting is a paid feature.
      • R5: no escalation for repeat offenders.
      • R10: since 1.6 it is not one plain sidecar. It needs a bunkerweb container, a bw-scheduler container and a database, or the all-in-one image that bundles nginx, the scheduler, the UI and Redis.
    • It is designed as the proxy in front of many sites, not as a sidecar for one app. AGPL-3.0.
    • Unverified: whether several rates (minute, hour, day) can be stacked on the same URL pattern; the documented design is one rate per URL pattern.
  • CrowdSec is the strongest detection and ban engine, but it is not a proxy.
    • Meets: R5 fully (profiles with duration_expr, for example four hours times the number of previous decisions), R3 (notification plugins for Slack, generic HTTP, email and others; ntfy through the HTTP plugin), R6 (community blocklist, further blocklists, reputation API), R7 (alerts are enriched with AS number and country), R9 (AppSec component with virtual patching and ModSecurity-syntax rules), R2 (allowlists).
    • Partly: R1 and R8. Detection is by leaky-bucket scenarios over logs, and a scenario can filter on AS number or country, so a stricter bucket for listed AS numbers is possible, but each is a hand-written YAML scenario, it reacts after the fact by banning, and it is not an inline limiter that answers 429.
    • Fails:
      • R10, because it needs:
        • the security engine container with persistent state
        • log acquisition from traefik
        • a bouncer such as the traefik plugin
        • YAML for acquisition, profiles, scenarios and notifications
      • R4's alerts on bytes.
  • CrowdSec plus traefik's own rateLimit middleware is the best combination with no new code. It gives inline limiting (one window per middleware, keyed by IP, with sourceCriterion exclusions), bans with escalation, alerts and reputation lookups. It still fails R8, R4's alerts on bytes, and R10 entirely: the configuration lives in traefik labels and CrowdSec YAML on each host, not in one sidecar's environment. It meets R1's three windows only by chaining three middlewares per router.

Candidate by candidate

  • CrowdSec with the traefik bouncer plugin (maxlerebourg/crowdsec-bouncer-traefik-plugin)
    • What it is: a traefik middleware plugin that asks a CrowdSec local API whether the client IP is banned, and can forward each request to the CrowdSec AppSec component for a WAF verdict. Modes: query per request, cached, streamed decision list (recommended), standalone against the central API, or AppSec only. Supports captcha remediation.
    • Covers: R5, R6, R9, R3 and R7 through the engine (see Verdict).
    • Misses: the plugin itself does no rate limiting; R8; R4's alerts on bytes.
    • Configuration: traefik static config to load the plugin, dynamic config or labels for the middleware, CrowdSec YAML for everything else.
    • Sidecar fit: no. It runs inside traefik and needs a separate engine container.
    • Maturity: widely used (about 900 stars, listed in the traefik plugin catalog, documented by CrowdSec itself), actively maintained. Traefik plugins run in an interpreter inside traefik, which costs some per-request time.
  • CrowdSec generic bouncers (nginx, Caddy, firewall)
    • Same engine, different enforcement point. The firewall bouncer blocks at nftables level on the host, which is cheap and covers every service on the host at once; worth considering fleet-wide regardless of this project. Not a sidecar, same misses as above.
  • BunkerWeb 1.6.14 (bunkerity/bunkerweb)
    • What it is: nginx with Lua plugins, ModSecurity and the Core Rule Set, configured by settings that are passed as env vars to its scheduler container. Documents running behind traefik with USE_REAL_IP and REAL_IP_FROM.
    • Coverage and misses: see Verdict. Also has bot challenges (cookie, JavaScript, captcha), country allow and deny, Tor and user-agent lists.
    • Sidecar fit: partial. Env vars yes; single small container no.
    • Maturity: mature, frequent releases, commercial company behind it with a paid edition; free plugins are AGPL-3.0.
  • SafeLine (chaitin/SafeLine)
    • What it is: nginx-derived (Tengine) reverse proxy with a proprietary semantic detection engine and a web console. About 22,000 stars.
    • Covers: R9 well, basic per-IP rate limiting with block duration, bot challenge, authentication in front of the app.
    • Misses: R8, R7, R4; R1 has one window per rule. Country blocking, alert messages, syslog forwarding and the better reputation database are paid (Lite about 10 USD per month, Pro about 100 USD per month); the free edition is capped at 10 applications.
    • Configuration: web UI backed by PostgreSQL. No env-var configuration.
    • Sidecar fit: no. Seven containers (postgres, management, detector, tengine, and three helpers); the proxy container uses host networking. The detection engine is closed source.
    • Maturity: very active, vendor-driven.
  • Coraza (corazawaf/coraza) and Coraza-based proxies
    • What it is: a Go library that implements the ModSecurity rule language and runs the OWASP Core Rule Set. OWASP project, actively maintained, the successor to ModSecurity, which is now in maintenance only.
    • Packagings: coraza-caddy (Caddy module), coraza-spoa (HAProxy), coraza-proxy-wasm (Envoy), a traefik WASM plugin, and coreruleset/coraza-crs-docker (Caddy plus Coraza plus the Core Rule Set, with env vars for backend address, engine mode and rule-set tuning).
    • Covers: R9 only. coraza-crs-docker fits R10 well: one container, env vars, backend address.
    • Misses: R1 to R8. ModSecurity-language rules can count requests per IP in a persistent collection, but Coraza's support for persistent collections is limited and this is not a practical rate limiter.
    • Value here: the right library to embed for R9 in a purpose-built sidecar.
  • ModSecurity Core Rule Set containers (owasp/modsecurity-crs)
    • What it is: official images of Apache or nginx with ModSecurity and the Core Rule Set, as a reverse proxy configured by env vars (BACKEND, paranoia level, anomaly thresholds, many more). Current images contain ModSecurity 2.9.x on Apache and Core Rule Set 4.x.
    • Covers: R9, and R10 (single container, env vars, one backend).
    • Misses: R1 to R8.
    • Maturity: rule set is very actively maintained; the ModSecurity engine itself is in maintenance under OWASP after Trustwave ended support in 2024.
  • Anubis (TecharoHQ/anubis), 1.27 current
    • What it is: a single-binary reverse proxy that makes browsers solve a proof-of-work challenge before passing them to TARGET. Aimed at scrapers, which are likely a large share of the unwanted traffic on a public code forge.
    • Covers: R10 well (one container, env vars for listener, target, difficulty, cookies). Policy rules can match on path, user agent, headers, IP ranges, and with the vendor's hosted data service also AS number and country, and can weigh a request toward a harder challenge.
    • Misses: R1, R3, R4, R5, R6, R9. Bot policy needs a YAML file, not env vars. AS number and country matching depend on the vendor's hosted service. Breaks non-browser clients unless paths are exempted; for a code forge, git-over-HTTP and API paths must be allowed through by rule.
    • Maturity: very active, widely deployed on code forges since 2025.
    • Value here: complementary. It can be chained (traefik, then the sidecar, then Anubis, then the app) if challenge pages are wanted.
  • go-away (git.gammaspectra.live/git/go-away)
    • What it is: a single Go reverse proxy with a rule language over request properties, several challenge types (including ones that need no JavaScript), built-in DNSBL check (default dnsbl.dronebl.org), network range lists, and metrics by network range, AS and user agent.
    • Covers: R6 partly, R7 partly (through range lists rather than a lookup database; unverified), bot filtering.
    • Misses: R1, R3, R4, R5, R8, R9. YAML policy configuration. Small maintainer base.
  • fail2ban-style traefik plugins
    • tomMoulard/fail2ban: watches request rate or response status per IP inside traefik and bans for a fixed time. Covers a slice of R5 (fixed temporary ban, no escalation, state lost on traefik restart) and R2 (allowlist). Nothing else.
    • juitde/traefik-plugin-fail2ban, fail2ban connector plugins, GeoBlock plugins (country allow or deny through an external lookup API): each adds one small feature, is configured in traefik and keeps its state in memory.
    • Sidecar fit: no; they live inside traefik.
  • Traefik built-in middlewares
    • rateLimit (one average-and-burst window per middleware, optional Redis in traefik 3.x), inFlightReq, ipAllowList. No bans, alerts, reputation lookups or AS number lookups.
  • caddy-waf (fabriziosalmi/caddy-waf), 0.4.x
    • What it is: a Caddy module with regex rules and anomaly scoring, per-IP and per-path rate limiting with one configurable window, IP and DNS blacklists, Tor exit list fetch, country and AS number allow or deny from MaxMind databases.
    • Misses: R8 (allow or deny only), R3, R4, R5 (no documented ban state or alerting), R1's three windows. Caddyfile configuration. One maintainer, pre-1.0, AGPL-3.0.
  • open-appsec (Check Point)
    • Machine-learning WAF agent attached to nginx, Kong, Envoy or similar, with a declarative policy file or the vendor's cloud console. Covers R9 only; rate limiting and richer features are in paid tiers. It attaches to an existing proxy, so it is not the sidecar R10 asks for.
  • iocaine
    • Serves generated garbage pages to clients the fronting proxy classifies as scrapers. Not a limiter, WAF or ban tool; out of scope, and only of passing interest for scraper traffic.
  • Pangolin
    • A tunnelled access platform that bundles traefik and optionally CrowdSec. Replaces the ingress rather than adding a sidecar; out of scope.

Requirement by requirement, across the field

  • R1 three windows: BunkerWeb (one rate per URL pattern; stacking unverified). Everyone else offers one window per rule or none.
  • R2 bypass netblocks: BunkerWeb, CrowdSec, traefik, fail2ban plugins.
  • R3 alerts: CrowdSec (best), BunkerWeb (denied requests only), SafeLine (paid).
  • R4 alerts on request counts and bytes: none. CrowdSec can approximate alerts on request counts with a custom scenario; no surveyed tool alerts on bytes.
  • R5 temporary then permanent: CrowdSec only. BunkerWeb's ban is either one fixed length or permanent, and does not change with a client's history.
  • R6 reputation: CrowdSec, BunkerWeb, go-away, caddy-waf (static lists).
  • R7 AS number lookup: CrowdSec (enrichment), BunkerWeb and caddy-waf (allow or deny), Anubis (hosted service).
  • R8 biased thresholds: none.
  • R9 attack detection: Coraza or ModSecurity with the Core Rule Set (open, standard), CrowdSec AppSec, SafeLine (closed engine), BunkerWeb (wraps Coraza or ModSecurity).
  • R10 env-var sidecar: owasp/modsecurity-crs, coraza-crs-docker, Anubis. None of these covers anything beyond its one job.

Not checked

  • No tool was installed, load-tested or run against real fleet traffic.
  • Licence terms of reputation data sources for this use (AbuseIPDB free tier, Spamhaus query policy, IPinfo Lite attribution) were not read in full.
  • How upaas describes a second container for a deployed service was not examined; no repo was opened for this task.
  • Paid editions (BunkerWeb PRO, SafeLine Pro, CrowdSec paid blocklists) were assessed from public feature lists only.
  • The owner's Go dependency defaults file was not found at the documented path, so library choices in SPEC.md are proposals to be checked against it.

Sources

Reference in new issue View Git Blame Copy Permalink