smallwebwaf/SPEC.md

smallwebwaf SPEC (draft): protective reverse-proxy sidecar

Status: second draft, with the owner's rulings on storage, logging and metrics applied. Nothing has been built yet. EVALUATION.md beside this file explains why no existing tool was chosen. Points that turn on an owner decision are collected under "Questions for the owner" and are marked "(open)" where they appear.

Purpose

One small container that sits between traefik and one application container. The traefik router for the public hostname points at the sidecar; the sidecar forwards to the application. The sidecar limits request and byte rates per client, detects attacks, blocks abusers for a time and bans repeat offenders, consults IP reputation sources, looks up the AS number and country of each client, scales its limits for listed AS numbers and countries, and sends alerts. Everything is configured by environment variables, apart from the attack detection rules, which are read from a directory of hand-editable text files.

Non-goals

• TLS termination, certificates, hostname routing: traefik's job.
• More than one upstream application per sidecar. Run one sidecar per app.
• Browser challenges (captcha, proof of work). If wanted, chain Anubis between the sidecar and the app.
• Defence against traffic floods that saturate the host's network link. That needs help upstream of the host.
• A web UI or a configuration file. Settings are environment variables; the only files read are the rule files, which hold one regex per line and nothing more elaborate.
• Sharing ban state between sidecars in the first version (open, see Questions).

Architecture

• One statically linked Go binary in one container, running as a non-root user, read-only root filesystem, one writable volume for state.
• Three listeners, of which only the first is routed by traefik:
  • the proxy listener;
  • a metrics listener serving Prometheus metrics, which can be switched off;
  • an admin listener for health and ban management.
• Components inside the process:
  • Client identification: works out the real client IP from X-Forwarded-For, trusting only configured proxy netblocks.
  • Lookup: AS number and country from local database files, held in memory.
  • Reputation: static blocklists fetched on a schedule; DNSBL and reputation API queries made in the background and cached.
  • Counters: per-client request and byte counts per minute, hour and day, held in memory; the hour and day counts are written out so they survive a restart.
  • Attack detection: regex rules read at start from a directory of plain text rule files (see "Rule files"), Coraza with the OWASP Core Rule Set, and simple signals (requests for listed trap paths, bursts of error responses from the app).
  • Ban ledger: offences, active bans and ban history, held in memory and written to disk as JSON (see "Persistent state"). No database of any kind is used.
  • Request log: one JSON object per request on stdout, optionally also sent to a remote syslog or RELP endpoint (see "Request log").
  • Metrics: Prometheus counters and gauges on their own listener (see "Metrics endpoint").
  • Alerting: a queue with de-duplication feeding webhook, Slack and ntfy senders.
  • Proxy: the standard library's net/http/httputil.ReverseProxy, streaming in both directions, with WebSocket upgrade support.
• Proposed libraries (to be checked against the owner's Go dependency defaults before any are added):
  • standard library for the proxy, HTTP clients, DNS, logging (log/slog), state files (encoding/json, os.Rename);
  • github.com/corazawaf/coraza/v3 and github.com/corazawaf/coraza-coreruleset for attack detection;
  • github.com/oschwald/maxminddb-golang to read .mmdb lookup databases;
  • github.com/prometheus/client_golang for metrics;
  • remote log sending: the standard library's log/syslog is frozen and does not write the current syslog format (RFC 5424), and no widely used Go RELP client was found, so the library for this is open (see Questions).

Data flow for one request

Steps run in this order; the first step that produces a final answer ends processing.

• Identify the client.
  • If the TCP peer is inside TRUSTED_PROXIES, walk X-Forwarded-For from the right and take the first address not inside TRUSTED_PROXIES. Otherwise use the TCP peer address and ignore the header.
  • IPv6 clients are grouped by prefix (IPV6_GROUP_PREFIX, default 64) for counting and banning, because one abuser usually controls a whole /64.
• Static lists.
  • In ALLOW_NETS: skip every check below and forward. Still counted for anomaly alerts.
  • In DENY_NETS: refuse.
• Ban ledger. An active ban: refuse (BAN_RESPONSE).
• Lookup AS number and country. Unknown is a valid result and changes nothing.
• Reputation.
  • Address inside a fetched blocklist: apply BLOCKLIST_ACTION.
  • Cached DNSBL or reputation API result: apply REPUTATION_ACTION.
  • No cached result: queue a background query and carry on. A first request is never delayed by a reputation query.
• Work out the client's limit percentage: the lowest of the percentages that apply (AS number, country, reputation). 100 if none applies. 0 means refuse. (Lowest-wins versus multiplying is open.)
• Request rate limits. Unless the client is in RATE_LIMIT_EXEMPT_NETS, check the minute, hour and day counters against each configured limit times the percentage. Over any limit: answer 429 with Retry-After, record an offence.
• Rule files. The request is checked against the rules loaded from RULES_DIR, in file name order then line order. Each rule that matches takes its action: record offences, refuse with 403, ban at once, or only log. Matching stops at the first rule that refuses or bans.
• Core Rule Set inspection of the request (headers, URL, and body up to WAF_BODY_LIMIT). In block mode a match at or over the anomaly threshold is refused with 403 and recorded as an offence; in detect mode it is only logged and alerted.
• Forward to UPSTREAM_URL, streaming. Add X-Forwarded-For and, if enabled, X-Client-ASN and X-Client-Country for the app's own logs.
• After the response.
  • Add response bytes (and request body bytes) to the byte counters. Once a client's byte total passes a byte limit times the percentage, following requests get 429 until the window moves on, and an offence is recorded. A response already in progress is not cut off.
  • Count 401, 403 and 404 responses from the app per client; passing ERROR_BURST_THRESHOLD within a minute records an offence.
  • Update anomaly counters and evaluate alert thresholds.
• Offences and bans.
  • BAN_AFTER_OFFENCES offences within OFFENCE_WINDOW creates a ban.
  • Ban length follows BAN_DURATIONS by the number of earlier bans for that client within BAN_HISTORY_WINDOW: first ban the first value, second ban the second value, and so on.
  • After PERMANENT_BAN_AFTER bans within BAN_HISTORY_WINDOW, the ban has no expiry.
  • A request for a path in TRAP_PATHS counts as TRAP_OFFENCE_WEIGHT offences at once.

Counting method

• Each window (minute, hour, day) uses two adjacent fixed buckets per client, with the previous bucket weighted by how much of it still overlaps the sliding window. This costs a few integers per client per window and avoids the burst at bucket boundaries that a single fixed bucket allows.
• Counters are read and updated in memory only. The hour and day counters are written to disk on a timer and at shutdown and loaded again at start (see "Persistent state"), so a restart does not hand every client a fresh daily allowance. Minute counters are not written; a restart forgives at most one minute of counting. Buckets whose time has passed are discarded on load.
• Memory is bounded by MAX_TRACKED_CLIENTS; when full, the least recently seen clients with no recent offences are dropped first.

Configuration surface

Conventions: lists are comma separated; netblocks are CIDR (a bare address means /32 or /128); durations use Go syntax plus d for days (90s, 15m, 24h, 7d); byte sizes accept K, M, G suffixes; an unset limit means that limit is off. Every variable may instead be given as NAME_FILE pointing at a file holding the value, for secrets and long lists. Invalid configuration stops the process at start with a message naming the variable. At start the effective configuration is logged with secrets masked.

• Core
  • UPSTREAM_URL (required): the application, for example http://gitea:3000.
  • LISTEN_ADDR (default :8080): proxy listener.
  • ADMIN_LISTEN_ADDR (default 127.0.0.1:9090): admin listener.
  • ADMIN_TOKEN: bearer token required for ban management endpoints.
  • INSTANCE_NAME: included in every log line, metric and alert, for example fsn1app1/gitea.
  • MODE (default enforce): enforce, or observe to log and alert on every decision while refusing nothing. Meant for the first days of a rollout.
  • TRUSTED_PROXIES (required): netblocks whose X-Forwarded-For is believed, normally the docker network traefik reaches the sidecar from.
  • IPV6_GROUP_PREFIX (default 64).
  • MAX_TRACKED_CLIENTS (default 500000).
• Persistent state
  • STATE_DIR (default /data): the JSON state files and downloaded lookup databases.
  • STATE_WRITE_DELAY (default 2s): after a ban or offence changes, wait this long before writing, so a burst of changes becomes one write.
  • STATE_COUNTER_INTERVAL (default 60s): how often the hour and day counters and the reputation cache are written.
• Logging
  • The request log on stdout is always on and has no switch.
  • LOG_LEVEL (default info): for the process's own messages (start-up, fetch failures, state writes), which are JSON lines on stdout too, marked "type":"process". It does not filter the request log.
  • LOG_REQUEST_HEADERS (default accept,accept-language,accept-encoding,content-type,origin,range): extra request headers to record. Authorization, Cookie and Set-Cookie values are never logged, only whether they were present.
  • LOG_REMOTE_URL: when set, every log line is also sent to this endpoint. Forms: syslog+udp://host:514, syslog+tcp://host:514, syslog+tls://host:6514, relp://host:2514, relp+tls://host:2514.
  • LOG_REMOTE_TLS_CA_FILE: optional CA certificate for the +tls forms.
  • LOG_REMOTE_BUFFER (default 10000): lines held in memory while the endpoint is unreachable; when full the oldest are dropped and counted.
  • LOG_REMOTE_FACILITY (default local0), LOG_REMOTE_APP_NAME (default INSTANCE_NAME): syslog header fields.
• Metrics
  • METRICS_ENABLED (default true).
  • METRICS_LISTEN_ADDR (default :9100): its own listener, so it can be bound to a monitoring network without exposing the admin endpoints.
  • METRICS_PATH (default /metrics).
  • METRICS_TOP_N (default 50): how many AS numbers and countries get their own series; the rest are summed as other.
  • METRICS_TOKEN: optional bearer token; unset means no authentication, which is the usual arrangement for a scraper on a private network.
• Static lists
  • ALLOW_NETS: bypass everything (monitoring, the owner's own networks).
  • RATE_LIMIT_EXEMPT_NETS: bypass request and byte limits only; attack detection and bans still apply.
  • DENY_NETS: always refused.
• Request rate limits, per client (R1, R2)
  • RATE_LIMIT_PER_MINUTE, RATE_LIMIT_PER_HOUR, RATE_LIMIT_PER_DAY.
  • RATE_LIMIT_EXEMPT_PATHS: path prefixes not counted (static assets, health checks).
• Byte limits, per client
  • BYTES_LIMIT_PER_MINUTE, BYTES_LIMIT_PER_HOUR, BYTES_LIMIT_PER_DAY.
  • BYTES_COUNT (default both): response, request or both.
• Lookup of AS number and country (R7)
  • ASN_DB_PATH, COUNTRY_DB_PATH: .mmdb files mounted into the container; or
  • ASN_DB_URL, COUNTRY_DB_URL: downloaded into STATE_DIR at start and every LOOKUP_DB_REFRESH (default 7d). One file may serve both.
  • Known free sources in this format: IPinfo Lite (country and AS number in one file, free account token), DB-IP Lite, MaxMind GeoLite2 (free account). Choice is open.
  • ADD_LOOKUP_HEADERS (default false): pass X-Client-ASN and X-Client-Country to the app.
  • Missing or unreadable database: start anyway, treat every client as unknown, log a warning and send one alert.
• Biased thresholds (R8)
  • ASN_LIMIT_PERCENT: for example AS14061:50,AS16276:50,AS45102:25. Listed AS numbers get that percentage of every request and byte limit. 0 refuses outright.
  • COUNTRY_LIMIT_PERCENT: same form with ISO country codes, for example CN:25,RU:50.
  • ASN_BYTES_PERCENT, COUNTRY_BYTES_PERCENT: optional overrides applied to byte limits only, when the byte percentage should differ from the request percentage.
  • UNKNOWN_LIMIT_PERCENT (default 100): for clients the lookup cannot place.
  • ASN_LIMIT_PERCENT_URL: optional URL of a text file of AS:percent lines, so one abuse-source list can be shared by every sidecar in the fleet.
• Attack detection (R9)
  • RULES_DIR (default /etc/smallwebwaf/rules.d): directory of rule files read once at start; format under "Rule files". The image ships a default file there. Mounting a directory over it replaces the defaults; mounting single files into it adds to them.
  • RULES_ENABLED (default true): false skips rule files entirely.
  • WAF_MODE (default detect): off, detect, block.
  • WAF_PARANOIA_LEVEL (default 1), WAF_ANOMALY_THRESHOLD (default 5): the Core Rule Set's own two tuning values.
  • WAF_DISABLED_RULES: rule ids to switch off when an app trips a false positive.
  • WAF_EXEMPT_PATHS: path prefixes not inspected.
  • WAF_BODY_LIMIT (default 128K): bodies are inspected up to this size and streamed beyond it without buffering, so large uploads and git pushes are not held in memory.
  • TRAP_PATHS: paths the app never serves and only scanners ask for, for example /.env,/wp-login.php,/.git/config. TRAP_OFFENCE_WEIGHT (default 3). This is the env-var short form of a path rule with an offence action, for deployments that mount no rule files.
  • ERROR_BURST_THRESHOLD (default off): 401, 403 and 404 responses per client per minute that count as an offence.
• Blocks and bans (R5)
  • BAN_AFTER_OFFENCES (default 3), OFFENCE_WINDOW (default 10m).
  • BAN_DURATIONS (default 1h,24h,7d).
  • PERMANENT_BAN_AFTER (default 4; 0 never bans permanently), BAN_HISTORY_WINDOW (default 90d).
  • BAN_RESPONSE (default 403): 403, 429, or close to drop the connection without an answer.
  • BAN_SCOPE_V4_PREFIX (default 32): widen to for example 24 to ban the surrounding netblock.
• Reputation (R6)
  • BLOCKLIST_URLS: text files of addresses and netblocks, for example Spamhaus DROP, FireHOL level 1, the Tor exit list. Refreshed every BLOCKLIST_REFRESH (default 6h); the last good copy is kept on failure.
  • BLOCKLIST_ACTION (default limit:25): deny, limit:<percent>, or log.
  • DNSBL_ZONES: for example dnsbl.dronebl.org. Zones meant for mail (lists of residential ranges) will block ordinary visitors and should not be used; Spamhaus zones need their keyed query service, given as a zone name containing the key.
  • DNSBL_RESOLVER: optional resolver address, since public resolvers are refused by several list operators.
  • ABUSEIPDB_KEY, ABUSEIPDB_MIN_SCORE (default 75), ABUSEIPDB_DAILY_BUDGET (default 900; the free tier allows 1000 checks a day). Only clients that have already committed one offence are queried, so the budget is spent on suspects.
  • CROWDSEC_LAPI_URL, CROWDSEC_LAPI_KEY: optional. If a CrowdSec engine exists on the host, pull its decision list on a schedule and treat listed addresses as banned. This is how a fleet-wide blocklist can arrive without the sidecar depending on CrowdSec.
  • REPUTATION_ACTION (default limit:25): deny, limit:<percent>, or log, for DNSBL and API hits.
  • REPUTATION_CACHE_TTL (default 24h), REPUTATION_TIMEOUT (default 2s).
• Alerting (R3)
  • ALERT_WEBHOOK_URL: JSON POST; schema below. ALERT_WEBHOOK_HEADERS: optional Name:value pairs for authentication.
  • ALERT_SLACK_WEBHOOK_URL: Slack incoming webhook, formatted message.
  • ALERT_NTFY_URL (full topic URL), ALERT_NTFY_TOKEN: title, priority and tags set from the event type.
  • ALERT_EVENTS (default ban,permanent_ban,waf_block,anomaly,reputation_hit,source_failure): which event types are sent.
  • ALERT_COOLDOWN (default 15m): the same event type for the same client or netblock is not repeated within this time; a count of suppressed repeats is included in the next one.
  • ALERT_MAX_PER_HOUR (default 60): beyond this, alerts are rolled into one summary per hour so a wide attack cannot flood the channel.
• Anomaly thresholds, alert only, nothing is blocked (R4)
  • Per client: ANOMALY_CLIENT_REQUESTS_PER_MINUTE, ANOMALY_CLIENT_REQUESTS_PER_HOUR, ANOMALY_CLIENT_BYTES_PER_MINUTE, ANOMALY_CLIENT_BYTES_PER_HOUR.
  • Per surrounding netblock (ANOMALY_NET_V4_PREFIX default 24, ANOMALY_NET_V6_PREFIX default 48): ANOMALY_NET_REQUESTS_PER_MINUTE, ..._PER_HOUR, ANOMALY_NET_BYTES_PER_MINUTE, ..._PER_HOUR.
  • Per AS number: ANOMALY_ASN_REQUESTS_PER_MINUTE, ..._PER_HOUR, ANOMALY_ASN_BYTES_PER_MINUTE, ..._PER_HOUR.
  • Whole service: ANOMALY_TOTAL_REQUESTS_PER_MINUTE, ..._PER_HOUR, ANOMALY_TOTAL_BYTES_PER_MINUTE, ..._PER_HOUR.
  • Named netblocks: WATCH_NETS, for example office=203.0.113.0/24,scraper-x=198.51.100.0/22, with WATCH_REQUESTS_PER_MINUTE, ..._PER_HOUR, WATCH_BYTES_PER_MINUTE, ..._PER_HOUR applied to each named block as a whole.
  • Anomaly counting includes clients in ALLOW_NETS and RATE_LIMIT_EXEMPT_NETS, since an exempt client misbehaving is worth knowing about.

Rule files

A required feature: at start the sidecar reads every *.rules file in RULES_DIR and checks each request against them. The files are plain text meant to be edited by hand, so that a new scanning pattern seen in the request log can be turned into a rule in one line.

• One rule per line, four fields separated by spaces or tabs; the fourth field runs to the end of the line:

  <id>  <target>  <action>  <regex>
  

• Blank lines and lines starting with # are ignored.

• id: a short name of letters, digits, - and _, unique across all files. It appears in the request log, metrics, alerts and offence history.

• target: what the regex is matched against.

  • path: the URL path as received, before any decoding.
  • query: the raw query string.
  • uri: path and query together, both as received and once percent-decoded, so an encoded probe cannot slip past.
  • method, host, user_agent, referer.
  • header:<Name>: any one request header.
  • Request bodies are not available to rule files; body inspection is the Core Rule Set's job.

• action:

  • log: note the match in the request log and do nothing else.
  • offence or offence:<n>: record one or n offences and forward the request. Enough offences within OFFENCE_WINDOW creates a ban.
  • block or block:<n>: refuse with 403 and record one or n offences.
  • ban: refuse and ban the client at once, at whatever length its ban history calls for.

• regex: Go regular expression syntax (RE2). It has no backreferences or lookaround, and in exchange matching time is linear in the input, so no rule can be made to stall the proxy. (?i) at the front makes a rule case-insensitive. A rule matches if the regex matches anywhere in the target; anchor with ^ and $ when that is not wanted.

• Files are read in name order (00-default.rules before 50-gitea.rules), rules in line order.

• Rules are compiled once at start and held in memory. A line that does not parse, a regex that does not compile, or a duplicate id stops the process with a message naming the file and line. A missing or empty directory is not an error: the log says that no rules were loaded. Changing rules means restarting the container; reloading while running is given up for now.

• In MODE=observe every action is logged as what would have happened and nothing is refused.

• Clients in ALLOW_NETS are not checked.

Example file:

# 00-default.rules: probes no real visitor sends

# id             target      action     regex
env-file         path        offence:3  (?i)/\.env(\.[a-z]+)?$
git-dir          path        offence:3  ^/\.git/(config|HEAD|index)$
wp-probe         path        offence:3  (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/)
php-shell        path        block:3    (?i)/(shell|c99|r57|wso|alfa)\.php$
path-traversal   uri         block      (\.\./){2,}
scanner-agent    user_agent  ban        (?i)\b(sqlmap|nikto|nuclei|masscan|zgrab|wpscan)\b
empty-agent      user_agent  log        ^$

The image ships one default file of this kind. It is kept short and limited to patterns that are wrong for every app; anything app-specific (for example blocking /wp-login.php is harmless in front of gitea and fatal in front of WordPress) belongs in a file the deployer mounts.

Persistent state

All state lives in memory. No database is used. What must survive a restart is written to STATE_DIR as JSON files; the files are read once at start and never during a request. The whole state is expected to stay under 10 MiB, and reading or writing even ten times that as JSON takes well under a second, so whole-file rewrites are cheap enough to need nothing cleverer.

• Files, each holding one kind of state:
  • bans.json: active and past bans. Per entry: client or netblock, start, expiry (null for permanent), reason, which ban this is for that client (first, second, third), source (local, admin, crowdsec), and when it was lifted, if it was.
  • offences.json: offender history. Per entry: client, time, kind (rate, bytes, waf, trap, error_burst), short detail. Entries older than BAN_HISTORY_WINDOW are dropped at each write.
  • counters.json: the hour and day request and byte counters per client.
  • reputation.json: cached DNSBL and reputation API verdicts with the time each was fetched.
• Format: indented JSON with a top-level version number, entries sorted by client address, times in RFC 3339 UTC, durations and sizes as plain numbers with the unit in the field name. The aim is that a person can open bans.json in an editor, find an address, and remove or add an entry.
• Writing:
  • serialise from a snapshot taken under the lock, so requests are not held up while the file is written;
  • write to a temporary file in the same directory, sync it, rename it over the real name, sync the directory. A crash at any point leaves either the old complete file or the new complete file, never a partial one;
  • bans.json and offences.json are written STATE_WRITE_DELAY after a change; counters.json and reputation.json every STATE_COUNTER_INTERVAL; all four on orderly shutdown (SIGTERM).
• Reading:
  • at start only. Active bans go into an in-memory prefix lookup; everything else into maps. Expired counters and cache entries are discarded.
  • a missing file means empty state and is normal on first run.
  • a file that does not parse, or has an unknown version, stops the process with a message naming the file and position. Starting with empty bans would silently forgive every repeat offender, and since writes are atomic a broken file can only come from a hand edit, which the editor should hear about. counters.json and reputation.json are the exception: if unreadable they are logged, set aside with a .bad suffix, and the process starts with them empty.
• Hand edits are made with the container stopped; a running process will overwrite the file at its next write. Changes to a running process go through the admin endpoints. Reloading files into a running process is given up for now.
• STATE_DIR not writable at start: the process exits. A write that fails while running: state stays correct in memory, the failure is logged, counted in metrics, alerted once per cooldown, and retried at the next write.
• What a hard kill can lose: bans and offences from the last STATE_WRITE_DELAY, counters from the last STATE_COUNTER_INTERVAL. Losing the volume loses ban history, not service.

Request log

One JSON object per line on stdout for every request, including refused ones. stdout is always on. When LOG_REMOTE_URL is set the same lines are also sent to the remote endpoint, so a deployment can stop depending on docker's log handling while docker logs keeps working.

• Standard web log fields: time (RFC 3339 with milliseconds), instance, client_ip, method, scheme, host, path, query, protocol, status, request_bytes, response_bytes, referer, user_agent.
• Request detail: request_id (generated if traefik did not supply one, and passed to the app), peer_ip (the TCP peer, normally traefik), forwarded_for (the header as received), client_group (the /64 or configured prefix used for counting), asn, as_name, country, content_type, content_length, the headers named in LOG_REQUEST_HEADERS, has_authorization and has_cookie as booleans, websocket when the connection was upgraded.
• Response detail: response_content_type, upstream_status (differs from status when the sidecar answered itself), cache_control, location on redirects, aborted when the client went away early.
• Decision: action (forward, rate_limited, bytes_limited, banned, denied, waf_blocked, upstream_error), would_action in observe mode, limit_percent and which rule set it, counts (the client's minute, hour and day request and byte totals after this request), limit_hit (which window), rule_ids (rule file rules that matched), waf_rule_ids, waf_score, reputation (sources that listed the client), offence when one was recorded, ban_expires.
• Timings in milliseconds: duration_total, duration_checks (everything the sidecar did before forwarding), duration_waf, duration_upstream_connect, duration_upstream_first_byte, duration_upstream_total.
• Bodies are never logged. Query strings are logged as received; an app that carries secrets in query strings needs that fixed in the app.
• The process's own messages share the stream as JSON lines with "type":"process"; request lines carry "type":"request".
• Remote sending:
  • syslog forms send each line as the message of an RFC 5424 record, with octet-counted framing on TCP and TLS;
  • RELP forms send the same record and wait for the receiver's acknowledgement, so lines are not lost across a receiver restart. This is the form to use with rsyslog;
  • sending happens on its own goroutine from a bounded buffer (LOG_REMOTE_BUFFER). An unreachable or slow endpoint never delays a request and never stops stdout; it reconnects with backoff, drops the oldest lines when the buffer is full, and counts the drops in metrics. UDP gives no delivery signal at all and is offered only for compatibility.

Metrics endpoint

Prometheus text format on METRICS_LISTEN_ADDR at METRICS_PATH, on by default, switched off with METRICS_ENABLED=false. It has its own listener so it can be reached by a scraper without exposing ban management.

• Traffic: requests and bytes in and out, by status class and action; request duration and upstream duration histograms; requests in flight.
• Limits and bans: limit hits by window and kind (requests or bytes), offences by kind, bans created by ordinal, permanent bans, active bans (gauge).
• Attack detection: rule file matches by rule id and action, and the number of rules loaded; Core Rule Set matches by mode and rule id (label limited to the rules that actually fired).
• Lookup and reputation: requests and bytes by AS number and by country, limited to the METRICS_TOP_N (default 50) busiest of each with the rest summed as other, so the label set stays bounded; reputation queries, hits, failures and remaining daily budget by source; age of each blocklist and lookup database.
• Housekeeping: tracked clients (gauge), state file writes, write failures, last successful write time and size per file; alerts sent, failed and suppressed by destination; remote log lines sent, dropped and buffer depth; the standard Go runtime and process metrics.
• No metric carries a client IP address as a label; per-address questions are answered by the request log and GET /clients/<ip>.

Admin listener

• GET /healthz: for the container health check.
• GET /bans, POST /bans (client or netblock, duration, reason), DELETE /bans/<client>: need ADMIN_TOKEN.
• GET /clients/<ip>: current counters, lookup result, reputation, offences; for answering "why was this address refused".

Alert webhook schema

One JSON object per alert:

• instance, time, event (one of the ALERT_EVENTS values)
• client, netblock, asn, as_name, country
• reason: short human-readable sentence
• detail: event-specific fields, for example window, count, limit, limit_percent, rule_ids, path, ban_ordinal, ban_expires
• suppressed_repeats: number of identical alerts held back by the cooldown

Deployment as a sidecar

docker-compose shape, using gitea as the example; upaas deployments follow the same shape if upaas can run a second container for a service (open):

services:
    gitea:
        image: gitea/gitea:1
        # no traefik labels, no published ports
        networks: [internal]

    gitea-guard:
        image: <registry>/<image>:<pinned digest>
        read_only: true
        user: "65532:65532"
        environment:
            UPSTREAM_URL: http://gitea:3000
            TRUSTED_PROXIES: 172.18.0.0/16
            INSTANCE_NAME: fsn1app1/gitea
            MODE: observe
            RATE_LIMIT_PER_MINUTE: "120"
            RATE_LIMIT_PER_HOUR: "2000"
            RATE_LIMIT_PER_DAY: "10000"
            BYTES_LIMIT_PER_HOUR: 2G
            ASN_DB_URL: https://example.invalid/asn-country.mmdb
            ASN_LIMIT_PERCENT: AS14061:50,AS16276:50
            COUNTRY_LIMIT_PERCENT: CN:25
            WAF_MODE: detect
            TRAP_PATHS: /.env,/wp-login.php,/.git/config
            BLOCKLIST_URLS: https://www.spamhaus.org/drop/drop.txt
            ALERT_NTFY_URL: https://ntfy.example.invalid/fleet-alerts
            LOG_REMOTE_URL: relp://logs.example.invalid:2514
            METRICS_LISTEN_ADDR: :9100
        volumes:
            - gitea-guard-state:/data
            - ./50-gitea.rules:/etc/smallwebwaf/rules.d/50-gitea.rules:ro
        networks: [internal, traefik]
        labels:
            traefik.enable: "true"
            traefik.http.routers.gitea.rule: Host(`git.example.invalid`)
            traefik.http.services.gitea.loadbalancer.server.port: "8080"

• The application container leaves the traefik network, so the sidecar cannot be bypassed.
• Traefik routes only port 8080. The metrics port is reached by the scraper over a docker network it shares with the sidecar; the admin port stays on loopback inside the container and is used through docker exec.
• The state volume holds a few small JSON files; it needs no backup beyond whatever the host already does.
• SSH access to gitea does not pass through the sidecar and is not protected by it.
• Rollout per service: start in MODE=observe with WAF_MODE=detect, read a few days of logs and alerts, set limits and rule exclusions from what real traffic looks like, then switch to enforce and block.
• Notes specific to gitea: clones and pushes over HTTP are large and long, so byte limits must be sized for a legitimate clone, the proxy sets no overall response timeout, and WAF_BODY_LIMIT keeps pack uploads out of memory. Archive download and blame or history pages are what scrapers hammer; request limits do most of the work there.

Failure behaviour

• Lookup database missing: clients are unknown, service continues, one alert.
• Reputation source down or over quota: no verdict, service continues, one source_failure alert per cooldown.
• Alert destination down: retried with backoff from a bounded queue, oldest dropped first, drops counted in metrics.
• Upstream down: 502 from the sidecar, not counted as client offences.
• Attack detection engine error on a request: request is forwarded, error logged and counted.
• Remote log endpoint down: stdout continues, lines are buffered then dropped oldest first, drops counted in metrics.
• State file write fails while running: memory stays authoritative, logged, counted, alerted, retried.
• In short: the only things that stop the process are invalid configuration, a rule file that does not parse, an unwritable STATE_DIR, and an unparseable bans.json or offences.json at start. A broken helper never takes the protected service down.

Risks the design has to handle

• Forged X-Forwarded-For: handled by TRUSTED_PROXIES being required and by walking the header from the right.
• Many clients behind one address (mobile carriers, offices, Tor): rate limits punish them collectively; RATE_LIMIT_EXEMPT_NETS and sensible day limits are the remedy, and bans on such addresses should stay short.
• IPv6 address rotation inside a /64: handled by grouping.
• Widely distributed scrapers using thousands of addresses at low rates each: per-client limits do not see them. The AS number and netblock anomaly alerts reveal them, and ASN_LIMIT_PERCENT at a low value or 0 is the response. A per-AS-number or per-country total limit is open.
• Core Rule Set false positives against real apps (gitea's editor, API payloads): detect by default, exclusions by rule id and path.
• Slow-request attacks: header read timeout and idle timeout on the listener.
• Alert floods: cooldown and hourly cap.

Build order

• First: proxy, client identification, static lists, three-window request limits, exemptions, observe mode, the full request log on stdout, the metrics endpoint, health.
• Second: rule files, ban ledger with escalation and permanent bans, the JSON state files, admin endpoints, alerting to all three destinations, remote log sending.
• Third: AS number and country lookup, biased thresholds, byte limits, anomaly thresholds.
• Fourth: blocklists, DNSBL, AbuseIPDB, optional CrowdSec decision feed.
• Fifth: attack detection with Coraza and the Core Rule Set, trap paths, error bursts.
• Each stage is usable on its own; the first three already cover the traffic problem the fleet has today.

Questions for the owner

• Combining percentages. A client in a listed AS number (50 percent) and a listed country (25 percent): should it get the lowest (25 percent of the normal limit) or the product (12.5 percent)? Lowest is easier to predict when reading a config; product punishes overlap harder. Recommendation: lowest.
• Meaning of "certain countries getting a percentage rate or byte limit". Read here as: each client from that country gets that percentage of the normal per-client limits. The other reading is a cap on the country as a whole, for example all of one country together may use at most 20 percent of the service's hourly byte budget. The second needs a configured total budget and means one heavy client can lock out its compatriots. Recommendation: build the per-client reading first; add whole-country and whole-AS-number caps only if distributed scrapers make it necessary.
• Sharing bans across the fleet. As drafted, each sidecar remembers only its own bans, so an abuser banned at gitea starts fresh at the next service. Options: keep it per sidecar; or run one CrowdSec engine per host and have every sidecar read its decision list (already supported above) and also report its own bans to it. Recommendation: per sidecar for the first version, decide on CrowdSec after seeing real ban volumes.
• Lookup database source. IPinfo Lite (one file with country and AS number, free account, attribution required), DB-IP Lite (no account, attribution required), or MaxMind GeoLite2 (free account, licence terms on redistribution). Recommendation: IPinfo Lite, fetched once centrally and served to the fleet from an internal URL so sidecars hold no account token.
• upaas. Whether upaas can deploy a second container beside an app and move the traefik labels onto it was not examined. If it cannot, that is a feature request for upaas, not something this sidecar can solve.
• Remote log library. Sending syslog in the current format (RFC 5424) is not covered by Go's standard library, whose log/syslog package is frozen and writes the older format, and no widely used Go RELP client was found. The choices are a small third-party package for each, or writing both inside this project; the syslog record is a few dozen lines, a RELP client with acknowledgement handling is a few hundred. Recommendation: write the syslog sender in-project, ship it first, and decide on RELP after looking at the available packages; say if RELP must be in the first release.
• Ban response. 403 tells the abuser they were noticed; close wastes less and tells them nothing. Recommendation: 403 by default for debuggability, close available.