fmt and feedback comments, no code changes

script/publish runs script/docker with DOCKER_HOST set to the Docker
host (ssh://root@fsn1app1 unless set) and pushes the image to
git.eeqj.de/sneak/smallwebwaf under four tags: the UTC date as
YYYY-MM-DD, the short commit hash, the branch name and latest. It
refuses a working tree with uncommitted changes, since the hash tag
names a commit.

Makefile

@@ -1,5 +1,5 @@
-.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
-	run example-app gitea-requests gitea-setup
+.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker publish \
+	hooks build run example-app gitea-requests gitea-setup
 
 # Makefile targets are thin shims; the implementations live in script/
 # per the scripts-to-rule-them-all pattern (see the Entrypoints section
@@ -7,7 +7,8 @@
 # which test checks. build and run are for working on the code by hand;
 # example-app checks the image with an app built on it; gitea-requests
 # reports the gitea requests the Core Rule Set refuses at the defaults;
-# gitea-setup checks the setup in deploy/gitea.
+# gitea-setup checks the setup in deploy/gitea; publish builds the image
+# on the Docker host and pushes it to the registry.
 
 bootstrap:
 	@script/bootstrap
@@ -36,6 +37,9 @@ check:
 docker:
 	@script/docker
 
+publish:
+	@script/publish
+
 hooks:
 	@script/install-precommit
 

README.md

@@ -743,6 +743,11 @@ on the host.
   `make check`.
 - `script/docker`: builds the image, whose build runs the tests and the linter
   first. `make docker`.
+- `script/publish`: runs `script/docker` on the Docker host `DOCKER_HOST`
+  (`ssh://root@fsn1app1` unless set) and pushes the image to `PUBLISH_IMAGE`
+  (`git.eeqj.de/sneak/smallwebwaf` unless set) under four tags: the UTC date as
+  `YYYY-MM-DD`, the short commit hash, the branch name and `latest`. Refuses a
+  working tree with uncommitted changes. `make publish`.
 - `script/cibuild`: what CI runs: `script/bootstrap`, `script/check`, then the
   image build.
 - `script/precommit`: run by the git pre-commit hook; runs `script/check`.

SPEC.md

@@ -123,9 +123,7 @@ runs steps 3 to 13, and does the counting of step 15. The first step that
 produces a final answer ends the checks. Step 15 runs for each request
 `servewaf` answers with `allow`, `deny` or `close`. `rproxy`'s size and time
 limits apply to the whole exchange. Each step names the section that holds its
-rules. Each check is a heuristic. Its name, in parentheses below, is the value
-of the request log's `heuristic` field. `refusal_limit` and `byte_limits` give
-theirs in `report_heuristic` instead.
+rules.
 
 1. `GET /_smallwebwaf/healthz` is answered at once by `rproxy`, before any
    check, so a health checker never needs an exemption and is never refused.
@@ -143,9 +141,9 @@ theirs in `report_heuristic` instead.
       around its address, a /64 by default, since one abuser usually controls a
       whole /64. A client is therefore one IPv4 address or one IPv6 group.
 3. The static lists. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 12, the
-   byte limits and the error burst. A client in `SWWAF_DENY_NETS` is refused
-   (`deny_nets`); a client in both lists is let through.
-4. Bans. A client under a ban is refused (`active_ban`; see "Bans").
+   byte limits and the error burst. A client in `SWWAF_DENY_NETS` is refused; a
+   client in both lists is let through.
+4. Bans. A client under a ban is refused (see "Bans").
 5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
    and country"). The request goes on at once. It waits for the first GeoJS
    answer, up to `SWWAF_LOOKUP_TIMEOUT`, only when one of these needs it:
@@ -155,26 +153,22 @@ theirs in `report_heuristic` instead.
 
     A request that ends before the answer comes is logged without it.
 
-6. The country lists (`countries`; see "Country lists").
-7. The reputation sources (see "Reputation"), in this order:
-    - A client on a fetched blocklist gets `SWWAF_BLOCKLIST_ACTION`
-      (`blocklists`).
-    - A client on the CrowdSec decision list is banned (`crowdsec`).
-    - A client listed by a DNS blocklist gets `SWWAF_REPUTATION_ACTION`
-      (`dnsbl`).
-    - A client scored at least `SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB gets
-      `SWWAF_REPUTATION_ACTION` (`abuseipdb`).
+6. The country lists (see "Country lists").
+7. The reputation sources (see "Reputation"). A client on a fetched blocklist
+   gets `SWWAF_BLOCKLIST_ACTION`. A client on the CrowdSec decision list is
+   banned. A client listed by a DNS blocklist, or scored at least
+   `SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB, gets `SWWAF_REPUTATION_ACTION`.
 8. The client's limit percentage (see "Biased thresholds").
-9. Rate limiting (`rate_limits`; see "Request rate limits, per client"). A
-   request over a limit times the percentage is refused and bans the client.
-10. The trap paths (`trap_paths`; see "Attack detection"). A request for one is
-    refused and bans the client.
-11. The rule files (`rule_files`; see "Rule files"). Matching stops at the first
-    rule that refuses.
-12. The header limits (`header_limits`; see "Answering in time"), and then the
-    Core Rule Set (`core_rule_set`; see "Attack detection"). A Core Rule Set
-    match is refused with `403` while `SWWAF_WAF_MODE` is `block`, and only
-    logged and alerted while it is `detect`.
+9. Rate limiting (see "Request rate limits, per client"). A request over a limit
+   times the percentage is refused and bans the client.
+10. The trap paths (see "Attack detection"). A request for one is refused and
+    bans the client.
+11. The rule files (see "Rule files"). Matching stops at the first rule that
+    refuses.
+12. The header limits (see "Answering in time"), and then the Core Rule Set (see
+    "Attack detection"). A Core Rule Set match is refused with `403` while
+    `SWWAF_WAF_MODE` is `block`, and only logged and alerted while it is
+    `detect`.
 13. A path under `/_smallwebwaf/` is answered `404`, so that the app never sees
     that prefix.
 14. `rproxy` passes the request to `SWRP_UPSTREAM_URL`, streaming (see "Passing
@@ -188,14 +182,14 @@ theirs in `report_heuristic` instead.
       are added to the byte counters, as `SWWAF_BYTES_COUNT` says. For a
       WebSocket, what it carried each way counts once it closes. A client whose
       total passes a byte limit times the percentage has broken that limit and
-      is banned (`byte_limits`). The response in progress is not cut off.
+      is banned. The response in progress is not cut off.
     - For a request it allowed, the client's history and the anomaly counters
       are updated, and the alert thresholds are evaluated.
 
     For a request it refused, `servewaf` adds nothing to the byte limits. It
     updates the history and the anomaly counters at its decision. It counts the
-    refusals the error burst counts at the decision too (`refusal_limit`; see
-    "Attack detection").
+    refusals the error burst counts at the decision too (see "Attack
+    detection").
 
 In `SWWAF_MODE=observe`, every refusal above is logged as what would have
 happened, nothing is refused, and `servewaf` makes no ban of its own. A path

docs/request-log.md

@@ -107,34 +107,6 @@ Every one of these lines has:
 - `would_action`: in `observe` mode, what `enforce` mode would have done with a
   request it would have refused. `action` then names what was done, normally
   `forward`.
-- `heuristic`: the heuristic that refused the request, in `observe` mode too. It
-  is left out when none did.
-- `heuristic_figures`: an object of the figures that heuristic acted on. Each
-  heuristic and its figures:
-    - `deny_nets`: `netblock`, the entry of `SWWAF_DENY_NETS` the client is in;
-    - `active_ban`: `netblock`, the ban's netblock, and `ban_expires`;
-    - `countries`: `country`, and `setting`, the country list that refused it;
-    - `blocklists`: `lists`, the blocklist URLs that list the client;
-    - `crowdsec`: `scenario`, the decision's scenario, and `expires`, when the
-      decision ends;
-    - `dnsbl`: `zones`, the DNS blocklist zones that list the client, their keys
-      masked;
-    - `abuseipdb`: `score`, and `min_score`, `SWWAF_ABUSEIPDB_MIN_SCORE`;
-    - `rate_limits`: `window`, `count` and `limit`;
-    - `trap_paths`: `path`, the trap path asked for;
-    - `rule_files`: `rule_id` and `rule_action`, of the rule that refused it;
-    - `header_limits`: `fields`, `fields_limit`, `cookies` and `cookies_limit`;
-    - `core_rule_set`: `score`, and `threshold`, `SWWAF_WAF_ANOMALY_THRESHOLD`;
-    - `refusal_limit`: `window`, `count` and `limit`, of the error burst;
-    - `byte_limits`: `window`, `count` and `limit`, the count in bytes.
-- `report_heuristic`: the heuristic that banned the client after `servewaf`
-  answered the request, in `observe` mode too: `refusal_limit` or `byte_limits`.
-- `report_heuristic_figures`: an object of the figures that heuristic acted on,
-  as listed above.
-
-    `byte_limits` bans when the report comes, so only the `report` line names
-    it.
-
 - `limit_percent` and `limit_percent_setting`: the client's percentage of the
   rate limits and the setting that lowered it, when one did. `bytes_percent` and
   `bytes_percent_setting` do the same for the byte limits.

internal/config/config.go

@@ -7,6 +7,9 @@
 // the image runs.
 package config
 
+// FIXME: the rproxy config is in rproxy.go, the servewaf is in config.go -
+// wrong and inconsistent.
+
 import (
 	"crypto/x509"
 	"errors"
@@ -363,13 +366,16 @@ var (
 		"is not a header name such as accept-language")
 	errHeaderTakenOut = errors.New(
 		"is taken out of every request by Go's HTTP server, so it can never " +
-			"be logged")
+			"be logged",
+	)
 	errOnBothLists     = errors.New("is in SWWAF_DENIED_COUNTRIES too")
 	errNotLookupSource = errors.New("is not geojs, file or off")
 	errNeedsLookups    = errors.New("it needs each client looked up")
-	errNeedsDBPath     = errors.New("it names the file to look clients up in")
-	errDBPathUnused    = errors.New("only file reads it")
-	errOver64K         = errors.New(
+	errNeedsDBPath     = errors.New(
+		"it names the file to look clients up in",
+	)
+	errDBPathUnused = errors.New("only file reads it")
+	errOver64K      = errors.New(
 		"is more than 64K, the most servewaf gets of a body")
 	errNotCommand           = errors.New("is not rproxy or servewaf")
 	errNotDurationAboveZero = errors.New(
@@ -394,22 +400,29 @@ var (
 	errNotPathPrefix = errors.New(
 		"is not a path prefix starting with /, such as /assets/")
 	errNotTrapPath = errors.New(
-		"is not a path starting with / and without a ?, such as /wp-login.php")
+		"is not a path starting with / and without a ?, such as /wp-login.php",
+	)
 	errNotWAFMode       = errors.New("is not off, detect or block")
 	errNotParanoiaLevel = errors.New("is not a paranoia level, from 1 to 4")
 	errNotRuleID        = errors.New(
-		"is not the id of a Core Rule Set rule, a whole number such as 942100")
+		"is not the id of a Core Rule Set rule, a whole number such as 942100",
+	)
 	errSetupRuleID = errors.New(
 		"is from 900000 to 900999, the ids of the rules that set the Core Rule Set " +
-			"up and of smallwebwaf's own, which cannot be switched off")
+			"up and of smallwebwaf's own, which cannot be switched off",
+	)
 	errNotBoolean      = errors.New("is not true or false")
 	errNotLogRemoteURL = errors.New(
 		"is not syslog+udp, syslog+tcp or syslog+tls with a host and a port, " +
-			"and nothing more, such as syslog+tls://logs.example:6514")
+			"and nothing more, such as syslog+tls://logs.example:6514",
+	)
 	errNoCertificate = errors.New("holds no PEM certificate")
-	errNotFacility   = errors.New("is not a syslog facility such as local0 or daemon")
-	errNotAppName    = errors.New(
-		"is not 1 to 48 printable ASCII characters without a space, such as gitea")
+	errNotFacility   = errors.New(
+		"is not a syslog facility such as local0 or daemon",
+	)
+	errNotAppName = errors.New(
+		"is not 1 to 48 printable ASCII characters without a space, such as gitea",
+	)
 	errSetTwice      = errors.New("set only one of them")
 	errNotWebhookURL = errors.New(
 		"is not an http or https URL without a user or a fragment, " +
@@ -418,35 +431,50 @@ var (
 		"is not a header name followed by : and the header's value, " +
 			"such as Authorization:Bearer <token>")
 	errControlCharacter = errors.New(
-		"holds a control character, such as the carriage return of a Windows line end")
+		"holds a control character, such as the carriage return of a Windows line end",
+	)
 	errNotAlertEvent = errors.New(
 		"is not ban, permanent_ban, waf_block, anomaly, reputation_hit, " +
 			"source_failure or file_error")
-	errNotNumberOrOff = errors.New("is not a whole number above zero, such as 60, or off")
+	errNotNumberOrOff = errors.New(
+		"is not a whole number above zero, such as 60, or off",
+	)
 	errNotUTF8        = errors.New("is not valid UTF-8")
 	errNotASN         = errors.New("is not an AS number such as AS64496")
 	errNotPercentItem = errors.New(
 		"is not a code, : and a percentage, such as AS64496:50 or cn:25")
-	errNotPercent  = errors.New("is not a percentage, a whole number from 0 to 100")
+	errNotPercent = errors.New(
+		"is not a percentage, a whole number from 0 to 100",
+	)
 	errListedTwice = errors.New("is listed twice")
 	errNotListURL  = errors.New(
 		"is not an http or https URL without a user or a fragment, " +
 			"such as https://www.spamhaus.org/drop/drop.txt")
 	errInBlocklistURLs = errors.New("is in SWWAF_BLOCKLIST_URLS too")
-	errNotAnHourOrMore = errors.New("is not a duration of 1h or more, such as 24h")
-	errNotAction       = errors.New(
+	errNotAnHourOrMore = errors.New(
+		"is not a duration of 1h or more, such as 24h",
+	)
+	errNotAction = errors.New(
 		"is not deny, limit:<percent> such as limit:25, or log")
-	errNotZone     = errors.New("is not a DNS zone such as dnsbl.dronebl.org")
-	errZoneTooLong = errors.New("is longer than 189 characters, too long for the " +
-		"names IPv6 clients are asked about by")
-	errNotResolver = errors.New("is not an IP address with an optional port, " +
-		"such as 192.0.2.53 or [2001:db8::53]:5353")
+	errNotZone = errors.New(
+		"is not a DNS zone such as dnsbl.dronebl.org",
+	)
+	errZoneTooLong = errors.New(
+		"is longer than 189 characters, too long for the " +
+			"names IPv6 clients are asked about by",
+	)
+	errNotResolver = errors.New(
+		"is not an IP address with an optional port, " +
+			"such as 192.0.2.53 or [2001:db8::53]:5353",
+	)
 	errNotLAPIURL = errors.New(
 		"is not an http or https URL without a user or a fragment, " +
 			"such as http://172.17.0.1:8080")
 	errNeedsLAPIKey  = errors.New("the engine answers no request without it")
-	errLAPIKeyUnused = errors.New("it is sent only to the engine at that URL")
-	errAnotherList   = errors.New(
+	errLAPIKeyUnused = errors.New(
+		"it is sent only to the engine at that URL",
+	)
+	errAnotherList = errors.New(
 		"is in SWWAF_BLOCKLIST_URLS or is SWWAF_ASN_LIMIT_PERCENT_URL too")
 )
 
@@ -457,97 +485,208 @@ var (
 // setting that is set but invalid is an error that names it.
 //
 //nolint:funlen // one line for each setting, a list that grows with them
-func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
+func FromEnvironment(
+	lookupEnv func(string) (string, bool),
+) (*Config, error) {
 	env := &environment{lookupEnv: lookupEnv}
 	cfg := &Config{
-		ListenAddr:           env.address("SWWAF_LISTEN_ADDR"),
-		InstanceName:         env.instanceName(),
-		Observe:              env.observe("SWWAF_MODE", "enforce"),
-		IPv6GroupPrefix:      env.ipv6GroupPrefix("SWWAF_IPV6_GROUP_PREFIX", "64"),
-		MaxTrackedClients:    env.numberNotOff("SWWAF_MAX_TRACKED_CLIENTS", "20000"),
-		AllowNets:            env.netblocks("SWWAF_ALLOW_NETS", ""),
-		RateLimitExemptNets:  env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
-		DenyNets:             env.netblocks("SWWAF_DENY_NETS", ""),
-		RateLimitPerMinute:   env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "120"),
-		RateLimitPerHour:     env.count("SWWAF_RATE_LIMIT_PER_HOUR", "1500"),
-		RateLimitPerDay:      env.count("SWWAF_RATE_LIMIT_PER_DAY", "6000"),
-		RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
-		BytesLimitPerMinute:  env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "off"),
-		BytesLimitPerHour:    env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "50G"),
-		BytesLimitPerDay:     env.size("SWWAF_BYTES_LIMIT_PER_DAY", "100G"),
-		BytesCount:           env.bytesCount("SWWAF_BYTES_COUNT", "both"),
-		LookupSource:         env.lookupSource("SWWAF_LOOKUP_SOURCE", "geojs"),
-		LookupDBPath:         env.value("SWWAF_LOOKUP_DB_PATH", ""),
-		LookupTimeout:        env.durationNotOff("SWWAF_LOOKUP_TIMEOUT", "800ms"),
-		AddLookupHeaders:     env.boolean("SWWAF_ADD_LOOKUP_HEADERS", "false"),
-		DeniedCountries:      env.countries("SWWAF_DENIED_COUNTRIES", ""),
+		ListenAddr:   env.address("SWWAF_LISTEN_ADDR"),
+		InstanceName: env.instanceName(),
+		Observe:      env.observe("SWWAF_MODE", "enforce"),
+		IPv6GroupPrefix: env.ipv6GroupPrefix(
+			"SWWAF_IPV6_GROUP_PREFIX",
+			"64",
+		),
+		MaxTrackedClients: env.numberNotOff(
+			"SWWAF_MAX_TRACKED_CLIENTS",
+			"20000",
+		),
+		AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""),
+		RateLimitExemptNets: env.netblocks(
+			"SWWAF_RATE_LIMIT_EXEMPT_NETS",
+			"",
+		),
+		DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
+		RateLimitPerMinute: env.count(
+			"SWWAF_RATE_LIMIT_PER_MINUTE",
+			"120",
+		),
+		RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "1500"),
+		RateLimitPerDay:  env.count("SWWAF_RATE_LIMIT_PER_DAY", "6000"),
+		RateLimitExemptPaths: env.pathPrefixes(
+			"SWWAF_RATE_LIMIT_EXEMPT_PATHS",
+			"",
+		),
+		BytesLimitPerMinute: env.size(
+			"SWWAF_BYTES_LIMIT_PER_MINUTE",
+			"off",
+		),
+		BytesLimitPerHour: env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "50G"),
+		BytesLimitPerDay:  env.size("SWWAF_BYTES_LIMIT_PER_DAY", "100G"),
+		BytesCount:        env.bytesCount("SWWAF_BYTES_COUNT", "both"),
+		LookupSource: env.lookupSource(
+			"SWWAF_LOOKUP_SOURCE",
+			"geojs",
+		),
+		LookupDBPath: env.value("SWWAF_LOOKUP_DB_PATH", ""),
+		LookupTimeout: env.durationNotOff(
+			"SWWAF_LOOKUP_TIMEOUT",
+			"800ms",
+		),
+		AddLookupHeaders: env.boolean(
+			"SWWAF_ADD_LOOKUP_HEADERS",
+			"false",
+		),
+		DeniedCountries: env.countries("SWWAF_DENIED_COUNTRIES", ""),
 		ExclusivelyAllowedCountries: env.countries(
 			"SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES", ""),
-		ASNLimitPercent:      env.percents("SWWAF_ASN_LIMIT_PERCENT", ParseASN),
-		CountryLimitPercent:  env.percents("SWWAF_COUNTRY_LIMIT_PERCENT", parseCountry),
-		ASNBytesPercent:      env.percents("SWWAF_ASN_BYTES_PERCENT", ParseASN),
-		CountryBytesPercent:  env.percents("SWWAF_COUNTRY_BYTES_PERCENT", parseCountry),
-		UnknownLimitPercent:  env.percent("SWWAF_UNKNOWN_LIMIT_PERCENT", "100"),
-		ASNLimitPercentURL:   env.listURL("SWWAF_ASN_LIMIT_PERCENT_URL"),
-		BlocklistURLs:        env.listURLs("SWWAF_BLOCKLIST_URLS"),
-		BlocklistRefresh:     env.refresh("SWWAF_BLOCKLIST_REFRESH", "24h"),
-		DNSBLZones:           env.zones("SWWAF_DNSBL_ZONES"),
-		DNSBLResolver:        env.resolver("SWWAF_DNSBL_RESOLVER"),
-		AbuseIPDBKey:         env.secret("SWWAF_ABUSEIPDB_KEY"),
-		AbuseIPDBMinScore:    env.percent("SWWAF_ABUSEIPDB_MIN_SCORE", "75"),
-		AbuseIPDBDailyBudget: env.numberNotOff("SWWAF_ABUSEIPDB_DAILY_BUDGET", "900"),
-		ReputationCacheTTL:   env.durationNotOff("SWWAF_REPUTATION_CACHE_TTL", "24h"),
-		ReputationTimeout:    env.durationNotOff("SWWAF_REPUTATION_TIMEOUT", "2s"),
-		CrowdSecDecisionsURL: env.crowdSecDecisionsURL("SWWAF_CROWDSEC_LAPI_URL"),
-		CrowdSecKey:          env.secret("SWWAF_CROWDSEC_LAPI_KEY"),
-		BanResponse:          env.banResponse("SWWAF_BAN_RESPONSE", "403"),
-		LimitBanDuration:     env.durationNotOff("SWWAF_LIMIT_BAN_DURATION", "1h"),
-		LimitBanRepeatWindow: env.durationNotOff("SWWAF_LIMIT_BAN_REPEAT_WINDOW", "24h"),
-		MaxBanDuration:       env.durationNotOff("SWWAF_MAX_BAN_DURATION", "7d"),
-		AttackBanDuration:    env.durationNotOff("SWWAF_ATTACK_BAN_DURATION", "7d"),
-		MaxBans:              env.numberNotOff("SWWAF_MAX_BANS", "5000"),
-		BanScopeV4Prefix:     env.v4Prefix("SWWAF_BAN_SCOPE_V4_PREFIX", "32"),
-		StateDir:             env.absolutePath("SWWAF_STATE_DIR", "/var/lib/smallwebwaf"),
-		StateWriteDelay:      env.durationNotOff("SWWAF_STATE_WRITE_DELAY", "10s"),
-		StateCounterInterval: env.durationNotOff("SWWAF_STATE_COUNTER_INTERVAL", "15m"),
-		LogRequestHeaders: env.headerNames("SWWAF_LOG_REQUEST_HEADERS",
-			"accept,accept-language,accept-encoding,content-type,origin,range"),
-		LogLevel:            env.logLevel("SWWAF_LOG_LEVEL", "info"),
-		AdminToken:          env.token("SWWAF_ADMIN_TOKEN"),
-		MetricsToken:        env.token("SWWAF_METRICS_TOKEN"),
-		MetricsTopN:         env.numberNotOff("SWWAF_METRICS_TOP_N", "50"),
-		RulesDir:            env.value("SWWAF_RULES_DIR", "/etc/smallwebwaf/rules.d"),
-		RulesEnabled:        env.boolean("SWWAF_RULES_ENABLED", "true"),
-		WAFMode:             env.wafMode("SWWAF_WAF_MODE", WAFModeBlock),
-		WAFParanoiaLevel:    env.paranoiaLevel("SWWAF_WAF_PARANOIA_LEVEL", "1"),
-		WAFAnomalyThreshold: env.numberOrOff("SWWAF_WAF_ANOMALY_THRESHOLD", "5"),
+		ASNLimitPercent: env.percents(
+			"SWWAF_ASN_LIMIT_PERCENT",
+			ParseASN,
+		),
+		CountryLimitPercent: env.percents(
+			"SWWAF_COUNTRY_LIMIT_PERCENT",
+			parseCountry,
+		),
+		ASNBytesPercent: env.percents(
+			"SWWAF_ASN_BYTES_PERCENT",
+			ParseASN,
+		),
+		CountryBytesPercent: env.percents(
+			"SWWAF_COUNTRY_BYTES_PERCENT",
+			parseCountry,
+		),
+		UnknownLimitPercent: env.percent(
+			"SWWAF_UNKNOWN_LIMIT_PERCENT",
+			"100",
+		),
+		ASNLimitPercentURL: env.listURL("SWWAF_ASN_LIMIT_PERCENT_URL"),
+		BlocklistURLs:      env.listURLs("SWWAF_BLOCKLIST_URLS"),
+		BlocklistRefresh:   env.refresh("SWWAF_BLOCKLIST_REFRESH", "24h"),
+		DNSBLZones:         env.zones("SWWAF_DNSBL_ZONES"),
+		DNSBLResolver:      env.resolver("SWWAF_DNSBL_RESOLVER"),
+		AbuseIPDBKey:       env.secret("SWWAF_ABUSEIPDB_KEY"),
+		AbuseIPDBMinScore:  env.percent("SWWAF_ABUSEIPDB_MIN_SCORE", "75"),
+		AbuseIPDBDailyBudget: env.numberNotOff(
+			"SWWAF_ABUSEIPDB_DAILY_BUDGET",
+			"900",
+		),
+		ReputationCacheTTL: env.durationNotOff(
+			"SWWAF_REPUTATION_CACHE_TTL",
+			"24h",
+		),
+		ReputationTimeout: env.durationNotOff(
+			"SWWAF_REPUTATION_TIMEOUT",
+			"2s",
+		),
+		CrowdSecDecisionsURL: env.crowdSecDecisionsURL(
+			"SWWAF_CROWDSEC_LAPI_URL",
+		),
+		CrowdSecKey: env.secret("SWWAF_CROWDSEC_LAPI_KEY"),
+		BanResponse: env.banResponse("SWWAF_BAN_RESPONSE", "403"),
+		LimitBanDuration: env.durationNotOff(
+			"SWWAF_LIMIT_BAN_DURATION",
+			"1h",
+		),
+		LimitBanRepeatWindow: env.durationNotOff(
+			"SWWAF_LIMIT_BAN_REPEAT_WINDOW",
+			"24h",
+		),
+		MaxBanDuration: env.durationNotOff(
+			"SWWAF_MAX_BAN_DURATION",
+			"7d",
+		),
+		AttackBanDuration: env.durationNotOff(
+			"SWWAF_ATTACK_BAN_DURATION",
+			"7d",
+		),
+		MaxBans: env.numberNotOff("SWWAF_MAX_BANS", "5000"),
+		BanScopeV4Prefix: env.v4Prefix(
+			"SWWAF_BAN_SCOPE_V4_PREFIX",
+			"32",
+		),
+		StateDir: env.absolutePath(
+			"SWWAF_STATE_DIR",
+			"/var/lib/smallwebwaf",
+		),
+		StateWriteDelay: env.durationNotOff(
+			"SWWAF_STATE_WRITE_DELAY",
+			"10s",
+		),
+		StateCounterInterval: env.durationNotOff(
+			"SWWAF_STATE_COUNTER_INTERVAL",
+			"15m",
+		),
+		LogRequestHeaders: env.headerNames(
+			"SWWAF_LOG_REQUEST_HEADERS",
+			"accept,accept-language,accept-encoding,content-type,origin,range",
+		),
+		LogLevel:     env.logLevel("SWWAF_LOG_LEVEL", "info"),
+		AdminToken:   env.token("SWWAF_ADMIN_TOKEN"),
+		MetricsToken: env.token("SWWAF_METRICS_TOKEN"),
+		MetricsTopN:  env.numberNotOff("SWWAF_METRICS_TOP_N", "50"),
+		RulesDir: env.value(
+			"SWWAF_RULES_DIR",
+			"/etc/smallwebwaf/rules.d",
+		),
+		RulesEnabled: env.boolean("SWWAF_RULES_ENABLED", "true"),
+		WAFMode:      env.wafMode("SWWAF_WAF_MODE", WAFModeBlock),
+		WAFParanoiaLevel: env.paranoiaLevel(
+			"SWWAF_WAF_PARANOIA_LEVEL",
+			"1",
+		),
+		WAFAnomalyThreshold: env.numberOrOff(
+			"SWWAF_WAF_ANOMALY_THRESHOLD",
+			"5",
+		),
 		WAFDisabledRules: env.ruleIDs("SWWAF_WAF_DISABLED_RULES",
 			"920340,920420,920440,920640,930130,930140"),
-		WAFExemptPaths:       env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
-		WAFBodyLimit:         env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
-		TrapPaths:            env.trapPaths("SWWAF_TRAP_PATHS"),
-		ErrorBurstThreshold:  env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
-		LogRemoteURL:         env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
-		LogRemoteTLSCAs:      env.certificates("SWWAF_LOG_REMOTE_TLS_CA_FILE"),
-		LogRemoteBuffer:      env.numberNotOff("SWWAF_LOG_REMOTE_BUFFER", "10000"),
-		LogRemoteFacility:    env.facility("SWWAF_LOG_REMOTE_FACILITY", "local0"),
-		AlertWebhookURL:      env.webhookURL("SWWAF_ALERT_WEBHOOK_URL"),
-		AlertWebhookHeaders:  env.webhookHeaders("SWWAF_ALERT_WEBHOOK_HEADERS"),
-		AlertSlackWebhookURL: env.webhookURL("SWWAF_ALERT_SLACK_WEBHOOK_URL"),
-		AlertNtfyURL:         env.webhookURL("SWWAF_ALERT_NTFY_URL"),
-		AlertNtfyToken:       env.secret("SWWAF_ALERT_NTFY_TOKEN"),
+		WAFExemptPaths:      env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
+		WAFBodyLimit:        env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
+		TrapPaths:           env.trapPaths("SWWAF_TRAP_PATHS"),
+		ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
+		LogRemoteURL:        env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
+		LogRemoteTLSCAs: env.certificates(
+			"SWWAF_LOG_REMOTE_TLS_CA_FILE",
+		),
+		LogRemoteBuffer: env.numberNotOff(
+			"SWWAF_LOG_REMOTE_BUFFER",
+			"10000",
+		),
+		LogRemoteFacility: env.facility(
+			"SWWAF_LOG_REMOTE_FACILITY",
+			"local0",
+		),
+		AlertWebhookURL: env.webhookURL("SWWAF_ALERT_WEBHOOK_URL"),
+		AlertWebhookHeaders: env.webhookHeaders(
+			"SWWAF_ALERT_WEBHOOK_HEADERS",
+		),
+		AlertSlackWebhookURL: env.webhookURL(
+			"SWWAF_ALERT_SLACK_WEBHOOK_URL",
+		),
+		AlertNtfyURL:   env.webhookURL("SWWAF_ALERT_NTFY_URL"),
+		AlertNtfyToken: env.secret("SWWAF_ALERT_NTFY_TOKEN"),
 		AlertEvents: env.alertEvents("SWWAF_ALERT_EVENTS",
 			strings.Join(alerts.Events(), ",")),
-		AlertCooldown:      env.duration("SWWAF_ALERT_COOLDOWN", "15m"),
-		AlertMaxPerHour:    env.numberOrOff("SWWAF_ALERT_MAX_PER_HOUR", "60"),
-		AnomalyClient:      env.thresholds("SWWAF_ANOMALY_CLIENT_"),
-		AnomalyNet:         env.thresholds("SWWAF_ANOMALY_NET_"),
-		AnomalyASN:         env.thresholds("SWWAF_ANOMALY_ASN_"),
-		AnomalyTotal:       env.thresholds("SWWAF_ANOMALY_TOTAL_"),
-		AnomalyWatch:       env.thresholds("SWWAF_WATCH_"),
-		AnomalyNetV4Prefix: env.v4Prefix("SWWAF_ANOMALY_NET_V4_PREFIX", "24"),
-		AnomalyNetV6Prefix: env.v6Prefix("SWWAF_ANOMALY_NET_V6_PREFIX", "48"),
-		WatchNets:          env.namedNetblocks("SWWAF_WATCH_NETS"),
+		AlertCooldown: env.duration("SWWAF_ALERT_COOLDOWN", "15m"),
+		AlertMaxPerHour: env.numberOrOff(
+			"SWWAF_ALERT_MAX_PER_HOUR",
+			"60",
+		),
+		AnomalyClient: env.thresholds("SWWAF_ANOMALY_CLIENT_"),
+		AnomalyNet:    env.thresholds("SWWAF_ANOMALY_NET_"),
+		AnomalyASN:    env.thresholds("SWWAF_ANOMALY_ASN_"),
+		AnomalyTotal:  env.thresholds("SWWAF_ANOMALY_TOTAL_"),
+		AnomalyWatch:  env.thresholds("SWWAF_WATCH_"),
+		AnomalyNetV4Prefix: env.v4Prefix(
+			"SWWAF_ANOMALY_NET_V4_PREFIX",
+			"24",
+		),
+		AnomalyNetV6Prefix: env.v6Prefix(
+			"SWWAF_ANOMALY_NET_V6_PREFIX",
+			"48",
+		),
+		WatchNets: env.namedNetblocks("SWWAF_WATCH_NETS"),
 	}
 
 	cfg.LogRemoteAppName = env.appName("SWWAF_LOG_REMOTE_APP_NAME",
@@ -584,14 +723,20 @@ func Command(lookupEnv func(string) (string, bool)) (string, error) {
 	case "servewaf":
 		return command, nil
 	default:
-		return "", fmt.Errorf("SMALLWEBWAF_COMMAND: %q %w", command, errNotCommand)
+		return "", fmt.Errorf(
+			"SMALLWEBWAF_COMMAND: %q %w",
+			command,
+			errNotCommand,
+		)
 	}
 }
 
 // ServeWAFListenAddr reads only SWWAF_LISTEN_ADDR, which may be given as a
 // file, as FromEnvironment does. The health check in servewaf's container needs
 // no other setting, so it reads no other, nor a file that another names.
-func ServeWAFListenAddr(lookupEnv func(string) (string, bool)) (string, error) {
+func ServeWAFListenAddr(
+	lookupEnv func(string) (string, bool),
+) (string, error) {
 	env := &environment{lookupEnv: lookupEnv}
 	listenAddr := env.address("SWWAF_LISTEN_ADDR")
 
@@ -653,14 +798,19 @@ func (e *environment) lookup(name string) (string, bool) {
 	}
 
 	if set {
-		e.check(name, fmt.Errorf("is set, and so is %s; %w", fileName, errSetTwice))
+		e.check(
+			name,
+			fmt.Errorf("is set, and so is %s; %w", fileName, errSetTwice),
+		)
 
 		return value, set
 	}
 
 	e.settings = append(e.settings, slog.String(fileName, path))
 
-	contents, err := os.ReadFile(path) //nolint:gosec // a file the admin names
+	contents, err := os.ReadFile(
+		path,
+	) //nolint:gosec // a file the admin names
 	if err != nil {
 		e.check(fileName, fmt.Errorf("cannot be read: %w", err))
 
@@ -911,7 +1061,10 @@ func (e *environment) zones(name string) []string {
 		logged[i] = MaskZoneKey(zone)
 	}
 
-	e.settings = append(e.settings, slog.String(name, strings.Join(logged, ",")))
+	e.settings = append(
+		e.settings,
+		slog.String(name, strings.Join(logged, ",")),
+	)
 
 	return zones
 }
@@ -963,10 +1116,15 @@ func (e *environment) checkLookupDBPath(cfg *Config) {
 	switch {
 	case cfg.LookupSource == fileSource && cfg.LookupDBPath == "":
 		e.check("SWWAF_LOOKUP_SOURCE", fmt.Errorf(
-			"is file while SWWAF_LOOKUP_DB_PATH is unset; %w", errNeedsDBPath))
+			"is file while SWWAF_LOOKUP_DB_PATH is unset; %w",
+			errNeedsDBPath,
+		))
 	case cfg.LookupSource != fileSource && cfg.LookupDBPath != "":
 		e.check("SWWAF_LOOKUP_DB_PATH", fmt.Errorf(
-			"is set while SWWAF_LOOKUP_SOURCE is %s; %w", cfg.LookupSource, errDBPathUnused))
+			"is set while SWWAF_LOOKUP_SOURCE is %s; %w",
+			cfg.LookupSource,
+			errDBPathUnused,
+		))
 	}
 }
 
@@ -1007,8 +1165,11 @@ func (e *environment) checkCountriesAndLookups(cfg *Config) {
 		{"SWWAF_ANOMALY_ASN_BYTES_PER_HOUR", cfg.AnomalyASN.BytesPerHour > 0},
 	} {
 		if setting.set {
-			e.check(setting.name, fmt.Errorf("is set while SWWAF_LOOKUP_SOURCE is off; %w",
-				errNeedsLookups))
+			e.check(
+				setting.name,
+				fmt.Errorf("is set while SWWAF_LOOKUP_SOURCE is off; %w",
+					errNeedsLookups),
+			)
 		}
 	}
 }
@@ -1031,14 +1192,21 @@ func (e *environment) checkCrowdSec(cfg *Config) {
 	switch {
 	case decisionsURL != "" && cfg.CrowdSecKey == "":
 		e.check("SWWAF_CROWDSEC_LAPI_URL", fmt.Errorf(
-			"is set while SWWAF_CROWDSEC_LAPI_KEY is unset; %w", errNeedsLAPIKey))
+			"is set while SWWAF_CROWDSEC_LAPI_KEY is unset; %w",
+			errNeedsLAPIKey,
+		))
 	case decisionsURL == "" && cfg.CrowdSecKey != "":
 		e.check("SWWAF_CROWDSEC_LAPI_KEY", fmt.Errorf(
-			"is set while SWWAF_CROWDSEC_LAPI_URL is unset; %w", errLAPIKeyUnused))
+			"is set while SWWAF_CROWDSEC_LAPI_URL is unset; %w",
+			errLAPIKeyUnused,
+		))
 	case decisionsURL != "" && (slices.Contains(cfg.BlocklistURLs, decisionsURL) ||
 		decisionsURL == cfg.ASNLimitPercentURL):
-		e.check("SWWAF_CROWDSEC_LAPI_URL", fmt.Errorf("gives the decision list %q, which %w",
-			decisionsURL, errAnotherList))
+		e.check(
+			"SWWAF_CROWDSEC_LAPI_URL",
+			fmt.Errorf("gives the decision list %q, which %w",
+				decisionsURL, errAnotherList),
+		)
 	}
 }
 
@@ -1053,7 +1221,9 @@ func (e *environment) headerNames(name, defaultValue string) []string {
 
 // durationNotOff reads a setting that is a duration and, unlike a
 // timeout, cannot be off.
-func (e *environment) durationNotOff(name, defaultValue string) time.Duration {
+func (e *environment) durationNotOff(
+	name, defaultValue string,
+) time.Duration {
 	duration, err := ParseDurationNotOff(e.value(name, defaultValue))
 	e.check(name, err)
 
@@ -1233,7 +1403,10 @@ func (e *environment) instanceName() string {
 
 	value := e.value("SWWAF_INSTANCE_NAME", hostname)
 	if !utf8.ValidString(value) {
-		e.check("SWWAF_INSTANCE_NAME", fmt.Errorf("%q %w", value, errNotUTF8))
+		e.check(
+			"SWWAF_INSTANCE_NAME",
+			fmt.Errorf("%q %w", value, errNotUTF8),
+		)
 	}
 
 	return value
@@ -1242,7 +1415,10 @@ func (e *environment) instanceName() string {
 // appName reads the setting that is the APP-NAME of the records the log
 // lines are sent in, by default the instance name. Its value is checked
 // when it is set, and, while lines are sent, when it is the instance name.
-func (e *environment) appName(name, instanceName string, sending bool) string {
+func (e *environment) appName(
+	name, instanceName string,
+	sending bool,
+) string {
 	value, set := e.lookup(name)
 	if !set {
 		value = instanceName
@@ -1255,8 +1431,14 @@ func (e *environment) appName(name, instanceName string, sending bool) string {
 	case set:
 		e.check(name, fmt.Errorf("%q %w", value, errNotAppName))
 	case sending:
-		e.check(name, fmt.Errorf("is unset, and SWWAF_INSTANCE_NAME %q, its default, %w",
-			value, errNotAppName))
+		e.check(
+			name,
+			fmt.Errorf(
+				"is unset, and SWWAF_INSTANCE_NAME %q, its default, %w",
+				value,
+				errNotAppName,
+			),
+		)
 	}
 
 	return value
@@ -1265,11 +1447,16 @@ func (e *environment) appName(name, instanceName string, sending bool) string {
 // checkInstanceNameForNtfy refuses an instance name that holds a control
 // character while ntfySet, SWWAF_ALERT_NTFY_URL being set: ntfy is sent
 // the instance name in a header, which cannot hold one.
-func (e *environment) checkInstanceNameForNtfy(instanceName string, ntfySet bool) {
+func (e *environment) checkInstanceNameForNtfy(
+	instanceName string,
+	ntfySet bool,
+) {
 	if ntfySet && strings.ContainsFunc(instanceName, unicode.IsControl) {
 		e.check("SWWAF_INSTANCE_NAME", fmt.Errorf(
 			"%q %w, and is sent to ntfy in a header while SWWAF_ALERT_NTFY_URL is set",
-			instanceName, errControlCharacter))
+			instanceName,
+			errControlCharacter,
+		))
 	}
 }
 
@@ -1589,7 +1776,10 @@ func parseNamedNetblocks(value string) ([]anomaly.NamedNetblock, error) {
 			return nil, fmt.Errorf("%q %w", name, errListedTwice)
 		}
 
-		named = append(named, anomaly.NamedNetblock{Name: name, Netblock: netblock})
+		named = append(
+			named,
+			anomaly.NamedNetblock{Name: name, Netblock: netblock},
+		)
 	}
 
 	return named, nil
@@ -1828,8 +2018,11 @@ func parseHeaderNames(value string) ([]string, error) {
 		header := strings.ToLower(item)
 		switch header {
 		case "host":
-			return nil, fmt.Errorf("%q %w; the request's host is the field host",
-				item, errHeaderTakenOut)
+			return nil, fmt.Errorf(
+				"%q %w; the request's host is the field host",
+				item,
+				errHeaderTakenOut,
+			)
 		case "transfer-encoding":
 			return nil, fmt.Errorf("%q %w", item, errHeaderTakenOut)
 		}
@@ -1866,9 +2059,11 @@ func parseUpstreamURL(value string) (*url.URL, error) {
 	}
 
 	onlySchemeAndHost := (upstream.Scheme == "http" || upstream.Scheme == "https") &&
-		upstream.Hostname() != "" && upstream.User == nil && upstream.Opaque == "" &&
+		upstream.Hostname() != "" && upstream.User == nil &&
+		upstream.Opaque == "" &&
 		(upstream.Path == "" || upstream.Path == "/") &&
-		upstream.RawQuery == "" && upstream.Fragment == ""
+		upstream.RawQuery == "" &&
+		upstream.Fragment == ""
 	if !onlySchemeAndHost {
 		return nil, fmt.Errorf("%q %w", value, errNotUpstreamURL)
 	}
@@ -1892,7 +2087,11 @@ func parseLogRemoteURL(value string) (*url.URL, error) {
 		return nil, fmt.Errorf("%q %w", value, errNotLogRemoteURL)
 	}
 
-	schemes := []string{remotelog.SchemeUDP, remotelog.SchemeTCP, remotelog.SchemeTLS}
+	schemes := []string{
+		remotelog.SchemeUDP,
+		remotelog.SchemeTCP,
+		remotelog.SchemeTLS,
+	}
 	port, err := strconv.ParseUint(remote.Port(), 10, 16)
 
 	onlySchemeHostAndPort := slices.Contains(schemes, remote.Scheme) &&
@@ -1955,7 +2154,9 @@ func isHTTPURL(u *url.URL) bool {
 
 	return (u.Scheme == "http" || u.Scheme == "https") && u.Hostname() != "" &&
 		(u.Port() == "" || (err == nil && port != 0)) &&
-		u.User == nil && u.Opaque == "" && u.Fragment == ""
+		u.User == nil &&
+		u.Opaque == "" &&
+		u.Fragment == ""
 }
 
 // isListURL reports whether value is a URL a list can be fetched from, as
@@ -2022,9 +2223,12 @@ func parseZones(value string) ([]string, error) {
 
 	for i, zone := range zones {
 		shown := MaskZoneKey(zone)
-		listedBefore := slices.ContainsFunc(zones[:i], func(earlier string) bool {
-			return strings.EqualFold(MaskZoneKey(earlier), shown)
-		})
+		listedBefore := slices.ContainsFunc(
+			zones[:i],
+			func(earlier string) bool {
+				return strings.EqualFold(MaskZoneKey(earlier), shown)
+			},
+		)
 
 		switch {
 		case len(zone) > maxZoneLength:
@@ -2111,8 +2315,13 @@ func parseWebhookHeaders(value string) (http.Header, string, error) {
 		name, headerValue, found := strings.Cut(item, ":")
 		name = strings.TrimSpace(name)
 
-		if !found || !IsHeaderName(name) || strings.ContainsAny(headerValue, "\r\n\x00") {
-			return nil, "", fmt.Errorf("item %d %w", i+1, errNotWebhookHeader)
+		if !found || !IsHeaderName(name) ||
+			strings.ContainsAny(headerValue, "\r\n\x00") {
+			return nil, "", fmt.Errorf(
+				"item %d %w",
+				i+1,
+				errNotWebhookHeader,
+			)
 		}
 
 		headers.Add(name, strings.TrimSpace(headerValue))

internal/lookup/file.go

@@ -16,6 +16,11 @@ import (
 	"sneak.berlin/go/smallwebwaf/internal/alerts"
 )
 
+// FIXME: you can't call things just "File" - that's super vague.
+
+// FIXME: "lookup database" - in memory on disk?  you must not be so
+// ambiguous when describing things.
+
 // quietTime is how long the lookup database must go without a change
 // before it is read again, so that a file still being copied in is read
 // only once whole.
@@ -133,8 +138,11 @@ func (f *File) Watch(ctx context.Context) {
 	}
 
 	if err != nil {
-		f.params.ProcessLog.Error("cannot watch the lookup database for replacements",
-			"error", err.Error())
+		f.params.ProcessLog.Error(
+			"cannot watch the lookup database for replacements",
+			"error",
+			err.Error(),
+		)
 
 		return
 	}
@@ -194,7 +202,10 @@ func (f *File) readAgain() {
 		f.params.Alerts.Raise(alerts.Alert{
 			Event:  alerts.EventFileError,
 			Reason: kept,
-			Detail: map[string]any{"file": f.params.Path, "error": err.Error()},
+			Detail: map[string]any{
+				"file":  f.params.Path,
+				"error": err.Error(),
+			},
 		})
 		f.params.ProcessLog.Error(kept, "error", err.Error())
 
@@ -212,7 +223,11 @@ func (f *File) use(reader *maxminddb.Reader) {
 	f.lastRead = f.params.Now()
 	f.mu.Unlock()
 
-	f.params.ProcessLog.Info("read the lookup database", "file", f.params.Path)
+	f.params.ProcessLog.Info(
+		"read the lookup database",
+		"file",
+		f.params.Path,
+	)
 }
 
 // read reads the lookup database at path. The whole file is read into
@@ -221,12 +236,19 @@ func (f *File) use(reader *maxminddb.Reader) {
 func read(path string) (*maxminddb.Reader, error) {
 	data, err := os.ReadFile(path) //nolint:gosec // the file the admin names
 	if err != nil {
-		return nil, fmt.Errorf("SWWAF_LOOKUP_DB_PATH cannot be read: %w", err)
+		return nil, fmt.Errorf(
+			"SWWAF_LOOKUP_DB_PATH cannot be read: %w",
+			err,
+		)
 	}
 
 	reader, err := maxminddb.OpenBytes(data)
 	if err != nil {
-		return nil, fmt.Errorf("SWWAF_LOOKUP_DB_PATH %s is not a .mmdb file: %w", path, err)
+		return nil, fmt.Errorf(
+			"SWWAF_LOOKUP_DB_PATH %s is not a .mmdb file: %w",
+			path,
+			err,
+		)
 	}
 
 	return reader, nil

internal/requestlog/requestlog.go

@@ -19,29 +19,51 @@ const (
 	// ActionForward is a request servewaf allowed, which rproxy passes to the
 	// app.
 	ActionForward = "forward"
+
+	//FIXME: "too_large" doesn't tell a reader what that means.
+	//request_header_too_large does.  you must not use ambiguous terms.
+
 	// ActionTooLarge is a request over the header limits.
 	ActionTooLarge = "too_large"
+
 	// ActionRateLimited is a request refused because it took its client
 	// over a rate limit, which bans the client.
+
+	//FIXME: better would be "client_is_rate_limited"
 	ActionRateLimited = "rate_limited"
+
 	// ActionBanned is a request refused because a ban covers its client,
 	// or because it matched a ban rule, asked for a trap path or the
 	// CrowdSec decision list lists its client, each of which bans the
 	// client.
+
+	// FIXME: if I saw this, i would assume they are permanently banned.  is that
+	// true?  if not, there should be TemporarilyBanned and
+	// PermanentlyBanned
 	ActionBanned = "banned"
+
 	// ActionRuleBlocked is a request refused because it matched a block
 	// rule.
+
+	// FIXME: "blocked_by_block_rule"
 	ActionRuleBlocked = "rule_blocked"
+
 	// ActionWAFBlocked is a request refused because the Core Rule Set
 	// scored it at or over SWWAF_WAF_ANOMALY_THRESHOLD.
 	ActionWAFBlocked = "waf_blocked"
+
 	// ActionDenied is a request refused because its client is in
 	// SWWAF_DENY_NETS, in a blocklist while SWWAF_BLOCKLIST_ACTION is deny,
 	// or listed by a DNSBL zone, or scored a hit by AbuseIPDB, while
 	// SWWAF_REPUTATION_ACTION is deny.
 	ActionDenied = "denied"
+
 	// ActionCountryDenied is a request refused for its client's country.
 	ActionCountryDenied = "country_denied"
+
+	// FIXME: you have country_denied and denied.  why isn't it
+	// client_denied?  ALWAYS BE CONSISTENT AND UNAMBIGUOUS
+
 	// ActionAdmin is a request servewaf answered at one of its own endpoints,
 	// or a question about a path under /_smallwebwaf/.
 	ActionAdmin = "admin"
@@ -131,15 +153,6 @@ type Line struct {
 	// ActionRuleBlocked, ActionWAFBlocked, or, in servewaf, ActionTooLarge
 	// for the header limits.
 	WouldAction string `json:"would_action,omitempty"`
-	// Heuristic is the name of the heuristic that refused the request, in
-	// observe mode too, and HeuristicFigures are the figures it acted on, by
-	// name.
-	Heuristic        string         `json:"heuristic,omitempty"`
-	HeuristicFigures map[string]any `json:"heuristic_figures,omitempty"`
-	// ReportHeuristic and ReportHeuristicFigures are the same for the
-	// heuristic that banned the client after servewaf answered the request.
-	ReportHeuristic        string         `json:"report_heuristic,omitempty"`
-	ReportHeuristicFigures map[string]any `json:"report_heuristic_figures,omitempty"`
 	// LimitPercent and LimitPercentSetting are, for a request the rate
 	// limits counted whose client a biased threshold gives a percentage of
 	// the rate limits below 100, that percentage and the setting that gave
@@ -254,7 +267,11 @@ func FormatTime(t time.Time) string {
 
 // Milliseconds is d in milliseconds, to the microsecond.
 func Milliseconds(d time.Duration) float64 {
-	return float64(d.Microseconds()) / float64(time.Millisecond/time.Microsecond)
+	return float64(
+		d.Microseconds(),
+	) / float64(
+		time.Millisecond/time.Microsecond,
+	)
 }
 
 // NewProcessLogger returns the logger for the process's own messages:
@@ -262,17 +279,25 @@ func Milliseconds(d time.Duration) float64 {
 // as a request line's, and instanceName, SWWAF_INSTANCE_NAME, as instance.
 // It writes only the messages at level, SWWAF_LOG_LEVEL, or more severe;
 // the request lines Write writes are never held back.
-func NewProcessLogger(w io.Writer, instanceName string, level slog.Level) *slog.Logger {
+func NewProcessLogger(
+	w io.Writer,
+	instanceName string,
+	level slog.Level,
+) *slog.Logger {
 	handler := slog.NewJSONHandler(w, &slog.HandlerOptions{
 		Level: level,
 		ReplaceAttr: func(groups []string, attr slog.Attr) slog.Attr {
 			if attr.Key == slog.TimeKey && len(groups) == 0 {
-				return slog.String(slog.TimeKey, FormatTime(attr.Value.Time()))
+				return slog.String(
+					slog.TimeKey,
+					FormatTime(attr.Value.Time()),
+				)
 			}
 
 			return attr
 		},
 	})
 
-	return slog.New(handler).With("type", "process", "instance", instanceName)
+	return slog.New(handler).
+		With("type", "process", "instance", instanceName)
 }

internal/servewaf/abuseipdb.go

@@ -1,55 +0,0 @@
-package servewaf
-
-// The heuristic abuseipdb. It reads SWWAF_ABUSEIPDB_KEY,
-// SWWAF_ABUSEIPDB_MIN_SCORE and SWWAF_REPUTATION_ACTION.
-
-import (
-	"context"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/bans"
-	"sneak.berlin/go/smallwebwaf/internal/ratelimit"
-	"sneak.berlin/go/smallwebwaf/internal/reputation"
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-// abuseIPDB notes AbuseIPDB, as noteHit does, with the score, when its score
-// of the client is a hit, and refuses the request when SWWAF_REPUTATION_ACTION
-// is deny, as dnsbl does for a zone. While SWWAF_ABUSEIPDB_KEY is unset it does
-// nothing. A client without a score is checked in the background, by the
-// request's address, if its history counts an offence, and the request does
-// not wait for the answer. The score is then used for each address of the
-// client. Its figures are the score and SWWAF_ABUSEIPDB_MIN_SCORE.
-type abuseIPDB struct{}
-
-func (abuseIPDB) name() string { return "abuseipdb" }
-
-func (abuseIPDB) check(ctx context.Context, rq *request, _ time.Time) *verdict {
-	if rq.h.config.AbuseIPDBKey == "" {
-		return nil
-	}
-
-	client := rq.h.clientGroup(rq.client)
-	held, _ := rq.h.limiter.Client(client)
-	offender := held.History.Offences != ratelimit.Offences{}
-
-	score, hit := rq.h.abuseIPDB.Hit(ctx, client, rq.client, offender)
-	if !hit {
-		return nil
-	}
-
-	rq.abuseIPDBHit = true
-	rq.noteHit(bans.ReputationHit{Source: reputation.AbuseIPDBSource, Score: &score},
-		"scored by AbuseIPDB at or over SWWAF_ABUSEIPDB_MIN_SCORE")
-
-	if rq.h.config.ReputationAction != deny {
-		return nil
-	}
-
-	return &verdict{
-		action: requestlog.ActionDenied,
-		figures: map[string]any{
-			figureScore: score, "min_score": rq.h.config.AbuseIPDBMinScore,
-		},
-	}
-}

internal/servewaf/answertime_test.go

@@ -91,18 +91,6 @@ func TestSixteenThousandCookiesAreRefusedWellWithinTheTimeLimit(t *testing.T) {
 	}
 }
 
-func TestHeaderLimitRefusalNamesItsHeuristicCountsAndLimits(t *testing.T) {
-	t.Parallel()
-
-	s := startServeWAF(t, "", nil)
-	req := s.question(siteA, client, "/")
-	fillHeaders(req, headerFieldLimit+1)
-	do(t, req)
-
-	wantHeuristic(t, s.out.requestLine(t), "header_limits",
-		`{"fields": 101, "fields_limit": 100, "cookies": 0, "cookies_limit": 50}`)
-}
-
 func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/bans.go

@@ -1,10 +1,7 @@
 package servewaf
 
-// The heuristic active_ban, and the code that makes bans, which several
-// heuristics share. active_ban reads SWWAF_MODE; the bans are the ban ledger's.
-
 import (
-	"context"
+	"net/http"
 	"net/netip"
 	"time"
 
@@ -21,15 +18,11 @@ func (rq *request) banResponse(action string) *refusal {
 	return &refusal{status: rq.h.config.BanResponse, action: action}
 }
 
-// activeBan refuses a client a ban covers: a ban on a netblock the client is
-// in, in force at now. It notes for the log line when that ban ends. A request
-// that makes the ban permanent, or in observe mode would have, raises the alert
-// for it. Its figures are the ban's netblock and when it ends.
-type activeBan struct{}
-
-func (activeBan) name() string { return "active_ban" }
-
-func (activeBan) check(_ context.Context, rq *request, now time.Time) *verdict {
+// banned reports whether a ban on a netblock the client is in covers the
+// request at now, and notes for the log line when that ban ends. A
+// request that makes the ban permanent, or in observe mode would have,
+// raises the alert for it.
+func (rq *request) banned(now time.Time) bool {
 	check := rq.h.ledger.Check
 	if rq.h.config.Observe {
 		check = rq.h.ledger.Find // the ban refuses nothing, and stays as it is
@@ -45,24 +38,85 @@ func (activeBan) check(_ context.Context, rq *request, now time.Time) *verdict {
 		rq.alertBan(ban)
 	}
 
-	if !banned {
-		return nil
+	return banned
+}
+
+// limitBroken counts the request for the rate limits at now, notes the
+// client's counts for the log line, and reports whether the request takes
+// the client over a rate limit, as its limit percentage lowers it, which
+// breaks it.
+func (rq *request) limitBroken(now time.Time) bool {
+	counts, hit, over := rq.h.limiter.Count(rq.h.clientGroup(rq.client), now,
+		rq.limitPercent.percent)
+	rq.line.Counts = counts
+
+	if over {
+		rq.banForLimit(now, hit, rq.h.config.BanResponse)
 	}
 
-	return &verdict{
-		action: requestlog.ActionBanned,
-		figures: map[string]any{
-			figureNetblock: ban.Netblock.String(), "ban_expires": rq.line.BanExpires,
-		},
+	return over
+}
+
+// countBytes counts the request's bytes, as countedBytes gives them, for
+// the byte limits, once its report has come, and notes the client's byte totals
+// for the log line; its requests stay there as the rate limits counted them.
+// Only an allowed request has them counted, and only one the rate limits
+// counted; in observe mode, not one that enforce mode would have refused. Bytes
+// that take the client over a byte limit, as its limit percentage for the byte
+// limits lowers it, break it; the response was passed on whole.
+func (rq *request) countBytes() {
+	if !rq.counted || rq.line.WouldAction != "" {
+		return
+	}
+
+	now := rq.h.now()
+
+	counts, hit, over := rq.h.limiter.CountBytes(rq.h.clientGroup(rq.client), now,
+		rq.countedBytes(), rq.bytesPercent.percent)
+	rq.line.Counts.MinuteBytes = counts.MinuteBytes
+	rq.line.Counts.HourBytes = counts.HourBytes
+	rq.line.Counts.DayBytes = counts.DayBytes
+
+	if over {
+		rq.banForLimit(now, hit, rq.out.status)
 	}
 }
 
-// limitFigures are the figures of a heuristic that broke hit, a limit: its
-// window, the count and the limit.
-func limitFigures(hit ratelimit.Hit) map[string]any {
-	return map[string]any{
-		figureWindow: hit.Window, figureCount: hit.Count, figureLimit: hit.Limit,
+// countRefusal counts the request for the error burst once it has been
+// answered, if servewaf refused it after a rule file match, a trap path or a
+// Core Rule Set match, and in observe mode if enforce mode would have: more than
+// SWWAF_ERROR_BURST_THRESHOLD such refusals of the client within a minute
+// break a limit. A client in SWWAF_ALLOW_NETS, which the checks skip, is
+// not counted, and nothing is while the threshold is off.
+func (rq *request) countRefusal() {
+	cfg := rq.h.config
+	if cfg.ErrorBurstThreshold == 0 {
+		return
 	}
+
+	if !rq.attack && !rq.ruleBlocked && !rq.wafBlocked {
+		return
+	}
+
+	now := rq.h.now()
+
+	hit, over := rq.h.limiter.CountRefusal(rq.h.clientGroup(rq.client), now,
+		cfg.ErrorBurstThreshold)
+	if !over {
+		return
+	}
+
+	// What the client was sent, or in observe mode would have been.
+	status := rq.out.status
+
+	switch rq.line.WouldAction {
+	case requestlog.ActionRuleBlocked, requestlog.ActionWAFBlocked:
+		status = http.StatusForbidden
+	case requestlog.ActionBanned:
+		status = cfg.BanResponse
+	}
+
+	rq.banForLimit(now, hit, status)
 }
 
 // countedBytes returns the request's bytes, once it has ended, as the

internal/servewaf/bans_test.go

@@ -228,22 +228,6 @@ func TestBannedClientIsRefusedBeforeItsCountryIsLookedUp(t *testing.T) {
 	}
 }
 
-func TestRefusalUnderABanNamesItsHeuristicNetblockAndEnd(t *testing.T) {
-	t.Parallel()
-
-	s, clk, _ := startWithClock(t, "", map[string]string{
-		rateLimitPerMinute: "1", banScopeV4Prefix: "24",
-	})
-	expires := requestlog.FormatTime(clk.Now().Add(time.Hour))
-
-	s.get(client, http.StatusOK, requestlog.ActionForward)
-	s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
-
-	line := s.get(otherClient, http.StatusForbidden, requestlog.ActionBanned)
-	wantHeuristic(t, line, "active_ban",
-		`{"netblock": "`+clientsNet+`", "ban_expires": "`+expires+`"}`)
-}
-
 func TestBanResponseAnswersEveryRefusalButTheSizeLimits(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/blocklists.go

@@ -1,41 +0,0 @@
-package servewaf
-
-// The heuristic blocklists. It reads SWWAF_BLOCKLIST_ACTION, and the blocklists
-// through the reputation lists.
-
-import (
-	"context"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-// blocklists notes the blocklists that list the client, as noteListed does,
-// and refuses the request when SWWAF_BLOCKLIST_ACTION is deny. Being limit, it
-// lowers the client's limits instead (see limitPercentages), and being log, it
-// does nothing more. Its figure is the blocklists that list the client.
-type blocklists struct{}
-
-func (blocklists) name() string { return "blocklists" }
-
-func (blocklists) check(_ context.Context, rq *request, _ time.Time) *verdict {
-	listedBy := rq.h.lists.ListedBy(rq.client)
-	rq.blocklisted = len(listedBy) > 0
-	rq.noteListed(listedBy, "listed by a blocklist")
-
-	if !rq.refusedByBlocklists(listedBy) {
-		return nil
-	}
-
-	return &verdict{
-		action:  requestlog.ActionDenied,
-		figures: map[string]any{"lists": listedBy},
-	}
-}
-
-// refusedByBlocklists reports whether the heuristic blocklists refuses a
-// client, given listedBy, the blocklists that list it: it does when one lists
-// it and SWWAF_BLOCKLIST_ACTION is deny.
-func (rq *request) refusedByBlocklists(listedBy []string) bool {
-	return len(listedBy) > 0 && rq.h.config.BlocklistAction == deny
-}

internal/servewaf/bytelimits.go

@@ -1,43 +0,0 @@
-package servewaf
-
-// The heuristic byte_limits. It reads SWWAF_BYTES_COUNT, and the byte limits
-// and biased thresholds through the limiter and limitPercentages.
-
-import (
-	"context"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-// byteLimits counts the request's bytes, as countedBytes gives them, for the
-// byte limits, once its report has come, and notes the client's byte totals for
-// the log line; its requests stay there as the rate limits counted them. Only
-// an allowed request has them counted, and only one the rate limits counted; in
-// observe mode, not one that enforce mode would have refused. Bytes that take
-// the client over a byte limit, as its limit percentage for the byte limits
-// lowers it, break it, which bans the client; the response was passed on whole.
-// Its figures are the window, the count and the limit.
-type byteLimits struct{}
-
-func (byteLimits) name() string { return "byte_limits" }
-
-func (byteLimits) check(_ context.Context, rq *request, now time.Time) *verdict {
-	if !rq.counted || rq.line.WouldAction != "" {
-		return nil
-	}
-
-	counts, hit, over := rq.h.limiter.CountBytes(rq.h.clientGroup(rq.client), now,
-		rq.countedBytes(), rq.bytesPercent.percent)
-	rq.line.Counts.MinuteBytes = counts.MinuteBytes
-	rq.line.Counts.HourBytes = counts.HourBytes
-	rq.line.Counts.DayBytes = counts.DayBytes
-
-	if !over {
-		return nil
-	}
-
-	rq.banForLimit(now, hit, rq.out.status)
-
-	return &verdict{action: requestlog.ActionBanned, figures: limitFigures(hit)}
-}

internal/servewaf/bytelimits_test.go

@@ -98,21 +98,6 @@ func TestEachByteLimitBansOnceTheResponseHasEnded(t *testing.T) {
 	}
 }
 
-func TestByteLimitBanNamesItsHeuristicWindowCountAndLimit(t *testing.T) {
-	t.Parallel()
-
-	s, _ := startWithAnswers(t, map[string]string{bytesLimitPerMinute: byteLimit})
-
-	line, _ := s.download()
-	wantReportHeuristic(t, line, "", "null")
-
-	// Two answers of 70 bytes are over the limit of 99.
-	line, _ = s.download()
-	wantHeuristic(t, line, "", "null")
-	wantReportHeuristic(t, line, "byte_limits",
-		`{"window": "minute", "count": 140, "limit": 99}`)
-}
-
 func TestResponseOverAByteLimitByItselfIsPassedOnWhole(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/coreruleset.go

@@ -1,10 +1,6 @@
 package servewaf
 
-// The heuristic core_rule_set. It reads SWWAF_WAF_MODE, SWWAF_WAF_EXEMPT_PATHS
-// and SWWAF_WAF_ANOMALY_THRESHOLD.
-
 import (
-	"context"
 	"time"
 
 	"sneak.berlin/go/smallwebwaf/internal/alerts"
@@ -13,21 +9,18 @@ import (
 	"sneak.berlin/go/smallwebwaf/internal/waf"
 )
 
-// coreRuleSet inspects the request with the Core Rule Set, unless
+// checkCoreRuleSet inspects the request with the Core Rule Set, unless
 // SWWAF_WAF_MODE is off or SWWAF_WAF_EXEMPT_PATHS exempts its path, as
 // pathExempt decides, and notes the rules it matched and its score in the
 // log line, and the rules in the metrics. A score at or over
 // SWWAF_WAF_ANOMALY_THRESHOLD is a match: it raises the waf_block alert,
-// and in block mode refuses the request with 403, which is an offence its
-// client's history counts. Its figures are the score and the threshold.
-type coreRuleSet struct{}
-
-func (coreRuleSet) name() string { return "core_rule_set" }
-
-func (coreRuleSet) check(_ context.Context, rq *request, _ time.Time) *verdict {
+// and in block mode refuses the request, which is an offence its client's
+// history counts, and so returns ActionWAFBlocked. It returns "" for a
+// request it does not refuse.
+func (rq *request) checkCoreRuleSet() string {
 	cfg := rq.h.config
 	if cfg.WAFMode == config.WAFModeOff || pathExempt(rq.in, cfg.WAFExemptPaths) {
-		return nil
+		return ""
 	}
 
 	start := time.Now()
@@ -47,21 +40,18 @@ func (coreRuleSet) check(_ context.Context, rq *request, _ time.Time) *verdict {
 
 	threshold := cfg.WAFAnomalyThreshold
 	if threshold == 0 || result.Score < threshold {
-		return nil
+		return ""
 	}
 
 	rq.alertWAFBlock(result)
 
 	if cfg.WAFMode == config.WAFModeDetect {
-		return nil
+		return ""
 	}
 
 	rq.wafBlocked = true
 
-	return &verdict{
-		action:  requestlog.ActionWAFBlocked,
-		figures: map[string]any{figureScore: result.Score, "threshold": threshold},
-	}
+	return requestlog.ActionWAFBlocked
 }
 
 // alertWAFBlock raises the waf_block alert for the request, which the Core

internal/servewaf/coreruleset_test.go

@@ -110,16 +110,6 @@ func TestCoreRuleSetRefusesAttacksInBlockModeAndOnlyLogsThemInDetectMode(t *test
 	}
 }
 
-func TestCoreRuleSetRefusalNamesItsHeuristicScoreAndThreshold(t *testing.T) {
-	t.Parallel()
-
-	s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
-
-	line := s.request(client, sqlInjection, http.StatusForbidden,
-		requestlog.ActionWAFBlocked)
-	wantHeuristic(t, line, "core_rule_set", `{"score": 5, "threshold": 5}`)
-}
-
 func TestOrdinaryRequestIsInspectedAndPassed(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/countries.go

@@ -1,42 +1,21 @@
 package servewaf
 
-// The heuristic countries. It reads SWWAF_DENIED_COUNTRIES and
-// SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
-
 import (
-	"context"
 	"slices"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
 )
 
-// countries refuses a request by the client's country as it was looked up. A
-// client without a country, or whose country cannot be found, is refused only
-// by SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES. Its figures are the country and the
-// setting that refused it.
-type countries struct{}
-
-func (countries) name() string { return "countries" }
-
-func (countries) check(_ context.Context, rq *request, _ time.Time) *verdict {
+// countryDenied reports whether the country lists refuse the request, by
+// the client's country as it was looked up. A client without a country,
+// or whose country cannot be found, is refused only by
+// SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
+func (rq *request) countryDenied() bool {
 	denied := rq.h.config.DeniedCountries
 	allowed := rq.h.config.ExclusivelyAllowedCountries
 	country := rq.line.Country
 
-	var setting string
-
-	switch {
-	case slices.Contains(denied, country):
-		setting = "SWWAF_DENIED_COUNTRIES"
-	case len(allowed) > 0 && !slices.Contains(allowed, country):
-		setting = "SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES"
-	default:
-		return nil
+	if slices.Contains(denied, country) {
+		return true
 	}
 
-	return &verdict{
-		action:  requestlog.ActionCountryDenied,
-		figures: map[string]any{"country": country, "setting": setting},
-	}
+	return len(allowed) > 0 && !slices.Contains(allowed, country)
 }

internal/servewaf/countries_test.go

@@ -90,23 +90,6 @@ func TestCountryLists(t *testing.T) {
 	}
 }
 
-func TestCountryRefusalNamesItsHeuristicCountryAndSetting(t *testing.T) {
-	t.Parallel()
-
-	geojsURL, _ := startGeoJS(t)
-	s, _, _ := startWithClock(t, geojsURL, map[string]string{
-		lookupTimeout: "1h", deniedCountries: "kp", allowedCountries: "de",
-	})
-
-	line := s.get(fromKP, http.StatusForbidden, requestlog.ActionCountryDenied)
-	wantHeuristic(t, line, "countries",
-		`{"country": "KP", "setting": "SWWAF_DENIED_COUNTRIES"}`)
-
-	line = s.get(unplaced, http.StatusForbidden, requestlog.ActionCountryDenied)
-	wantHeuristic(t, line, "countries",
-		`{"country": "", "setting": "SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES"}`)
-}
-
 func TestCountryRefusalKeepsARequestFromTheApp(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/crowdsec.go

@@ -1,49 +0,0 @@
-package servewaf
-
-// The heuristic crowdsec. It reads SWWAF_CROWDSEC_LAPI_URL, and the decisions
-// through the reputation lists.
-
-import (
-	"context"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/bans"
-	"sneak.berlin/go/smallwebwaf/internal/reputation"
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-// crowdSec refuses a request when a decision of the CrowdSec decision list on
-// the client is in force at now. It then notes the list, as noteHit does, and
-// bans the client until that decision ends. Its figures are the decision's
-// scenario and when it ends.
-type crowdSec struct{}
-
-func (crowdSec) name() string { return "crowdsec" }
-
-func (crowdSec) check(_ context.Context, rq *request, now time.Time) *verdict {
-	decision, listed := rq.crowdSecDecisionInForce(now)
-	if !listed {
-		return nil
-	}
-
-	rq.noteHit(bans.ReputationHit{Source: rq.h.config.CrowdSecDecisionsURL},
-		"listed by the CrowdSec decision list")
-	rq.banForCrowdSec(now, decision)
-
-	return &verdict{
-		action: requestlog.ActionBanned,
-		figures: map[string]any{
-			"scenario": decision.Scenario,
-			"expires":  requestlog.FormatTime(decision.Expires),
-		},
-	}
-}
-
-// crowdSecDecisionInForce returns the decision of the CrowdSec decision list on
-// the client that is in force at now, and reports whether there is one, which
-// the heuristic crowdsec refuses the request for.
-func (rq *request) crowdSecDecisionInForce(
-	now time.Time,
-) (reputation.Decision, bool) {
-	return rq.h.lists.CrowdSecDecision(rq.client, now)
-}

internal/servewaf/crowdsec_test.go

@@ -95,21 +95,6 @@ func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(t *testi
 		float64(fetched.Unix()))
 }
 
-func TestCrowdSecBanNamesItsHeuristicScenarioAndEnd(t *testing.T) {
-	t.Parallel()
-
-	s, clk, server, _ := startWithAlerts(t, map[string]string{
-		crowdSecURL: lapi, crowdSecKey: bouncerKey,
-	})
-	loadDecisions(t, server, clk.Now(), `[{"duration": "4h0m0s", `+
-		`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
-		`"value": "`+client+`"}]`)
-
-	line := s.get(client, http.StatusForbidden, requestlog.ActionBanned)
-	wantHeuristic(t, line, "crowdsec", `{"scenario": "crowdsecurity/ssh-bf", `+
-		`"expires": "`+requestlog.FormatTime(clk.Now().Add(4*time.Hour))+`"}`)
-}
-
 func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/denynets.go

@@ -1,29 +0,0 @@
-package servewaf
-
-// The heuristic deny_nets. It reads SWWAF_DENY_NETS.
-
-import (
-	"context"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-// denyNets refuses a client in SWWAF_DENY_NETS. Its figure is the netblock of
-// the setting the client is in.
-type denyNets struct{}
-
-func (denyNets) name() string { return "deny_nets" }
-
-func (denyNets) check(_ context.Context, rq *request, _ time.Time) *verdict {
-	for _, netblock := range rq.h.config.DenyNets {
-		if netblock.Contains(rq.client) {
-			return &verdict{
-				action:  requestlog.ActionDenied,
-				figures: map[string]any{figureNetblock: netblock.String()},
-			}
-		}
-	}
-
-	return nil
-}

internal/servewaf/dnsbl.go

@@ -1,36 +0,0 @@
-package servewaf
-
-// The heuristic dnsbl. It reads SWWAF_REPUTATION_ACTION, and the DNSBL zones'
-// verdicts through the DNSBL checker.
-
-import (
-	"context"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-// dnsbl notes the DNSBL zones whose verdict lists the client, as noteListed
-// does, and refuses the request when SWWAF_REPUTATION_ACTION is deny. Being
-// limit, it lowers the client's limits instead (see limitPercentages), and
-// being log, it does nothing more. A zone without a verdict on the client is
-// asked about it in the background, and the request does not wait for the
-// answer. Its figure is the zones that list the client, their keys masked.
-type dnsbl struct{}
-
-func (dnsbl) name() string { return "dnsbl" }
-
-func (dnsbl) check(ctx context.Context, rq *request, _ time.Time) *verdict {
-	listedBy := rq.h.dnsbl.ListedBy(ctx, rq.client)
-	rq.dnsblListed = len(listedBy) > 0
-	rq.noteListed(listedBy, "listed by a DNSBL zone")
-
-	if !rq.dnsblListed || rq.h.config.ReputationAction != deny {
-		return nil
-	}
-
-	return &verdict{
-		action:  requestlog.ActionDenied,
-		figures: map[string]any{"zones": listedBy},
-	}
-}

internal/servewaf/errorburst_test.go

@@ -106,25 +106,6 @@ func TestErrorBurstBreaksAtOneOverTheThreshold(t *testing.T) {
 	}
 }
 
-func TestRefusalLimitBanNamesItsHeuristicWindowCountAndLimit(t *testing.T) {
-	t.Parallel()
-
-	s, _, _ := startForErrorBurst(t, "1", nil)
-
-	const ruleFigures = `{"rule_id": "blocked", "rule_action": "block"}`
-
-	line := blockRule.send(s, client)
-	wantHeuristic(t, line, "rule_files", ruleFigures)
-	wantReportHeuristic(t, line, "", "null")
-
-	// The second refusal is the request refusal_limit bans for, and the line
-	// names both.
-	line = blockRule.send(s, client)
-	wantHeuristic(t, line, "rule_files", ruleFigures)
-	wantReportHeuristic(t, line, "refusal_limit",
-		`{"window": "minute", "count": 2, "limit": 1}`)
-}
-
 func TestErrorBurstBanNotesHistoryAndMetrics(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/headerlimits.go

@@ -1,73 +0,0 @@
-package servewaf
-
-// The heuristic header_limits. It reads no setting: its limits are fixed.
-
-import (
-	"context"
-	"strings"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-// The header limits: the most cookies and header fields a request may have
-// when the checks reach the Core Rule Set. Real clients never reach them.
-const (
-	maxCookies      = 50
-	maxHeaderFields = 100
-)
-
-// headerLimits refuses a request over the header limits, with 431. Its figures
-// are the request's header fields and cookies, and the limit of each.
-type headerLimits struct{}
-
-func (headerLimits) name() string { return "header_limits" }
-
-func (headerLimits) check(_ context.Context, rq *request, _ time.Time) *verdict {
-	fields, cookies := rq.headerCounts()
-	if fields <= maxHeaderFields && cookies <= maxCookies {
-		return nil
-	}
-
-	return &verdict{
-		action: requestlog.ActionTooLarge,
-		figures: map[string]any{
-			"fields": fields, "fields_limit": maxHeaderFields,
-			"cookies": cookies, "cookies_limit": maxCookies,
-		},
-	}
-}
-
-// overHeaderLimits reports whether the request is over the header limits: it
-// has more than maxHeaderFields fields or more than maxCookies cookies, as
-// headerCounts counts them.
-func (rq *request) overHeaderLimits() bool {
-	fields, cookies := rq.headerCounts()
-
-	return fields > maxHeaderFields || cookies > maxCookies
-}
-
-// headerCounts returns the request's header fields, each value of a header sent
-// twice being one, and its cookies, each part of a Cookie header between
-// semicolons being one unless it is empty or only spaces and tabs. The Core
-// Rule Set runs each of its rules on each cookie, so that thousands of small
-// cookies keep it busy for over a second. It trims only spaces and control
-// characters, of which a header value can hold only tabs, so a part of Unicode
-// spaces is a cookie to it.
-func (rq *request) headerCounts() (int, int) {
-	fields, cookies := 0, 0
-
-	for _, values := range rq.in.Header {
-		fields += len(values)
-	}
-
-	for _, value := range rq.in.Header.Values("Cookie") {
-		for part := range strings.SplitSeq(value, ";") {
-			if strings.Trim(part, " \t") != "" {
-				cookies++
-			}
-		}
-	}
-
-	return fields, cookies
-}

internal/servewaf/heuristics.go

@@ -1,133 +0,0 @@
-package servewaf
-
-import (
-	"context"
-	"time"
-)
-
-// To add a heuristic, write one file with a type that implements heuristic,
-// and add one line for it to one of the lists below. Its settings go in
-// config.Config, and the comment at the top of its file names them.
-
-// heuristic is one of the checks that decide what servewaf does with a request.
-type heuristic interface {
-	// name is the heuristic's name, which the log line gives.
-	name() string
-	// check looks at the request at now, and returns nil to let it through.
-	// ctx is the request's own context.
-	check(ctx context.Context, rq *request, now time.Time) *verdict
-}
-
-// verdict is a heuristic refusing a request, or banning its client: the action
-// the log line names, one of requestlog's Action constants, and the figures the
-// heuristic acted on, by name, such as the count and the limit.
-type verdict struct {
-	action  string
-	figures map[string]any
-}
-
-// The names of the figures that more than one heuristic gives.
-const (
-	figureWindow   = "window"
-	figureCount    = "count"
-	figureLimit    = "limit"
-	figureScore    = "score"
-	figureNetblock = "netblock"
-)
-
-// The lists of heuristics, each in the order its heuristics run. A client in
-// SWWAF_ALLOW_NETS skips them all.
-
-// beforeLookup are the heuristics that run before the client's AS number and
-// country are looked up, so that a client they refuse is not looked up.
-func beforeLookup() []heuristic {
-	return []heuristic{
-		denyNets{},
-		activeBan{},
-	}
-}
-
-// afterLookup are the heuristics that run once the lookup has been made.
-func afterLookup() []heuristic {
-	return []heuristic{
-		countries{},
-		blocklists{},
-		crowdSec{},
-		dnsbl{},
-		abuseIPDB{},
-		rateLimits{},
-		trapPaths{},
-		ruleFiles{},
-		headerLimits{},
-		coreRuleSet{},
-	}
-}
-
-// afterAnswer are the heuristics that run on a request once servewaf has
-// answered it, before its decision line is written. They ban the client, and do
-// not change what the request got: their verdict's action is not used.
-func afterAnswer() []heuristic {
-	return []heuristic{
-		refusalLimit{},
-	}
-}
-
-// onReport are the heuristics that run on an allowed request when rproxy's
-// report of it comes, before its report line is written. Like afterAnswer, they
-// only ban.
-func onReport() []heuristic {
-	return []heuristic{
-		byteLimits{},
-	}
-}
-
-// runAfterAnswer runs the heuristics of afterAnswer on the request, at the time
-// it is called.
-func (rq *request) runAfterAnswer(ctx context.Context) {
-	rq.runBanningHeuristics(ctx, afterAnswer())
-}
-
-// runHeuristics runs list, beforeLookup or afterLookup, on the request at now,
-// and returns the first verdict, or nil when none gives one. The log line's
-// heuristic and heuristic_figures name the heuristic that gave it, and give its
-// figures. ctx is the request's own context.
-func (rq *request) runHeuristics(
-	ctx context.Context, list []heuristic, now time.Time,
-) *verdict {
-	name, v := rq.firstVerdict(ctx, list, now)
-	if v != nil {
-		rq.line.Heuristic = name
-		rq.line.HeuristicFigures = v.figures
-	}
-
-	return v
-}
-
-// runBanningHeuristics runs list, afterAnswer or onReport, on the request at
-// the time it is called. The log line's report_heuristic and
-// report_heuristic_figures name the heuristic that gave a verdict, and give its
-// figures, so that heuristic and heuristic_figures still name the one that
-// refused the request. ctx is the request's own context.
-func (rq *request) runBanningHeuristics(ctx context.Context, list []heuristic) {
-	name, v := rq.firstVerdict(ctx, list, rq.h.now())
-	if v != nil {
-		rq.line.ReportHeuristic = name
-		rq.line.ReportHeuristicFigures = v.figures
-	}
-}
-
-// firstVerdict runs list on the request at now, in order, until a heuristic
-// gives a verdict, and returns that heuristic's name and its verdict, or "" and
-// nil when none gives one. ctx is the request's own context.
-func (rq *request) firstVerdict(
-	ctx context.Context, list []heuristic, now time.Time,
-) (string, *verdict) {
-	for _, each := range list {
-		v := each.check(ctx, rq, now)
-		if v != nil {
-			return each.name(), v
-		}
-	}
-
-	return "", nil
-}

internal/servewaf/observe_test.go

@@ -14,18 +14,6 @@ import (
 // observe is the value of SWWAF_MODE for observe mode.
 const observe = "observe"
 
-func TestObserveModeNamesTheHeuristicThatWouldHaveRefused(t *testing.T) {
-	t.Parallel()
-
-	s, _, _ := startWithClock(t, "", map[string]string{
-		mode: observe, denyNets: client,
-	})
-
-	line := s.get(client, http.StatusOK, requestlog.ActionForward)
-	wantWouldAction(t, line, requestlog.ActionDenied)
-	wantHeuristic(t, line, "deny_nets", `{"netblock": "`+client+`/32"}`)
-}
-
 func TestObserveModeForwardsWhatEnforceModeRefuses(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/ratelimits.go

@@ -1,65 +0,0 @@
-package servewaf
-
-// The heuristic rate_limits. It reads SWWAF_RATE_LIMIT_EXEMPT_NETS,
-// SWWAF_RATE_LIMIT_EXEMPT_PATHS and SWWAF_BAN_RESPONSE, and the rate limits and
-// biased thresholds through the limiter and limitPercentages.
-
-import (
-	"context"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-// rateLimits counts the request for the rate limits, unless
-// countedByRateLimits exempts it, by the client's limit percentages, which it
-// notes for the log line with the client's counts. A request that takes the
-// client over a rate limit, as its limit percentage lowers it, breaks it: it is
-// refused, and bans the client. A request exempt from the rate limits is exempt
-// from the byte limits too. Its figures are the window, the count and the
-// limit.
-type rateLimits struct{}
-
-func (rateLimits) name() string { return "rate_limits" }
-
-func (rateLimits) check(_ context.Context, rq *request, now time.Time) *verdict {
-	rq.counted = rq.countedByRateLimits()
-	if !rq.counted {
-		return nil
-	}
-
-	rq.limitPercent, rq.bytesPercent = rq.limitPercentages()
-	rq.line.LimitPercent, rq.line.LimitPercentSetting = rq.limitPercent.logged()
-	rq.line.BytesPercent, rq.line.BytesPercentSetting = rq.bytesPercent.logged()
-
-	counts, hit, over := rq.h.limiter.Count(rq.h.clientGroup(rq.client), now,
-		rq.limitPercent.percent)
-	rq.line.Counts = counts
-
-	if !over {
-		return nil
-	}
-
-	rq.banForLimit(now, hit, rq.h.config.BanResponse)
-
-	return &verdict{action: requestlog.ActionRateLimited, figures: limitFigures(hit)}
-}
-
-// overAWholeRateLimit reports whether the heuristic rate_limits will refuse the
-// request at now, whatever its client's limit percentages: the rate limits
-// count it, and one more request from its client takes it over a whole rate
-// limit. It counts nothing.
-func (rq *request) overAWholeRateLimit(now time.Time) bool {
-	return rq.countedByRateLimits() &&
-		rq.h.limiter.OverARateLimit(rq.h.clientGroup(rq.client), now)
-}
-
-// countedByRateLimits reports whether the rate limits count the request: its
-// client is not in SWWAF_RATE_LIMIT_EXEMPT_NETS, and its path is not exempt
-// under SWWAF_RATE_LIMIT_EXEMPT_PATHS.
-func (rq *request) countedByRateLimits() bool {
-	cfg := rq.h.config
-
-	return !isInside(rq.client, cfg.RateLimitExemptNets) &&
-		!pathExempt(rq.in, cfg.RateLimitExemptPaths)
-}

internal/servewaf/ratelimits_test.go

@@ -72,17 +72,6 @@ func TestRateLimitRefusesBeforeTheApp(t *testing.T) {
 	}
 }
 
-func TestRateLimitRefusalNamesItsHeuristicWindowCountAndLimit(t *testing.T) {
-	t.Parallel()
-
-	s, _, _ := startWithClock(t, "", map[string]string{rateLimitPerMinute: "1"})
-
-	s.get(client, http.StatusOK, requestlog.ActionForward)
-
-	line := s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
-	wantHeuristic(t, line, "rate_limits", `{"window": "minute", "count": 2, "limit": 1}`)
-}
-
 func TestIPv6GroupPrefixSetsTheClientTheLimitsCount(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/refusallimit.go

@@ -1,54 +0,0 @@
-package servewaf
-
-// The heuristic refusal_limit. It reads SWWAF_ERROR_BURST_THRESHOLD and
-// SWWAF_BAN_RESPONSE.
-
-import (
-	"context"
-	"net/http"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-)
-
-// refusalLimit counts the request once it has been answered, if servewaf
-// refused it after a rule file match, a trap path or a Core Rule Set match, and
-// in observe mode if enforce mode would have: more than
-// SWWAF_ERROR_BURST_THRESHOLD such refusals of the client within a minute break
-// a limit, which bans the client. A client in SWWAF_ALLOW_NETS, which the
-// heuristics skip, is not counted, and nothing is while the threshold is off.
-// Its figures are the window, the count and the limit.
-type refusalLimit struct{}
-
-func (refusalLimit) name() string { return "refusal_limit" }
-
-func (refusalLimit) check(_ context.Context, rq *request, now time.Time) *verdict {
-	cfg := rq.h.config
-	if cfg.ErrorBurstThreshold == 0 {
-		return nil
-	}
-
-	if !rq.attack && !rq.ruleBlocked && !rq.wafBlocked {
-		return nil
-	}
-
-	hit, over := rq.h.limiter.CountRefusal(rq.h.clientGroup(rq.client), now,
-		cfg.ErrorBurstThreshold)
-	if !over {
-		return nil
-	}
-
-	// What the client was sent, or in observe mode would have been.
-	status := rq.out.status
-
-	switch rq.line.WouldAction {
-	case requestlog.ActionRuleBlocked, requestlog.ActionWAFBlocked:
-		status = http.StatusForbidden
-	case requestlog.ActionBanned:
-		status = cfg.BanResponse
-	}
-
-	rq.banForLimit(now, hit, status)
-
-	return &verdict{action: requestlog.ActionBanned, figures: limitFigures(hit)}
-}

internal/servewaf/reputation.go

@@ -1,17 +1,91 @@
 package servewaf
 
-// What the reputation heuristics share: blocklists, crowdsec, dnsbl and
-// abuseipdb.
-
 import (
+	"context"
+	"time"
+
 	"sneak.berlin/go/smallwebwaf/internal/alerts"
 	"sneak.berlin/go/smallwebwaf/internal/bans"
+	"sneak.berlin/go/smallwebwaf/internal/ratelimit"
+	"sneak.berlin/go/smallwebwaf/internal/reputation"
 )
 
 // deny is the SWWAF_BLOCKLIST_ACTION and the SWWAF_REPUTATION_ACTION that
 // refuses the requests of a client a source lists.
 const deny = "deny"
 
+// blocklistDenied notes the blocklists that list the client, as
+// noteListed does, and reports whether SWWAF_BLOCKLIST_ACTION, being deny,
+// refuses the request. Being limit, it lowers the client's limits instead
+// (see limitPercentages), and being log, it does nothing more.
+func (rq *request) blocklistDenied() bool {
+	listedBy := rq.h.lists.ListedBy(rq.client)
+	rq.blocklisted = len(listedBy) > 0
+	rq.noteListed(listedBy, "listed by a blocklist")
+
+	return rq.blocklisted && rq.h.config.BlocklistAction == deny
+}
+
+// crowdSecBanned reports whether a decision of the CrowdSec decision list
+// on the client is in force at now. If one is, it notes the list, as
+// noteHit does, and bans the client until that decision ends.
+func (rq *request) crowdSecBanned(now time.Time) bool {
+	decision, listed := rq.h.lists.CrowdSecDecision(rq.client, now)
+	if !listed {
+		return false
+	}
+
+	rq.noteHit(bans.ReputationHit{Source: rq.h.config.CrowdSecDecisionsURL},
+		"listed by the CrowdSec decision list")
+	rq.banForCrowdSec(now, decision)
+
+	return true
+}
+
+// dnsblDenied notes the DNSBL zones whose verdict lists the client, as
+// noteListed does, and reports whether SWWAF_REPUTATION_ACTION, being
+// deny, refuses the request. Being limit, it lowers the client's limits
+// instead (see limitPercentages), and being log, it does nothing more. A
+// zone without a verdict on the client is asked about it in the
+// background, and the request does not wait for the answer. ctx is the
+// request's own context.
+func (rq *request) dnsblDenied(ctx context.Context) bool {
+	listedBy := rq.h.dnsbl.ListedBy(ctx, rq.client)
+	rq.dnsblListed = len(listedBy) > 0
+	rq.noteListed(listedBy, "listed by a DNSBL zone")
+
+	return rq.dnsblListed && rq.h.config.ReputationAction == deny
+}
+
+// abuseIPDBDenied notes AbuseIPDB, as noteHit does, with the score, when
+// its score of the client is a hit, and reports whether
+// SWWAF_REPUTATION_ACTION, being deny, refuses the request, as dnsblDenied
+// does for a zone. While SWWAF_ABUSEIPDB_KEY is unset it does nothing. A
+// client without a score is checked in the background, by the request's
+// address, if its history counts an offence, and the request does not
+// wait for the answer. The score is then used for each address of the
+// client. ctx is the request's own context.
+func (rq *request) abuseIPDBDenied(ctx context.Context) bool {
+	if rq.h.config.AbuseIPDBKey == "" {
+		return false
+	}
+
+	client := rq.h.clientGroup(rq.client)
+	held, _ := rq.h.limiter.Client(client)
+	offender := held.History.Offences != ratelimit.Offences{}
+
+	score, hit := rq.h.abuseIPDB.Hit(ctx, client, rq.client, offender)
+	if !hit {
+		return false
+	}
+
+	rq.abuseIPDBHit = true
+	rq.noteHit(bans.ReputationHit{Source: reputation.AbuseIPDBSource, Score: &score},
+		"scored by AbuseIPDB at or over SWWAF_ABUSEIPDB_MIN_SCORE")
+
+	return rq.h.config.ReputationAction == deny
+}
+
 // noteListed notes each of sources, the URLs of the blocklists or the
 // DNSBL zones, their keys masked, that list the client, as noteHit does,
 // with reason.

internal/servewaf/reputation_test.go

@@ -118,56 +118,6 @@ func TestEachBlocklistActionForAListedAddressAndAListedNetblock(t *testing.T) {
 	}
 }
 
-func TestReputationRefusalNamesItsHeuristicAndWhatListedTheClient(t *testing.T) {
-	t.Parallel()
-
-	for _, tc := range []struct {
-		heuristic string
-		env       map[string]string
-		load      func(*testing.T, *servewaf.Server)
-		figures   string
-	}{
-		{
-			"blocklists",
-			map[string]string{blocklistURLs: dropURL, blocklistAction: actionDeny},
-			func(t *testing.T, server *servewaf.Server) {
-				t.Helper()
-				loadLists(t, server, map[string][]string{dropURL: {client}})
-			},
-			`{"lists": ["` + dropURL + `"]}`,
-		},
-		{
-			"dnsbl",
-			map[string]string{
-				dnsblZones: dnsblZone, dnsblResolver: noResolver,
-				reputationAction: actionDeny,
-			},
-			func(_ *testing.T, server *servewaf.Server) {
-				loadVerdicts(server, map[string][]string{client: {dnsblZone}})
-			},
-			`{"zones": ["` + dnsblZone + `"]}`,
-		},
-		{
-			"abuseipdb",
-			map[string]string{abuseIPDBKey: accountKey, reputationAction: actionDeny},
-			func(_ *testing.T, server *servewaf.Server) {
-				loadScores(server, map[string]int64{client: 80})
-			},
-			`{"score": 80, "min_score": 75}`,
-		},
-	} {
-		t.Run(tc.heuristic, func(t *testing.T) {
-			t.Parallel()
-
-			s, server, _ := startWithLookups(t, tc.env)
-			tc.load(t, server)
-
-			line := s.get(client, http.StatusForbidden, requestlog.ActionDenied)
-			wantHeuristic(t, line, tc.heuristic, tc.figures)
-		})
-	}
-}
-
 func TestBlocklistsComeAfterTheCountryListsAndSkipAllowNets(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/request.go

@@ -160,20 +160,16 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
 }
 
 // check is the one place where a request can be refused. It returns nil to let
-// the request through. The heuristics of checkClient decide: a request they
-// refuse is answered with SWWAF_BAN_RESPONSE, 403 for a block rule or the Core
-// Rule Set, or 431 for the header limits. In observe mode nothing is refused,
-// and the log line names what enforce mode would have done. ctx is the
-// request's own context.
+// the request through. The checks of checkClient decide: a request they refuse
+// is answered with SWWAF_BAN_RESPONSE, 403 for a block rule or the Core Rule Set,
+// or 431 for the header limits. In observe mode nothing is refused, and the log
+// line names what enforce mode would have done. ctx is the request's own context.
 func (rq *request) check(ctx context.Context) *refusal {
-	v := rq.checkClient(ctx)
-	if v == nil {
-		return nil
-	}
-
-	action := v.action
+	action := rq.checkClient(ctx)
 
 	switch {
+	case action == "":
+		return nil
 	case rq.h.config.Observe:
 		rq.line.WouldAction = action
 
@@ -187,21 +183,35 @@ func (rq *request) check(ctx context.Context) *refusal {
 	}
 }
 
-// checkClient runs the heuristics of beforeLookup on the request, then looks
-// up the client's AS number and country, then runs those of afterLookup. It
-// returns the first verdict, or nil when none gives one. A client in
-// SWWAF_ALLOW_NETS skips them all, and is not looked up. ctx is the request's
-// own context.
-func (rq *request) checkClient(ctx context.Context) *verdict {
-	if isInside(rq.client, rq.h.config.AllowNets) {
-		return nil
+// checkClient runs the checks on the request's client, and returns the
+// action of the first that refuses the request, or "" when none does. A
+// client in SWWAF_ALLOW_NETS skips them, and is not looked up. For any
+// other client, SWWAF_DENY_NETS comes first, then a ban on its netblock,
+// so that a client either refuses is not looked up, then the lookup of
+// its AS number and country, then the country lists, then the blocklists,
+// then the CrowdSec decision list, which bans the client it lists, then
+// the DNSBL zones' verdicts, and then AbuseIPDB's score; a request any of
+// them refuses is not counted for the rate limits. Then come the
+// rate limits, unless the client is in SWWAF_RATE_LIMIT_EXEMPT_NETS or the
+// request's path is exempt under SWWAF_RATE_LIMIT_EXEMPT_PATHS, so that
+// every other request is counted, each of them by the client's limit
+// percentages, then SWWAF_TRAP_PATHS, then the rule files, then the header
+// limits, and last the Core Rule Set. A request exempt from the rate limits
+// is exempt from the byte limits too. ctx is the request's own context.
+func (rq *request) checkClient(ctx context.Context) string {
+	cfg := rq.h.config
+	if isInside(rq.client, cfg.AllowNets) {
+		return ""
 	}
 
 	now := rq.h.now()
 
-	v := rq.runHeuristics(ctx, beforeLookup(), now)
-	if v != nil {
-		return v
+	if isInside(rq.client, cfg.DenyNets) {
+		return requestlog.ActionDenied
+	}
+
+	if rq.banned(now) {
+		return requestlog.ActionBanned
 	}
 
 	rq.lookUp(ctx, now)
@@ -210,7 +220,59 @@ func (rq *request) checkClient(ctx context.Context) *verdict {
 	// again: a request counted at the time it arrived could be counted
 	// after a later one of its client's, which the rate limits would take
 	// for the clock set back.
-	return rq.runHeuristics(ctx, afterLookup(), rq.h.now())
+	now = rq.h.now()
+
+	if rq.countryDenied() {
+		return requestlog.ActionCountryDenied
+	}
+
+	if rq.blocklistDenied() {
+		return requestlog.ActionDenied
+	}
+
+	if rq.crowdSecBanned(now) {
+		return requestlog.ActionBanned
+	}
+
+	if rq.dnsblDenied(ctx) || rq.abuseIPDBDenied(ctx) {
+		return requestlog.ActionDenied
+	}
+
+	rq.counted = rq.countedByRateLimits()
+	if rq.counted {
+		rq.limitPercent, rq.bytesPercent = rq.limitPercentages()
+		rq.line.LimitPercent, rq.line.LimitPercentSetting = rq.limitPercent.logged()
+		rq.line.BytesPercent, rq.line.BytesPercentSetting = rq.bytesPercent.logged()
+	}
+
+	if rq.counted && rq.limitBroken(now) {
+		return requestlog.ActionRateLimited
+	}
+
+	if rq.trapPath(now) {
+		return requestlog.ActionBanned
+	}
+
+	action := rq.checkRules(now)
+	if action != "" {
+		return action
+	}
+
+	if rq.overHeaderLimits() {
+		return requestlog.ActionTooLarge
+	}
+
+	return rq.checkCoreRuleSet()
+}
+
+// countedByRateLimits reports whether the rate limits count the request: its
+// client is not in SWWAF_RATE_LIMIT_EXEMPT_NETS, and its path is not exempt
+// under SWWAF_RATE_LIMIT_EXEMPT_PATHS.
+func (rq *request) countedByRateLimits() bool {
+	cfg := rq.h.config
+
+	return !isInside(rq.client, cfg.RateLimitExemptNets) &&
+		!pathExempt(rq.in, cfg.RateLimitExemptPaths)
 }
 
 // pathExempt reports whether r is exempt under prefixes,

internal/servewaf/requestlog_test.go

@@ -1,7 +1,6 @@
 package servewaf_test
 
 import (
-	"encoding/json"
 	"io"
 	"math"
 	"net/http"
@@ -273,47 +272,6 @@ func wantTimings(t *testing.T, line logLine, want ...string) {
 	}
 }
 
-// wantHeuristic checks that line names heuristic as the one that refused the
-// request, with figures, a JSON object. A heuristic of "" with figures "null" is
-// a line that names none.
-func wantHeuristic(t *testing.T, line logLine, heuristic, figures string) {
-	t.Helper()
-
-	if line.Heuristic != heuristic ||
-		!reflect.DeepEqual(line.HeuristicFigures, decodeFigures(t, figures)) {
-		t.Errorf("log line has heuristic %q with heuristic_figures %v, want %q with %s",
-			line.Heuristic, line.HeuristicFigures, heuristic, figures)
-	}
-}
-
-// wantReportHeuristic checks report_heuristic and report_heuristic_figures as
-// wantHeuristic checks heuristic and heuristic_figures.
-func wantReportHeuristic(t *testing.T, line logLine, heuristic, figures string) {
-	t.Helper()
-
-	if line.ReportHeuristic != heuristic ||
-		!reflect.DeepEqual(line.ReportHeuristicFigures, decodeFigures(t, figures)) {
-		t.Errorf("log line has report_heuristic %q with report_heuristic_figures %v, "+
-			"want %q with %s",
-			line.ReportHeuristic, line.ReportHeuristicFigures, heuristic, figures)
-	}
-}
-
-// decodeFigures decodes figures, a JSON object of a heuristic's figures, as
-// the log line's figures are decoded.
-func decodeFigures(t *testing.T, figures string) map[string]any {
-	t.Helper()
-
-	var decoded map[string]any
-
-	err := json.Unmarshal([]byte(figures), &decoded)
-	if err != nil {
-		t.Fatalf("decode %s: %v", figures, err)
-	}
-
-	return decoded
-}
-
 // microseconds is a timing in whole microseconds.
 func microseconds(milliseconds float64) int64 {
 	return int64(math.Round(milliseconds * 1000))

internal/servewaf/rulefiles.go

@@ -1,55 +0,0 @@
-package servewaf
-
-// The heuristic rule_files. It reads the rules of the rule files, which
-// SWWAF_RULES_DIR and SWWAF_RULES_ENABLED name.
-
-import (
-	"context"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/bans"
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-	"sneak.berlin/go/smallwebwaf/internal/rules"
-)
-
-// ruleFiles checks the request against the rules of the rule files at now,
-// notes the ids of those it matches in the log line, and refuses the request
-// when the last rule it matches refuses it: with ActionRuleBlocked for a block
-// rule, and ActionBanned for a ban rule. A ban rule bans the client's netblock
-// for a clear sign of attack, or in observe mode raises the alert for the ban
-// it would have made. Either rule's match is noted as an offence, for the
-// client's history. Its figures are the rule's id and action.
-type ruleFiles struct{}
-
-func (ruleFiles) name() string { return "rule_files" }
-
-func (ruleFiles) check(_ context.Context, rq *request, now time.Time) *verdict {
-	matched := rq.h.rules.Match(rq.in)
-
-	for _, rule := range matched {
-		rq.line.RuleIDs = append(rq.line.RuleIDs, rule.ID)
-		rq.h.metrics.RuleMatched(rule.ID, rule.Action)
-	}
-
-	if len(matched) == 0 {
-		return nil
-	}
-
-	// Only the last rule matched can refuse the request.
-	last := matched[len(matched)-1]
-	figures := map[string]any{"rule_id": last.ID, "rule_action": last.Action}
-
-	switch last.Action {
-	case rules.ActionBlock:
-		rq.ruleBlocked = true
-
-		return &verdict{action: requestlog.ActionRuleBlocked, figures: figures}
-	case rules.ActionBan:
-		rq.attack = true
-		rq.banForAttack(now, bans.Notes{RuleID: last.ID, Target: last.Target})
-
-		return &verdict{action: requestlog.ActionBanned, figures: figures}
-	default:
-		return nil
-	}
-}

internal/servewaf/rulefiles_test.go

@@ -93,21 +93,6 @@ func TestEachRuleAction(t *testing.T) {
 	s.get(client, http.StatusTooManyRequests, requestlog.ActionBanned)
 }
 
-func TestRuleRefusalNamesItsHeuristicAndTheRule(t *testing.T) {
-	t.Parallel()
-
-	s, _, _ := startWithClock(t, "", map[string]string{
-		rulesDir: writeRules(t, "noted path log ^/\n"+testRules),
-	})
-
-	line := s.request(client, "/blocked", http.StatusForbidden,
-		requestlog.ActionRuleBlocked)
-	wantHeuristic(t, line, "rule_files", `{"rule_id": "blocked", "rule_action": "block"}`)
-
-	line = s.request(client, "/.env", http.StatusForbidden, requestlog.ActionBanned)
-	wantHeuristic(t, line, "rule_files", `{"rule_id": "probe", "rule_action": "ban"}`)
-}
-
 func TestNextClearSignOfAttackAfterABanBansPermanently(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/rules.go

@@ -0,0 +1,64 @@
+package servewaf
+
+import (
+	"slices"
+	"time"
+
+	"sneak.berlin/go/smallwebwaf/internal/bans"
+	"sneak.berlin/go/smallwebwaf/internal/requestlog"
+	"sneak.berlin/go/smallwebwaf/internal/rules"
+)
+
+// trapPath reports whether the request asks for a path in
+// SWWAF_TRAP_PATHS: its path as a path rule sees it, without the query,
+// as sent or once percent-decoded, is one of them. Such a request is a
+// clear sign of attack, as a ban rule's match is: it bans the client's
+// netblock, or in observe mode raises the alert for the ban it would have
+// made. The ban's notes give the path as sent.
+func (rq *request) trapPath(now time.Time) bool {
+	path := rules.Path(rq.in)
+	if !slices.Contains(rq.h.config.TrapPaths, path) &&
+		!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
+		return false
+	}
+
+	rq.attack = true
+	rq.banForAttack(now, bans.Notes{TrapPath: path})
+
+	return true
+}
+
+// checkRules checks the request against the rules of the rule files at
+// now, notes the ids of those it matches in the log line, and returns the
+// action of the rule that refuses it, ActionRuleBlocked for a block rule
+// and ActionBanned for a ban rule, or "" when none does. A ban rule bans
+// the client's netblock for a clear sign of attack, or in observe mode
+// raises the alert for the ban it would have made. Either rule's match
+// is noted as an offence, for the client's history.
+func (rq *request) checkRules(now time.Time) string {
+	matched := rq.h.rules.Match(rq.in)
+
+	for _, rule := range matched {
+		rq.line.RuleIDs = append(rq.line.RuleIDs, rule.ID)
+		rq.h.metrics.RuleMatched(rule.ID, rule.Action)
+	}
+
+	if len(matched) == 0 {
+		return ""
+	}
+
+	// Only the last rule matched can refuse the request.
+	switch last := matched[len(matched)-1]; last.Action {
+	case rules.ActionBlock:
+		rq.ruleBlocked = true
+
+		return requestlog.ActionRuleBlocked
+	case rules.ActionBan:
+		rq.attack = true
+		rq.banForAttack(now, bans.Notes{RuleID: last.ID, Target: last.Target})
+
+		return requestlog.ActionBanned
+	default:
+		return ""
+	}
+}

internal/servewaf/servewaf.go

@@ -2,7 +2,6 @@ package servewaf
 
 import (
 	"bytes"
-	"context"
 	"encoding/json"
 	"errors"
 	"fmt"
@@ -53,6 +52,13 @@ const ReportPath = "/_smallwebwaf/report"
 // QuestionBodyMaxBytes is the most of a visitor's body a question carries.
 const QuestionBodyMaxBytes = 64 << 10
 
+// The header limits: the most cookies and header fields a request may have
+// when the checks reach the Core Rule Set. Real clients never reach them.
+const (
+	maxCookies      = 50
+	maxHeaderFields = 100
+)
+
 const (
 	// maxAwaitingReport is the most answered requests held until their
 	// reports come. Past it, the request answered longest ago is dropped.
@@ -141,20 +147,48 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
 
 // refusedWhateverTheAnswer reports whether the request is one that does not wait
 // for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
-// its client: the heuristic crowdsec, header_limits, blocklists or rate_limits
-// will refuse it, as a function in that heuristic's file says. In observe mode,
-// which refuses nothing, none is. rproxy lets a request through when servewaf
-// answers too late.
+// its client: it is over the header limits, a blocklist lists its client while
+// SWWAF_BLOCKLIST_ACTION is deny, the CrowdSec decision list lists it, or it is
+// over a whole rate limit. In observe mode, which refuses nothing, none is.
+// rproxy lets a request through when servewaf answers too late.
 func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
 	if rq.h.config.Observe {
 		return false
 	}
 
-	_, decided := rq.crowdSecDecisionInForce(now)
+	lists := rq.h.lists
+	_, decided := lists.CrowdSecDecision(rq.client, now)
 
 	return decided || rq.overHeaderLimits() ||
-		rq.refusedByBlocklists(rq.h.lists.ListedBy(rq.client)) ||
-		rq.overAWholeRateLimit(now)
+		(rq.h.config.BlocklistAction == deny && len(lists.ListedBy(rq.client)) > 0) ||
+		(rq.countedByRateLimits() &&
+			rq.h.limiter.OverARateLimit(rq.h.clientGroup(rq.client), now))
+}
+
+// overHeaderLimits reports whether servewaf refuses the request for its
+// headers: it has more than maxHeaderFields fields, each value of a header sent
+// twice being one, or more than maxCookies cookies, each part of a Cookie header
+// between semicolons being one unless it is empty or only spaces and tabs. The
+// Core Rule Set runs each of its rules on each cookie, so that thousands of small
+// cookies keep it busy for over a second. It trims only spaces and control
+// characters, of which a header value can hold only tabs, so a part of Unicode
+// spaces is a cookie to it.
+func (rq *request) overHeaderLimits() bool {
+	fields, cookies := 0, 0
+
+	for _, values := range rq.in.Header {
+		fields += len(values)
+	}
+
+	for _, value := range rq.in.Header.Values("Cookie") {
+		for part := range strings.SplitSeq(value, ";") {
+			if strings.Trim(part, " \t") != "" {
+				cookies++
+			}
+		}
+	}
+
+	return fields > maxHeaderFields || cookies > maxCookies
 }
 
 // answerQuestion runs the checks on the visitor's request a question
@@ -185,18 +219,18 @@ func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
 
 	if refused == nil && !underOwnPrefix {
 		// In observe mode, what enforce mode would have refused counts here.
-		defer rq.runAfterAnswer(r.Context())
+		defer rq.countRefusal()
 
 		rq.allow()
 
 		return
 	}
 
-	// The last deferred runs first: runAfterAnswer before addToHistory, so that
-	// a broken refusal limit is in the client's history.
+	// The last deferred runs first: countRefusal before addToHistory, so that a
+	// broken error burst is in the client's history.
 	defer rq.addToHistory()
 	defer rq.countAnomalies()
-	defer rq.runAfterAnswer(r.Context())
+	defer rq.countRefusal()
 
 	if refused != nil {
 		rq.deny(*refused)
@@ -378,7 +412,7 @@ func (h *handler) takeReport(w http.ResponseWriter, r *http.Request) {
 	// Remove reports true to one report only, of several that name the id. A
 	// question still being handled is held as nil.
 	if found && held != nil && h.awaitingReport.Remove(report.RequestID) {
-		h.finishWith(r.Context(), held, report)
+		h.finishWith(held, report)
 	}
 
 	w.WriteHeader(http.StatusNoContent)
@@ -441,11 +475,11 @@ func (report *Report) checkRanges() error {
 
 // finishWith finishes a request with what its report says, and writes the
 // request's last log line, which holds its decision's fields and the report's.
-// For an allowed request it first runs the heuristics of onReport, and adds the
-// request to its client's history, the anomaly counters and the metrics, with
-// the status and bytes the report gives. A denied request's counting was
-// finished at its decision. ctx is the report's own context.
-func (h *handler) finishWith(ctx context.Context, held *decision, report Report) {
+// For an allowed request it first counts the request's bytes, and adds it to its
+// client's history, the anomaly counters and the metrics, with the status and
+// bytes the report gives. A denied request's counting was finished at its
+// decision.
+func (h *handler) finishWith(held *decision, report Report) {
 	rq := &request{
 		decision:     *held,
 		h:            h,
@@ -462,7 +496,7 @@ func (h *handler) finishWith(ctx context.Context, held *decision, report Report)
 	if rq.allowed {
 		rq.line.DurationUpstreamTotal = &report.DurationUpstreamTotal
 
-		rq.runBanningHeuristics(ctx, onReport())
+		rq.countBytes()
 		rq.countAnomalies()
 		rq.addToHistory()
 

internal/servewaf/staticlists_test.go

@@ -104,15 +104,6 @@ func TestDenyNetsRefuseBeforeTheLookup(t *testing.T) {
 	}
 }
 
-func TestDenyNetsRefusalNamesItsHeuristicAndNetblock(t *testing.T) {
-	t.Parallel()
-
-	s, _, _ := startWithClock(t, "", map[string]string{denyNets: clientsNet})
-
-	line := s.get(client, http.StatusForbidden, requestlog.ActionDenied)
-	wantHeuristic(t, line, "deny_nets", `{"netblock": "`+clientsNet+`"}`)
-}
-
 func TestRequestRefusedByDenyNetsIsNotCounted(t *testing.T) {
 	t.Parallel()
 

internal/servewaf/trappaths.go

@@ -1,39 +0,0 @@
-package servewaf
-
-// The heuristic trap_paths. It reads SWWAF_TRAP_PATHS.
-
-import (
-	"context"
-	"slices"
-	"time"
-
-	"sneak.berlin/go/smallwebwaf/internal/bans"
-	"sneak.berlin/go/smallwebwaf/internal/requestlog"
-	"sneak.berlin/go/smallwebwaf/internal/rules"
-)
-
-// trapPaths refuses a request for a path in SWWAF_TRAP_PATHS: its path as a
-// path rule sees it, without the query, as sent or once percent-decoded, is one
-// of them. Such a request is a clear sign of attack, as a ban rule's match is:
-// it bans the client's netblock, or in observe mode raises the alert for the
-// ban it would have made. The ban's notes give the path as sent, and so does
-// its figure.
-type trapPaths struct{}
-
-func (trapPaths) name() string { return "trap_paths" }
-
-func (trapPaths) check(_ context.Context, rq *request, now time.Time) *verdict {
-	path := rules.Path(rq.in)
-	if !slices.Contains(rq.h.config.TrapPaths, path) &&
-		!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
-		return nil
-	}
-
-	rq.attack = true
-	rq.banForAttack(now, bans.Notes{TrapPath: path})
-
-	return &verdict{
-		action:  requestlog.ActionBanned,
-		figures: map[string]any{"path": path},
-	}
-}

internal/servewaf/trappaths_test.go

@@ -83,16 +83,6 @@ func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
 	}
 }
 
-func TestTrapPathRefusalNamesItsHeuristicAndPath(t *testing.T) {
-	t.Parallel()
-
-	s, _, _ := startWithClock(t, "", map[string]string{trapPaths: trapPathList})
-
-	line := s.request(client, "/wp-login.php?redirect_to=x", http.StatusForbidden,
-		requestlog.ActionBanned)
-	wantHeuristic(t, line, "trap_paths", `{"path": "/wp-login.php"}`)
-}
-
 func TestTrapPathMatchesThePathDecodedOnce(t *testing.T) {
 	t.Parallel()
 

script/publish

@@ -0,0 +1,51 @@
+#!/bin/sh
+# script/publish: build the image on the Docker host and push it to the
+# registry under four tags: the UTC date (YYYY-MM-DD), the short commit
+# hash, the branch name and latest. The build is script/docker's, so the
+# tests and the linter run first. The host and the registry name are
+# environment variables with defaults, for running it against another.
+#
+# The working tree must be clean: the hash tag names a commit, and an
+# image built from uncommitted changes would carry a hash that does not
+# describe it.
+set -eu
+
+SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
+ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
+
+: "${DOCKER_HOST:=ssh://root@fsn1app1}"
+: "${PUBLISH_IMAGE:=git.eeqj.de/sneak/smallwebwaf}"
+export DOCKER_HOST
+
+main() {
+    cd "$ROOT"
+
+    if [ -n "$(git status --porcelain)" ]; then
+        echo "script/publish: the working tree is not clean; commit or stash first" >&2
+        exit 1
+    fi
+
+    date="$(date -u +%Y-%m-%d)"
+    hash="$(git rev-parse --short HEAD)"
+    branch="$(git rev-parse --abbrev-ref HEAD)"
+    if [ "$branch" = "HEAD" ]; then
+        echo "script/publish: detached HEAD; check out a branch first" >&2
+        exit 1
+    fi
+    # A tag is [A-Za-z0-9_.-]; a branch name such as feature/x is not.
+    branch="$(printf '%s' "$branch" | tr -c 'A-Za-z0-9_.-' '-')"
+
+    local_tag="$("$SCRIPT_DIR/projectname")"
+    "$SCRIPT_DIR/docker"
+
+    for tag in "$date" "$hash" "$branch" latest; do
+        docker tag "$local_tag" "$PUBLISH_IMAGE:$tag"
+    done
+    for tag in "$date" "$hash" "$branch" latest; do
+        docker push "$PUBLISH_IMAGE:$tag"
+    done
+
+    echo "pushed $PUBLISH_IMAGE with tags: $date $hash $branch latest"
+}
+
+main "$@"