Default request limits per minute, hour and day, sized for API use (closes #176)

The default request limits now suit programmatic use of an API, such
as an automation fleet on another host, and still catch abusive bots:

- per minute: 500, room for a fleet's busiest minute and a few first
  visits with an empty cache, and passed within seconds by a client
  firing as fast as it can;
- per hour: 30000, which a client at the minute limit for a whole
  hour reaches exactly;
- per day: 300000, passed by a crawler that keeps up 4 a second.

The byte limits and ban lengths are unchanged. SPEC.md, README.md and
docs/settings.md give the reasons; docs/gitea.md redoes the Actions
runner sums.

Judgement call: the docs/alerts.md ban example shows the minute limit.

Model: opus-5-5

README.md

@@ -17,11 +17,20 @@ reference documents under [`docs/`](docs/) are listed under "Documents" below.
 ## What it protects against
 
 - Scrapers that fetch every page of a site, all day. Rate limiting gives each
-  client 6000 requests a minute, 100,000 an hour and 1,000,000 a day. Byte
-  limits give it 4 GiB a minute, 20 GiB an hour and 200 GiB a day. A client over
-  a limit is refused and banned. The first ban lasts an hour. Each repeat within
-  a day makes the next ban three times as long. A ban that would last more than
-  seven days is permanent.
+  client 500 requests a minute, 30,000 an hour and 300,000 a day. That leaves
+  room for programmatic use of an API, such as an automation fleet on another
+  host. A fleet of about 20 agents makes about 120 requests a minute at its
+  busiest. A gitea page opened with an empty cache loads about 40 files, so a
+  few such first visits fit in the minute limit too. The minute limit catches,
+  within seconds, a client firing requests as fast as it can. A client at the
+  minute limit for a whole hour reaches the hour limit exactly. The day limit
+  catches a client that keeps up 4 a second around the clock. Counting cannot
+  catch a client that stays under all three limits, such as one that keeps up 3
+  a second; only the protections below can. Byte limits give each client 4 GiB a
+  minute, 20 GiB an hour and 200 GiB a day. A client over a limit is refused and
+  banned. The first ban lasts an hour. Each repeat within a day makes the next
+  ban three times as long. A ban that would last more than seven days is
+  permanent.
 - Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
   default rule file bans such a client for seven days at its first probe. Its
   next request during the ban makes the ban permanent. The default rule file

SPEC.md

@@ -369,22 +369,37 @@ Conventions:
   owner's own networks. `SWWAF_RATE_LIMIT_EXEMPT_NETS` bypasses the request and
   byte limits only; the error burst, attack detection and bans still apply.
 - Many people can share one address, as in an office, on a mobile carrier or
-  through Tor. They are one client, and share its limits and its bans. The
-  default limits are sized for about 40 busy people on one address. A known
-  shared address busier than that belongs in `SWWAF_ALLOW_NETS` or
+  through Tor. They are one client, and share its limits and its bans. A known
+  shared address busier than the limits allow belongs in `SWWAF_ALLOW_NETS` or
   `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
 
 ### Request rate limits, per client
 
-Breaking one bans the client, so the defaults sit about twice above what 40
-people busy with heavy web apps make together on one address. Each such person
-makes about 75 requests in a busy minute, and 1,250 an hour for ten hours a day.
+Breaking one bans the client. The defaults suit programmatic use of an API, such
+as an automation fleet on another host, and still stop abusive bots. A fleet of
+20 agents, each making a request every 10 seconds at its busiest, makes 120
+requests a minute and 7,200 an hour. Working around the clock, it makes 172,800
+a day. It stays under all three limits.
 
-A client slower than those 40 people cannot be told from them by counting. Only
-the other checks can catch an abuser that slow. They include the trap paths, the
-rule files, the Core Rule Set and the blocklists. An abuser that trips none of
-them is not caught. A machine that talks to the app all day can still pass the
-day limit. Its address belongs in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
+- Per minute: 500, about 8 requests a second. It leaves room for the fleet's 120
+  a minute and a few quick first visits with an empty cache. A gitea page opened
+  with an empty cache loads about 40 files. A client firing requests as fast as
+  it can passes the limit within seconds.
+- Per hour: 30,000, about 8 requests a second for a whole hour. A client at the
+  minute limit for a whole hour reaches it exactly. The fleet's 7,200 an hour is
+  about a quarter of it.
+- Per day: 300,000, about 3.5 requests a second around the clock. A crawler that
+  keeps up 4 a second makes 345,600 a day, and passes it. At 240 a minute and
+  14,400 an hour, it stays under the other two limits.
+
+Counting cannot catch a client that stays under all three limits, such as one
+that keeps up 3 requests a second. Only the other checks can catch an abuser
+that slow. They include the trap paths, the rule files, the Core Rule Set and
+the blocklists. An abuser that trips none of them is not caught. A machine that
+keeps up more than 500 requests a minute passes the minute limit. One that keeps
+up more than about 3.5 a second all day passes the day limit, as two busy
+Actions runners on one address do. Such a machine's address belongs in
+`SWWAF_RATE_LIMIT_EXEMPT_NETS`.
 
 `SWWAF_RATE_LIMIT_EXEMPT_PATHS` takes path prefixes out of the count, such as
 static assets or health checks; their bytes are not counted either. A prefix is
@@ -397,12 +412,12 @@ matched the same way.
 
 ### Byte limits, per client
 
-Each of the same 40 people moves about 256 MiB an hour, for ten hours a day. The
-hour and day limits sit about twice above what they move together. A response's
-bytes are counted when it ends (see step 15). So the minute limit holds two of
-the largest responses, with their requests, ending in the same minute. No single
-download then breaks a limit. An operator who raises `SWWAF_RESPONSE_MAX_BYTES`
-raises the byte limits with it.
+The hour and day limits sit about twice above what 40 people busy with heavy web
+apps move together on one address. Each moves about 256 MiB an hour, for ten
+hours a day. A response's bytes are counted when it ends (see step 15). So the
+minute limit holds two of the largest responses, with their requests, ending in
+the same minute. No single download then breaks a limit. An operator who raises
+`SWWAF_RESPONSE_MAX_BYTES` raises the byte limits with it.
 
 ### Size and time limits
 

docs/alerts.md

@@ -35,34 +35,34 @@ on one line:
 ```json
 {
     "instance": "fsn1app1/gitea",
-    "time": "2026-10-06T12:00:00.123461Z",
+    "time": "2026-10-06T12:00:50.123461Z",
     "event": "ban",
     "client": "203.0.113.9",
     "netblock": "203.0.113.9/32",
     "asn": "",
     "as_name": "",
     "country": "",
-    "reason": "requests per minute over the limit of 6000",
+    "reason": "requests per minute over the limit of 500",
     "detail": {
-        "ban_expires": "2026-10-06T13:00:00.123Z",
+        "ban_expires": "2026-10-06T13:00:50.123Z",
         "cause": "limit",
         "notes": {
             "asn": "",
             "as_name": "",
             "country": "",
             "kind": "requests",
-            "limit": 6000,
+            "limit": 500,
             "window": "minute",
-            "count": 6001,
+            "count": 501,
             "request": {
-                "time": "2026-10-06T12:00:00.123456789Z",
+                "time": "2026-10-06T12:00:50.123456789Z",
                 "method": "GET",
                 "host": "app.example",
-                "path": "/owner/repo/commits/branch/main?page=812",
+                "path": "/owner/repo/commits/branch/main?page=501",
                 "status": 403,
                 "user_agent": "scraper/1.0"
             },
-            "requests": 8210,
+            "requests": 501,
             "refused": 0,
             "earlier_bans": {
                 "limit": 0,
@@ -89,7 +89,7 @@ on one line:
 - `asn`, `as_name` and `country`: the client's, as known when the alert was
   raised. They are empty when GeoJS had not answered yet.
 - `reason`: a short sentence, such as
-  `requests per minute over the limit of 6000` or `listed by a blocklist`.
+  `requests per minute over the limit of 500` or `listed by a blocklist`.
 - `detail`: what is particular to the event:
     - for a ban, its `cause`, `ban_expires` and `notes` (the notes are listed
       under `bans.json` in [`state-files.md`](state-files.md));

docs/gitea.md

@@ -34,11 +34,12 @@ checked again when gitea or the Core Rule Set changes version.
   day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
   43,200 requests a day. While it runs a job, it reports the job's log and state
   every second instead, two requests a second. So one runner busy all day sends
-  172,800 requests, under the request limits. Six such runners on one address,
-  each busy all day, pass the day limit of 1,000,000 within two days; five stay
-  under it. Their address is then banned, their running jobs fail, and each
-  repeat within a day makes the ban longer. Newer runners ask for work less
-  often. Put the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
+  120 requests a minute and 172,800 a day, under the request limits. Two such
+  runners on one address, each busy all day, make 345,600 and pass the day limit
+  of 300,000. Five busy at once make 600 a minute and pass the minute limit
+  of 500. Their address is then banned, their running jobs fail, and each repeat
+  within a day makes the ban longer. Newer runners ask for work less often. Put
+  the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
 
 ## What the Core Rule Set lets through
 

docs/settings.md

@@ -99,9 +99,21 @@ that take `off`, the `_FILE` form, and what each group does and why.
 
 ## Request rate limits, per client
 
-- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `6000`), `SWWAF_RATE_LIMIT_PER_HOUR`
-  (default `100000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `1000000`): the
-  most requests a client may make in each window. A count or `off`.
+- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `500`, about 8 a second): the most
+  requests a client may make in a minute. A count or `off`. The default catches,
+  within seconds, a client firing requests as fast as it can. It leaves room for
+  a fleet of about 20 agents at its busiest, about 120 requests a minute. It
+  also leaves room for a few quick first visits to gitea with an empty cache,
+  about 40 files each.
+- `SWWAF_RATE_LIMIT_PER_HOUR` (default `30000`, about 8 a second): the most
+  requests a client may make in an hour. A count or `off`. The default leaves
+  room for an automation fleet at work. A client at the minute limit for a whole
+  hour reaches it exactly.
+- `SWWAF_RATE_LIMIT_PER_DAY` (default `300000`, about 3.5 a second): the most
+  requests a client may make in a day. A count or `off`. The default catches a
+  crawler that keeps up 4 a second around the clock.
+- Counting cannot catch a client that stays under all three limits (see "Request
+  rate limits, per client" in [`SPEC.md`](../SPEC.md)).
 - `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
   rate limiting and the byte limits, such as `/assets/`.
 

internal/config/config.go

@@ -506,9 +506,9 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
 		AllowNets:               env.netblocks("SWWAF_ALLOW_NETS", ""),
 		RateLimitExemptNets:     env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
 		DenyNets:                env.netblocks("SWWAF_DENY_NETS", ""),
-		RateLimitPerMinute:      env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "6000"),
-		RateLimitPerHour:        env.count("SWWAF_RATE_LIMIT_PER_HOUR", "100000"),
-		RateLimitPerDay:         env.count("SWWAF_RATE_LIMIT_PER_DAY", "1000000"),
+		RateLimitPerMinute:      env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "500"),
+		RateLimitPerHour:        env.count("SWWAF_RATE_LIMIT_PER_HOUR", "30000"),
+		RateLimitPerDay:         env.count("SWWAF_RATE_LIMIT_PER_DAY", "300000"),
 		RateLimitExemptPaths:    env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
 		BytesLimitPerMinute:     env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "4G"),
 		BytesLimitPerHour:       env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "20G"),

internal/config/config_test.go

@@ -262,9 +262,9 @@ func TestDefaults(t *testing.T) {
 		UpstreamResponseTimeout:     30 * time.Minute,
 		RequestMaxBytes:             100 << 20,
 		ResponseMaxBytes:            1 << 30,
-		RateLimitPerMinute:          6000,
-		RateLimitPerHour:            100000,
-		RateLimitPerDay:             1000000,
+		RateLimitPerMinute:          500,
+		RateLimitPerHour:            30000,
+		RateLimitPerDay:             300000,
 		BanResponse:                 403,
 		LimitBanDuration:            time.Hour,
 		LimitBanRepeatWindow:        24 * time.Hour,
@@ -2375,9 +2375,9 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
 		allowNets:               "",
 		rateLimitExemptNets:     "",
 		denyNets:                "",
-		rateLimitPerMinute:      "6000",
-		rateLimitPerHour:        "100000",
-		rateLimitPerDay:         "1000000",
+		rateLimitPerMinute:      "500",
+		rateLimitPerHour:        "30000",
+		rateLimitPerDay:         "300000",
 		rateLimitExemptPaths:    "",
 		bytesLimitPerMinute:     "4G",
 		bytesLimitPerHour:       "20G",

internal/smallwebwaf/smallwebwaf_test.go

@@ -1244,9 +1244,9 @@ func wantStartingLine(t *testing.T, line map[string]any, appURL, dir string) {
 		"SWWAF_ALLOW_NETS":                      "",
 		rateLimitExemptNets:                     "",
 		"SWWAF_DENY_NETS":                       "",
-		"SWWAF_RATE_LIMIT_PER_MINUTE":           "6000",
-		"SWWAF_RATE_LIMIT_PER_HOUR":             "100000",
-		rateLimitPerDay:                         "1000000",
+		"SWWAF_RATE_LIMIT_PER_MINUTE":           "500",
+		"SWWAF_RATE_LIMIT_PER_HOUR":             "30000",
+		rateLimitPerDay:                         "300000",
 		"SWWAF_RATE_LIMIT_EXEMPT_PATHS":         "",
 		"SWWAF_DENIED_COUNTRIES":                "",
 		"SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES":   "",