Default request limits per minute, hour and day, sized for API use (closes #176)

The default request limits now suit programmatic use of an API, such
as an automation fleet on another host, and still catch abusive bots:

- per minute: 500, room for a fleet's busiest minute and a few first
  visits with an empty cache, and passed within seconds by a client
  firing as fast as it can;
- per hour: 20000, passed by a client that keeps up 6 a second;
- per day: 300000, passed by a scraper that keeps up 4 a second.

The byte limits and ban lengths are unchanged. SPEC.md, README.md and
docs/settings.md give the reasons; docs/gitea.md redoes the Actions
runner sums.

Judgement call: the docs/alerts.md ban example shows the minute limit.

Model: opus-5-5

README.md

@@ -17,20 +17,20 @@ reference documents under [`docs/`](docs/) are listed under "Documents" below.
 ## What it protects against
 
 - Scrapers that fetch every page of a site, all day. Rate limiting gives each
-  client 500 requests a minute, 30,000 an hour and 300,000 a day. That leaves
+  client 500 requests a minute, 20,000 an hour and 300,000 a day. That leaves
   room for programmatic use of an API, such as an automation fleet on another
   host. A fleet of about 20 agents makes about 120 requests a minute at its
   busiest. A gitea page opened with an empty cache loads about 40 files, so a
   few such first visits fit in the minute limit too. The minute limit catches,
-  within seconds, a client firing requests as fast as it can. A client at the
+  within seconds, a client firing requests as fast as it can. The hour limit
-  minute limit for a whole hour reaches the hour limit exactly. The day limit
+  catches a client that keeps up more than about 5.5 a second for an hour. The
-  catches a client that keeps up 4 a second around the clock. Counting cannot
+  day limit catches a client that keeps up 4 a second around the clock. Counting
-  catch a client that stays under all three limits, such as one that keeps up 3
+  cannot catch a client that stays under all three limits, such as one that
-  a second; only the protections below can. Byte limits give each client 4 GiB a
+  keeps up 3 a second; only the protections below can. Byte limits give each
-  minute, 20 GiB an hour and 200 GiB a day. A client over a limit is refused and
+  client 4 GiB a minute, 20 GiB an hour and 200 GiB a day. A client over a limit
-  banned. The first ban lasts an hour. Each repeat within a day makes the next
+  is refused and banned. The first ban lasts an hour. Each repeat within a day
-  ban three times as long. A ban that would last more than seven days is
+  makes the next ban three times as long. A ban that would last more than seven
-  permanent.
+  days is permanent.
 - Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
   default rule file bans such a client for seven days at its first probe. Its
   next request during the ban makes the ban permanent. The default rule file

SPEC.md

@@ -385,9 +385,9 @@ a day. It stays under all three limits.
   a minute and a few quick first visits with an empty cache. A gitea page opened
   with an empty cache loads about 40 files. A client firing requests as fast as
   it can passes the limit within seconds.
-- Per hour: 30,000, about 8 requests a second for a whole hour. A client at the
+- Per hour: 20,000, about 5.5 requests a second for a whole hour. A client that
-  minute limit for a whole hour reaches it exactly. The fleet's 7,200 an hour is
+  keeps up 6 a second makes 21,600 in an hour, and passes it, at 360 a minute,
-  about a quarter of it.
+  under the minute limit. The fleet's 7,200 an hour is about a third of it.
 - Per day: 300,000, about 3.5 requests a second around the clock. A scraper that
   keeps up 4 a second makes 345,600 a day, and passes it. At 240 a minute and
   14,400 an hour, it stays under the other two limits.
@@ -397,7 +397,8 @@ that keeps up 3 requests a second. Only the other checks can catch an abuser
 that slow. They include the trap paths, the rule files, the Core Rule Set and
 the blocklists. An abuser that trips none of them is not caught. A machine that
 keeps up more than 500 requests a minute passes the minute limit. One that keeps
-up more than about 3.5 a second all day passes the day limit, as two busy
+up more than about 5.5 a second for an hour passes the hour limit. One that
+keeps up more than about 3.5 a second all day passes the day limit, as two busy
 Actions runners on one address do. Such a machine's address belongs in
 `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
 

docs/gitea.md

@@ -36,10 +36,11 @@ checked again when gitea or the Core Rule Set changes version.
   every second instead, two requests a second. So one runner busy all day sends
   120 requests a minute and 172,800 a day, under the request limits. Two such
   runners on one address, each busy all day, make 345,600 and pass the day limit
-  of 300,000. Five busy at once make 600 a minute and pass the minute limit
+  of 300,000. Three busy at once make 21,600 an hour and pass the hour limit of
-  of 500. Their address is then banned, their running jobs fail, and each repeat
+  20,000. Five busy at once make 600 a minute and pass the minute limit of 500.
-  within a day makes the ban longer. Newer runners ask for work less often. Put
+  Their address is then banned, their running jobs fail, and each repeat within
-  the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
+  a day makes the ban longer. Newer runners ask for work less often. Put the
+  runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
 
 ## What the Core Rule Set lets through
 

docs/settings.md

@@ -105,10 +105,10 @@ that take `off`, the `_FILE` form, and what each group does and why.
   a fleet of about 20 agents at its busiest, about 120 requests a minute. It
   also leaves room for a few quick first visits to gitea with an empty cache,
   about 40 files each.
-- `SWWAF_RATE_LIMIT_PER_HOUR` (default `30000`, about 8 a second): the most
+- `SWWAF_RATE_LIMIT_PER_HOUR` (default `20000`, about 5.5 a second): the most
   requests a client may make in an hour. A count or `off`. The default leaves
-  room for an automation fleet at work. A client at the minute limit for a whole
+  room for an automation fleet at work. It catches a client that keeps up more
-  hour reaches it exactly.
+  than about 5.5 a second for an hour.
 - `SWWAF_RATE_LIMIT_PER_DAY` (default `300000`, about 3.5 a second): the most
   requests a client may make in a day. A count or `off`. The default catches a
   scraper that keeps up 4 a second around the clock.

internal/config/config.go

@@ -507,7 +507,7 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
 		RateLimitExemptNets:     env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
 		DenyNets:                env.netblocks("SWWAF_DENY_NETS", ""),
 		RateLimitPerMinute:      env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "500"),
-		RateLimitPerHour:        env.count("SWWAF_RATE_LIMIT_PER_HOUR", "30000"),
+		RateLimitPerHour:        env.count("SWWAF_RATE_LIMIT_PER_HOUR", "20000"),
 		RateLimitPerDay:         env.count("SWWAF_RATE_LIMIT_PER_DAY", "300000"),
 		RateLimitExemptPaths:    env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
 		BytesLimitPerMinute:     env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "4G"),

internal/config/config_test.go

@@ -263,7 +263,7 @@ func TestDefaults(t *testing.T) {
 		RequestMaxBytes:             100 << 20,
 		ResponseMaxBytes:            1 << 30,
 		RateLimitPerMinute:          500,
-		RateLimitPerHour:            30000,
+		RateLimitPerHour:            20000,
 		RateLimitPerDay:             300000,
 		BanResponse:                 403,
 		LimitBanDuration:            time.Hour,
@@ -2376,7 +2376,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
 		rateLimitExemptNets:     "",
 		denyNets:                "",
 		rateLimitPerMinute:      "500",
-		rateLimitPerHour:        "30000",
+		rateLimitPerHour:        "20000",
 		rateLimitPerDay:         "300000",
 		rateLimitExemptPaths:    "",
 		bytesLimitPerMinute:     "4G",

internal/smallwebwaf/smallwebwaf_test.go

@@ -1245,7 +1245,7 @@ func wantStartingLine(t *testing.T, line map[string]any, appURL, dir string) {
 		rateLimitExemptNets:                     "",
 		"SWWAF_DENY_NETS":                       "",
 		"SWWAF_RATE_LIMIT_PER_MINUTE":           "500",
-		"SWWAF_RATE_LIMIT_PER_HOUR":             "30000",
+		"SWWAF_RATE_LIMIT_PER_HOUR":             "20000",
 		rateLimitPerDay:                         "300000",
 		"SWWAF_RATE_LIMIT_EXEMPT_PATHS":         "",
 		"SWWAF_DENIED_COUNTRIES":                "",