1 Commits
Author SHA1 Message Date
sneak a55b4c789c Default request limits per minute, hour and day, sized for API use (closes #176)
check / check (push) Successful in 4m45s
The default request limits now suit programmatic use of an API, such
as an automation fleet on another host, and still catch abusive bots:

- per minute: 500, room for a fleet's busiest minute and a few first
  visits with an empty cache, and passed within seconds by a client
  firing as fast as it can;
- per hour: 30000, which a client at the minute limit for a whole
  hour reaches exactly;
- per day: 300000, passed by a crawler that keeps up 4 a second.

The byte limits and ban lengths are unchanged. SPEC.md, README.md and
docs/settings.md give the reasons; docs/gitea.md redoes the Actions
runner sums.

Judgement call: the docs/alerts.md ban example shows the minute limit.

Model: opus-5-5
2026-11-04 20:30:11 +00:00
8 changed files with 44 additions and 41 deletions

No files matched your search

+13 -10
View File
@@ -17,17 +17,20 @@ reference documents under [`docs/`](docs/) are listed under "Documents" below.
## What it protects against ## What it protects against
- Scrapers that fetch every page of a site, all day. Rate limiting gives each - Scrapers that fetch every page of a site, all day. Rate limiting gives each
client 300 requests a minute, 30,000 an hour and 300,000 a day. That leaves client 500 requests a minute, 30,000 an hour and 300,000 a day. That leaves
room for programmatic use of an API, such as an automation fleet on another room for programmatic use of an API, such as an automation fleet on another
host. The minute limit catches, within seconds, a client firing requests as host. A fleet of about 20 agents makes about 120 requests a minute at its
fast as it can. With the minute limit raised or off, the hour limit catches busiest. A Gitea page opened with an empty cache loads about 40 files, so a
one that keeps up 10 a second. The day limit catches one that keeps up 4 a few such first visits fit in the minute limit too. The minute limit catches,
second around the clock. Counting cannot catch a client that stays under all within seconds, a client firing requests as fast as it can. A client at the
three limits, such as one that keeps up 3 a second; only the protections below minute limit for a whole hour reaches the hour limit exactly. The day limit
can. Byte limits give each client 4 GiB a minute, 20 GiB an hour and 200 GiB a catches a client that keeps up 4 a second around the clock. Counting cannot
day. A client over a limit is refused and banned. The first ban lasts an hour. catch a client that stays under all three limits, such as one that keeps up 3
Each repeat within a day makes the next ban three times as long. A ban that a second; only the protections below can. Byte limits give each client 4 GiB a
would last more than seven days is permanent. minute, 20 GiB an hour and 200 GiB a day. A client over a limit is refused and
banned. The first ban lasts an hour. Each repeat within a day makes the next
ban three times as long. A ban that would last more than seven days is
permanent.
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The - Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
default rule file bans such a client for seven days at its first probe. Its default rule file bans such a client for seven days at its first probe. Its
next request during the ban makes the ban permanent. The default rule file next request during the ban makes the ban permanent. The default rule file
+8 -9
View File
@@ -381,14 +381,13 @@ as an automation fleet on another host, and still stop abusive bots. A fleet of
requests a minute and 7,200 an hour. Working around the clock, it makes 172,800 requests a minute and 7,200 an hour. Working around the clock, it makes 172,800
a day. It stays under all three limits. a day. It stays under all three limits.
- Per minute: 300, 5 requests a second. A client firing requests as fast as it - Per minute: 500, about 8 requests a second. It leaves room for the fleet's 120
can passes it within seconds. It leaves room for the fleet and a few quick a minute and a few quick first visits with an empty cache. A Gitea page opened
first visits with an empty cache. A Gitea page opened with an empty cache with an empty cache loads about 40 files. A client firing requests as fast as
loads about 40 files. it can passes the limit within seconds.
- Per hour: 30,000, about 8 requests a second for a whole hour. At the default - Per hour: 30,000, about 8 requests a second for a whole hour. A client at the
minute limit no client reaches it, since 300 a minute is at most 18,000 an minute limit for a whole hour reaches it exactly. The fleet's 7,200 an hour is
hour. With the minute limit raised or off, a crawler that keeps up 10 a second about a quarter of it.
makes 36,000 an hour, and passes it.
- Per day: 300,000, about 3.5 requests a second around the clock. A crawler that - Per day: 300,000, about 3.5 requests a second around the clock. A crawler that
keeps up 4 a second makes 345,600 a day, and passes it. At 240 a minute and keeps up 4 a second makes 345,600 a day, and passes it. At 240 a minute and
14,400 an hour, it stays under the other two limits. 14,400 an hour, it stays under the other two limits.
@@ -397,7 +396,7 @@ Counting cannot catch a client that stays under all three limits, such as one
that keeps up 3 requests a second. Only the other checks can catch an abuser that keeps up 3 requests a second. Only the other checks can catch an abuser
that slow. They include the trap paths, the rule files, the Core Rule Set and that slow. They include the trap paths, the rule files, the Core Rule Set and
the blocklists. An abuser that trips none of them is not caught. A machine that the blocklists. An abuser that trips none of them is not caught. A machine that
keeps up more than 5 requests a second passes the minute limit. One that keeps keeps up more than 500 requests a minute passes the minute limit. One that keeps
up more than about 3.5 a second all day passes the day limit, as two busy up more than about 3.5 a second all day passes the day limit, as two busy
Actions runners on one address do. Such a machine's address belongs in Actions runners on one address do. Such a machine's address belongs in
`SWWAF_RATE_LIMIT_EXEMPT_NETS`. `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
+9 -9
View File
@@ -35,34 +35,34 @@ on one line:
```json ```json
{ {
"instance": "fsn1app1/gitea", "instance": "fsn1app1/gitea",
"time": "2026-10-06T12:00:30.123461Z", "time": "2026-10-06T12:00:50.123461Z",
"event": "ban", "event": "ban",
"client": "203.0.113.9", "client": "203.0.113.9",
"netblock": "203.0.113.9/32", "netblock": "203.0.113.9/32",
"asn": "", "asn": "",
"as_name": "", "as_name": "",
"country": "", "country": "",
"reason": "requests per minute over the limit of 300", "reason": "requests per minute over the limit of 500",
"detail": { "detail": {
"ban_expires": "2026-10-06T13:00:30.123Z", "ban_expires": "2026-10-06T13:00:50.123Z",
"cause": "limit", "cause": "limit",
"notes": { "notes": {
"asn": "", "asn": "",
"as_name": "", "as_name": "",
"country": "", "country": "",
"kind": "requests", "kind": "requests",
"limit": 300, "limit": 500,
"window": "minute", "window": "minute",
"count": 301, "count": 501,
"request": { "request": {
"time": "2026-10-06T12:00:30.123456789Z", "time": "2026-10-06T12:00:50.123456789Z",
"method": "GET", "method": "GET",
"host": "app.example", "host": "app.example",
"path": "/owner/repo/commits/branch/main?page=301", "path": "/owner/repo/commits/branch/main?page=501",
"status": 403, "status": 403,
"user_agent": "scraper/1.0" "user_agent": "scraper/1.0"
}, },
"requests": 301, "requests": 501,
"refused": 0, "refused": 0,
"earlier_bans": { "earlier_bans": {
"limit": 0, "limit": 0,
@@ -89,7 +89,7 @@ on one line:
- `asn`, `as_name` and `country`: the client's, as known when the alert was - `asn`, `as_name` and `country`: the client's, as known when the alert was
raised. They are empty when GeoJS had not answered yet. raised. They are empty when GeoJS had not answered yet.
- `reason`: a short sentence, such as - `reason`: a short sentence, such as
`requests per minute over the limit of 300` or `listed by a blocklist`. `requests per minute over the limit of 500` or `listed by a blocklist`.
- `detail`: what is particular to the event: - `detail`: what is particular to the event:
- for a ban, its `cause`, `ban_expires` and `notes` (the notes are listed - for a ban, its `cause`, `ban_expires` and `notes` (the notes are listed
under `bans.json` in [`state-files.md`](state-files.md)); under `bans.json` in [`state-files.md`](state-files.md));
+2 -2
View File
@@ -36,8 +36,8 @@ checked again when gitea or the Core Rule Set changes version.
every second instead, two requests a second. So one runner busy all day sends every second instead, two requests a second. So one runner busy all day sends
120 requests a minute and 172,800 a day, under the request limits. Two such 120 requests a minute and 172,800 a day, under the request limits. Two such
runners on one address, each busy all day, make 345,600 and pass the day limit runners on one address, each busy all day, make 345,600 and pass the day limit
of 300,000. Three busy at once make 360 a minute and pass the minute limit of 300,000. Five busy at once make 600 a minute and pass the minute limit
of 300. Their address is then banned, their running jobs fail, and each repeat of 500. Their address is then banned, their running jobs fail, and each repeat
within a day makes the ban longer. Newer runners ask for work less often. Put within a day makes the ban longer. Newer runners ask for work less often. Put
the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
+8 -7
View File
@@ -99,15 +99,16 @@ that take `off`, the `_FILE` form, and what each group does and why.
## Request rate limits, per client ## Request rate limits, per client
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `300`, 5 a second): the most requests a - `SWWAF_RATE_LIMIT_PER_MINUTE` (default `500`, about 8 a second): the most
client may make in a minute. A count or `off`. The default catches, within requests a client may make in a minute. A count or `off`. The default catches,
seconds, a client firing requests as fast as it can. It leaves room for an within seconds, a client firing requests as fast as it can. It leaves room for
automation fleet's busiest minute and a few quick first visits with an empty a fleet of about 20 agents at its busiest, about 120 requests a minute. It
cache. also leaves room for a few quick first visits to Gitea with an empty cache,
about 40 files each.
- `SWWAF_RATE_LIMIT_PER_HOUR` (default `30000`, about 8 a second): the most - `SWWAF_RATE_LIMIT_PER_HOUR` (default `30000`, about 8 a second): the most
requests a client may make in an hour. A count or `off`. The default leaves requests a client may make in an hour. A count or `off`. The default leaves
room for an automation fleet at work. With the minute limit raised or off, it room for an automation fleet at work. A client at the minute limit for a whole
catches a crawler that keeps up 10 a second. hour reaches it exactly.
- `SWWAF_RATE_LIMIT_PER_DAY` (default `300000`, about 3.5 a second): the most - `SWWAF_RATE_LIMIT_PER_DAY` (default `300000`, about 3.5 a second): the most
requests a client may make in a day. A count or `off`. The default catches a requests a client may make in a day. A count or `off`. The default catches a
crawler that keeps up 4 a second around the clock. crawler that keeps up 4 a second around the clock.
+1 -1
View File
@@ -506,7 +506,7 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""), AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""),
RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""), RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
DenyNets: env.netblocks("SWWAF_DENY_NETS", ""), DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "300"), RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "500"),
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "30000"), RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "30000"),
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "300000"), RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "300000"),
RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""), RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
+2 -2
View File
@@ -262,7 +262,7 @@ func TestDefaults(t *testing.T) {
UpstreamResponseTimeout: 30 * time.Minute, UpstreamResponseTimeout: 30 * time.Minute,
RequestMaxBytes: 100 << 20, RequestMaxBytes: 100 << 20,
ResponseMaxBytes: 1 << 30, ResponseMaxBytes: 1 << 30,
RateLimitPerMinute: 300, RateLimitPerMinute: 500,
RateLimitPerHour: 30000, RateLimitPerHour: 30000,
RateLimitPerDay: 300000, RateLimitPerDay: 300000,
BanResponse: 403, BanResponse: 403,
@@ -2375,7 +2375,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
allowNets: "", allowNets: "",
rateLimitExemptNets: "", rateLimitExemptNets: "",
denyNets: "", denyNets: "",
rateLimitPerMinute: "300", rateLimitPerMinute: "500",
rateLimitPerHour: "30000", rateLimitPerHour: "30000",
rateLimitPerDay: "300000", rateLimitPerDay: "300000",
rateLimitExemptPaths: "", rateLimitExemptPaths: "",
+1 -1
View File
@@ -1244,7 +1244,7 @@ func wantStartingLine(t *testing.T, line map[string]any, appURL, dir string) {
"SWWAF_ALLOW_NETS": "", "SWWAF_ALLOW_NETS": "",
rateLimitExemptNets: "", rateLimitExemptNets: "",
"SWWAF_DENY_NETS": "", "SWWAF_DENY_NETS": "",
"SWWAF_RATE_LIMIT_PER_MINUTE": "300", "SWWAF_RATE_LIMIT_PER_MINUTE": "500",
"SWWAF_RATE_LIMIT_PER_HOUR": "30000", "SWWAF_RATE_LIMIT_PER_HOUR": "30000",
rateLimitPerDay: "300000", rateLimitPerDay: "300000",
"SWWAF_RATE_LIMIT_EXEMPT_PATHS": "", "SWWAF_RATE_LIMIT_EXEMPT_PATHS": "",