Default request limits per hour and per day, sized for API use (closes #176)
check / check (push) Failing after 1m45s

The default request limits now suit programmatic use of an API, such
as an automation fleet on another host, and still catch crawlers:

- per minute: off;
- per hour: 30000, passed by a crawler that keeps up 10 a second;
- per day: 300000, passed by one that keeps up 4 a second.

The byte limits and ban lengths are unchanged. SPEC.md, README.md and
docs/settings.md give the reasons and what counting cannot catch;
docs/gitea.md redoes the Actions runner sums.

Judgement call: the ban alert example in docs/alerts.md now shows the
hour limit, since the minute limit is off by default.

Model: opus-5-5
This commit is contained in:
2026-11-04 19:50:41 +00:00
parent fdabf4974b
commit 35054fe94b
8 changed files with 78 additions and 52 deletions
+9 -9
View File
@@ -35,34 +35,34 @@ on one line:
```json
{
"instance": "fsn1app1/gitea",
"time": "2026-10-06T12:00:00.123461Z",
"time": "2026-10-06T12:50:00.123461Z",
"event": "ban",
"client": "203.0.113.9",
"netblock": "203.0.113.9/32",
"asn": "",
"as_name": "",
"country": "",
"reason": "requests per minute over the limit of 6000",
"reason": "requests per hour over the limit of 30000",
"detail": {
"ban_expires": "2026-10-06T13:00:00.123Z",
"ban_expires": "2026-10-06T13:50:00.123Z",
"cause": "limit",
"notes": {
"asn": "",
"as_name": "",
"country": "",
"kind": "requests",
"limit": 6000,
"window": "minute",
"count": 6001,
"limit": 30000,
"window": "hour",
"count": 30001,
"request": {
"time": "2026-10-06T12:00:00.123456789Z",
"time": "2026-10-06T12:50:00.123456789Z",
"method": "GET",
"host": "app.example",
"path": "/owner/repo/commits/branch/main?page=812",
"status": 403,
"user_agent": "scraper/1.0"
},
"requests": 8210,
"requests": 30001,
"refused": 0,
"earlier_bans": {
"limit": 0,
@@ -89,7 +89,7 @@ on one line:
- `asn`, `as_name` and `country`: the client's, as known when the alert was
raised. They are empty when GeoJS had not answered yet.
- `reason`: a short sentence, such as
`requests per minute over the limit of 6000` or `listed by a blocklist`.
`requests per hour over the limit of 30000` or `listed by a blocklist`.
- `detail`: what is particular to the event:
- for a ban, its `cause`, `ban_expires` and `notes` (the notes are listed
under `bans.json` in [`state-files.md`](state-files.md));
+2 -2
View File
@@ -34,8 +34,8 @@ checked again when gitea or the Core Rule Set changes version.
day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
43,200 requests a day. While it runs a job, it reports the job's log and state
every second instead, two requests a second. So one runner busy all day sends
172,800 requests, under the request limits. Six such runners on one address,
each busy all day, pass the day limit of 1,000,000 within two days; five stay
172,800 requests, under the request limits. Two such runners on one address,
each busy all day, make 345,600 and pass the day limit of 300,000; one stays
under it. Their address is then banned, their running jobs fail, and each
repeat within a day makes the ban longer. Newer runners ask for work less
often. Put the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
+12 -3
View File
@@ -99,9 +99,18 @@ that take `off`, the `_FILE` form, and what each group does and why.
## Request rate limits, per client
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `6000`), `SWWAF_RATE_LIMIT_PER_HOUR`
(default `100000`) and `SWWAF_RATE_LIMIT_PER_DAY` (default `1000000`): the
most requests a client may make in each window. A count or `off`.
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `off`): the most requests a client may
make in a minute. A count or `off`. It is off by default so that a short burst
of API requests breaks no limit on its own.
- `SWWAF_RATE_LIMIT_PER_HOUR` (default `30000`, about 8 a second): the most
requests a client may make in an hour. A count or `off`. The default leaves
room for an automation fleet at work, and catches a crawler that keeps up 10 a
second.
- `SWWAF_RATE_LIMIT_PER_DAY` (default `300000`, about 3.5 a second): the most
requests a client may make in a day. A count or `off`. The default catches a
crawler that keeps up 4 a second around the clock.
- Counting cannot catch a client slower than these limits (see "Request rate
limits, per client" in [`SPEC.md`](../SPEC.md)).
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
rate limiting and the byte limits, such as `/assets/`.