Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
776f988956 |
@@ -17,20 +17,20 @@ reference documents under [`docs/`](docs/) are listed under "Documents" below.
|
||||
## What it protects against
|
||||
|
||||
- Scrapers that fetch every page of a site, all day. Rate limiting gives each
|
||||
client 500 requests a minute, 20,000 an hour and 300,000 a day. That leaves
|
||||
client 500 requests a minute, 30,000 an hour and 300,000 a day. That leaves
|
||||
room for programmatic use of an API, such as an automation fleet on another
|
||||
host. A fleet of about 20 agents makes about 120 requests a minute at its
|
||||
busiest. A gitea page opened with an empty cache loads about 40 files, so a
|
||||
few such first visits fit in the minute limit too. The minute limit catches,
|
||||
within seconds, a client firing requests as fast as it can. The hour limit
|
||||
catches a client that keeps up more than about 5.5 a second for an hour. The
|
||||
day limit catches a client that keeps up 4 a second around the clock. Counting
|
||||
cannot catch a client that stays under all three limits, such as one that
|
||||
keeps up 3 a second; only the protections below can. Byte limits give each
|
||||
client 4 GiB a minute, 20 GiB an hour and 200 GiB a day. A client over a limit
|
||||
is refused and banned. The first ban lasts an hour. Each repeat within a day
|
||||
makes the next ban three times as long. A ban that would last more than seven
|
||||
days is permanent.
|
||||
within seconds, a client firing requests as fast as it can. A client at the
|
||||
minute limit for a whole hour reaches the hour limit exactly. The day limit
|
||||
catches a client that keeps up 4 a second around the clock. Counting cannot
|
||||
catch a client that stays under all three limits, such as one that keeps up 3
|
||||
a second; only the protections below can. Byte limits give each client 4 GiB a
|
||||
minute, 20 GiB an hour and 200 GiB a day. A client over a limit is refused and
|
||||
banned. The first ban lasts an hour. Each repeat within a day makes the next
|
||||
ban three times as long. A ban that would last more than seven days is
|
||||
permanent.
|
||||
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
|
||||
default rule file bans such a client for seven days at its first probe. Its
|
||||
next request during the ban makes the ban permanent. The default rule file
|
||||
|
||||
@@ -385,9 +385,9 @@ a day. It stays under all three limits.
|
||||
a minute and a few quick first visits with an empty cache. A gitea page opened
|
||||
with an empty cache loads about 40 files. A client firing requests as fast as
|
||||
it can passes the limit within seconds.
|
||||
- Per hour: 20,000, about 5.5 requests a second for a whole hour. A client that
|
||||
keeps up 6 a second makes 21,600 in an hour, and passes it, at 360 a minute,
|
||||
under the minute limit. The fleet's 7,200 an hour is about a third of it.
|
||||
- Per hour: 30,000, about 8 requests a second for a whole hour. A client at the
|
||||
minute limit for a whole hour reaches it exactly. The fleet's 7,200 an hour is
|
||||
about a quarter of it.
|
||||
- Per day: 300,000, about 3.5 requests a second around the clock. A scraper that
|
||||
keeps up 4 a second makes 345,600 a day, and passes it. At 240 a minute and
|
||||
14,400 an hour, it stays under the other two limits.
|
||||
@@ -397,8 +397,7 @@ that keeps up 3 requests a second. Only the other checks can catch an abuser
|
||||
that slow. They include the trap paths, the rule files, the Core Rule Set and
|
||||
the blocklists. An abuser that trips none of them is not caught. A machine that
|
||||
keeps up more than 500 requests a minute passes the minute limit. One that keeps
|
||||
up more than about 5.5 a second for an hour passes the hour limit. One that
|
||||
keeps up more than about 3.5 a second all day passes the day limit, as two busy
|
||||
up more than about 3.5 a second all day passes the day limit, as two busy
|
||||
Actions runners on one address do. Such a machine's address belongs in
|
||||
`SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||
|
||||
|
||||
+4
-5
@@ -36,11 +36,10 @@ checked again when gitea or the Core Rule Set changes version.
|
||||
every second instead, two requests a second. So one runner busy all day sends
|
||||
120 requests a minute and 172,800 a day, under the request limits. Two such
|
||||
runners on one address, each busy all day, make 345,600 and pass the day limit
|
||||
of 300,000. Three busy at once make 21,600 an hour and pass the hour limit of
|
||||
20,000. Five busy at once make 600 a minute and pass the minute limit of 500.
|
||||
Their address is then banned, their running jobs fail, and each repeat within
|
||||
a day makes the ban longer. Newer runners ask for work less often. Put the
|
||||
runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||
of 300,000. Five busy at once make 600 a minute and pass the minute limit
|
||||
of 500. Their address is then banned, their running jobs fail, and each repeat
|
||||
within a day makes the ban longer. Newer runners ask for work less often. Put
|
||||
the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||
|
||||
## What the Core Rule Set lets through
|
||||
|
||||
|
||||
+3
-3
@@ -105,10 +105,10 @@ that take `off`, the `_FILE` form, and what each group does and why.
|
||||
a fleet of about 20 agents at its busiest, about 120 requests a minute. It
|
||||
also leaves room for a few quick first visits to gitea with an empty cache,
|
||||
about 40 files each.
|
||||
- `SWWAF_RATE_LIMIT_PER_HOUR` (default `20000`, about 5.5 a second): the most
|
||||
- `SWWAF_RATE_LIMIT_PER_HOUR` (default `30000`, about 8 a second): the most
|
||||
requests a client may make in an hour. A count or `off`. The default leaves
|
||||
room for an automation fleet at work. It catches a client that keeps up more
|
||||
than about 5.5 a second for an hour.
|
||||
room for an automation fleet at work. A client at the minute limit for a whole
|
||||
hour reaches it exactly.
|
||||
- `SWWAF_RATE_LIMIT_PER_DAY` (default `300000`, about 3.5 a second): the most
|
||||
requests a client may make in a day. A count or `off`. The default catches a
|
||||
scraper that keeps up 4 a second around the clock.
|
||||
|
||||
@@ -507,7 +507,7 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
|
||||
RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
|
||||
DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
|
||||
RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "500"),
|
||||
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "20000"),
|
||||
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "30000"),
|
||||
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "300000"),
|
||||
RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
|
||||
BytesLimitPerMinute: env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "4G"),
|
||||
|
||||
@@ -263,7 +263,7 @@ func TestDefaults(t *testing.T) {
|
||||
RequestMaxBytes: 100 << 20,
|
||||
ResponseMaxBytes: 1 << 30,
|
||||
RateLimitPerMinute: 500,
|
||||
RateLimitPerHour: 20000,
|
||||
RateLimitPerHour: 30000,
|
||||
RateLimitPerDay: 300000,
|
||||
BanResponse: 403,
|
||||
LimitBanDuration: time.Hour,
|
||||
@@ -2376,7 +2376,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
||||
rateLimitExemptNets: "",
|
||||
denyNets: "",
|
||||
rateLimitPerMinute: "500",
|
||||
rateLimitPerHour: "20000",
|
||||
rateLimitPerHour: "30000",
|
||||
rateLimitPerDay: "300000",
|
||||
rateLimitExemptPaths: "",
|
||||
bytesLimitPerMinute: "4G",
|
||||
|
||||
@@ -1245,7 +1245,7 @@ func wantStartingLine(t *testing.T, line map[string]any, appURL, dir string) {
|
||||
rateLimitExemptNets: "",
|
||||
"SWWAF_DENY_NETS": "",
|
||||
"SWWAF_RATE_LIMIT_PER_MINUTE": "500",
|
||||
"SWWAF_RATE_LIMIT_PER_HOUR": "20000",
|
||||
"SWWAF_RATE_LIMIT_PER_HOUR": "30000",
|
||||
rateLimitPerDay: "300000",
|
||||
"SWWAF_RATE_LIMIT_EXEMPT_PATHS": "",
|
||||
"SWWAF_DENIED_COUNTRIES": "",
|
||||
|
||||
Reference in New Issue
Block a user