Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
a41d61e9ad | ||
|
|
84a9190d2b | ||
|
|
1bdf66cd21 |
@@ -1,13 +1,12 @@
|
|||||||
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
|
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
|
||||||
run example-app gitea-requests gitea-setup
|
run example-app gitea-requests
|
||||||
|
|
||||||
# Makefile targets are thin shims; the implementations live in script/
|
# Makefile targets are thin shims; the implementations live in script/
|
||||||
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
|
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
|
||||||
# of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
|
# of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
|
||||||
# which test checks. build and run are for working on the code by hand;
|
# which test checks. build and run are for working on the code by hand;
|
||||||
# example-app checks the image with an app built on it; gitea-requests
|
# example-app checks the image with an app built on it; gitea-requests
|
||||||
# reports the gitea requests the Core Rule Set refuses at the defaults;
|
# reports the gitea requests the Core Rule Set refuses at the defaults.
|
||||||
# gitea-setup checks the setup in deploy/gitea.
|
|
||||||
|
|
||||||
bootstrap:
|
bootstrap:
|
||||||
@script/bootstrap
|
@script/bootstrap
|
||||||
@@ -50,6 +49,3 @@ example-app:
|
|||||||
|
|
||||||
gitea-requests:
|
gitea-requests:
|
||||||
@script/gitea-requests
|
@script/gitea-requests
|
||||||
|
|
||||||
gitea-setup:
|
|
||||||
@script/gitea-setup
|
|
||||||
|
|||||||
@@ -26,10 +26,10 @@ It protects against:
|
|||||||
|
|
||||||
- Scrapers that fetch every page of a site, all day. Each client gets a limit on
|
- Scrapers that fetch every page of a site, all day. Each client gets a limit on
|
||||||
requests and on bytes per minute, per hour and per day. The defaults leave
|
requests and on bytes per minute, per hour and per day. The defaults leave
|
||||||
room for a person reading a site for hours. They catch a scraper that keeps up
|
room for programmatic use of an API, such as an automation fleet, and catch a
|
||||||
one request every 2 seconds. A client over a limit is refused and banned: for
|
client firing requests as fast as it can within seconds. A client over a limit
|
||||||
a while at first, for three times as long at each repeat, and permanently when
|
is refused and banned: for a while at first, for three times as long at each
|
||||||
it keeps at it.
|
repeat, and permanently when it keeps at it.
|
||||||
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
|
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
|
||||||
default rule file bans such a client at its first probe, and its next request
|
default rule file bans such a client at its first probe, and its next request
|
||||||
during the ban makes the ban permanent. Your own rules, or a list of trap
|
during the ban makes the ban permanent. Your own rules, or a list of trap
|
||||||
@@ -192,9 +192,9 @@ account, a key, a token or an alert destination stays off until you give one.
|
|||||||
- Any `SWWAF_` setting can be given as a file instead, for secrets and long
|
- Any `SWWAF_` setting can be given as a file instead, for secrets and long
|
||||||
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
|
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
|
||||||
The file's contents are the value, less one newline at the end. Setting both
|
The file's contents are the value, less one newline at the end. Setting both
|
||||||
forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` and
|
forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` already names a file, so
|
||||||
`SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` already name a file, so they have no
|
it has no `_FILE` form, and no `SWRP_` setting has one, since `rproxy` never
|
||||||
`_FILE` form. No `SWRP_` setting has one, since `rproxy` never touches disk.
|
touches disk.
|
||||||
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
|
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
|
||||||
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
|
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
|
||||||
1024: `512K`, `100M`, `5G`.
|
1024: `512K`, `100M`, `5G`.
|
||||||
@@ -236,13 +236,13 @@ Every setting the code reads, with its default:
|
|||||||
| `SWWAF_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
|
| `SWWAF_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
|
||||||
| `SWWAF_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients the request and byte limits neither count nor refuse. |
|
| `SWWAF_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients the request and byte limits neither count nor refuse. |
|
||||||
| `SWWAF_DENY_NETS` | empty | Netblocks whose clients are refused. |
|
| `SWWAF_DENY_NETS` | empty | Netblocks whose clients are refused. |
|
||||||
| `SWWAF_RATE_LIMIT_PER_MINUTE` | `120` | The most requests a client may make in a minute, a count or `off`; one more bans it. |
|
| `SWWAF_RATE_LIMIT_PER_MINUTE` | `500` | The most requests a client may make in a minute, a count or `off`; one more bans it. |
|
||||||
| `SWWAF_RATE_LIMIT_PER_HOUR` | `1500` | The most requests a client may make in an hour, a count or `off`; one more bans it. |
|
| `SWWAF_RATE_LIMIT_PER_HOUR` | `20000` | The most requests a client may make in an hour, a count or `off`; one more bans it. |
|
||||||
| `SWWAF_RATE_LIMIT_PER_DAY` | `6000` | The most requests a client may make in a day, a count or `off`; one more bans it. |
|
| `SWWAF_RATE_LIMIT_PER_DAY` | `300000` | The most requests a client may make in a day, a count or `off`; one more bans it. |
|
||||||
| `SWWAF_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes whose requests the request and byte limits neither count nor refuse, such as `/assets/`. |
|
| `SWWAF_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes whose requests the request and byte limits neither count nor refuse, such as `/assets/`. |
|
||||||
| `SWWAF_BYTES_LIMIT_PER_MINUTE` | `off` | The most bytes a client's requests may carry in a minute, a size or `off`; the request that takes a client past any byte limit bans it. |
|
| `SWWAF_BYTES_LIMIT_PER_MINUTE` | `4G` | The most bytes a client's requests may carry in a minute, a size or `off`; the request that takes a client past any byte limit bans it. |
|
||||||
| `SWWAF_BYTES_LIMIT_PER_HOUR` | `50G` | The most bytes a client's requests may carry in an hour, a size or `off`. |
|
| `SWWAF_BYTES_LIMIT_PER_HOUR` | `20G` | The most bytes a client's requests may carry in an hour, a size or `off`. |
|
||||||
| `SWWAF_BYTES_LIMIT_PER_DAY` | `100G` | The most bytes a client's requests may carry in a day, a size or `off`. |
|
| `SWWAF_BYTES_LIMIT_PER_DAY` | `200G` | The most bytes a client's requests may carry in a day, a size or `off`. |
|
||||||
| `SWWAF_BYTES_COUNT` | `both` | Which body bytes the byte limits count: `response`, `request` or `both`. |
|
| `SWWAF_BYTES_COUNT` | `both` | Which body bytes the byte limits count: `response`, `request` or `both`. |
|
||||||
| `SWWAF_ERROR_BURST_THRESHOLD` | `30` | The most refusals by a rule file, a trap path or the Core Rule Set a client may get in a minute, a count or `off`; one more bans it. |
|
| `SWWAF_ERROR_BURST_THRESHOLD` | `30` | The most refusals by a rule file, a trap path or the Core Rule Set a client may get in a minute, a count or `off`; one more bans it. |
|
||||||
| `SWWAF_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. |
|
| `SWWAF_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. |
|
||||||
@@ -272,7 +272,6 @@ Every setting the code reads, with its default:
|
|||||||
| `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
|
| `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
|
||||||
| `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
| `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
||||||
| `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
| `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
||||||
| `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only; `deploy/gitea/crs-exclusions.conf` is gitea's. |
|
|
||||||
| `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
| `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
||||||
| `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
| `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
||||||
| `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
| `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
||||||
@@ -408,10 +407,10 @@ services:
|
|||||||
SWWAF_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7
|
SWWAF_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7
|
||||||
# A netblock that sends nothing wanted.
|
# A netblock that sends nothing wanted.
|
||||||
SWWAF_DENY_NETS: 198.51.100.128/25
|
SWWAF_DENY_NETS: 198.51.100.128/25
|
||||||
# Tighter limits for everyone else: a small static site.
|
# Tighter limits for everyone else: a site with no API.
|
||||||
SWWAF_RATE_LIMIT_PER_MINUTE: "60"
|
SWWAF_RATE_LIMIT_PER_MINUTE: "120"
|
||||||
SWWAF_RATE_LIMIT_PER_HOUR: "600"
|
SWWAF_RATE_LIMIT_PER_HOUR: "3000"
|
||||||
SWWAF_RATE_LIMIT_PER_DAY: "3000"
|
SWWAF_RATE_LIMIT_PER_DAY: "30000"
|
||||||
# Hosting networks and one country get a share of those limits.
|
# Hosting networks and one country get a share of those limits.
|
||||||
SWWAF_ASN_LIMIT_PERCENT: AS14061:50,AS16276:25
|
SWWAF_ASN_LIMIT_PERCENT: AS14061:50,AS16276:25
|
||||||
SWWAF_COUNTRY_LIMIT_PERCENT: cn:25
|
SWWAF_COUNTRY_LIMIT_PERCENT: cn:25
|
||||||
@@ -432,9 +431,10 @@ volumes:
|
|||||||
gitea behind traefik, set up for a code forge:
|
gitea behind traefik, set up for a code forge:
|
||||||
|
|
||||||
- the upload limits raised, in size and in time;
|
- the upload limits raised, in size and in time;
|
||||||
- the response size limit raised, for clones over 1 GiB;
|
- the response size limit and the byte limits raised, for clones over 1 GiB;
|
||||||
- the Actions runners exempt from counting;
|
- the Actions runners exempt from counting;
|
||||||
- the default request and byte limits kept.
|
- the default request limits kept, since they already leave room for API
|
||||||
|
clients.
|
||||||
|
|
||||||
`traefik` is the network traefik is on, made outside this file.
|
`traefik` is the network traefik is on, made outside this file.
|
||||||
|
|
||||||
@@ -488,6 +488,11 @@ services:
|
|||||||
SWWAF_INSTANCE_NAME: fsn1/servewaf
|
SWWAF_INSTANCE_NAME: fsn1/servewaf
|
||||||
# The Actions runners ask for work all day; never count them.
|
# The Actions runners ask for work all day; never count them.
|
||||||
SWWAF_RATE_LIMIT_EXEMPT_NETS: 10.0.4.0/24
|
SWWAF_RATE_LIMIT_EXEMPT_NETS: 10.0.4.0/24
|
||||||
|
# Raised with rproxy's response limit, to four times the default
|
||||||
|
# each, so that one clone of that size breaks no byte limit.
|
||||||
|
SWWAF_BYTES_LIMIT_PER_MINUTE: 16G
|
||||||
|
SWWAF_BYTES_LIMIT_PER_HOUR: 80G
|
||||||
|
SWWAF_BYTES_LIMIT_PER_DAY: 800G
|
||||||
# gitea never serves these; a request for one bans the client.
|
# gitea never serves these; a request for one bans the client.
|
||||||
SWWAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php,/wp-admin/
|
SWWAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php,/wp-admin/
|
||||||
volumes:
|
volumes:
|
||||||
@@ -505,9 +510,8 @@ volumes:
|
|||||||
servewaf-state:
|
servewaf-state:
|
||||||
```
|
```
|
||||||
|
|
||||||
`deploy/gitea` holds a complete setup of gitea, traefik included, as a Docker
|
[`docs/gitea.md`](docs/gitea.md) records what the defaults do in front of gitea,
|
||||||
Compose file. [`docs/gitea.md`](docs/gitea.md) explains it, and records what the
|
request by request.
|
||||||
defaults do in front of gitea, request by request.
|
|
||||||
|
|
||||||
### `rproxy` beside the app
|
### `rproxy` beside the app
|
||||||
|
|
||||||
@@ -637,14 +641,10 @@ and the status for a size or time limit, such as `413` for a body too large.
|
|||||||
headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In
|
headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In
|
||||||
front of a code forge, bodies carry what people write and publish, which the
|
front of a code forge, bodies carry what people write and publish, which the
|
||||||
rules take for attacks.
|
rules take for attacks.
|
||||||
- `smallwebwaf` changes the Core Rule Set in seven ways so that it lets ordinary
|
- `smallwebwaf` changes the Core Rule Set in six ways so that it lets ordinary
|
||||||
requests to a code forge through, such as git over HTTP and sign-in through
|
requests to a code forge through, such as git over HTTP and sign-in through
|
||||||
Git Credential Manager. No setting undoes them; "Attack detection" in
|
Git Credential Manager. No setting undoes them; "Attack detection" in
|
||||||
[`SPEC.md`](SPEC.md) lists them.
|
[`SPEC.md`](SPEC.md) lists them.
|
||||||
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's
|
|
||||||
own fields or paths. gitea's exclusions, in
|
|
||||||
`deploy/gitea/crs-exclusions.conf`, let its searches, file names and branch
|
|
||||||
names through.
|
|
||||||
- Rule files are plain text, one rule to a line, edited by hand and read again
|
- Rule files are plain text, one rule to a line, edited by hand and read again
|
||||||
while `servewaf` runs. The image ships one file of probes no real visitor
|
while `servewaf` runs. The image ships one file of probes no real visitor
|
||||||
sends (see [`docs/rules.md`](docs/rules.md)).
|
sends (see [`docs/rules.md`](docs/rules.md)).
|
||||||
@@ -687,8 +687,8 @@ aside as `<name>.bad`, and never stops the running process (see
|
|||||||
health check.
|
health check.
|
||||||
- `internal/config`: reads the settings, the one place they are read.
|
- `internal/config`: reads the settings, the one place they are read.
|
||||||
- `internal/rproxy`: the reverse proxy in front of one app.
|
- `internal/rproxy`: the reverse proxy in front of one app.
|
||||||
- `internal/servewaf`: `servewaf`'s server, which runs the checks and answers
|
- `internal/proxy`: `servewaf`'s server, which runs the checks and answers its
|
||||||
its own endpoints.
|
own endpoints.
|
||||||
- `internal/ratelimit`, `internal/bans`, `internal/rules`, `internal/waf`,
|
- `internal/ratelimit`, `internal/bans`, `internal/rules`, `internal/waf`,
|
||||||
`internal/lookup`, `internal/reputation`, `internal/anomaly` and
|
`internal/lookup`, `internal/reputation`, `internal/anomaly` and
|
||||||
`internal/alerts`: one check or source each.
|
`internal/alerts`: one check or source each.
|
||||||
@@ -784,10 +784,6 @@ on the host.
|
|||||||
Core Rule Set is known to refuse, and prints what `servewaf` did with each.
|
Core Rule Set is known to refuse, and prints what `servewaf` did with each.
|
||||||
Needs network access, and a docker daemon that pushes to a registry on
|
Needs network access, and a docker daemon that pushes to a registry on
|
||||||
`localhost` over plain HTTP. `make gitea-requests`.
|
`localhost` over plain HTTP. `make gitea-requests`.
|
||||||
- `script/gitea-setup`: builds the image, brings up the setup in `deploy/gitea`,
|
|
||||||
and checks it as [`docs/gitea.md`](docs/gitea.md) describes. Then it removes
|
|
||||||
the containers, the network, the volumes and the image it built. Needs network
|
|
||||||
access. `make gitea-setup`.
|
|
||||||
|
|
||||||
## TODO
|
## TODO
|
||||||
|
|
||||||
@@ -811,8 +807,7 @@ on the host.
|
|||||||
- [`docs/state-files.md`](docs/state-files.md): the state files and what each
|
- [`docs/state-files.md`](docs/state-files.md): the state files and what each
|
||||||
holds.
|
holds.
|
||||||
- [`docs/image.md`](docs/image.md): what the image holds and how it is pinned.
|
- [`docs/image.md`](docs/image.md): what the image holds and how it is pinned.
|
||||||
- [`docs/gitea.md`](docs/gitea.md): `smallwebwaf` in front of gitea: the setup
|
- [`docs/gitea.md`](docs/gitea.md): `smallwebwaf` in front of gitea.
|
||||||
in `deploy/gitea`, and what the defaults do there.
|
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
@@ -36,8 +36,7 @@ to them.
|
|||||||
- the state files;
|
- the state files;
|
||||||
- a lookup database;
|
- a lookup database;
|
||||||
- a `SWWAF_` setting given as a file;
|
- a `SWWAF_` setting given as a file;
|
||||||
- the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names;
|
- the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names.
|
||||||
- the file `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names.
|
|
||||||
|
|
||||||
## Non-goals
|
## Non-goals
|
||||||
|
|
||||||
@@ -67,10 +66,10 @@ to them.
|
|||||||
one it does not know, the binary prints the commands and exits with
|
one it does not know, the binary prints the commands and exits with
|
||||||
status 2.
|
status 2.
|
||||||
- `internal/config`: reads the settings, the one place they are read.
|
- `internal/config`: reads the settings, the one place they are read.
|
||||||
- `internal/servewaf`: `servewaf`'s server. It runs the checks in `check`,
|
- `internal/proxy`: `servewaf`'s server. It runs the checks in `check`, and
|
||||||
and holds `servewaf`'s question, answer and report. It answers
|
holds `servewaf`'s question, answer and report. It answers `servewaf`'s
|
||||||
`servewaf`'s own endpoints in `answerOwnEndpoint`. It counts bytes,
|
own endpoints in `answerOwnEndpoint`. It counts bytes, refusals and
|
||||||
refusals and anomalies, and writes the log lines.
|
anomalies, and writes the log lines.
|
||||||
- `internal/rproxy`: the reverse proxy in front of one app that asks
|
- `internal/rproxy`: the reverse proxy in front of one app that asks
|
||||||
`servewaf` about each request. It passes a request to the app with the
|
`servewaf` about each request. It passes a request to the app with the
|
||||||
standard library's `httputil.ReverseProxy`, within the time and size
|
standard library's `httputil.ReverseProxy`, within the time and size
|
||||||
@@ -82,7 +81,7 @@ to them.
|
|||||||
of bans is full.
|
of bans is full.
|
||||||
- `internal/rules`: reads the rule files at start and again when they
|
- `internal/rules`: reads the rule files at start and again when they
|
||||||
change, and tells which rules a request matches.
|
change, and tells which rules a request matches.
|
||||||
- `internal/waf`: the Core Rule Set, with the seven changes listed under
|
- `internal/waf`: the Core Rule Set, with the six changes listed under
|
||||||
"Attack detection", as Coraza directives.
|
"Attack detection", as Coraza directives.
|
||||||
- `internal/lookup`: looks clients up through GeoJS or in the lookup
|
- `internal/lookup`: looks clients up through GeoJS or in the lookup
|
||||||
database.
|
database.
|
||||||
@@ -326,10 +325,9 @@ Conventions:
|
|||||||
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
|
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
|
||||||
The file's contents are the value, less one newline at the end, and are
|
The file's contents are the value, less one newline at the end, and are
|
||||||
checked as the value itself would be. Setting both forms stops the start, and
|
checked as the value itself would be. Setting both forms stops the start, and
|
||||||
so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` and
|
so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE`
|
||||||
`SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` already hold a file name, so they have
|
already holds a file name, so it has no `_FILE` form. No `SWRP_` setting has
|
||||||
no `_FILE` form. No `SWRP_` setting has one, since `rproxy` never touches
|
one, since `rproxy` never touches disk.
|
||||||
disk.
|
|
||||||
- Settings, files included, are read once at start. Changing one means
|
- Settings, files included, are read once at start. Changing one means
|
||||||
restarting the container. The files `servewaf` watches while it runs are its
|
restarting the container. The files `servewaf` watches while it runs are its
|
||||||
state files, its rule files and the lookup database.
|
state files, its rule files and the lookup database.
|
||||||
@@ -399,31 +397,32 @@ Conventions:
|
|||||||
|
|
||||||
### Request rate limits, per client
|
### Request rate limits, per client
|
||||||
|
|
||||||
Breaking one bans the client. The defaults let people read a site and stop
|
Breaking one bans the client. The defaults suit programmatic use of an API, such
|
||||||
scrapers that send many requests slowly over hours or days. The hour limit is
|
as an automation fleet on another host, and still stop abusive bots. A fleet of
|
||||||
the main one.
|
20 agents, each making a request every 10 seconds at its busiest, makes 120
|
||||||
|
requests a minute and 7,200 an hour. Working around the clock, it makes 172,800
|
||||||
|
a day. It stays under all three limits.
|
||||||
|
|
||||||
- Per minute: 120, 2 requests a second. It leaves room for a person opening two
|
- Per minute: 500, about 8 requests a second. It leaves room for the fleet's 120
|
||||||
pages at once with an empty cache. A gitea page opened with an empty cache
|
a minute and a few quick first visits with an empty cache. A gitea page opened
|
||||||
loads 20 to 60 files, and a handful once cached.
|
with an empty cache loads about 40 files. A client firing requests as fast as
|
||||||
- Per hour: 1,500. A person reading code in gitea for a whole hour, a page every
|
it can passes the limit within seconds.
|
||||||
20 seconds, makes about 900 requests. A scraper making one request every 2
|
- Per hour: 20,000, about 5.5 requests a second for a whole hour. A client that
|
||||||
seconds makes 1,800 an hour, more than the limit.
|
keeps up 6 a second makes 21,600 in an hour, and passes it, at 360 a minute,
|
||||||
- Per day: 6,000, four hours at the hour limit. A scraper making one request
|
under the minute limit. The fleet's 7,200 an hour is about a third of it.
|
||||||
every 10 seconds makes 8,640 a day, more than the limit.
|
- Per day: 300,000, about 3.5 requests a second around the clock. A scraper that
|
||||||
|
keeps up 4 a second makes 345,600 a day, and passes it. At 240 a minute and
|
||||||
|
14,400 an hour, it stays under the other two limits.
|
||||||
|
|
||||||
Counting cannot catch a client that stays under all three limits, about 4
|
Counting cannot catch a client that stays under all three limits, such as one
|
||||||
requests a minute around the clock. Only the other checks can catch an abuser
|
that keeps up 3 requests a second. Only the other checks can catch an abuser
|
||||||
that slow. They include the trap paths, the rule files, the Core Rule Set and
|
that slow. They include the trap paths, the rule files, the Core Rule Set and
|
||||||
the blocklists. An abuser that trips none of them is not caught.
|
the blocklists. An abuser that trips none of them is not caught. A machine that
|
||||||
|
keeps up more than 500 requests a minute passes the minute limit. One that keeps
|
||||||
Some legitimate clients pass the limits:
|
up more than about 5.5 a second for an hour passes the hour limit. One that
|
||||||
|
keeps up more than about 3.5 a second all day passes the day limit, as two busy
|
||||||
- An Actions runner asks gitea for work all day and can pass the limits.
|
Actions runners on one address do. Such a machine's address belongs in
|
||||||
- Everyone behind one shared address, such as an office, shares one client's
|
`SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||||
limits.
|
|
||||||
|
|
||||||
Their addresses belong in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
|
||||||
|
|
||||||
`SWWAF_RATE_LIMIT_EXEMPT_PATHS` takes path prefixes out of the count, such as
|
`SWWAF_RATE_LIMIT_EXEMPT_PATHS` takes path prefixes out of the count, such as
|
||||||
static assets or health checks; their bytes are not counted either. A prefix is
|
static assets or health checks; their bytes are not counted either. A prefix is
|
||||||
@@ -436,10 +435,12 @@ matched the same way.
|
|||||||
|
|
||||||
### Byte limits, per client
|
### Byte limits, per client
|
||||||
|
|
||||||
By default there is no minute limit, so a client can download in a burst at full
|
The hour and day limits sit about twice above what 40 people busy with heavy web
|
||||||
speed. The hour limit defaults to 50 GiB and the day limit to 100 GiB. They
|
apps move together on one address. Each moves about 256 MiB an hour, for ten
|
||||||
allow several downloads of a few GiB each. A response's bytes are counted when
|
hours a day. A response's bytes are counted when it ends (see step 15). So the
|
||||||
it ends (see step 15).
|
minute limit holds three of the largest responses, with their requests, ending
|
||||||
|
in the same minute. No single download then breaks a limit. An operator who
|
||||||
|
raises `SWRP_MAX_RESPONSE_SIZE` must raise the byte limits too.
|
||||||
|
|
||||||
### Lookup of AS number and country
|
### Lookup of AS number and country
|
||||||
|
|
||||||
@@ -585,24 +586,6 @@ sends it.
|
|||||||
probes these rules caught (see "Rule files"). A list you give replaces the
|
probes these rules caught (see "Rule files"). A list you give replaces the
|
||||||
default, so name these six ids in it to keep them off.
|
default, so name these six ids in it to keep them off.
|
||||||
|
|
||||||
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names a file of exclusions, written as
|
|
||||||
Coraza directives. An exclusion switches one rule off for some fields or paths
|
|
||||||
only, where `SWWAF_WAF_DISABLED_RULES` switches it off everywhere.
|
|
||||||
`deploy/gitea/crs-exclusions.conf` holds gitea's.
|
|
||||||
- `servewaf` loads the file after `smallwebwaf`'s own setup rules and before
|
|
||||||
the Core Rule Set's rules. The Core Rule Set puts its own file of
|
|
||||||
exclusions that run before the rules in the same place.
|
|
||||||
- So the file holds only `SecAction` and `SecRule` lines that exclude with
|
|
||||||
`ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. These run with each
|
|
||||||
request, before the rules they exclude, those of phase 1 included.
|
|
||||||
- A rule that `SWWAF_WAF_DISABLED_RULES` switches off and the file names is
|
|
||||||
simply off.
|
|
||||||
- The start stops on a file that cannot be read or does not parse, whatever
|
|
||||||
`SWWAF_WAF_MODE` is.
|
|
||||||
- It stops too on a file that names a number from 900000 to 900999, the ids
|
|
||||||
of `smallwebwaf`'s own rules, alone or in a range. Comment lines are not
|
|
||||||
checked.
|
|
||||||
|
|
||||||
- `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. It takes
|
- `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. It takes
|
||||||
at most `64K`, since `servewaf` gets no more of a body than that (see "The
|
at most `64K`, since `servewaf` gets no more of a body than that (see "The
|
||||||
answer"). Form data and multipart bodies are read up to the size. JSON and XML
|
answer"). Form data and multipart bodies are read up to the size. JSON and XML
|
||||||
@@ -632,7 +615,7 @@ sends it.
|
|||||||
scanner from an ordinary client. In front of gitea, registry and git clients
|
scanner from an ordinary client. In front of gitea, registry and git clients
|
||||||
get `404` and `401` by design, hundreds of times a minute.
|
get `404` and `401` by design, hundreds of times a minute.
|
||||||
|
|
||||||
`smallwebwaf` changes the Core Rule Set 4.25.0 in seven ways, since in front of
|
`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of
|
||||||
gitea it would otherwise refuse ordinary requests. The changes hold in front of
|
gitea it would otherwise refuse ordinary requests. The changes hold in front of
|
||||||
every app, and no setting undoes them:
|
every app, and no setting undoes them:
|
||||||
|
|
||||||
@@ -689,10 +672,6 @@ every app, and no setting undoes them:
|
|||||||
must check it itself.
|
must check it itself.
|
||||||
6. Responses are not inspected. A raw file from a repository, such as a shell
|
6. Responses are not inspected. A raw file from a repository, such as a shell
|
||||||
script, looks to the response rules like source code leaking from the server.
|
script, looks to the response rules like source code leaking from the server.
|
||||||
7. The `from` parameter of a container registry's upload path,
|
|
||||||
`/v2/<image>/blobs/uploads/`, is not checked for command names (932260). When
|
|
||||||
a push reuses a layer, `from` names the image the layer comes from. An image
|
|
||||||
such as `owner/python3-app` would have that push refused.
|
|
||||||
|
|
||||||
The parameter and cookie names in these changes are matched without regard to
|
The parameter and cookie names in these changes are matched without regard to
|
||||||
case, as Coraza matches them.
|
case, as Coraza matches them.
|
||||||
|
|||||||
@@ -1,136 +0,0 @@
|
|||||||
# Gitea behind traefik, with rproxy and servewaf in front of it.
|
|
||||||
# docs/gitea.md says what each setting here does and why. Before starting it:
|
|
||||||
# - set SMALLWEBWAF_IMAGE to the smallwebwaf image, named by digest;
|
|
||||||
# - replace git.example.org with the site's host name;
|
|
||||||
# - replace each 10.99.0 with the first three numbers of a /24 subnet that
|
|
||||||
# no other network on the host uses. The /24 must lie inside 10.0.0.0/8,
|
|
||||||
# 172.16.0.0/12 or 192.168.0.0/16, because rproxy believes
|
|
||||||
# X-Forwarded-For only from those by default. Replace each fd10:99 with
|
|
||||||
# the first two hextets of an unused /64 subnet whose third and fourth
|
|
||||||
# hextets are 0, as in fd10:99::/64;
|
|
||||||
# - set GITEA_SECRET_KEY to the output of
|
|
||||||
# `docker run --rm <gitea's image below> gitea generate secret SECRET_KEY`,
|
|
||||||
# once per install, and keep it.
|
|
||||||
# Then run `docker compose up --detach` in this directory.
|
|
||||||
# script/gitea-setup checks this setup.
|
|
||||||
|
|
||||||
name: gitea
|
|
||||||
|
|
||||||
x-logging: &logging
|
|
||||||
driver: json-file
|
|
||||||
options:
|
|
||||||
max-size: 10m
|
|
||||||
max-file: "5"
|
|
||||||
|
|
||||||
services:
|
|
||||||
traefik:
|
|
||||||
# traefik v3.7.17, 2026-11-03
|
|
||||||
image: traefik@sha256:68ae5fe5b64ca166704b746a9e8afcd385e36b3f569a5691401bf652d57b3748
|
|
||||||
# traefik replaces the X-Forwarded-For header a visitor sends. Keep it
|
|
||||||
# so: otherwise a visitor could name an address on the shared network,
|
|
||||||
# and skip servewaf's checks.
|
|
||||||
command:
|
|
||||||
- --providers.docker.exposedbydefault=false
|
|
||||||
- --entrypoints.web.address=:80
|
|
||||||
- --entrypoints.web.http.redirections.entrypoint.to=websecure
|
|
||||||
- --entrypoints.websecure.address=:443
|
|
||||||
# Until a certificate resolver is added here, as for your other
|
|
||||||
# sites, traefik answers with a certificate it made itself.
|
|
||||||
- --entrypoints.websecure.http.tls=true
|
|
||||||
# traefik's own limit of 60 seconds for a whole request would cut
|
|
||||||
# off an upload long before rproxy's limit of 30 minutes.
|
|
||||||
- --entrypoints.websecure.transport.respondingTimeouts.readTimeout=30m
|
|
||||||
ports:
|
|
||||||
- 80:80
|
|
||||||
- 443:443
|
|
||||||
volumes:
|
|
||||||
- /var/run/docker.sock:/var/run/docker.sock:ro
|
|
||||||
networks:
|
|
||||||
shared:
|
|
||||||
# Containers on the shared network find traefik under the
|
|
||||||
# site's name, and so reach it from their own address.
|
|
||||||
aliases:
|
|
||||||
- git.example.org
|
|
||||||
restart: unless-stopped
|
|
||||||
|
|
||||||
gitea:
|
|
||||||
# gitea 1.27.3, 2026-08-29
|
|
||||||
image: gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef
|
|
||||||
environment:
|
|
||||||
GITEA__server__ROOT_URL: https://git.example.org/
|
|
||||||
# Gitea starts at its defaults, with SQLite, instead of showing its
|
|
||||||
# install page to whoever opens it first.
|
|
||||||
GITEA__security__INSTALL_LOCK: "true"
|
|
||||||
GITEA__security__SECRET_KEY: ${GITEA_SECRET_KEY:?set GITEA_SECRET_KEY, see docs/gitea.md}
|
|
||||||
# Accounts are made with `gitea admin user create`.
|
|
||||||
GITEA__service__DISABLE_REGISTRATION: "true"
|
|
||||||
# rproxy's addresses. Gitea then takes each visitor's address from
|
|
||||||
# the X-Real-Ip header traefik sets and rproxy passes on.
|
|
||||||
GITEA__security__REVERSE_PROXY_TRUSTED_PROXIES: 10.99.0.2,fd10:99::2
|
|
||||||
volumes:
|
|
||||||
- gitea-data:/data
|
|
||||||
networks:
|
|
||||||
- shared
|
|
||||||
restart: unless-stopped
|
|
||||||
|
|
||||||
rproxy:
|
|
||||||
image: ${SMALLWEBWAF_IMAGE:?set SMALLWEBWAF_IMAGE to the smallwebwaf image}
|
|
||||||
environment:
|
|
||||||
SWRP_WAF_BACKEND: http://servewaf:8080
|
|
||||||
SWRP_UPSTREAM_URL: http://gitea:3000
|
|
||||||
# Pushes, LFS objects, packages and release attachments of up to
|
|
||||||
# 2 GiB, sent in up to 30 minutes.
|
|
||||||
SWRP_MAX_REQUEST_SIZE: 2G
|
|
||||||
SWRP_MAX_REQUEST_TIME_MS: "1800000"
|
|
||||||
# 15 minutes for a clone or a download, half the default.
|
|
||||||
SWRP_MAX_RESPONSE_TIME_MS: "900000"
|
|
||||||
labels:
|
|
||||||
traefik.enable: "true"
|
|
||||||
traefik.http.routers.gitea.rule: Host(`git.example.org`)
|
|
||||||
traefik.http.routers.gitea.entrypoints: websecure
|
|
||||||
traefik.http.services.gitea.loadbalancer.server.port: "8080"
|
|
||||||
networks:
|
|
||||||
shared:
|
|
||||||
ipv4_address: 10.99.0.2
|
|
||||||
ipv6_address: fd10:99::2
|
|
||||||
logging: *logging
|
|
||||||
restart: unless-stopped
|
|
||||||
|
|
||||||
servewaf:
|
|
||||||
image: ${SMALLWEBWAF_IMAGE:?set SMALLWEBWAF_IMAGE to the smallwebwaf image}
|
|
||||||
environment:
|
|
||||||
SMALLWEBWAF_COMMAND: servewaf
|
|
||||||
# The containers on the shared network skip the checks. Add the
|
|
||||||
# address of an automation fleet on another host here too; without
|
|
||||||
# that, it is counted like any client.
|
|
||||||
SWWAF_ALLOW_NETS: 10.99.0.128/25,fd10:99:0:0:8000::/65
|
|
||||||
# The Core Rule Set's exclusions for Gitea's searches, file names
|
|
||||||
# and branch names.
|
|
||||||
SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: /etc/smallwebwaf/crs-exclusions.conf
|
|
||||||
volumes:
|
|
||||||
- servewaf-state:/var/lib/smallwebwaf
|
|
||||||
- ./crs-exclusions.conf:/etc/smallwebwaf/crs-exclusions.conf:ro
|
|
||||||
networks:
|
|
||||||
- shared
|
|
||||||
logging: *logging
|
|
||||||
restart: unless-stopped
|
|
||||||
|
|
||||||
networks:
|
|
||||||
# The network traefik and Gitea share. Its containers get addresses in the
|
|
||||||
# upper half of each subnet, which servewaf lets skip the checks. Each
|
|
||||||
# subnet's gateway, the host's own address on it, stays out of that half:
|
|
||||||
# requests docker passes on from a published port can come from it.
|
|
||||||
shared:
|
|
||||||
enable_ipv6: true
|
|
||||||
ipam:
|
|
||||||
config:
|
|
||||||
- subnet: 10.99.0.0/24
|
|
||||||
ip_range: 10.99.0.128/25
|
|
||||||
gateway: 10.99.0.1
|
|
||||||
- subnet: fd10:99::/64
|
|
||||||
ip_range: fd10:99:0:0:8000::/65
|
|
||||||
gateway: fd10:99::1
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
gitea-data:
|
|
||||||
servewaf-state:
|
|
||||||
@@ -1,150 +0,0 @@
|
|||||||
# The Core Rule Set's exclusions for Gitea, which servewaf loads from
|
|
||||||
# SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE. servewaf loads this file before the Core
|
|
||||||
# Rule Set's rules, so it holds only exclusions made while a request is checked:
|
|
||||||
# SecAction and SecRule lines with ctl:ruleRemoveTargetById or
|
|
||||||
# ctl:ruleRemoveById. Each switches one rule off for the Gitea fields or paths
|
|
||||||
# where it refused ordinary Gitea requests. A SecAction applies on every path, a
|
|
||||||
# SecRule on the paths it matches. Gitea sends these in the query string:
|
|
||||||
# - q: the text of every search;
|
|
||||||
# - file, path, template, body and redirect_to: a file name, alone or in
|
|
||||||
# JSON or a link;
|
|
||||||
# - name, tag, rule_name and refSubUrl: a branch or tag name.
|
|
||||||
# docs/gitea.md says how to add an exclusion.
|
|
||||||
|
|
||||||
# Searches.
|
|
||||||
# 930110: a search for ../
|
|
||||||
SecAction "id:10001,phase:1,pass,nolog,ctl:ruleRemoveTargetById=930110;ARGS:q"
|
|
||||||
# 931100: a search for http://127.0.0.1:8080
|
|
||||||
SecAction "id:10002,phase:1,pass,nolog,ctl:ruleRemoveTargetById=931100;ARGS:q"
|
|
||||||
# 932140: a search for if (a == b) {
|
|
||||||
SecAction "id:10003,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932140;ARGS:q"
|
|
||||||
# 932235: a search for `make check`
|
|
||||||
SecAction "id:10004,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932235;ARGS:q"
|
|
||||||
# 932250: a search for ls -la
|
|
||||||
SecAction "id:10005,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932250;ARGS:q"
|
|
||||||
# 932380: a search for `whoami`
|
|
||||||
SecAction "id:10006,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932380;ARGS:q"
|
|
||||||
# 933100: a search for <?php echo
|
|
||||||
SecAction "id:10007,phase:1,pass,nolog,ctl:ruleRemoveTargetById=933100;ARGS:q"
|
|
||||||
# 933130: a search for $_GET[
|
|
||||||
SecAction "id:10008,phase:1,pass,nolog,ctl:ruleRemoveTargetById=933130;ARGS:q"
|
|
||||||
# 933135: a search for ${VAR}
|
|
||||||
SecAction "id:10009,phase:1,pass,nolog,ctl:ruleRemoveTargetById=933135;ARGS:q"
|
|
||||||
# 934110: a search for http://127.0.0.1:8080
|
|
||||||
SecAction "id:10010,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934110;ARGS:q"
|
|
||||||
# 941100: a search for <script>
|
|
||||||
SecAction "id:10011,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941100;ARGS:q"
|
|
||||||
# 941110: a search for <script>, or a page opened from its results
|
|
||||||
SecAction "id:10012,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=941110;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=941110;REQUEST_HEADERS:Referer"
|
|
||||||
# 941120: a page opened from the results of a search for onClick={
|
|
||||||
SecAction "id:10013,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=941120;REQUEST_HEADERS:Referer"
|
|
||||||
# 941160: a search for <img src=, or a page opened from its results
|
|
||||||
SecAction "id:10014,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=941160;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=941160;REQUEST_HEADERS:Referer"
|
|
||||||
# 941390: a search for fetch(
|
|
||||||
SecAction "id:10015,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941390;ARGS:q"
|
|
||||||
# 942100: a search for SELECT * FROM users WHERE, or a page opened from the
|
|
||||||
# results of a search for LIKE '%foo%'
|
|
||||||
SecAction "id:10016,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=942100;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=942100;REQUEST_HEADERS:Referer"
|
|
||||||
# 942190: a search for union select
|
|
||||||
SecAction "id:10017,phase:1,pass,nolog,ctl:ruleRemoveTargetById=942190;ARGS:q"
|
|
||||||
# 942230: a search for if (a == b) {
|
|
||||||
SecAction "id:10018,phase:1,pass,nolog,ctl:ruleRemoveTargetById=942230;ARGS:q"
|
|
||||||
# 944100: a search for java.lang.Runtime, or a page opened from its results
|
|
||||||
SecAction "id:10019,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=944100;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=944100;REQUEST_HEADERS:Referer"
|
|
||||||
|
|
||||||
# Searches, and file, branch and tag names.
|
|
||||||
# 930120: a search for package.json, or the commit graph of .gitignore
|
|
||||||
SecAction "id:10020,phase:1,pass,nolog,ctl:ruleRemoveTargetById=930120;\
|
|
||||||
ARGS:/^(?:q|file|template|body|refsuburl|name|tag|rule_name)$/"
|
|
||||||
# 932160: a search for /bin/sh, or the API's contents of bin/docker-entrypoint
|
|
||||||
SecAction "id:10021,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932160;\
|
|
||||||
ARGS:/^(?:q|file|template|body|refsuburl|name|tag|rule_name)$/"
|
|
||||||
# 932260: a search for python3, or deleting the branch docker-build
|
|
||||||
SecAction "id:10022,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;\
|
|
||||||
ARGS:/^(?:q|file|template|body|refsuburl|name|tag|rule_name)$/"
|
|
||||||
|
|
||||||
# Searches, and file names in the query string.
|
|
||||||
# 932120: a search for Invoke-WebRequest, or the commits of install-script.sh
|
|
||||||
SecAction "id:10023,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932120;\
|
|
||||||
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
|
|
||||||
# 932130: a search for $(pwd), or an issue opened from the template [draft].md
|
|
||||||
SecAction "id:10024,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932130;\
|
|
||||||
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
|
|
||||||
# 934100: a search for process.env, or the commits of process.env.d.ts
|
|
||||||
SecAction "id:10025,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934100;\
|
|
||||||
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
|
|
||||||
# 934130: a search for __proto__, or the commits of __proto__.json
|
|
||||||
SecAction "id:10026,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934130;\
|
|
||||||
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
|
|
||||||
# 941130: the commits of index.xhtml
|
|
||||||
SecAction "id:10027,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941130;\
|
|
||||||
ARGS:/^(?:file|path|template|body|redirect_to)$/"
|
|
||||||
# 941180: a search for document.cookie, or the commits of document.write.js
|
|
||||||
SecAction "id:10028,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941180;\
|
|
||||||
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
|
|
||||||
# 942140: a search for information_schema, or the commits of
|
|
||||||
# information_schema.sql
|
|
||||||
SecAction "id:10029,phase:1,pass,nolog,ctl:ruleRemoveTargetById=942140;\
|
|
||||||
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
|
|
||||||
# 944110: a search for java.lang.Runtime, or the commits of Runtime.java
|
|
||||||
SecAction "id:10030,phase:1,pass,nolog,ctl:ruleRemoveTargetById=944110;\
|
|
||||||
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
|
|
||||||
# 944130: a search for java.lang.Runtime, the commits of java.lang.String.md,
|
|
||||||
# or any page opened from that file's page, which names it in the Referer
|
|
||||||
SecAction "id:10031,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;\
|
|
||||||
ARGS:/^(?:q|file|path|template|body|redirect_to)$/,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;REQUEST_HEADERS:Referer"
|
|
||||||
|
|
||||||
# File names in the path of a file's page, raw file, blame, history and edit
|
|
||||||
# page, and of the API's calls for its contents.
|
|
||||||
# 920500: the page of notes.txt~
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
|
|
||||||
"id:10032,phase:1,pass,nolog,ctl:ruleRemoveTargetById=920500;REQUEST_FILENAME"
|
|
||||||
# 934100: the page of process.env.d.ts
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
|
|
||||||
"id:10033,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934100;REQUEST_FILENAME"
|
|
||||||
# 941130: the page of docs/a b.xhtml
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
|
|
||||||
"id:10034,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941130;REQUEST_FILENAME"
|
|
||||||
# 944130: the page of java.lang.String.md
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
|
|
||||||
"id:10035,phase:1,pass,nolog,ctl:ruleRemoveTargetById=944130;REQUEST_FILENAME"
|
|
||||||
|
|
||||||
# The file name, in name, of an attachment uploaded through the API to a
|
|
||||||
# release, an issue or a comment.
|
|
||||||
# 932120: the attachment install-script.sh
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
|
|
||||||
"id:10036,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932120;ARGS:name"
|
|
||||||
# 934100: the attachment process.env.d.ts
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
|
|
||||||
"id:10037,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934100;ARGS:name"
|
|
||||||
# 934130: the attachment __proto__.json
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
|
|
||||||
"id:10038,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934130;ARGS:name"
|
|
||||||
# 941130: the attachment index.xhtml
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
|
|
||||||
"id:10039,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941130;ARGS:name"
|
|
||||||
# 941180: the attachment document.write.js
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
|
|
||||||
"id:10040,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941180;ARGS:name"
|
|
||||||
# 942140: the attachment information_schema.sql
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
|
|
||||||
"id:10041,phase:1,pass,nolog,ctl:ruleRemoveTargetById=942140;ARGS:name"
|
|
||||||
# 944130: the attachment java.lang.String.md
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
|
|
||||||
"id:10042,phase:1,pass,nolog,ctl:ruleRemoveTargetById=944130;ARGS:name"
|
|
||||||
|
|
||||||
# The header Content-Range of an artifact upload.
|
|
||||||
# 920450: an artifact upload from actions/upload-artifact@v3
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/api/actions_pipeline/_apis/pipelines/workflows/[0-9]+/artifacts/[^/]+/upload$" \
|
|
||||||
"id:10043,phase:1,pass,nolog,ctl:ruleRemoveTargetById=920450;REQUEST_HEADERS_NAMES:Content-Range"
|
|
||||||
+181
-167
@@ -1,206 +1,220 @@
|
|||||||
# smallwebwaf in front of gitea
|
# smallwebwaf in front of gitea
|
||||||
|
|
||||||
`deploy/gitea/compose.yaml` runs gitea behind traefik, with `rproxy` and
|
These notes record what `smallwebwaf` does in front of gitea at the default
|
||||||
`servewaf` in front of it. This document says what that setup sets and why, and
|
settings. They are observations, not requirements; the requirements are in
|
||||||
what the defaults would do without it. [`SPEC.md`](../SPEC.md) says how
|
[`SPEC.md`](../SPEC.md). `script/gitea-requests` runs gitea behind `rproxy` and
|
||||||
`smallwebwaf` behaves.
|
`servewaf`, and sends through them a sample of gitea's ordinary use and the
|
||||||
|
requests named below. It reports each request the Core Rule Set refuses, so that
|
||||||
|
these notes can be checked again when gitea or the Core Rule Set changes
|
||||||
|
version.
|
||||||
|
|
||||||
## The setup
|
## What the defaults cover
|
||||||
|
|
||||||
Four containers share one Docker network, the shared network:
|
|
||||||
|
|
||||||
- traefik, which routes the site's HTTPS requests to `rproxy`, and redirects
|
|
||||||
plain HTTP to HTTPS;
|
|
||||||
- `rproxy`, in front of gitea;
|
|
||||||
- gitea 1.27.3, as its image comes;
|
|
||||||
- `servewaf`, with a volume for its state files, and with
|
|
||||||
`deploy/gitea/crs-exclusions.conf` mounted read-only.
|
|
||||||
|
|
||||||
To start it:
|
|
||||||
|
|
||||||
1. In `deploy/gitea/compose.yaml`, replace `git.example.org` with the site's
|
|
||||||
host name.
|
|
||||||
2. Replace each `10.99.0` with the first three numbers of a `/24` subnet that no
|
|
||||||
other network on the host uses. The `/24` must lie inside `10.0.0.0/8`,
|
|
||||||
`172.16.0.0/12` or `192.168.0.0/16`, because `rproxy` believes
|
|
||||||
`X-Forwarded-For` only from those by default. Replace each `fd10:99` with the
|
|
||||||
first two hextets of an unused `/64` subnet whose third and fourth hextets
|
|
||||||
are 0, as in `fd10:99::/64`.
|
|
||||||
3. Set `SMALLWEBWAF_IMAGE` to the `smallwebwaf` image, named by digest.
|
|
||||||
4. Set `GITEA_SECRET_KEY` to the output of
|
|
||||||
`docker run --rm <gitea's image> gitea generate secret SECRET_KEY`, with
|
|
||||||
gitea's image as `deploy/gitea/compose.yaml` names it. Do this once per
|
|
||||||
install, and keep the key.
|
|
||||||
5. Run `docker compose up --detach` in `deploy/gitea`, with
|
|
||||||
`crs-exclusions.conf` beside `compose.yaml`.
|
|
||||||
6. Make gitea's first user with `gitea admin user create`, in gitea's container,
|
|
||||||
as the user `git`.
|
|
||||||
|
|
||||||
Sign-up is off: accounts are made with `gitea admin user create`. gitea starts
|
|
||||||
at its defaults, with SQLite. traefik answers with a certificate it made itself
|
|
||||||
until you add a certificate resolver, as for your other sites. SSH to gitea does
|
|
||||||
not pass through `smallwebwaf`; publish gitea's SSH port yourself if you use it.
|
|
||||||
|
|
||||||
`make gitea-setup` checks the setup. It builds the image, brings the setup up
|
|
||||||
under a name of its own, and checks:
|
|
||||||
|
|
||||||
- that a page load, a push and a clone sent through traefik pass through
|
|
||||||
`rproxy`, and that `servewaf` checks them;
|
|
||||||
- that gitea logs the visitor's address, not `rproxy`'s;
|
|
||||||
- that a request from a container on the shared network skips the checks;
|
|
||||||
- that the Core Rule Set refuses none of the ordinary gitea requests in
|
|
||||||
`internal/waf/testdata/gitea-requests.txt`, sent by a signed-in user;
|
|
||||||
- that it refuses none of the API calls with JSON bodies;
|
|
||||||
- that it refuses no fetch;
|
|
||||||
- that it refuses SQL injection, path traversal and script injection in a query
|
|
||||||
string;
|
|
||||||
- that with `servewaf` stopped, gitea still answers through `rproxy`.
|
|
||||||
|
|
||||||
## Uploads and downloads
|
|
||||||
|
|
||||||
An upload is a request:
|
|
||||||
|
|
||||||
|
- SSH access to gitea does not pass through `smallwebwaf` and is not protected
|
||||||
|
by it.
|
||||||
|
- A clone is a response. It fits the defaults of 30 minutes and 1 GiB unless the
|
||||||
|
repository is large or the link slow. A repository whose clone is larger than
|
||||||
|
1 GiB needs `SWRP_MAX_RESPONSE_SIZE` raised, and the byte limits with it
|
||||||
|
("Byte limits, per client" in `SPEC.md`).
|
||||||
|
- A push is a request, and so is every other upload:
|
||||||
- a git push;
|
- a git push;
|
||||||
- an LFS object;
|
- an LFS object;
|
||||||
- a container image layer;
|
- a container image layer;
|
||||||
- a package file;
|
- a package file;
|
||||||
- a release attachment.
|
- a release attachment.
|
||||||
|
|
||||||
The setup lets an upload of up to 2 GiB, sent in up to 30 minutes, through:
|
At the defaults, an upload larger than 100 MiB or taking more than 60
|
||||||
|
seconds is cut off ("`rproxy`'s limits" in `SPEC.md`). A gitea that takes
|
||||||
|
large uploads needs `SWRP_MAX_REQUEST_SIZE` and `SWRP_MAX_REQUEST_TIME_MS`
|
||||||
|
raised to fit.
|
||||||
|
|
||||||
- `SWRP_MAX_REQUEST_SIZE` is `2G`. At the default, an upload over 100 MiB is cut
|
- Archive download and blame or history pages are what scrapers hammer. The
|
||||||
off.
|
request limits do most of the work there.
|
||||||
- `SWRP_MAX_REQUEST_TIME_MS` is `1800000`, 30 minutes. At the default, an upload
|
- An Actions runner that reaches gitea through `smallwebwaf` sends requests all
|
||||||
that takes over 60 seconds is cut off.
|
day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
|
||||||
- traefik's `readTimeout` is 30 minutes. traefik's own default of 60 seconds for
|
43,200 requests a day. While it runs a job, it reports the job's log and state
|
||||||
a whole request would cut off an upload first.
|
every second instead, two requests a second. So one runner busy all day sends
|
||||||
|
120 requests a minute and 172,800 a day, under the request limits. Two such
|
||||||
|
runners on one address, each busy all day, make 345,600 and pass the day limit
|
||||||
|
of 300,000. Three busy at once make 21,600 an hour and pass the hour limit of
|
||||||
|
20,000. Five busy at once make 600 a minute and pass the minute limit of 500.
|
||||||
|
Their address is then banned, their running jobs fail, and each repeat within
|
||||||
|
a day makes the ban longer. Newer runners ask for work less often. Put the
|
||||||
|
runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||||
|
|
||||||
A clone or a download is a response:
|
## What the Core Rule Set lets through
|
||||||
|
|
||||||
- `SWRP_MAX_RESPONSE_TIME_MS` is `900000`, 15 minutes, half the default. A clone
|
At the defaults, the Core Rule Set lets gitea's ordinary use through, apart from
|
||||||
or download that takes longer is cut off.
|
the refusals in the next section:
|
||||||
- `SWRP_MAX_RESPONSE_SIZE` stays at its default, 1 GiB. A repository whose clone
|
|
||||||
is larger needs it raised.
|
|
||||||
|
|
||||||
The byte limits stay at their defaults: no minute limit, 50 GiB an hour and 100
|
- browsing and views of files in a repository, with their history, blame and the
|
||||||
GiB a day.
|
file tree;
|
||||||
|
- diffs, including their hidden lines and large files, and pull request review;
|
||||||
|
- git's clone, fetch and push over HTTP;
|
||||||
|
- signing in, including the return to the page a visitor came from, and sign-in
|
||||||
|
with Git Credential Manager, git-credential-oauth or tea;
|
||||||
|
- the API's calls for a file and its commits;
|
||||||
|
- pushing and pulling container images and packages;
|
||||||
|
- Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`;
|
||||||
|
- posting issues, pull requests, comments, wiki pages and files saved in the web
|
||||||
|
editor, code included, since no body is read;
|
||||||
|
- the page gitea shows after a change, with its message naming what was done.
|
||||||
|
|
||||||
## The checks
|
## What the Core Rule Set still refuses
|
||||||
|
|
||||||
The Core Rule Set runs in `block` mode, the default, with gitea's exclusions
|
Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts
|
||||||
(see "The Core Rule Set's exclusions" below).
|
toward). The request log names the rule in `waf_rule_ids`, which
|
||||||
|
`SWWAF_WAF_DISABLED_RULES` can switch off.
|
||||||
|
|
||||||
Every other check stays at its default. A client may make 120 requests a minute,
|
### A query string that reads as an attack
|
||||||
1,500 an hour and 6,000 a day. Archive downloads and blame and history pages are
|
|
||||||
what scrapers hammer, and these limits do most of the work there.
|
|
||||||
|
|
||||||
## The shared network
|
Most often this is a search. A search term is a query parameter, which the Core
|
||||||
|
Rule Set checks against its lists of system files and commands (see the fourth
|
||||||
|
change under "Attack detection" in `SPEC.md`). A release attachment uploaded
|
||||||
|
through the API as `docker-compose.yml` is refused the same way. Examples:
|
||||||
|
|
||||||
Containers on the shared network skip `servewaf`'s checks:
|
- a name on those lists, such as `package.json`, `.gitignore` or
|
||||||
|
`docker-compose.yml`;
|
||||||
|
- text that starts with a command name, such as `python3` or `ssh key`, or has
|
||||||
|
one right after a `/` in its first word, such as `feat/docker-support`;
|
||||||
|
- a shell command with its options or a system path (`ls -la`, `sed -i`,
|
||||||
|
`/bin/sh`), or a command in backticks;
|
||||||
|
- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`) or an SQL
|
||||||
|
statement (`SELECT * FROM users WHERE`);
|
||||||
|
- a URL naming an IP address or `localhost`.
|
||||||
|
|
||||||
- The shared network's subnets are `10.99.0.0/24` and `fd10:99::/64`. Docker
|
### A branch, tag or file name that starts with a command name
|
||||||
gives its containers addresses in the upper half of each, `10.99.0.128/25` and
|
|
||||||
`fd10:99:0:0:8000::/65`, which `SWWAF_ALLOW_NETS` names.
|
|
||||||
- traefik is known on the shared network by the site's host name. So a container
|
|
||||||
there that asks for the site reaches traefik from its own address. An
|
|
||||||
automation fleet in such a container skips the checks.
|
|
||||||
- Each subnet's gateway, `10.99.0.1` and `fd10:99::1`, is the host's own address
|
|
||||||
on it. Requests docker passes on from a published port can come from the
|
|
||||||
gateway, so the gateways are left out of `SWWAF_ALLOW_NETS`. The network has
|
|
||||||
IPv6, so a visitor over IPv6 reaches traefik from its own address, and
|
|
||||||
`servewaf` checks it as itself.
|
|
||||||
- A container on the shared network that asks for the site at the host's public
|
|
||||||
address arrives from the gateway. It is checked like any visitor.
|
|
||||||
- A fleet on another host can have its address added to `SWWAF_ALLOW_NETS`.
|
|
||||||
Without that, it is counted like any client.
|
|
||||||
- traefik replaces the `X-Forwarded-For` a visitor sends, by default. Keep it
|
|
||||||
so: otherwise a visitor could name an address on the shared network, and skip
|
|
||||||
the checks.
|
|
||||||
- Anything on the shared network can reach gitea and `servewaf` around `rproxy`.
|
|
||||||
Put only containers you trust on it.
|
|
||||||
|
|
||||||
## gitea's settings
|
The Core Rule Set's list of commands (932260) holds names such as `docker-`,
|
||||||
|
`python3`, `ansible`, `base64` and `whoami`. A query parameter holding a branch,
|
||||||
|
tag or file name that starts with one is refused. So is one that has such a name
|
||||||
|
right after a `/`, such as `feat/docker-support`, `fix/docker-build` or
|
||||||
|
`renovate/docker-build-push-action-6.x`. gitea's own pages send the name in a
|
||||||
|
query parameter that 930120, 932160 and 932260 still check:
|
||||||
|
|
||||||
- `REVERSE_PROXY_TRUSTED_PROXIES` names `rproxy`'s fixed addresses, `10.99.0.2`
|
- `refSubUrl`, the branch or tag of a directory listing. A listing sends it when
|
||||||
and `fd10:99::2`. gitea then takes each visitor's address from the `X-Real-Ip`
|
it asks for its entries' last commits in a second request. It does so whenever
|
||||||
header, which traefik sets and `rproxy` passes on. Without it, gitea logs
|
gitea takes more than a second to work them out;
|
||||||
every visitor as `rproxy`.
|
- `name`, when a branch is deleted or restored on the branches page;
|
||||||
- `INSTALL_LOCK` is `true`, as "The setup" above says.
|
- `tag`, when a release is started from a tag;
|
||||||
|
- `template`, the file of an issue template, when an issue is opened from it;
|
||||||
|
- `rule_name`, when a branch protection rule is opened for editing in the
|
||||||
|
repository's settings or has just been saved.
|
||||||
|
|
||||||
## Methods
|
What a visitor sees, for a branch named `docker-build`:
|
||||||
|
|
||||||
`rproxy` passes on only these methods: `GET`, `HEAD`, `POST`, `OPTIONS`, `PUT`,
|
- A directory listing that makes that second request leaves those last commits
|
||||||
`DELETE`, `PATCH` and `QUERY`. gitea 1.27.3 needs no other. Its routes use:
|
out and shows an error.
|
||||||
|
- The branches page shows an error instead of deleting or restoring the branch.
|
||||||
|
- A release started from such a tag gets the `403` answer of `smallwebwaf` in
|
||||||
|
place of its form. So does an issue opened from such a template, and such a
|
||||||
|
rule opened for editing.
|
||||||
|
- Saving such a rule stores it. gitea then returns to the branch settings page,
|
||||||
|
which gets the `403` answer, so the save looks as if it failed.
|
||||||
|
|
||||||
- pages and the API: `GET`, `HEAD`, `POST`, `PUT`, `PATCH` and `DELETE`;
|
### A file name in a query parameter that trips the code rules
|
||||||
- git's push, clone and fetch over HTTP: `GET`, `POST` and `OPTIONS`;
|
|
||||||
- LFS: `GET`, `POST` and `PUT`;
|
|
||||||
- the container registry and the other package registries: `GET`, `HEAD`,
|
|
||||||
`POST`, `PUT`, `PATCH` and `DELETE`;
|
|
||||||
- Actions runners: `POST`.
|
|
||||||
|
|
||||||
## The log
|
The Core Rule Set's rules for Java, script, PowerShell and SQL find what they
|
||||||
|
look for in some file names. This holds too in the ten query parameters that
|
||||||
|
930120, 932160 and 932260 do not check (see the fourth change under "Attack
|
||||||
|
detection" in `SPEC.md`). Examples:
|
||||||
|
|
||||||
`rproxy` and `servewaf` write each query string to the log whole. A token sent
|
- `runtime` in OpenJDK's `src/java.base/share/classes/java/lang/Runtime.java`
|
||||||
in the query string, as `?token=`, is then in the log. Send it in the
|
(944110);
|
||||||
`Authorization` header instead.
|
- a Java class, as in `java.lang.String.md` (944130);
|
||||||
|
- `document.write.js` (941180);
|
||||||
|
- `process.env.d.ts` (934100);
|
||||||
|
- `__proto__` (934130);
|
||||||
|
- the PowerShell command `Install-Script`, as in `install-script.sh` (932120);
|
||||||
|
- a database's name, as in PostgreSQL's `information_schema.sql` (942140).
|
||||||
|
|
||||||
## Actions runners
|
For such a file, these requests are refused:
|
||||||
|
|
||||||
A runner on the shared network skips the checks. One on another host is counted
|
- the API's list of its commits (`path`);
|
||||||
like any client. A runner older than version 0.4 (April 2026) asks for work
|
- the API's call for the contents of files that names it in its query (`body`);
|
||||||
every 2 seconds, 1,800 requests an hour. While it runs a job, it reports the
|
- the commit graph of that file alone (`file`);
|
||||||
job's log and state every second instead, two requests a second.
|
- the hidden lines of its diff (`path`), which stay hidden;
|
||||||
|
- the sign-in page, for a visitor who opens a page of the file that needs
|
||||||
|
signing in, such as its edit page (`redirect_to`). The visitor can still sign
|
||||||
|
in by opening the sign-in page itself.
|
||||||
|
|
||||||
So one idle runner makes 1,800 requests an hour, more than the hour limit of
|
### A path
|
||||||
1,500. A busy one, at two requests a second, passes the minute limit of 120
|
|
||||||
within two minutes. Its address is then banned, its running jobs fail, and each
|
|
||||||
repeat within a day makes the ban longer. Newer runners ask for work less often.
|
|
||||||
Put the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
|
||||||
|
|
||||||
## The Core Rule Set's exclusions
|
- A file name ending in `~`, or an `.xhtml` file whose path holds a space.
|
||||||
|
- A path that holds what the Core Rule Set takes for a Java class (944130), as
|
||||||
|
`java.lang.String.md` and
|
||||||
|
`META-INF/services/org.apache.commons.logging.LogFactory` do.
|
||||||
|
- A path that holds what it takes for Node.js code (934100), as
|
||||||
|
`process.env.d.ts` does.
|
||||||
|
- A path with a directory named `...` (930100, 930110), such as `.../notes.txt`.
|
||||||
|
|
||||||
`deploy/gitea/crs-exclusions.conf` switches single Core Rule Set rules off where
|
For such a file, these are refused:
|
||||||
they refused ordinary gitea requests. `servewaf` loads it from
|
|
||||||
`SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE`, before the Core Rule Set's rules. Each
|
|
||||||
exclusion covers one rule, in these places only:
|
|
||||||
|
|
||||||
- `q`, the text of every search;
|
- the page of the file;
|
||||||
- the query parameters that hold a file name: `file`, `path`, `template`, `body`
|
- its raw file;
|
||||||
and `redirect_to`;
|
- its history and its blame;
|
||||||
- the query parameters that hold a branch or tag name: `name`, `tag`,
|
- its edit page.
|
||||||
`rule_name` and `refSubUrl`;
|
|
||||||
- `Referer`, on a page opened from search results or from a file's page;
|
|
||||||
- the path of a file's page, raw file, blame, history and edit page, and of the
|
|
||||||
API's call for its contents;
|
|
||||||
- `name`, the file name of an attachment uploaded through the API;
|
|
||||||
- the header `Content-Range` of an Actions artifact upload.
|
|
||||||
|
|
||||||
gitea reads these as text to search for, as names in a repository, or, for
|
The directory that holds it lists it, and a clone carries it.
|
||||||
`Content-Range`, as the part of a file an upload carries. It never runs them as
|
|
||||||
SQL, a command or a page's script. So an attack that only the excluded rules
|
|
||||||
would catch there reaches gitea and does nothing. Every other field and path
|
|
||||||
keeps every rule.
|
|
||||||
|
|
||||||
To add an exclusion for a request the Core Rule Set refuses:
|
### A container image push that reuses a layer
|
||||||
|
|
||||||
1. Find the request in `servewaf`'s log. Its line gives the path and query, and
|
A push can reuse a layer of an image docker has pushed to that gitea before.
|
||||||
names the rules in `waf_rule_ids`.
|
docker then asks gitea to take the layer from that image, naming it in the query
|
||||||
2. For each rule, add a line with `ctl:ruleRemoveTargetById` for the field it
|
parameter `from`, which 932260 checks. The request is refused when the image's
|
||||||
matched. For a field on every path, copy one of the file's `SecAction` lines.
|
owner starts with an entry on the list of commands. It is refused too when the
|
||||||
For some paths only, copy one of its `SecRule REQUEST_FILENAME` lines. Give
|
image's name after the owner starts with one. Examples are `owner/python3-app`,
|
||||||
it a new id. An id from 900000 to 900999 stops the start.
|
`owner/docker-base`, and any image of an owner named `ansible` or `docker-team`.
|
||||||
3. Above it, write a one-line comment naming the request it refused.
|
|
||||||
4. Add the request to `internal/waf/testdata/gitea-requests.txt`, and run
|
|
||||||
`make check`.
|
|
||||||
5. Restart `servewaf`.
|
|
||||||
|
|
||||||
The Core Rule Set still refuses the page of a file in a directory named `...`,
|
What happens next depends on docker's image store. A docker that keeps its
|
||||||
which reads as path traversal.
|
images in containerd's image store fails the push, and fails each new try the
|
||||||
|
same way. A new install of Docker Engine 29 does this. A docker that keeps them
|
||||||
|
in its own older image store uploads the layer instead. The push then succeeds
|
||||||
|
after one refusal for each layer it reuses, and each of those refusals counts
|
||||||
|
toward the error burst ("Attack detection" in `SPEC.md`).
|
||||||
|
|
||||||
With `SWWAF_WAF_BODY_LIMIT` set, it also refuses issue text, comments, wiki
|
### Artifact uploads from `actions/upload-artifact@v3`
|
||||||
pages and files saved in the web editor that hold shell commands or code.
|
|
||||||
|
|
||||||
`make gitea-requests` runs gitea behind `rproxy` and `servewaf` at the defaults,
|
This action sends the header `Content-Range`, which the Core Rule Set refuses
|
||||||
without these exclusions. It reports each request the Core Rule Set refuses
|
(920450). The action sends two files at a time, does not retry a `403`, and
|
||||||
there.
|
sends no further file once one is refused. So an upload is refused once or
|
||||||
|
twice, however many files it holds, and the step fails. One upload does not ban
|
||||||
|
the runner. The error burst ("Attack detection" in `SPEC.md`) bans it after more
|
||||||
|
than 30 such refusals from its address in a minute, as when many jobs upload at
|
||||||
|
once. `actions/upload-artifact@v4` does not send the header.
|
||||||
|
|
||||||
|
## What body inspection refuses
|
||||||
|
|
||||||
|
With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In
|
||||||
|
front of gitea it then refuses these, until their rule ids, named in the request
|
||||||
|
log, are added to `SWWAF_WAF_DISABLED_RULES`:
|
||||||
|
|
||||||
|
- issue and comment text, wiki pages and files saved in the web editor that hold
|
||||||
|
shell commands or code (932125, 932235, 932250 and others);
|
||||||
|
- package descriptions that show code;
|
||||||
|
- PyPI uploads (922130);
|
||||||
|
- attachments named like `debug.log` or `config.yml` (932180).
|
||||||
|
|
||||||
|
## What `script/gitea-requests` sends
|
||||||
|
|
||||||
|
Of the requests above, the script sends:
|
||||||
|
|
||||||
|
- for each file named under "A file name in a query parameter that trips the
|
||||||
|
code rules", the API's list of its commits;
|
||||||
|
- for `Runtime.java` alone, the commit graph of that file, the hidden lines of
|
||||||
|
its diff, and the API's call for the contents of files that names it;
|
||||||
|
- the sign-in page for a visitor who opens the edit page of `document.write.js`;
|
||||||
|
- the pages of `java.lang.String.md`,
|
||||||
|
`META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
|
||||||
|
and `.../notes.txt`;
|
||||||
|
- one container image push that reuses a layer, which it does not try again.
|
||||||
|
|
||||||
|
It does not send the raw file, history, blame or edit page of those four files,
|
||||||
|
or the pages of the directories `META-INF/services` and `...`. It also sends a
|
||||||
|
short sample of gitea's ordinary use:
|
||||||
|
|
||||||
|
- signing in;
|
||||||
|
- pages of files and commits, and a diff;
|
||||||
|
- the API;
|
||||||
|
- a push, and clone and fetch over HTTP;
|
||||||
|
- two container image pushes.
|
||||||
|
|||||||
+15
-25
@@ -99,22 +99,19 @@ chooses between the two (see "The image's command" below).
|
|||||||
|
|
||||||
## Request rate limits, per client
|
## Request rate limits, per client
|
||||||
|
|
||||||
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `120`, 2 a second): the most requests a
|
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `500`, about 8 a second): the most
|
||||||
client may make in a minute. A count or `off`. The default leaves room for a
|
requests a client may make in a minute. A count or `off`. The default catches,
|
||||||
person opening two gitea pages at once with an empty cache, up to 60 files
|
within seconds, a client firing requests as fast as it can. It leaves room for
|
||||||
each.
|
a fleet of about 20 agents at its busiest, about 120 requests a minute. It
|
||||||
- `SWWAF_RATE_LIMIT_PER_HOUR` (default `1500`): the most requests a client may
|
also leaves room for a few quick first visits to gitea with an empty cache,
|
||||||
make in an hour. A count or `off`. It is the main limit against scrapers. A
|
about 40 files each.
|
||||||
person reading code in gitea for an hour, a page every 20 seconds, makes
|
- `SWWAF_RATE_LIMIT_PER_HOUR` (default `20000`, about 5.5 a second): the most
|
||||||
about 900. A scraper making one request every 2 seconds makes 1,800 an hour,
|
requests a client may make in an hour. A count or `off`. The default leaves
|
||||||
more than the limit.
|
room for an automation fleet at work. It catches a client that keeps up more
|
||||||
- `SWWAF_RATE_LIMIT_PER_DAY` (default `6000`): the most requests a client may
|
than about 5.5 a second for an hour.
|
||||||
make in a day. A count or `off`. The default is four hours at the hour limit.
|
- `SWWAF_RATE_LIMIT_PER_DAY` (default `300000`, about 3.5 a second): the most
|
||||||
A scraper making one request every 10 seconds makes 8,640 a day, more than the
|
requests a client may make in a day. A count or `off`. The default catches a
|
||||||
limit.
|
scraper that keeps up 4 a second around the clock.
|
||||||
- An Actions runner asks gitea for work all day and can pass the limits. Put the
|
|
||||||
runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. Do the same for a
|
|
||||||
network where many people share one address, such as an office.
|
|
||||||
- Counting cannot catch a client that stays under all three limits (see "Request
|
- Counting cannot catch a client that stays under all three limits (see "Request
|
||||||
rate limits, per client" in [`SPEC.md`](../SPEC.md)).
|
rate limits, per client" in [`SPEC.md`](../SPEC.md)).
|
||||||
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
|
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
|
||||||
@@ -122,8 +119,8 @@ chooses between the two (see "The image's command" below).
|
|||||||
|
|
||||||
## Byte limits, per client
|
## Byte limits, per client
|
||||||
|
|
||||||
- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `off`), `SWWAF_BYTES_LIMIT_PER_HOUR`
|
- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `4G`), `SWWAF_BYTES_LIMIT_PER_HOUR`
|
||||||
(default `50G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `100G`): the most
|
(default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `200G`): the most
|
||||||
bytes counted for a client in each window. A size or `off`.
|
bytes counted for a client in each window. A size or `off`.
|
||||||
- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
|
- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
|
||||||
`response`, `request` or `both`.
|
`response`, `request` or `both`.
|
||||||
@@ -192,13 +189,6 @@ chooses between the two (see "The image's command" below).
|
|||||||
the Core Rule Set reads, a size of at most `64K`. A question carries no more
|
the Core Rule Set reads, a size of at most `64K`. A question carries no more
|
||||||
of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one
|
of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one
|
||||||
byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
|
byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
|
||||||
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule Set
|
|
||||||
exclusions, written as Coraza directives. It is loaded before the Core Rule
|
|
||||||
Set's rules, so it holds only `SecAction` and `SecRule` lines that exclude
|
|
||||||
with `ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. A file that cannot be
|
|
||||||
read or does not parse stops the start, whatever `SWWAF_WAF_MODE` is. So does
|
|
||||||
a file that names a number from 900000 to 900999 outside a comment, alone or
|
|
||||||
in a range. It has no `_FILE` form.
|
|
||||||
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
|
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
|
||||||
in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
|
in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
|
||||||
|
|
||||||
|
|||||||
@@ -221,15 +221,12 @@ type Config struct {
|
|||||||
// WAFExemptPaths the path prefixes it does not inspect
|
// WAFExemptPaths the path prefixes it does not inspect
|
||||||
// (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body
|
// (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body
|
||||||
// it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
// it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
||||||
// CoreRuleSetExclusionsFile is the path of the file of its exclusions
|
|
||||||
// (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while there is none.
|
|
||||||
WAFMode string
|
WAFMode string
|
||||||
WAFParanoiaLevel int
|
WAFParanoiaLevel int
|
||||||
WAFAnomalyThreshold int
|
WAFAnomalyThreshold int
|
||||||
WAFDisabledRules []int
|
WAFDisabledRules []int
|
||||||
WAFExemptPaths []string
|
WAFExemptPaths []string
|
||||||
WAFBodyLimit int64
|
WAFBodyLimit int64
|
||||||
CoreRuleSetExclusionsFile string
|
|
||||||
// TrapPaths are the paths a request for which is a clear sign of
|
// TrapPaths are the paths a request for which is a clear sign of
|
||||||
// attack (SWWAF_TRAP_PATHS), each starting with / and without a ?.
|
// attack (SWWAF_TRAP_PATHS), each starting with / and without a ?.
|
||||||
TrapPaths []string
|
TrapPaths []string
|
||||||
@@ -471,13 +468,13 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
|
|||||||
AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""),
|
AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""),
|
||||||
RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
|
RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
|
||||||
DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
|
DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
|
||||||
RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "120"),
|
RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "500"),
|
||||||
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "1500"),
|
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "20000"),
|
||||||
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "6000"),
|
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "300000"),
|
||||||
RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
|
RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
|
||||||
BytesLimitPerMinute: env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "off"),
|
BytesLimitPerMinute: env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "4G"),
|
||||||
BytesLimitPerHour: env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "50G"),
|
BytesLimitPerHour: env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "20G"),
|
||||||
BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "100G"),
|
BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "200G"),
|
||||||
BytesCount: env.bytesCount("SWWAF_BYTES_COUNT", "both"),
|
BytesCount: env.bytesCount("SWWAF_BYTES_COUNT", "both"),
|
||||||
LookupSource: env.lookupSource("SWWAF_LOOKUP_SOURCE", "geojs"),
|
LookupSource: env.lookupSource("SWWAF_LOOKUP_SOURCE", "geojs"),
|
||||||
LookupDBPath: env.value("SWWAF_LOOKUP_DB_PATH", ""),
|
LookupDBPath: env.value("SWWAF_LOOKUP_DB_PATH", ""),
|
||||||
@@ -528,8 +525,6 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
|
|||||||
"920340,920420,920440,920640,930130,930140"),
|
"920340,920420,920440,920640,930130,930140"),
|
||||||
WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
|
WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
|
||||||
WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
|
WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
|
||||||
CoreRuleSetExclusionsFile: env.filePath(
|
|
||||||
"SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"),
|
|
||||||
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
|
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
|
||||||
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
|
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
|
||||||
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
|
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
|
||||||
@@ -1194,16 +1189,6 @@ func (e *environment) logRemoteURL(name string) *url.URL {
|
|||||||
return remote
|
return remote
|
||||||
}
|
}
|
||||||
|
|
||||||
// filePath reads a setting that is the path of a file, "" while it is
|
|
||||||
// unset. Its value names a file already, so, unlike the other settings,
|
|
||||||
// it has no _FILE form. Whoever uses the file reads it.
|
|
||||||
func (e *environment) filePath(name string) string {
|
|
||||||
path, _ := e.lookupEnv(name)
|
|
||||||
e.settings = append(e.settings, slog.String(name, path))
|
|
||||||
|
|
||||||
return path
|
|
||||||
}
|
|
||||||
|
|
||||||
// certificates reads a setting that is the path of a file of PEM
|
// certificates reads a setting that is the path of a file of PEM
|
||||||
// certificates. Unset or empty, it is nil. Its value names a file
|
// certificates. Unset or empty, it is nil. Its value names a file
|
||||||
// already, so, unlike the other settings, it has no _FILE form.
|
// already, so, unlike the other settings, it has no _FILE form.
|
||||||
|
|||||||
@@ -87,7 +87,6 @@ const (
|
|||||||
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
|
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
|
||||||
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
|
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
|
||||||
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
|
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
|
||||||
exclusionsFile = "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"
|
|
||||||
trapPaths = "SWWAF_TRAP_PATHS"
|
trapPaths = "SWWAF_TRAP_PATHS"
|
||||||
errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
|
errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
|
||||||
logRemoteURL = "SWWAF_LOG_REMOTE_URL"
|
logRemoteURL = "SWWAF_LOG_REMOTE_URL"
|
||||||
@@ -245,9 +244,9 @@ func TestDefaults(t *testing.T) {
|
|||||||
wantSettings(t, cfg, config.Config{
|
wantSettings(t, cfg, config.Config{
|
||||||
ListenAddr: ":8080",
|
ListenAddr: ":8080",
|
||||||
Observe: false,
|
Observe: false,
|
||||||
RateLimitPerMinute: 120,
|
RateLimitPerMinute: 500,
|
||||||
RateLimitPerHour: 1500,
|
RateLimitPerHour: 20000,
|
||||||
RateLimitPerDay: 6000,
|
RateLimitPerDay: 300000,
|
||||||
BanResponse: 403,
|
BanResponse: 403,
|
||||||
LimitBanDuration: time.Hour,
|
LimitBanDuration: time.Hour,
|
||||||
LimitBanRepeatWindow: 24 * time.Hour,
|
LimitBanRepeatWindow: 24 * time.Hour,
|
||||||
@@ -267,8 +266,8 @@ func TestDefaults(t *testing.T) {
|
|||||||
LookupSource: defaultLookupSource, LookupTimeout: 800 * time.Millisecond,
|
LookupSource: defaultLookupSource, LookupTimeout: 800 * time.Millisecond,
|
||||||
})
|
})
|
||||||
wantByteLimitSettings(t, cfg, config.Config{
|
wantByteLimitSettings(t, cfg, config.Config{
|
||||||
BytesLimitPerMinute: 0, BytesLimitPerHour: 50 << 30,
|
BytesLimitPerMinute: 4 << 30, BytesLimitPerHour: 20 << 30,
|
||||||
BytesLimitPerDay: 100 << 30, BytesCount: "both",
|
BytesLimitPerDay: 200 << 30, BytesCount: "both",
|
||||||
})
|
})
|
||||||
|
|
||||||
wantNetblocks(t, cfg.AllowNets)
|
wantNetblocks(t, cfg.AllowNets)
|
||||||
@@ -309,7 +308,7 @@ func TestNoSingleRequestBreaksAByteLimitAtTheDefaults(t *testing.T) {
|
|||||||
for _, limit := range []int64{
|
for _, limit := range []int64{
|
||||||
cfg.BytesLimitPerMinute, cfg.BytesLimitPerHour, cfg.BytesLimitPerDay,
|
cfg.BytesLimitPerMinute, cfg.BytesLimitPerHour, cfg.BytesLimitPerDay,
|
||||||
} {
|
} {
|
||||||
if limit != 0 && largest > limit { // 0 is off
|
if largest > limit {
|
||||||
t.Errorf("a request of %d bytes breaks the byte limit of %d", largest, limit)
|
t.Errorf("a request of %d bytes breaks the byte limit of %d", largest, limit)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -540,14 +539,13 @@ func TestCoreRuleSetSettings(t *testing.T) {
|
|||||||
environment{
|
environment{
|
||||||
wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10",
|
wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10",
|
||||||
wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/",
|
wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/",
|
||||||
wafBodyLimit: "32K", exclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
|
wafBodyLimit: "32K",
|
||||||
},
|
},
|
||||||
config.Config{
|
config.Config{
|
||||||
WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10,
|
WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10,
|
||||||
WAFDisabledRules: []int{942100, 920350},
|
WAFDisabledRules: []int{942100, 920350},
|
||||||
WAFExemptPaths: []string{"/api/", "/static/"},
|
WAFExemptPaths: []string{"/api/", "/static/"},
|
||||||
WAFBodyLimit: 32 << 10,
|
WAFBodyLimit: 32 << 10,
|
||||||
CoreRuleSetExclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
|
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -568,7 +566,6 @@ func TestCoreRuleSetSettings(t *testing.T) {
|
|||||||
WAFAnomalyThreshold: cfg.WAFAnomalyThreshold,
|
WAFAnomalyThreshold: cfg.WAFAnomalyThreshold,
|
||||||
WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths,
|
WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths,
|
||||||
WAFBodyLimit: cfg.WAFBodyLimit,
|
WAFBodyLimit: cfg.WAFBodyLimit,
|
||||||
CoreRuleSetExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
|
||||||
}
|
}
|
||||||
if !reflect.DeepEqual(got, tc.want) {
|
if !reflect.DeepEqual(got, tc.want) {
|
||||||
t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want)
|
t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want)
|
||||||
@@ -2274,12 +2271,12 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
|||||||
rateLimitExemptNets: "",
|
rateLimitExemptNets: "",
|
||||||
denyNets: "",
|
denyNets: "",
|
||||||
rateLimitPerMinute: "45",
|
rateLimitPerMinute: "45",
|
||||||
rateLimitPerHour: "1500",
|
rateLimitPerHour: "20000",
|
||||||
rateLimitPerDay: "6000",
|
rateLimitPerDay: "300000",
|
||||||
rateLimitExemptPaths: "",
|
rateLimitExemptPaths: "",
|
||||||
bytesLimitPerMinute: "off",
|
bytesLimitPerMinute: "4G",
|
||||||
bytesLimitPerHour: "50G",
|
bytesLimitPerHour: "20G",
|
||||||
bytesLimitPerDay: "100G",
|
bytesLimitPerDay: "200G",
|
||||||
bytesCount: "both",
|
bytesCount: "both",
|
||||||
lookupSource: defaultLookupSource,
|
lookupSource: defaultLookupSource,
|
||||||
lookupDBPath: "",
|
lookupDBPath: "",
|
||||||
@@ -2330,7 +2327,6 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
|||||||
wafDisabledRules: defaultWAFDisabledRules,
|
wafDisabledRules: defaultWAFDisabledRules,
|
||||||
wafExemptPaths: "",
|
wafExemptPaths: "",
|
||||||
wafBodyLimit: off,
|
wafBodyLimit: off,
|
||||||
exclusionsFile: "",
|
|
||||||
trapPaths: "",
|
trapPaths: "",
|
||||||
errorBurstThreshold: "30",
|
errorBurstThreshold: "30",
|
||||||
logRemoteURL: "",
|
logRemoteURL: "",
|
||||||
|
|||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
@@ -10,9 +10,9 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/state"
|
"sneak.berlin/go/smallwebwaf/internal/state"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -88,11 +88,11 @@ func TestAdminEndpointsNeedTheAdminToken(t *testing.T) {
|
|||||||
|
|
||||||
// Any other request under /_smallwebwaf/ is not found.
|
// Any other request under /_smallwebwaf/ is not found.
|
||||||
for _, e := range []adminEndpoint{
|
for _, e := range []adminEndpoint{
|
||||||
{http.MethodPut, servewaf.BansPath, banOtherClient},
|
{http.MethodPut, proxy.BansPath, banOtherClient},
|
||||||
{http.MethodDelete, servewaf.BansPath, ""},
|
{http.MethodDelete, proxy.BansPath, ""},
|
||||||
{http.MethodGet, servewaf.BansPath + "/" + otherClient, ""},
|
{http.MethodGet, proxy.BansPath + "/" + otherClient, ""},
|
||||||
{http.MethodPost, servewaf.ClientsPath + otherClient, ""},
|
{http.MethodPost, proxy.ClientsPath + otherClient, ""},
|
||||||
{http.MethodGet, strings.TrimSuffix(servewaf.ClientsPath, "/"), ""},
|
{http.MethodGet, strings.TrimSuffix(proxy.ClientsPath, "/"), ""},
|
||||||
} {
|
} {
|
||||||
s.admin(e.method, e.path, e.body, http.StatusNotFound)
|
s.admin(e.method, e.path, e.body, http.StatusNotFound)
|
||||||
}
|
}
|
||||||
@@ -118,7 +118,7 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
|
|||||||
Reason: "probes for logins",
|
Reason: "probes for logins",
|
||||||
}
|
}
|
||||||
|
|
||||||
wantBans(t, s.admin(http.MethodPost, servewaf.BansPath, banOtherClient, http.StatusOK),
|
wantBans(t, s.admin(http.MethodPost, proxy.BansPath, banOtherClient, http.StatusOK),
|
||||||
want)
|
want)
|
||||||
|
|
||||||
line := s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
line := s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
@@ -129,7 +129,7 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
|
|||||||
// Its notes count the request it refused.
|
// Its notes count the request it refused.
|
||||||
want.Notes.Requests, want.Notes.Refused = 1, 1
|
want.Notes.Requests, want.Notes.Refused = 1, 1
|
||||||
|
|
||||||
wantBans(t, s.admin(http.MethodGet, servewaf.BansPath, "", http.StatusOK), want)
|
wantBans(t, s.admin(http.MethodGet, proxy.BansPath, "", http.StatusOK), want)
|
||||||
|
|
||||||
// Ten minutes on, lifting the bans on client lifts that one, which is
|
// Ten minutes on, lifting the bans on client lifts that one, which is
|
||||||
// kept, marked lifted.
|
// kept, marked lifted.
|
||||||
@@ -138,13 +138,13 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
|
|||||||
lifted := clk.Now()
|
lifted := clk.Now()
|
||||||
want.Lifted = &lifted
|
want.Lifted = &lifted
|
||||||
|
|
||||||
wantBans(t, s.admin(http.MethodDelete, servewaf.BansPath+"/"+client, "",
|
wantBans(t, s.admin(http.MethodDelete, proxy.BansPath+"/"+client, "",
|
||||||
http.StatusOK), want)
|
http.StatusOK), want)
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
wantBans(t, s.admin(http.MethodGet, servewaf.BansPath, "", http.StatusOK), want)
|
wantBans(t, s.admin(http.MethodGet, proxy.BansPath, "", http.StatusOK), want)
|
||||||
|
|
||||||
// No ban on it is active any more.
|
// No ban on it is active any more.
|
||||||
s.admin(http.MethodDelete, servewaf.BansPath+"/"+client, "", http.StatusNotFound)
|
s.admin(http.MethodDelete, proxy.BansPath+"/"+client, "", http.StatusNotFound)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
|
func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
|
||||||
@@ -181,7 +181,7 @@ func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
|
|||||||
want.Expires = &expires
|
want.Expires = &expires
|
||||||
}
|
}
|
||||||
|
|
||||||
wantBans(t, s.admin(http.MethodPost, servewaf.BansPath, body, http.StatusOK), want)
|
wantBans(t, s.admin(http.MethodPost, proxy.BansPath, body, http.StatusOK), want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -250,7 +250,7 @@ func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) {
|
|||||||
"request body too large",
|
"request body too large",
|
||||||
},
|
},
|
||||||
} {
|
} {
|
||||||
got := s.admin(http.MethodPost, servewaf.BansPath, tc.body, http.StatusBadRequest)
|
got := s.admin(http.MethodPost, proxy.BansPath, tc.body, http.StatusBadRequest)
|
||||||
if !strings.Contains(string(got.body), tc.want) {
|
if !strings.Contains(string(got.body), tc.want) {
|
||||||
t.Errorf("%.80s was answered %q, want it to say %q", tc.body, got.body, tc.want)
|
t.Errorf("%.80s was answered %q, want it to say %q", tc.body, got.body, tc.want)
|
||||||
}
|
}
|
||||||
@@ -283,7 +283,7 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
|
|||||||
Bans []state.BanEntry `json:"bans"`
|
Bans []state.BanEntry `json:"bans"`
|
||||||
}
|
}
|
||||||
|
|
||||||
decode(t, s.admin(http.MethodGet, servewaf.ClientsPath+addr, "", http.StatusOK), &got)
|
decode(t, s.admin(http.MethodGet, proxy.ClientsPath+addr, "", http.StatusOK), &got)
|
||||||
|
|
||||||
if got.Client == nil {
|
if got.Client == nil {
|
||||||
t.Fatalf("%s: no client", addr)
|
t.Fatalf("%s: no client", addr)
|
||||||
@@ -305,13 +305,13 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
|
|||||||
|
|
||||||
// Of an address no request came from and no ban covers, nothing is
|
// Of an address no request came from and no ban covers, nothing is
|
||||||
// known.
|
// known.
|
||||||
got := s.admin(http.MethodGet, servewaf.ClientsPath+"198.51.100.99", "", http.StatusOK)
|
got := s.admin(http.MethodGet, proxy.ClientsPath+"198.51.100.99", "", http.StatusOK)
|
||||||
if string(got.body) != "{\n \"client\": null,\n \"bans\": []\n}\n" {
|
if string(got.body) != "{\n \"client\": null,\n \"bans\": []\n}\n" {
|
||||||
t.Errorf("an unknown client is answered\n%s", got.body)
|
t.Errorf("an unknown client is answered\n%s", got.body)
|
||||||
}
|
}
|
||||||
|
|
||||||
s.admin(http.MethodGet, servewaf.ClientsPath+"203.0.113", "", http.StatusBadRequest)
|
s.admin(http.MethodGet, proxy.ClientsPath+"203.0.113", "", http.StatusBadRequest)
|
||||||
s.admin(http.MethodDelete, servewaf.BansPath+"/203.0.113.0/24", "",
|
s.admin(http.MethodDelete, proxy.BansPath+"/203.0.113.0/24", "",
|
||||||
http.StatusBadRequest)
|
http.StatusBadRequest)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -342,10 +342,10 @@ type adminEndpoint struct {
|
|||||||
// bans on otherClient, and asking about otherClient.
|
// bans on otherClient, and asking about otherClient.
|
||||||
func adminEndpoints() []adminEndpoint {
|
func adminEndpoints() []adminEndpoint {
|
||||||
return []adminEndpoint{
|
return []adminEndpoint{
|
||||||
{http.MethodGet, servewaf.BansPath, ""},
|
{http.MethodGet, proxy.BansPath, ""},
|
||||||
{http.MethodPost, servewaf.BansPath, banOtherClient},
|
{http.MethodPost, proxy.BansPath, banOtherClient},
|
||||||
{http.MethodDelete, servewaf.BansPath + "/" + otherClient, ""},
|
{http.MethodDelete, proxy.BansPath + "/" + otherClient, ""},
|
||||||
{http.MethodGet, servewaf.ClientsPath + otherClient, ""},
|
{http.MethodGet, proxy.ClientsPath + otherClient, ""},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"maps"
|
"maps"
|
||||||
@@ -10,8 +10,8 @@ import (
|
|||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -201,7 +201,7 @@ func TestObserveModeWorksOutABanOnlyWhenItsAlertWouldBeSent(t *testing.T) {
|
|||||||
// never sent them, and returns the queue they wait in as well.
|
// never sent them, and returns the queue they wait in as well.
|
||||||
func startWithAlerts(
|
func startWithAlerts(
|
||||||
t *testing.T, env map[string]string,
|
t *testing.T, env map[string]string,
|
||||||
) (*sender, *clock, *servewaf.Server, *alerts.Queue) {
|
) (*sender, *clock, *proxy.Server, *alerts.Queue) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
return startAppWithAlerts(t, func(http.ResponseWriter, *http.Request) {}, env)
|
return startAppWithAlerts(t, func(http.ResponseWriter, *http.Request) {}, env)
|
||||||
@@ -210,7 +210,7 @@ func startWithAlerts(
|
|||||||
// startAppWithAlerts is startWithAlerts in front of the app handler.
|
// startAppWithAlerts is startWithAlerts in front of the app handler.
|
||||||
func startAppWithAlerts(
|
func startAppWithAlerts(
|
||||||
t *testing.T, handler http.HandlerFunc, env map[string]string,
|
t *testing.T, handler http.HandlerFunc, env map[string]string,
|
||||||
) (*sender, *clock, *servewaf.Server, *alerts.Queue) {
|
) (*sender, *clock, *proxy.Server, *alerts.Queue) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
app := startApp(t, handler)
|
app := startApp(t, handler)
|
||||||
+1
-1
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"net/netip"
|
"net/netip"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"maps"
|
"maps"
|
||||||
@@ -13,9 +13,9 @@ import (
|
|||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// The anomaly thresholds: the prefix of a scope followed by the end of a
|
// The anomaly thresholds: the prefix of a scope followed by the end of a
|
||||||
@@ -319,7 +319,7 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
|
|||||||
// kept.
|
// kept.
|
||||||
geojsURL, _, release := startHeldGeoJS(t)
|
geojsURL, _, release := startHeldGeoJS(t)
|
||||||
|
|
||||||
var server atomic.Pointer[servewaf.Server]
|
var server atomic.Pointer[proxy.Server]
|
||||||
|
|
||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
release()
|
release()
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
@@ -16,10 +16,10 @@ import (
|
|||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -81,7 +81,7 @@ func TestSixteenThousandCookiesAreRefusedWellWithinTheTimeLimit(t *testing.T) {
|
|||||||
t.Errorf("answered after %s, want well within %s", took, answerTimeLimit)
|
t.Errorf("answered after %s, want well within %s", took, answerTimeLimit)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantDisposition(t, got, tooLarge, servewaf.DispositionDeny)
|
wantDisposition(t, got, tooLarge, proxy.DispositionDeny)
|
||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
|
wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
|
||||||
@@ -144,13 +144,13 @@ func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
|
|||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
|
|
||||||
if tc.refused {
|
if tc.refused {
|
||||||
wantDisposition(t, got, tooLarge, servewaf.DispositionDeny)
|
wantDisposition(t, got, tooLarge, proxy.DispositionDeny)
|
||||||
wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
|
wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
|
||||||
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
|
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
if line.DurationWAF == nil {
|
if line.DurationWAF == nil {
|
||||||
@@ -175,7 +175,7 @@ func TestHeaderLimitsHoldWhateverTheCoreRuleSetSettingsSay(t *testing.T) {
|
|||||||
req := s.question(siteA, client, "/")
|
req := s.question(siteA, client, "/")
|
||||||
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
||||||
|
|
||||||
wantDisposition(t, do(t, req), tooLarge, servewaf.DispositionDeny)
|
wantDisposition(t, do(t, req), tooLarge, proxy.DispositionDeny)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -187,7 +187,7 @@ func TestClientInTheAllowNetsSkipsTheHeaderLimits(t *testing.T) {
|
|||||||
req := s.question(siteA, client, "/")
|
req := s.question(siteA, client, "/")
|
||||||
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
||||||
|
|
||||||
wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
|
func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
|
||||||
@@ -204,7 +204,7 @@ func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
|
|||||||
t.Errorf("answered after %s, want well within %s", took, answerTimeLimit)
|
t.Errorf("answered after %s, want well within %s", took, answerTimeLimit)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
if line.WouldAction != requestlog.ActionTooLarge || line.DurationWAF != nil {
|
if line.WouldAction != requestlog.ActionTooLarge || line.DurationWAF != nil {
|
||||||
@@ -224,8 +224,8 @@ func TestRequestsOverTheHeaderLimitsCountTowardTheRateLimits(t *testing.T) {
|
|||||||
return do(t, req)
|
return do(t, req)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantDisposition(t, ask(), tooLarge, servewaf.DispositionDeny)
|
wantDisposition(t, ask(), tooLarge, proxy.DispositionDeny)
|
||||||
wantDisposition(t, ask(), http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, ask(), http.StatusForbidden, proxy.DispositionDeny)
|
||||||
|
|
||||||
lines := s.out.requestLines(t, 2)
|
lines := s.out.requestLines(t, 2)
|
||||||
wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionRateLimited)
|
wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
@@ -249,7 +249,7 @@ func TestTheHeaderLimitsComeAfterTheTrapPathsAndTheRuleFiles(t *testing.T) {
|
|||||||
req := s.question(siteA, client, tc.path)
|
req := s.question(siteA, client, tc.path)
|
||||||
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
||||||
|
|
||||||
wantDisposition(t, do(t, req), http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, do(t, req), http.StatusForbidden, proxy.DispositionDeny)
|
||||||
wantLine(t, s.out.requestLine(t), http.StatusForbidden, tc.action)
|
wantLine(t, s.out.requestLine(t), http.StatusForbidden, tc.action)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -265,7 +265,7 @@ func TestNotWaitingForGeoJSNeverLowersAClientsLimit(t *testing.T) {
|
|||||||
req := s.question(siteA, fromDE, "/")
|
req := s.question(siteA, fromDE, "/")
|
||||||
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
||||||
|
|
||||||
wantDisposition(t, do(t, req), tooLarge, servewaf.DispositionDeny)
|
wantDisposition(t, do(t, req), tooLarge, proxy.DispositionDeny)
|
||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
|
wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
|
||||||
@@ -275,7 +275,7 @@ func TestNotWaitingForGeoJSNeverLowersAClientsLimit(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// The client is under no ban, and GeoJS places it in a country.
|
// The client is under no ban, and GeoJS places it in a country.
|
||||||
wantDisposition(t, s.ask(fromDE, "/"), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, s.ask(fromDE, "/"), http.StatusOK, proxy.DispositionAllow)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T) {
|
func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T) {
|
||||||
@@ -291,7 +291,7 @@ func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T
|
|||||||
req := s.question(siteA, client, "/")
|
req := s.question(siteA, client, "/")
|
||||||
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
||||||
|
|
||||||
wantDisposition(t, do(t, req), http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, do(t, req), http.StatusForbidden, proxy.DispositionDeny)
|
||||||
wantLine(t, s.out.requestLine(t), http.StatusForbidden, requestlog.ActionRateLimited)
|
wantLine(t, s.out.requestLine(t), http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -306,7 +306,7 @@ func TestObserveModeWaitsForGeoJSForARequestOverTheHeaderLimits(t *testing.T) {
|
|||||||
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
req.Header.Set(cookieHeader, cookies(cookieLimit+1))
|
||||||
|
|
||||||
got := do(t, req)
|
got := do(t, req)
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country")
|
asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country")
|
||||||
if asn != asnDE || country != "DE" {
|
if asn != asnDE || country != "DE" {
|
||||||
@@ -318,9 +318,7 @@ func TestObserveModeWaitsForGeoJSForARequestOverTheHeaderLimits(t *testing.T) {
|
|||||||
// newServeWAFInBubble makes a servewaf for a synctest bubble: GeoJS never
|
// newServeWAFInBubble makes a servewaf for a synctest bubble: GeoJS never
|
||||||
// answers, a setting needs its answer, and everything is at its default but
|
// answers, a setting needs its answer, and everything is at its default but
|
||||||
// for env. No request to it may use the network.
|
// for env. No request to it may use the network.
|
||||||
func newServeWAFInBubble(
|
func newServeWAFInBubble(t *testing.T, env map[string]string) (*proxy.Server, *output) {
|
||||||
t *testing.T, env map[string]string,
|
|
||||||
) (*servewaf.Server, *output) {
|
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
settings := map[string]string{deniedCountries: "KP"}
|
settings := map[string]string{deniedCountries: "KP"}
|
||||||
@@ -334,13 +332,13 @@ func newServeWAFInBubble(
|
|||||||
// askDirectly hands a question about a request from client straight to server,
|
// askDirectly hands a question about a request from client straight to server,
|
||||||
// as change leaves it, and returns the answer's status and how long it took.
|
// as change leaves it, and returns the answer's status and how long it took.
|
||||||
func askDirectly(
|
func askDirectly(
|
||||||
t *testing.T, server *servewaf.Server, change func(*http.Request),
|
t *testing.T, server *proxy.Server, change func(*http.Request),
|
||||||
) (int, time.Duration) {
|
) (int, time.Duration) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
|
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
|
||||||
req.RemoteAddr = net.JoinHostPort(rproxyPeer, "1234")
|
req.RemoteAddr = net.JoinHostPort(rproxyPeer, "1234")
|
||||||
req.Header.Set(servewaf.ClientHeader, client)
|
req.Header.Set(proxy.ClientHeader, client)
|
||||||
req.Header.Set(requestIDHeader, "the-id")
|
req.Header.Set(requestIDHeader, "the-id")
|
||||||
change(req)
|
change(req)
|
||||||
|
|
||||||
@@ -353,17 +351,17 @@ func askDirectly(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// banClient bans client at now.
|
// banClient bans client at now.
|
||||||
func banClient(_ *testing.T, server *servewaf.Server, now time.Time) {
|
func banClient(_ *testing.T, server *proxy.Server, now time.Time) {
|
||||||
server.Ledger.BanForLimit(netip.MustParsePrefix(client+"/32"), now, bans.Notes{})
|
server.Ledger.BanForLimit(netip.MustParsePrefix(client+"/32"), now, bans.Notes{})
|
||||||
}
|
}
|
||||||
|
|
||||||
// countClient counts one request from client at now for the rate limits.
|
// countClient counts one request from client at now for the rate limits.
|
||||||
func countClient(_ *testing.T, server *servewaf.Server, now time.Time) {
|
func countClient(_ *testing.T, server *proxy.Server, now time.Time) {
|
||||||
server.Limiter.Count(netip.MustParsePrefix(client+"/32"), now, wholeLimit)
|
server.Limiter.Count(netip.MustParsePrefix(client+"/32"), now, wholeLimit)
|
||||||
}
|
}
|
||||||
|
|
||||||
// listClient puts client on the blocklist at dropURL.
|
// listClient puts client on the blocklist at dropURL.
|
||||||
func listClient(t *testing.T, server *servewaf.Server, now time.Time) {
|
func listClient(t *testing.T, server *proxy.Server, now time.Time) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
err := server.Lists.Load([]reputation.List{{
|
err := server.Lists.Load([]reputation.List{{
|
||||||
@@ -375,13 +373,13 @@ func listClient(t *testing.T, server *servewaf.Server, now time.Time) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// decideOnClient puts client on the CrowdSec decision list.
|
// decideOnClient puts client on the CrowdSec decision list.
|
||||||
func decideOnClient(t *testing.T, server *servewaf.Server, now time.Time) {
|
func decideOnClient(t *testing.T, server *proxy.Server, now time.Time) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
loadDecisions(t, server, now, clientsDecision)
|
loadDecisions(t, server, now, clientsDecision)
|
||||||
}
|
}
|
||||||
|
|
||||||
// leaveAsItIs prepares nothing.
|
// leaveAsItIs prepares nothing.
|
||||||
func leaveAsItIs(*testing.T, *servewaf.Server, time.Time) {}
|
func leaveAsItIs(*testing.T, *proxy.Server, time.Time) {}
|
||||||
|
|
||||||
func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
|
func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
@@ -392,7 +390,7 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
|
|||||||
name string
|
name string
|
||||||
env map[string]string
|
env map[string]string
|
||||||
// prepare gets the server ready, at now, before the question.
|
// prepare gets the server ready, at now, before the question.
|
||||||
prepare func(t *testing.T, server *servewaf.Server, now time.Time)
|
prepare func(t *testing.T, server *proxy.Server, now time.Time)
|
||||||
// cookie is the request's Cookie header.
|
// cookie is the request's Cookie header.
|
||||||
cookie string
|
cookie string
|
||||||
status int
|
status int
|
||||||
@@ -463,7 +461,7 @@ func TestAQuestionWaitsForASlowGeoJSOnlyAsLongAsTheDefaultSays(t *testing.T) {
|
|||||||
name string
|
name string
|
||||||
env map[string]string
|
env map[string]string
|
||||||
// prepare gets the server ready, at now, before the question.
|
// prepare gets the server ready, at now, before the question.
|
||||||
prepare func(t *testing.T, server *servewaf.Server, now time.Time)
|
prepare func(t *testing.T, server *proxy.Server, now time.Time)
|
||||||
status int
|
status int
|
||||||
action string
|
action string
|
||||||
}{
|
}{
|
||||||
@@ -521,10 +519,10 @@ func TestServeWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(t *testin
|
|||||||
|
|
||||||
t.Cleanup(func() { _ = resolver.Close() })
|
t.Cleanup(func() { _ = resolver.Close() })
|
||||||
|
|
||||||
server, out, _ := newServeWAFWith(t, func(params servewaf.Params) *servewaf.Server {
|
server, out, _ := newServeWAFWith(t, func(params proxy.Params) *proxy.Server {
|
||||||
params.AbuseIPDBURL = silentAbuseIPDBURL
|
params.AbuseIPDBURL = silentAbuseIPDBURL
|
||||||
|
|
||||||
return servewaf.New(params)
|
return proxy.NewServeWAF(params)
|
||||||
}, held, time.Now, map[string]string{
|
}, held, time.Now, map[string]string{
|
||||||
deniedCountries: "KP", wafMode: block,
|
deniedCountries: "KP", wafMode: block,
|
||||||
dnsblZones: dnsblZone, dnsblResolver: resolver.LocalAddr().String(),
|
dnsblZones: dnsblZone, dnsblResolver: resolver.LocalAddr().String(),
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"bufio"
|
"bufio"
|
||||||
@@ -15,8 +15,8 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -382,7 +382,7 @@ func (c *clock) advance(d time.Duration) {
|
|||||||
// window.
|
// window.
|
||||||
func startWithClock(
|
func startWithClock(
|
||||||
t *testing.T, geojsURL string, env map[string]string,
|
t *testing.T, geojsURL string, env map[string]string,
|
||||||
) (*sender, *clock, *servewaf.Server) {
|
) (*sender, *clock, *proxy.Server) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
@@ -16,8 +16,8 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// The biased thresholds.
|
// The biased thresholds.
|
||||||
@@ -419,7 +419,7 @@ func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(t *testing.T
|
|||||||
// clock.
|
// clock.
|
||||||
func startWithLookups(
|
func startWithLookups(
|
||||||
t *testing.T, env map[string]string,
|
t *testing.T, env map[string]string,
|
||||||
) (*sender, *servewaf.Server, *alerts.Queue) {
|
) (*sender, *proxy.Server, *alerts.Queue) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
s, _, server, queue := startWithLookupsAndClock(t, env)
|
s, _, server, queue := startWithLookupsAndClock(t, env)
|
||||||
@@ -434,7 +434,7 @@ func startWithLookups(
|
|||||||
// country, and no other address.
|
// country, and no other address.
|
||||||
func startWithLookupsAndClock(
|
func startWithLookupsAndClock(
|
||||||
t *testing.T, env map[string]string,
|
t *testing.T, env map[string]string,
|
||||||
) (*sender, *clock, *servewaf.Server, *alerts.Queue) {
|
) (*sender, *clock, *proxy.Server, *alerts.Queue) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
|
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
|
||||||
@@ -463,11 +463,11 @@ func (s *sender) uploadFrom(from string) logLine {
|
|||||||
// serveFromDE hands a question about a request from fromDE with method and body
|
// serveFromDE hands a question about a request from fromDE with method and body
|
||||||
// straight to server's handler, without the network, and returns once it is
|
// straight to server's handler, without the network, and returns once it is
|
||||||
// answered.
|
// answered.
|
||||||
func serveFromDE(t *testing.T, server *servewaf.Server, method string, body io.Reader) {
|
func serveFromDE(t *testing.T, server *proxy.Server, method string, body io.Reader) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(), method, "/", body)
|
req := httptest.NewRequestWithContext(t.Context(), method, "/", body)
|
||||||
req.Header.Set(servewaf.ClientHeader, fromDE)
|
req.Header.Set(proxy.ClientHeader, fromDE)
|
||||||
req.Header.Set(requestIDHeader, "the-id")
|
req.Header.Set(requestIDHeader, "the-id")
|
||||||
|
|
||||||
server.Handler.ServeHTTP(httptest.NewRecorder(), req)
|
server.Handler.ServeHTTP(httptest.NewRecorder(), req)
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"bufio"
|
"bufio"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"crypto/rand"
|
"crypto/rand"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"time"
|
"time"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"io"
|
"io"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"slices"
|
"slices"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -9,9 +9,9 @@ import (
|
|||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// The CrowdSec settings, and the tests' engine, which is never asked: each
|
// The CrowdSec settings, and the tests' engine, which is never asked: each
|
||||||
@@ -168,7 +168,7 @@ func TestObserveModeForwardsAClientTheCrowdSecDecisionListListsAndAlertsTheBan(
|
|||||||
// decisionsURL, answer, the engine's answer, fetched at fetched, as
|
// decisionsURL, answer, the engine's answer, fetched at fetched, as
|
||||||
// reputation.json would at start.
|
// reputation.json would at start.
|
||||||
func loadDecisions(
|
func loadDecisions(
|
||||||
t *testing.T, server *servewaf.Server, fetched time.Time, answer string,
|
t *testing.T, server *proxy.Server, fetched time.Time, answer string,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"maps"
|
"maps"
|
||||||
@@ -10,9 +10,9 @@ import (
|
|||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
const errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
|
const errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
|
||||||
@@ -51,7 +51,7 @@ func (r refused) send(s *sender, from string) logLine {
|
|||||||
// SWWAF_ERROR_BURST_THRESHOLD at threshold, and the settings in env.
|
// SWWAF_ERROR_BURST_THRESHOLD at threshold, and the settings in env.
|
||||||
func startForErrorBurst(
|
func startForErrorBurst(
|
||||||
t *testing.T, threshold string, env map[string]string,
|
t *testing.T, threshold string, env map[string]string,
|
||||||
) (*sender, *clock, *servewaf.Server) {
|
) (*sender, *clock, *proxy.Server) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
settings := map[string]string{
|
settings := map[string]string{
|
||||||
@@ -1,11 +1,11 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestHealthEndpointCountsTowardNoLimit(t *testing.T) {
|
func TestHealthEndpointCountsTowardNoLimit(t *testing.T) {
|
||||||
@@ -21,7 +21,7 @@ func TestHealthEndpointCountsTowardNoLimit(t *testing.T) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
for range healthChecks {
|
for range healthChecks {
|
||||||
got := s.adminRequest("", http.MethodGet, servewaf.HealthPath, "", http.StatusOK)
|
got := s.adminRequest("", http.MethodGet, proxy.HealthPath, "", http.StatusOK)
|
||||||
|
|
||||||
if string(got.body) != "ok\n" || got.header.Get("Content-Type") != contentType {
|
if string(got.body) != "ok\n" || got.header.Get("Content-Type") != contentType {
|
||||||
t.Errorf("health endpoint answered %q with Content-Type %q, want ok "+
|
t.Errorf("health endpoint answered %q with Content-Type %q, want ok "+
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"io"
|
"io"
|
||||||
@@ -8,9 +8,9 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestHistoryKeepsEachRequestOfTheClient(t *testing.T) {
|
func TestHistoryKeepsEachRequestOfTheClient(t *testing.T) {
|
||||||
@@ -100,8 +100,8 @@ func TestOwnEndpointsAreNotInTheHistory(t *testing.T) {
|
|||||||
|
|
||||||
s, _, server := startWithClock(t, "", map[string]string{metricsToken: token})
|
s, _, server := startWithClock(t, "", map[string]string{metricsToken: token})
|
||||||
|
|
||||||
s.adminRequest("", http.MethodGet, servewaf.HealthPath, "", http.StatusOK)
|
s.adminRequest("", http.MethodGet, proxy.HealthPath, "", http.StatusOK)
|
||||||
s.adminRequest("", http.MethodGet, servewaf.MetricsPath, "", http.StatusUnauthorized)
|
s.adminRequest("", http.MethodGet, proxy.MetricsPath, "", http.StatusUnauthorized)
|
||||||
s.scrape()
|
s.scrape()
|
||||||
|
|
||||||
if clients := server.Limiter.Snapshot(); len(clients) != 0 {
|
if clients := server.Limiter.Snapshot(); len(clients) != 0 {
|
||||||
@@ -126,7 +126,7 @@ func TestPathUnderTheOwnPrefixIsNeitherForwardedNorRefused(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// historyOf returns the history of the client at addr.
|
// historyOf returns the history of the client at addr.
|
||||||
func historyOf(t *testing.T, server *servewaf.Server, addr string) ratelimit.History {
|
func historyOf(t *testing.T, server *proxy.Server, addr string) ratelimit.History {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
client := netip.MustParsePrefix(addr + "/32")
|
client := netip.MustParsePrefix(addr + "/32")
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -379,7 +379,7 @@ func TestRequestWaitsAsLongAsTheLookupTimeoutSays(t *testing.T) {
|
|||||||
// clock of the test's own: the wait lasts exactly as long as it should,
|
// clock of the test's own: the wait lasts exactly as long as it should,
|
||||||
// however slowly the test process runs. Nothing in it may wait on the
|
// however slowly the test process runs. Nothing in it may wait on the
|
||||||
// network, which would keep that clock from moving on: the request is
|
// network, which would keep that clock from moving on: the request is
|
||||||
// handed to servewaf's handler, and GeoJS is one that never answers.
|
// handed to the proxy's handler, and GeoJS is one that never answers.
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
// Not the default. The exclusive list needs the answer, and the app
|
// Not the default. The exclusive list needs the answer, and the app
|
||||||
// is never reached.
|
// is never reached.
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"io"
|
"io"
|
||||||
@@ -12,8 +12,8 @@ import (
|
|||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -32,7 +32,7 @@ func TestMetricsAreOffWhileTheTokenIsUnset(t *testing.T) {
|
|||||||
|
|
||||||
// An empty token does not match the unset one either.
|
// An empty token does not match the unset one either.
|
||||||
for _, authorization := range []string{bearer, "Bearer ", ""} {
|
for _, authorization := range []string{bearer, "Bearer ", ""} {
|
||||||
s.adminRequest(authorization, http.MethodGet, servewaf.MetricsPath, "",
|
s.adminRequest(authorization, http.MethodGet, proxy.MetricsPath, "",
|
||||||
http.StatusNotFound)
|
http.StatusNotFound)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -46,18 +46,18 @@ func TestMetricsNeedTheTokenAndOtherPathsAreNotFound(t *testing.T) {
|
|||||||
method, path, authorization string
|
method, path, authorization string
|
||||||
status int
|
status int
|
||||||
}{
|
}{
|
||||||
{http.MethodGet, servewaf.MetricsPath, "", http.StatusUnauthorized},
|
{http.MethodGet, proxy.MetricsPath, "", http.StatusUnauthorized},
|
||||||
{
|
{
|
||||||
http.MethodGet, servewaf.MetricsPath, "Bearer " + strings.ToUpper(token),
|
http.MethodGet, proxy.MetricsPath, "Bearer " + strings.ToUpper(token),
|
||||||
http.StatusUnauthorized,
|
http.StatusUnauthorized,
|
||||||
},
|
},
|
||||||
{http.MethodGet, servewaf.MetricsPath, "Basic " + token, http.StatusUnauthorized},
|
{http.MethodGet, proxy.MetricsPath, "Basic " + token, http.StatusUnauthorized},
|
||||||
{http.MethodGet, servewaf.MetricsPath, bearer, http.StatusOK},
|
{http.MethodGet, proxy.MetricsPath, bearer, http.StatusOK},
|
||||||
{http.MethodGet, servewaf.MetricsPath, "bearer " + token, http.StatusOK},
|
{http.MethodGet, proxy.MetricsPath, "bearer " + token, http.StatusOK},
|
||||||
{http.MethodPost, servewaf.MetricsPath, bearer, http.StatusNotFound},
|
{http.MethodPost, proxy.MetricsPath, bearer, http.StatusNotFound},
|
||||||
{http.MethodGet, servewaf.MetricsPath + "/", bearer, http.StatusNotFound},
|
{http.MethodGet, proxy.MetricsPath + "/", bearer, http.StatusNotFound},
|
||||||
{http.MethodGet, "/_smallwebwaf/bans", bearer, http.StatusNotFound},
|
{http.MethodGet, "/_smallwebwaf/bans", bearer, http.StatusNotFound},
|
||||||
{http.MethodPost, servewaf.HealthPath, "", http.StatusNotFound},
|
{http.MethodPost, proxy.HealthPath, "", http.StatusNotFound},
|
||||||
} {
|
} {
|
||||||
got := s.adminRequest(tc.authorization, tc.method, tc.path, "", tc.status)
|
got := s.adminRequest(tc.authorization, tc.method, tc.path, "", tc.status)
|
||||||
|
|
||||||
@@ -180,7 +180,7 @@ func TestMetricsCountTheBansAnAdminMakes(t *testing.T) {
|
|||||||
wantMetric(t, s.scrape(), admins, 1)
|
wantMetric(t, s.scrape(), admins, 1)
|
||||||
|
|
||||||
// And a ban made through the endpoint.
|
// And a ban made through the endpoint.
|
||||||
s.admin(http.MethodPost, servewaf.BansPath, banOtherClient, http.StatusOK)
|
s.admin(http.MethodPost, proxy.BansPath, banOtherClient, http.StatusOK)
|
||||||
wantMetric(t, s.scrape(), admins, 2)
|
wantMetric(t, s.scrape(), admins, 2)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -365,7 +365,7 @@ func keptAnswer(addr, country string) lookup.Answer {
|
|||||||
func scrape(t *testing.T, addr string) string {
|
func scrape(t *testing.T, addr string) string {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
req := newRequest(t, http.MethodGet, addr, servewaf.MetricsPath, http.NoBody)
|
req := newRequest(t, http.MethodGet, addr, proxy.MetricsPath, http.NoBody)
|
||||||
req.Header.Set("Authorization", bearer)
|
req.Header.Set("Authorization", bearer)
|
||||||
|
|
||||||
got := do(t, req)
|
got := do(t, req)
|
||||||
@@ -380,7 +380,7 @@ func scrape(t *testing.T, addr string) string {
|
|||||||
func (s *sender) scrape() string {
|
func (s *sender) scrape() string {
|
||||||
s.t.Helper()
|
s.t.Helper()
|
||||||
|
|
||||||
return string(s.adminRequest(bearer, http.MethodGet, servewaf.MetricsPath, "",
|
return string(s.adminRequest(bearer, http.MethodGet, proxy.MetricsPath, "",
|
||||||
http.StatusOK).body)
|
http.StatusOK).body)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -1,10 +1,12 @@
|
|||||||
// Package servewaf is the server the servewaf command runs. It answers rproxy's
|
// Package proxy is servewaf's server. It runs the checks on the request each of
|
||||||
// questions, takes rproxy's reports and answers its own endpoints, and writes a
|
// rproxy's questions carries, and answers rproxy what to do with that request. It
|
||||||
// log line for each.
|
// takes rproxy's report once the visitor's answer has ended, which counts the
|
||||||
package servewaf
|
// request's bytes and writes its last log line. It answers servewaf's own
|
||||||
|
// endpoints: the health check, the metrics, the bans and the clients. It passes
|
||||||
|
// nothing to an app: internal/rproxy does that.
|
||||||
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
|
||||||
"io"
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -44,7 +46,7 @@ const BansPath = "/_smallwebwaf/bans"
|
|||||||
// client's address after it, with SWWAF_ADMIN_TOKEN.
|
// client's address after it, with SWWAF_ADMIN_TOKEN.
|
||||||
const ClientsPath = "/_smallwebwaf/clients/"
|
const ClientsPath = "/_smallwebwaf/clients/"
|
||||||
|
|
||||||
// Params are what New needs.
|
// Params are what NewServeWAF needs.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
Config *config.Config
|
Config *config.Config
|
||||||
// RequestLog receives two JSON lines per question, one at its decision and
|
// RequestLog receives two JSON lines per question, one at its decision and
|
||||||
@@ -61,9 +63,6 @@ type Params struct {
|
|||||||
// LookupFile is the lookup database they are looked up in while
|
// LookupFile is the lookup database they are looked up in while
|
||||||
// SWWAF_LOOKUP_SOURCE is file, and nil otherwise.
|
// SWWAF_LOOKUP_SOURCE is file, and nil otherwise.
|
||||||
LookupFile *lookup.File
|
LookupFile *lookup.File
|
||||||
// CoreRuleSet is what NewCoreRuleSet returns: what inspects requests
|
|
||||||
// while SWWAF_WAF_MODE is not off, and nil while it is.
|
|
||||||
CoreRuleSet *waf.CoreRuleSet
|
|
||||||
// Now tells the time by which requests are counted for the rate
|
// Now tells the time by which requests are counted for the rate
|
||||||
// limits, bans are made and run out, and GeoJS's answers are kept,
|
// limits, bans are made and run out, and GeoJS's answers are kept,
|
||||||
// normally time.Now in UTC, the time the state files give.
|
// normally time.Now in UTC, the time the state files give.
|
||||||
@@ -148,7 +147,7 @@ func newHandler(params Params) *handler {
|
|||||||
dnsbl: dnsbl,
|
dnsbl: dnsbl,
|
||||||
abuseIPDB: abuseIPDB,
|
abuseIPDB: abuseIPDB,
|
||||||
rules: params.Rules,
|
rules: params.Rules,
|
||||||
coreRuleSet: params.CoreRuleSet,
|
coreRuleSet: newCoreRuleSet(params.Config),
|
||||||
alerts: params.Alerts,
|
alerts: params.Alerts,
|
||||||
}
|
}
|
||||||
h.geojs = lookup.New(lookup.Params{
|
h.geojs = lookup.New(lookup.Params{
|
||||||
@@ -206,35 +205,29 @@ func newReputation(
|
|||||||
return lists, dnsbl, abuseIPDB
|
return lists, dnsbl, abuseIPDB
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL,
|
// newCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL,
|
||||||
// without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies
|
// without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies
|
||||||
// up to SWWAF_WAF_BODY_LIMIT, with the exclusions in
|
// up to SWWAF_WAF_BODY_LIMIT, or nil while SWWAF_WAF_MODE is off. It reads at
|
||||||
// SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, or nil while SWWAF_WAF_MODE is off. It
|
// most QuestionBodyMaxBytes less one byte: a question's body of
|
||||||
// reads at most QuestionBodyMaxBytes less one byte: a question's body of
|
|
||||||
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
|
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
|
||||||
// JSON or XML body over its limit. An exclusions file that cannot be read, does
|
// JSON or XML body over its limit.
|
||||||
// not parse or names an id of smallwebwaf's own rules is an error in every
|
func newCoreRuleSet(cfg *config.Config) *waf.CoreRuleSet {
|
||||||
// mode. While the mode is off, a set file is loaded with the Core Rule Set only
|
if cfg.WAFMode == config.WAFModeOff {
|
||||||
// to check it.
|
return nil
|
||||||
func NewCoreRuleSet(cfg *config.Config) (*waf.CoreRuleSet, error) {
|
|
||||||
if cfg.WAFMode == config.WAFModeOff && cfg.CoreRuleSetExclusionsFile == "" {
|
|
||||||
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
|
|
||||||
}
|
}
|
||||||
|
|
||||||
coreRuleSet, err := waf.New(waf.Params{
|
coreRuleSet, err := waf.New(waf.Params{
|
||||||
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
|
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
|
||||||
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
|
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
|
||||||
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: %w", err)
|
// The Core Rule Set is built in, and the settings cannot break it:
|
||||||
|
// the paranoia level is from 1 to 4, and the id of no rule switches
|
||||||
|
// nothing off.
|
||||||
|
panic(err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.WAFMode == config.WAFModeOff {
|
return coreRuleSet
|
||||||
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
|
|
||||||
}
|
|
||||||
|
|
||||||
return coreRuleSet, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// handler is servewaf's handler. It holds what every request shares; what
|
// handler is servewaf's handler. It holds what every request shares; what
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"bufio"
|
"bufio"
|
||||||
@@ -18,10 +18,10 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -300,7 +300,7 @@ func startProxyWithGeoJS(
|
|||||||
func startProxyWithClock(
|
func startProxyWithClock(
|
||||||
t *testing.T, appURL, geojsURL string, now func() time.Time,
|
t *testing.T, appURL, geojsURL string, now func() time.Time,
|
||||||
env map[string]string,
|
env map[string]string,
|
||||||
) (string, *output, *servewaf.Server) {
|
) (string, *output, *proxy.Server) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
addr, out, server, _ := startProxyWithAlerts(t, appURL, geojsURL, now, env)
|
addr, out, server, _ := startProxyWithAlerts(t, appURL, geojsURL, now, env)
|
||||||
@@ -313,7 +313,7 @@ func startProxyWithClock(
|
|||||||
func startProxyWithAlerts(
|
func startProxyWithAlerts(
|
||||||
t *testing.T, appURL, geojsURL string, now func() time.Time,
|
t *testing.T, appURL, geojsURL string, now func() time.Time,
|
||||||
env map[string]string,
|
env map[string]string,
|
||||||
) (string, *output, *servewaf.Server, *alerts.Queue) {
|
) (string, *output, *proxy.Server, *alerts.Queue) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
server, out, alertQueue := newServeWAF(t, geojsURL, now, env)
|
server, out, alertQueue := newServeWAF(t, geojsURL, now, env)
|
||||||
@@ -324,7 +324,7 @@ func startProxyWithAlerts(
|
|||||||
|
|
||||||
// serve serves server on a port of its own until the test ends, and sets its
|
// serve serves server on a port of its own until the test ends, and sets its
|
||||||
// Addr to where it listens.
|
// Addr to where it listens.
|
||||||
func serve(t *testing.T, server *servewaf.Server) {
|
func serve(t *testing.T, server *proxy.Server) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
|
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
|
||||||
@@ -401,17 +401,17 @@ func startRProxy(
|
|||||||
// inspected by it.
|
// inspected by it.
|
||||||
func newServeWAF(
|
func newServeWAF(
|
||||||
t *testing.T, geojsURL string, now func() time.Time, env map[string]string,
|
t *testing.T, geojsURL string, now func() time.Time, env map[string]string,
|
||||||
) (*servewaf.Server, *output, *alerts.Queue) {
|
) (*proxy.Server, *output, *alerts.Queue) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
return newServeWAFWith(t, servewaf.New, geojsURL, now, env)
|
return newServeWAFWith(t, proxy.NewServeWAF, geojsURL, now, env)
|
||||||
}
|
}
|
||||||
|
|
||||||
// newServeWAFWith is newServeWAF for the server build makes.
|
// newServeWAFWith is newServeWAF for the server build makes.
|
||||||
func newServeWAFWith(
|
func newServeWAFWith(
|
||||||
t *testing.T, build func(servewaf.Params) *servewaf.Server,
|
t *testing.T, build func(proxy.Params) *proxy.Server,
|
||||||
geojsURL string, now func() time.Time, env map[string]string,
|
geojsURL string, now func() time.Time, env map[string]string,
|
||||||
) (*servewaf.Server, *output, *alerts.Queue) {
|
) (*proxy.Server, *output, *alerts.Queue) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
settings := map[string]string{
|
settings := map[string]string{
|
||||||
@@ -463,19 +463,13 @@ func newServeWAFWith(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
coreRuleSet, err := servewaf.NewCoreRuleSet(cfg)
|
server := build(proxy.Params{
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Core Rule Set: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
server := build(servewaf.Params{
|
|
||||||
Config: cfg,
|
Config: cfg,
|
||||||
RequestLog: out,
|
RequestLog: out,
|
||||||
ProcessLog: processLog,
|
ProcessLog: processLog,
|
||||||
GeoJSURL: geojsURL,
|
GeoJSURL: geojsURL,
|
||||||
AbuseIPDBURL: abuseIPDBURL,
|
AbuseIPDBURL: abuseIPDBURL,
|
||||||
LookupFile: lookupFile,
|
LookupFile: lookupFile,
|
||||||
CoreRuleSet: coreRuleSet,
|
|
||||||
Now: now,
|
Now: now,
|
||||||
Rules: ruleFiles,
|
Rules: ruleFiles,
|
||||||
Alerts: alertQueue,
|
Alerts: alertQueue,
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
"fmt"
|
||||||
@@ -15,10 +15,10 @@ import (
|
|||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// The reputation settings.
|
// The reputation settings.
|
||||||
@@ -617,7 +617,7 @@ const (
|
|||||||
// AbuseIPDB.
|
// AbuseIPDB.
|
||||||
const abuseipdb = reputation.AbuseIPDBSource
|
const abuseipdb = reputation.AbuseIPDBSource
|
||||||
|
|
||||||
// abuseIPDBURL is where newServeWAF has clients checked with AbuseIPDB: at
|
// abuseIPDBURL is where newProxy has clients checked with AbuseIPDB: at
|
||||||
// abuseIPDBStandIn, which TestMain registers with Go's default transport,
|
// abuseIPDBStandIn, which TestMain registers with Go's default transport,
|
||||||
// through which AbuseIPDB is asked.
|
// through which AbuseIPDB is asked.
|
||||||
const abuseIPDBURL = "abuseipdb://stand-in/api/v2/check"
|
const abuseIPDBURL = "abuseipdb://stand-in/api/v2/check"
|
||||||
@@ -975,7 +975,7 @@ func verdictsFetched() time.Time {
|
|||||||
// verdictsFetched, as reputation.json would at start: one that lists the
|
// verdictsFetched, as reputation.json would at start: one that lists the
|
||||||
// client from each zone listedBy gives for it, and one that does not from
|
// client from each zone listedBy gives for it, and one that does not from
|
||||||
// each other zone.
|
// each other zone.
|
||||||
func loadVerdicts(server *servewaf.Server, listedBy map[string][]string) {
|
func loadVerdicts(server *proxy.Server, listedBy map[string][]string) {
|
||||||
verdicts := make([]reputation.Verdict, 0, len(listedBy)*len(server.DNSBL.Zones()))
|
verdicts := make([]reputation.Verdict, 0, len(listedBy)*len(server.DNSBL.Zones()))
|
||||||
|
|
||||||
for client, zones := range listedBy {
|
for client, zones := range listedBy {
|
||||||
@@ -993,7 +993,7 @@ func loadVerdicts(server *servewaf.Server, listedBy map[string][]string) {
|
|||||||
// loadScores puts into server's AbuseIPDB the score scores gives each
|
// loadScores puts into server's AbuseIPDB the score scores gives each
|
||||||
// client, an IPv4 address, fetched at verdictsFetched, as reputation.json
|
// client, an IPv4 address, fetched at verdictsFetched, as reputation.json
|
||||||
// would at start.
|
// would at start.
|
||||||
func loadScores(server *servewaf.Server, scores map[string]int64) {
|
func loadScores(server *proxy.Server, scores map[string]int64) {
|
||||||
kept := make([]reputation.Score, 0, len(scores))
|
kept := make([]reputation.Score, 0, len(scores))
|
||||||
for client, score := range scores {
|
for client, score := range scores {
|
||||||
kept = append(kept, reputation.Score{
|
kept = append(kept, reputation.Score{
|
||||||
@@ -1007,7 +1007,7 @@ func loadScores(server *servewaf.Server, scores map[string]int64) {
|
|||||||
|
|
||||||
// wantAbuseIPDBChecks checks how many clients server has checked with
|
// wantAbuseIPDBChecks checks how many clients server has checked with
|
||||||
// AbuseIPDB.
|
// AbuseIPDB.
|
||||||
func wantAbuseIPDBChecks(t *testing.T, server *servewaf.Server, want int) {
|
func wantAbuseIPDBChecks(t *testing.T, server *proxy.Server, want int) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
if got := server.AbuseIPDB.Checked(); got != want {
|
if got := server.AbuseIPDB.Checked(); got != want {
|
||||||
@@ -1017,7 +1017,7 @@ func wantAbuseIPDBChecks(t *testing.T, server *servewaf.Server, want int) {
|
|||||||
|
|
||||||
// loadLists puts copies of lists into server's lists, by URL, each with
|
// loadLists puts copies of lists into server's lists, by URL, each with
|
||||||
// its lines, fetched at listsFetched, as reputation.json would at start.
|
// its lines, fetched at listsFetched, as reputation.json would at start.
|
||||||
func loadLists(t *testing.T, server *servewaf.Server, copies map[string][]string) {
|
func loadLists(t *testing.T, server *proxy.Server, copies map[string][]string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
lists := make([]reputation.List, 0, len(copies))
|
lists := make([]reputation.List, 0, len(copies))
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"io"
|
"io"
|
||||||
@@ -10,9 +10,9 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -220,7 +220,7 @@ func TestTimingsAreInOrder(t *testing.T) {
|
|||||||
wantStatus(t, do(t, req), http.StatusForbidden)
|
wantStatus(t, do(t, req), http.StatusForbidden)
|
||||||
refused := out.endedLines(t, 2)[1]
|
refused := out.endedLines(t, 2)[1]
|
||||||
|
|
||||||
wantStatus(t, get(t, server.Addr, servewaf.HealthPath), http.StatusOK)
|
wantStatus(t, get(t, server.Addr, proxy.HealthPath), http.StatusOK)
|
||||||
health := out.endedLines(t, 3)[2]
|
health := out.endedLines(t, 3)[2]
|
||||||
|
|
||||||
// A request passed to the app has every timing; one refused, none of
|
// A request passed to the app has every timing; one refused, none of
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"slices"
|
"slices"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf
|
package proxy
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
@@ -103,13 +103,13 @@ type Report struct {
|
|||||||
DurationUpstreamTotal float64 `json:"duration_upstream_total"`
|
DurationUpstreamTotal float64 `json:"duration_upstream_total"`
|
||||||
}
|
}
|
||||||
|
|
||||||
// New returns the server `smallwebwaf servewaf` runs. It answers
|
// NewServeWAF returns the server `smallwebwaf servewaf` runs. It answers
|
||||||
// rproxy's questions, takes rproxy's reports and answers its own endpoints. It
|
// rproxy's questions, takes rproxy's reports and answers its own endpoints. It
|
||||||
// passes nothing to an app. rproxy applies the size and time limits to the
|
// passes nothing to an app. rproxy applies the size and time limits to the
|
||||||
// visitor. Go's default limit on a request's line and headers, 1 MiB, stays: it
|
// visitor. Go's default limit on a request's line and headers, 1 MiB, stays: it
|
||||||
// is far above rproxy's, and a question refused for its size would have rproxy
|
// is far above rproxy's, and a question refused for its size would have rproxy
|
||||||
// let the request through.
|
// let the request through.
|
||||||
func New(params Params) *Server {
|
func NewServeWAF(params Params) *Server {
|
||||||
h := newHandler(params)
|
h := newHandler(params)
|
||||||
|
|
||||||
return &Server{
|
return &Server{
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -19,9 +19,9 @@ import (
|
|||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -57,7 +57,7 @@ type serveWAF struct {
|
|||||||
t *testing.T
|
t *testing.T
|
||||||
addr string
|
addr string
|
||||||
out *output
|
out *output
|
||||||
server *servewaf.Server
|
server *proxy.Server
|
||||||
clock *clock
|
clock *clock
|
||||||
queue *alerts.Queue
|
queue *alerts.Queue
|
||||||
}
|
}
|
||||||
@@ -67,13 +67,13 @@ type serveWAF struct {
|
|||||||
func startServeWAF(t *testing.T, geojsURL string, env map[string]string) *serveWAF {
|
func startServeWAF(t *testing.T, geojsURL string, env map[string]string) *serveWAF {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
return startServeWAFWith(t, geojsURL, env, func(*servewaf.Server) {})
|
return startServeWAFWith(t, geojsURL, env, func(*proxy.Server) {})
|
||||||
}
|
}
|
||||||
|
|
||||||
// startServeWAFWith is startServeWAF with change applied to the server before
|
// startServeWAFWith is startServeWAF with change applied to the server before
|
||||||
// it starts.
|
// it starts.
|
||||||
func startServeWAFWith(
|
func startServeWAFWith(
|
||||||
t *testing.T, geojsURL string, env map[string]string, change func(*servewaf.Server),
|
t *testing.T, geojsURL string, env map[string]string, change func(*proxy.Server),
|
||||||
) *serveWAF {
|
) *serveWAF {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -103,8 +103,8 @@ func (s *serveWAF) questionWith(
|
|||||||
|
|
||||||
req := newRequest(s.t, method, s.addr, path, body)
|
req := newRequest(s.t, method, s.addr, path, body)
|
||||||
req.Host = host
|
req.Host = host
|
||||||
req.Header.Set(servewaf.ClientHeader, from)
|
req.Header.Set(proxy.ClientHeader, from)
|
||||||
req.Header.Set(servewaf.PeerHeader, rproxyPeer)
|
req.Header.Set(proxy.PeerHeader, rproxyPeer)
|
||||||
req.Header.Set(requestIDHeader, rand.Text())
|
req.Header.Set(requestIDHeader, rand.Text())
|
||||||
|
|
||||||
return req
|
return req
|
||||||
@@ -131,13 +131,13 @@ func (s *serveWAF) allowed(from string) string {
|
|||||||
s.t.Helper()
|
s.t.Helper()
|
||||||
|
|
||||||
req := s.question(siteA, from, "/")
|
req := s.question(siteA, from, "/")
|
||||||
wantDisposition(s.t, do(s.t, req), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(s.t, do(s.t, req), http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
return req.Header.Get(requestIDHeader)
|
return req.Header.Get(requestIDHeader)
|
||||||
}
|
}
|
||||||
|
|
||||||
// report sends rproxy's report and returns the answer.
|
// report sends rproxy's report and returns the answer.
|
||||||
func (s *serveWAF) report(report servewaf.Report) answer {
|
func (s *serveWAF) report(report proxy.Report) answer {
|
||||||
s.t.Helper()
|
s.t.Helper()
|
||||||
|
|
||||||
body, err := json.Marshal(report)
|
body, err := json.Marshal(report)
|
||||||
@@ -152,7 +152,7 @@ func (s *serveWAF) report(report servewaf.Report) answer {
|
|||||||
func (s *serveWAF) reportBody(body string) answer {
|
func (s *serveWAF) reportBody(body string) answer {
|
||||||
s.t.Helper()
|
s.t.Helper()
|
||||||
|
|
||||||
return do(s.t, newRequest(s.t, http.MethodPost, s.addr, servewaf.ReportPath,
|
return do(s.t, newRequest(s.t, http.MethodPost, s.addr, proxy.ReportPath,
|
||||||
strings.NewReader(body)))
|
strings.NewReader(body)))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -161,7 +161,7 @@ func (s *serveWAF) reportBody(body string) answer {
|
|||||||
func (s *serveWAF) passedOn(from string) {
|
func (s *serveWAF) passedOn(from string) {
|
||||||
s.t.Helper()
|
s.t.Helper()
|
||||||
|
|
||||||
got := s.report(servewaf.Report{
|
got := s.report(proxy.Report{
|
||||||
RequestID: s.allowed(from),
|
RequestID: s.allowed(from),
|
||||||
Status: http.StatusOK,
|
Status: http.StatusOK,
|
||||||
RequestBytes: bodyBytes,
|
RequestBytes: bodyBytes,
|
||||||
@@ -182,7 +182,7 @@ func (s *serveWAF) history() ratelimit.History {
|
|||||||
func (s *serveWAF) metrics() string {
|
func (s *serveWAF) metrics() string {
|
||||||
s.t.Helper()
|
s.t.Helper()
|
||||||
|
|
||||||
req := newRequest(s.t, http.MethodGet, s.addr, servewaf.MetricsPath, http.NoBody)
|
req := newRequest(s.t, http.MethodGet, s.addr, proxy.MetricsPath, http.NoBody)
|
||||||
req.Header.Set("Authorization", bearer)
|
req.Header.Set("Authorization", bearer)
|
||||||
|
|
||||||
got := do(s.t, req)
|
got := do(s.t, req)
|
||||||
@@ -196,7 +196,7 @@ func wantDisposition(t *testing.T, got answer, status int, disposition string) {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
wantStatus(t, got, status)
|
wantStatus(t, got, status)
|
||||||
|
|
||||||
if answered := got.header.Get(servewaf.DispositionHeader); answered != disposition {
|
if answered := got.header.Get(proxy.DispositionHeader); answered != disposition {
|
||||||
t.Errorf("answered with disposition %q, want %q", answered, disposition)
|
t.Errorf("answered with disposition %q, want %q", answered, disposition)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -248,10 +248,10 @@ func TestQuestionAllowedIsAnsweredAndLogged(t *testing.T) {
|
|||||||
req := s.question(siteA, client, "/page?q=1&token=visible")
|
req := s.question(siteA, client, "/page?q=1&token=visible")
|
||||||
req.Header.Set(requestIDHeader, "the-id")
|
req.Header.Set(requestIDHeader, "the-id")
|
||||||
req.Header.Set("X-Forwarded-Proto", "https")
|
req.Header.Set("X-Forwarded-Proto", "https")
|
||||||
req.Header.Set(servewaf.ContentLengthHeader, "1234")
|
req.Header.Set(proxy.ContentLengthHeader, "1234")
|
||||||
|
|
||||||
got := do(t, req)
|
got := do(t, req)
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
|
||||||
wantBody(t, got, "")
|
wantBody(t, got, "")
|
||||||
|
|
||||||
if id := got.header.Get(requestIDHeader); id != "" {
|
if id := got.header.Get(requestIDHeader); id != "" {
|
||||||
@@ -278,7 +278,7 @@ func TestIPv6VisitorIsCountedInItsGroup(t *testing.T) {
|
|||||||
s.allowed("2001:db8::1")
|
s.allowed("2001:db8::1")
|
||||||
|
|
||||||
got := s.ask("2001:db8::2", "/")
|
got := s.ask("2001:db8::2", "/")
|
||||||
wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestIPv4VisitorInIPv6FormIsTheIPv4Client(t *testing.T) {
|
func TestIPv4VisitorInIPv6FormIsTheIPv4Client(t *testing.T) {
|
||||||
@@ -287,9 +287,9 @@ func TestIPv4VisitorInIPv6FormIsTheIPv4Client(t *testing.T) {
|
|||||||
s := startServeWAF(t, "", map[string]string{denyNets: client})
|
s := startServeWAF(t, "", map[string]string{denyNets: client})
|
||||||
|
|
||||||
req := s.question(siteA, "::ffff:"+client, "/")
|
req := s.question(siteA, "::ffff:"+client, "/")
|
||||||
req.Header.Set(servewaf.PeerHeader, "::ffff:"+rproxyPeer)
|
req.Header.Set(proxy.PeerHeader, "::ffff:"+rproxyPeer)
|
||||||
|
|
||||||
wantDisposition(t, do(t, req), http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, do(t, req), http.StatusForbidden, proxy.DispositionDeny)
|
||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
if line.ClientIP != client || line.PeerIP != rproxyPeer {
|
if line.ClientIP != client || line.PeerIP != rproxyPeer {
|
||||||
@@ -305,13 +305,13 @@ func TestMalformedQuestionGets400AndAWarningAndNoLine(t *testing.T) {
|
|||||||
change func(*http.Request)
|
change func(*http.Request)
|
||||||
}{
|
}{
|
||||||
{"client not an address", func(req *http.Request) {
|
{"client not an address", func(req *http.Request) {
|
||||||
req.Header.Set(servewaf.ClientHeader, "not an address")
|
req.Header.Set(proxy.ClientHeader, "not an address")
|
||||||
}},
|
}},
|
||||||
{"client empty", func(req *http.Request) {
|
{"client empty", func(req *http.Request) {
|
||||||
req.Header.Set(servewaf.ClientHeader, "")
|
req.Header.Set(proxy.ClientHeader, "")
|
||||||
}},
|
}},
|
||||||
{"client address with a zone", func(req *http.Request) {
|
{"client address with a zone", func(req *http.Request) {
|
||||||
req.Header.Set(servewaf.ClientHeader, "2001:db8::1%eth0")
|
req.Header.Set(proxy.ClientHeader, "2001:db8::1%eth0")
|
||||||
}},
|
}},
|
||||||
{"id absent", func(req *http.Request) {
|
{"id absent", func(req *http.Request) {
|
||||||
req.Header.Del(requestIDHeader)
|
req.Header.Del(requestIDHeader)
|
||||||
@@ -353,7 +353,7 @@ func TestLongestValidIDIsTaken(t *testing.T) {
|
|||||||
|
|
||||||
req := s.question(siteA, client, "/")
|
req := s.question(siteA, client, "/")
|
||||||
req.Header.Set(requestIDHeader, strings.Repeat("x", 128))
|
req.Header.Set(requestIDHeader, strings.Repeat("x", 128))
|
||||||
wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestQuestionWithABodyOver64KiBGets400(t *testing.T) {
|
func TestQuestionWithABodyOver64KiBGets400(t *testing.T) {
|
||||||
@@ -361,15 +361,15 @@ func TestQuestionWithABodyOver64KiBGets400(t *testing.T) {
|
|||||||
|
|
||||||
s := startServeWAF(t, "", nil)
|
s := startServeWAF(t, "", nil)
|
||||||
|
|
||||||
over := strings.NewReader(strings.Repeat("x", servewaf.QuestionBodyMaxBytes+1))
|
over := strings.NewReader(strings.Repeat("x", proxy.QuestionBodyMaxBytes+1))
|
||||||
got := do(t, s.questionWith(http.MethodPost, siteA, client, "/", over))
|
got := do(t, s.questionWith(http.MethodPost, siteA, client, "/", over))
|
||||||
wantDisposition(t, got, http.StatusBadRequest, "")
|
wantDisposition(t, got, http.StatusBadRequest, "")
|
||||||
wantProcessMessages(t, s, "a question is refused", 1)
|
wantProcessMessages(t, s, "a question is refused", 1)
|
||||||
wantNoRequestLines(t, s)
|
wantNoRequestLines(t, s)
|
||||||
|
|
||||||
whole := strings.NewReader(strings.Repeat("x", servewaf.QuestionBodyMaxBytes))
|
whole := strings.NewReader(strings.Repeat("x", proxy.QuestionBodyMaxBytes))
|
||||||
got = do(t, s.questionWith(http.MethodPost, siteA, client, "/", whole))
|
got = do(t, s.questionWith(http.MethodPost, siteA, client, "/", whole))
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestQuestionWithAHeldIDIsRefused(t *testing.T) {
|
func TestQuestionWithAHeldIDIsRefused(t *testing.T) {
|
||||||
@@ -385,15 +385,15 @@ func TestQuestionWithAHeldIDIsRefused(t *testing.T) {
|
|||||||
wantProcessMessages(t, s, "a question is refused", 1)
|
wantProcessMessages(t, s, "a question is refused", 1)
|
||||||
|
|
||||||
// A denied request's id is held as well.
|
// A denied request's id is held as well.
|
||||||
denied := s.question(siteA, otherClient, servewaf.HealthPath)
|
denied := s.question(siteA, otherClient, proxy.HealthPath)
|
||||||
wantDisposition(t, do(t, denied), http.StatusNotFound, servewaf.DispositionDeny)
|
wantDisposition(t, do(t, denied), http.StatusNotFound, proxy.DispositionDeny)
|
||||||
|
|
||||||
again = s.question(siteA, client, "/")
|
again = s.question(siteA, client, "/")
|
||||||
again.Header.Set(requestIDHeader, denied.Header.Get(requestIDHeader))
|
again.Header.Set(requestIDHeader, denied.Header.Get(requestIDHeader))
|
||||||
wantDisposition(t, do(t, again), http.StatusBadRequest, "")
|
wantDisposition(t, do(t, again), http.StatusBadRequest, "")
|
||||||
|
|
||||||
// The report ends the holding, and the refused questions changed nothing.
|
// The report ends the holding, and the refused questions changed nothing.
|
||||||
got := s.report(servewaf.Report{RequestID: id, ResponseBytes: answerBytes})
|
got := s.report(proxy.Report{RequestID: id, ResponseBytes: answerBytes})
|
||||||
wantStatus(t, got, http.StatusNoContent)
|
wantStatus(t, got, http.StatusNoContent)
|
||||||
|
|
||||||
if history := s.history(); history.ResponseBytes != answerBytes {
|
if history := s.history(); history.ResponseBytes != answerBytes {
|
||||||
@@ -402,7 +402,7 @@ func TestQuestionWithAHeldIDIsRefused(t *testing.T) {
|
|||||||
|
|
||||||
again = s.question(siteA, client, "/")
|
again = s.question(siteA, client, "/")
|
||||||
again.Header.Set(requestIDHeader, id)
|
again.Header.Set(requestIDHeader, id)
|
||||||
wantDisposition(t, do(t, again), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, do(t, again), http.StatusOK, proxy.DispositionAllow)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestReportNeverReachesAQuestionStillBeingHandled(t *testing.T) {
|
func TestReportNeverReachesAQuestionStillBeingHandled(t *testing.T) {
|
||||||
@@ -436,7 +436,7 @@ func TestReportNeverReachesAQuestionStillBeingHandled(t *testing.T) {
|
|||||||
|
|
||||||
// A report for a question not yet answered is ignored, and that question's
|
// A report for a question not yet answered is ignored, and that question's
|
||||||
// id is taken.
|
// id is taken.
|
||||||
got := s.report(servewaf.Report{RequestID: id, ResponseBytes: 1000})
|
got := s.report(proxy.Report{RequestID: id, ResponseBytes: 1000})
|
||||||
wantStatus(t, got, http.StatusNoContent)
|
wantStatus(t, got, http.StatusNoContent)
|
||||||
|
|
||||||
// This would wait on the same lookup were the id not taken.
|
// This would wait on the same lookup were the id not taken.
|
||||||
@@ -451,7 +451,7 @@ func TestReportNeverReachesAQuestionStillBeingHandled(t *testing.T) {
|
|||||||
|
|
||||||
select {
|
select {
|
||||||
case got := <-answered:
|
case got := <-answered:
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
|
||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Fatal("the question got no answer")
|
t.Fatal("the question got no answer")
|
||||||
}
|
}
|
||||||
@@ -466,7 +466,7 @@ func TestReportNeverReachesAQuestionStillBeingHandled(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// The true report comes after, and is the one that counts.
|
// The true report comes after, and is the one that counts.
|
||||||
got = s.report(servewaf.Report{RequestID: id, ResponseBytes: answerBytes})
|
got = s.report(proxy.Report{RequestID: id, ResponseBytes: answerBytes})
|
||||||
wantStatus(t, got, http.StatusNoContent)
|
wantStatus(t, got, http.StatusNoContent)
|
||||||
|
|
||||||
if history := s.history(); history.ResponseBytes != answerBytes {
|
if history := s.history(); history.ResponseBytes != answerBytes {
|
||||||
@@ -484,13 +484,13 @@ func TestPeerAbsentOrUnreadableIsAnsweredAndLoggedInvalid(t *testing.T) {
|
|||||||
s := startServeWAF(t, "", nil)
|
s := startServeWAF(t, "", nil)
|
||||||
|
|
||||||
req := s.question(siteA, client, "/")
|
req := s.question(siteA, client, "/")
|
||||||
req.Header.Del(servewaf.PeerHeader)
|
req.Header.Del(proxy.PeerHeader)
|
||||||
|
|
||||||
if peer != "" {
|
if peer != "" {
|
||||||
req.Header.Set(servewaf.PeerHeader, peer)
|
req.Header.Set(proxy.PeerHeader, peer)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
if line := s.out.requestLine(t); line.PeerIP != "invalid IP" {
|
if line := s.out.requestLine(t); line.PeerIP != "invalid IP" {
|
||||||
t.Errorf("log line has peer_ip %q, want invalid IP", line.PeerIP)
|
t.Errorf("log line has peer_ip %q, want invalid IP", line.PeerIP)
|
||||||
@@ -511,8 +511,8 @@ func TestContentLengthUnreadableOrNegativeCountsAsAbsent(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
req := s.question(siteA, client, "/")
|
req := s.question(siteA, client, "/")
|
||||||
req.Header.Set(servewaf.ContentLengthHeader, announced)
|
req.Header.Set(proxy.ContentLengthHeader, announced)
|
||||||
wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
if line := s.out.requestLine(t); line.ContentLength != 0 {
|
if line := s.out.requestLine(t); line.ContentLength != 0 {
|
||||||
t.Errorf("log line has content_length %d, want none", line.ContentLength)
|
t.Errorf("log line has content_length %d, want none", line.ContentLength)
|
||||||
@@ -527,13 +527,13 @@ func TestHeaderSentTwiceUsesItsFirstValue(t *testing.T) {
|
|||||||
s := startServeWAF(t, "", nil)
|
s := startServeWAF(t, "", nil)
|
||||||
|
|
||||||
req := s.question(siteA, client, "/")
|
req := s.question(siteA, client, "/")
|
||||||
req.Header.Add(servewaf.ClientHeader, otherClient)
|
req.Header.Add(proxy.ClientHeader, otherClient)
|
||||||
req.Header.Add(servewaf.PeerHeader, "172.18.0.3")
|
req.Header.Add(proxy.PeerHeader, "172.18.0.3")
|
||||||
req.Header.Set(requestIDHeader, "first")
|
req.Header.Set(requestIDHeader, "first")
|
||||||
req.Header.Add(requestIDHeader, "second")
|
req.Header.Add(requestIDHeader, "second")
|
||||||
req.Header.Set(servewaf.ContentLengthHeader, "1234")
|
req.Header.Set(proxy.ContentLengthHeader, "1234")
|
||||||
req.Header.Add(servewaf.ContentLengthHeader, "5678")
|
req.Header.Add(proxy.ContentLengthHeader, "5678")
|
||||||
wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
if line.ClientIP != client || line.PeerIP != rproxyPeer ||
|
if line.ClientIP != client || line.PeerIP != rproxyPeer ||
|
||||||
@@ -562,7 +562,7 @@ func sendQuestionWithPartOfItsBody(t *testing.T, conn net.Conn) {
|
|||||||
_, err := fmt.Fprintf(conn,
|
_, err := fmt.Fprintf(conn,
|
||||||
"POST / HTTP/1.1\r\nHost: %s\r\n%s: %s\r\n%s: an-id\r\n"+
|
"POST / HTTP/1.1\r\nHost: %s\r\n%s: %s\r\n%s: an-id\r\n"+
|
||||||
"Content-Length: 100\r\n\r\npart",
|
"Content-Length: 100\r\n\r\npart",
|
||||||
siteA, servewaf.ClientHeader, client, requestIDHeader)
|
siteA, proxy.ClientHeader, client, requestIDHeader)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("send the question: %v", err)
|
t.Fatalf("send the question: %v", err)
|
||||||
}
|
}
|
||||||
@@ -592,7 +592,7 @@ func TestQuestionWhoseBodyStopsArrivingGetsNoAnswer(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s := startServeWAFWith(t, "", map[string]string{metricsToken: token},
|
s := startServeWAFWith(t, "", map[string]string{metricsToken: token},
|
||||||
func(server *servewaf.Server) { server.ReadTimeout = readTimeout })
|
func(server *proxy.Server) { server.ReadTimeout = readTimeout })
|
||||||
|
|
||||||
conn := dial(t, s.addr)
|
conn := dial(t, s.addr)
|
||||||
sendQuestionWithPartOfItsBody(t, conn)
|
sendQuestionWithPartOfItsBody(t, conn)
|
||||||
@@ -615,7 +615,7 @@ func TestQuestionWhoseBodyStopsArrivingGetsNoAnswer(t *testing.T) {
|
|||||||
// The id was not taken.
|
// The id was not taken.
|
||||||
again := s.question(siteA, client, "/")
|
again := s.question(siteA, client, "/")
|
||||||
again.Header.Set(requestIDHeader, "an-id")
|
again.Header.Set(requestIDHeader, "an-id")
|
||||||
wantDisposition(t, do(t, again), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, do(t, again), http.StatusOK, proxy.DispositionAllow)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -632,7 +632,7 @@ func TestQuestionDeniedGetsTheBanResponse(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
got := s.ask(client, "/")
|
got := s.ask(client, "/")
|
||||||
wantDisposition(t, got, status, servewaf.DispositionDeny)
|
wantDisposition(t, got, status, proxy.DispositionDeny)
|
||||||
wantBody(t, got, http.StatusText(status)+"\n")
|
wantBody(t, got, http.StatusText(status)+"\n")
|
||||||
wantLine(t, s.out.requestLine(t), status, requestlog.ActionDenied)
|
wantLine(t, s.out.requestLine(t), status, requestlog.ActionDenied)
|
||||||
})
|
})
|
||||||
@@ -647,7 +647,7 @@ func TestBanResponseCloseIsAnAnswerOfItsOwn(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
got := s.ask(client, "/")
|
got := s.ask(client, "/")
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionClose)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionClose)
|
||||||
wantBody(t, got, "")
|
wantBody(t, got, "")
|
||||||
// The visitor is sent nothing.
|
// The visitor is sent nothing.
|
||||||
wantLine(t, s.out.requestLine(t), 0, requestlog.ActionDenied)
|
wantLine(t, s.out.requestLine(t), 0, requestlog.ActionDenied)
|
||||||
@@ -664,12 +664,12 @@ func TestBlockRuleIsDeniedWith403AndABanRuleWithTheBanResponse(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
got := s.ask(client, "/blocked")
|
got := s.ask(client, "/blocked")
|
||||||
wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny)
|
||||||
wantLine(t, s.out.requestLine(t), http.StatusForbidden,
|
wantLine(t, s.out.requestLine(t), http.StatusForbidden,
|
||||||
requestlog.ActionRuleBlocked)
|
requestlog.ActionRuleBlocked)
|
||||||
|
|
||||||
got = s.ask(client, "/.env")
|
got = s.ask(client, "/.env")
|
||||||
wantDisposition(t, got, http.StatusTooManyRequests, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusTooManyRequests, proxy.DispositionDeny)
|
||||||
wantLine(t, s.out.requestLines(t, 2)[1], http.StatusTooManyRequests,
|
wantLine(t, s.out.requestLines(t, 2)[1], http.StatusTooManyRequests,
|
||||||
requestlog.ActionBanned)
|
requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
@@ -681,17 +681,15 @@ func TestQuestionUnderTheOwnPrefixIsDeniedWith404AfterTheChecks(t *testing.T) {
|
|||||||
metricsToken: token, denyNets: otherClient,
|
metricsToken: token, denyNets: otherClient,
|
||||||
})
|
})
|
||||||
|
|
||||||
for _, path := range []string{
|
for _, path := range []string{proxy.HealthPath, proxy.MetricsPath, proxy.ReportPath} {
|
||||||
servewaf.HealthPath, servewaf.MetricsPath, servewaf.ReportPath,
|
|
||||||
} {
|
|
||||||
got := s.ask(client, path)
|
got := s.ask(client, path)
|
||||||
wantDisposition(t, got, http.StatusNotFound, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusNotFound, proxy.DispositionDeny)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantLine(t, s.out.requestLine(t), http.StatusNotFound, requestlog.ActionAdmin)
|
wantLine(t, s.out.requestLine(t), http.StatusNotFound, requestlog.ActionAdmin)
|
||||||
|
|
||||||
got := s.ask(otherClient, servewaf.MetricsPath)
|
got := s.ask(otherClient, proxy.MetricsPath)
|
||||||
wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestObserveModeAllowsAndLogsWhatEnforceModeWouldHaveDone(t *testing.T) {
|
func TestObserveModeAllowsAndLogsWhatEnforceModeWouldHaveDone(t *testing.T) {
|
||||||
@@ -700,7 +698,7 @@ func TestObserveModeAllowsAndLogsWhatEnforceModeWouldHaveDone(t *testing.T) {
|
|||||||
s := startServeWAF(t, "", map[string]string{mode: observe, denyNets: client})
|
s := startServeWAF(t, "", map[string]string{mode: observe, denyNets: client})
|
||||||
|
|
||||||
got := s.ask(client, "/")
|
got := s.ask(client, "/")
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
|
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
|
||||||
@@ -716,7 +714,7 @@ func TestObserveModeRefusalAnsweredAllowCountsForTheErrorBurst(t *testing.T) {
|
|||||||
|
|
||||||
for range 2 {
|
for range 2 {
|
||||||
got := s.ask(client, "/blocked")
|
got := s.ask(client, "/blocked")
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
|
||||||
}
|
}
|
||||||
|
|
||||||
lines := s.out.requestLines(t, 2)
|
lines := s.out.requestLines(t, 2)
|
||||||
@@ -735,7 +733,7 @@ func TestDeniedQuestionCountsForTheErrorBurst(t *testing.T) {
|
|||||||
|
|
||||||
for range 2 {
|
for range 2 {
|
||||||
got := s.ask(client, "/blocked")
|
got := s.ask(client, "/blocked")
|
||||||
wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny)
|
||||||
}
|
}
|
||||||
|
|
||||||
lines := s.out.requestLines(t, 2)
|
lines := s.out.requestLines(t, 2)
|
||||||
@@ -764,7 +762,7 @@ func TestAllowCarriesTheLookupHeadersWhileTheSettingIsSet(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
got := s.ask(fromDE, "/")
|
got := s.ask(fromDE, "/")
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country")
|
asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country")
|
||||||
if asn != tc.wantASN || country != tc.wantCountry {
|
if asn != tc.wantASN || country != tc.wantCountry {
|
||||||
@@ -789,7 +787,7 @@ func TestReportCountsTowardAByteLimitWhichThenBans(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// 100 bytes are over the limit of 99.
|
// 100 bytes are over the limit of 99.
|
||||||
got := s.report(servewaf.Report{
|
got := s.report(proxy.Report{
|
||||||
RequestID: id, Status: http.StatusBadGateway,
|
RequestID: id, Status: http.StatusBadGateway,
|
||||||
RequestBytes: bodyBytes, ResponseBytes: answerBytes,
|
RequestBytes: bodyBytes, ResponseBytes: answerBytes,
|
||||||
})
|
})
|
||||||
@@ -813,7 +811,7 @@ func TestReportCountsTowardAByteLimitWhichThenBans(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
got = s.ask(client, "/")
|
got = s.ask(client, "/")
|
||||||
wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestReportCountsAtThePercentageItsQuestionWasGiven(t *testing.T) {
|
func TestReportCountsAtThePercentageItsQuestionWasGiven(t *testing.T) {
|
||||||
@@ -827,10 +825,10 @@ func TestReportCountsAtThePercentageItsQuestionWasGiven(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
s.passedOn(fromKP)
|
s.passedOn(fromKP)
|
||||||
wantDisposition(t, s.ask(fromKP, "/"), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, s.ask(fromKP, "/"), http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
s.passedOn(fromDE)
|
s.passedOn(fromDE)
|
||||||
wantDisposition(t, s.ask(fromDE, "/"), http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, s.ask(fromDE, "/"), http.StatusForbidden, proxy.DispositionDeny)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestReportAddsTheRequestToItsClientsHistoryAndTheAnomalyCounters(t *testing.T) {
|
func TestReportAddsTheRequestToItsClientsHistoryAndTheAnomalyCounters(t *testing.T) {
|
||||||
@@ -848,7 +846,7 @@ func TestReportAddsTheRequestToItsClientsHistoryAndTheAnomalyCounters(t *testing
|
|||||||
t.Errorf("anomaly counters %v before the report, want none", counters)
|
t.Errorf("anomaly counters %v before the report, want none", counters)
|
||||||
}
|
}
|
||||||
|
|
||||||
got := s.report(servewaf.Report{
|
got := s.report(proxy.Report{
|
||||||
RequestID: id, Status: http.StatusBadGateway,
|
RequestID: id, Status: http.StatusBadGateway,
|
||||||
RequestBytes: bodyBytes, ResponseBytes: answerBytes,
|
RequestBytes: bodyBytes, ResponseBytes: answerBytes,
|
||||||
})
|
})
|
||||||
@@ -876,7 +874,7 @@ func TestDeniedRequestIsFinishedAtItsDecision(t *testing.T) {
|
|||||||
s.allowed(client)
|
s.allowed(client)
|
||||||
|
|
||||||
denied := s.question(siteA, client, "/")
|
denied := s.question(siteA, client, "/")
|
||||||
wantDisposition(t, do(t, denied), http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, do(t, denied), http.StatusForbidden, proxy.DispositionDeny)
|
||||||
|
|
||||||
counters := s.server.Anomalies.Snapshot()
|
counters := s.server.Anomalies.Snapshot()
|
||||||
if len(counters) != 1 || counters[0].Minute.Current != 1 {
|
if len(counters) != 1 || counters[0].Minute.Current != 1 {
|
||||||
@@ -893,7 +891,7 @@ func TestDeniedRequestIsFinishedAtItsDecision(t *testing.T) {
|
|||||||
|
|
||||||
wantMetric(t, s.metrics(), series, 1)
|
wantMetric(t, s.metrics(), series, 1)
|
||||||
|
|
||||||
got := s.report(servewaf.Report{
|
got := s.report(proxy.Report{
|
||||||
RequestID: denied.Header.Get(requestIDHeader), Status: http.StatusForbidden,
|
RequestID: denied.Header.Get(requestIDHeader), Status: http.StatusForbidden,
|
||||||
ResponseBytes: 1000, DurationTotal: 2,
|
ResponseBytes: 1000, DurationTotal: 2,
|
||||||
})
|
})
|
||||||
@@ -926,7 +924,7 @@ func TestMetricsCountAnAllowedRequestFromItsReport(t *testing.T) {
|
|||||||
wantMetric(t, metrics, `smallwebwaf_requests_in_flight{instance="`+alertInstance+`"}`,
|
wantMetric(t, metrics, `smallwebwaf_requests_in_flight{instance="`+alertInstance+`"}`,
|
||||||
1) // the scrape itself
|
1) // the scrape itself
|
||||||
|
|
||||||
got := s.report(servewaf.Report{
|
got := s.report(proxy.Report{
|
||||||
RequestID: id, Status: http.StatusBadGateway,
|
RequestID: id, Status: http.StatusBadGateway,
|
||||||
RequestBytes: bodyBytes, ResponseBytes: answerBytes,
|
RequestBytes: bodyBytes, ResponseBytes: answerBytes,
|
||||||
DurationTotal: 5, DurationUpstreamTotal: 4,
|
DurationTotal: 5, DurationUpstreamTotal: 4,
|
||||||
@@ -944,7 +942,7 @@ func TestMetricsCountAnAllowedRequestFromItsReport(t *testing.T) {
|
|||||||
// check needs a server that has not been scraped yet.
|
// check needs a server that has not been scraped yet.
|
||||||
unscraped := startServeWAF(t, "", map[string]string{metricsToken: token})
|
unscraped := startServeWAF(t, "", map[string]string{metricsToken: token})
|
||||||
|
|
||||||
got = unscraped.report(servewaf.Report{
|
got = unscraped.report(proxy.Report{
|
||||||
RequestID: unscraped.allowed(client), Status: http.StatusBadGateway, DurationTotal: 5,
|
RequestID: unscraped.allowed(client), Status: http.StatusBadGateway, DurationTotal: 5,
|
||||||
})
|
})
|
||||||
wantStatus(t, got, http.StatusNoContent)
|
wantStatus(t, got, http.StatusNoContent)
|
||||||
@@ -957,7 +955,7 @@ func TestReportForARequestNotHeldIsIgnored(t *testing.T) {
|
|||||||
|
|
||||||
s := startServeWAF(t, "", map[string]string{bytesLimitPerMinute: byteLimit})
|
s := startServeWAF(t, "", map[string]string{bytesLimitPerMinute: byteLimit})
|
||||||
|
|
||||||
got := s.report(servewaf.Report{RequestID: "no-such-request", ResponseBytes: 1000})
|
got := s.report(proxy.Report{RequestID: "no-such-request", ResponseBytes: 1000})
|
||||||
wantStatus(t, got, http.StatusNoContent)
|
wantStatus(t, got, http.StatusNoContent)
|
||||||
|
|
||||||
if clients := s.server.Limiter.Snapshot(); len(clients) != 0 {
|
if clients := s.server.Limiter.Snapshot(); len(clients) != 0 {
|
||||||
@@ -974,7 +972,7 @@ func TestReportForARequestNotHeldIsIgnored(t *testing.T) {
|
|||||||
// A second report of the same request does not count it twice.
|
// A second report of the same request does not count it twice.
|
||||||
id := s.allowed(client)
|
id := s.allowed(client)
|
||||||
for range 2 {
|
for range 2 {
|
||||||
got = s.report(servewaf.Report{RequestID: id, ResponseBytes: answerBytes})
|
got = s.report(proxy.Report{RequestID: id, ResponseBytes: answerBytes})
|
||||||
wantStatus(t, got, http.StatusNoContent)
|
wantStatus(t, got, http.StatusNoContent)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1162,7 +1160,7 @@ func TestBytesAtTheirMostEachWayBanRatherThanOverflow(t *testing.T) {
|
|||||||
|
|
||||||
s := startServeWAF(t, "", map[string]string{bytesLimitPerMinute: byteLimit})
|
s := startServeWAF(t, "", map[string]string{bytesLimitPerMinute: byteLimit})
|
||||||
|
|
||||||
got := s.report(servewaf.Report{
|
got := s.report(proxy.Report{
|
||||||
RequestID: s.allowed(client), Status: http.StatusOK,
|
RequestID: s.allowed(client), Status: http.StatusOK,
|
||||||
RequestBytes: reportedBytesLimit, ResponseBytes: reportedBytesLimit,
|
RequestBytes: reportedBytesLimit, ResponseBytes: reportedBytesLimit,
|
||||||
})
|
})
|
||||||
@@ -1247,9 +1245,9 @@ func TestReportWritesTheLastLineWithTheDecisionsAndTheReportsFields(t *testing.T
|
|||||||
req.Header.Set("X-Forwarded-Proto", "https")
|
req.Header.Set("X-Forwarded-Proto", "https")
|
||||||
id := req.Header.Get(requestIDHeader)
|
id := req.Header.Get(requestIDHeader)
|
||||||
|
|
||||||
wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
got := s.report(servewaf.Report{
|
got := s.report(proxy.Report{
|
||||||
RequestID: id, Status: http.StatusCreated,
|
RequestID: id, Status: http.StatusCreated,
|
||||||
RequestBytes: bodyBytes, ResponseBytes: answerBytes,
|
RequestBytes: bodyBytes, ResponseBytes: answerBytes,
|
||||||
DurationTotal: 12.5, DurationUpstreamTotal: 10.25,
|
DurationTotal: 12.5, DurationUpstreamTotal: 10.25,
|
||||||
@@ -1296,7 +1294,7 @@ func TestEveryDeniedRequestsReportGetsItsLastLine(t *testing.T) {
|
|||||||
"close", map[string]string{denyNets: client, banResponse: banResponseClose}, "/",
|
"close", map[string]string{denyNets: client, banResponse: banResponseClose}, "/",
|
||||||
0, requestlog.ActionDenied,
|
0, requestlog.ActionDenied,
|
||||||
},
|
},
|
||||||
{"own prefix", nil, servewaf.HealthPath, http.StatusNotFound, requestlog.ActionAdmin},
|
{"own prefix", nil, proxy.HealthPath, http.StatusNotFound, requestlog.ActionAdmin},
|
||||||
} {
|
} {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
@@ -1307,7 +1305,7 @@ func TestEveryDeniedRequestsReportGetsItsLastLine(t *testing.T) {
|
|||||||
id := req.Header.Get(requestIDHeader)
|
id := req.Header.Get(requestIDHeader)
|
||||||
do(t, req)
|
do(t, req)
|
||||||
|
|
||||||
got := s.report(servewaf.Report{
|
got := s.report(proxy.Report{
|
||||||
RequestID: id, Status: tc.status, ResponseBytes: 8, DurationTotal: 3,
|
RequestID: id, Status: tc.status, ResponseBytes: 8, DurationTotal: 3,
|
||||||
})
|
})
|
||||||
wantStatus(t, got, http.StatusNoContent)
|
wantStatus(t, got, http.StatusNoContent)
|
||||||
@@ -1343,18 +1341,18 @@ func TestOldestAnsweredRequestIsDroppedPastTheBound(t *testing.T) {
|
|||||||
t.Fatalf("question: %v", err)
|
t.Fatalf("question: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantDisposition(t, readAnswer(res), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, readAnswer(res), http.StatusOK, proxy.DispositionAllow)
|
||||||
}
|
}
|
||||||
|
|
||||||
// The table is full again with next and those, without oldest.
|
// The table is full again with next and those, without oldest.
|
||||||
got := s.report(servewaf.Report{RequestID: oldest, ResponseBytes: answerBytes})
|
got := s.report(proxy.Report{RequestID: oldest, ResponseBytes: answerBytes})
|
||||||
wantStatus(t, got, http.StatusNoContent)
|
wantStatus(t, got, http.StatusNoContent)
|
||||||
|
|
||||||
if counted := s.history().Requests; counted != 0 {
|
if counted := s.history().Requests; counted != 0 {
|
||||||
t.Errorf("history holds %d requests, want the dropped one not counted", counted)
|
t.Errorf("history holds %d requests, want the dropped one not counted", counted)
|
||||||
}
|
}
|
||||||
|
|
||||||
got = s.report(servewaf.Report{RequestID: next, ResponseBytes: answerBytes})
|
got = s.report(proxy.Report{RequestID: next, ResponseBytes: answerBytes})
|
||||||
wantStatus(t, got, http.StatusNoContent)
|
wantStatus(t, got, http.StatusNoContent)
|
||||||
|
|
||||||
if counted := s.history().Requests; counted != 1 {
|
if counted := s.history().Requests; counted != 1 {
|
||||||
@@ -1368,10 +1366,10 @@ func TestBanMadeThroughOneHostRefusesTheClientOnAnother(t *testing.T) {
|
|||||||
s := startServeWAF(t, "", map[string]string{rulesDir: writeRules(t, testRules)})
|
s := startServeWAF(t, "", map[string]string{rulesDir: writeRules(t, testRules)})
|
||||||
|
|
||||||
got := do(t, s.question(siteA, client, "/.env"))
|
got := do(t, s.question(siteA, client, "/.env"))
|
||||||
wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny)
|
||||||
|
|
||||||
got = do(t, s.question(siteB, client, "/"))
|
got = do(t, s.question(siteB, client, "/"))
|
||||||
wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny)
|
||||||
|
|
||||||
lines := s.out.requestLines(t, 2)
|
lines := s.out.requestLines(t, 2)
|
||||||
if lines[0].Host != siteA || lines[1].Host != siteB {
|
if lines[0].Host != siteA || lines[1].Host != siteB {
|
||||||
@@ -1397,7 +1395,7 @@ func TestRequestsOnEverySiteCountTowardOneSetOfLimits(t *testing.T) {
|
|||||||
s.allowed(client)
|
s.allowed(client)
|
||||||
|
|
||||||
got := do(t, s.question(siteB, client, "/"))
|
got := do(t, s.question(siteB, client, "/"))
|
||||||
wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestOwnEndpointsAnswerOnlyWithoutTheClientHeader(t *testing.T) {
|
func TestOwnEndpointsAnswerOnlyWithoutTheClientHeader(t *testing.T) {
|
||||||
@@ -1412,13 +1410,13 @@ func TestOwnEndpointsAnswerOnlyWithoutTheClientHeader(t *testing.T) {
|
|||||||
path, authorization string
|
path, authorization string
|
||||||
want int
|
want int
|
||||||
}{
|
}{
|
||||||
{servewaf.HealthPath, "", http.StatusOK},
|
{proxy.HealthPath, "", http.StatusOK},
|
||||||
{servewaf.MetricsPath, bearer, http.StatusOK},
|
{proxy.MetricsPath, bearer, http.StatusOK},
|
||||||
{servewaf.MetricsPath, "", http.StatusUnauthorized},
|
{proxy.MetricsPath, "", http.StatusUnauthorized},
|
||||||
{servewaf.BansPath, adminBearer, http.StatusOK},
|
{proxy.BansPath, adminBearer, http.StatusOK},
|
||||||
{servewaf.BansPath, "", http.StatusUnauthorized},
|
{proxy.BansPath, "", http.StatusUnauthorized},
|
||||||
{servewaf.ClientsPath + client, adminBearer, http.StatusOK},
|
{proxy.ClientsPath + client, adminBearer, http.StatusOK},
|
||||||
{servewaf.ReportPath, "", http.StatusNotFound},
|
{proxy.ReportPath, "", http.StatusNotFound},
|
||||||
{"/_smallwebwaf/other", "", http.StatusNotFound},
|
{"/_smallwebwaf/other", "", http.StatusNotFound},
|
||||||
{"/", "", http.StatusNotFound},
|
{"/", "", http.StatusNotFound},
|
||||||
} {
|
} {
|
||||||
@@ -1431,9 +1429,9 @@ func TestOwnEndpointsAnswerOnlyWithoutTheClientHeader(t *testing.T) {
|
|||||||
req.Header.Set("Authorization", tc.authorization)
|
req.Header.Set("Authorization", tc.authorization)
|
||||||
|
|
||||||
if strings.HasPrefix(tc.path, "/_smallwebwaf/") {
|
if strings.HasPrefix(tc.path, "/_smallwebwaf/") {
|
||||||
wantDisposition(t, do(t, req), http.StatusNotFound, servewaf.DispositionDeny)
|
wantDisposition(t, do(t, req), http.StatusNotFound, proxy.DispositionDeny)
|
||||||
} else {
|
} else {
|
||||||
wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1443,11 +1441,10 @@ func TestOwnEndpointsLinesDoNotSayWhichOfTheTwoTheyAre(t *testing.T) {
|
|||||||
|
|
||||||
s := startServeWAF(t, "", nil)
|
s := startServeWAF(t, "", nil)
|
||||||
|
|
||||||
wantStatus(t, get(t, s.addr, servewaf.HealthPath), http.StatusOK)
|
wantStatus(t, get(t, s.addr, proxy.HealthPath), http.StatusOK)
|
||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
if _, there := line.fields["servewaf_line"]; there ||
|
if _, there := line.fields["servewaf_line"]; there || line.Path != proxy.HealthPath {
|
||||||
line.Path != servewaf.HealthPath {
|
|
||||||
t.Errorf("the health check's log line is %+v", line.fields)
|
t.Errorf("the health check's log line is %+v", line.fields)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1459,12 +1456,12 @@ func TestALargeBodyAnnouncedAndLongHeadersAreNoReasonToRefuse(t *testing.T) {
|
|||||||
|
|
||||||
req := s.questionWith(http.MethodPost, siteA, client, "/",
|
req := s.questionWith(http.MethodPost, siteA, client, "/",
|
||||||
strings.NewReader("a body"))
|
strings.NewReader("a body"))
|
||||||
req.Header.Set(servewaf.ContentLengthHeader, "1073741824")
|
req.Header.Set(proxy.ContentLengthHeader, "1073741824")
|
||||||
req.Header.Set(forwardedFor, otherClient)
|
req.Header.Set(forwardedFor, otherClient)
|
||||||
req.Header.Set("X-Long", strings.Repeat("x", 40<<10))
|
req.Header.Set("X-Long", strings.Repeat("x", 40<<10))
|
||||||
|
|
||||||
got := do(t, req)
|
got := do(t, req)
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
line := s.out.requestLine(t)
|
line := s.out.requestLine(t)
|
||||||
if line.ClientIP != client || line.ContentLength != 1<<30 {
|
if line.ClientIP != client || line.ContentLength != 1<<30 {
|
||||||
@@ -1532,7 +1529,7 @@ carried header:Content-Length block ^6$
|
|||||||
strings.NewReader("a body"))
|
strings.NewReader("a body"))
|
||||||
|
|
||||||
if tc.announced != "" {
|
if tc.announced != "" {
|
||||||
req.Header.Set(servewaf.ContentLengthHeader, tc.announced)
|
req.Header.Set(proxy.ContentLengthHeader, tc.announced)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantStatus(t, do(t, req), tc.want)
|
wantStatus(t, do(t, req), tc.want)
|
||||||
@@ -1550,19 +1547,19 @@ func TestCoreRuleSetReadsTheBodyAQuestionCarries(t *testing.T) {
|
|||||||
req := s.questionWith(http.MethodPost, siteA, client, "/",
|
req := s.questionWith(http.MethodPost, siteA, client, "/",
|
||||||
strings.NewReader(body))
|
strings.NewReader(body))
|
||||||
req.Header.Set("Content-Type", contentType)
|
req.Header.Set("Content-Type", contentType)
|
||||||
req.Header.Set(servewaf.ContentLengthHeader, "100000")
|
req.Header.Set(proxy.ContentLengthHeader, "100000")
|
||||||
|
|
||||||
return do(t, req)
|
return do(t, req)
|
||||||
}
|
}
|
||||||
|
|
||||||
got := ask(formData, "id=1'%20OR%20'1'='1")
|
got := ask(formData, "id=1'%20OR%20'1'='1")
|
||||||
wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny)
|
||||||
|
|
||||||
// The first 64 KiB of a longer JSON body cannot be parsed, which is no
|
// The first 64 KiB of a longer JSON body cannot be parsed, which is no
|
||||||
// attack, so the Core Rule Set does not read it.
|
// attack, so the Core Rule Set does not read it.
|
||||||
cut := `{"text":"` + strings.Repeat("x", servewaf.QuestionBodyMaxBytes)
|
cut := `{"text":"` + strings.Repeat("x", proxy.QuestionBodyMaxBytes)
|
||||||
got = ask("application/json", cut[:servewaf.QuestionBodyMaxBytes])
|
got = ask("application/json", cut[:proxy.QuestionBodyMaxBytes])
|
||||||
wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
|
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCoreRuleSetAlertNamesTheSite(t *testing.T) {
|
func TestCoreRuleSetAlertNamesTheSite(t *testing.T) {
|
||||||
@@ -1571,7 +1568,7 @@ func TestCoreRuleSetAlertNamesTheSite(t *testing.T) {
|
|||||||
s := startServeWAF(t, "", map[string]string{wafMode: block})
|
s := startServeWAF(t, "", map[string]string{wafMode: block})
|
||||||
|
|
||||||
got := s.ask(client, sqlInjection)
|
got := s.ask(client, sqlInjection)
|
||||||
wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny)
|
||||||
|
|
||||||
waiting := s.queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
waiting := s.queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||||
if len(waiting) != 1 || waiting[0].Event != alerts.EventWAFBlock ||
|
if len(waiting) != 1 || waiting[0].Event != alerts.EventWAFBlock ||
|
||||||
@@ -1586,7 +1583,7 @@ func TestReputationHitAlertNamesTheSite(t *testing.T) {
|
|||||||
s := startServeWAF(t, "", map[string]string{blocklistURLs: dropURL})
|
s := startServeWAF(t, "", map[string]string{blocklistURLs: dropURL})
|
||||||
loadLists(t, s.server, map[string][]string{dropURL: {client}})
|
loadLists(t, s.server, map[string][]string{dropURL: {client}})
|
||||||
|
|
||||||
wantDisposition(t, s.ask(client, "/"), http.StatusForbidden, servewaf.DispositionDeny)
|
wantDisposition(t, s.ask(client, "/"), http.StatusForbidden, proxy.DispositionDeny)
|
||||||
|
|
||||||
waiting := s.queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
waiting := s.queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||||
if len(waiting) != 1 || waiting[0].Event != alerts.EventReputationHit ||
|
if len(waiting) != 1 || waiting[0].Event != alerts.EventReputationHit ||
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -1,4 +1,4 @@
|
|||||||
package servewaf_test
|
package proxy_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -16,9 +16,9 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// post is the start of a POST whose headers end with what follows it.
|
// post is the start of a POST whose headers end with what follows it.
|
||||||
@@ -71,11 +71,11 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceServeWAFAllowsIt(t *testing.T) {
|
|||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxRequestSize: tc.limit,
|
maxRequestSize: tc.limit,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -85,11 +85,11 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceServeWAFAllowsIt(t *testing.T) {
|
|||||||
t.Errorf("answered %+v, want 413", got)
|
t.Errorf("answered %+v, want 413", got)
|
||||||
}
|
}
|
||||||
|
|
||||||
if asked := standIn.question(t); len(asked.body) != tc.asked {
|
if asked := servewaf.question(t); len(asked.body) != tc.asked {
|
||||||
t.Errorf("the question carries %d bytes, want %d", len(asked.body), tc.asked)
|
t.Errorf("the question carries %d bytes, want %d", len(asked.body), tc.asked)
|
||||||
}
|
}
|
||||||
|
|
||||||
report := standIn.report(t)
|
report := servewaf.report(t)
|
||||||
if report.Status != http.StatusRequestEntityTooLarge {
|
if report.Status != http.StatusRequestEntityTooLarge {
|
||||||
t.Errorf("report %+v, want status 413", report)
|
t.Errorf("report %+v, want status 413", report)
|
||||||
}
|
}
|
||||||
@@ -100,9 +100,9 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceServeWAFAllowsIt(t *testing.T) {
|
|||||||
func TestADenyOutranksTheBodySizeLimit(t *testing.T) {
|
func TestADenyOutranksTheBodySizeLimit(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, deny)
|
servewaf := startServeWAF(t, deny)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxRequestSize: "1K",
|
maxRequestSize: "1K",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -132,12 +132,12 @@ func TestABodyNotAnnouncedOverItsLimitIsRefused(t *testing.T) {
|
|||||||
|
|
||||||
var asked atomic.Bool
|
var asked atomic.Bool
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
asked.Store(true)
|
asked.Store(true)
|
||||||
greet(w, r)
|
greet(w, r)
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxRequestSize: tc.limit,
|
maxRequestSize: tc.limit,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -165,12 +165,12 @@ func TestAResponseOverItsLimit(t *testing.T) {
|
|||||||
t.Run("announced, is refused", func(t *testing.T) {
|
t.Run("announced, is refused", func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
w.Header().Set(contentLengthHeader, strconv.Itoa(len(large)))
|
w.Header().Set(contentLengthHeader, strconv.Itoa(len(large)))
|
||||||
_, _ = io.WriteString(w, large)
|
_, _ = io.WriteString(w, large)
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxResponseSize: "1K",
|
maxResponseSize: "1K",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -179,7 +179,7 @@ func TestAResponseOverItsLimit(t *testing.T) {
|
|||||||
t.Errorf("answered %d, want 502", got.status)
|
t.Errorf("answered %d, want 502", got.status)
|
||||||
}
|
}
|
||||||
|
|
||||||
if report := standIn.report(t); report.Status != http.StatusBadGateway {
|
if report := servewaf.report(t); report.Status != http.StatusBadGateway {
|
||||||
t.Errorf("report %+v, want status 502", report)
|
t.Errorf("report %+v, want status 502", report)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -187,13 +187,13 @@ func TestAResponseOverItsLimit(t *testing.T) {
|
|||||||
t.Run("not announced, is cut off", func(t *testing.T) {
|
t.Run("not announced, is cut off", func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
_, _ = io.WriteString(w, large[:1])
|
_, _ = io.WriteString(w, large[:1])
|
||||||
_ = http.NewResponseController(w).Flush()
|
_ = http.NewResponseController(w).Flush()
|
||||||
_, _ = io.WriteString(w, large[1:])
|
_, _ = io.WriteString(w, large[1:])
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxResponseSize: "1K",
|
maxResponseSize: "1K",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -205,7 +205,7 @@ func TestAResponseOverItsLimit(t *testing.T) {
|
|||||||
t.Errorf("read %d bytes (%v), want the answer cut off", len(text), err)
|
t.Errorf("read %d bytes (%v), want the answer cut off", len(text), err)
|
||||||
}
|
}
|
||||||
|
|
||||||
report := standIn.report(t)
|
report := servewaf.report(t)
|
||||||
if report.Status != http.StatusOK || report.ResponseBytes > 1<<10 {
|
if report.Status != http.StatusOK || report.ResponseBytes > 1<<10 {
|
||||||
t.Errorf("report %+v, want status 200 and at most 1K of the answer", report)
|
t.Errorf("report %+v, want status 200 and at most 1K of the answer", report)
|
||||||
}
|
}
|
||||||
@@ -217,11 +217,11 @@ func TestAHeadWhoseAnswerAnnouncesMoreThanTheResponseLimitIsPassedOn(t *testing.
|
|||||||
|
|
||||||
const announced = "2048"
|
const announced = "2048"
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
w.Header().Set(contentLengthHeader, announced)
|
w.Header().Set(contentLengthHeader, announced)
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxResponseSize: "1K",
|
maxResponseSize: "1K",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -240,9 +240,9 @@ func TestASettingOfOffSetsNoLimit(t *testing.T) {
|
|||||||
t.Run(setting, func(t *testing.T) {
|
t.Run(setting, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
setting: "off",
|
setting: "off",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -251,7 +251,7 @@ func TestASettingOfOffSetsNoLimit(t *testing.T) {
|
|||||||
wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/",
|
wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/",
|
||||||
strings.NewReader("a body"))))
|
strings.NewReader("a body"))))
|
||||||
|
|
||||||
if got := get(t, addr, servewaf.HealthPath); got.status != http.StatusOK {
|
if got := get(t, addr, proxy.HealthPath); got.status != http.StatusOK {
|
||||||
t.Errorf("answered %+v, want rproxy's own 200", got)
|
t.Errorf("answered %+v, want rproxy's own 200", got)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -270,9 +270,9 @@ func TestAVisitorTooSlowToSendTheFirstPartIsAnswered408(t *testing.T) {
|
|||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxRequestTime: "500",
|
maxRequestTime: "500",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -288,11 +288,11 @@ func TestAVisitorTooSlowToSendTheFirstPartIsAnswered408(t *testing.T) {
|
|||||||
t.Errorf("line %v, want no disposition and 10 bytes", line)
|
t.Errorf("line %v, want no disposition and 10 bytes", line)
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(standIn.questions) != 0 {
|
if len(servewaf.questions) != 0 {
|
||||||
t.Error("servewaf was asked")
|
t.Error("servewaf was asked")
|
||||||
}
|
}
|
||||||
|
|
||||||
standIn.wantNoReport(t)
|
servewaf.wantNoReport(t)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -300,9 +300,9 @@ func TestAVisitorTooSlowToSendTheFirstPartIsAnswered408(t *testing.T) {
|
|||||||
func TestAVisitorThatGoesAwayBeforeTheFirstPartIsNotAsked(t *testing.T) {
|
func TestAVisitorThatGoesAwayBeforeTheFirstPartIsNotAsked(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, post+announcedLength(100)+"some of it")
|
send(t, conn, post+announcedLength(100)+"some of it")
|
||||||
@@ -313,7 +313,7 @@ func TestAVisitorThatGoesAwayBeforeTheFirstPartIsNotAsked(t *testing.T) {
|
|||||||
t.Errorf("line %v, want no answer and no disposition", line)
|
t.Errorf("line %v, want no answer and no disposition", line)
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(standIn.questions) != 0 {
|
if len(servewaf.questions) != 0 {
|
||||||
t.Error("servewaf was asked")
|
t.Error("servewaf was asked")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -334,9 +334,9 @@ func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
|||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, tc.request)
|
send(t, conn, tc.request)
|
||||||
@@ -364,7 +364,7 @@ func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
|||||||
t.Errorf("line %v, want no answer and no disposition", line)
|
t.Errorf("line %v, want no answer and no disposition", line)
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(standIn.questions) != 0 {
|
if len(servewaf.questions) != 0 {
|
||||||
t.Error("servewaf was asked")
|
t.Error("servewaf was asked")
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -404,11 +404,11 @@ func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
|
|||||||
announced = 100 * questionBodyMaxBytes
|
announced = 100 * questionBodyMaxBytes
|
||||||
)
|
)
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
w.WriteHeader(http.StatusNoContent)
|
w.WriteHeader(http.StatusNoContent)
|
||||||
})
|
})
|
||||||
_, _, server := startServer(t, standIn.URL, app.URL, map[string]string{
|
_, _, server := startServer(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxRequestTime: strconv.FormatInt(limit.Milliseconds(), 10),
|
maxRequestTime: strconv.FormatInt(limit.Milliseconds(), 10),
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -454,9 +454,9 @@ func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
|
|||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, deny)
|
servewaf := startServeWAF(t, deny)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
_, _, server := startServer(t, standIn.URL, app.URL, map[string]string{
|
_, _, server := startServer(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
|
maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -484,9 +484,9 @@ func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
|
|||||||
func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
|
func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxRequestTime: "3000",
|
maxRequestTime: "3000",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -498,11 +498,11 @@ func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
|
|||||||
t.Errorf("status %d, want 408", status)
|
t.Errorf("status %d, want 408", status)
|
||||||
}
|
}
|
||||||
|
|
||||||
if asked := standIn.question(t); len(asked.body) != questionBodyMaxBytes {
|
if asked := servewaf.question(t); len(asked.body) != questionBodyMaxBytes {
|
||||||
t.Errorf("the question carries %d bytes", len(asked.body))
|
t.Errorf("the question carries %d bytes", len(asked.body))
|
||||||
}
|
}
|
||||||
|
|
||||||
if report := standIn.report(t); report.Status != http.StatusRequestTimeout {
|
if report := servewaf.report(t); report.Status != http.StatusRequestTimeout {
|
||||||
t.Errorf("report %+v, want status 408", report)
|
t.Errorf("report %+v, want status 408", report)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -553,8 +553,8 @@ func TestAnAppTooSlowToTakeTheRequestIsAnswered504(t *testing.T) {
|
|||||||
writeSize = 32 << 10
|
writeSize = 32 << 10
|
||||||
)
|
)
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
addr, _ := startRProxy(t, standIn.URL, startAppThatTakesNothing(t),
|
addr, _ := startRProxy(t, servewaf.URL, startAppThatTakesNothing(t),
|
||||||
map[string]string{maxRequestTime: "3000"})
|
map[string]string{maxRequestTime: "3000"})
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
@@ -580,14 +580,14 @@ func TestAnAppTooSlowToAnswerIsAnswered504(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
release := make(chan struct{})
|
release := make(chan struct{})
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
<-release
|
<-release
|
||||||
})
|
})
|
||||||
|
|
||||||
t.Cleanup(func() { close(release) })
|
t.Cleanup(func() { close(release) })
|
||||||
|
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxResponseTime: "300",
|
maxResponseTime: "300",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -596,7 +596,7 @@ func TestAnAppTooSlowToAnswerIsAnswered504(t *testing.T) {
|
|||||||
t.Errorf("answered %d, want 504", got.status)
|
t.Errorf("answered %d, want 504", got.status)
|
||||||
}
|
}
|
||||||
|
|
||||||
if report := standIn.report(t); report.Status != http.StatusGatewayTimeout {
|
if report := servewaf.report(t); report.Status != http.StatusGatewayTimeout {
|
||||||
t.Errorf("report %+v, want status 504", report)
|
t.Errorf("report %+v, want status 504", report)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -607,7 +607,7 @@ func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
|
|||||||
const start = "the start"
|
const start = "the start"
|
||||||
|
|
||||||
release := make(chan struct{})
|
release := make(chan struct{})
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
_, _ = io.WriteString(w, start)
|
_, _ = io.WriteString(w, start)
|
||||||
_ = http.NewResponseController(w).Flush()
|
_ = http.NewResponseController(w).Flush()
|
||||||
@@ -619,7 +619,7 @@ func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
|
|||||||
|
|
||||||
// The time limit is long enough for a loaded host to pass on the start of the
|
// The time limit is long enough for a loaded host to pass on the start of the
|
||||||
// answer before it runs out.
|
// answer before it runs out.
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxResponseTime: "3000",
|
maxResponseTime: "3000",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -642,7 +642,7 @@ func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
|
|||||||
res.StatusCode, body, err, start)
|
res.StatusCode, body, err, start)
|
||||||
}
|
}
|
||||||
|
|
||||||
if report := standIn.report(t); report.Status != http.StatusOK {
|
if report := servewaf.report(t); report.Status != http.StatusOK {
|
||||||
t.Errorf("report %+v, want the status the visitor got", report)
|
t.Errorf("report %+v, want the status the visitor got", report)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -656,7 +656,7 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
|
|||||||
writeSize = 32 << 10
|
writeSize = 32 << 10
|
||||||
)
|
)
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
part := make([]byte, writeSize)
|
part := make([]byte, writeSize)
|
||||||
|
|
||||||
@@ -669,7 +669,7 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
|
|||||||
})
|
})
|
||||||
// The time limit is long enough for a loaded host to start passing on the
|
// The time limit is long enough for a loaded host to start passing on the
|
||||||
// answer before it runs out.
|
// answer before it runs out.
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxResponseTime: "3000",
|
maxResponseTime: "3000",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -684,7 +684,7 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
|
|||||||
t.Errorf("line %v, want status 200 and the answer cut off", line)
|
t.Errorf("line %v, want status 200 and the answer cut off", line)
|
||||||
}
|
}
|
||||||
|
|
||||||
report := standIn.report(t)
|
report := servewaf.report(t)
|
||||||
if report.Status != http.StatusOK || report.ResponseBytes >= bodySize {
|
if report.Status != http.StatusOK || report.ResponseBytes >= bodySize {
|
||||||
t.Errorf("report %+v, want status 200 and the answer cut off", report)
|
t.Errorf("report %+v, want status 200 and the answer cut off", report)
|
||||||
}
|
}
|
||||||
@@ -693,15 +693,15 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
|
|||||||
func TestAnAppThatCannotBeReachedIsAnswered502WithAReport(t *testing.T) {
|
func TestAnAppThatCannotBeReachedIsAnswered502WithAReport(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
|
addr, _ := startRProxy(t, servewaf.URL, nowhere, nil)
|
||||||
|
|
||||||
got := get(t, addr, "/")
|
got := get(t, addr, "/")
|
||||||
if got.status != http.StatusBadGateway {
|
if got.status != http.StatusBadGateway {
|
||||||
t.Errorf("answered %d, want 502", got.status)
|
t.Errorf("answered %d, want 502", got.status)
|
||||||
}
|
}
|
||||||
|
|
||||||
if report := standIn.report(t); report.Status != http.StatusBadGateway {
|
if report := servewaf.report(t); report.Status != http.StatusBadGateway {
|
||||||
t.Errorf("report %+v, want status 502", report)
|
t.Errorf("report %+v, want status 502", report)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -709,11 +709,11 @@ func TestAnAppThatCannotBeReachedIsAnswered502WithAReport(t *testing.T) {
|
|||||||
func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer(t *testing.T) {
|
func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
_, _ = io.WriteString(w, greeting)
|
_, _ = io.WriteString(w, greeting)
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxRequestTime: "3000",
|
maxRequestTime: "3000",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -732,12 +732,12 @@ func TestAVisitorThatStopsSendingWhileTheAppWorksGetsNoAnswer(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
working := make(chan struct{})
|
working := make(chan struct{})
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
|
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
|
||||||
close(working)
|
close(working)
|
||||||
<-r.Context().Done()
|
<-r.Context().Done()
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
||||||
@@ -773,7 +773,7 @@ func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(t *testing
|
|||||||
|
|
||||||
const answer = "the end of a long answer"
|
const answer = "the end of a long answer"
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
_, _ = io.Copy(io.Discard, r.Body)
|
_, _ = io.Copy(io.Discard, r.Body)
|
||||||
_ = http.NewResponseController(w).Flush()
|
_ = http.NewResponseController(w).Flush()
|
||||||
@@ -782,7 +782,7 @@ func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(t *testing
|
|||||||
|
|
||||||
_, _ = io.WriteString(w, answer)
|
_, _ = io.WriteString(w, answer)
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxRequestTime: "2000",
|
maxRequestTime: "2000",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -802,18 +802,18 @@ func TestAVisitorThatStopsSendingGetsAnAnswerRProxyGivesByTheDeadline(t *testing
|
|||||||
{"a deny", post, http.StatusForbidden},
|
{"a deny", post, http.StatusForbidden},
|
||||||
{
|
{
|
||||||
"the health check",
|
"the health check",
|
||||||
"GET " + servewaf.HealthPath + " HTTP/1.1\r\nHost: site.example\r\n",
|
"GET " + proxy.HealthPath + " HTTP/1.1\r\nHost: site.example\r\n",
|
||||||
http.StatusOK,
|
http.StatusOK,
|
||||||
},
|
},
|
||||||
} {
|
} {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, deny)
|
servewaf := startServeWAF(t, deny)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
// The time limit is long enough for a loaded host to read the first part of
|
// The time limit is long enough for a loaded host to read the first part of
|
||||||
// the body before it runs out.
|
// the body before it runs out.
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxRequestTime: "3000",
|
maxRequestTime: "3000",
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|||||||
@@ -12,11 +12,11 @@ import (
|
|||||||
"strings"
|
"strings"
|
||||||
"testing"
|
"testing"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
)
|
)
|
||||||
|
|
||||||
// questionBodyMaxBytes is the most of a body a question carries.
|
// questionBodyMaxBytes is the most of a body a question carries.
|
||||||
const questionBodyMaxBytes = servewaf.QuestionBodyMaxBytes
|
const questionBodyMaxBytes = proxy.QuestionBodyMaxBytes
|
||||||
|
|
||||||
// chunked is the encoding of a body whose length the visitor did not announce.
|
// chunked is the encoding of a body whose length the visitor did not announce.
|
||||||
const chunked = "chunked"
|
const chunked = "chunked"
|
||||||
@@ -29,9 +29,9 @@ const pathGoReencodes = "/some%2Fpath/with%20space|pipe"
|
|||||||
func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
|
func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
const target = pathGoReencodes + "?b=2&a=1&bad=%zz;x"
|
const target = pathGoReencodes + "?b=2&a=1&bad=%zz;x"
|
||||||
|
|
||||||
@@ -42,16 +42,16 @@ func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
|
|||||||
req.Header.Set(forwardedForHeader, "198.51.100.1")
|
req.Header.Set(forwardedForHeader, "198.51.100.1")
|
||||||
req.Header.Set(forwardedProtoHeader, "gopher")
|
req.Header.Set(forwardedProtoHeader, "gopher")
|
||||||
req.Header.Set(requestIDHeader, "the-visitor's")
|
req.Header.Set(requestIDHeader, "the-visitor's")
|
||||||
req.Header.Set(servewaf.ClientHeader, "198.51.100.2")
|
req.Header.Set(proxy.ClientHeader, "198.51.100.2")
|
||||||
req.Header.Set(servewaf.PeerHeader, "198.51.100.3")
|
req.Header.Set(proxy.PeerHeader, "198.51.100.3")
|
||||||
req.Header.Set(servewaf.ContentLengthHeader, "12345")
|
req.Header.Set(proxy.ContentLengthHeader, "12345")
|
||||||
// Go's client then sends no User-Agent, so that one in the question would be
|
// Go's client then sends no User-Agent, so that one in the question would be
|
||||||
// Go's own.
|
// Go's own.
|
||||||
req.Header["User-Agent"] = nil
|
req.Header["User-Agent"] = nil
|
||||||
|
|
||||||
wantGreeting(t, visit(t, req))
|
wantGreeting(t, visit(t, req))
|
||||||
|
|
||||||
asked := standIn.question(t)
|
asked := servewaf.question(t)
|
||||||
if asked.method != http.MethodDelete || asked.target != target ||
|
if asked.method != http.MethodDelete || asked.target != target ||
|
||||||
asked.host != site {
|
asked.host != site {
|
||||||
t.Errorf("asked about %s %s at %s", asked.method, asked.target, asked.host)
|
t.Errorf("asked about %s %s at %s", asked.method, asked.target, asked.host)
|
||||||
@@ -68,8 +68,8 @@ func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
|
|||||||
want.Set(forwardedForHeader, "198.51.100.1")
|
want.Set(forwardedForHeader, "198.51.100.1")
|
||||||
want.Set(forwardedProtoHeader, "http")
|
want.Set(forwardedProtoHeader, "http")
|
||||||
want.Set(requestIDHeader, asked.header.Get(requestIDHeader))
|
want.Set(requestIDHeader, asked.header.Get(requestIDHeader))
|
||||||
want.Set(servewaf.ClientHeader, localhost)
|
want.Set(proxy.ClientHeader, localhost)
|
||||||
want.Set(servewaf.PeerHeader, localhost)
|
want.Set(proxy.PeerHeader, localhost)
|
||||||
|
|
||||||
if !maps.EqualFunc(asked.header, want, slices.Equal) {
|
if !maps.EqualFunc(asked.header, want, slices.Equal) {
|
||||||
t.Errorf("the question's headers are %v, want %v", asked.header, want)
|
t.Errorf("the question's headers are %v, want %v", asked.header, want)
|
||||||
@@ -93,14 +93,14 @@ func TestQuestionNamesTheVisitorATrustedProxySpeaksFor(t *testing.T) {
|
|||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
forwarded := make(chan http.Header, 1)
|
forwarded := make(chan http.Header, 1)
|
||||||
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
forwarded <- r.Header.Clone()
|
forwarded <- r.Header.Clone()
|
||||||
|
|
||||||
greet(w, r)
|
greet(w, r)
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
trustedProxies: tc.trusted,
|
trustedProxies: tc.trusted,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -109,9 +109,9 @@ func TestQuestionNamesTheVisitorATrustedProxySpeaksFor(t *testing.T) {
|
|||||||
req.Header.Set(forwardedProtoHeader, "https")
|
req.Header.Set(forwardedProtoHeader, "https")
|
||||||
wantGreeting(t, visit(t, req))
|
wantGreeting(t, visit(t, req))
|
||||||
|
|
||||||
asked := standIn.question(t)
|
asked := servewaf.question(t)
|
||||||
if asked.header.Get(servewaf.ClientHeader) != tc.client ||
|
if asked.header.Get(proxy.ClientHeader) != tc.client ||
|
||||||
asked.header.Get(servewaf.PeerHeader) != localhost ||
|
asked.header.Get(proxy.PeerHeader) != localhost ||
|
||||||
asked.header.Get(forwardedForHeader) != visitor ||
|
asked.header.Get(forwardedForHeader) != visitor ||
|
||||||
asked.header.Get(forwardedProtoHeader) != tc.proto {
|
asked.header.Get(forwardedProtoHeader) != tc.proto {
|
||||||
t.Errorf("the question's headers are %v", asked.header)
|
t.Errorf("the question's headers are %v", asked.header)
|
||||||
@@ -157,9 +157,9 @@ func TestQuestionNamesTheClientAsTheForwardedAddressesSay(t *testing.T) {
|
|||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
trustedProxies: localhost + "/32,10.0.0.0/8",
|
trustedProxies: localhost + "/32,10.0.0.0/8",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -167,7 +167,7 @@ func TestQuestionNamesTheClientAsTheForwardedAddressesSay(t *testing.T) {
|
|||||||
req.Header[forwardedForHeader] = tc.header
|
req.Header[forwardedForHeader] = tc.header
|
||||||
wantGreeting(t, visit(t, req))
|
wantGreeting(t, visit(t, req))
|
||||||
|
|
||||||
if got := standIn.question(t).header.Get(servewaf.ClientHeader); got != tc.client {
|
if got := servewaf.question(t).header.Get(proxy.ClientHeader); got != tc.client {
|
||||||
t.Errorf("the question names %s, want %s", got, tc.client)
|
t.Errorf("the question names %s, want %s", got, tc.client)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -197,7 +197,7 @@ func TestFirst64KiBReachServeWAFAndTheWholeBodyTheApp(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
body := randomBytes(size)
|
body := randomBytes(size)
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
received := make(chan []byte, 1)
|
received := make(chan []byte, 1)
|
||||||
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
whole, _ := io.ReadAll(r.Body)
|
whole, _ := io.ReadAll(r.Body)
|
||||||
@@ -205,7 +205,7 @@ func TestFirst64KiBReachServeWAFAndTheWholeBodyTheApp(t *testing.T) {
|
|||||||
|
|
||||||
greet(w, r)
|
greet(w, r)
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
req := newRequest(t, http.MethodPost, addr, "/", bytes.NewReader(body))
|
req := newRequest(t, http.MethodPost, addr, "/", bytes.NewReader(body))
|
||||||
if !announced {
|
if !announced {
|
||||||
@@ -215,7 +215,7 @@ func TestFirst64KiBReachServeWAFAndTheWholeBodyTheApp(t *testing.T) {
|
|||||||
|
|
||||||
wantGreeting(t, visit(t, req))
|
wantGreeting(t, visit(t, req))
|
||||||
|
|
||||||
asked := standIn.question(t)
|
asked := servewaf.question(t)
|
||||||
want := body[:min(size, questionBodyMaxBytes)]
|
want := body[:min(size, questionBodyMaxBytes)]
|
||||||
wantQuestionBody(t, asked, want, size, announced)
|
wantQuestionBody(t, asked, want, size, announced)
|
||||||
|
|
||||||
@@ -223,7 +223,7 @@ func TestFirst64KiBReachServeWAFAndTheWholeBodyTheApp(t *testing.T) {
|
|||||||
t.Error("the app did not get the whole body")
|
t.Error("the app did not get the whole body")
|
||||||
}
|
}
|
||||||
|
|
||||||
if report := standIn.report(t); report.RequestBytes != int64(size) {
|
if report := servewaf.report(t); report.RequestBytes != int64(size) {
|
||||||
t.Errorf("report %+v, of %d bytes", report, size)
|
t.Errorf("report %+v, of %d bytes", report, size)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -243,7 +243,7 @@ func wantQuestionBody(
|
|||||||
len(want))
|
len(want))
|
||||||
}
|
}
|
||||||
|
|
||||||
values := asked.header.Values(servewaf.ContentLengthHeader)
|
values := asked.header.Values(proxy.ContentLengthHeader)
|
||||||
|
|
||||||
if announced {
|
if announced {
|
||||||
if !slices.Equal(values, []string{strconv.Itoa(size)}) ||
|
if !slices.Equal(values, []string{strconv.Itoa(size)}) ||
|
||||||
@@ -264,9 +264,9 @@ func wantQuestionBody(
|
|||||||
func TestDenyNeedsOnlyTheFirstPartOfALargeBody(t *testing.T) {
|
func TestDenyNeedsOnlyTheFirstPartOfALargeBody(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, deny)
|
servewaf := startServeWAF(t, deny)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
// Far more announced than sent. Go's server does not wait for that much, so
|
// Far more announced than sent. Go's server does not wait for that much, so
|
||||||
// the answer comes at once.
|
// the answer comes at once.
|
||||||
@@ -278,7 +278,7 @@ func TestDenyNeedsOnlyTheFirstPartOfALargeBody(t *testing.T) {
|
|||||||
t.Errorf("status %d, want servewaf's 403", status)
|
t.Errorf("status %d, want servewaf's 403", status)
|
||||||
}
|
}
|
||||||
|
|
||||||
if report := standIn.report(t); report.RequestBytes != questionBodyMaxBytes {
|
if report := servewaf.report(t); report.RequestBytes != questionBodyMaxBytes {
|
||||||
t.Errorf("report %+v, want the first part's bytes", report)
|
t.Errorf("report %+v, want the first part's bytes", report)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -306,8 +306,8 @@ func TestAQuestionCarriesItsBodyWhateverTheMethod(t *testing.T) {
|
|||||||
t.Run(method+" "+tc.name, func(t *testing.T) {
|
t.Run(method+" "+tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, deny)
|
servewaf := startServeWAF(t, deny)
|
||||||
addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
|
addr, _ := startRProxy(t, servewaf.URL, nowhere, nil)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, method+" /x HTTP/1.1\r\nHost: site.example\r\n"+tc.framing)
|
send(t, conn, method+" /x HTTP/1.1\r\nHost: site.example\r\n"+tc.framing)
|
||||||
@@ -316,7 +316,7 @@ func TestAQuestionCarriesItsBodyWhateverTheMethod(t *testing.T) {
|
|||||||
t.Errorf("status %d, want servewaf's 403", status)
|
t.Errorf("status %d, want servewaf's 403", status)
|
||||||
}
|
}
|
||||||
|
|
||||||
asked := standIn.question(t)
|
asked := servewaf.question(t)
|
||||||
if string(asked.body) != tc.body ||
|
if string(asked.body) != tc.body ||
|
||||||
!slices.Equal(asked.transferEncoding, tc.encoding) {
|
!slices.Equal(asked.transferEncoding, tc.encoding) {
|
||||||
t.Errorf("the question carries %q, encoded %v, want %q, encoded %v",
|
t.Errorf("the question carries %q, encoded %v, want %q, encoded %v",
|
||||||
@@ -334,11 +334,11 @@ func TestAMethodNotPassedOnIsAnswered405(t *testing.T) {
|
|||||||
t.Run(method, func(t *testing.T) {
|
t.Run(method, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
got := visit(t, newRequest(t, method, addr, "/", http.NoBody))
|
got := visit(t, newRequest(t, method, addr, "/", http.NoBody))
|
||||||
if got.status != http.StatusMethodNotAllowed ||
|
if got.status != http.StatusMethodNotAllowed ||
|
||||||
@@ -352,11 +352,11 @@ func TestAMethodNotPassedOnIsAnswered405(t *testing.T) {
|
|||||||
t.Errorf("line %v, want status 405 and no disposition", line)
|
t.Errorf("line %v, want status 405 and no disposition", line)
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(standIn.questions) != 0 {
|
if len(servewaf.questions) != 0 {
|
||||||
t.Error("servewaf was asked")
|
t.Error("servewaf was asked")
|
||||||
}
|
}
|
||||||
|
|
||||||
standIn.wantNoReport(t)
|
servewaf.wantNoReport(t)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -372,18 +372,18 @@ func TestEachMethodPassedOnReachesServeWAFAndTheApp(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
reached := make(chan string, 1)
|
reached := make(chan string, 1)
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
|
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
|
||||||
reached <- r.Method
|
reached <- r.Method
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
got := visit(t, newRequest(t, method, addr, "/", http.NoBody))
|
got := visit(t, newRequest(t, method, addr, "/", http.NoBody))
|
||||||
if got.status != http.StatusOK {
|
if got.status != http.StatusOK {
|
||||||
t.Errorf("answered %+v, want the app's 200", got)
|
t.Errorf("answered %+v, want the app's 200", got)
|
||||||
}
|
}
|
||||||
|
|
||||||
if asked := standIn.question(t); asked.method != method {
|
if asked := servewaf.question(t); asked.method != method {
|
||||||
t.Errorf("servewaf was asked about %s", asked.method)
|
t.Errorf("servewaf was asked about %s", asked.method)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -416,9 +416,9 @@ func TestARequestThatCannotGoToServeWAFAsSentIsRefused(t *testing.T) {
|
|||||||
t.Run(name, func(t *testing.T) {
|
t.Run(name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, request)
|
send(t, conn, request)
|
||||||
@@ -433,11 +433,11 @@ func TestARequestThatCannotGoToServeWAFAsSentIsRefused(t *testing.T) {
|
|||||||
t.Errorf("line %v, want status 400 and no disposition", line)
|
t.Errorf("line %v, want status 400 and no disposition", line)
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(standIn.questions) != 0 {
|
if len(servewaf.questions) != 0 {
|
||||||
t.Error("servewaf was asked")
|
t.Error("servewaf was asked")
|
||||||
}
|
}
|
||||||
|
|
||||||
standIn.wantNoReport(t)
|
servewaf.wantNoReport(t)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -447,9 +447,9 @@ func TestAHostWithAPercentSignButNoBracketGoesToServeWAF(t *testing.T) {
|
|||||||
|
|
||||||
const host = "a%b"
|
const host = "a%b"
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, "GET / HTTP/1.1\r\nHost: "+host+"\r\n\r\n")
|
send(t, conn, "GET / HTTP/1.1\r\nHost: "+host+"\r\n\r\n")
|
||||||
@@ -458,7 +458,7 @@ func TestAHostWithAPercentSignButNoBracketGoesToServeWAF(t *testing.T) {
|
|||||||
t.Errorf("status %d, want the app's 200", status)
|
t.Errorf("status %d, want the app's 200", status)
|
||||||
}
|
}
|
||||||
|
|
||||||
if asked := standIn.question(t); asked.host != host {
|
if asked := servewaf.question(t); asked.host != host {
|
||||||
t.Errorf("asked about the host %q, want %q", asked.host, host)
|
t.Errorf("asked about the host %q, want %q", asked.host, host)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -468,15 +468,15 @@ func TestATrustedPeerWithNoForwardedProtoIsAskedAboutAsHTTP(t *testing.T) {
|
|||||||
|
|
||||||
const want = "http"
|
const want = "http"
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
trustedProxies: localhost + "/32",
|
trustedProxies: localhost + "/32",
|
||||||
})
|
})
|
||||||
|
|
||||||
wantGreeting(t, get(t, addr, "/"))
|
wantGreeting(t, get(t, addr, "/"))
|
||||||
|
|
||||||
asked := standIn.question(t)
|
asked := servewaf.question(t)
|
||||||
if got := asked.header.Get(forwardedProtoHeader); got != want {
|
if got := asked.header.Get(forwardedProtoHeader); got != want {
|
||||||
t.Errorf("the question's %s is %q, want %s", forwardedProtoHeader, got, want)
|
t.Errorf("the question's %s is %q, want %s", forwardedProtoHeader, got, want)
|
||||||
}
|
}
|
||||||
@@ -497,9 +497,9 @@ func TestAQuestionNamesNoZone(t *testing.T) {
|
|||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, deny)
|
servewaf := startServeWAF(t, deny)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
_, out, server := startServer(t, standIn.URL, app.URL, map[string]string{
|
_, out, server := startServer(t, servewaf.URL, app.URL, map[string]string{
|
||||||
trustedProxies: localhost + "/32",
|
trustedProxies: localhost + "/32",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -509,9 +509,9 @@ func TestAQuestionNamesNoZone(t *testing.T) {
|
|||||||
|
|
||||||
server.Handler.ServeHTTP(httptest.NewRecorder(), req)
|
server.Handler.ServeHTTP(httptest.NewRecorder(), req)
|
||||||
|
|
||||||
asked := standIn.question(t)
|
asked := servewaf.question(t)
|
||||||
if asked.header.Get(servewaf.ClientHeader) != tc.client ||
|
if asked.header.Get(proxy.ClientHeader) != tc.client ||
|
||||||
asked.header.Get(servewaf.PeerHeader) != tc.peer {
|
asked.header.Get(proxy.PeerHeader) != tc.peer {
|
||||||
t.Errorf("the question's headers are %v", asked.header)
|
t.Errorf("the question's headers are %v", asked.header)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -16,9 +16,9 @@ import (
|
|||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// flushAfterEachWrite has ReverseProxy pass on each part of the app's answer as
|
// flushAfterEachWrite has ReverseProxy pass on each part of the app's answer as
|
||||||
@@ -129,7 +129,7 @@ func (rq *request) readFirstPartOfBody() bool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
_ = rq.rc.SetReadDeadline(rq.requestDeadline())
|
_ = rq.rc.SetReadDeadline(rq.requestDeadline())
|
||||||
held, err := io.ReadAll(io.LimitReader(rq.body, servewaf.QuestionBodyMaxBytes))
|
held, err := io.ReadAll(io.LimitReader(rq.body, proxy.QuestionBodyMaxBytes))
|
||||||
// From here on, the request timer ends a read of the rest of the body, so that
|
// From here on, the request timer ends a read of the rest of the body, so that
|
||||||
// the answer can name the side rproxy was waiting on.
|
// the answer can name the side rproxy was waiting on.
|
||||||
_ = rq.rc.SetReadDeadline(time.Time{})
|
_ = rq.rc.SetReadDeadline(time.Time{})
|
||||||
@@ -384,7 +384,7 @@ func (rq *request) finish(ctx context.Context) {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
report := &servewaf.Report{
|
report := &proxy.Report{
|
||||||
RequestID: rq.id,
|
RequestID: rq.id,
|
||||||
Status: min(rq.out.status, reportStatusMax),
|
Status: min(rq.out.status, reportStatusMax),
|
||||||
RequestBytes: requestBytes,
|
RequestBytes: requestBytes,
|
||||||
|
|||||||
@@ -22,7 +22,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
)
|
)
|
||||||
|
|
||||||
// How rproxy keeps connections to the app and to servewaf open between
|
// How rproxy keeps connections to the app and to servewaf open between
|
||||||
@@ -85,7 +85,7 @@ func New(params Params) *Server {
|
|||||||
transport: newTransport(),
|
transport: newTransport(),
|
||||||
serveWAFTransport: newServeWAFTransport(),
|
serveWAFTransport: newServeWAFTransport(),
|
||||||
reportURL: (&url.URL{
|
reportURL: (&url.URL{
|
||||||
Scheme: backend.Scheme, Host: backend.Host, Path: servewaf.ReportPath,
|
Scheme: backend.Scheme, Host: backend.Host, Path: proxy.ReportPath,
|
||||||
}).String(),
|
}).String(),
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -166,7 +166,7 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
rq := h.newRequest(w, r)
|
rq := h.newRequest(w, r)
|
||||||
defer rq.finish(r.Context())
|
defer rq.finish(r.Context())
|
||||||
|
|
||||||
if r.Method == http.MethodGet && r.URL.Path == servewaf.HealthPath {
|
if r.Method == http.MethodGet && r.URL.Path == proxy.HealthPath {
|
||||||
rq.answerHealth()
|
rq.answerHealth()
|
||||||
|
|
||||||
return
|
return
|
||||||
@@ -199,9 +199,9 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
rq.noteAnswer(answer)
|
rq.noteAnswer(answer)
|
||||||
|
|
||||||
switch rq.disposition {
|
switch rq.disposition {
|
||||||
case servewaf.DispositionAllow:
|
case proxy.DispositionAllow:
|
||||||
rq.passToApp(r.Context())
|
rq.passToApp(r.Context())
|
||||||
case servewaf.DispositionClose:
|
case proxy.DispositionClose:
|
||||||
// This panic has Go's server close the connection without an answer,
|
// This panic has Go's server close the connection without an answer,
|
||||||
// and log nothing; the deferred finish still runs.
|
// and log nothing; the deferred finish still runs.
|
||||||
panic(http.ErrAbortHandler)
|
panic(http.ErrAbortHandler)
|
||||||
|
|||||||
@@ -19,9 +19,9 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -54,7 +54,7 @@ const (
|
|||||||
|
|
||||||
// The headers the tests set and look at.
|
// The headers the tests set and look at.
|
||||||
const (
|
const (
|
||||||
dispositionHeader = servewaf.DispositionHeader
|
dispositionHeader = proxy.DispositionHeader
|
||||||
forwardedForHeader = "X-Forwarded-For"
|
forwardedForHeader = "X-Forwarded-For"
|
||||||
forwardedProtoHeader = "X-Forwarded-Proto"
|
forwardedProtoHeader = "X-Forwarded-Proto"
|
||||||
requestIDHeader = "X-Request-ID"
|
requestIDHeader = "X-Request-ID"
|
||||||
@@ -70,9 +70,9 @@ const (
|
|||||||
|
|
||||||
// What the stand-in for servewaf answers.
|
// What the stand-in for servewaf answers.
|
||||||
const (
|
const (
|
||||||
dispositionAllow = servewaf.DispositionAllow
|
dispositionAllow = proxy.DispositionAllow
|
||||||
dispositionDeny = servewaf.DispositionDeny
|
dispositionDeny = proxy.DispositionDeny
|
||||||
dispositionClose = servewaf.DispositionClose
|
dispositionClose = proxy.DispositionClose
|
||||||
// deniedBody is the body of a deny.
|
// deniedBody is the body of a deny.
|
||||||
deniedBody = "refused by servewaf"
|
deniedBody = "refused by servewaf"
|
||||||
)
|
)
|
||||||
@@ -173,7 +173,7 @@ type serveWAF struct {
|
|||||||
*httptest.Server
|
*httptest.Server
|
||||||
|
|
||||||
questions chan question
|
questions chan question
|
||||||
reports chan servewaf.Report
|
reports chan proxy.Report
|
||||||
}
|
}
|
||||||
|
|
||||||
// startServeWAF starts a stand-in for servewaf.
|
// startServeWAF starts a stand-in for servewaf.
|
||||||
@@ -182,11 +182,11 @@ func startServeWAF(t *testing.T, answer http.HandlerFunc) *serveWAF {
|
|||||||
|
|
||||||
s := &serveWAF{
|
s := &serveWAF{
|
||||||
questions: make(chan question, 10),
|
questions: make(chan question, 10),
|
||||||
reports: make(chan servewaf.Report, 10),
|
reports: make(chan proxy.Report, 10),
|
||||||
}
|
}
|
||||||
s.Server = httptest.NewServer(http.HandlerFunc(
|
s.Server = httptest.NewServer(http.HandlerFunc(
|
||||||
func(w http.ResponseWriter, r *http.Request) {
|
func(w http.ResponseWriter, r *http.Request) {
|
||||||
if r.Header.Get(servewaf.ClientHeader) == "" {
|
if r.Header.Get(proxy.ClientHeader) == "" {
|
||||||
s.takeReport(t, w, r)
|
s.takeReport(t, w, r)
|
||||||
|
|
||||||
return
|
return
|
||||||
@@ -213,10 +213,10 @@ func startServeWAF(t *testing.T, answer http.HandlerFunc) *serveWAF {
|
|||||||
func (s *serveWAF) takeReport(t *testing.T, w http.ResponseWriter, r *http.Request) {
|
func (s *serveWAF) takeReport(t *testing.T, w http.ResponseWriter, r *http.Request) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
var report servewaf.Report
|
var report proxy.Report
|
||||||
|
|
||||||
err := json.NewDecoder(r.Body).Decode(&report)
|
err := json.NewDecoder(r.Body).Decode(&report)
|
||||||
if err != nil || r.Method != http.MethodPost || r.URL.Path != servewaf.ReportPath {
|
if err != nil || r.Method != http.MethodPost || r.URL.Path != proxy.ReportPath {
|
||||||
t.Errorf("%s %s is no report: %v", r.Method, r.URL.Path, err)
|
t.Errorf("%s %s is no report: %v", r.Method, r.URL.Path, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -237,7 +237,7 @@ func (s *serveWAF) wantNoReport(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// report waits for a report.
|
// report waits for a report.
|
||||||
func (s *serveWAF) report(t *testing.T) servewaf.Report {
|
func (s *serveWAF) report(t *testing.T) proxy.Report {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
select {
|
select {
|
||||||
@@ -246,7 +246,7 @@ func (s *serveWAF) report(t *testing.T) servewaf.Report {
|
|||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Fatal("no report")
|
t.Fatal("no report")
|
||||||
|
|
||||||
return servewaf.Report{}
|
return proxy.Report{}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -273,7 +273,7 @@ func startServeWAFAnsweringReports(
|
|||||||
|
|
||||||
backend := httptest.NewServer(http.HandlerFunc(
|
backend := httptest.NewServer(http.HandlerFunc(
|
||||||
func(w http.ResponseWriter, r *http.Request) {
|
func(w http.ResponseWriter, r *http.Request) {
|
||||||
if r.Header.Get(servewaf.ClientHeader) != "" {
|
if r.Header.Get(proxy.ClientHeader) != "" {
|
||||||
answer(w, r)
|
answer(w, r)
|
||||||
|
|
||||||
return
|
return
|
||||||
@@ -498,7 +498,7 @@ func readStatus(t *testing.T, conn net.Conn) int {
|
|||||||
func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
|
func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, func(w http.ResponseWriter, r *http.Request) {
|
servewaf := startServeWAF(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
w.Header().Set(asnHeader, "AS64496")
|
w.Header().Set(asnHeader, "AS64496")
|
||||||
w.Header().Set(countryHeader, "DE")
|
w.Header().Set(countryHeader, "DE")
|
||||||
allow(w, r)
|
allow(w, r)
|
||||||
@@ -512,7 +512,7 @@ func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
|
|||||||
w.WriteHeader(http.StatusTeapot)
|
w.WriteHeader(http.StatusTeapot)
|
||||||
_, _ = io.WriteString(w, greeting)
|
_, _ = io.WriteString(w, greeting)
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
const target = pathGoReencodes + "?b=2&a=1&bad=%zz;x"
|
const target = pathGoReencodes + "?b=2&a=1&bad=%zz;x"
|
||||||
|
|
||||||
@@ -530,7 +530,7 @@ func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
received := <-seen
|
received := <-seen
|
||||||
asked := standIn.question(t)
|
asked := servewaf.question(t)
|
||||||
|
|
||||||
if received.RequestURI != target || received.Host != site {
|
if received.RequestURI != target || received.Host != site {
|
||||||
t.Errorf("the app saw %s for %s", received.RequestURI, received.Host)
|
t.Errorf("the app saw %s for %s", received.RequestURI, received.Host)
|
||||||
@@ -557,14 +557,14 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
|
|||||||
"X-Forwarded-Tls-Client-Cert-Info", "X-Real-Ip",
|
"X-Forwarded-Tls-Client-Cert-Info", "X-Real-Ip",
|
||||||
}
|
}
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
seen := make(chan http.Header, 1)
|
seen := make(chan http.Header, 1)
|
||||||
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
seen <- r.Header.Clone()
|
seen <- r.Header.Clone()
|
||||||
|
|
||||||
greet(w, r)
|
greet(w, r)
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
trustedProxies: "10.0.0.0/8",
|
trustedProxies: "10.0.0.0/8",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -602,14 +602,14 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
|
|||||||
func TestATrustedProxysForwardedHeadersPassOn(t *testing.T) {
|
func TestATrustedProxysForwardedHeadersPassOn(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
seen := make(chan http.Header, 1)
|
seen := make(chan http.Header, 1)
|
||||||
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
||||||
seen <- r.Header.Clone()
|
seen <- r.Header.Clone()
|
||||||
|
|
||||||
greet(w, r)
|
greet(w, r)
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
trustedProxies: localhost + "/32",
|
trustedProxies: localhost + "/32",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -641,12 +641,12 @@ func TestATrustedProxysForwardedHeadersPassOn(t *testing.T) {
|
|||||||
func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
|
func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
w.Header()["Content-Type"] = nil
|
w.Header()["Content-Type"] = nil
|
||||||
_, _ = io.WriteString(w, "<html>")
|
_, _ = io.WriteString(w, "<html>")
|
||||||
})
|
})
|
||||||
_, _, server := startServer(t, standIn.URL, app.URL, nil)
|
_, _, server := startServer(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
|
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
|
||||||
w := httptest.NewRecorder()
|
w := httptest.NewRecorder()
|
||||||
@@ -704,11 +704,11 @@ func TestRemovesTheVisitorsLookupHeadersWhenServeWAFGivesNone(t *testing.T) {
|
|||||||
func TestDenyGivesTheVisitorServeWAFsAnswer(t *testing.T) {
|
func TestDenyGivesTheVisitorServeWAFsAnswer(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, deny)
|
servewaf := startServeWAF(t, deny)
|
||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
got := get(t, addr, "/")
|
got := get(t, addr, "/")
|
||||||
if got.status != http.StatusForbidden || got.body != deniedBody ||
|
if got.status != http.StatusForbidden || got.body != deniedBody ||
|
||||||
@@ -720,7 +720,7 @@ func TestDenyGivesTheVisitorServeWAFsAnswer(t *testing.T) {
|
|||||||
t.Errorf("the visitor got %s", dispositionHeader)
|
t.Errorf("the visitor got %s", dispositionHeader)
|
||||||
}
|
}
|
||||||
|
|
||||||
report := standIn.report(t)
|
report := servewaf.report(t)
|
||||||
if report.Status != http.StatusForbidden ||
|
if report.Status != http.StatusForbidden ||
|
||||||
report.ResponseBytes != int64(len(deniedBody)) ||
|
report.ResponseBytes != int64(len(deniedBody)) ||
|
||||||
report.DurationUpstreamTotal != 0 {
|
report.DurationUpstreamTotal != 0 {
|
||||||
@@ -736,9 +736,9 @@ func TestDenyGivesTheVisitorServeWAFsAnswer(t *testing.T) {
|
|||||||
func TestADeniedHeadKeepsServeWAFsContentLength(t *testing.T) {
|
func TestADeniedHeadKeepsServeWAFsContentLength(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, deny)
|
servewaf := startServeWAF(t, deny)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
|
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
|
||||||
if got.status != http.StatusForbidden ||
|
if got.status != http.StatusForbidden ||
|
||||||
@@ -758,15 +758,15 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
|
|||||||
}{
|
}{
|
||||||
{"a deny", "GET /" + host, deniedBody, http.StatusForbidden},
|
{"a deny", "GET /" + host, deniedBody, http.StatusForbidden},
|
||||||
{"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest},
|
{"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest},
|
||||||
{"the health check", "GET " + servewaf.HealthPath + host, "ok\n", http.StatusOK},
|
{"the health check", "GET " + proxy.HealthPath + host, "ok\n", http.StatusOK},
|
||||||
} {
|
} {
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
log := heldLog{release: make(chan struct{})}
|
log := heldLog{release: make(chan struct{})}
|
||||||
standIn := startServeWAF(t, deny)
|
servewaf := startServeWAF(t, deny)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startServerLogging(t, standIn.URL, app.URL, nil, log, io.Discard)
|
addr, _ := startServerLogging(t, servewaf.URL, app.URL, nil, log, io.Discard)
|
||||||
|
|
||||||
t.Cleanup(func() { close(log.release) })
|
t.Cleanup(func() { close(log.release) })
|
||||||
|
|
||||||
@@ -794,9 +794,9 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
|
|||||||
func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
|
func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, "GET x:admin HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
send(t, conn, "GET x:admin HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
||||||
@@ -817,14 +817,14 @@ func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
|
|||||||
func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
servewaf := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
w.Header().Set(dispositionHeader, dispositionClose)
|
w.Header().Set(dispositionHeader, dispositionClose)
|
||||||
w.WriteHeader(http.StatusOK)
|
w.WriteHeader(http.StatusOK)
|
||||||
})
|
})
|
||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
||||||
@@ -835,7 +835,7 @@ func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
|||||||
text, err)
|
text, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
report := standIn.report(t)
|
report := servewaf.report(t)
|
||||||
if report.Status != 0 || report.ResponseBytes != 0 {
|
if report.Status != 0 || report.ResponseBytes != 0 {
|
||||||
t.Errorf("report %+v, of a visitor sent nothing", report)
|
t.Errorf("report %+v, of a visitor sent nothing", report)
|
||||||
}
|
}
|
||||||
@@ -935,7 +935,7 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsServeWAFGaveIt(t *testing.T) {
|
|||||||
t.Run(tc.name+" with the default "+setting, func(t *testing.T) {
|
t.Run(tc.name+" with the default "+setting, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
servewaf := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
if tc.disposition != "" {
|
if tc.disposition != "" {
|
||||||
w.Header().Set(dispositionHeader, tc.disposition)
|
w.Header().Set(dispositionHeader, tc.disposition)
|
||||||
}
|
}
|
||||||
@@ -947,7 +947,7 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsServeWAFGaveIt(t *testing.T) {
|
|||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
defaultDisposition: setting,
|
defaultDisposition: setting,
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -977,11 +977,11 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsServeWAFGaveIt(t *testing.T) {
|
|||||||
func TestAnAnswerWithoutAValidDispositionGetsNoReport(t *testing.T) {
|
func TestAnAnswerWithoutAValidDispositionGetsNoReport(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
servewaf := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
http.Error(w, "refused", http.StatusBadRequest)
|
http.Error(w, "refused", http.StatusBadRequest)
|
||||||
})
|
})
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
if got := get(t, addr, "/"); got.status != http.StatusBadRequest {
|
if got := get(t, addr, "/"); got.status != http.StatusBadRequest {
|
||||||
t.Errorf("answered %+v, want servewaf's 400", got)
|
t.Errorf("answered %+v, want servewaf's 400", got)
|
||||||
@@ -991,18 +991,18 @@ func TestAnAnswerWithoutAValidDispositionGetsNoReport(t *testing.T) {
|
|||||||
t.Errorf("line %v, want no disposition", line)
|
t.Errorf("line %v, want no disposition", line)
|
||||||
}
|
}
|
||||||
|
|
||||||
standIn.wantNoReport(t)
|
servewaf.wantNoReport(t)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAnAnswerWithoutAContentTypeGoesToTheVisitorWithNone(t *testing.T) {
|
func TestAnAnswerWithoutAContentTypeGoesToTheVisitorWithNone(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
servewaf := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
w.Header()["Content-Type"] = nil
|
w.Header()["Content-Type"] = nil
|
||||||
w.WriteHeader(http.StatusInternalServerError)
|
w.WriteHeader(http.StatusInternalServerError)
|
||||||
_, _ = io.WriteString(w, "<html>broken")
|
_, _ = io.WriteString(w, "<html>broken")
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
|
addr, _ := startRProxy(t, servewaf.URL, nowhere, nil)
|
||||||
|
|
||||||
got := get(t, addr, "/")
|
got := get(t, addr, "/")
|
||||||
if contentType, sent := got.header["Content-Type"]; sent {
|
if contentType, sent := got.header["Content-Type"]; sent {
|
||||||
@@ -1015,11 +1015,11 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
|
|||||||
|
|
||||||
const size = 1 << 20
|
const size = 1 << 20
|
||||||
|
|
||||||
standIn := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
servewaf := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
w.WriteHeader(http.StatusInternalServerError)
|
w.WriteHeader(http.StatusInternalServerError)
|
||||||
_, _ = w.Write(make([]byte, size))
|
_, _ = w.Write(make([]byte, size))
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
|
addr, _ := startRProxy(t, servewaf.URL, nowhere, nil)
|
||||||
|
|
||||||
got := get(t, addr, "/")
|
got := get(t, addr, "/")
|
||||||
if got.status != http.StatusInternalServerError || len(got.body) != size {
|
if got.status != http.StatusInternalServerError || len(got.body) != size {
|
||||||
@@ -1135,15 +1135,15 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
|
|||||||
func TestAnAnswerThatCannotBePassedOnGetsNoReport(t *testing.T) {
|
func TestAnAnswerThatCannotBePassedOnGetsNoReport(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, overOneMiB(dispositionDeny))
|
servewaf := startServeWAF(t, overOneMiB(dispositionDeny))
|
||||||
addr, out := startRProxy(t, standIn.URL, nowhere, nil)
|
addr, out := startRProxy(t, servewaf.URL, nowhere, nil)
|
||||||
|
|
||||||
if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
|
if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
|
||||||
t.Errorf("answered %+v, want 502", got)
|
t.Errorf("answered %+v, want 502", got)
|
||||||
}
|
}
|
||||||
|
|
||||||
out.requestLine(t)
|
out.requestLine(t)
|
||||||
standIn.wantNoReport(t)
|
servewaf.wantNoReport(t)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
|
func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
|
||||||
@@ -1199,20 +1199,20 @@ func TestAVisitorThatGoesAwayWhileServeWAFIsAskedGetsNoDefault(t *testing.T) {
|
|||||||
t.Run("with the default "+disposition, func(t *testing.T) {
|
t.Run("with the default "+disposition, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, func(_ http.ResponseWriter, r *http.Request) {
|
servewaf := startServeWAF(t, func(_ http.ResponseWriter, r *http.Request) {
|
||||||
<-r.Context().Done()
|
<-r.Context().Done()
|
||||||
})
|
})
|
||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
timeoutSetting: "60000",
|
timeoutSetting: "60000",
|
||||||
defaultDisposition: disposition,
|
defaultDisposition: disposition,
|
||||||
})
|
})
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
||||||
standIn.question(t)
|
servewaf.question(t)
|
||||||
|
|
||||||
_ = conn.Close()
|
_ = conn.Close()
|
||||||
|
|
||||||
@@ -1226,7 +1226,7 @@ func TestAVisitorThatGoesAwayWhileServeWAFIsAskedGetsNoDefault(t *testing.T) {
|
|||||||
t.Error("rproxy warned that servewaf gave no answer")
|
t.Error("rproxy warned that servewaf gave no answer")
|
||||||
}
|
}
|
||||||
|
|
||||||
standIn.wantNoReport(t)
|
servewaf.wantNoReport(t)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1244,17 +1244,17 @@ func TestRProxyWarnsWhenTheDefaultDecides(t *testing.T) {
|
|||||||
func TestSendsNoReportWithoutAnAnswer(t *testing.T) {
|
func TestSendsNoReportWithoutAnAnswer(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, func(_ http.ResponseWriter, r *http.Request) {
|
servewaf := startServeWAF(t, func(_ http.ResponseWriter, r *http.Request) {
|
||||||
<-r.Context().Done()
|
<-r.Context().Done()
|
||||||
})
|
})
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
timeoutSetting: "200",
|
timeoutSetting: "200",
|
||||||
})
|
})
|
||||||
|
|
||||||
wantGreeting(t, get(t, addr, "/"))
|
wantGreeting(t, get(t, addr, "/"))
|
||||||
out.requestLine(t)
|
out.requestLine(t)
|
||||||
standIn.wantNoReport(t)
|
servewaf.wantNoReport(t)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAPathUnderOwnPrefixNeverReachesTheApp(t *testing.T) {
|
func TestAPathUnderOwnPrefixNeverReachesTheApp(t *testing.T) {
|
||||||
@@ -1274,25 +1274,25 @@ func TestAPathUnderOwnPrefixNeverReachesTheApp(t *testing.T) {
|
|||||||
func TestHealthCheckIsAnsweredWhileServeWAFIsDown(t *testing.T) {
|
func TestHealthCheckIsAnsweredWhileServeWAFIsDown(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, nowhere, app.URL, nil)
|
addr, out := startRProxy(t, nowhere, app.URL, nil)
|
||||||
|
|
||||||
got := get(t, addr, servewaf.HealthPath)
|
got := get(t, addr, proxy.HealthPath)
|
||||||
if got.status != http.StatusOK || got.body != "ok\n" {
|
if got.status != http.StatusOK || got.body != "ok\n" {
|
||||||
t.Errorf("answered %+v, want ok", got)
|
t.Errorf("answered %+v, want ok", got)
|
||||||
}
|
}
|
||||||
|
|
||||||
line := out.requestLine(t)
|
line := out.requestLine(t)
|
||||||
if line["path"] != servewaf.HealthPath || line["disposition"] != nil {
|
if line["path"] != proxy.HealthPath || line["disposition"] != nil {
|
||||||
t.Errorf("line %v, want one with no disposition", line)
|
t.Errorf("line %v, want one with no disposition", line)
|
||||||
}
|
}
|
||||||
|
|
||||||
// With servewaf up too, it is not asked.
|
// With servewaf up too, it is not asked.
|
||||||
addr, _ = startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ = startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
get(t, addr, servewaf.HealthPath)
|
get(t, addr, proxy.HealthPath)
|
||||||
|
|
||||||
if len(standIn.questions) != 0 {
|
if len(servewaf.questions) != 0 {
|
||||||
t.Error("servewaf was asked about the health check")
|
t.Error("servewaf was asked about the health check")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1396,15 +1396,15 @@ func TestAReportAnswered101IsDroppedAtOnce(t *testing.T) {
|
|||||||
func TestAReportDueAfterWaitForReportsIsDropped(t *testing.T) {
|
func TestAReportDueAfterWaitForReportsIsDropped(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out, server := startServer(t, standIn.URL, app.URL, nil)
|
addr, out, server := startServer(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
server.WaitForReports()
|
server.WaitForReports()
|
||||||
|
|
||||||
wantGreeting(t, get(t, addr, "/"))
|
wantGreeting(t, get(t, addr, "/"))
|
||||||
out.requestLine(t)
|
out.requestLine(t)
|
||||||
standIn.wantNoReport(t)
|
servewaf.wantNoReport(t)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestReportGivesWhatHappened(t *testing.T) {
|
func TestReportGivesWhatHappened(t *testing.T) {
|
||||||
@@ -1412,14 +1412,14 @@ func TestReportGivesWhatHappened(t *testing.T) {
|
|||||||
|
|
||||||
const body = "the visitor's body"
|
const body = "the visitor's body"
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/",
|
wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/",
|
||||||
strings.NewReader(body))))
|
strings.NewReader(body))))
|
||||||
|
|
||||||
report := standIn.report(t)
|
report := servewaf.report(t)
|
||||||
|
|
||||||
if report.RequestID == "" || report.Status != http.StatusOK ||
|
if report.RequestID == "" || report.Status != http.StatusOK ||
|
||||||
report.RequestBytes != int64(len(body)) ||
|
report.RequestBytes != int64(len(body)) ||
|
||||||
@@ -1442,7 +1442,7 @@ func TestWaitForReportsWaitsForAReportBeingSent(t *testing.T) {
|
|||||||
release := make(chan struct{})
|
release := make(chan struct{})
|
||||||
backend := httptest.NewServer(http.HandlerFunc(
|
backend := httptest.NewServer(http.HandlerFunc(
|
||||||
func(w http.ResponseWriter, r *http.Request) {
|
func(w http.ResponseWriter, r *http.Request) {
|
||||||
if r.Header.Get(servewaf.ClientHeader) != "" {
|
if r.Header.Get(proxy.ClientHeader) != "" {
|
||||||
allow(w, r)
|
allow(w, r)
|
||||||
|
|
||||||
return
|
return
|
||||||
@@ -1497,9 +1497,9 @@ func TestWaitForReportsWaitsForAReportBeingSent(t *testing.T) {
|
|||||||
func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
|
func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, echoAfterUpgrade)
|
app := startApp(t, echoAfterUpgrade)
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
maxRequestTime: "2000",
|
maxRequestTime: "2000",
|
||||||
maxResponseTime: "2000",
|
maxResponseTime: "2000",
|
||||||
})
|
})
|
||||||
@@ -1535,7 +1535,7 @@ func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
|
|||||||
|
|
||||||
_ = conn.Close()
|
_ = conn.Close()
|
||||||
|
|
||||||
report := standIn.report(t)
|
report := servewaf.report(t)
|
||||||
if report.Status != http.StatusSwitchingProtocols ||
|
if report.Status != http.StatusSwitchingProtocols ||
|
||||||
report.RequestBytes != int64(len(message)) ||
|
report.RequestBytes != int64(len(message)) ||
|
||||||
report.ResponseBytes != int64(len(message)) {
|
report.ResponseBytes != int64(len(message)) {
|
||||||
@@ -1552,7 +1552,7 @@ func TestAWebSocketPassesOnThatTheVisitorStoppedSending(t *testing.T) {
|
|||||||
|
|
||||||
const goodbye = "sent after the visitor stopped sending\n"
|
const goodbye = "sent after the visitor stopped sending\n"
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
conn, buffered, err := http.NewResponseController(w).Hijack()
|
conn, buffered, err := http.NewResponseController(w).Hijack()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -1572,7 +1572,7 @@ func TestAWebSocketPassesOnThatTheVisitorStoppedSending(t *testing.T) {
|
|||||||
_, _ = buffered.WriteString(goodbye)
|
_, _ = buffered.WriteString(goodbye)
|
||||||
_ = buffered.Flush()
|
_ = buffered.Flush()
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+
|
send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+
|
||||||
@@ -1610,7 +1610,7 @@ func TestAWebSocketPassesOnThatTheVisitorStoppedSending(t *testing.T) {
|
|||||||
//nolint:paralleltest // see above
|
//nolint:paralleltest // see above
|
||||||
func TestASwitchNotPassedOnClosesTheAppsConnection(t *testing.T) {
|
func TestASwitchNotPassedOnClosesTheAppsConnection(t *testing.T) {
|
||||||
closed := make(chan error, 1)
|
closed := make(chan error, 1)
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
conn, buffered, err := http.NewResponseController(w).Hijack()
|
conn, buffered, err := http.NewResponseController(w).Hijack()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -1633,7 +1633,7 @@ func TestASwitchNotPassedOnClosesTheAppsConnection(t *testing.T) {
|
|||||||
_, err = buffered.ReadByte()
|
_, err = buffered.ReadByte()
|
||||||
closed <- err
|
closed <- err
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+
|
send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+
|
||||||
@@ -1681,9 +1681,9 @@ func TestEachRequestGetsAFreshID(t *testing.T) {
|
|||||||
|
|
||||||
const fromTraefik = "an-id-from-traefik"
|
const fromTraefik = "an-id-from-traefik"
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||||
trustedProxies: localhost + "/32",
|
trustedProxies: localhost + "/32",
|
||||||
})
|
})
|
||||||
|
|
||||||
@@ -1694,7 +1694,7 @@ func TestEachRequestGetsAFreshID(t *testing.T) {
|
|||||||
req.Header.Set(requestIDHeader, fromTraefik)
|
req.Header.Set(requestIDHeader, fromTraefik)
|
||||||
wantGreeting(t, visit(t, req))
|
wantGreeting(t, visit(t, req))
|
||||||
|
|
||||||
ids = append(ids, standIn.question(t).header.Get(requestIDHeader))
|
ids = append(ids, servewaf.question(t).header.Get(requestIDHeader))
|
||||||
}
|
}
|
||||||
|
|
||||||
if slices.Contains(ids, fromTraefik) || ids[0] == "" || ids[0] == ids[1] {
|
if slices.Contains(ids, fromTraefik) || ids[0] == "" || ids[0] == ids[1] {
|
||||||
@@ -1711,16 +1711,16 @@ func TestLogLineGivesTheRequestAsSent(t *testing.T) {
|
|||||||
body = "the visitor's body"
|
body = "the visitor's body"
|
||||||
)
|
)
|
||||||
|
|
||||||
standIn := startServeWAF(t, allow)
|
servewaf := startServeWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||||
|
|
||||||
req := newRequest(t, http.MethodPost, addr, path+"?"+query, strings.NewReader(body))
|
req := newRequest(t, http.MethodPost, addr, path+"?"+query, strings.NewReader(body))
|
||||||
req.Host = site
|
req.Host = site
|
||||||
wantGreeting(t, visit(t, req))
|
wantGreeting(t, visit(t, req))
|
||||||
|
|
||||||
line := out.requestLine(t)
|
line := out.requestLine(t)
|
||||||
asked := standIn.question(t)
|
asked := servewaf.question(t)
|
||||||
|
|
||||||
for name, want := range map[string]any{
|
for name, want := range map[string]any{
|
||||||
"type": "request",
|
"type": "request",
|
||||||
|
|||||||
+11
-11
@@ -11,7 +11,7 @@ import (
|
|||||||
"net/http"
|
"net/http"
|
||||||
"strings"
|
"strings"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
)
|
)
|
||||||
|
|
||||||
// answerBodyMaxBytes is the most of the body of servewaf's answer that is read.
|
// answerBodyMaxBytes is the most of the body of servewaf's answer that is read.
|
||||||
@@ -38,10 +38,10 @@ type servewafAnswer struct {
|
|||||||
// notes nothing, and no report is due for it: servewaf holds no decision to report
|
// notes nothing, and no report is due for it: servewaf holds no decision to report
|
||||||
// on.
|
// on.
|
||||||
func (rq *request) noteAnswer(answer *servewafAnswer) {
|
func (rq *request) noteAnswer(answer *servewafAnswer) {
|
||||||
disposition := answer.header.Get(servewaf.DispositionHeader)
|
disposition := answer.header.Get(proxy.DispositionHeader)
|
||||||
|
|
||||||
switch disposition {
|
switch disposition {
|
||||||
case servewaf.DispositionAllow, servewaf.DispositionDeny, servewaf.DispositionClose:
|
case proxy.DispositionAllow, proxy.DispositionDeny, proxy.DispositionClose:
|
||||||
rq.disposition = disposition
|
rq.disposition = disposition
|
||||||
rq.answered = true
|
rq.answered = true
|
||||||
rq.asn = answer.header.Get(asnHeader)
|
rq.asn = answer.header.Get(asnHeader)
|
||||||
@@ -74,7 +74,7 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
|
|||||||
rq.disposition = rq.h.config.DefaultDisposition
|
rq.disposition = rq.h.config.DefaultDisposition
|
||||||
rq.byDefault = true
|
rq.byDefault = true
|
||||||
|
|
||||||
if rq.disposition == servewaf.DispositionAllow {
|
if rq.disposition == proxy.DispositionAllow {
|
||||||
rq.passToApp(ctx)
|
rq.passToApp(ctx)
|
||||||
} else {
|
} else {
|
||||||
rq.answer(http.StatusServiceUnavailable)
|
rq.answer(http.StatusServiceUnavailable)
|
||||||
@@ -169,13 +169,13 @@ func (rq *request) newQuestion(ctx context.Context) (*http.Request, error) {
|
|||||||
question.TransferEncoding = in.TransferEncoding
|
question.TransferEncoding = in.TransferEncoding
|
||||||
|
|
||||||
question.Header = in.Header.Clone()
|
question.Header = in.Header.Clone()
|
||||||
question.Header.Set(servewaf.ClientHeader, rq.client.String())
|
question.Header.Set(proxy.ClientHeader, rq.client.String())
|
||||||
question.Header.Set(servewaf.PeerHeader, rq.peer.String())
|
question.Header.Set(proxy.PeerHeader, rq.peer.String())
|
||||||
question.Header.Del(servewaf.ContentLengthHeader)
|
question.Header.Del(proxy.ContentLengthHeader)
|
||||||
|
|
||||||
announced := in.Header.Get("Content-Length")
|
announced := in.Header.Get("Content-Length")
|
||||||
if announced != "" {
|
if announced != "" {
|
||||||
question.Header.Set(servewaf.ContentLengthHeader, announced)
|
question.Header.Set(proxy.ContentLengthHeader, announced)
|
||||||
}
|
}
|
||||||
|
|
||||||
question.Header.Set(requestIDHeader, rq.id)
|
question.Header.Set(requestIDHeader, rq.id)
|
||||||
@@ -196,7 +196,7 @@ func (rq *request) answerWith(answer *servewafAnswer) {
|
|||||||
|
|
||||||
header := rq.out.Header()
|
header := rq.out.Header()
|
||||||
maps.Copy(header, answer.header)
|
maps.Copy(header, answer.header)
|
||||||
header.Del(servewaf.DispositionHeader)
|
header.Del(proxy.DispositionHeader)
|
||||||
|
|
||||||
// Go's server adds a Content-Type it guesses from the body to a response whose
|
// Go's server adds a Content-Type it guesses from the body to a response whose
|
||||||
// headers have no Content-Type key. The key with no value sends none.
|
// headers have no Content-Type key. The key with no value sends none.
|
||||||
@@ -210,7 +210,7 @@ func (rq *request) answerWith(answer *servewafAnswer) {
|
|||||||
// sendReport sends servewaf report on a goroutine of its own, so that the
|
// sendReport sends servewaf report on a goroutine of its own, so that the
|
||||||
// visitor does not wait for it. The report goes on after ctx, the request's own
|
// visitor does not wait for it. The report goes on after ctx, the request's own
|
||||||
// context, has ended.
|
// context, has ended.
|
||||||
func (h *handler) sendReport(ctx context.Context, report *servewaf.Report) {
|
func (h *handler) sendReport(ctx context.Context, report *proxy.Report) {
|
||||||
h.reportsMu.Lock()
|
h.reportsMu.Lock()
|
||||||
defer h.reportsMu.Unlock()
|
defer h.reportsMu.Unlock()
|
||||||
|
|
||||||
@@ -228,7 +228,7 @@ func (h *handler) sendReport(ctx context.Context, report *servewaf.Report) {
|
|||||||
|
|
||||||
// postReport posts report to servewaf, and waits for its answer for up to
|
// postReport posts report to servewaf, and waits for its answer for up to
|
||||||
// SWRP_TIMEOUT_MS.
|
// SWRP_TIMEOUT_MS.
|
||||||
func (h *handler) postReport(ctx context.Context, report *servewaf.Report) error {
|
func (h *handler) postReport(ctx context.Context, report *proxy.Report) error {
|
||||||
body, err := json.Marshal(report)
|
body, err := json.Marshal(report)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("encode the report: %w", err)
|
return fmt.Errorf("encode the report: %w", err)
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
)
|
)
|
||||||
|
|
||||||
// healthCheckTimeout bounds the whole health check.
|
// healthCheckTimeout bounds the whole health check.
|
||||||
@@ -107,7 +107,7 @@ func checkRProxyAndApp(
|
|||||||
// listenAddr, which the settings have checked has one.
|
// listenAddr, which the settings have checked has one.
|
||||||
func askHealthEndpoint(ctx context.Context, listenAddr string) error {
|
func askHealthEndpoint(ctx context.Context, listenAddr string) error {
|
||||||
_, port, _ := net.SplitHostPort(listenAddr)
|
_, port, _ := net.SplitHostPort(listenAddr)
|
||||||
health := "http://" + net.JoinHostPort("127.0.0.1", port) + servewaf.HealthPath
|
health := "http://" + net.JoinHostPort("127.0.0.1", port) + proxy.HealthPath
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, health, http.NoBody)
|
req, err := http.NewRequestWithContext(ctx, http.MethodGet, health, http.NoBody)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
@@ -19,7 +19,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
|
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -153,7 +153,7 @@ func TestABanThroughOneSiteRefusesTheClientOnTheOther(t *testing.T) {
|
|||||||
|
|
||||||
// One request's three lines carry one id, rproxy's with servewaf's answer.
|
// One request's three lines carry one id, rproxy's with servewaf's answer.
|
||||||
line := firstOut.line(t, "path", bannedProbe)
|
line := firstOut.line(t, "path", bannedProbe)
|
||||||
if line["disposition"] != servewaf.DispositionDeny || line["client_ip"] != placed {
|
if line["disposition"] != proxy.DispositionDeny || line["client_ip"] != placed {
|
||||||
t.Errorf("rproxy's line is %v", line)
|
t.Errorf("rproxy's line is %v", line)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -225,7 +225,7 @@ func TestTheSiteAnswersWhileServeWAFIsStoppedAndStartedAgain(t *testing.T) {
|
|||||||
wantStatus(t, site+"/after-stop", placed, http.StatusOK)
|
wantStatus(t, site+"/after-stop", placed, http.StatusOK)
|
||||||
|
|
||||||
line := out.line(t, "path", "/after-stop")
|
line := out.line(t, "path", "/after-stop")
|
||||||
if line["disposition"] != servewaf.DispositionAllow ||
|
if line["disposition"] != proxy.DispositionAllow ||
|
||||||
line["default_disposition"] != true {
|
line["default_disposition"] != true {
|
||||||
t.Errorf("rproxy's line is %v", line)
|
t.Errorf("rproxy's line is %v", line)
|
||||||
}
|
}
|
||||||
@@ -318,7 +318,7 @@ func TestAChunkedBodyStaysABodyAtServeWAF(t *testing.T) {
|
|||||||
const smuggledID = "smuggled-request"
|
const smuggledID = "smuggled-request"
|
||||||
|
|
||||||
smuggled := "GET " + bannedProbe + " HTTP/1.1\r\nHost: site.example\r\n" +
|
smuggled := "GET " + bannedProbe + " HTTP/1.1\r\nHost: site.example\r\n" +
|
||||||
servewaf.ClientHeader + ": " + otherClient + "\r\n" +
|
proxy.ClientHeader + ": " + otherClient + "\r\n" +
|
||||||
"X-Request-ID: " + smuggledID + "\r\n\r\n"
|
"X-Request-ID: " + smuggledID + "\r\n\r\n"
|
||||||
|
|
||||||
status := sendRaw(t, site, "POST /upload HTTP/1.1\r\nHost: site.example\r\n"+
|
status := sendRaw(t, site, "POST /upload HTTP/1.1\r\nHost: site.example\r\n"+
|
||||||
@@ -459,10 +459,10 @@ func TestRProxyAsksAServeWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
|
|||||||
protocols := make(chan string, 10)
|
protocols := make(chan string, 10)
|
||||||
backend := httptest.NewUnstartedServer(http.HandlerFunc(
|
backend := httptest.NewUnstartedServer(http.HandlerFunc(
|
||||||
func(w http.ResponseWriter, r *http.Request) {
|
func(w http.ResponseWriter, r *http.Request) {
|
||||||
if r.Header.Get(servewaf.ClientHeader) != "" {
|
if r.Header.Get(proxy.ClientHeader) != "" {
|
||||||
protocols <- r.Proto
|
protocols <- r.Proto
|
||||||
|
|
||||||
w.Header().Set(servewaf.DispositionHeader, servewaf.DispositionDeny)
|
w.Header().Set(proxy.DispositionHeader, proxy.DispositionDeny)
|
||||||
w.WriteHeader(http.StatusForbidden)
|
w.WriteHeader(http.StatusForbidden)
|
||||||
}
|
}
|
||||||
}))
|
}))
|
||||||
@@ -532,8 +532,8 @@ func TestRProxyStoppingWaitsForAReportBeingSent(t *testing.T) {
|
|||||||
release := make(chan struct{})
|
release := make(chan struct{})
|
||||||
backend := httptest.NewServer(http.HandlerFunc(
|
backend := httptest.NewServer(http.HandlerFunc(
|
||||||
func(w http.ResponseWriter, r *http.Request) {
|
func(w http.ResponseWriter, r *http.Request) {
|
||||||
if r.Header.Get(servewaf.ClientHeader) != "" {
|
if r.Header.Get(proxy.ClientHeader) != "" {
|
||||||
w.Header().Set(servewaf.DispositionHeader, servewaf.DispositionAllow)
|
w.Header().Set(proxy.DispositionHeader, proxy.DispositionAllow)
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -640,7 +640,7 @@ func TestRProxyStoppingCutsOffARequestAfterFiveSeconds(t *testing.T) {
|
|||||||
|
|
||||||
backend := httptest.NewServer(http.HandlerFunc(
|
backend := httptest.NewServer(http.HandlerFunc(
|
||||||
func(w http.ResponseWriter, _ *http.Request) {
|
func(w http.ResponseWriter, _ *http.Request) {
|
||||||
w.Header().Set(servewaf.DispositionHeader, servewaf.DispositionAllow)
|
w.Header().Set(proxy.DispositionHeader, proxy.DispositionAllow)
|
||||||
}))
|
}))
|
||||||
t.Cleanup(backend.Close)
|
t.Cleanup(backend.Close)
|
||||||
|
|
||||||
@@ -757,7 +757,7 @@ func TestMainRunsRProxy(t *testing.T) {
|
|||||||
addr, cmd, out := startRProxyProcess(t, wafBackend+"=http://127.0.0.1:9")
|
addr, cmd, out := startRProxyProcess(t, wafBackend+"=http://127.0.0.1:9")
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(t.Context(), http.MethodGet,
|
req, err := http.NewRequestWithContext(t.Context(), http.MethodGet,
|
||||||
"http://"+addr+servewaf.HealthPath, http.NoBody)
|
"http://"+addr+proxy.HealthPath, http.NoBody)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("new request: %v", err)
|
t.Fatalf("new request: %v", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -14,7 +14,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
|
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -67,7 +67,7 @@ func TestMainRunsServeWAF(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
addr, _ := out.line(t, "msg", "starting")["address"].(string)
|
addr, _ := out.line(t, "msg", "starting")["address"].(string)
|
||||||
ask(t, addr, "/", http.StatusOK, servewaf.DispositionAllow)
|
ask(t, addr, "/", http.StatusOK, proxy.DispositionAllow)
|
||||||
|
|
||||||
err = cmd.Process.Signal(syscall.SIGTERM)
|
err = cmd.Process.Signal(syscall.SIGTERM)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -132,7 +132,7 @@ func ask(t *testing.T, addr, path string, status int, disposition string) {
|
|||||||
t.Fatalf("new request: %v", err)
|
t.Fatalf("new request: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
req.Header.Set(servewaf.ClientHeader, placed)
|
req.Header.Set(proxy.ClientHeader, placed)
|
||||||
req.Header.Set("X-Request-ID", rand.Text())
|
req.Header.Set("X-Request-ID", rand.Text())
|
||||||
|
|
||||||
transport := &http.Transport{}
|
transport := &http.Transport{}
|
||||||
@@ -147,9 +147,9 @@ func ask(t *testing.T, addr, path string, status int, disposition string) {
|
|||||||
_ = res.Body.Close()
|
_ = res.Body.Close()
|
||||||
|
|
||||||
if res.StatusCode != status ||
|
if res.StatusCode != status ||
|
||||||
res.Header.Get(servewaf.DispositionHeader) != disposition {
|
res.Header.Get(proxy.DispositionHeader) != disposition {
|
||||||
t.Errorf("answered %d with disposition %q, want %d with %q", res.StatusCode,
|
t.Errorf("answered %d with disposition %q, want %d with %q", res.StatusCode,
|
||||||
res.Header.Get(servewaf.DispositionHeader), status, disposition)
|
res.Header.Get(proxy.DispositionHeader), status, disposition)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -193,12 +193,12 @@ func TestServeWAFKeepsItsBansAcrossARestartAndNeedsNoApp(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
runServeWAF(t, env, func(addr string) {
|
runServeWAF(t, env, func(addr string) {
|
||||||
ask(t, addr, "/", http.StatusOK, servewaf.DispositionAllow)
|
ask(t, addr, "/", http.StatusOK, proxy.DispositionAllow)
|
||||||
// The default rule file bans a probe for this.
|
// The default rule file bans a probe for this.
|
||||||
ask(t, addr, "/.env", http.StatusForbidden, servewaf.DispositionDeny)
|
ask(t, addr, "/.env", http.StatusForbidden, proxy.DispositionDeny)
|
||||||
})
|
})
|
||||||
|
|
||||||
runServeWAF(t, env, func(addr string) {
|
runServeWAF(t, env, func(addr string) {
|
||||||
ask(t, addr, "/", http.StatusForbidden, servewaf.DispositionDeny)
|
ask(t, addr, "/", http.StatusForbidden, proxy.DispositionDeny)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -23,13 +23,12 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/remotelog"
|
"sneak.berlin/go/smallwebwaf/internal/remotelog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/state"
|
"sneak.berlin/go/smallwebwaf/internal/state"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// shutdownTimeout is how long requests in progress may take to finish
|
// shutdownTimeout is how long requests in progress may take to finish
|
||||||
@@ -151,14 +150,7 @@ func RunServeWAF(ctx context.Context, params Params) int {
|
|||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
|
|
||||||
crs, err := servewaf.NewCoreRuleSet(cfg)
|
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue)
|
||||||
if err != nil {
|
|
||||||
processLog.Error("cannot use the Core Rule Set", "error", err.Error())
|
|
||||||
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
|
|
||||||
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue, crs)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("cannot use the lookup database", "error", err.Error())
|
processLog.Error("cannot use the lookup database", "error", err.Error())
|
||||||
|
|
||||||
@@ -178,7 +170,8 @@ func RunServeWAF(ctx context.Context, params Params) int {
|
|||||||
|
|
||||||
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
|
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("cannot listen on SWWAF_LISTEN_ADDR", "error", err.Error())
|
processLog.Error("cannot listen on SWWAF_LISTEN_ADDR",
|
||||||
|
"error", err.Error())
|
||||||
|
|
||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
@@ -197,8 +190,7 @@ func RunServeWAF(ctx context.Context, params Params) int {
|
|||||||
func newServer(
|
func newServer(
|
||||||
cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
|
cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
|
||||||
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
|
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
|
||||||
crs *waf.CoreRuleSet,
|
) (*proxy.Server, error) {
|
||||||
) (*servewaf.Server, error) {
|
|
||||||
var lookupFile *lookup.File
|
var lookupFile *lookup.File
|
||||||
|
|
||||||
if cfg.LookupSource == "file" {
|
if cfg.LookupSource == "file" {
|
||||||
@@ -215,14 +207,13 @@ func newServer(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
server := servewaf.New(servewaf.Params{
|
server := proxy.NewServeWAF(proxy.Params{
|
||||||
Config: cfg,
|
Config: cfg,
|
||||||
RequestLog: stdout,
|
RequestLog: stdout,
|
||||||
ProcessLog: processLog,
|
ProcessLog: processLog,
|
||||||
GeoJSURL: lookup.URL,
|
GeoJSURL: lookup.URL,
|
||||||
AbuseIPDBURL: reputation.AbuseIPDBURL,
|
AbuseIPDBURL: reputation.AbuseIPDBURL,
|
||||||
LookupFile: lookupFile,
|
LookupFile: lookupFile,
|
||||||
CoreRuleSet: crs,
|
|
||||||
Now: now,
|
Now: now,
|
||||||
Rules: ruleFiles,
|
Rules: ruleFiles,
|
||||||
Alerts: alertQueue,
|
Alerts: alertQueue,
|
||||||
@@ -239,7 +230,7 @@ func newServer(
|
|||||||
// loadStateFiles reads the state files into the parts of server and into
|
// loadStateFiles reads the state files into the parts of server and into
|
||||||
// alertQueue, as state.Load does.
|
// alertQueue, as state.Load does.
|
||||||
func loadStateFiles(
|
func loadStateFiles(
|
||||||
cfg *config.Config, server *servewaf.Server, alertQueue *alerts.Queue,
|
cfg *config.Config, server *proxy.Server, alertQueue *alerts.Queue,
|
||||||
now func() time.Time, processLog *slog.Logger,
|
now func() time.Time, processLog *slog.Logger,
|
||||||
) (*state.Files, error) {
|
) (*state.Files, error) {
|
||||||
return state.Load(state.Params{
|
return state.Load(state.Params{
|
||||||
@@ -325,7 +316,7 @@ func startSending(
|
|||||||
// shutdownTimeout to finish, and writes every state file, alerts.json with
|
// shutdownTimeout to finish, and writes every state file, alerts.json with
|
||||||
// the alerts still waiting.
|
// the alerts still waiting.
|
||||||
func serve(
|
func serve(
|
||||||
ctx context.Context, server *servewaf.Server, listener net.Listener,
|
ctx context.Context, server *proxy.Server, listener net.Listener,
|
||||||
files *state.Files, ruleFiles *rules.Files, alertQueue *alerts.Queue,
|
files *state.Files, ruleFiles *rules.Files, alertQueue *alerts.Queue,
|
||||||
processLog *slog.Logger,
|
processLog *slog.Logger,
|
||||||
) int {
|
) int {
|
||||||
|
|||||||
@@ -19,9 +19,8 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
"sneak.berlin/go/smallwebwaf/internal/proxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
|
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -552,56 +551,6 @@ func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
|
|||||||
": no such file or directory")
|
": no such file or directory")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
path := filepath.Join(t.TempDir(), "crs-exclusions.conf")
|
|
||||||
out := &output{}
|
|
||||||
|
|
||||||
status := run(t.Context(), map[string]string{
|
|
||||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
|
||||||
"SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
|
||||||
}, out)
|
|
||||||
if status != 1 {
|
|
||||||
t.Fatalf("exit status %d, want 1", status)
|
|
||||||
}
|
|
||||||
|
|
||||||
line := out.line(t, "msg", "cannot use the Core Rule Set")
|
|
||||||
want := "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: read the exclusions file: open " +
|
|
||||||
path + ": no such file or directory"
|
|
||||||
|
|
||||||
if line["error"] != want {
|
|
||||||
t.Errorf("start refused with %v, want the error %q", line, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCoreRuleSetExclusionsFileThatCannotBeUsedStopsTheStartWhileTheCoreRuleSetIsOff(
|
|
||||||
t *testing.T,
|
|
||||||
) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
broken := filepath.Join(t.TempDir(), "broken.conf")
|
|
||||||
|
|
||||||
err := os.WriteFile(broken, []byte("SecNoSuchDirective On\n"), 0o600)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, path := range []string{filepath.Join(t.TempDir(), "missing.conf"), broken} {
|
|
||||||
out := &output{}
|
|
||||||
|
|
||||||
status := run(t.Context(), map[string]string{
|
|
||||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
|
||||||
"SWWAF_WAF_MODE": config.WAFModeOff, "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
|
||||||
}, out)
|
|
||||||
if status != 1 {
|
|
||||||
t.Fatalf("%s: exit status %d, want 1", path, status)
|
|
||||||
}
|
|
||||||
|
|
||||||
out.line(t, "msg", "cannot use the Core Rule Set")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
|
func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -1276,9 +1225,9 @@ func wantStartingLine(t *testing.T, line map[string]any, dir string) {
|
|||||||
"SWWAF_ALLOW_NETS": "",
|
"SWWAF_ALLOW_NETS": "",
|
||||||
rateLimitExemptNets: "",
|
rateLimitExemptNets: "",
|
||||||
"SWWAF_DENY_NETS": "",
|
"SWWAF_DENY_NETS": "",
|
||||||
"SWWAF_RATE_LIMIT_PER_MINUTE": "120",
|
"SWWAF_RATE_LIMIT_PER_MINUTE": "500",
|
||||||
"SWWAF_RATE_LIMIT_PER_HOUR": "1500",
|
"SWWAF_RATE_LIMIT_PER_HOUR": "20000",
|
||||||
rateLimitPerDay: "6000",
|
rateLimitPerDay: "300000",
|
||||||
"SWWAF_RATE_LIMIT_EXEMPT_PATHS": "",
|
"SWWAF_RATE_LIMIT_EXEMPT_PATHS": "",
|
||||||
"SWWAF_DENIED_COUNTRIES": "",
|
"SWWAF_DENIED_COUNTRIES": "",
|
||||||
"SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES": "",
|
"SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES": "",
|
||||||
@@ -1678,7 +1627,7 @@ func statusFrom(t *testing.T, url, from string) int {
|
|||||||
}
|
}
|
||||||
|
|
||||||
req.Header.Set("X-Forwarded-For", from)
|
req.Header.Set("X-Forwarded-For", from)
|
||||||
req.Header.Set(servewaf.ClientHeader, from)
|
req.Header.Set(proxy.ClientHeader, from)
|
||||||
req.Header.Set("X-Request-ID", rand.Text())
|
req.Header.Set("X-Request-ID", rand.Text())
|
||||||
|
|
||||||
transport := &http.Transport{}
|
transport := &http.Transport{}
|
||||||
|
|||||||
-337
@@ -1,337 +0,0 @@
|
|||||||
# Ordinary gitea requests that the Core Rule Set must let through with
|
|
||||||
# deploy/gitea/crs-exclusions.conf. Each line is a path and query, and may end
|
|
||||||
# with one header after a space. The tests of internal/waf inspect each as a GET,
|
|
||||||
# and script/gitea-setup sends each to gitea through the setup as a GET. The Core
|
|
||||||
# Rule Set reads the query and the headers the same way for any method.
|
|
||||||
|
|
||||||
# Searches with ordinary words and symbols.
|
|
||||||
/owner/repo/search?q=package.json
|
|
||||||
/owner/repo/search?q=.gitignore
|
|
||||||
/owner/repo/search?q=docker-compose.yml
|
|
||||||
/owner/repo/search?q=python3
|
|
||||||
/owner/repo/search?q=ssh%20key
|
|
||||||
/owner/repo/search?q=feat%2Fdocker-support
|
|
||||||
/owner/repo/search?q=ls%20-la
|
|
||||||
/owner/repo/search?q=sed%20-i
|
|
||||||
/owner/repo/search?q=%2Fbin%2Fsh
|
|
||||||
/owner/repo/search?q=%60make%20check%60
|
|
||||||
/owner/repo/search?q=%60whoami%60
|
|
||||||
/owner/repo/search?q=fetch%28
|
|
||||||
/owner/repo/search?q=%24%7BVAR%7D
|
|
||||||
/owner/repo/search?q=%24%28pwd%29
|
|
||||||
/owner/repo/search?q=process.env
|
|
||||||
/owner/repo/search?q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/search?q=%3Cscript%3E
|
|
||||||
/owner/repo/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/search?q=union%20select
|
|
||||||
/owner/repo/search?q=http%3A%2F%2F127.0.0.1%3A8080
|
|
||||||
/owner/repo/search?q=..%2F
|
|
||||||
/owner/repo/search?q=%2Fetc%2Fpasswd
|
|
||||||
/owner/repo/search?q=eval%28
|
|
||||||
/owner/repo/search?q=chmod%20%2Bx
|
|
||||||
/owner/repo/search?q=echo%20%24PATH
|
|
||||||
/owner/repo/search?q=java.lang.Runtime
|
|
||||||
/owner/repo/search?q=document.cookie
|
|
||||||
/owner/repo/search?q=alert%281%29
|
|
||||||
/owner/repo/search?q=__proto__
|
|
||||||
/owner/repo/search?q=C%3A%5CWindows%5CSystem32
|
|
||||||
/owner/repo/search?q=Invoke-WebRequest
|
|
||||||
/owner/repo/search?q=if%20%28a%20%3D%3D%20b%29%20%7B
|
|
||||||
/owner/repo/search?q=sudo%20apt%20install
|
|
||||||
/owner/repo/search?q=~%2F.ssh%2Fconfig
|
|
||||||
/owner/repo/search?q=.htaccess
|
|
||||||
/owner/repo/search?q=%24_GET%5B
|
|
||||||
/owner/repo/search?q=%24_POST
|
|
||||||
/owner/repo/search?q=%24_SERVER%5B%27REQUEST_URI%27%5D
|
|
||||||
/owner/repo/search?q=%3C%3Fphp%20echo
|
|
||||||
/owner/repo/search?q=information_schema
|
|
||||||
/owner/repo/search?q=LIKE%20%27%25foo%25%27
|
|
||||||
/owner/repo/search?q=onClick%3D%7B
|
|
||||||
/explore/repos?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/explore/repos?q=%3Cimg%20src%3D
|
|
||||||
/explore/code?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/explore/code?q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/issues?type=all&state=open&q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/issues?type=all&state=open&q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/pulls?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/pulls?q=%3Cimg%20src%3D
|
|
||||||
/issues?type=your_repositories&q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/issues?type=your_repositories&q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/commits/branch/main/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/commits/branch/main/search?q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/wiki/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/wiki/search?q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/branches?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/branches?q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/releases?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/releases?q=%3Cimg%20src%3D
|
|
||||||
/owner/-/packages?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/-/packages?q=%3Cimg%20src%3D
|
|
||||||
/user/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/user/search?q=%3Cimg%20src%3D
|
|
||||||
/api/v1/repos/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/api/v1/repos/search?q=%3Cimg%20src%3D
|
|
||||||
/api/v1/repos/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/api/v1/repos/issues/search?q=%3Cimg%20src%3D
|
|
||||||
/api/v1/repos/owner/repo/issues?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/api/v1/repos/owner/repo/issues?q=%3Cimg%20src%3D
|
|
||||||
|
|
||||||
# Pages, raw files and edit pages of files, and the API's call for their
|
|
||||||
# contents.
|
|
||||||
/owner/repo/src/branch/main/package.json
|
|
||||||
/owner/repo/raw/branch/main/package.json
|
|
||||||
/owner/repo/_edit/main/package.json
|
|
||||||
/api/v1/repos/owner/repo/contents/package.json
|
|
||||||
/owner/repo/src/branch/main/.gitignore
|
|
||||||
/owner/repo/raw/branch/main/.gitignore
|
|
||||||
/owner/repo/_edit/main/.gitignore
|
|
||||||
/api/v1/repos/owner/repo/contents/.gitignore
|
|
||||||
/owner/repo/src/branch/main/docker-compose.yml
|
|
||||||
/owner/repo/raw/branch/main/docker-compose.yml
|
|
||||||
/owner/repo/_edit/main/docker-compose.yml
|
|
||||||
/api/v1/repos/owner/repo/contents/docker-compose.yml
|
|
||||||
/owner/repo/src/branch/main/Dockerfile
|
|
||||||
/owner/repo/raw/branch/main/Dockerfile
|
|
||||||
/owner/repo/_edit/main/Dockerfile
|
|
||||||
/api/v1/repos/owner/repo/contents/Dockerfile
|
|
||||||
/owner/repo/src/branch/main/.htaccess
|
|
||||||
/owner/repo/raw/branch/main/.htaccess
|
|
||||||
/owner/repo/_edit/main/.htaccess
|
|
||||||
/api/v1/repos/owner/repo/contents/.htaccess
|
|
||||||
/owner/repo/src/branch/main/rootfs/etc/passwd
|
|
||||||
/owner/repo/raw/branch/main/rootfs/etc/passwd
|
|
||||||
/owner/repo/_edit/main/rootfs/etc/passwd
|
|
||||||
/api/v1/repos/owner/repo/contents/rootfs/etc/passwd
|
|
||||||
/owner/repo/src/branch/main/bin/docker-entrypoint
|
|
||||||
/owner/repo/raw/branch/main/bin/docker-entrypoint
|
|
||||||
/owner/repo/_edit/main/bin/docker-entrypoint
|
|
||||||
/api/v1/repos/owner/repo/contents/bin/docker-entrypoint
|
|
||||||
/owner/repo/src/branch/main/scripts/python3-setup.sh
|
|
||||||
/owner/repo/raw/branch/main/scripts/python3-setup.sh
|
|
||||||
/owner/repo/_edit/main/scripts/python3-setup.sh
|
|
||||||
/api/v1/repos/owner/repo/contents/scripts/python3-setup.sh
|
|
||||||
/owner/repo/src/branch/main/src/java.base/share/classes/java/lang/Runtime.java
|
|
||||||
/owner/repo/raw/branch/main/src/java.base/share/classes/java/lang/Runtime.java
|
|
||||||
/owner/repo/_edit/main/src/java.base/share/classes/java/lang/Runtime.java
|
|
||||||
/api/v1/repos/owner/repo/contents/src/java.base/share/classes/java/lang/Runtime.java
|
|
||||||
/owner/repo/src/branch/main/java.lang.String.md
|
|
||||||
/owner/repo/raw/branch/main/java.lang.String.md
|
|
||||||
/owner/repo/_edit/main/java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/contents/java.lang.String.md
|
|
||||||
/owner/repo/src/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
|
|
||||||
/owner/repo/raw/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
|
|
||||||
/owner/repo/_edit/main/META-INF/services/org.apache.commons.logging.LogFactory
|
|
||||||
/api/v1/repos/owner/repo/contents/META-INF/services/org.apache.commons.logging.LogFactory
|
|
||||||
/owner/repo/src/branch/main/process.env.d.ts
|
|
||||||
/owner/repo/raw/branch/main/process.env.d.ts
|
|
||||||
/owner/repo/_edit/main/process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/contents/process.env.d.ts
|
|
||||||
/owner/repo/src/branch/main/document.write.js
|
|
||||||
/owner/repo/raw/branch/main/document.write.js
|
|
||||||
/owner/repo/_edit/main/document.write.js
|
|
||||||
/api/v1/repos/owner/repo/contents/document.write.js
|
|
||||||
/owner/repo/src/branch/main/test/fixtures/__proto__.json
|
|
||||||
/owner/repo/raw/branch/main/test/fixtures/__proto__.json
|
|
||||||
/owner/repo/_edit/main/test/fixtures/__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/contents/test/fixtures/__proto__.json
|
|
||||||
/owner/repo/src/branch/main/install-script.sh
|
|
||||||
/owner/repo/raw/branch/main/install-script.sh
|
|
||||||
/owner/repo/_edit/main/install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/contents/install-script.sh
|
|
||||||
/owner/repo/src/branch/main/src/backend/catalog/information_schema.sql
|
|
||||||
/owner/repo/raw/branch/main/src/backend/catalog/information_schema.sql
|
|
||||||
/owner/repo/_edit/main/src/backend/catalog/information_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/contents/src/backend/catalog/information_schema.sql
|
|
||||||
/owner/repo/src/branch/main/docs/a%20b.xhtml
|
|
||||||
/owner/repo/raw/branch/main/docs/a%20b.xhtml
|
|
||||||
/owner/repo/_edit/main/docs/a%20b.xhtml
|
|
||||||
/api/v1/repos/owner/repo/contents/docs/a%20b.xhtml
|
|
||||||
/owner/repo/src/branch/main/%5Bdraft%5D.md
|
|
||||||
/owner/repo/raw/branch/main/%5Bdraft%5D.md
|
|
||||||
/owner/repo/_edit/main/%5Bdraft%5D.md
|
|
||||||
/api/v1/repos/owner/repo/contents/%5Bdraft%5D.md
|
|
||||||
/owner/repo/src/branch/main/notes.txt~
|
|
||||||
/owner/repo/raw/branch/main/notes.txt~
|
|
||||||
/owner/repo/_edit/main/notes.txt~
|
|
||||||
/api/v1/repos/owner/repo/contents/notes.txt~
|
|
||||||
|
|
||||||
# File names in the query string: the commit graph of a file, the API's lists
|
|
||||||
# of its commits and of its contents, an issue opened from a template, and the
|
|
||||||
# sign-in page on the way to its edit page.
|
|
||||||
/owner/repo/graph?file=package.json
|
|
||||||
/api/v1/repos/owner/repo/commits?path=package.json
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22package.json%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fpackage.json
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fpackage.json
|
|
||||||
/owner/repo/graph?file=.gitignore
|
|
||||||
/api/v1/repos/owner/repo/commits?path=.gitignore
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.gitignore%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.gitignore
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.gitignore
|
|
||||||
/owner/repo/graph?file=docker-compose.yml
|
|
||||||
/api/v1/repos/owner/repo/commits?path=docker-compose.yml
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docker-compose.yml%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocker-compose.yml
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocker-compose.yml
|
|
||||||
/owner/repo/graph?file=.htaccess
|
|
||||||
/api/v1/repos/owner/repo/commits?path=.htaccess
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.htaccess%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.htaccess
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.htaccess
|
|
||||||
/owner/repo/graph?file=rootfs%2Fetc%2Fpasswd
|
|
||||||
/api/v1/repos/owner/repo/commits?path=rootfs%2Fetc%2Fpasswd
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22rootfs%2Fetc%2Fpasswd%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Frootfs%2Fetc%2Fpasswd
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Frootfs%2Fetc%2Fpasswd
|
|
||||||
/owner/repo/graph?file=bin%2Fdocker-entrypoint
|
|
||||||
/api/v1/repos/owner/repo/commits?path=bin%2Fdocker-entrypoint
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22bin%2Fdocker-entrypoint%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fbin%2Fdocker-entrypoint
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fbin%2Fdocker-entrypoint
|
|
||||||
/owner/repo/graph?file=scripts%2Fpython3-setup.sh
|
|
||||||
/api/v1/repos/owner/repo/commits?path=scripts%2Fpython3-setup.sh
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22scripts%2Fpython3-setup.sh%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fscripts%2Fpython3-setup.sh
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fscripts%2Fpython3-setup.sh
|
|
||||||
/owner/repo/graph?file=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
|
||||||
/api/v1/repos/owner/repo/commits?path=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
|
||||||
/owner/repo/graph?file=java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/commits?path=java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22java.lang.String.md%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fjava.lang.String.md
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fjava.lang.String.md
|
|
||||||
/owner/repo/graph?file=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
|
||||||
/api/v1/repos/owner/repo/commits?path=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
|
||||||
/owner/repo/graph?file=process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/commits?path=process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22process.env.d.ts%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fprocess.env.d.ts
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fprocess.env.d.ts
|
|
||||||
/owner/repo/graph?file=document.write.js
|
|
||||||
/api/v1/repos/owner/repo/commits?path=document.write.js
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22document.write.js%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocument.write.js
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocument.write.js
|
|
||||||
/owner/repo/graph?file=test%2Ffixtures%2F__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/commits?path=test%2Ffixtures%2F__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22test%2Ffixtures%2F__proto__.json%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Ftest%2Ffixtures%2F__proto__.json
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Ftest%2Ffixtures%2F__proto__.json
|
|
||||||
/owner/repo/graph?file=install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/commits?path=install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22install-script.sh%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Finstall-script.sh
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Finstall-script.sh
|
|
||||||
/owner/repo/graph?file=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/commits?path=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fbackend%2Fcatalog%2Finformation_schema.sql%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
|
||||||
/owner/repo/graph?file=docs%2Fa%20b.xhtml
|
|
||||||
/api/v1/repos/owner/repo/commits?path=docs%2Fa%20b.xhtml
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docs%2Fa%20b.xhtml%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocs%2Fa%20b.xhtml
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocs%2Fa%20b.xhtml
|
|
||||||
/owner/repo/graph?file=%5Bdraft%5D.md
|
|
||||||
/api/v1/repos/owner/repo/commits?path=%5Bdraft%5D.md
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22%5Bdraft%5D.md%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F%5Bdraft%5D.md
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F%5Bdraft%5D.md
|
|
||||||
|
|
||||||
# File names of attachments uploaded through the API, to a release, an issue and
|
|
||||||
# a comment.
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=package.json
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=.gitignore
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=docker-compose.yml
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=.htaccess
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=docker-entrypoint
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=python3-setup.sh
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=Runtime.java
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=org.apache.commons.logging.LogFactory
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=%5Bdraft%5D.md
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=notes.txt~
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=information_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=information_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=information_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=document.write.js
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=document.write.js
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=document.write.js
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=index.xhtml
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=index.xhtml
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=index.xhtml
|
|
||||||
|
|
||||||
# An artifact upload from actions/upload-artifact@v3, which sends Content-Range.
|
|
||||||
/api/actions_pipeline/_apis/pipelines/workflows/1/artifacts/9b7c1f3a/upload?itemPath=artifact%2Freport.txt Content-Range: bytes 0-9/10
|
|
||||||
|
|
||||||
# Branch and tag names in the query string: deleting a branch, starting a
|
|
||||||
# release from a tag, a branch protection rule, a directory listing's last
|
|
||||||
# commits, and a container image push that reuses a layer.
|
|
||||||
/owner/repo/branches?name=docker-build
|
|
||||||
/owner/repo/releases/new?tag=docker-build
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=docker-build
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fdocker-build
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fdocker-build
|
|
||||||
/owner/repo/branches?name=feat%2Fdocker-support
|
|
||||||
/owner/repo/releases/new?tag=feat%2Fdocker-support
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=feat%2Fdocker-support
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Ffeat%2Fdocker-support
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Ffeat%2Fdocker-support
|
|
||||||
/owner/repo/branches?name=renovate%2Fdocker-build-push-action-6.x
|
|
||||||
/owner/repo/releases/new?tag=renovate%2Fdocker-build-push-action-6.x
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=renovate%2Fdocker-build-push-action-6.x
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Frenovate%2Fdocker-build-push-action-6.x
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Frenovate%2Fdocker-build-push-action-6.x
|
|
||||||
/owner/repo/branches?name=python3-migration
|
|
||||||
/owner/repo/releases/new?tag=python3-migration
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=python3-migration
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fpython3-migration
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fpython3-migration
|
|
||||||
/owner/repo/branches?name=ansible-roles
|
|
||||||
/owner/repo/releases/new?tag=ansible-roles
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=ansible-roles
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fansible-roles
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fansible-roles
|
|
||||||
/owner/repo/branches?name=whoami
|
|
||||||
/owner/repo/releases/new?tag=whoami
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=whoami
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fwhoami
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fwhoami
|
|
||||||
|
|
||||||
# Pages a signed-in user opens.
|
|
||||||
/
|
|
||||||
/owner/repo
|
|
||||||
/owner/repo/issues
|
|
||||||
/owner/repo/pulls
|
|
||||||
/owner/repo/releases
|
|
||||||
/owner/repo/wiki
|
|
||||||
/owner/repo/activity
|
|
||||||
/owner/repo/branches
|
|
||||||
/owner/repo/tags
|
|
||||||
/owner/repo/settings
|
|
||||||
/owner/repo/actions
|
|
||||||
/owner/repo/compare/main...main
|
|
||||||
/owner/repo/commits/branch/main
|
|
||||||
/owner/repo/archive/main.zip
|
|
||||||
/notifications
|
|
||||||
/user/settings
|
|
||||||
/user/settings/applications
|
|
||||||
/explore/repos
|
|
||||||
/-/admin
|
|
||||||
+13
-94
@@ -1,9 +1,8 @@
|
|||||||
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
||||||
// method, the URL with its query and the headers of a request, and on its
|
// method, the URL with its query and the headers of a request, and on its
|
||||||
// body while SWWAF_WAF_BODY_LIMIT is set, with the seven changes smallwebwaf
|
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
|
||||||
// makes to it and the exclusions in SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, as
|
// makes to it, as "Attack detection" under "Settings" in SPEC.md describes
|
||||||
// "Attack detection" under "Settings" in SPEC.md describes them. It reads no
|
// them. It reads no response.
|
||||||
// response.
|
|
||||||
//
|
//
|
||||||
// smallwebwaf writes only to its state directory, so Coraza is built with
|
// smallwebwaf writes only to its state directory, so Coraza is built with
|
||||||
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
||||||
@@ -12,13 +11,10 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"errors"
|
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"os"
|
|
||||||
"regexp"
|
|
||||||
"slices"
|
"slices"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -30,13 +26,12 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// directives are the Core Rule Set as smallwebwaf runs it, with the
|
// directives are the Core Rule Set as smallwebwaf runs it, with the
|
||||||
// paranoia level for %d, bodyDirectives for the first %s while
|
// paranoia level for %d, and bodyDirectives for %s while
|
||||||
// SWWAF_WAF_BODY_LIMIT is set, and the exclusions file for the second.
|
// SWWAF_WAF_BODY_LIMIT is set. Each rule smallwebwaf adds has an id from
|
||||||
// Each rule smallwebwaf adds has an id from 900000 to 900999, the ids the
|
// 900000 to 900999, the ids the Core Rule Set keeps for the rules that set
|
||||||
// Core Rule Set keeps for the rules that set it up, which
|
// it up, which SWWAF_WAF_DISABLED_RULES refuses, so that no setting
|
||||||
// SWWAF_WAF_DISABLED_RULES and the exclusions file refuse, so that no
|
// switches one off. Coraza joins a line ending in \ to the next, without
|
||||||
// setting switches one off. Coraza joins a line ending in \ to the next,
|
// the spaces at the start of the next.
|
||||||
// without the spaces at the start of the next.
|
|
||||||
const directives = `
|
const directives = `
|
||||||
# The engine only detects. smallwebwaf compares the request's anomaly
|
# The engine only detects. smallwebwaf compares the request's anomaly
|
||||||
# score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
|
# score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
|
||||||
@@ -74,11 +69,6 @@ SecArgumentsLimit 1000
|
|||||||
SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
|
SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
|
||||||
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
||||||
|
|
||||||
# The exclusions file comes before the rules, where the Core Rule Set puts
|
|
||||||
# its own file of exclusions that run before the rules, so that they run
|
|
||||||
# before the rules of phase 1 too.
|
|
||||||
%s
|
|
||||||
|
|
||||||
# The sixth: only the rules for requests are loaded, and no response is
|
# The sixth: only the rules for requests are loaded, and no response is
|
||||||
# inspected.
|
# inspected.
|
||||||
Include @owasp_crs/REQUEST-*.conf
|
Include @owasp_crs/REQUEST-*.conf
|
||||||
@@ -110,13 +100,6 @@ SecRuleUpdateTargetById 932260 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
|
|||||||
# Inspect.
|
# Inspect.
|
||||||
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
|
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
|
||||||
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
|
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
|
||||||
|
|
||||||
# The seventh: a container image push that reuses a layer names the image
|
|
||||||
# it takes the layer from in the query parameter from, which is not
|
|
||||||
# checked for command names. 932260 refused the push of a layer from
|
|
||||||
# owner/python3-app.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/v2/.+/blobs/uploads/$" \
|
|
||||||
"id:900500,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:from"
|
|
||||||
`
|
`
|
||||||
|
|
||||||
// bodyDirectives have the Core Rule Set read the part of a request body
|
// bodyDirectives have the Core Rule Set read the part of a request body
|
||||||
@@ -177,51 +160,6 @@ type Params struct {
|
|||||||
// BodyLimit is the most of a request body the Core Rule Set reads
|
// BodyLimit is the most of a request body the Core Rule Set reads
|
||||||
// (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
// (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
||||||
BodyLimit int64
|
BodyLimit int64
|
||||||
// ExclusionsFile is the path of a file of exclusions loaded before the
|
|
||||||
// Core Rule Set's rules (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
|
|
||||||
// there is none.
|
|
||||||
ExclusionsFile string
|
|
||||||
}
|
|
||||||
|
|
||||||
// The ids of smallwebwaf's own rules, which an exclusions file may not
|
|
||||||
// name.
|
|
||||||
const (
|
|
||||||
firstOwnRuleID = 900000
|
|
||||||
lastOwnRuleID = 900999
|
|
||||||
)
|
|
||||||
|
|
||||||
// numberOrRange matches a number, or a range of numbers such as
|
|
||||||
// 900000-900999.
|
|
||||||
var numberOrRange = regexp.MustCompile(`(\d+)(?:-(\d+))?`)
|
|
||||||
|
|
||||||
// errSmallwebwafRuleID is the error of an exclusions file that names an
|
|
||||||
// id of smallwebwaf's own rules.
|
|
||||||
var errSmallwebwafRuleID = errors.New("an id of smallwebwaf's own rules")
|
|
||||||
|
|
||||||
// ownRuleNamed returns the first number or range in exclusions, the text
|
|
||||||
// of an exclusions file, that takes in an id of smallwebwaf's own rules,
|
|
||||||
// or "" if none does. Comment lines are skipped, as Coraza skips them.
|
|
||||||
func ownRuleNamed(exclusions string) string {
|
|
||||||
for line := range strings.Lines(exclusions) {
|
|
||||||
if strings.HasPrefix(strings.TrimSpace(line), "#") {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, match := range numberOrRange.FindAllStringSubmatch(line, -1) {
|
|
||||||
first, _ := strconv.Atoi(match[1])
|
|
||||||
|
|
||||||
last := first
|
|
||||||
if match[2] != "" {
|
|
||||||
last, _ = strconv.Atoi(match[2])
|
|
||||||
}
|
|
||||||
|
|
||||||
if first <= lastOwnRuleID && last >= firstOwnRuleID {
|
|
||||||
return match[0]
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return ""
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
|
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
|
||||||
@@ -231,35 +169,16 @@ type CoreRuleSet struct {
|
|||||||
bodyLimit int64
|
bodyLimit int64
|
||||||
}
|
}
|
||||||
|
|
||||||
// New returns the Core Rule Set with the seven changes, at params'
|
// New returns the Core Rule Set with the six changes, at params'
|
||||||
// paranoia level, without the rules it switches off, reading request
|
// paranoia level, without the rules it switches off, and reading request
|
||||||
// bodies up to params' limit, and with params' exclusions file loaded
|
// bodies up to params' limit.
|
||||||
// before the Core Rule Set's rules. An exclusions file that cannot be
|
|
||||||
// read, does not parse or names an id of smallwebwaf's own rules is an
|
|
||||||
// error.
|
|
||||||
func New(params Params) (*CoreRuleSet, error) {
|
func New(params Params) (*CoreRuleSet, error) {
|
||||||
body := ""
|
body := ""
|
||||||
if params.BodyLimit > 0 {
|
if params.BodyLimit > 0 {
|
||||||
body = fmt.Sprintf(bodyDirectives, params.BodyLimit)
|
body = fmt.Sprintf(bodyDirectives, params.BodyLimit)
|
||||||
}
|
}
|
||||||
|
|
||||||
exclusions := ""
|
text := fmt.Sprintf(directives, params.ParanoiaLevel, body)
|
||||||
|
|
||||||
if params.ExclusionsFile != "" {
|
|
||||||
file, err := os.ReadFile(params.ExclusionsFile)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("read the exclusions file: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
exclusions = string(file)
|
|
||||||
|
|
||||||
if named := ownRuleNamed(exclusions); named != "" {
|
|
||||||
return nil, fmt.Errorf("the exclusions file names %s: %w", named,
|
|
||||||
errSmallwebwafRuleID)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions)
|
|
||||||
|
|
||||||
if len(params.DisabledRules) > 0 {
|
if len(params.DisabledRules) > 0 {
|
||||||
ids := make([]string, len(params.DisabledRules))
|
ids := make([]string, len(params.DisabledRules))
|
||||||
|
|||||||
+1
-163
@@ -5,7 +5,6 @@ import (
|
|||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"net/url"
|
"net/url"
|
||||||
"os"
|
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"reflect"
|
"reflect"
|
||||||
"strconv"
|
"strconv"
|
||||||
@@ -120,7 +119,7 @@ func atDefaults(t *testing.T) *waf.CoreRuleSet {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// wantChange checks that crs lets through passes, a gitea request one of
|
// wantChange checks that crs lets through passes, a gitea request one of
|
||||||
// the seven changes is for, and still finds result in refused, a request
|
// the six changes is for, and still finds result in refused, a request
|
||||||
// like it that the change is not for.
|
// like it that the change is not for.
|
||||||
func wantChange(
|
func wantChange(
|
||||||
t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result,
|
t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result,
|
||||||
@@ -363,167 +362,6 @@ func TestEachDisabledRuleIsSwitchedOff(t *testing.T) {
|
|||||||
wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{})
|
wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestImagePushThatReusesALayerIsNotCheckedForCommandNames(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
crs := atDefaults(t)
|
|
||||||
reuse := "?mount=sha256:0000&from=owner/python3-app"
|
|
||||||
|
|
||||||
wantChange(t, crs, get("/v2/owner/app/blobs/uploads/"+reuse), get("/upload/"+reuse),
|
|
||||||
matched(932260))
|
|
||||||
|
|
||||||
// A command after the image's name is still refused.
|
|
||||||
wantResult(t, crs,
|
|
||||||
get("/v2/owner/app/blobs/uploads/?from=owner/app;cat%20/etc/passwd"),
|
|
||||||
matched(930120, 932160))
|
|
||||||
}
|
|
||||||
|
|
||||||
// withExclusions returns the Core Rule Set at the defaults, with the
|
|
||||||
// exclusions file at path.
|
|
||||||
func withExclusions(path string) (*waf.CoreRuleSet, error) {
|
|
||||||
return waf.New(waf.Params{
|
|
||||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, ExclusionsFile: path,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// exclusionsFile writes text to a file of exclusions and returns its path.
|
|
||||||
func exclusionsFile(t *testing.T, text string) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
path := filepath.Join(t.TempDir(), "exclusions.conf")
|
|
||||||
|
|
||||||
err := os.WriteFile(path, []byte(text), 0o600)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return path
|
|
||||||
}
|
|
||||||
|
|
||||||
// backupFileExclusion is an exclusions file that switches 920500, a rule
|
|
||||||
// of phase 1, off for the path of a backup file under /files/.
|
|
||||||
const backupFileExclusion = `SecRule REQUEST_FILENAME "@beginsWith /files/" ` +
|
|
||||||
`"id:1,phase:1,pass,nolog,ctl:ruleRemoveTargetById=920500;REQUEST_FILENAME"` + "\n"
|
|
||||||
|
|
||||||
func TestExclusionsFileIsLoadedBeforeTheRules(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
crs, err := withExclusions(exclusionsFile(t, backupFileExclusion))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("load the exclusions: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
wantChange(t, crs, get("/files/notes.txt~"), get("/notes.txt~"), matched(920500))
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRuleSwitchedOffAndNamedInTheExclusionsFileIsOff(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
crs, err := waf.New(waf.Params{
|
|
||||||
ParanoiaLevel: 1, DisabledRules: []int{920500},
|
|
||||||
ExclusionsFile: exclusionsFile(t, backupFileExclusion),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("load the exclusions: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
wantResult(t, crs, get("/notes.txt~"), waf.Result{})
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestExclusionsFileMayNameAnyNumberInAComment(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
_, err := withExclusions(exclusionsFile(t,
|
|
||||||
"# 942100: GET /issues/900123\n"+backupFileExclusion))
|
|
||||||
if err != nil {
|
|
||||||
t.Errorf("the exclusions did not load: %v", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestExclusionsFileThatCannotBeUsedIsAnError(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
for _, path := range []string{
|
|
||||||
filepath.Join(t.TempDir(), "missing.conf"),
|
|
||||||
exclusionsFile(t, "SecNoSuchDirective On\n"),
|
|
||||||
// The Core Rule Set's rules are not loaded yet.
|
|
||||||
exclusionsFile(t, "SecRuleUpdateTargetById 942100 \"!ARGS:id\"\n"),
|
|
||||||
// smallwebwaf's own rules may not be switched off or changed, by
|
|
||||||
// id or in a range.
|
|
||||||
exclusionsFile(t, "SecRuleRemoveById 900300\n"),
|
|
||||||
exclusionsFile(t, "SecRuleRemoveById 899999-901000\n"),
|
|
||||||
exclusionsFile(t, "SecAction \"id:1,phase:1,pass,nolog,"+
|
|
||||||
"ctl:ruleRemoveById=899999-901000\"\n"),
|
|
||||||
exclusionsFile(t, "SecAction \"id:1,phase:1,pass,nolog,"+
|
|
||||||
"ctl:ruleRemoveTargetById=900500;ARGS:from\"\n"),
|
|
||||||
} {
|
|
||||||
_, err := withExclusions(path)
|
|
||||||
if err == nil {
|
|
||||||
t.Errorf("%s loaded", path)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// giteaRequests returns the recorded ordinary gitea requests in testdata,
|
|
||||||
// which script/gitea-setup sends too. Each has the Referer of a page that
|
|
||||||
// names the same, as a page opened from it has.
|
|
||||||
func giteaRequests(t *testing.T) []request {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
text, err := os.ReadFile(filepath.Join("testdata", "gitea-requests.txt"))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
var requests []request
|
|
||||||
|
|
||||||
for line := range strings.Lines(string(text)) {
|
|
||||||
line = strings.TrimSpace(line)
|
|
||||||
if line == "" || strings.HasPrefix(line, "#") {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
target, header, _ := strings.Cut(line, " ")
|
|
||||||
|
|
||||||
headers := []string{"Referer: http://git.example" + target}
|
|
||||||
if header != "" {
|
|
||||||
headers = append(headers, header)
|
|
||||||
}
|
|
||||||
|
|
||||||
requests = append(requests, get(target, headers...))
|
|
||||||
}
|
|
||||||
|
|
||||||
return requests
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestGiteaExclusionsLetOrdinaryGiteaRequestsThroughAndRefuseAttacks(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
crs, err := withExclusions(filepath.Join("..", "..", "deploy", "gitea",
|
|
||||||
"crs-exclusions.conf"))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("load gitea's exclusions: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
const threshold = 5 // SWWAF_WAF_ANOMALY_THRESHOLD's default
|
|
||||||
|
|
||||||
for _, r := range giteaRequests(t) {
|
|
||||||
if got := inspect(t, crs, r); got.Score >= threshold {
|
|
||||||
t.Errorf("%s %q refused: %+v", r.target, r.headers[1:], got)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, attack := range []string{
|
|
||||||
"/owner/repo?id=1'%20OR%20'1'='1",
|
|
||||||
"/owner/repo?id=..%2Fcustom%2Fconf%2Fapp.ini",
|
|
||||||
"/owner/repo?id=%3Cscript%3Ealert(1)%3C%2Fscript%3E",
|
|
||||||
} {
|
|
||||||
if got := inspect(t, crs, get(attack)); got.Score < threshold {
|
|
||||||
t.Errorf("%s let through: %+v", attack, got)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies.
|
// bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies.
|
||||||
const bodyLimit = 8 << 10
|
const bodyLimit = 8 << 10
|
||||||
|
|
||||||
|
|||||||
@@ -4,8 +4,7 @@
|
|||||||
# default but these: SWRP_UPSTREAM_URL names gitea's container, SWRP_WAF_BACKEND
|
# default but these: SWRP_UPSTREAM_URL names gitea's container, SWRP_WAF_BACKEND
|
||||||
# names servewaf's, and SWWAF_LOG_REQUEST_HEADERS adds the header that labels a
|
# names servewaf's, and SWWAF_LOG_REQUEST_HEADERS adds the header that labels a
|
||||||
# request. Then send through rproxy the gitea requests that reading gitea's
|
# request. Then send through rproxy the gitea requests that reading gitea's
|
||||||
# source found the Core Rule Set refuses without the exclusions in
|
# source found the Core Rule Set refuses, which docs/gitea.md records, and a
|
||||||
# deploy/gitea/crs-exclusions.conf, and a
|
|
||||||
# short sample of gitea's ordinary use: signing in, pages of files and commits,
|
# short sample of gitea's ordinary use: signing in, pages of files and commits,
|
||||||
# a diff, the API, a push, clone and fetch over HTTP, and two container image
|
# a diff, the API, a push, clone and fetch over HTTP, and two container image
|
||||||
# pushes. Last, print each request servewaf logged, with what it did with it and
|
# pushes. Last, print each request servewaf logged, with what it did with it and
|
||||||
|
|||||||
@@ -1,293 +0,0 @@
|
|||||||
#!/bin/sh
|
|
||||||
# script/gitea-setup: build the image, bring up deploy/gitea under a project
|
|
||||||
# name of its own, and check it:
|
|
||||||
# - that a page load, a push and a clone sent through traefik pass through
|
|
||||||
# rproxy, and that servewaf checks them;
|
|
||||||
# - that gitea logs the visitor's address, not rproxy's;
|
|
||||||
# - that a request from a container on the shared network skips the checks;
|
|
||||||
# - that the Core Rule Set refuses none of the ordinary gitea requests in
|
|
||||||
# internal/waf/testdata/gitea-requests.txt;
|
|
||||||
# - that it refuses none of the API calls with JSON bodies;
|
|
||||||
# - that it refuses no fetch;
|
|
||||||
# - that it refuses SQL injection, path traversal and script injection in a
|
|
||||||
# query string;
|
|
||||||
# - that with servewaf stopped, gitea still answers through rproxy.
|
|
||||||
# It publishes no port of traefik's, since this host's ports 80 and 443 may be
|
|
||||||
# taken. It reaches traefik at its address on the shared network instead, so
|
|
||||||
# its own requests come from the network's gateway, which servewaf checks like
|
|
||||||
# any visitor. Two runs at once on one docker host cannot both start, since
|
|
||||||
# both would use the subnet compose.yaml names. The containers, the network,
|
|
||||||
# the volumes, the image and the files the script makes are removed however it
|
|
||||||
# ends; the gitea and traefik images stay, as the base images of a build do.
|
|
||||||
# It needs network access. script/check does not run this.
|
|
||||||
set -eu
|
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
|
||||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
|
||||||
|
|
||||||
# Named after this run, so that runs in other clones on the same host
|
|
||||||
# never touch each other's.
|
|
||||||
NAME="$("$SCRIPT_DIR/projectname")-gitea-setup-$$"
|
|
||||||
IMAGE="$NAME-image"
|
|
||||||
NETWORK="${NAME}_shared"
|
|
||||||
VISITOR="$NAME-visitor"
|
|
||||||
|
|
||||||
# The site's host name and gitea's image, as deploy/gitea/compose.yaml names
|
|
||||||
# them. gitea's image also runs the container on the shared network.
|
|
||||||
SITE="git.example.org"
|
|
||||||
# gitea 1.27.3, 2026-08-29
|
|
||||||
GITEA_IMAGE="gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef"
|
|
||||||
|
|
||||||
# gitea's one user, who owns the repository.
|
|
||||||
PASSWORD="owner-password"
|
|
||||||
|
|
||||||
# The files the script makes, gitea's secret key for this run, and traefik's
|
|
||||||
# and the gateway's addresses on the shared network once the setup runs.
|
|
||||||
WORK=""
|
|
||||||
secret_key=""
|
|
||||||
traefik=""
|
|
||||||
gateway=""
|
|
||||||
|
|
||||||
cleanup() {
|
|
||||||
# So that a signal cannot stop it partway through.
|
|
||||||
trap '' HUP INT TERM
|
|
||||||
docker rm --force "$VISITOR" >/dev/null 2>&1 || true
|
|
||||||
if [ -n "$WORK" ]; then
|
|
||||||
compose down --volumes --remove-orphans --timeout 5 >/dev/null 2>&1 || true
|
|
||||||
rm -rf "$WORK"
|
|
||||||
fi
|
|
||||||
docker rmi --force "$IMAGE" >/dev/null 2>&1 || true
|
|
||||||
}
|
|
||||||
|
|
||||||
# compose <argument>...: docker compose on the setup, under this run's project
|
|
||||||
# name, with the image built here, this run's secret key and no port of
|
|
||||||
# traefik's published.
|
|
||||||
compose() {
|
|
||||||
SMALLWEBWAF_IMAGE="$IMAGE" GITEA_SECRET_KEY="$secret_key" \
|
|
||||||
docker compose --project-name "$NAME" \
|
|
||||||
--file "$ROOT/deploy/gitea/compose.yaml" --file "$WORK/no-ports.yaml" "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
fail() {
|
|
||||||
echo "gitea-setup: $*; the last lines of each container's output:" >&2
|
|
||||||
compose logs --no-color --tail 30 >&2 || true
|
|
||||||
exit 1
|
|
||||||
}
|
|
||||||
|
|
||||||
# wait_for <what fails> <command>...: run the command every second until
|
|
||||||
# it succeeds, for at most a minute.
|
|
||||||
wait_for() {
|
|
||||||
failure="$1"
|
|
||||||
shift
|
|
||||||
tries=0
|
|
||||||
until "$@"; do
|
|
||||||
tries=$((tries + 1))
|
|
||||||
[ "$tries" -lt 60 ] || fail "$failure"
|
|
||||||
sleep 1
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
# logged <service> <text>...: a line of the service's output holds every text,
|
|
||||||
# in any order.
|
|
||||||
logged() {
|
|
||||||
lines="$(compose logs --no-color --no-log-prefix "$1" 2>&1)"
|
|
||||||
shift
|
|
||||||
for text in "$@"; do
|
|
||||||
lines="$(printf '%s\n' "$lines" | grep -F -e "$text")" || return 1
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
# send <path> <curl option>...: send a request for path to the site, at
|
|
||||||
# traefik's address, and print the status of the answer, 000 for none. traefik
|
|
||||||
# answers with a certificate it made itself, since the setup names no
|
|
||||||
# certificate resolver.
|
|
||||||
send() {
|
|
||||||
path="$1"
|
|
||||||
shift
|
|
||||||
curl --silent --show-error --insecure --max-time 30 --output /dev/null \
|
|
||||||
--write-out '%{http_code}' --resolve "$SITE:443:$traefik" "$@" \
|
|
||||||
"https://$SITE$path" || true
|
|
||||||
}
|
|
||||||
|
|
||||||
# routed: traefik passes a request for the site's front page on to gitea.
|
|
||||||
routed() {
|
|
||||||
[ "$(send / 2>/dev/null)" = 200 ]
|
|
||||||
}
|
|
||||||
|
|
||||||
# site_git <git argument>...: run git as the owner, without the host's own git
|
|
||||||
# settings, reaching the site at traefik's address. git gives up on a transfer
|
|
||||||
# slower than 1000 bytes a second for 30 seconds.
|
|
||||||
site_git() {
|
|
||||||
GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 git \
|
|
||||||
-c user.name=owner -c user.email=owner@example.invalid \
|
|
||||||
-c http.sslVerify=false -c "http.curloptResolve=$SITE:443:$traefik" \
|
|
||||||
-c http.lowSpeedLimit=1000 -c http.lowSpeedTime=30 "$@"
|
|
||||||
}
|
|
||||||
|
|
||||||
# address_of <service>: print the service's address on the shared network.
|
|
||||||
address_of() {
|
|
||||||
docker inspect --format "{{(index .NetworkSettings.Networks \"$NETWORK\").IPAddress}}" \
|
|
||||||
"$(compose ps --quiet "$1")"
|
|
||||||
}
|
|
||||||
|
|
||||||
# passed_through <method> <path>: rproxy passed a request from the gateway for
|
|
||||||
# path to gitea on servewaf's allow, and servewaf checked it: its report line
|
|
||||||
# gives the client's counts and the Core Rule Set's score.
|
|
||||||
passed_through() {
|
|
||||||
wait_for "rproxy logged no $1 $2 that servewaf allowed" logged rproxy \
|
|
||||||
"\"method\":\"$1\"" "\"path\":\"$2\"" "\"client_ip\":\"$gateway\"" \
|
|
||||||
'"disposition":"allow"'
|
|
||||||
wait_for "servewaf logged no check of $1 $2" logged servewaf \
|
|
||||||
'"servewaf_line":"report"' "\"method\":\"$1\"" "\"path\":\"$2\"" \
|
|
||||||
"\"client_ip\":\"$gateway\"" '"action":"forward"' '"counts":' '"waf_score":'
|
|
||||||
}
|
|
||||||
|
|
||||||
# refused: print each request servewaf logged as refused, for any reason.
|
|
||||||
refused() {
|
|
||||||
compose logs --no-color --no-log-prefix servewaf 2>&1 |
|
|
||||||
grep -F '"servewaf_line":"report"' | grep -v -F '"action":"forward"' || true
|
|
||||||
}
|
|
||||||
|
|
||||||
# ordinary_requests: sign in as the owner, save a file and open an issue
|
|
||||||
# through the API with JSON bodies, then send, signed in, each request in
|
|
||||||
# internal/waf/testdata/gitea-requests.txt, with the header its line ends with,
|
|
||||||
# if any, and that request's own address as its Referer, as a page opened from it
|
|
||||||
# has. One a second keeps them under the default limit of 120 requests a minute,
|
|
||||||
# since a request refused for a limit is never inspected. Fail if servewaf
|
|
||||||
# refuses any of them.
|
|
||||||
ordinary_requests() {
|
|
||||||
send /user/login --cookie-jar "$WORK/cookies" >/dev/null
|
|
||||||
status="$(send /user/login --cookie "$WORK/cookies" --cookie-jar "$WORK/cookies" \
|
|
||||||
--header "Origin: https://$SITE" --data-urlencode user_name=owner \
|
|
||||||
--data-urlencode "password=$PASSWORD")"
|
|
||||||
[ "$status" = 303 ] || fail "signing in was answered $status"
|
|
||||||
|
|
||||||
content="$(printf 'declare const env: Record<string, string>;\n' | base64)"
|
|
||||||
status="$(send /api/v1/repos/owner/repo/contents/process.env.d.ts \
|
|
||||||
--user "owner:$PASSWORD" --header "Content-Type: application/json" \
|
|
||||||
--data "{\"message\":\"Add process.env.d.ts\",\"content\":\"$content\"}")"
|
|
||||||
[ "$status" = 201 ] || fail "saving a file through the API was answered $status"
|
|
||||||
status="$(send /api/v1/repos/owner/repo/issues --user "owner:$PASSWORD" \
|
|
||||||
--header "Content-Type: application/json" \
|
|
||||||
--data '{"title":"ls -la fails","body":"SELECT * FROM users; <img src=x>"}')"
|
|
||||||
[ "$status" = 201 ] || fail "opening an issue through the API was answered $status"
|
|
||||||
|
|
||||||
grep -v -e '^#' -e '^$' "$ROOT/internal/waf/testdata/gitea-requests.txt" |
|
|
||||||
while read -r target header; do
|
|
||||||
if [ -n "$header" ]; then
|
|
||||||
send "$target" --globoff --cookie "$WORK/cookies" \
|
|
||||||
--header "Referer: https://$SITE$target" --header "$header" >/dev/null
|
|
||||||
else
|
|
||||||
send "$target" --globoff --cookie "$WORK/cookies" \
|
|
||||||
--header "Referer: https://$SITE$target" >/dev/null
|
|
||||||
fi
|
|
||||||
sleep 1
|
|
||||||
done
|
|
||||||
|
|
||||||
# The last request's report is logged once its answer has ended.
|
|
||||||
send /api/v1/settings/api >/dev/null
|
|
||||||
wait_for "servewaf logged no report of the last ordinary request" logged servewaf \
|
|
||||||
'"servewaf_line":"report"' '"path":"/api/v1/settings/api"'
|
|
||||||
refusals="$(refused)"
|
|
||||||
[ -z "$refusals" ] || fail "servewaf refused ordinary requests:
|
|
||||||
$refusals"
|
|
||||||
}
|
|
||||||
|
|
||||||
# attacks: send SQL injection, path traversal and script injection in a query
|
|
||||||
# string, and fail unless the Core Rule Set refuses each.
|
|
||||||
attacks() {
|
|
||||||
# The traversal climbs one directory: the default rule file refuses two or
|
|
||||||
# more first, before the Core Rule Set runs.
|
|
||||||
for query in "id=1'%20OR%20'1'%3D'1" "id=..%2Fcustom%2Fconf%2Fapp.ini" \
|
|
||||||
"id=%3Cscript%3Ealert(1)%3C%2Fscript%3E"; do
|
|
||||||
status="$(send "/owner/repo?$query")"
|
|
||||||
[ "$status" = 403 ] || fail "the attack ?$query was answered $status"
|
|
||||||
wait_for "servewaf logged no refusal of ?$query" logged servewaf \
|
|
||||||
'"servewaf_line":"report"' '"action":"waf_blocked"' "\"query\":\"$query\""
|
|
||||||
done
|
|
||||||
}
|
|
||||||
|
|
||||||
main() {
|
|
||||||
cd "$ROOT"
|
|
||||||
trap cleanup EXIT
|
|
||||||
trap 'exit 1' HUP INT TERM
|
|
||||||
WORK="$(mktemp -d)"
|
|
||||||
printf 'services:\n traefik:\n ports: !reset []\n' >"$WORK/no-ports.yaml"
|
|
||||||
|
|
||||||
docker build --no-cache -t "$IMAGE" .
|
|
||||||
secret_key="$(docker run --rm "$GITEA_IMAGE" gitea generate secret SECRET_KEY)"
|
|
||||||
|
|
||||||
# rproxy is healthy once gitea takes connections.
|
|
||||||
compose up --detach --quiet-pull --wait --wait-timeout 120 ||
|
|
||||||
fail "the setup did not start"
|
|
||||||
traefik="$(address_of traefik)"
|
|
||||||
gateway="$(docker network inspect --format '{{(index .IPAM.Config 0).Gateway}}' "$NETWORK")"
|
|
||||||
wait_for "traefik did not route the site to rproxy" routed
|
|
||||||
|
|
||||||
compose exec --user git gitea gitea admin user create --admin \
|
|
||||||
--username owner --password "$PASSWORD" --email owner@example.invalid \
|
|
||||||
--must-change-password=false >/dev/null || fail "creating gitea's user failed"
|
|
||||||
status="$(send /api/v1/user/repos --user "owner:$PASSWORD" \
|
|
||||||
--header "Content-Type: application/json" --data '{"name":"repo"}')"
|
|
||||||
[ "$status" = 201 ] || fail "creating the repository was answered $status"
|
|
||||||
|
|
||||||
status="$(send /owner/repo)"
|
|
||||||
[ "$status" = 200 ] || fail "the repository's page was answered $status"
|
|
||||||
passed_through GET /owner/repo
|
|
||||||
echo "gitea-setup: a page load passes through rproxy, and servewaf checks it"
|
|
||||||
|
|
||||||
wait_for "gitea did not log the visitor's address" logged gitea \
|
|
||||||
"GET /owner/repo for $gateway:"
|
|
||||||
echo "gitea-setup: gitea logs the visitor's address, not rproxy's"
|
|
||||||
|
|
||||||
site_git init --quiet --initial-branch=main "$WORK/repo"
|
|
||||||
echo "hello" >"$WORK/repo/README.md"
|
|
||||||
site_git -C "$WORK/repo" add README.md
|
|
||||||
site_git -C "$WORK/repo" commit --quiet --message "Add a README"
|
|
||||||
site_git -C "$WORK/repo" push --quiet \
|
|
||||||
"https://owner:$PASSWORD@$SITE/owner/repo.git" main ||
|
|
||||||
fail "the push failed"
|
|
||||||
passed_through POST /owner/repo.git/git-receive-pack
|
|
||||||
echo "gitea-setup: a push passes through rproxy, and servewaf checks it"
|
|
||||||
|
|
||||||
site_git clone --quiet "https://owner:$PASSWORD@$SITE/owner/repo.git" \
|
|
||||||
"$WORK/clone" || fail "the clone failed"
|
|
||||||
[ "$(cat "$WORK/clone/README.md")" = hello ] || fail "the clone lacks the README"
|
|
||||||
passed_through POST /owner/repo.git/git-upload-pack
|
|
||||||
echo "gitea-setup: a clone passes through rproxy, and servewaf checks it"
|
|
||||||
|
|
||||||
# A container on the shared network, as sandcastle is, asks for the site
|
|
||||||
# by its name, and so reaches traefik from its own address.
|
|
||||||
status="$(docker run --rm --name "$VISITOR" --network "$NETWORK" \
|
|
||||||
--entrypoint curl "$GITEA_IMAGE" --silent --insecure --max-time 30 \
|
|
||||||
--output /dev/null --write-out '%{http_code}' "https://$SITE/api/v1/version" ||
|
|
||||||
true)"
|
|
||||||
[ "$status" = 200 ] || fail "the request from the shared network was answered $status"
|
|
||||||
wait_for "servewaf logged no request from the shared network" logged servewaf \
|
|
||||||
'"servewaf_line":"report"' '"path":"/api/v1/version"' '"action":"forward"'
|
|
||||||
line="$(compose logs --no-color --no-log-prefix servewaf |
|
|
||||||
grep -F '"servewaf_line":"report"' | grep -F '"path":"/api/v1/version"')"
|
|
||||||
case "$line" in
|
|
||||||
*"\"client_ip\":\"$gateway\""*)
|
|
||||||
fail "the request from the shared network came from the gateway" ;;
|
|
||||||
*'"counts":'* | *'"waf_score":'*)
|
|
||||||
fail "servewaf checked the request from the shared network" ;;
|
|
||||||
esac
|
|
||||||
echo "gitea-setup: a request from the shared network skips the checks"
|
|
||||||
|
|
||||||
site_git -C "$WORK/clone" fetch --quiet || fail "the fetch failed"
|
|
||||||
ordinary_requests
|
|
||||||
echo "gitea-setup: the Core Rule Set refuses no ordinary gitea request"
|
|
||||||
attacks
|
|
||||||
echo "gitea-setup: the Core Rule Set refuses the attacks"
|
|
||||||
|
|
||||||
compose stop servewaf >/dev/null 2>&1 || fail "servewaf did not stop"
|
|
||||||
status="$(send /owner/repo)"
|
|
||||||
[ "$status" = 200 ] ||
|
|
||||||
fail "with servewaf stopped, the repository's page was answered $status"
|
|
||||||
wait_for "rproxy logged no request SWRP_DEFAULT_DISPOSITION decided" \
|
|
||||||
logged rproxy '"path":"/owner/repo"' '"default_disposition":true'
|
|
||||||
echo "gitea-setup: with servewaf stopped, gitea answers through rproxy"
|
|
||||||
}
|
|
||||||
|
|
||||||
main "$@"
|
|
||||||
Reference in New Issue
Block a user