Author SHA1 Message Date
clawbot 866b948ec0 Gitea setup: Core Rule Set blocks attacks, not ordinary Gitea traffic (closes #198)
check / check (push) Waiting to run
The Gitea setup runs the Core Rule Set in block mode. Rules that refused
ordinary Gitea requests are switched off only where they misfired:

- deploy/gitea/crs-exclusions.conf, loaded through the new setting
  SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE before the Core Rule Set's rules,
  covers Gitea's search, file-name and branch-name fields, Referer, file
  paths, API attachment names and Actions artifact uploads;
- smallwebwaf's defaults gain a seventh change: the registry upload
  path's from parameter skips 932260.

A file that cannot be read, does not parse or names an id from 900000
to 900999 stops servewaf at start, in every mode. internal/waf's tests
and script/gitea-setup send the recorded requests in
internal/waf/testdata/gitea-requests.txt and three attacks.

Model: opus-5-5
2026-11-06 20:17:07 +00:00
clawbot 21cac70b50 Lower the default request and byte limits (closes #193)
check / check (push) Successful in 3m12s
New defaults, as sneak approved: 120 requests a minute, 1,500 an hour and 6,000 a day; bytes off a minute, 50G an hour and 100G a day. The README, `SPEC.md`, `docs/settings.md` and `docs/gitea.md` give the new figures and reasoning, and point Actions runners and shared addresses at `SWWAF_RATE_LIMIT_EXEMPT_NETS`. The README examples now hold against the new defaults.

An Actions runner that is not exempted makes more requests an hour than the new limit and will be banned.

Model: opus-5-5
2026-11-06 20:14:23 +01:00
clawbot 2ab57eb278 Install the image's Ubuntu packages from the normal mirrors (closes #190)
check / check (push) Successful in 3m6s
The image's `ca-certificates`, `nix-bin` and `runit` now come from Ubuntu's archive instead of a dated snapshot, whose service had failed image builds. apt fetches them over plain HTTP and checks each against the archive's signed package lists, so no CA certificates are borrowed from the Go image. The README, `SPEC.md` and `docs/image.md` say so; the amd64-only limit, which came from the removed hash check, is gone.

Rule cut against: `REPO_POLICIES.md` pins every external reference by hash; sneak's ruling at sneak/prompts#72 (comment) allows Ubuntu's own package index.

Model: deepseek-v4-pro (implementation), opus-5-5 (rework)
2026-11-06 20:14:18 +01:00
clawbot a63b7e4d2d Rename internal/proxy to internal/servewaf (closes #173)
check / check (push) Successful in 3m44s
The package passes nothing on since the one-process mode went: it answers rproxy's questions, takes its reports and answers servewaf's own endpoints. It is now named after the command it serves, with `git mv`, and its constructor is `servewaf.New`. `SPEC.md` and the other documents follow. No change of behaviour.

Model: opus-5-5
2026-11-06 20:02:17 +01:00
clawbot 3cb41f27a8 deploy/gitea: Gitea behind traefik, rproxy and servewaf, with its test (closes #178)
check / check (push) Successful in 3m50s
A compose setup that runs gitea 1.27.3 behind traefik, rproxy and servewaf on one shared network with IPv4 and IPv6, and `script/gitea-setup` (`make gitea-setup`), which brings it up and checks that a page load, a push and a clone pass through rproxy and are checked by servewaf. Containers on the shared network skip servewaf's checks; the gateways do not. gitea gets a secret key made per install and has sign-up off. `docs/gitea.md` explains each setting.

Judgement call: the Core Rule Set runs in `detect` mode here until #198 lands.

Model: deepseek-v4-pro (implementation), opus-5-5 (rework)
2026-11-06 19:46:38 +01:00
62 changed files with 2096 additions and 813 deletions
+6 -2
View File
@@ -1,12 +1,13 @@
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \ .PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
run example-app gitea-requests run example-app gitea-requests gitea-setup
# Makefile targets are thin shims; the implementations live in script/ # Makefile targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern (see the Entrypoints section # per the scripts-to-rule-them-all pattern (see the Entrypoints section
# of README.md). tidy writes go.mod and go.sum as `go mod tidy` does, # of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
# which test checks. build and run are for working on the code by hand; # which test checks. build and run are for working on the code by hand;
# example-app checks the image with an app built on it; gitea-requests # example-app checks the image with an app built on it; gitea-requests
# reports the gitea requests the Core Rule Set refuses at the defaults. # reports the gitea requests the Core Rule Set refuses at the defaults;
# gitea-setup checks the setup in deploy/gitea.
bootstrap: bootstrap:
@script/bootstrap @script/bootstrap
@@ -49,3 +50,6 @@ example-app:
gitea-requests: gitea-requests:
@script/gitea-requests @script/gitea-requests
gitea-setup:
@script/gitea-setup
+36 -31
View File
@@ -26,10 +26,10 @@ It protects against:
- Scrapers that fetch every page of a site, all day. Each client gets a limit on - Scrapers that fetch every page of a site, all day. Each client gets a limit on
requests and on bytes per minute, per hour and per day. The defaults leave requests and on bytes per minute, per hour and per day. The defaults leave
room for programmatic use of an API, such as an automation fleet, and catch a room for a person reading a site for hours. They catch a scraper that keeps up
client firing requests as fast as it can within seconds. A client over a limit one request every 2 seconds. A client over a limit is refused and banned: for
is refused and banned: for a while at first, for three times as long at each a while at first, for three times as long at each repeat, and permanently when
repeat, and permanently when it keeps at it. it keeps at it.
- Scanners that probe for secrets, such as `/.env` or `/.git/config`. The - Scanners that probe for secrets, such as `/.env` or `/.git/config`. The
default rule file bans such a client at its first probe, and its next request default rule file bans such a client at its first probe, and its next request
during the ban makes the ban permanent. Your own rules, or a list of trap during the ban makes the ban permanent. Your own rules, or a list of trap
@@ -192,9 +192,9 @@ account, a key, a token or an alert destination stays off until you give one.
- Any `SWWAF_` setting can be given as a file instead, for secrets and long - Any `SWWAF_` setting can be given as a file instead, for secrets and long
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
The file's contents are the value, less one newline at the end. Setting both The file's contents are the value, less one newline at the end. Setting both
forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` already names a file, so forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` and
it has no `_FILE` form, and no `SWRP_` setting has one, since `rproxy` never `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` already name a file, so they have no
touches disk. `_FILE` form. No `SWRP_` setting has one, since `rproxy` never touches disk.
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. - A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of - A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
1024: `512K`, `100M`, `5G`. 1024: `512K`, `100M`, `5G`.
@@ -236,13 +236,13 @@ Every setting the code reads, with its default:
| `SWWAF_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. | | `SWWAF_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
| `SWWAF_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients the request and byte limits neither count nor refuse. | | `SWWAF_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients the request and byte limits neither count nor refuse. |
| `SWWAF_DENY_NETS` | empty | Netblocks whose clients are refused. | | `SWWAF_DENY_NETS` | empty | Netblocks whose clients are refused. |
| `SWWAF_RATE_LIMIT_PER_MINUTE` | `500` | The most requests a client may make in a minute, a count or `off`; one more bans it. | | `SWWAF_RATE_LIMIT_PER_MINUTE` | `120` | The most requests a client may make in a minute, a count or `off`; one more bans it. |
| `SWWAF_RATE_LIMIT_PER_HOUR` | `20000` | The most requests a client may make in an hour, a count or `off`; one more bans it. | | `SWWAF_RATE_LIMIT_PER_HOUR` | `1500` | The most requests a client may make in an hour, a count or `off`; one more bans it. |
| `SWWAF_RATE_LIMIT_PER_DAY` | `300000` | The most requests a client may make in a day, a count or `off`; one more bans it. | | `SWWAF_RATE_LIMIT_PER_DAY` | `6000` | The most requests a client may make in a day, a count or `off`; one more bans it. |
| `SWWAF_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes whose requests the request and byte limits neither count nor refuse, such as `/assets/`. | | `SWWAF_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes whose requests the request and byte limits neither count nor refuse, such as `/assets/`. |
| `SWWAF_BYTES_LIMIT_PER_MINUTE` | `4G` | The most bytes a client's requests may carry in a minute, a size or `off`; the request that takes a client past any byte limit bans it. | | `SWWAF_BYTES_LIMIT_PER_MINUTE` | `off` | The most bytes a client's requests may carry in a minute, a size or `off`; the request that takes a client past any byte limit bans it. |
| `SWWAF_BYTES_LIMIT_PER_HOUR` | `20G` | The most bytes a client's requests may carry in an hour, a size or `off`. | | `SWWAF_BYTES_LIMIT_PER_HOUR` | `50G` | The most bytes a client's requests may carry in an hour, a size or `off`. |
| `SWWAF_BYTES_LIMIT_PER_DAY` | `200G` | The most bytes a client's requests may carry in a day, a size or `off`. | | `SWWAF_BYTES_LIMIT_PER_DAY` | `100G` | The most bytes a client's requests may carry in a day, a size or `off`. |
| `SWWAF_BYTES_COUNT` | `both` | Which body bytes the byte limits count: `response`, `request` or `both`. | | `SWWAF_BYTES_COUNT` | `both` | Which body bytes the byte limits count: `response`, `request` or `both`. |
| `SWWAF_ERROR_BURST_THRESHOLD` | `30` | The most refusals by a rule file, a trap path or the Core Rule Set a client may get in a minute, a count or `off`; one more bans it. | | `SWWAF_ERROR_BURST_THRESHOLD` | `30` | The most refusals by a rule file, a trap path or the Core Rule Set a client may get in a minute, a count or `off`; one more bans it. |
| `SWWAF_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. | | `SWWAF_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. |
@@ -272,6 +272,7 @@ Every setting the code reads, with its default:
| `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. | | `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
| `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. | | `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
| `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. | | `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
| `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only; `deploy/gitea/crs-exclusions.conf` is gitea's. |
| `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. | | `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
| `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. | | `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
| `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. | | `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
@@ -407,10 +408,10 @@ services:
SWWAF_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7 SWWAF_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7
# A netblock that sends nothing wanted. # A netblock that sends nothing wanted.
SWWAF_DENY_NETS: 198.51.100.128/25 SWWAF_DENY_NETS: 198.51.100.128/25
# Tighter limits for everyone else: a site with no API. # Tighter limits for everyone else: a small static site.
SWWAF_RATE_LIMIT_PER_MINUTE: "120" SWWAF_RATE_LIMIT_PER_MINUTE: "60"
SWWAF_RATE_LIMIT_PER_HOUR: "3000" SWWAF_RATE_LIMIT_PER_HOUR: "600"
SWWAF_RATE_LIMIT_PER_DAY: "30000" SWWAF_RATE_LIMIT_PER_DAY: "3000"
# Hosting networks and one country get a share of those limits. # Hosting networks and one country get a share of those limits.
SWWAF_ASN_LIMIT_PERCENT: AS14061:50,AS16276:25 SWWAF_ASN_LIMIT_PERCENT: AS14061:50,AS16276:25
SWWAF_COUNTRY_LIMIT_PERCENT: cn:25 SWWAF_COUNTRY_LIMIT_PERCENT: cn:25
@@ -431,10 +432,9 @@ volumes:
gitea behind traefik, set up for a code forge: gitea behind traefik, set up for a code forge:
- the upload limits raised, in size and in time; - the upload limits raised, in size and in time;
- the response size limit and the byte limits raised, for clones over 1 GiB; - the response size limit raised, for clones over 1 GiB;
- the Actions runners exempt from counting; - the Actions runners exempt from counting;
- the default request limits kept, since they already leave room for API - the default request and byte limits kept.
clients.
`traefik` is the network traefik is on, made outside this file. `traefik` is the network traefik is on, made outside this file.
@@ -488,11 +488,6 @@ services:
SWWAF_INSTANCE_NAME: fsn1/servewaf SWWAF_INSTANCE_NAME: fsn1/servewaf
# The Actions runners ask for work all day; never count them. # The Actions runners ask for work all day; never count them.
SWWAF_RATE_LIMIT_EXEMPT_NETS: 10.0.4.0/24 SWWAF_RATE_LIMIT_EXEMPT_NETS: 10.0.4.0/24
# Raised with rproxy's response limit, to four times the default
# each, so that one clone of that size breaks no byte limit.
SWWAF_BYTES_LIMIT_PER_MINUTE: 16G
SWWAF_BYTES_LIMIT_PER_HOUR: 80G
SWWAF_BYTES_LIMIT_PER_DAY: 800G
# gitea never serves these; a request for one bans the client. # gitea never serves these; a request for one bans the client.
SWWAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php,/wp-admin/ SWWAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php,/wp-admin/
volumes: volumes:
@@ -510,8 +505,9 @@ volumes:
servewaf-state: servewaf-state:
``` ```
[`docs/gitea.md`](docs/gitea.md) records what the defaults do in front of gitea, `deploy/gitea` holds a complete setup of gitea, traefik included, as a Docker
request by request. Compose file. [`docs/gitea.md`](docs/gitea.md) explains it, and records what the
defaults do in front of gitea, request by request.
### `rproxy` beside the app ### `rproxy` beside the app
@@ -641,10 +637,14 @@ and the status for a size or time limit, such as `413` for a body too large.
headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In
front of a code forge, bodies carry what people write and publish, which the front of a code forge, bodies carry what people write and publish, which the
rules take for attacks. rules take for attacks.
- `smallwebwaf` changes the Core Rule Set in six ways so that it lets ordinary - `smallwebwaf` changes the Core Rule Set in seven ways so that it lets ordinary
requests to a code forge through, such as git over HTTP and sign-in through requests to a code forge through, such as git over HTTP and sign-in through
Git Credential Manager. No setting undoes them; "Attack detection" in Git Credential Manager. No setting undoes them; "Attack detection" in
[`SPEC.md`](SPEC.md) lists them. [`SPEC.md`](SPEC.md) lists them.
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's
own fields or paths. gitea's exclusions, in
`deploy/gitea/crs-exclusions.conf`, let its searches, file names and branch
names through.
- Rule files are plain text, one rule to a line, edited by hand and read again - Rule files are plain text, one rule to a line, edited by hand and read again
while `servewaf` runs. The image ships one file of probes no real visitor while `servewaf` runs. The image ships one file of probes no real visitor
sends (see [`docs/rules.md`](docs/rules.md)). sends (see [`docs/rules.md`](docs/rules.md)).
@@ -687,8 +687,8 @@ aside as `<name>.bad`, and never stops the running process (see
health check. health check.
- `internal/config`: reads the settings, the one place they are read. - `internal/config`: reads the settings, the one place they are read.
- `internal/rproxy`: the reverse proxy in front of one app. - `internal/rproxy`: the reverse proxy in front of one app.
- `internal/proxy`: `servewaf`'s server, which runs the checks and answers its - `internal/servewaf`: `servewaf`'s server, which runs the checks and answers
own endpoints. its own endpoints.
- `internal/ratelimit`, `internal/bans`, `internal/rules`, `internal/waf`, - `internal/ratelimit`, `internal/bans`, `internal/rules`, `internal/waf`,
`internal/lookup`, `internal/reputation`, `internal/anomaly` and `internal/lookup`, `internal/reputation`, `internal/anomaly` and
`internal/alerts`: one check or source each. `internal/alerts`: one check or source each.
@@ -784,6 +784,10 @@ on the host.
Core Rule Set is known to refuse, and prints what `servewaf` did with each. Core Rule Set is known to refuse, and prints what `servewaf` did with each.
Needs network access, and a docker daemon that pushes to a registry on Needs network access, and a docker daemon that pushes to a registry on
`localhost` over plain HTTP. `make gitea-requests`. `localhost` over plain HTTP. `make gitea-requests`.
- `script/gitea-setup`: builds the image, brings up the setup in `deploy/gitea`,
and checks it as [`docs/gitea.md`](docs/gitea.md) describes. Then it removes
the containers, the network, the volumes and the image it built. Needs network
access. `make gitea-setup`.
## TODO ## TODO
@@ -807,7 +811,8 @@ on the host.
- [`docs/state-files.md`](docs/state-files.md): the state files and what each - [`docs/state-files.md`](docs/state-files.md): the state files and what each
holds. holds.
- [`docs/image.md`](docs/image.md): what the image holds and how it is pinned. - [`docs/image.md`](docs/image.md): what the image holds and how it is pinned.
- [`docs/gitea.md`](docs/gitea.md): `smallwebwaf` in front of gitea. - [`docs/gitea.md`](docs/gitea.md): `smallwebwaf` in front of gitea: the setup
in `deploy/gitea`, and what the defaults do there.
## License ## License
+60 -39
View File
@@ -36,7 +36,8 @@ to them.
- the state files; - the state files;
- a lookup database; - a lookup database;
- a `SWWAF_` setting given as a file; - a `SWWAF_` setting given as a file;
- the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names. - the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names;
- the file `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names.
## Non-goals ## Non-goals
@@ -66,10 +67,10 @@ to them.
one it does not know, the binary prints the commands and exits with one it does not know, the binary prints the commands and exits with
status 2. status 2.
- `internal/config`: reads the settings, the one place they are read. - `internal/config`: reads the settings, the one place they are read.
- `internal/proxy`: `servewaf`'s server. It runs the checks in `check`, and - `internal/servewaf`: `servewaf`'s server. It runs the checks in `check`,
holds `servewaf`'s question, answer and report. It answers `servewaf`'s and holds `servewaf`'s question, answer and report. It answers
own endpoints in `answerOwnEndpoint`. It counts bytes, refusals and `servewaf`'s own endpoints in `answerOwnEndpoint`. It counts bytes,
anomalies, and writes the log lines. refusals and anomalies, and writes the log lines.
- `internal/rproxy`: the reverse proxy in front of one app that asks - `internal/rproxy`: the reverse proxy in front of one app that asks
`servewaf` about each request. It passes a request to the app with the `servewaf` about each request. It passes a request to the app with the
standard library's `httputil.ReverseProxy`, within the time and size standard library's `httputil.ReverseProxy`, within the time and size
@@ -81,7 +82,7 @@ to them.
of bans is full. of bans is full.
- `internal/rules`: reads the rule files at start and again when they - `internal/rules`: reads the rule files at start and again when they
change, and tells which rules a request matches. change, and tells which rules a request matches.
- `internal/waf`: the Core Rule Set, with the six changes listed under - `internal/waf`: the Core Rule Set, with the seven changes listed under
"Attack detection", as Coraza directives. "Attack detection", as Coraza directives.
- `internal/lookup`: looks clients up through GeoJS or in the lookup - `internal/lookup`: looks clients up through GeoJS or in the lookup
database. database.
@@ -325,9 +326,10 @@ Conventions:
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
The file's contents are the value, less one newline at the end, and are The file's contents are the value, less one newline at the end, and are
checked as the value itself would be. Setting both forms stops the start, and checked as the value itself would be. Setting both forms stops the start, and
so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` and
already holds a file name, so it has no `_FILE` form. No `SWRP_` setting has `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` already hold a file name, so they have
one, since `rproxy` never touches disk. no `_FILE` form. No `SWRP_` setting has one, since `rproxy` never touches
disk.
- Settings, files included, are read once at start. Changing one means - Settings, files included, are read once at start. Changing one means
restarting the container. The files `servewaf` watches while it runs are its restarting the container. The files `servewaf` watches while it runs are its
state files, its rule files and the lookup database. state files, its rule files and the lookup database.
@@ -397,32 +399,31 @@ Conventions:
### Request rate limits, per client ### Request rate limits, per client
Breaking one bans the client. The defaults suit programmatic use of an API, such Breaking one bans the client. The defaults let people read a site and stop
as an automation fleet on another host, and still stop abusive bots. A fleet of scrapers that send many requests slowly over hours or days. The hour limit is
20 agents, each making a request every 10 seconds at its busiest, makes 120 the main one.
requests a minute and 7,200 an hour. Working around the clock, it makes 172,800
a day. It stays under all three limits.
- Per minute: 500, about 8 requests a second. It leaves room for the fleet's 120 - Per minute: 120, 2 requests a second. It leaves room for a person opening two
a minute and a few quick first visits with an empty cache. A gitea page opened pages at once with an empty cache. A gitea page opened with an empty cache
with an empty cache loads about 40 files. A client firing requests as fast as loads 20 to 60 files, and a handful once cached.
it can passes the limit within seconds. - Per hour: 1,500. A person reading code in gitea for a whole hour, a page every
- Per hour: 20,000, about 5.5 requests a second for a whole hour. A client that 20 seconds, makes about 900 requests. A scraper making one request every 2
keeps up 6 a second makes 21,600 in an hour, and passes it, at 360 a minute, seconds makes 1,800 an hour, more than the limit.
under the minute limit. The fleet's 7,200 an hour is about a third of it. - Per day: 6,000, four hours at the hour limit. A scraper making one request
- Per day: 300,000, about 3.5 requests a second around the clock. A scraper that every 10 seconds makes 8,640 a day, more than the limit.
keeps up 4 a second makes 345,600 a day, and passes it. At 240 a minute and
14,400 an hour, it stays under the other two limits.
Counting cannot catch a client that stays under all three limits, such as one Counting cannot catch a client that stays under all three limits, about 4
that keeps up 3 requests a second. Only the other checks can catch an abuser requests a minute around the clock. Only the other checks can catch an abuser
that slow. They include the trap paths, the rule files, the Core Rule Set and that slow. They include the trap paths, the rule files, the Core Rule Set and
the blocklists. An abuser that trips none of them is not caught. A machine that the blocklists. An abuser that trips none of them is not caught.
keeps up more than 500 requests a minute passes the minute limit. One that keeps
up more than about 5.5 a second for an hour passes the hour limit. One that Some legitimate clients pass the limits:
keeps up more than about 3.5 a second all day passes the day limit, as two busy
Actions runners on one address do. Such a machine's address belongs in - An Actions runner asks gitea for work all day and can pass the limits.
`SWWAF_RATE_LIMIT_EXEMPT_NETS`. - Everyone behind one shared address, such as an office, shares one client's
limits.
Their addresses belong in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
`SWWAF_RATE_LIMIT_EXEMPT_PATHS` takes path prefixes out of the count, such as `SWWAF_RATE_LIMIT_EXEMPT_PATHS` takes path prefixes out of the count, such as
static assets or health checks; their bytes are not counted either. A prefix is static assets or health checks; their bytes are not counted either. A prefix is
@@ -435,12 +436,10 @@ matched the same way.
### Byte limits, per client ### Byte limits, per client
The hour and day limits sit about twice above what 40 people busy with heavy web By default there is no minute limit, so a client can download in a burst at full
apps move together on one address. Each moves about 256 MiB an hour, for ten speed. The hour limit defaults to 50 GiB and the day limit to 100 GiB. They
hours a day. A response's bytes are counted when it ends (see step 15). So the allow several downloads of a few GiB each. A response's bytes are counted when
minute limit holds three of the largest responses, with their requests, ending it ends (see step 15).
in the same minute. No single download then breaks a limit. An operator who
raises `SWRP_MAX_RESPONSE_SIZE` must raise the byte limits too.
### Lookup of AS number and country ### Lookup of AS number and country
@@ -586,6 +585,24 @@ sends it.
probes these rules caught (see "Rule files"). A list you give replaces the probes these rules caught (see "Rule files"). A list you give replaces the
default, so name these six ids in it to keep them off. default, so name these six ids in it to keep them off.
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names a file of exclusions, written as
Coraza directives. An exclusion switches one rule off for some fields or paths
only, where `SWWAF_WAF_DISABLED_RULES` switches it off everywhere.
`deploy/gitea/crs-exclusions.conf` holds gitea's.
- `servewaf` loads the file after `smallwebwaf`'s own setup rules and before
the Core Rule Set's rules. The Core Rule Set puts its own file of
exclusions that run before the rules in the same place.
- So the file holds only `SecAction` and `SecRule` lines that exclude with
`ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. These run with each
request, before the rules they exclude, those of phase 1 included.
- A rule that `SWWAF_WAF_DISABLED_RULES` switches off and the file names is
simply off.
- The start stops on a file that cannot be read or does not parse, whatever
`SWWAF_WAF_MODE` is.
- It stops too on a file that names a number from 900000 to 900999, the ids
of `smallwebwaf`'s own rules, alone or in a range. Comment lines are not
checked.
- `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. It takes - `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. It takes
at most `64K`, since `servewaf` gets no more of a body than that (see "The at most `64K`, since `servewaf` gets no more of a body than that (see "The
answer"). Form data and multipart bodies are read up to the size. JSON and XML answer"). Form data and multipart bodies are read up to the size. JSON and XML
@@ -615,7 +632,7 @@ sends it.
scanner from an ordinary client. In front of gitea, registry and git clients scanner from an ordinary client. In front of gitea, registry and git clients
get `404` and `401` by design, hundreds of times a minute. get `404` and `401` by design, hundreds of times a minute.
`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of `smallwebwaf` changes the Core Rule Set 4.25.0 in seven ways, since in front of
gitea it would otherwise refuse ordinary requests. The changes hold in front of gitea it would otherwise refuse ordinary requests. The changes hold in front of
every app, and no setting undoes them: every app, and no setting undoes them:
@@ -672,6 +689,10 @@ every app, and no setting undoes them:
must check it itself. must check it itself.
6. Responses are not inspected. A raw file from a repository, such as a shell 6. Responses are not inspected. A raw file from a repository, such as a shell
script, looks to the response rules like source code leaking from the server. script, looks to the response rules like source code leaking from the server.
7. The `from` parameter of a container registry's upload path,
`/v2/<image>/blobs/uploads/`, is not checked for command names (932260). When
a push reuses a layer, `from` names the image the layer comes from. An image
such as `owner/python3-app` would have that push refused.
The parameter and cookie names in these changes are matched without regard to The parameter and cookie names in these changes are matched without regard to
case, as Coraza matches them. case, as Coraza matches them.
+136
View File
@@ -0,0 +1,136 @@
# Gitea behind traefik, with rproxy and servewaf in front of it.
# docs/gitea.md says what each setting here does and why. Before starting it:
# - set SMALLWEBWAF_IMAGE to the smallwebwaf image, named by digest;
# - replace git.example.org with the site's host name;
# - replace each 10.99.0 with the first three numbers of a /24 subnet that
# no other network on the host uses. The /24 must lie inside 10.0.0.0/8,
# 172.16.0.0/12 or 192.168.0.0/16, because rproxy believes
# X-Forwarded-For only from those by default. Replace each fd10:99 with
# the first two hextets of an unused /64 subnet whose third and fourth
# hextets are 0, as in fd10:99::/64;
# - set GITEA_SECRET_KEY to the output of
# `docker run --rm <gitea's image below> gitea generate secret SECRET_KEY`,
# once per install, and keep it.
# Then run `docker compose up --detach` in this directory.
# script/gitea-setup checks this setup.
name: gitea
x-logging: &logging
driver: json-file
options:
max-size: 10m
max-file: "5"
services:
traefik:
# traefik v3.7.17, 2026-11-03
image: traefik@sha256:68ae5fe5b64ca166704b746a9e8afcd385e36b3f569a5691401bf652d57b3748
# traefik replaces the X-Forwarded-For header a visitor sends. Keep it
# so: otherwise a visitor could name an address on the shared network,
# and skip servewaf's checks.
command:
- --providers.docker.exposedbydefault=false
- --entrypoints.web.address=:80
- --entrypoints.web.http.redirections.entrypoint.to=websecure
- --entrypoints.websecure.address=:443
# Until a certificate resolver is added here, as for your other
# sites, traefik answers with a certificate it made itself.
- --entrypoints.websecure.http.tls=true
# traefik's own limit of 60 seconds for a whole request would cut
# off an upload long before rproxy's limit of 30 minutes.
- --entrypoints.websecure.transport.respondingTimeouts.readTimeout=30m
ports:
- 80:80
- 443:443
volumes:
- /var/run/docker.sock:/var/run/docker.sock:ro
networks:
shared:
# Containers on the shared network find traefik under the
# site's name, and so reach it from their own address.
aliases:
- git.example.org
restart: unless-stopped
gitea:
# gitea 1.27.3, 2026-08-29
image: gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef
environment:
GITEA__server__ROOT_URL: https://git.example.org/
# Gitea starts at its defaults, with SQLite, instead of showing its
# install page to whoever opens it first.
GITEA__security__INSTALL_LOCK: "true"
GITEA__security__SECRET_KEY: ${GITEA_SECRET_KEY:?set GITEA_SECRET_KEY, see docs/gitea.md}
# Accounts are made with `gitea admin user create`.
GITEA__service__DISABLE_REGISTRATION: "true"
# rproxy's addresses. Gitea then takes each visitor's address from
# the X-Real-Ip header traefik sets and rproxy passes on.
GITEA__security__REVERSE_PROXY_TRUSTED_PROXIES: 10.99.0.2,fd10:99::2
volumes:
- gitea-data:/data
networks:
- shared
restart: unless-stopped
rproxy:
image: ${SMALLWEBWAF_IMAGE:?set SMALLWEBWAF_IMAGE to the smallwebwaf image}
environment:
SWRP_WAF_BACKEND: http://servewaf:8080
SWRP_UPSTREAM_URL: http://gitea:3000
# Pushes, LFS objects, packages and release attachments of up to
# 2 GiB, sent in up to 30 minutes.
SWRP_MAX_REQUEST_SIZE: 2G
SWRP_MAX_REQUEST_TIME_MS: "1800000"
# 15 minutes for a clone or a download, half the default.
SWRP_MAX_RESPONSE_TIME_MS: "900000"
labels:
traefik.enable: "true"
traefik.http.routers.gitea.rule: Host(`git.example.org`)
traefik.http.routers.gitea.entrypoints: websecure
traefik.http.services.gitea.loadbalancer.server.port: "8080"
networks:
shared:
ipv4_address: 10.99.0.2
ipv6_address: fd10:99::2
logging: *logging
restart: unless-stopped
servewaf:
image: ${SMALLWEBWAF_IMAGE:?set SMALLWEBWAF_IMAGE to the smallwebwaf image}
environment:
SMALLWEBWAF_COMMAND: servewaf
# The containers on the shared network skip the checks. Add the
# address of an automation fleet on another host here too; without
# that, it is counted like any client.
SWWAF_ALLOW_NETS: 10.99.0.128/25,fd10:99:0:0:8000::/65
# The Core Rule Set's exclusions for Gitea's searches, file names
# and branch names.
SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: /etc/smallwebwaf/crs-exclusions.conf
volumes:
- servewaf-state:/var/lib/smallwebwaf
- ./crs-exclusions.conf:/etc/smallwebwaf/crs-exclusions.conf:ro
networks:
- shared
logging: *logging
restart: unless-stopped
networks:
# The network traefik and Gitea share. Its containers get addresses in the
# upper half of each subnet, which servewaf lets skip the checks. Each
# subnet's gateway, the host's own address on it, stays out of that half:
# requests docker passes on from a published port can come from it.
shared:
enable_ipv6: true
ipam:
config:
- subnet: 10.99.0.0/24
ip_range: 10.99.0.128/25
gateway: 10.99.0.1
- subnet: fd10:99::/64
ip_range: fd10:99:0:0:8000::/65
gateway: fd10:99::1
volumes:
gitea-data:
servewaf-state:
+150
View File
@@ -0,0 +1,150 @@
# The Core Rule Set's exclusions for Gitea, which servewaf loads from
# SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE. servewaf loads this file before the Core
# Rule Set's rules, so it holds only exclusions made while a request is checked:
# SecAction and SecRule lines with ctl:ruleRemoveTargetById or
# ctl:ruleRemoveById. Each switches one rule off for the Gitea fields or paths
# where it refused ordinary Gitea requests. A SecAction applies on every path, a
# SecRule on the paths it matches. Gitea sends these in the query string:
# - q: the text of every search;
# - file, path, template, body and redirect_to: a file name, alone or in
# JSON or a link;
# - name, tag, rule_name and refSubUrl: a branch or tag name.
# docs/gitea.md says how to add an exclusion.
# Searches.
# 930110: a search for ../
SecAction "id:10001,phase:1,pass,nolog,ctl:ruleRemoveTargetById=930110;ARGS:q"
# 931100: a search for http://127.0.0.1:8080
SecAction "id:10002,phase:1,pass,nolog,ctl:ruleRemoveTargetById=931100;ARGS:q"
# 932140: a search for if (a == b) {
SecAction "id:10003,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932140;ARGS:q"
# 932235: a search for `make check`
SecAction "id:10004,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932235;ARGS:q"
# 932250: a search for ls -la
SecAction "id:10005,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932250;ARGS:q"
# 932380: a search for `whoami`
SecAction "id:10006,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932380;ARGS:q"
# 933100: a search for <?php echo
SecAction "id:10007,phase:1,pass,nolog,ctl:ruleRemoveTargetById=933100;ARGS:q"
# 933130: a search for $_GET[
SecAction "id:10008,phase:1,pass,nolog,ctl:ruleRemoveTargetById=933130;ARGS:q"
# 933135: a search for ${VAR}
SecAction "id:10009,phase:1,pass,nolog,ctl:ruleRemoveTargetById=933135;ARGS:q"
# 934110: a search for http://127.0.0.1:8080
SecAction "id:10010,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934110;ARGS:q"
# 941100: a search for <script>
SecAction "id:10011,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941100;ARGS:q"
# 941110: a search for <script>, or a page opened from its results
SecAction "id:10012,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=941110;ARGS:q,\
ctl:ruleRemoveTargetById=941110;REQUEST_HEADERS:Referer"
# 941120: a page opened from the results of a search for onClick={
SecAction "id:10013,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=941120;REQUEST_HEADERS:Referer"
# 941160: a search for <img src=, or a page opened from its results
SecAction "id:10014,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=941160;ARGS:q,\
ctl:ruleRemoveTargetById=941160;REQUEST_HEADERS:Referer"
# 941390: a search for fetch(
SecAction "id:10015,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941390;ARGS:q"
# 942100: a search for SELECT * FROM users WHERE, or a page opened from the
# results of a search for LIKE '%foo%'
SecAction "id:10016,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=942100;ARGS:q,\
ctl:ruleRemoveTargetById=942100;REQUEST_HEADERS:Referer"
# 942190: a search for union select
SecAction "id:10017,phase:1,pass,nolog,ctl:ruleRemoveTargetById=942190;ARGS:q"
# 942230: a search for if (a == b) {
SecAction "id:10018,phase:1,pass,nolog,ctl:ruleRemoveTargetById=942230;ARGS:q"
# 944100: a search for java.lang.Runtime, or a page opened from its results
SecAction "id:10019,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=944100;ARGS:q,\
ctl:ruleRemoveTargetById=944100;REQUEST_HEADERS:Referer"
# Searches, and file, branch and tag names.
# 930120: a search for package.json, or the commit graph of .gitignore
SecAction "id:10020,phase:1,pass,nolog,ctl:ruleRemoveTargetById=930120;\
ARGS:/^(?:q|file|template|body|refsuburl|name|tag|rule_name)$/"
# 932160: a search for /bin/sh, or the API's contents of bin/docker-entrypoint
SecAction "id:10021,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932160;\
ARGS:/^(?:q|file|template|body|refsuburl|name|tag|rule_name)$/"
# 932260: a search for python3, or deleting the branch docker-build
SecAction "id:10022,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;\
ARGS:/^(?:q|file|template|body|refsuburl|name|tag|rule_name)$/"
# Searches, and file names in the query string.
# 932120: a search for Invoke-WebRequest, or the commits of install-script.sh
SecAction "id:10023,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932120;\
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
# 932130: a search for $(pwd), or an issue opened from the template [draft].md
SecAction "id:10024,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932130;\
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
# 934100: a search for process.env, or the commits of process.env.d.ts
SecAction "id:10025,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934100;\
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
# 934130: a search for __proto__, or the commits of __proto__.json
SecAction "id:10026,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934130;\
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
# 941130: the commits of index.xhtml
SecAction "id:10027,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941130;\
ARGS:/^(?:file|path|template|body|redirect_to)$/"
# 941180: a search for document.cookie, or the commits of document.write.js
SecAction "id:10028,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941180;\
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
# 942140: a search for information_schema, or the commits of
# information_schema.sql
SecAction "id:10029,phase:1,pass,nolog,ctl:ruleRemoveTargetById=942140;\
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
# 944110: a search for java.lang.Runtime, or the commits of Runtime.java
SecAction "id:10030,phase:1,pass,nolog,ctl:ruleRemoveTargetById=944110;\
ARGS:/^(?:q|file|path|template|body|redirect_to)$/"
# 944130: a search for java.lang.Runtime, the commits of java.lang.String.md,
# or any page opened from that file's page, which names it in the Referer
SecAction "id:10031,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=944130;\
ARGS:/^(?:q|file|path|template|body|redirect_to)$/,\
ctl:ruleRemoveTargetById=944130;REQUEST_HEADERS:Referer"
# File names in the path of a file's page, raw file, blame, history and edit
# page, and of the API's calls for its contents.
# 920500: the page of notes.txt~
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
"id:10032,phase:1,pass,nolog,ctl:ruleRemoveTargetById=920500;REQUEST_FILENAME"
# 934100: the page of process.env.d.ts
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
"id:10033,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934100;REQUEST_FILENAME"
# 941130: the page of docs/a b.xhtml
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
"id:10034,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941130;REQUEST_FILENAME"
# 944130: the page of java.lang.String.md
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
"id:10035,phase:1,pass,nolog,ctl:ruleRemoveTargetById=944130;REQUEST_FILENAME"
# The file name, in name, of an attachment uploaded through the API to a
# release, an issue or a comment.
# 932120: the attachment install-script.sh
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
"id:10036,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932120;ARGS:name"
# 934100: the attachment process.env.d.ts
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
"id:10037,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934100;ARGS:name"
# 934130: the attachment __proto__.json
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
"id:10038,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934130;ARGS:name"
# 941130: the attachment index.xhtml
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
"id:10039,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941130;ARGS:name"
# 941180: the attachment document.write.js
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
"id:10040,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941180;ARGS:name"
# 942140: the attachment information_schema.sql
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
"id:10041,phase:1,pass,nolog,ctl:ruleRemoveTargetById=942140;ARGS:name"
# 944130: the attachment java.lang.String.md
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
"id:10042,phase:1,pass,nolog,ctl:ruleRemoveTargetById=944130;ARGS:name"
# The header Content-Range of an artifact upload.
# 920450: an artifact upload from actions/upload-artifact@v3
SecRule REQUEST_FILENAME "@rx ^/api/actions_pipeline/_apis/pipelines/workflows/[0-9]+/artifacts/[^/]+/upload$" \
"id:10043,phase:1,pass,nolog,ctl:ruleRemoveTargetById=920450;REQUEST_HEADERS_NAMES:Content-Range"
+164 -178
View File
@@ -1,220 +1,206 @@
# smallwebwaf in front of gitea # smallwebwaf in front of gitea
These notes record what `smallwebwaf` does in front of gitea at the default `deploy/gitea/compose.yaml` runs gitea behind traefik, with `rproxy` and
settings. They are observations, not requirements; the requirements are in `servewaf` in front of it. This document says what that setup sets and why, and
[`SPEC.md`](../SPEC.md). `script/gitea-requests` runs gitea behind `rproxy` and what the defaults would do without it. [`SPEC.md`](../SPEC.md) says how
`servewaf`, and sends through them a sample of gitea's ordinary use and the `smallwebwaf` behaves.
requests named below. It reports each request the Core Rule Set refuses, so that
these notes can be checked again when gitea or the Core Rule Set changes
version.
## What the defaults cover ## The setup
- SSH access to gitea does not pass through `smallwebwaf` and is not protected Four containers share one Docker network, the shared network:
by it.
- A clone is a response. It fits the defaults of 30 minutes and 1 GiB unless the
repository is large or the link slow. A repository whose clone is larger than
1 GiB needs `SWRP_MAX_RESPONSE_SIZE` raised, and the byte limits with it
("Byte limits, per client" in `SPEC.md`).
- A push is a request, and so is every other upload:
- a git push;
- an LFS object;
- a container image layer;
- a package file;
- a release attachment.
At the defaults, an upload larger than 100 MiB or taking more than 60 - traefik, which routes the site's HTTPS requests to `rproxy`, and redirects
seconds is cut off ("`rproxy`'s limits" in `SPEC.md`). A gitea that takes plain HTTP to HTTPS;
large uploads needs `SWRP_MAX_REQUEST_SIZE` and `SWRP_MAX_REQUEST_TIME_MS` - `rproxy`, in front of gitea;
raised to fit. - gitea 1.27.3, as its image comes;
- `servewaf`, with a volume for its state files, and with
`deploy/gitea/crs-exclusions.conf` mounted read-only.
- Archive download and blame or history pages are what scrapers hammer. The To start it:
request limits do most of the work there.
- An Actions runner that reaches gitea through `smallwebwaf` sends requests all
day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
43,200 requests a day. While it runs a job, it reports the job's log and state
every second instead, two requests a second. So one runner busy all day sends
120 requests a minute and 172,800 a day, under the request limits. Two such
runners on one address, each busy all day, make 345,600 and pass the day limit
of 300,000. Three busy at once make 21,600 an hour and pass the hour limit of
20,000. Five busy at once make 600 a minute and pass the minute limit of 500.
Their address is then banned, their running jobs fail, and each repeat within
a day makes the ban longer. Newer runners ask for work less often. Put the
runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
## What the Core Rule Set lets through 1. In `deploy/gitea/compose.yaml`, replace `git.example.org` with the site's
host name.
2. Replace each `10.99.0` with the first three numbers of a `/24` subnet that no
other network on the host uses. The `/24` must lie inside `10.0.0.0/8`,
`172.16.0.0/12` or `192.168.0.0/16`, because `rproxy` believes
`X-Forwarded-For` only from those by default. Replace each `fd10:99` with the
first two hextets of an unused `/64` subnet whose third and fourth hextets
are 0, as in `fd10:99::/64`.
3. Set `SMALLWEBWAF_IMAGE` to the `smallwebwaf` image, named by digest.
4. Set `GITEA_SECRET_KEY` to the output of
`docker run --rm <gitea's image> gitea generate secret SECRET_KEY`, with
gitea's image as `deploy/gitea/compose.yaml` names it. Do this once per
install, and keep the key.
5. Run `docker compose up --detach` in `deploy/gitea`, with
`crs-exclusions.conf` beside `compose.yaml`.
6. Make gitea's first user with `gitea admin user create`, in gitea's container,
as the user `git`.
At the defaults, the Core Rule Set lets gitea's ordinary use through, apart from Sign-up is off: accounts are made with `gitea admin user create`. gitea starts
the refusals in the next section: at its defaults, with SQLite. traefik answers with a certificate it made itself
until you add a certificate resolver, as for your other sites. SSH to gitea does
not pass through `smallwebwaf`; publish gitea's SSH port yourself if you use it.
- browsing and views of files in a repository, with their history, blame and the `make gitea-setup` checks the setup. It builds the image, brings the setup up
file tree; under a name of its own, and checks:
- diffs, including their hidden lines and large files, and pull request review;
- git's clone, fetch and push over HTTP;
- signing in, including the return to the page a visitor came from, and sign-in
with Git Credential Manager, git-credential-oauth or tea;
- the API's calls for a file and its commits;
- pushing and pulling container images and packages;
- Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`;
- posting issues, pull requests, comments, wiki pages and files saved in the web
editor, code included, since no body is read;
- the page gitea shows after a change, with its message naming what was done.
## What the Core Rule Set still refuses - that a page load, a push and a clone sent through traefik pass through
`rproxy`, and that `servewaf` checks them;
- that gitea logs the visitor's address, not `rproxy`'s;
- that a request from a container on the shared network skips the checks;
- that the Core Rule Set refuses none of the ordinary gitea requests in
`internal/waf/testdata/gitea-requests.txt`, sent by a signed-in user;
- that it refuses none of the API calls with JSON bodies;
- that it refuses no fetch;
- that it refuses SQL injection, path traversal and script injection in a query
string;
- that with `servewaf` stopped, gitea still answers through `rproxy`.
Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts ## Uploads and downloads
toward). The request log names the rule in `waf_rule_ids`, which
`SWWAF_WAF_DISABLED_RULES` can switch off.
### A query string that reads as an attack An upload is a request:
Most often this is a search. A search term is a query parameter, which the Core - a git push;
Rule Set checks against its lists of system files and commands (see the fourth - an LFS object;
change under "Attack detection" in `SPEC.md`). A release attachment uploaded - a container image layer;
through the API as `docker-compose.yml` is refused the same way. Examples: - a package file;
- a release attachment.
- a name on those lists, such as `package.json`, `.gitignore` or The setup lets an upload of up to 2 GiB, sent in up to 30 minutes, through:
`docker-compose.yml`;
- text that starts with a command name, such as `python3` or `ssh key`, or has
one right after a `/` in its first word, such as `feat/docker-support`;
- a shell command with its options or a system path (`ls -la`, `sed -i`,
`/bin/sh`), or a command in backticks;
- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`) or an SQL
statement (`SELECT * FROM users WHERE`);
- a URL naming an IP address or `localhost`.
### A branch, tag or file name that starts with a command name - `SWRP_MAX_REQUEST_SIZE` is `2G`. At the default, an upload over 100 MiB is cut
off.
- `SWRP_MAX_REQUEST_TIME_MS` is `1800000`, 30 minutes. At the default, an upload
that takes over 60 seconds is cut off.
- traefik's `readTimeout` is 30 minutes. traefik's own default of 60 seconds for
a whole request would cut off an upload first.
The Core Rule Set's list of commands (932260) holds names such as `docker-`, A clone or a download is a response:
`python3`, `ansible`, `base64` and `whoami`. A query parameter holding a branch,
tag or file name that starts with one is refused. So is one that has such a name
right after a `/`, such as `feat/docker-support`, `fix/docker-build` or
`renovate/docker-build-push-action-6.x`. gitea's own pages send the name in a
query parameter that 930120, 932160 and 932260 still check:
- `refSubUrl`, the branch or tag of a directory listing. A listing sends it when - `SWRP_MAX_RESPONSE_TIME_MS` is `900000`, 15 minutes, half the default. A clone
it asks for its entries' last commits in a second request. It does so whenever or download that takes longer is cut off.
gitea takes more than a second to work them out; - `SWRP_MAX_RESPONSE_SIZE` stays at its default, 1 GiB. A repository whose clone
- `name`, when a branch is deleted or restored on the branches page; is larger needs it raised.
- `tag`, when a release is started from a tag;
- `template`, the file of an issue template, when an issue is opened from it;
- `rule_name`, when a branch protection rule is opened for editing in the
repository's settings or has just been saved.
What a visitor sees, for a branch named `docker-build`: The byte limits stay at their defaults: no minute limit, 50 GiB an hour and 100
GiB a day.
- A directory listing that makes that second request leaves those last commits ## The checks
out and shows an error.
- The branches page shows an error instead of deleting or restoring the branch.
- A release started from such a tag gets the `403` answer of `smallwebwaf` in
place of its form. So does an issue opened from such a template, and such a
rule opened for editing.
- Saving such a rule stores it. gitea then returns to the branch settings page,
which gets the `403` answer, so the save looks as if it failed.
### A file name in a query parameter that trips the code rules The Core Rule Set runs in `block` mode, the default, with gitea's exclusions
(see "The Core Rule Set's exclusions" below).
The Core Rule Set's rules for Java, script, PowerShell and SQL find what they Every other check stays at its default. A client may make 120 requests a minute,
look for in some file names. This holds too in the ten query parameters that 1,500 an hour and 6,000 a day. Archive downloads and blame and history pages are
930120, 932160 and 932260 do not check (see the fourth change under "Attack what scrapers hammer, and these limits do most of the work there.
detection" in `SPEC.md`). Examples:
- `runtime` in OpenJDK's `src/java.base/share/classes/java/lang/Runtime.java` ## The shared network
(944110);
- a Java class, as in `java.lang.String.md` (944130);
- `document.write.js` (941180);
- `process.env.d.ts` (934100);
- `__proto__` (934130);
- the PowerShell command `Install-Script`, as in `install-script.sh` (932120);
- a database's name, as in PostgreSQL's `information_schema.sql` (942140).
For such a file, these requests are refused: Containers on the shared network skip `servewaf`'s checks:
- the API's list of its commits (`path`); - The shared network's subnets are `10.99.0.0/24` and `fd10:99::/64`. Docker
- the API's call for the contents of files that names it in its query (`body`); gives its containers addresses in the upper half of each, `10.99.0.128/25` and
- the commit graph of that file alone (`file`); `fd10:99:0:0:8000::/65`, which `SWWAF_ALLOW_NETS` names.
- the hidden lines of its diff (`path`), which stay hidden; - traefik is known on the shared network by the site's host name. So a container
- the sign-in page, for a visitor who opens a page of the file that needs there that asks for the site reaches traefik from its own address. An
signing in, such as its edit page (`redirect_to`). The visitor can still sign automation fleet in such a container skips the checks.
in by opening the sign-in page itself. - Each subnet's gateway, `10.99.0.1` and `fd10:99::1`, is the host's own address
on it. Requests docker passes on from a published port can come from the
gateway, so the gateways are left out of `SWWAF_ALLOW_NETS`. The network has
IPv6, so a visitor over IPv6 reaches traefik from its own address, and
`servewaf` checks it as itself.
- A container on the shared network that asks for the site at the host's public
address arrives from the gateway. It is checked like any visitor.
- A fleet on another host can have its address added to `SWWAF_ALLOW_NETS`.
Without that, it is counted like any client.
- traefik replaces the `X-Forwarded-For` a visitor sends, by default. Keep it
so: otherwise a visitor could name an address on the shared network, and skip
the checks.
- Anything on the shared network can reach gitea and `servewaf` around `rproxy`.
Put only containers you trust on it.
### A path ## gitea's settings
- A file name ending in `~`, or an `.xhtml` file whose path holds a space. - `REVERSE_PROXY_TRUSTED_PROXIES` names `rproxy`'s fixed addresses, `10.99.0.2`
- A path that holds what the Core Rule Set takes for a Java class (944130), as and `fd10:99::2`. gitea then takes each visitor's address from the `X-Real-Ip`
`java.lang.String.md` and header, which traefik sets and `rproxy` passes on. Without it, gitea logs
`META-INF/services/org.apache.commons.logging.LogFactory` do. every visitor as `rproxy`.
- A path that holds what it takes for Node.js code (934100), as - `INSTALL_LOCK` is `true`, as "The setup" above says.
`process.env.d.ts` does.
- A path with a directory named `...` (930100, 930110), such as `.../notes.txt`.
For such a file, these are refused: ## Methods
- the page of the file; `rproxy` passes on only these methods: `GET`, `HEAD`, `POST`, `OPTIONS`, `PUT`,
- its raw file; `DELETE`, `PATCH` and `QUERY`. gitea 1.27.3 needs no other. Its routes use:
- its history and its blame;
- its edit page.
The directory that holds it lists it, and a clone carries it. - pages and the API: `GET`, `HEAD`, `POST`, `PUT`, `PATCH` and `DELETE`;
- git's push, clone and fetch over HTTP: `GET`, `POST` and `OPTIONS`;
- LFS: `GET`, `POST` and `PUT`;
- the container registry and the other package registries: `GET`, `HEAD`,
`POST`, `PUT`, `PATCH` and `DELETE`;
- Actions runners: `POST`.
### A container image push that reuses a layer ## The log
A push can reuse a layer of an image docker has pushed to that gitea before. `rproxy` and `servewaf` write each query string to the log whole. A token sent
docker then asks gitea to take the layer from that image, naming it in the query in the query string, as `?token=`, is then in the log. Send it in the
parameter `from`, which 932260 checks. The request is refused when the image's `Authorization` header instead.
owner starts with an entry on the list of commands. It is refused too when the
image's name after the owner starts with one. Examples are `owner/python3-app`,
`owner/docker-base`, and any image of an owner named `ansible` or `docker-team`.
What happens next depends on docker's image store. A docker that keeps its ## Actions runners
images in containerd's image store fails the push, and fails each new try the
same way. A new install of Docker Engine 29 does this. A docker that keeps them
in its own older image store uploads the layer instead. The push then succeeds
after one refusal for each layer it reuses, and each of those refusals counts
toward the error burst ("Attack detection" in `SPEC.md`).
### Artifact uploads from `actions/upload-artifact@v3` A runner on the shared network skips the checks. One on another host is counted
like any client. A runner older than version 0.4 (April 2026) asks for work
every 2 seconds, 1,800 requests an hour. While it runs a job, it reports the
job's log and state every second instead, two requests a second.
This action sends the header `Content-Range`, which the Core Rule Set refuses So one idle runner makes 1,800 requests an hour, more than the hour limit of
(920450). The action sends two files at a time, does not retry a `403`, and 1,500. A busy one, at two requests a second, passes the minute limit of 120
sends no further file once one is refused. So an upload is refused once or within two minutes. Its address is then banned, its running jobs fail, and each
twice, however many files it holds, and the step fails. One upload does not ban repeat within a day makes the ban longer. Newer runners ask for work less often.
the runner. The error burst ("Attack detection" in `SPEC.md`) bans it after more Put the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
than 30 such refusals from its address in a minute, as when many jobs upload at
once. `actions/upload-artifact@v4` does not send the header.
## What body inspection refuses ## The Core Rule Set's exclusions
With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In `deploy/gitea/crs-exclusions.conf` switches single Core Rule Set rules off where
front of gitea it then refuses these, until their rule ids, named in the request they refused ordinary gitea requests. `servewaf` loads it from
log, are added to `SWWAF_WAF_DISABLED_RULES`: `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE`, before the Core Rule Set's rules. Each
exclusion covers one rule, in these places only:
- issue and comment text, wiki pages and files saved in the web editor that hold - `q`, the text of every search;
shell commands or code (932125, 932235, 932250 and others); - the query parameters that hold a file name: `file`, `path`, `template`, `body`
- package descriptions that show code; and `redirect_to`;
- PyPI uploads (922130); - the query parameters that hold a branch or tag name: `name`, `tag`,
- attachments named like `debug.log` or `config.yml` (932180). `rule_name` and `refSubUrl`;
- `Referer`, on a page opened from search results or from a file's page;
- the path of a file's page, raw file, blame, history and edit page, and of the
API's call for its contents;
- `name`, the file name of an attachment uploaded through the API;
- the header `Content-Range` of an Actions artifact upload.
## What `script/gitea-requests` sends gitea reads these as text to search for, as names in a repository, or, for
`Content-Range`, as the part of a file an upload carries. It never runs them as
SQL, a command or a page's script. So an attack that only the excluded rules
would catch there reaches gitea and does nothing. Every other field and path
keeps every rule.
Of the requests above, the script sends: To add an exclusion for a request the Core Rule Set refuses:
- for each file named under "A file name in a query parameter that trips the 1. Find the request in `servewaf`'s log. Its line gives the path and query, and
code rules", the API's list of its commits; names the rules in `waf_rule_ids`.
- for `Runtime.java` alone, the commit graph of that file, the hidden lines of 2. For each rule, add a line with `ctl:ruleRemoveTargetById` for the field it
its diff, and the API's call for the contents of files that names it; matched. For a field on every path, copy one of the file's `SecAction` lines.
- the sign-in page for a visitor who opens the edit page of `document.write.js`; For some paths only, copy one of its `SecRule REQUEST_FILENAME` lines. Give
- the pages of `java.lang.String.md`, it a new id. An id from 900000 to 900999 stops the start.
`META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts` 3. Above it, write a one-line comment naming the request it refused.
and `.../notes.txt`; 4. Add the request to `internal/waf/testdata/gitea-requests.txt`, and run
- one container image push that reuses a layer, which it does not try again. `make check`.
5. Restart `servewaf`.
It does not send the raw file, history, blame or edit page of those four files, The Core Rule Set still refuses the page of a file in a directory named `...`,
or the pages of the directories `META-INF/services` and `...`. It also sends a which reads as path traversal.
short sample of gitea's ordinary use:
- signing in; With `SWWAF_WAF_BODY_LIMIT` set, it also refuses issue text, comments, wiki
- pages of files and commits, and a diff; pages and files saved in the web editor that hold shell commands or code.
- the API;
- a push, and clone and fetch over HTTP; `make gitea-requests` runs gitea behind `rproxy` and `servewaf` at the defaults,
- two container image pushes. without these exclusions. It reports each request the Core Rule Set refuses
there.
+25 -15
View File
@@ -99,19 +99,22 @@ chooses between the two (see "The image's command" below).
## Request rate limits, per client ## Request rate limits, per client
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `500`, about 8 a second): the most - `SWWAF_RATE_LIMIT_PER_MINUTE` (default `120`, 2 a second): the most requests a
requests a client may make in a minute. A count or `off`. The default catches, client may make in a minute. A count or `off`. The default leaves room for a
within seconds, a client firing requests as fast as it can. It leaves room for person opening two gitea pages at once with an empty cache, up to 60 files
a fleet of about 20 agents at its busiest, about 120 requests a minute. It each.
also leaves room for a few quick first visits to gitea with an empty cache, - `SWWAF_RATE_LIMIT_PER_HOUR` (default `1500`): the most requests a client may
about 40 files each. make in an hour. A count or `off`. It is the main limit against scrapers. A
- `SWWAF_RATE_LIMIT_PER_HOUR` (default `20000`, about 5.5 a second): the most person reading code in gitea for an hour, a page every 20 seconds, makes
requests a client may make in an hour. A count or `off`. The default leaves about 900. A scraper making one request every 2 seconds makes 1,800 an hour,
room for an automation fleet at work. It catches a client that keeps up more more than the limit.
than about 5.5 a second for an hour. - `SWWAF_RATE_LIMIT_PER_DAY` (default `6000`): the most requests a client may
- `SWWAF_RATE_LIMIT_PER_DAY` (default `300000`, about 3.5 a second): the most make in a day. A count or `off`. The default is four hours at the hour limit.
requests a client may make in a day. A count or `off`. The default catches a A scraper making one request every 10 seconds makes 8,640 a day, more than the
scraper that keeps up 4 a second around the clock. limit.
- An Actions runner asks gitea for work all day and can pass the limits. Put the
runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. Do the same for a
network where many people share one address, such as an office.
- Counting cannot catch a client that stays under all three limits (see "Request - Counting cannot catch a client that stays under all three limits (see "Request
rate limits, per client" in [`SPEC.md`](../SPEC.md)). rate limits, per client" in [`SPEC.md`](../SPEC.md)).
- `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from - `SWWAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
@@ -119,8 +122,8 @@ chooses between the two (see "The image's command" below).
## Byte limits, per client ## Byte limits, per client
- `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `4G`), `SWWAF_BYTES_LIMIT_PER_HOUR` - `SWWAF_BYTES_LIMIT_PER_MINUTE` (default `off`), `SWWAF_BYTES_LIMIT_PER_HOUR`
(default `20G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `200G`): the most (default `50G`) and `SWWAF_BYTES_LIMIT_PER_DAY` (default `100G`): the most
bytes counted for a client in each window. A size or `off`. bytes counted for a client in each window. A size or `off`.
- `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count: - `SWWAF_BYTES_COUNT` (default `both`): which body bytes the byte limits count:
`response`, `request` or `both`. `response`, `request` or `both`.
@@ -189,6 +192,13 @@ chooses between the two (see "The image's command" below).
the Core Rule Set reads, a size of at most `64K`. A question carries no more the Core Rule Set reads, a size of at most `64K`. A question carries no more
of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one
byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)). byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule Set
exclusions, written as Coraza directives. It is loaded before the Core Rule
Set's rules, so it holds only `SecAction` and `SecRule` lines that exclude
with `ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. A file that cannot be
read or does not parse stops the start, whatever `SWWAF_WAF_MODE` is. So does
a file that names a number from 900000 to 900999 outside a comment, alone or
in a range. It has no `_FILE` form.
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client - `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
in a minute (see "Attack detection" in `SPEC.md`). A count or `off`. in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
+21 -6
View File
@@ -221,12 +221,15 @@ type Config struct {
// WAFExemptPaths the path prefixes it does not inspect // WAFExemptPaths the path prefixes it does not inspect
// (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body // (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body
// it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none. // it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
// CoreRuleSetExclusionsFile is the path of the file of its exclusions
// (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while there is none.
WAFMode string WAFMode string
WAFParanoiaLevel int WAFParanoiaLevel int
WAFAnomalyThreshold int WAFAnomalyThreshold int
WAFDisabledRules []int WAFDisabledRules []int
WAFExemptPaths []string WAFExemptPaths []string
WAFBodyLimit int64 WAFBodyLimit int64
CoreRuleSetExclusionsFile string
// TrapPaths are the paths a request for which is a clear sign of // TrapPaths are the paths a request for which is a clear sign of
// attack (SWWAF_TRAP_PATHS), each starting with / and without a ?. // attack (SWWAF_TRAP_PATHS), each starting with / and without a ?.
TrapPaths []string TrapPaths []string
@@ -468,13 +471,13 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""), AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""),
RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""), RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
DenyNets: env.netblocks("SWWAF_DENY_NETS", ""), DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "500"), RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "120"),
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "20000"), RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "1500"),
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "300000"), RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "6000"),
RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""), RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
BytesLimitPerMinute: env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "4G"), BytesLimitPerMinute: env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "off"),
BytesLimitPerHour: env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "20G"), BytesLimitPerHour: env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "50G"),
BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "200G"), BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "100G"),
BytesCount: env.bytesCount("SWWAF_BYTES_COUNT", "both"), BytesCount: env.bytesCount("SWWAF_BYTES_COUNT", "both"),
LookupSource: env.lookupSource("SWWAF_LOOKUP_SOURCE", "geojs"), LookupSource: env.lookupSource("SWWAF_LOOKUP_SOURCE", "geojs"),
LookupDBPath: env.value("SWWAF_LOOKUP_DB_PATH", ""), LookupDBPath: env.value("SWWAF_LOOKUP_DB_PATH", ""),
@@ -525,6 +528,8 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
"920340,920420,920440,920640,930130,930140"), "920340,920420,920440,920640,930130,930140"),
WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""), WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off), WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
CoreRuleSetExclusionsFile: env.filePath(
"SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"),
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"), TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"), ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"), LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
@@ -1189,6 +1194,16 @@ func (e *environment) logRemoteURL(name string) *url.URL {
return remote return remote
} }
// filePath reads a setting that is the path of a file, "" while it is
// unset. Its value names a file already, so, unlike the other settings,
// it has no _FILE form. Whoever uses the file reads it.
func (e *environment) filePath(name string) string {
path, _ := e.lookupEnv(name)
e.settings = append(e.settings, slog.String(name, path))
return path
}
// certificates reads a setting that is the path of a file of PEM // certificates reads a setting that is the path of a file of PEM
// certificates. Unset or empty, it is nil. Its value names a file // certificates. Unset or empty, it is nil. Its value names a file
// already, so, unlike the other settings, it has no _FILE form. // already, so, unlike the other settings, it has no _FILE form.
+16 -12
View File
@@ -87,6 +87,7 @@ const (
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES" wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS" wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT" wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
exclusionsFile = "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"
trapPaths = "SWWAF_TRAP_PATHS" trapPaths = "SWWAF_TRAP_PATHS"
errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD" errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
logRemoteURL = "SWWAF_LOG_REMOTE_URL" logRemoteURL = "SWWAF_LOG_REMOTE_URL"
@@ -244,9 +245,9 @@ func TestDefaults(t *testing.T) {
wantSettings(t, cfg, config.Config{ wantSettings(t, cfg, config.Config{
ListenAddr: ":8080", ListenAddr: ":8080",
Observe: false, Observe: false,
RateLimitPerMinute: 500, RateLimitPerMinute: 120,
RateLimitPerHour: 20000, RateLimitPerHour: 1500,
RateLimitPerDay: 300000, RateLimitPerDay: 6000,
BanResponse: 403, BanResponse: 403,
LimitBanDuration: time.Hour, LimitBanDuration: time.Hour,
LimitBanRepeatWindow: 24 * time.Hour, LimitBanRepeatWindow: 24 * time.Hour,
@@ -266,8 +267,8 @@ func TestDefaults(t *testing.T) {
LookupSource: defaultLookupSource, LookupTimeout: 800 * time.Millisecond, LookupSource: defaultLookupSource, LookupTimeout: 800 * time.Millisecond,
}) })
wantByteLimitSettings(t, cfg, config.Config{ wantByteLimitSettings(t, cfg, config.Config{
BytesLimitPerMinute: 4 << 30, BytesLimitPerHour: 20 << 30, BytesLimitPerMinute: 0, BytesLimitPerHour: 50 << 30,
BytesLimitPerDay: 200 << 30, BytesCount: "both", BytesLimitPerDay: 100 << 30, BytesCount: "both",
}) })
wantNetblocks(t, cfg.AllowNets) wantNetblocks(t, cfg.AllowNets)
@@ -308,7 +309,7 @@ func TestNoSingleRequestBreaksAByteLimitAtTheDefaults(t *testing.T) {
for _, limit := range []int64{ for _, limit := range []int64{
cfg.BytesLimitPerMinute, cfg.BytesLimitPerHour, cfg.BytesLimitPerDay, cfg.BytesLimitPerMinute, cfg.BytesLimitPerHour, cfg.BytesLimitPerDay,
} { } {
if largest > limit { if limit != 0 && largest > limit { // 0 is off
t.Errorf("a request of %d bytes breaks the byte limit of %d", largest, limit) t.Errorf("a request of %d bytes breaks the byte limit of %d", largest, limit)
} }
} }
@@ -539,13 +540,14 @@ func TestCoreRuleSetSettings(t *testing.T) {
environment{ environment{
wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10", wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10",
wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/", wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/",
wafBodyLimit: "32K", wafBodyLimit: "32K", exclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
}, },
config.Config{ config.Config{
WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10, WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10,
WAFDisabledRules: []int{942100, 920350}, WAFDisabledRules: []int{942100, 920350},
WAFExemptPaths: []string{"/api/", "/static/"}, WAFExemptPaths: []string{"/api/", "/static/"},
WAFBodyLimit: 32 << 10, WAFBodyLimit: 32 << 10,
CoreRuleSetExclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
}, },
}, },
{ {
@@ -566,6 +568,7 @@ func TestCoreRuleSetSettings(t *testing.T) {
WAFAnomalyThreshold: cfg.WAFAnomalyThreshold, WAFAnomalyThreshold: cfg.WAFAnomalyThreshold,
WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths, WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths,
WAFBodyLimit: cfg.WAFBodyLimit, WAFBodyLimit: cfg.WAFBodyLimit,
CoreRuleSetExclusionsFile: cfg.CoreRuleSetExclusionsFile,
} }
if !reflect.DeepEqual(got, tc.want) { if !reflect.DeepEqual(got, tc.want) {
t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want) t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want)
@@ -2271,12 +2274,12 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
rateLimitExemptNets: "", rateLimitExemptNets: "",
denyNets: "", denyNets: "",
rateLimitPerMinute: "45", rateLimitPerMinute: "45",
rateLimitPerHour: "20000", rateLimitPerHour: "1500",
rateLimitPerDay: "300000", rateLimitPerDay: "6000",
rateLimitExemptPaths: "", rateLimitExemptPaths: "",
bytesLimitPerMinute: "4G", bytesLimitPerMinute: "off",
bytesLimitPerHour: "20G", bytesLimitPerHour: "50G",
bytesLimitPerDay: "200G", bytesLimitPerDay: "100G",
bytesCount: "both", bytesCount: "both",
lookupSource: defaultLookupSource, lookupSource: defaultLookupSource,
lookupDBPath: "", lookupDBPath: "",
@@ -2327,6 +2330,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
wafDisabledRules: defaultWAFDisabledRules, wafDisabledRules: defaultWAFDisabledRules,
wafExemptPaths: "", wafExemptPaths: "",
wafBodyLimit: off, wafBodyLimit: off,
exclusionsFile: "",
trapPaths: "", trapPaths: "",
errorBurstThreshold: "30", errorBurstThreshold: "30",
logRemoteURL: "", logRemoteURL: "",
+61 -61
View File
@@ -16,9 +16,9 @@ import (
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rproxy" "sneak.berlin/go/smallwebwaf/internal/rproxy"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
// post is the start of a POST whose headers end with what follows it. // post is the start of a POST whose headers end with what follows it.
@@ -71,11 +71,11 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceServeWAFAllowsIt(t *testing.T) {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestSize: tc.limit, maxRequestSize: tc.limit,
}) })
@@ -85,11 +85,11 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceServeWAFAllowsIt(t *testing.T) {
t.Errorf("answered %+v, want 413", got) t.Errorf("answered %+v, want 413", got)
} }
if asked := servewaf.question(t); len(asked.body) != tc.asked { if asked := standIn.question(t); len(asked.body) != tc.asked {
t.Errorf("the question carries %d bytes, want %d", len(asked.body), tc.asked) t.Errorf("the question carries %d bytes, want %d", len(asked.body), tc.asked)
} }
report := servewaf.report(t) report := standIn.report(t)
if report.Status != http.StatusRequestEntityTooLarge { if report.Status != http.StatusRequestEntityTooLarge {
t.Errorf("report %+v, want status 413", report) t.Errorf("report %+v, want status 413", report)
} }
@@ -100,9 +100,9 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceServeWAFAllowsIt(t *testing.T) {
func TestADenyOutranksTheBodySizeLimit(t *testing.T) { func TestADenyOutranksTheBodySizeLimit(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, deny) standIn := startServeWAF(t, deny)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestSize: "1K", maxRequestSize: "1K",
}) })
@@ -132,12 +132,12 @@ func TestABodyNotAnnouncedOverItsLimitIsRefused(t *testing.T) {
var asked atomic.Bool var asked atomic.Bool
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, r *http.Request) { app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
asked.Store(true) asked.Store(true)
greet(w, r) greet(w, r)
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestSize: tc.limit, maxRequestSize: tc.limit,
}) })
@@ -165,12 +165,12 @@ func TestAResponseOverItsLimit(t *testing.T) {
t.Run("announced, is refused", func(t *testing.T) { t.Run("announced, is refused", func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) { app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(contentLengthHeader, strconv.Itoa(len(large))) w.Header().Set(contentLengthHeader, strconv.Itoa(len(large)))
_, _ = io.WriteString(w, large) _, _ = io.WriteString(w, large)
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxResponseSize: "1K", maxResponseSize: "1K",
}) })
@@ -179,7 +179,7 @@ func TestAResponseOverItsLimit(t *testing.T) {
t.Errorf("answered %d, want 502", got.status) t.Errorf("answered %d, want 502", got.status)
} }
if report := servewaf.report(t); report.Status != http.StatusBadGateway { if report := standIn.report(t); report.Status != http.StatusBadGateway {
t.Errorf("report %+v, want status 502", report) t.Errorf("report %+v, want status 502", report)
} }
}) })
@@ -187,13 +187,13 @@ func TestAResponseOverItsLimit(t *testing.T) {
t.Run("not announced, is cut off", func(t *testing.T) { t.Run("not announced, is cut off", func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) { app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, large[:1]) _, _ = io.WriteString(w, large[:1])
_ = http.NewResponseController(w).Flush() _ = http.NewResponseController(w).Flush()
_, _ = io.WriteString(w, large[1:]) _, _ = io.WriteString(w, large[1:])
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxResponseSize: "1K", maxResponseSize: "1K",
}) })
@@ -205,7 +205,7 @@ func TestAResponseOverItsLimit(t *testing.T) {
t.Errorf("read %d bytes (%v), want the answer cut off", len(text), err) t.Errorf("read %d bytes (%v), want the answer cut off", len(text), err)
} }
report := servewaf.report(t) report := standIn.report(t)
if report.Status != http.StatusOK || report.ResponseBytes > 1<<10 { if report.Status != http.StatusOK || report.ResponseBytes > 1<<10 {
t.Errorf("report %+v, want status 200 and at most 1K of the answer", report) t.Errorf("report %+v, want status 200 and at most 1K of the answer", report)
} }
@@ -217,11 +217,11 @@ func TestAHeadWhoseAnswerAnnouncesMoreThanTheResponseLimitIsPassedOn(t *testing.
const announced = "2048" const announced = "2048"
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) { app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(contentLengthHeader, announced) w.Header().Set(contentLengthHeader, announced)
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxResponseSize: "1K", maxResponseSize: "1K",
}) })
@@ -240,9 +240,9 @@ func TestASettingOfOffSetsNoLimit(t *testing.T) {
t.Run(setting, func(t *testing.T) { t.Run(setting, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
setting: "off", setting: "off",
}) })
@@ -251,7 +251,7 @@ func TestASettingOfOffSetsNoLimit(t *testing.T) {
wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/", wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/",
strings.NewReader("a body")))) strings.NewReader("a body"))))
if got := get(t, addr, proxy.HealthPath); got.status != http.StatusOK { if got := get(t, addr, servewaf.HealthPath); got.status != http.StatusOK {
t.Errorf("answered %+v, want rproxy's own 200", got) t.Errorf("answered %+v, want rproxy's own 200", got)
} }
}) })
@@ -270,9 +270,9 @@ func TestAVisitorTooSlowToSendTheFirstPartIsAnswered408(t *testing.T) {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "500", maxRequestTime: "500",
}) })
@@ -288,11 +288,11 @@ func TestAVisitorTooSlowToSendTheFirstPartIsAnswered408(t *testing.T) {
t.Errorf("line %v, want no disposition and 10 bytes", line) t.Errorf("line %v, want no disposition and 10 bytes", line)
} }
if len(servewaf.questions) != 0 { if len(standIn.questions) != 0 {
t.Error("servewaf was asked") t.Error("servewaf was asked")
} }
servewaf.wantNoReport(t) standIn.wantNoReport(t)
}) })
} }
} }
@@ -300,9 +300,9 @@ func TestAVisitorTooSlowToSendTheFirstPartIsAnswered408(t *testing.T) {
func TestAVisitorThatGoesAwayBeforeTheFirstPartIsNotAsked(t *testing.T) { func TestAVisitorThatGoesAwayBeforeTheFirstPartIsNotAsked(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy(t, servewaf.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, post+announcedLength(100)+"some of it") send(t, conn, post+announcedLength(100)+"some of it")
@@ -313,7 +313,7 @@ func TestAVisitorThatGoesAwayBeforeTheFirstPartIsNotAsked(t *testing.T) {
t.Errorf("line %v, want no answer and no disposition", line) t.Errorf("line %v, want no answer and no disposition", line)
} }
if len(servewaf.questions) != 0 { if len(standIn.questions) != 0 {
t.Error("servewaf was asked") t.Error("servewaf was asked")
} }
} }
@@ -334,9 +334,9 @@ func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy(t, servewaf.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, tc.request) send(t, conn, tc.request)
@@ -364,7 +364,7 @@ func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
t.Errorf("line %v, want no answer and no disposition", line) t.Errorf("line %v, want no answer and no disposition", line)
} }
if len(servewaf.questions) != 0 { if len(standIn.questions) != 0 {
t.Error("servewaf was asked") t.Error("servewaf was asked")
} }
}) })
@@ -404,11 +404,11 @@ func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
announced = 100 * questionBodyMaxBytes announced = 100 * questionBodyMaxBytes
) )
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) { app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNoContent) w.WriteHeader(http.StatusNoContent)
}) })
_, _, server := startServer(t, servewaf.URL, app.URL, map[string]string{ _, _, server := startServer(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: strconv.FormatInt(limit.Milliseconds(), 10), maxRequestTime: strconv.FormatInt(limit.Milliseconds(), 10),
}) })
@@ -454,9 +454,9 @@ func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, deny) standIn := startServeWAF(t, deny)
app := startApp(t, greet) app := startApp(t, greet)
_, _, server := startServer(t, servewaf.URL, app.URL, map[string]string{ _, _, server := startServer(t, standIn.URL, app.URL, map[string]string{
maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10), maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
}) })
@@ -484,9 +484,9 @@ func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) { func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "3000", maxRequestTime: "3000",
}) })
@@ -498,11 +498,11 @@ func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
t.Errorf("status %d, want 408", status) t.Errorf("status %d, want 408", status)
} }
if asked := servewaf.question(t); len(asked.body) != questionBodyMaxBytes { if asked := standIn.question(t); len(asked.body) != questionBodyMaxBytes {
t.Errorf("the question carries %d bytes", len(asked.body)) t.Errorf("the question carries %d bytes", len(asked.body))
} }
if report := servewaf.report(t); report.Status != http.StatusRequestTimeout { if report := standIn.report(t); report.Status != http.StatusRequestTimeout {
t.Errorf("report %+v, want status 408", report) t.Errorf("report %+v, want status 408", report)
} }
} }
@@ -553,8 +553,8 @@ func TestAnAppTooSlowToTakeTheRequestIsAnswered504(t *testing.T) {
writeSize = 32 << 10 writeSize = 32 << 10
) )
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
addr, _ := startRProxy(t, servewaf.URL, startAppThatTakesNothing(t), addr, _ := startRProxy(t, standIn.URL, startAppThatTakesNothing(t),
map[string]string{maxRequestTime: "3000"}) map[string]string{maxRequestTime: "3000"})
conn := dial(t, addr) conn := dial(t, addr)
@@ -580,14 +580,14 @@ func TestAnAppTooSlowToAnswerIsAnswered504(t *testing.T) {
t.Parallel() t.Parallel()
release := make(chan struct{}) release := make(chan struct{})
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
<-release <-release
}) })
t.Cleanup(func() { close(release) }) t.Cleanup(func() { close(release) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxResponseTime: "300", maxResponseTime: "300",
}) })
@@ -596,7 +596,7 @@ func TestAnAppTooSlowToAnswerIsAnswered504(t *testing.T) {
t.Errorf("answered %d, want 504", got.status) t.Errorf("answered %d, want 504", got.status)
} }
if report := servewaf.report(t); report.Status != http.StatusGatewayTimeout { if report := standIn.report(t); report.Status != http.StatusGatewayTimeout {
t.Errorf("report %+v, want status 504", report) t.Errorf("report %+v, want status 504", report)
} }
} }
@@ -607,7 +607,7 @@ func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
const start = "the start" const start = "the start"
release := make(chan struct{}) release := make(chan struct{})
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) { app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, start) _, _ = io.WriteString(w, start)
_ = http.NewResponseController(w).Flush() _ = http.NewResponseController(w).Flush()
@@ -619,7 +619,7 @@ func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
// The time limit is long enough for a loaded host to pass on the start of the // The time limit is long enough for a loaded host to pass on the start of the
// answer before it runs out. // answer before it runs out.
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxResponseTime: "3000", maxResponseTime: "3000",
}) })
@@ -642,7 +642,7 @@ func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
res.StatusCode, body, err, start) res.StatusCode, body, err, start)
} }
if report := servewaf.report(t); report.Status != http.StatusOK { if report := standIn.report(t); report.Status != http.StatusOK {
t.Errorf("report %+v, want the status the visitor got", report) t.Errorf("report %+v, want the status the visitor got", report)
} }
} }
@@ -656,7 +656,7 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
writeSize = 32 << 10 writeSize = 32 << 10
) )
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) { app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
part := make([]byte, writeSize) part := make([]byte, writeSize)
@@ -669,7 +669,7 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
}) })
// The time limit is long enough for a loaded host to start passing on the // The time limit is long enough for a loaded host to start passing on the
// answer before it runs out. // answer before it runs out.
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxResponseTime: "3000", maxResponseTime: "3000",
}) })
@@ -684,7 +684,7 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
t.Errorf("line %v, want status 200 and the answer cut off", line) t.Errorf("line %v, want status 200 and the answer cut off", line)
} }
report := servewaf.report(t) report := standIn.report(t)
if report.Status != http.StatusOK || report.ResponseBytes >= bodySize { if report.Status != http.StatusOK || report.ResponseBytes >= bodySize {
t.Errorf("report %+v, want status 200 and the answer cut off", report) t.Errorf("report %+v, want status 200 and the answer cut off", report)
} }
@@ -693,15 +693,15 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
func TestAnAppThatCannotBeReachedIsAnswered502WithAReport(t *testing.T) { func TestAnAppThatCannotBeReachedIsAnswered502WithAReport(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
addr, _ := startRProxy(t, servewaf.URL, nowhere, nil) addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
got := get(t, addr, "/") got := get(t, addr, "/")
if got.status != http.StatusBadGateway { if got.status != http.StatusBadGateway {
t.Errorf("answered %d, want 502", got.status) t.Errorf("answered %d, want 502", got.status)
} }
if report := servewaf.report(t); report.Status != http.StatusBadGateway { if report := standIn.report(t); report.Status != http.StatusBadGateway {
t.Errorf("report %+v, want status 502", report) t.Errorf("report %+v, want status 502", report)
} }
} }
@@ -709,11 +709,11 @@ func TestAnAppThatCannotBeReachedIsAnswered502WithAReport(t *testing.T) {
func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer(t *testing.T) { func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) { app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, greeting) _, _ = io.WriteString(w, greeting)
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "3000", maxRequestTime: "3000",
}) })
@@ -732,12 +732,12 @@ func TestAVisitorThatStopsSendingWhileTheAppWorksGetsNoAnswer(t *testing.T) {
t.Parallel() t.Parallel()
working := make(chan struct{}) working := make(chan struct{})
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) { app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
close(working) close(working)
<-r.Context().Done() <-r.Context().Done()
}) })
addr, out := startRProxy(t, servewaf.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n") send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
@@ -773,7 +773,7 @@ func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(t *testing
const answer = "the end of a long answer" const answer = "the end of a long answer"
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, r *http.Request) { app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
_, _ = io.Copy(io.Discard, r.Body) _, _ = io.Copy(io.Discard, r.Body)
_ = http.NewResponseController(w).Flush() _ = http.NewResponseController(w).Flush()
@@ -782,7 +782,7 @@ func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(t *testing
_, _ = io.WriteString(w, answer) _, _ = io.WriteString(w, answer)
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "2000", maxRequestTime: "2000",
}) })
@@ -802,18 +802,18 @@ func TestAVisitorThatStopsSendingGetsAnAnswerRProxyGivesByTheDeadline(t *testing
{"a deny", post, http.StatusForbidden}, {"a deny", post, http.StatusForbidden},
{ {
"the health check", "the health check",
"GET " + proxy.HealthPath + " HTTP/1.1\r\nHost: site.example\r\n", "GET " + servewaf.HealthPath + " HTTP/1.1\r\nHost: site.example\r\n",
http.StatusOK, http.StatusOK,
}, },
} { } {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, deny) standIn := startServeWAF(t, deny)
app := startApp(t, greet) app := startApp(t, greet)
// The time limit is long enough for a loaded host to read the first part of // The time limit is long enough for a loaded host to read the first part of
// the body before it runs out. // the body before it runs out.
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "3000", maxRequestTime: "3000",
}) })
+51 -51
View File
@@ -12,11 +12,11 @@ import (
"strings" "strings"
"testing" "testing"
"sneak.berlin/go/smallwebwaf/internal/proxy" "sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
// questionBodyMaxBytes is the most of a body a question carries. // questionBodyMaxBytes is the most of a body a question carries.
const questionBodyMaxBytes = proxy.QuestionBodyMaxBytes const questionBodyMaxBytes = servewaf.QuestionBodyMaxBytes
// chunked is the encoding of a body whose length the visitor did not announce. // chunked is the encoding of a body whose length the visitor did not announce.
const chunked = "chunked" const chunked = "chunked"
@@ -29,9 +29,9 @@ const pathGoReencodes = "/some%2Fpath/with%20space|pipe"
func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) { func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
const target = pathGoReencodes + "?b=2&a=1&bad=%zz;x" const target = pathGoReencodes + "?b=2&a=1&bad=%zz;x"
@@ -42,16 +42,16 @@ func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
req.Header.Set(forwardedForHeader, "198.51.100.1") req.Header.Set(forwardedForHeader, "198.51.100.1")
req.Header.Set(forwardedProtoHeader, "gopher") req.Header.Set(forwardedProtoHeader, "gopher")
req.Header.Set(requestIDHeader, "the-visitor's") req.Header.Set(requestIDHeader, "the-visitor's")
req.Header.Set(proxy.ClientHeader, "198.51.100.2") req.Header.Set(servewaf.ClientHeader, "198.51.100.2")
req.Header.Set(proxy.PeerHeader, "198.51.100.3") req.Header.Set(servewaf.PeerHeader, "198.51.100.3")
req.Header.Set(proxy.ContentLengthHeader, "12345") req.Header.Set(servewaf.ContentLengthHeader, "12345")
// Go's client then sends no User-Agent, so that one in the question would be // Go's client then sends no User-Agent, so that one in the question would be
// Go's own. // Go's own.
req.Header["User-Agent"] = nil req.Header["User-Agent"] = nil
wantGreeting(t, visit(t, req)) wantGreeting(t, visit(t, req))
asked := servewaf.question(t) asked := standIn.question(t)
if asked.method != http.MethodDelete || asked.target != target || if asked.method != http.MethodDelete || asked.target != target ||
asked.host != site { asked.host != site {
t.Errorf("asked about %s %s at %s", asked.method, asked.target, asked.host) t.Errorf("asked about %s %s at %s", asked.method, asked.target, asked.host)
@@ -68,8 +68,8 @@ func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
want.Set(forwardedForHeader, "198.51.100.1") want.Set(forwardedForHeader, "198.51.100.1")
want.Set(forwardedProtoHeader, "http") want.Set(forwardedProtoHeader, "http")
want.Set(requestIDHeader, asked.header.Get(requestIDHeader)) want.Set(requestIDHeader, asked.header.Get(requestIDHeader))
want.Set(proxy.ClientHeader, localhost) want.Set(servewaf.ClientHeader, localhost)
want.Set(proxy.PeerHeader, localhost) want.Set(servewaf.PeerHeader, localhost)
if !maps.EqualFunc(asked.header, want, slices.Equal) { if !maps.EqualFunc(asked.header, want, slices.Equal) {
t.Errorf("the question's headers are %v, want %v", asked.header, want) t.Errorf("the question's headers are %v, want %v", asked.header, want)
@@ -93,14 +93,14 @@ func TestQuestionNamesTheVisitorATrustedProxySpeaksFor(t *testing.T) {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
forwarded := make(chan http.Header, 1) forwarded := make(chan http.Header, 1)
app := startApp(t, func(w http.ResponseWriter, r *http.Request) { app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
forwarded <- r.Header.Clone() forwarded <- r.Header.Clone()
greet(w, r) greet(w, r)
}) })
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
trustedProxies: tc.trusted, trustedProxies: tc.trusted,
}) })
@@ -109,9 +109,9 @@ func TestQuestionNamesTheVisitorATrustedProxySpeaksFor(t *testing.T) {
req.Header.Set(forwardedProtoHeader, "https") req.Header.Set(forwardedProtoHeader, "https")
wantGreeting(t, visit(t, req)) wantGreeting(t, visit(t, req))
asked := servewaf.question(t) asked := standIn.question(t)
if asked.header.Get(proxy.ClientHeader) != tc.client || if asked.header.Get(servewaf.ClientHeader) != tc.client ||
asked.header.Get(proxy.PeerHeader) != localhost || asked.header.Get(servewaf.PeerHeader) != localhost ||
asked.header.Get(forwardedForHeader) != visitor || asked.header.Get(forwardedForHeader) != visitor ||
asked.header.Get(forwardedProtoHeader) != tc.proto { asked.header.Get(forwardedProtoHeader) != tc.proto {
t.Errorf("the question's headers are %v", asked.header) t.Errorf("the question's headers are %v", asked.header)
@@ -157,9 +157,9 @@ func TestQuestionNamesTheClientAsTheForwardedAddressesSay(t *testing.T) {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
trustedProxies: localhost + "/32,10.0.0.0/8", trustedProxies: localhost + "/32,10.0.0.0/8",
}) })
@@ -167,7 +167,7 @@ func TestQuestionNamesTheClientAsTheForwardedAddressesSay(t *testing.T) {
req.Header[forwardedForHeader] = tc.header req.Header[forwardedForHeader] = tc.header
wantGreeting(t, visit(t, req)) wantGreeting(t, visit(t, req))
if got := servewaf.question(t).header.Get(proxy.ClientHeader); got != tc.client { if got := standIn.question(t).header.Get(servewaf.ClientHeader); got != tc.client {
t.Errorf("the question names %s, want %s", got, tc.client) t.Errorf("the question names %s, want %s", got, tc.client)
} }
}) })
@@ -197,7 +197,7 @@ func TestFirst64KiBReachServeWAFAndTheWholeBodyTheApp(t *testing.T) {
t.Parallel() t.Parallel()
body := randomBytes(size) body := randomBytes(size)
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
received := make(chan []byte, 1) received := make(chan []byte, 1)
app := startApp(t, func(w http.ResponseWriter, r *http.Request) { app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
whole, _ := io.ReadAll(r.Body) whole, _ := io.ReadAll(r.Body)
@@ -205,7 +205,7 @@ func TestFirst64KiBReachServeWAFAndTheWholeBodyTheApp(t *testing.T) {
greet(w, r) greet(w, r)
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
req := newRequest(t, http.MethodPost, addr, "/", bytes.NewReader(body)) req := newRequest(t, http.MethodPost, addr, "/", bytes.NewReader(body))
if !announced { if !announced {
@@ -215,7 +215,7 @@ func TestFirst64KiBReachServeWAFAndTheWholeBodyTheApp(t *testing.T) {
wantGreeting(t, visit(t, req)) wantGreeting(t, visit(t, req))
asked := servewaf.question(t) asked := standIn.question(t)
want := body[:min(size, questionBodyMaxBytes)] want := body[:min(size, questionBodyMaxBytes)]
wantQuestionBody(t, asked, want, size, announced) wantQuestionBody(t, asked, want, size, announced)
@@ -223,7 +223,7 @@ func TestFirst64KiBReachServeWAFAndTheWholeBodyTheApp(t *testing.T) {
t.Error("the app did not get the whole body") t.Error("the app did not get the whole body")
} }
if report := servewaf.report(t); report.RequestBytes != int64(size) { if report := standIn.report(t); report.RequestBytes != int64(size) {
t.Errorf("report %+v, of %d bytes", report, size) t.Errorf("report %+v, of %d bytes", report, size)
} }
}) })
@@ -243,7 +243,7 @@ func wantQuestionBody(
len(want)) len(want))
} }
values := asked.header.Values(proxy.ContentLengthHeader) values := asked.header.Values(servewaf.ContentLengthHeader)
if announced { if announced {
if !slices.Equal(values, []string{strconv.Itoa(size)}) || if !slices.Equal(values, []string{strconv.Itoa(size)}) ||
@@ -264,9 +264,9 @@ func wantQuestionBody(
func TestDenyNeedsOnlyTheFirstPartOfALargeBody(t *testing.T) { func TestDenyNeedsOnlyTheFirstPartOfALargeBody(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, deny) standIn := startServeWAF(t, deny)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
// Far more announced than sent. Go's server does not wait for that much, so // Far more announced than sent. Go's server does not wait for that much, so
// the answer comes at once. // the answer comes at once.
@@ -278,7 +278,7 @@ func TestDenyNeedsOnlyTheFirstPartOfALargeBody(t *testing.T) {
t.Errorf("status %d, want servewaf's 403", status) t.Errorf("status %d, want servewaf's 403", status)
} }
if report := servewaf.report(t); report.RequestBytes != questionBodyMaxBytes { if report := standIn.report(t); report.RequestBytes != questionBodyMaxBytes {
t.Errorf("report %+v, want the first part's bytes", report) t.Errorf("report %+v, want the first part's bytes", report)
} }
} }
@@ -306,8 +306,8 @@ func TestAQuestionCarriesItsBodyWhateverTheMethod(t *testing.T) {
t.Run(method+" "+tc.name, func(t *testing.T) { t.Run(method+" "+tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, deny) standIn := startServeWAF(t, deny)
addr, _ := startRProxy(t, servewaf.URL, nowhere, nil) addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, method+" /x HTTP/1.1\r\nHost: site.example\r\n"+tc.framing) send(t, conn, method+" /x HTTP/1.1\r\nHost: site.example\r\n"+tc.framing)
@@ -316,7 +316,7 @@ func TestAQuestionCarriesItsBodyWhateverTheMethod(t *testing.T) {
t.Errorf("status %d, want servewaf's 403", status) t.Errorf("status %d, want servewaf's 403", status)
} }
asked := servewaf.question(t) asked := standIn.question(t)
if string(asked.body) != tc.body || if string(asked.body) != tc.body ||
!slices.Equal(asked.transferEncoding, tc.encoding) { !slices.Equal(asked.transferEncoding, tc.encoding) {
t.Errorf("the question carries %q, encoded %v, want %q, encoded %v", t.Errorf("the question carries %q, encoded %v, want %q, encoded %v",
@@ -334,11 +334,11 @@ func TestAMethodNotPassedOnIsAnswered405(t *testing.T) {
t.Run(method, func(t *testing.T) { t.Run(method, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, out := startRProxy(t, servewaf.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
got := visit(t, newRequest(t, method, addr, "/", http.NoBody)) got := visit(t, newRequest(t, method, addr, "/", http.NoBody))
if got.status != http.StatusMethodNotAllowed || if got.status != http.StatusMethodNotAllowed ||
@@ -352,11 +352,11 @@ func TestAMethodNotPassedOnIsAnswered405(t *testing.T) {
t.Errorf("line %v, want status 405 and no disposition", line) t.Errorf("line %v, want status 405 and no disposition", line)
} }
if len(servewaf.questions) != 0 { if len(standIn.questions) != 0 {
t.Error("servewaf was asked") t.Error("servewaf was asked")
} }
servewaf.wantNoReport(t) standIn.wantNoReport(t)
}) })
} }
} }
@@ -372,18 +372,18 @@ func TestEachMethodPassedOnReachesServeWAFAndTheApp(t *testing.T) {
t.Parallel() t.Parallel()
reached := make(chan string, 1) reached := make(chan string, 1)
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) { app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
reached <- r.Method reached <- r.Method
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
got := visit(t, newRequest(t, method, addr, "/", http.NoBody)) got := visit(t, newRequest(t, method, addr, "/", http.NoBody))
if got.status != http.StatusOK { if got.status != http.StatusOK {
t.Errorf("answered %+v, want the app's 200", got) t.Errorf("answered %+v, want the app's 200", got)
} }
if asked := servewaf.question(t); asked.method != method { if asked := standIn.question(t); asked.method != method {
t.Errorf("servewaf was asked about %s", asked.method) t.Errorf("servewaf was asked about %s", asked.method)
} }
@@ -416,9 +416,9 @@ func TestARequestThatCannotGoToServeWAFAsSentIsRefused(t *testing.T) {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy(t, servewaf.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, request) send(t, conn, request)
@@ -433,11 +433,11 @@ func TestARequestThatCannotGoToServeWAFAsSentIsRefused(t *testing.T) {
t.Errorf("line %v, want status 400 and no disposition", line) t.Errorf("line %v, want status 400 and no disposition", line)
} }
if len(servewaf.questions) != 0 { if len(standIn.questions) != 0 {
t.Error("servewaf was asked") t.Error("servewaf was asked")
} }
servewaf.wantNoReport(t) standIn.wantNoReport(t)
}) })
} }
} }
@@ -447,9 +447,9 @@ func TestAHostWithAPercentSignButNoBracketGoesToServeWAF(t *testing.T) {
const host = "a%b" const host = "a%b"
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, "GET / HTTP/1.1\r\nHost: "+host+"\r\n\r\n") send(t, conn, "GET / HTTP/1.1\r\nHost: "+host+"\r\n\r\n")
@@ -458,7 +458,7 @@ func TestAHostWithAPercentSignButNoBracketGoesToServeWAF(t *testing.T) {
t.Errorf("status %d, want the app's 200", status) t.Errorf("status %d, want the app's 200", status)
} }
if asked := servewaf.question(t); asked.host != host { if asked := standIn.question(t); asked.host != host {
t.Errorf("asked about the host %q, want %q", asked.host, host) t.Errorf("asked about the host %q, want %q", asked.host, host)
} }
} }
@@ -468,15 +468,15 @@ func TestATrustedPeerWithNoForwardedProtoIsAskedAboutAsHTTP(t *testing.T) {
const want = "http" const want = "http"
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
trustedProxies: localhost + "/32", trustedProxies: localhost + "/32",
}) })
wantGreeting(t, get(t, addr, "/")) wantGreeting(t, get(t, addr, "/"))
asked := servewaf.question(t) asked := standIn.question(t)
if got := asked.header.Get(forwardedProtoHeader); got != want { if got := asked.header.Get(forwardedProtoHeader); got != want {
t.Errorf("the question's %s is %q, want %s", forwardedProtoHeader, got, want) t.Errorf("the question's %s is %q, want %s", forwardedProtoHeader, got, want)
} }
@@ -497,9 +497,9 @@ func TestAQuestionNamesNoZone(t *testing.T) {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, deny) standIn := startServeWAF(t, deny)
app := startApp(t, greet) app := startApp(t, greet)
_, out, server := startServer(t, servewaf.URL, app.URL, map[string]string{ _, out, server := startServer(t, standIn.URL, app.URL, map[string]string{
trustedProxies: localhost + "/32", trustedProxies: localhost + "/32",
}) })
@@ -509,9 +509,9 @@ func TestAQuestionNamesNoZone(t *testing.T) {
server.Handler.ServeHTTP(httptest.NewRecorder(), req) server.Handler.ServeHTTP(httptest.NewRecorder(), req)
asked := servewaf.question(t) asked := standIn.question(t)
if asked.header.Get(proxy.ClientHeader) != tc.client || if asked.header.Get(servewaf.ClientHeader) != tc.client ||
asked.header.Get(proxy.PeerHeader) != tc.peer { asked.header.Get(servewaf.PeerHeader) != tc.peer {
t.Errorf("the question's headers are %v", asked.header) t.Errorf("the question's headers are %v", asked.header)
} }
+3 -3
View File
@@ -16,9 +16,9 @@ import (
"sync/atomic" "sync/atomic"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules" "sneak.berlin/go/smallwebwaf/internal/rules"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
// flushAfterEachWrite has ReverseProxy pass on each part of the app's answer as // flushAfterEachWrite has ReverseProxy pass on each part of the app's answer as
@@ -129,7 +129,7 @@ func (rq *request) readFirstPartOfBody() bool {
} }
_ = rq.rc.SetReadDeadline(rq.requestDeadline()) _ = rq.rc.SetReadDeadline(rq.requestDeadline())
held, err := io.ReadAll(io.LimitReader(rq.body, proxy.QuestionBodyMaxBytes)) held, err := io.ReadAll(io.LimitReader(rq.body, servewaf.QuestionBodyMaxBytes))
// From here on, the request timer ends a read of the rest of the body, so that // From here on, the request timer ends a read of the rest of the body, so that
// the answer can name the side rproxy was waiting on. // the answer can name the side rproxy was waiting on.
_ = rq.rc.SetReadDeadline(time.Time{}) _ = rq.rc.SetReadDeadline(time.Time{})
@@ -384,7 +384,7 @@ func (rq *request) finish(ctx context.Context) {
return return
} }
report := &proxy.Report{ report := &servewaf.Report{
RequestID: rq.id, RequestID: rq.id,
Status: min(rq.out.status, reportStatusMax), Status: min(rq.out.status, reportStatusMax),
RequestBytes: requestBytes, RequestBytes: requestBytes,
+5 -5
View File
@@ -22,7 +22,7 @@ import (
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/proxy" "sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
// How rproxy keeps connections to the app and to servewaf open between // How rproxy keeps connections to the app and to servewaf open between
@@ -85,7 +85,7 @@ func New(params Params) *Server {
transport: newTransport(), transport: newTransport(),
serveWAFTransport: newServeWAFTransport(), serveWAFTransport: newServeWAFTransport(),
reportURL: (&url.URL{ reportURL: (&url.URL{
Scheme: backend.Scheme, Host: backend.Host, Path: proxy.ReportPath, Scheme: backend.Scheme, Host: backend.Host, Path: servewaf.ReportPath,
}).String(), }).String(),
} }
@@ -166,7 +166,7 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
rq := h.newRequest(w, r) rq := h.newRequest(w, r)
defer rq.finish(r.Context()) defer rq.finish(r.Context())
if r.Method == http.MethodGet && r.URL.Path == proxy.HealthPath { if r.Method == http.MethodGet && r.URL.Path == servewaf.HealthPath {
rq.answerHealth() rq.answerHealth()
return return
@@ -199,9 +199,9 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
rq.noteAnswer(answer) rq.noteAnswer(answer)
switch rq.disposition { switch rq.disposition {
case proxy.DispositionAllow: case servewaf.DispositionAllow:
rq.passToApp(r.Context()) rq.passToApp(r.Context())
case proxy.DispositionClose: case servewaf.DispositionClose:
// This panic has Go's server close the connection without an answer, // This panic has Go's server close the connection without an answer,
// and log nothing; the deferred finish still runs. // and log nothing; the deferred finish still runs.
panic(http.ErrAbortHandler) panic(http.ErrAbortHandler)
+80 -80
View File
@@ -19,9 +19,9 @@ import (
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rproxy" "sneak.berlin/go/smallwebwaf/internal/rproxy"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
const ( const (
@@ -54,7 +54,7 @@ const (
// The headers the tests set and look at. // The headers the tests set and look at.
const ( const (
dispositionHeader = proxy.DispositionHeader dispositionHeader = servewaf.DispositionHeader
forwardedForHeader = "X-Forwarded-For" forwardedForHeader = "X-Forwarded-For"
forwardedProtoHeader = "X-Forwarded-Proto" forwardedProtoHeader = "X-Forwarded-Proto"
requestIDHeader = "X-Request-ID" requestIDHeader = "X-Request-ID"
@@ -70,9 +70,9 @@ const (
// What the stand-in for servewaf answers. // What the stand-in for servewaf answers.
const ( const (
dispositionAllow = proxy.DispositionAllow dispositionAllow = servewaf.DispositionAllow
dispositionDeny = proxy.DispositionDeny dispositionDeny = servewaf.DispositionDeny
dispositionClose = proxy.DispositionClose dispositionClose = servewaf.DispositionClose
// deniedBody is the body of a deny. // deniedBody is the body of a deny.
deniedBody = "refused by servewaf" deniedBody = "refused by servewaf"
) )
@@ -173,7 +173,7 @@ type serveWAF struct {
*httptest.Server *httptest.Server
questions chan question questions chan question
reports chan proxy.Report reports chan servewaf.Report
} }
// startServeWAF starts a stand-in for servewaf. // startServeWAF starts a stand-in for servewaf.
@@ -182,11 +182,11 @@ func startServeWAF(t *testing.T, answer http.HandlerFunc) *serveWAF {
s := &serveWAF{ s := &serveWAF{
questions: make(chan question, 10), questions: make(chan question, 10),
reports: make(chan proxy.Report, 10), reports: make(chan servewaf.Report, 10),
} }
s.Server = httptest.NewServer(http.HandlerFunc( s.Server = httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) { func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get(proxy.ClientHeader) == "" { if r.Header.Get(servewaf.ClientHeader) == "" {
s.takeReport(t, w, r) s.takeReport(t, w, r)
return return
@@ -213,10 +213,10 @@ func startServeWAF(t *testing.T, answer http.HandlerFunc) *serveWAF {
func (s *serveWAF) takeReport(t *testing.T, w http.ResponseWriter, r *http.Request) { func (s *serveWAF) takeReport(t *testing.T, w http.ResponseWriter, r *http.Request) {
t.Helper() t.Helper()
var report proxy.Report var report servewaf.Report
err := json.NewDecoder(r.Body).Decode(&report) err := json.NewDecoder(r.Body).Decode(&report)
if err != nil || r.Method != http.MethodPost || r.URL.Path != proxy.ReportPath { if err != nil || r.Method != http.MethodPost || r.URL.Path != servewaf.ReportPath {
t.Errorf("%s %s is no report: %v", r.Method, r.URL.Path, err) t.Errorf("%s %s is no report: %v", r.Method, r.URL.Path, err)
} }
@@ -237,7 +237,7 @@ func (s *serveWAF) wantNoReport(t *testing.T) {
} }
// report waits for a report. // report waits for a report.
func (s *serveWAF) report(t *testing.T) proxy.Report { func (s *serveWAF) report(t *testing.T) servewaf.Report {
t.Helper() t.Helper()
select { select {
@@ -246,7 +246,7 @@ func (s *serveWAF) report(t *testing.T) proxy.Report {
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Fatal("no report") t.Fatal("no report")
return proxy.Report{} return servewaf.Report{}
} }
} }
@@ -273,7 +273,7 @@ func startServeWAFAnsweringReports(
backend := httptest.NewServer(http.HandlerFunc( backend := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) { func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get(proxy.ClientHeader) != "" { if r.Header.Get(servewaf.ClientHeader) != "" {
answer(w, r) answer(w, r)
return return
@@ -498,7 +498,7 @@ func readStatus(t *testing.T, conn net.Conn) int {
func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) { func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, func(w http.ResponseWriter, r *http.Request) { standIn := startServeWAF(t, func(w http.ResponseWriter, r *http.Request) {
w.Header().Set(asnHeader, "AS64496") w.Header().Set(asnHeader, "AS64496")
w.Header().Set(countryHeader, "DE") w.Header().Set(countryHeader, "DE")
allow(w, r) allow(w, r)
@@ -512,7 +512,7 @@ func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
w.WriteHeader(http.StatusTeapot) w.WriteHeader(http.StatusTeapot)
_, _ = io.WriteString(w, greeting) _, _ = io.WriteString(w, greeting)
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
const target = pathGoReencodes + "?b=2&a=1&bad=%zz;x" const target = pathGoReencodes + "?b=2&a=1&bad=%zz;x"
@@ -530,7 +530,7 @@ func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
} }
received := <-seen received := <-seen
asked := servewaf.question(t) asked := standIn.question(t)
if received.RequestURI != target || received.Host != site { if received.RequestURI != target || received.Host != site {
t.Errorf("the app saw %s for %s", received.RequestURI, received.Host) t.Errorf("the app saw %s for %s", received.RequestURI, received.Host)
@@ -557,14 +557,14 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
"X-Forwarded-Tls-Client-Cert-Info", "X-Real-Ip", "X-Forwarded-Tls-Client-Cert-Info", "X-Real-Ip",
} }
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
seen := make(chan http.Header, 1) seen := make(chan http.Header, 1)
app := startApp(t, func(w http.ResponseWriter, r *http.Request) { app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
seen <- r.Header.Clone() seen <- r.Header.Clone()
greet(w, r) greet(w, r)
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
trustedProxies: "10.0.0.0/8", trustedProxies: "10.0.0.0/8",
}) })
@@ -602,14 +602,14 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
func TestATrustedProxysForwardedHeadersPassOn(t *testing.T) { func TestATrustedProxysForwardedHeadersPassOn(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
seen := make(chan http.Header, 1) seen := make(chan http.Header, 1)
app := startApp(t, func(w http.ResponseWriter, r *http.Request) { app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
seen <- r.Header.Clone() seen <- r.Header.Clone()
greet(w, r) greet(w, r)
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
trustedProxies: localhost + "/32", trustedProxies: localhost + "/32",
}) })
@@ -641,12 +641,12 @@ func TestATrustedProxysForwardedHeadersPassOn(t *testing.T) {
func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) { func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) { app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header()["Content-Type"] = nil w.Header()["Content-Type"] = nil
_, _ = io.WriteString(w, "<html>") _, _ = io.WriteString(w, "<html>")
}) })
_, _, server := startServer(t, servewaf.URL, app.URL, nil) _, _, server := startServer(t, standIn.URL, app.URL, nil)
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody) req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
w := httptest.NewRecorder() w := httptest.NewRecorder()
@@ -704,11 +704,11 @@ func TestRemovesTheVisitorsLookupHeadersWhenServeWAFGivesNone(t *testing.T) {
func TestDenyGivesTheVisitorServeWAFsAnswer(t *testing.T) { func TestDenyGivesTheVisitorServeWAFsAnswer(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, deny) standIn := startServeWAF(t, deny)
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, out := startRProxy(t, servewaf.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
got := get(t, addr, "/") got := get(t, addr, "/")
if got.status != http.StatusForbidden || got.body != deniedBody || if got.status != http.StatusForbidden || got.body != deniedBody ||
@@ -720,7 +720,7 @@ func TestDenyGivesTheVisitorServeWAFsAnswer(t *testing.T) {
t.Errorf("the visitor got %s", dispositionHeader) t.Errorf("the visitor got %s", dispositionHeader)
} }
report := servewaf.report(t) report := standIn.report(t)
if report.Status != http.StatusForbidden || if report.Status != http.StatusForbidden ||
report.ResponseBytes != int64(len(deniedBody)) || report.ResponseBytes != int64(len(deniedBody)) ||
report.DurationUpstreamTotal != 0 { report.DurationUpstreamTotal != 0 {
@@ -736,9 +736,9 @@ func TestDenyGivesTheVisitorServeWAFsAnswer(t *testing.T) {
func TestADeniedHeadKeepsServeWAFsContentLength(t *testing.T) { func TestADeniedHeadKeepsServeWAFsContentLength(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, deny) standIn := startServeWAF(t, deny)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody)) got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
if got.status != http.StatusForbidden || if got.status != http.StatusForbidden ||
@@ -758,15 +758,15 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
}{ }{
{"a deny", "GET /" + host, deniedBody, http.StatusForbidden}, {"a deny", "GET /" + host, deniedBody, http.StatusForbidden},
{"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest}, {"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest},
{"the health check", "GET " + proxy.HealthPath + host, "ok\n", http.StatusOK}, {"the health check", "GET " + servewaf.HealthPath + host, "ok\n", http.StatusOK},
} { } {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
log := heldLog{release: make(chan struct{})} log := heldLog{release: make(chan struct{})}
servewaf := startServeWAF(t, deny) standIn := startServeWAF(t, deny)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startServerLogging(t, servewaf.URL, app.URL, nil, log, io.Discard) addr, _ := startServerLogging(t, standIn.URL, app.URL, nil, log, io.Discard)
t.Cleanup(func() { close(log.release) }) t.Cleanup(func() { close(log.release) })
@@ -794,9 +794,9 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
func TestRProxysOwnAnswerIsPlainText(t *testing.T) { func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, "GET x:admin HTTP/1.1\r\nHost: site.example\r\n\r\n") send(t, conn, "GET x:admin HTTP/1.1\r\nHost: site.example\r\n\r\n")
@@ -817,14 +817,14 @@ func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) { func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) { standIn := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(dispositionHeader, dispositionClose) w.Header().Set(dispositionHeader, dispositionClose)
w.WriteHeader(http.StatusOK) w.WriteHeader(http.StatusOK)
}) })
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, out := startRProxy(t, servewaf.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n") send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
@@ -835,7 +835,7 @@ func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
text, err) text, err)
} }
report := servewaf.report(t) report := standIn.report(t)
if report.Status != 0 || report.ResponseBytes != 0 { if report.Status != 0 || report.ResponseBytes != 0 {
t.Errorf("report %+v, of a visitor sent nothing", report) t.Errorf("report %+v, of a visitor sent nothing", report)
} }
@@ -935,7 +935,7 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsServeWAFGaveIt(t *testing.T) {
t.Run(tc.name+" with the default "+setting, func(t *testing.T) { t.Run(tc.name+" with the default "+setting, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) { standIn := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
if tc.disposition != "" { if tc.disposition != "" {
w.Header().Set(dispositionHeader, tc.disposition) w.Header().Set(dispositionHeader, tc.disposition)
} }
@@ -947,7 +947,7 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsServeWAFGaveIt(t *testing.T) {
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
defaultDisposition: setting, defaultDisposition: setting,
}) })
@@ -977,11 +977,11 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsServeWAFGaveIt(t *testing.T) {
func TestAnAnswerWithoutAValidDispositionGetsNoReport(t *testing.T) { func TestAnAnswerWithoutAValidDispositionGetsNoReport(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) { standIn := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
http.Error(w, "refused", http.StatusBadRequest) http.Error(w, "refused", http.StatusBadRequest)
}) })
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy(t, servewaf.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
if got := get(t, addr, "/"); got.status != http.StatusBadRequest { if got := get(t, addr, "/"); got.status != http.StatusBadRequest {
t.Errorf("answered %+v, want servewaf's 400", got) t.Errorf("answered %+v, want servewaf's 400", got)
@@ -991,18 +991,18 @@ func TestAnAnswerWithoutAValidDispositionGetsNoReport(t *testing.T) {
t.Errorf("line %v, want no disposition", line) t.Errorf("line %v, want no disposition", line)
} }
servewaf.wantNoReport(t) standIn.wantNoReport(t)
} }
func TestAnAnswerWithoutAContentTypeGoesToTheVisitorWithNone(t *testing.T) { func TestAnAnswerWithoutAContentTypeGoesToTheVisitorWithNone(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) { standIn := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header()["Content-Type"] = nil w.Header()["Content-Type"] = nil
w.WriteHeader(http.StatusInternalServerError) w.WriteHeader(http.StatusInternalServerError)
_, _ = io.WriteString(w, "<html>broken") _, _ = io.WriteString(w, "<html>broken")
}) })
addr, _ := startRProxy(t, servewaf.URL, nowhere, nil) addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
got := get(t, addr, "/") got := get(t, addr, "/")
if contentType, sent := got.header["Content-Type"]; sent { if contentType, sent := got.header["Content-Type"]; sent {
@@ -1015,11 +1015,11 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
const size = 1 << 20 const size = 1 << 20
servewaf := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) { standIn := startServeWAF(t, func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusInternalServerError) w.WriteHeader(http.StatusInternalServerError)
_, _ = w.Write(make([]byte, size)) _, _ = w.Write(make([]byte, size))
}) })
addr, _ := startRProxy(t, servewaf.URL, nowhere, nil) addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
got := get(t, addr, "/") got := get(t, addr, "/")
if got.status != http.StatusInternalServerError || len(got.body) != size { if got.status != http.StatusInternalServerError || len(got.body) != size {
@@ -1135,15 +1135,15 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
func TestAnAnswerThatCannotBePassedOnGetsNoReport(t *testing.T) { func TestAnAnswerThatCannotBePassedOnGetsNoReport(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, overOneMiB(dispositionDeny)) standIn := startServeWAF(t, overOneMiB(dispositionDeny))
addr, out := startRProxy(t, servewaf.URL, nowhere, nil) addr, out := startRProxy(t, standIn.URL, nowhere, nil)
if got := get(t, addr, "/"); got.status != http.StatusBadGateway { if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
t.Errorf("answered %+v, want 502", got) t.Errorf("answered %+v, want 502", got)
} }
out.requestLine(t) out.requestLine(t)
servewaf.wantNoReport(t) standIn.wantNoReport(t)
} }
func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) { func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
@@ -1199,20 +1199,20 @@ func TestAVisitorThatGoesAwayWhileServeWAFIsAskedGetsNoDefault(t *testing.T) {
t.Run("with the default "+disposition, func(t *testing.T) { t.Run("with the default "+disposition, func(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, func(_ http.ResponseWriter, r *http.Request) { standIn := startServeWAF(t, func(_ http.ResponseWriter, r *http.Request) {
<-r.Context().Done() <-r.Context().Done()
}) })
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
timeoutSetting: "60000", timeoutSetting: "60000",
defaultDisposition: disposition, defaultDisposition: disposition,
}) })
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n") send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
servewaf.question(t) standIn.question(t)
_ = conn.Close() _ = conn.Close()
@@ -1226,7 +1226,7 @@ func TestAVisitorThatGoesAwayWhileServeWAFIsAskedGetsNoDefault(t *testing.T) {
t.Error("rproxy warned that servewaf gave no answer") t.Error("rproxy warned that servewaf gave no answer")
} }
servewaf.wantNoReport(t) standIn.wantNoReport(t)
}) })
} }
} }
@@ -1244,17 +1244,17 @@ func TestRProxyWarnsWhenTheDefaultDecides(t *testing.T) {
func TestSendsNoReportWithoutAnAnswer(t *testing.T) { func TestSendsNoReportWithoutAnAnswer(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, func(_ http.ResponseWriter, r *http.Request) { standIn := startServeWAF(t, func(_ http.ResponseWriter, r *http.Request) {
<-r.Context().Done() <-r.Context().Done()
}) })
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
timeoutSetting: "200", timeoutSetting: "200",
}) })
wantGreeting(t, get(t, addr, "/")) wantGreeting(t, get(t, addr, "/"))
out.requestLine(t) out.requestLine(t)
servewaf.wantNoReport(t) standIn.wantNoReport(t)
} }
func TestAPathUnderOwnPrefixNeverReachesTheApp(t *testing.T) { func TestAPathUnderOwnPrefixNeverReachesTheApp(t *testing.T) {
@@ -1274,25 +1274,25 @@ func TestAPathUnderOwnPrefixNeverReachesTheApp(t *testing.T) {
func TestHealthCheckIsAnsweredWhileServeWAFIsDown(t *testing.T) { func TestHealthCheckIsAnsweredWhileServeWAFIsDown(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy(t, nowhere, app.URL, nil) addr, out := startRProxy(t, nowhere, app.URL, nil)
got := get(t, addr, proxy.HealthPath) got := get(t, addr, servewaf.HealthPath)
if got.status != http.StatusOK || got.body != "ok\n" { if got.status != http.StatusOK || got.body != "ok\n" {
t.Errorf("answered %+v, want ok", got) t.Errorf("answered %+v, want ok", got)
} }
line := out.requestLine(t) line := out.requestLine(t)
if line["path"] != proxy.HealthPath || line["disposition"] != nil { if line["path"] != servewaf.HealthPath || line["disposition"] != nil {
t.Errorf("line %v, want one with no disposition", line) t.Errorf("line %v, want one with no disposition", line)
} }
// With servewaf up too, it is not asked. // With servewaf up too, it is not asked.
addr, _ = startRProxy(t, servewaf.URL, app.URL, nil) addr, _ = startRProxy(t, standIn.URL, app.URL, nil)
get(t, addr, proxy.HealthPath) get(t, addr, servewaf.HealthPath)
if len(servewaf.questions) != 0 { if len(standIn.questions) != 0 {
t.Error("servewaf was asked about the health check") t.Error("servewaf was asked about the health check")
} }
} }
@@ -1396,15 +1396,15 @@ func TestAReportAnswered101IsDroppedAtOnce(t *testing.T) {
func TestAReportDueAfterWaitForReportsIsDropped(t *testing.T) { func TestAReportDueAfterWaitForReportsIsDropped(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, out, server := startServer(t, servewaf.URL, app.URL, nil) addr, out, server := startServer(t, standIn.URL, app.URL, nil)
server.WaitForReports() server.WaitForReports()
wantGreeting(t, get(t, addr, "/")) wantGreeting(t, get(t, addr, "/"))
out.requestLine(t) out.requestLine(t)
servewaf.wantNoReport(t) standIn.wantNoReport(t)
} }
func TestReportGivesWhatHappened(t *testing.T) { func TestReportGivesWhatHappened(t *testing.T) {
@@ -1412,14 +1412,14 @@ func TestReportGivesWhatHappened(t *testing.T) {
const body = "the visitor's body" const body = "the visitor's body"
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy(t, servewaf.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/", wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/",
strings.NewReader(body)))) strings.NewReader(body))))
report := servewaf.report(t) report := standIn.report(t)
if report.RequestID == "" || report.Status != http.StatusOK || if report.RequestID == "" || report.Status != http.StatusOK ||
report.RequestBytes != int64(len(body)) || report.RequestBytes != int64(len(body)) ||
@@ -1442,7 +1442,7 @@ func TestWaitForReportsWaitsForAReportBeingSent(t *testing.T) {
release := make(chan struct{}) release := make(chan struct{})
backend := httptest.NewServer(http.HandlerFunc( backend := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) { func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get(proxy.ClientHeader) != "" { if r.Header.Get(servewaf.ClientHeader) != "" {
allow(w, r) allow(w, r)
return return
@@ -1497,9 +1497,9 @@ func TestWaitForReportsWaitsForAReportBeingSent(t *testing.T) {
func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) { func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
t.Parallel() t.Parallel()
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, echoAfterUpgrade) app := startApp(t, echoAfterUpgrade)
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "2000", maxRequestTime: "2000",
maxResponseTime: "2000", maxResponseTime: "2000",
}) })
@@ -1535,7 +1535,7 @@ func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
_ = conn.Close() _ = conn.Close()
report := servewaf.report(t) report := standIn.report(t)
if report.Status != http.StatusSwitchingProtocols || if report.Status != http.StatusSwitchingProtocols ||
report.RequestBytes != int64(len(message)) || report.RequestBytes != int64(len(message)) ||
report.ResponseBytes != int64(len(message)) { report.ResponseBytes != int64(len(message)) {
@@ -1552,7 +1552,7 @@ func TestAWebSocketPassesOnThatTheVisitorStoppedSending(t *testing.T) {
const goodbye = "sent after the visitor stopped sending\n" const goodbye = "sent after the visitor stopped sending\n"
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) { app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
conn, buffered, err := http.NewResponseController(w).Hijack() conn, buffered, err := http.NewResponseController(w).Hijack()
if err != nil { if err != nil {
@@ -1572,7 +1572,7 @@ func TestAWebSocketPassesOnThatTheVisitorStoppedSending(t *testing.T) {
_, _ = buffered.WriteString(goodbye) _, _ = buffered.WriteString(goodbye)
_ = buffered.Flush() _ = buffered.Flush()
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+ send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+
@@ -1610,7 +1610,7 @@ func TestAWebSocketPassesOnThatTheVisitorStoppedSending(t *testing.T) {
//nolint:paralleltest // see above //nolint:paralleltest // see above
func TestASwitchNotPassedOnClosesTheAppsConnection(t *testing.T) { func TestASwitchNotPassedOnClosesTheAppsConnection(t *testing.T) {
closed := make(chan error, 1) closed := make(chan error, 1)
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) { app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
conn, buffered, err := http.NewResponseController(w).Hijack() conn, buffered, err := http.NewResponseController(w).Hijack()
if err != nil { if err != nil {
@@ -1633,7 +1633,7 @@ func TestASwitchNotPassedOnClosesTheAppsConnection(t *testing.T) {
_, err = buffered.ReadByte() _, err = buffered.ReadByte()
closed <- err closed <- err
}) })
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr) conn := dial(t, addr)
send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+ send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+
@@ -1681,9 +1681,9 @@ func TestEachRequestGetsAFreshID(t *testing.T) {
const fromTraefik = "an-id-from-traefik" const fromTraefik = "an-id-from-traefik"
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
trustedProxies: localhost + "/32", trustedProxies: localhost + "/32",
}) })
@@ -1694,7 +1694,7 @@ func TestEachRequestGetsAFreshID(t *testing.T) {
req.Header.Set(requestIDHeader, fromTraefik) req.Header.Set(requestIDHeader, fromTraefik)
wantGreeting(t, visit(t, req)) wantGreeting(t, visit(t, req))
ids = append(ids, servewaf.question(t).header.Get(requestIDHeader)) ids = append(ids, standIn.question(t).header.Get(requestIDHeader))
} }
if slices.Contains(ids, fromTraefik) || ids[0] == "" || ids[0] == ids[1] { if slices.Contains(ids, fromTraefik) || ids[0] == "" || ids[0] == ids[1] {
@@ -1711,16 +1711,16 @@ func TestLogLineGivesTheRequestAsSent(t *testing.T) {
body = "the visitor's body" body = "the visitor's body"
) )
servewaf := startServeWAF(t, allow) standIn := startServeWAF(t, allow)
app := startApp(t, greet) app := startApp(t, greet)
addr, out := startRProxy(t, servewaf.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
req := newRequest(t, http.MethodPost, addr, path+"?"+query, strings.NewReader(body)) req := newRequest(t, http.MethodPost, addr, path+"?"+query, strings.NewReader(body))
req.Host = site req.Host = site
wantGreeting(t, visit(t, req)) wantGreeting(t, visit(t, req))
line := out.requestLine(t) line := out.requestLine(t)
asked := servewaf.question(t) asked := standIn.question(t)
for name, want := range map[string]any{ for name, want := range map[string]any{
"type": "request", "type": "request",
+11 -11
View File
@@ -11,7 +11,7 @@ import (
"net/http" "net/http"
"strings" "strings"
"sneak.berlin/go/smallwebwaf/internal/proxy" "sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
// answerBodyMaxBytes is the most of the body of servewaf's answer that is read. // answerBodyMaxBytes is the most of the body of servewaf's answer that is read.
@@ -38,10 +38,10 @@ type servewafAnswer struct {
// notes nothing, and no report is due for it: servewaf holds no decision to report // notes nothing, and no report is due for it: servewaf holds no decision to report
// on. // on.
func (rq *request) noteAnswer(answer *servewafAnswer) { func (rq *request) noteAnswer(answer *servewafAnswer) {
disposition := answer.header.Get(proxy.DispositionHeader) disposition := answer.header.Get(servewaf.DispositionHeader)
switch disposition { switch disposition {
case proxy.DispositionAllow, proxy.DispositionDeny, proxy.DispositionClose: case servewaf.DispositionAllow, servewaf.DispositionDeny, servewaf.DispositionClose:
rq.disposition = disposition rq.disposition = disposition
rq.answered = true rq.answered = true
rq.asn = answer.header.Get(asnHeader) rq.asn = answer.header.Get(asnHeader)
@@ -74,7 +74,7 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
rq.disposition = rq.h.config.DefaultDisposition rq.disposition = rq.h.config.DefaultDisposition
rq.byDefault = true rq.byDefault = true
if rq.disposition == proxy.DispositionAllow { if rq.disposition == servewaf.DispositionAllow {
rq.passToApp(ctx) rq.passToApp(ctx)
} else { } else {
rq.answer(http.StatusServiceUnavailable) rq.answer(http.StatusServiceUnavailable)
@@ -169,13 +169,13 @@ func (rq *request) newQuestion(ctx context.Context) (*http.Request, error) {
question.TransferEncoding = in.TransferEncoding question.TransferEncoding = in.TransferEncoding
question.Header = in.Header.Clone() question.Header = in.Header.Clone()
question.Header.Set(proxy.ClientHeader, rq.client.String()) question.Header.Set(servewaf.ClientHeader, rq.client.String())
question.Header.Set(proxy.PeerHeader, rq.peer.String()) question.Header.Set(servewaf.PeerHeader, rq.peer.String())
question.Header.Del(proxy.ContentLengthHeader) question.Header.Del(servewaf.ContentLengthHeader)
announced := in.Header.Get("Content-Length") announced := in.Header.Get("Content-Length")
if announced != "" { if announced != "" {
question.Header.Set(proxy.ContentLengthHeader, announced) question.Header.Set(servewaf.ContentLengthHeader, announced)
} }
question.Header.Set(requestIDHeader, rq.id) question.Header.Set(requestIDHeader, rq.id)
@@ -196,7 +196,7 @@ func (rq *request) answerWith(answer *servewafAnswer) {
header := rq.out.Header() header := rq.out.Header()
maps.Copy(header, answer.header) maps.Copy(header, answer.header)
header.Del(proxy.DispositionHeader) header.Del(servewaf.DispositionHeader)
// Go's server adds a Content-Type it guesses from the body to a response whose // Go's server adds a Content-Type it guesses from the body to a response whose
// headers have no Content-Type key. The key with no value sends none. // headers have no Content-Type key. The key with no value sends none.
@@ -210,7 +210,7 @@ func (rq *request) answerWith(answer *servewafAnswer) {
// sendReport sends servewaf report on a goroutine of its own, so that the // sendReport sends servewaf report on a goroutine of its own, so that the
// visitor does not wait for it. The report goes on after ctx, the request's own // visitor does not wait for it. The report goes on after ctx, the request's own
// context, has ended. // context, has ended.
func (h *handler) sendReport(ctx context.Context, report *proxy.Report) { func (h *handler) sendReport(ctx context.Context, report *servewaf.Report) {
h.reportsMu.Lock() h.reportsMu.Lock()
defer h.reportsMu.Unlock() defer h.reportsMu.Unlock()
@@ -228,7 +228,7 @@ func (h *handler) sendReport(ctx context.Context, report *proxy.Report) {
// postReport posts report to servewaf, and waits for its answer for up to // postReport posts report to servewaf, and waits for its answer for up to
// SWRP_TIMEOUT_MS. // SWRP_TIMEOUT_MS.
func (h *handler) postReport(ctx context.Context, report *proxy.Report) error { func (h *handler) postReport(ctx context.Context, report *servewaf.Report) error {
body, err := json.Marshal(report) body, err := json.Marshal(report)
if err != nil { if err != nil {
return fmt.Errorf("encode the report: %w", err) return fmt.Errorf("encode the report: %w", err)
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"bytes" "bytes"
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"encoding/json" "encoding/json"
@@ -10,9 +10,9 @@ import (
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
"sneak.berlin/go/smallwebwaf/internal/state" "sneak.berlin/go/smallwebwaf/internal/state"
) )
@@ -88,11 +88,11 @@ func TestAdminEndpointsNeedTheAdminToken(t *testing.T) {
// Any other request under /_smallwebwaf/ is not found. // Any other request under /_smallwebwaf/ is not found.
for _, e := range []adminEndpoint{ for _, e := range []adminEndpoint{
{http.MethodPut, proxy.BansPath, banOtherClient}, {http.MethodPut, servewaf.BansPath, banOtherClient},
{http.MethodDelete, proxy.BansPath, ""}, {http.MethodDelete, servewaf.BansPath, ""},
{http.MethodGet, proxy.BansPath + "/" + otherClient, ""}, {http.MethodGet, servewaf.BansPath + "/" + otherClient, ""},
{http.MethodPost, proxy.ClientsPath + otherClient, ""}, {http.MethodPost, servewaf.ClientsPath + otherClient, ""},
{http.MethodGet, strings.TrimSuffix(proxy.ClientsPath, "/"), ""}, {http.MethodGet, strings.TrimSuffix(servewaf.ClientsPath, "/"), ""},
} { } {
s.admin(e.method, e.path, e.body, http.StatusNotFound) s.admin(e.method, e.path, e.body, http.StatusNotFound)
} }
@@ -118,7 +118,7 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
Reason: "probes for logins", Reason: "probes for logins",
} }
wantBans(t, s.admin(http.MethodPost, proxy.BansPath, banOtherClient, http.StatusOK), wantBans(t, s.admin(http.MethodPost, servewaf.BansPath, banOtherClient, http.StatusOK),
want) want)
line := s.get(client, http.StatusForbidden, requestlog.ActionBanned) line := s.get(client, http.StatusForbidden, requestlog.ActionBanned)
@@ -129,7 +129,7 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
// Its notes count the request it refused. // Its notes count the request it refused.
want.Notes.Requests, want.Notes.Refused = 1, 1 want.Notes.Requests, want.Notes.Refused = 1, 1
wantBans(t, s.admin(http.MethodGet, proxy.BansPath, "", http.StatusOK), want) wantBans(t, s.admin(http.MethodGet, servewaf.BansPath, "", http.StatusOK), want)
// Ten minutes on, lifting the bans on client lifts that one, which is // Ten minutes on, lifting the bans on client lifts that one, which is
// kept, marked lifted. // kept, marked lifted.
@@ -138,13 +138,13 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
lifted := clk.Now() lifted := clk.Now()
want.Lifted = &lifted want.Lifted = &lifted
wantBans(t, s.admin(http.MethodDelete, proxy.BansPath+"/"+client, "", wantBans(t, s.admin(http.MethodDelete, servewaf.BansPath+"/"+client, "",
http.StatusOK), want) http.StatusOK), want)
s.get(client, http.StatusOK, requestlog.ActionForward) s.get(client, http.StatusOK, requestlog.ActionForward)
wantBans(t, s.admin(http.MethodGet, proxy.BansPath, "", http.StatusOK), want) wantBans(t, s.admin(http.MethodGet, servewaf.BansPath, "", http.StatusOK), want)
// No ban on it is active any more. // No ban on it is active any more.
s.admin(http.MethodDelete, proxy.BansPath+"/"+client, "", http.StatusNotFound) s.admin(http.MethodDelete, servewaf.BansPath+"/"+client, "", http.StatusNotFound)
} }
func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) { func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
@@ -181,7 +181,7 @@ func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
want.Expires = &expires want.Expires = &expires
} }
wantBans(t, s.admin(http.MethodPost, proxy.BansPath, body, http.StatusOK), want) wantBans(t, s.admin(http.MethodPost, servewaf.BansPath, body, http.StatusOK), want)
} }
} }
@@ -250,7 +250,7 @@ func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) {
"request body too large", "request body too large",
}, },
} { } {
got := s.admin(http.MethodPost, proxy.BansPath, tc.body, http.StatusBadRequest) got := s.admin(http.MethodPost, servewaf.BansPath, tc.body, http.StatusBadRequest)
if !strings.Contains(string(got.body), tc.want) { if !strings.Contains(string(got.body), tc.want) {
t.Errorf("%.80s was answered %q, want it to say %q", tc.body, got.body, tc.want) t.Errorf("%.80s was answered %q, want it to say %q", tc.body, got.body, tc.want)
} }
@@ -283,7 +283,7 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
Bans []state.BanEntry `json:"bans"` Bans []state.BanEntry `json:"bans"`
} }
decode(t, s.admin(http.MethodGet, proxy.ClientsPath+addr, "", http.StatusOK), &got) decode(t, s.admin(http.MethodGet, servewaf.ClientsPath+addr, "", http.StatusOK), &got)
if got.Client == nil { if got.Client == nil {
t.Fatalf("%s: no client", addr) t.Fatalf("%s: no client", addr)
@@ -305,13 +305,13 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
// Of an address no request came from and no ban covers, nothing is // Of an address no request came from and no ban covers, nothing is
// known. // known.
got := s.admin(http.MethodGet, proxy.ClientsPath+"198.51.100.99", "", http.StatusOK) got := s.admin(http.MethodGet, servewaf.ClientsPath+"198.51.100.99", "", http.StatusOK)
if string(got.body) != "{\n \"client\": null,\n \"bans\": []\n}\n" { if string(got.body) != "{\n \"client\": null,\n \"bans\": []\n}\n" {
t.Errorf("an unknown client is answered\n%s", got.body) t.Errorf("an unknown client is answered\n%s", got.body)
} }
s.admin(http.MethodGet, proxy.ClientsPath+"203.0.113", "", http.StatusBadRequest) s.admin(http.MethodGet, servewaf.ClientsPath+"203.0.113", "", http.StatusBadRequest)
s.admin(http.MethodDelete, proxy.BansPath+"/203.0.113.0/24", "", s.admin(http.MethodDelete, servewaf.BansPath+"/203.0.113.0/24", "",
http.StatusBadRequest) http.StatusBadRequest)
} }
@@ -342,10 +342,10 @@ type adminEndpoint struct {
// bans on otherClient, and asking about otherClient. // bans on otherClient, and asking about otherClient.
func adminEndpoints() []adminEndpoint { func adminEndpoints() []adminEndpoint {
return []adminEndpoint{ return []adminEndpoint{
{http.MethodGet, proxy.BansPath, ""}, {http.MethodGet, servewaf.BansPath, ""},
{http.MethodPost, proxy.BansPath, banOtherClient}, {http.MethodPost, servewaf.BansPath, banOtherClient},
{http.MethodDelete, proxy.BansPath + "/" + otherClient, ""}, {http.MethodDelete, servewaf.BansPath + "/" + otherClient, ""},
{http.MethodGet, proxy.ClientsPath + otherClient, ""}, {http.MethodGet, servewaf.ClientsPath + otherClient, ""},
} }
} }
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"maps" "maps"
@@ -10,8 +10,8 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
const ( const (
@@ -201,7 +201,7 @@ func TestObserveModeWorksOutABanOnlyWhenItsAlertWouldBeSent(t *testing.T) {
// never sent them, and returns the queue they wait in as well. // never sent them, and returns the queue they wait in as well.
func startWithAlerts( func startWithAlerts(
t *testing.T, env map[string]string, t *testing.T, env map[string]string,
) (*sender, *clock, *proxy.Server, *alerts.Queue) { ) (*sender, *clock, *servewaf.Server, *alerts.Queue) {
t.Helper() t.Helper()
return startAppWithAlerts(t, func(http.ResponseWriter, *http.Request) {}, env) return startAppWithAlerts(t, func(http.ResponseWriter, *http.Request) {}, env)
@@ -210,7 +210,7 @@ func startWithAlerts(
// startAppWithAlerts is startWithAlerts in front of the app handler. // startAppWithAlerts is startWithAlerts in front of the app handler.
func startAppWithAlerts( func startAppWithAlerts(
t *testing.T, handler http.HandlerFunc, env map[string]string, t *testing.T, handler http.HandlerFunc, env map[string]string,
) (*sender, *clock, *proxy.Server, *alerts.Queue) { ) (*sender, *clock, *servewaf.Server, *alerts.Queue) {
t.Helper() t.Helper()
app := startApp(t, handler) app := startApp(t, handler)
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"net/netip" "net/netip"
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"maps" "maps"
@@ -13,9 +13,9 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly" "sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
// The anomaly thresholds: the prefix of a scope followed by the end of a // The anomaly thresholds: the prefix of a scope followed by the end of a
@@ -319,7 +319,7 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
// kept. // kept.
geojsURL, _, release := startHeldGeoJS(t) geojsURL, _, release := startHeldGeoJS(t)
var server atomic.Pointer[proxy.Server] var server atomic.Pointer[servewaf.Server]
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
release() release()
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"fmt" "fmt"
@@ -16,10 +16,10 @@ import (
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/lookup" "sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation" "sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
const ( const (
@@ -81,7 +81,7 @@ func TestSixteenThousandCookiesAreRefusedWellWithinTheTimeLimit(t *testing.T) {
t.Errorf("answered after %s, want well within %s", took, answerTimeLimit) t.Errorf("answered after %s, want well within %s", took, answerTimeLimit)
} }
wantDisposition(t, got, tooLarge, proxy.DispositionDeny) wantDisposition(t, got, tooLarge, servewaf.DispositionDeny)
line := s.out.requestLine(t) line := s.out.requestLine(t)
wantLine(t, line, tooLarge, requestlog.ActionTooLarge) wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
@@ -144,13 +144,13 @@ func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
line := s.out.requestLine(t) line := s.out.requestLine(t)
if tc.refused { if tc.refused {
wantDisposition(t, got, tooLarge, proxy.DispositionDeny) wantDisposition(t, got, tooLarge, servewaf.DispositionDeny)
wantLine(t, line, tooLarge, requestlog.ActionTooLarge) wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
return return
} }
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
wantLine(t, line, http.StatusOK, requestlog.ActionForward) wantLine(t, line, http.StatusOK, requestlog.ActionForward)
if line.DurationWAF == nil { if line.DurationWAF == nil {
@@ -175,7 +175,7 @@ func TestHeaderLimitsHoldWhateverTheCoreRuleSetSettingsSay(t *testing.T) {
req := s.question(siteA, client, "/") req := s.question(siteA, client, "/")
req.Header.Set(cookieHeader, cookies(cookieLimit+1)) req.Header.Set(cookieHeader, cookies(cookieLimit+1))
wantDisposition(t, do(t, req), tooLarge, proxy.DispositionDeny) wantDisposition(t, do(t, req), tooLarge, servewaf.DispositionDeny)
}) })
} }
} }
@@ -187,7 +187,7 @@ func TestClientInTheAllowNetsSkipsTheHeaderLimits(t *testing.T) {
req := s.question(siteA, client, "/") req := s.question(siteA, client, "/")
req.Header.Set(cookieHeader, cookies(cookieLimit+1)) req.Header.Set(cookieHeader, cookies(cookieLimit+1))
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
} }
func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) { func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
@@ -204,7 +204,7 @@ func TestObserveModeAllowsARequestOverTheHeaderLimitsUnread(t *testing.T) {
t.Errorf("answered after %s, want well within %s", took, answerTimeLimit) t.Errorf("answered after %s, want well within %s", took, answerTimeLimit)
} }
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
line := s.out.requestLine(t) line := s.out.requestLine(t)
if line.WouldAction != requestlog.ActionTooLarge || line.DurationWAF != nil { if line.WouldAction != requestlog.ActionTooLarge || line.DurationWAF != nil {
@@ -224,8 +224,8 @@ func TestRequestsOverTheHeaderLimitsCountTowardTheRateLimits(t *testing.T) {
return do(t, req) return do(t, req)
} }
wantDisposition(t, ask(), tooLarge, proxy.DispositionDeny) wantDisposition(t, ask(), tooLarge, servewaf.DispositionDeny)
wantDisposition(t, ask(), http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, ask(), http.StatusForbidden, servewaf.DispositionDeny)
lines := s.out.requestLines(t, 2) lines := s.out.requestLines(t, 2)
wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionRateLimited) wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionRateLimited)
@@ -249,7 +249,7 @@ func TestTheHeaderLimitsComeAfterTheTrapPathsAndTheRuleFiles(t *testing.T) {
req := s.question(siteA, client, tc.path) req := s.question(siteA, client, tc.path)
req.Header.Set(cookieHeader, cookies(cookieLimit+1)) req.Header.Set(cookieHeader, cookies(cookieLimit+1))
wantDisposition(t, do(t, req), http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, do(t, req), http.StatusForbidden, servewaf.DispositionDeny)
wantLine(t, s.out.requestLine(t), http.StatusForbidden, tc.action) wantLine(t, s.out.requestLine(t), http.StatusForbidden, tc.action)
}) })
} }
@@ -265,7 +265,7 @@ func TestNotWaitingForGeoJSNeverLowersAClientsLimit(t *testing.T) {
req := s.question(siteA, fromDE, "/") req := s.question(siteA, fromDE, "/")
req.Header.Set(cookieHeader, cookies(cookieLimit+1)) req.Header.Set(cookieHeader, cookies(cookieLimit+1))
wantDisposition(t, do(t, req), tooLarge, proxy.DispositionDeny) wantDisposition(t, do(t, req), tooLarge, servewaf.DispositionDeny)
line := s.out.requestLine(t) line := s.out.requestLine(t)
wantLine(t, line, tooLarge, requestlog.ActionTooLarge) wantLine(t, line, tooLarge, requestlog.ActionTooLarge)
@@ -275,7 +275,7 @@ func TestNotWaitingForGeoJSNeverLowersAClientsLimit(t *testing.T) {
} }
// The client is under no ban, and GeoJS places it in a country. // The client is under no ban, and GeoJS places it in a country.
wantDisposition(t, s.ask(fromDE, "/"), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, s.ask(fromDE, "/"), http.StatusOK, servewaf.DispositionAllow)
} }
func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T) { func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T) {
@@ -291,7 +291,7 @@ func TestAClientGeoJSPlacedInNoCountryCountsAsUnknownWithoutWaiting(t *testing.T
req := s.question(siteA, client, "/") req := s.question(siteA, client, "/")
req.Header.Set(cookieHeader, cookies(cookieLimit+1)) req.Header.Set(cookieHeader, cookies(cookieLimit+1))
wantDisposition(t, do(t, req), http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, do(t, req), http.StatusForbidden, servewaf.DispositionDeny)
wantLine(t, s.out.requestLine(t), http.StatusForbidden, requestlog.ActionRateLimited) wantLine(t, s.out.requestLine(t), http.StatusForbidden, requestlog.ActionRateLimited)
} }
@@ -306,7 +306,7 @@ func TestObserveModeWaitsForGeoJSForARequestOverTheHeaderLimits(t *testing.T) {
req.Header.Set(cookieHeader, cookies(cookieLimit+1)) req.Header.Set(cookieHeader, cookies(cookieLimit+1))
got := do(t, req) got := do(t, req)
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country") asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country")
if asn != asnDE || country != "DE" { if asn != asnDE || country != "DE" {
@@ -318,7 +318,9 @@ func TestObserveModeWaitsForGeoJSForARequestOverTheHeaderLimits(t *testing.T) {
// newServeWAFInBubble makes a servewaf for a synctest bubble: GeoJS never // newServeWAFInBubble makes a servewaf for a synctest bubble: GeoJS never
// answers, a setting needs its answer, and everything is at its default but // answers, a setting needs its answer, and everything is at its default but
// for env. No request to it may use the network. // for env. No request to it may use the network.
func newServeWAFInBubble(t *testing.T, env map[string]string) (*proxy.Server, *output) { func newServeWAFInBubble(
t *testing.T, env map[string]string,
) (*servewaf.Server, *output) {
t.Helper() t.Helper()
settings := map[string]string{deniedCountries: "KP"} settings := map[string]string{deniedCountries: "KP"}
@@ -332,13 +334,13 @@ func newServeWAFInBubble(t *testing.T, env map[string]string) (*proxy.Server, *o
// askDirectly hands a question about a request from client straight to server, // askDirectly hands a question about a request from client straight to server,
// as change leaves it, and returns the answer's status and how long it took. // as change leaves it, and returns the answer's status and how long it took.
func askDirectly( func askDirectly(
t *testing.T, server *proxy.Server, change func(*http.Request), t *testing.T, server *servewaf.Server, change func(*http.Request),
) (int, time.Duration) { ) (int, time.Duration) {
t.Helper() t.Helper()
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody) req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
req.RemoteAddr = net.JoinHostPort(rproxyPeer, "1234") req.RemoteAddr = net.JoinHostPort(rproxyPeer, "1234")
req.Header.Set(proxy.ClientHeader, client) req.Header.Set(servewaf.ClientHeader, client)
req.Header.Set(requestIDHeader, "the-id") req.Header.Set(requestIDHeader, "the-id")
change(req) change(req)
@@ -351,17 +353,17 @@ func askDirectly(
} }
// banClient bans client at now. // banClient bans client at now.
func banClient(_ *testing.T, server *proxy.Server, now time.Time) { func banClient(_ *testing.T, server *servewaf.Server, now time.Time) {
server.Ledger.BanForLimit(netip.MustParsePrefix(client+"/32"), now, bans.Notes{}) server.Ledger.BanForLimit(netip.MustParsePrefix(client+"/32"), now, bans.Notes{})
} }
// countClient counts one request from client at now for the rate limits. // countClient counts one request from client at now for the rate limits.
func countClient(_ *testing.T, server *proxy.Server, now time.Time) { func countClient(_ *testing.T, server *servewaf.Server, now time.Time) {
server.Limiter.Count(netip.MustParsePrefix(client+"/32"), now, wholeLimit) server.Limiter.Count(netip.MustParsePrefix(client+"/32"), now, wholeLimit)
} }
// listClient puts client on the blocklist at dropURL. // listClient puts client on the blocklist at dropURL.
func listClient(t *testing.T, server *proxy.Server, now time.Time) { func listClient(t *testing.T, server *servewaf.Server, now time.Time) {
t.Helper() t.Helper()
err := server.Lists.Load([]reputation.List{{ err := server.Lists.Load([]reputation.List{{
@@ -373,13 +375,13 @@ func listClient(t *testing.T, server *proxy.Server, now time.Time) {
} }
// decideOnClient puts client on the CrowdSec decision list. // decideOnClient puts client on the CrowdSec decision list.
func decideOnClient(t *testing.T, server *proxy.Server, now time.Time) { func decideOnClient(t *testing.T, server *servewaf.Server, now time.Time) {
t.Helper() t.Helper()
loadDecisions(t, server, now, clientsDecision) loadDecisions(t, server, now, clientsDecision)
} }
// leaveAsItIs prepares nothing. // leaveAsItIs prepares nothing.
func leaveAsItIs(*testing.T, *proxy.Server, time.Time) {} func leaveAsItIs(*testing.T, *servewaf.Server, time.Time) {}
func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) { func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
t.Parallel() t.Parallel()
@@ -390,7 +392,7 @@ func TestARefusedRequestDoesNotWaitForASlowGeoJS(t *testing.T) {
name string name string
env map[string]string env map[string]string
// prepare gets the server ready, at now, before the question. // prepare gets the server ready, at now, before the question.
prepare func(t *testing.T, server *proxy.Server, now time.Time) prepare func(t *testing.T, server *servewaf.Server, now time.Time)
// cookie is the request's Cookie header. // cookie is the request's Cookie header.
cookie string cookie string
status int status int
@@ -461,7 +463,7 @@ func TestAQuestionWaitsForASlowGeoJSOnlyAsLongAsTheDefaultSays(t *testing.T) {
name string name string
env map[string]string env map[string]string
// prepare gets the server ready, at now, before the question. // prepare gets the server ready, at now, before the question.
prepare func(t *testing.T, server *proxy.Server, now time.Time) prepare func(t *testing.T, server *servewaf.Server, now time.Time)
status int status int
action string action string
}{ }{
@@ -519,10 +521,10 @@ func TestServeWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs(t *testin
t.Cleanup(func() { _ = resolver.Close() }) t.Cleanup(func() { _ = resolver.Close() })
server, out, _ := newServeWAFWith(t, func(params proxy.Params) *proxy.Server { server, out, _ := newServeWAFWith(t, func(params servewaf.Params) *servewaf.Server {
params.AbuseIPDBURL = silentAbuseIPDBURL params.AbuseIPDBURL = silentAbuseIPDBURL
return proxy.NewServeWAF(params) return servewaf.New(params)
}, held, time.Now, map[string]string{ }, held, time.Now, map[string]string{
deniedCountries: "KP", wafMode: block, deniedCountries: "KP", wafMode: block,
dnsblZones: dnsblZone, dnsblResolver: resolver.LocalAddr().String(), dnsblZones: dnsblZone, dnsblResolver: resolver.LocalAddr().String(),
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"net/http" "net/http"
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"bufio" "bufio"
@@ -15,8 +15,8 @@ import (
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
const ( const (
@@ -382,7 +382,7 @@ func (c *clock) advance(d time.Duration) {
// window. // window.
func startWithClock( func startWithClock(
t *testing.T, geojsURL string, env map[string]string, t *testing.T, geojsURL string, env map[string]string,
) (*sender, *clock, *proxy.Server) { ) (*sender, *clock, *servewaf.Server) {
t.Helper() t.Helper()
app := startApp(t, func(http.ResponseWriter, *http.Request) {}) app := startApp(t, func(http.ResponseWriter, *http.Request) {})
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"fmt" "fmt"
@@ -16,8 +16,8 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest" "sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
// The biased thresholds. // The biased thresholds.
@@ -419,7 +419,7 @@ func TestBanForALoweredLimitGivesThePercentageInItsNotesAndItsAlert(t *testing.T
// clock. // clock.
func startWithLookups( func startWithLookups(
t *testing.T, env map[string]string, t *testing.T, env map[string]string,
) (*sender, *proxy.Server, *alerts.Queue) { ) (*sender, *servewaf.Server, *alerts.Queue) {
t.Helper() t.Helper()
s, _, server, queue := startWithLookupsAndClock(t, env) s, _, server, queue := startWithLookupsAndClock(t, env)
@@ -434,7 +434,7 @@ func startWithLookups(
// country, and no other address. // country, and no other address.
func startWithLookupsAndClock( func startWithLookupsAndClock(
t *testing.T, env map[string]string, t *testing.T, env map[string]string,
) (*sender, *clock, *proxy.Server, *alerts.Queue) { ) (*sender, *clock, *servewaf.Server, *alerts.Queue) {
t.Helper() t.Helper()
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb") path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
@@ -463,11 +463,11 @@ func (s *sender) uploadFrom(from string) logLine {
// serveFromDE hands a question about a request from fromDE with method and body // serveFromDE hands a question about a request from fromDE with method and body
// straight to server's handler, without the network, and returns once it is // straight to server's handler, without the network, and returns once it is
// answered. // answered.
func serveFromDE(t *testing.T, server *proxy.Server, method string, body io.Reader) { func serveFromDE(t *testing.T, server *servewaf.Server, method string, body io.Reader) {
t.Helper() t.Helper()
req := httptest.NewRequestWithContext(t.Context(), method, "/", body) req := httptest.NewRequestWithContext(t.Context(), method, "/", body)
req.Header.Set(proxy.ClientHeader, fromDE) req.Header.Set(servewaf.ClientHeader, fromDE)
req.Header.Set(requestIDHeader, "the-id") req.Header.Set(requestIDHeader, "the-id")
server.Handler.ServeHTTP(httptest.NewRecorder(), req) server.Handler.ServeHTTP(httptest.NewRecorder(), req)
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"bufio" "bufio"
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"crypto/rand" "crypto/rand"
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"time" "time"
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"io" "io"
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"slices" "slices"
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"encoding/json" "encoding/json"
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"net/http" "net/http"
@@ -9,9 +9,9 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/reputation" "sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
// The CrowdSec settings, and the tests' engine, which is never asked: each // The CrowdSec settings, and the tests' engine, which is never asked: each
@@ -168,7 +168,7 @@ func TestObserveModeForwardsAClientTheCrowdSecDecisionListListsAndAlertsTheBan(
// decisionsURL, answer, the engine's answer, fetched at fetched, as // decisionsURL, answer, the engine's answer, fetched at fetched, as
// reputation.json would at start. // reputation.json would at start.
func loadDecisions( func loadDecisions(
t *testing.T, server *proxy.Server, fetched time.Time, answer string, t *testing.T, server *servewaf.Server, fetched time.Time, answer string,
) { ) {
t.Helper() t.Helper()
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"maps" "maps"
@@ -10,9 +10,9 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
const errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD" const errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
@@ -51,7 +51,7 @@ func (r refused) send(s *sender, from string) logLine {
// SWWAF_ERROR_BURST_THRESHOLD at threshold, and the settings in env. // SWWAF_ERROR_BURST_THRESHOLD at threshold, and the settings in env.
func startForErrorBurst( func startForErrorBurst(
t *testing.T, threshold string, env map[string]string, t *testing.T, threshold string, env map[string]string,
) (*sender, *clock, *proxy.Server) { ) (*sender, *clock, *servewaf.Server) {
t.Helper() t.Helper()
settings := map[string]string{ settings := map[string]string{
@@ -1,11 +1,11 @@
package proxy_test package servewaf_test
import ( import (
"net/http" "net/http"
"testing" "testing"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
func TestHealthEndpointCountsTowardNoLimit(t *testing.T) { func TestHealthEndpointCountsTowardNoLimit(t *testing.T) {
@@ -21,7 +21,7 @@ func TestHealthEndpointCountsTowardNoLimit(t *testing.T) {
) )
for range healthChecks { for range healthChecks {
got := s.adminRequest("", http.MethodGet, proxy.HealthPath, "", http.StatusOK) got := s.adminRequest("", http.MethodGet, servewaf.HealthPath, "", http.StatusOK)
if string(got.body) != "ok\n" || got.header.Get("Content-Type") != contentType { if string(got.body) != "ok\n" || got.header.Get("Content-Type") != contentType {
t.Errorf("health endpoint answered %q with Content-Type %q, want ok "+ t.Errorf("health endpoint answered %q with Content-Type %q, want ok "+
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"io" "io"
@@ -8,9 +8,9 @@ import (
"testing" "testing"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
func TestHistoryKeepsEachRequestOfTheClient(t *testing.T) { func TestHistoryKeepsEachRequestOfTheClient(t *testing.T) {
@@ -100,8 +100,8 @@ func TestOwnEndpointsAreNotInTheHistory(t *testing.T) {
s, _, server := startWithClock(t, "", map[string]string{metricsToken: token}) s, _, server := startWithClock(t, "", map[string]string{metricsToken: token})
s.adminRequest("", http.MethodGet, proxy.HealthPath, "", http.StatusOK) s.adminRequest("", http.MethodGet, servewaf.HealthPath, "", http.StatusOK)
s.adminRequest("", http.MethodGet, proxy.MetricsPath, "", http.StatusUnauthorized) s.adminRequest("", http.MethodGet, servewaf.MetricsPath, "", http.StatusUnauthorized)
s.scrape() s.scrape()
if clients := server.Limiter.Snapshot(); len(clients) != 0 { if clients := server.Limiter.Snapshot(); len(clients) != 0 {
@@ -126,7 +126,7 @@ func TestPathUnderTheOwnPrefixIsNeitherForwardedNorRefused(t *testing.T) {
} }
// historyOf returns the history of the client at addr. // historyOf returns the history of the client at addr.
func historyOf(t *testing.T, server *proxy.Server, addr string) ratelimit.History { func historyOf(t *testing.T, server *servewaf.Server, addr string) ratelimit.History {
t.Helper() t.Helper()
client := netip.MustParsePrefix(addr + "/32") client := netip.MustParsePrefix(addr + "/32")
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"context" "context"
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"net/http" "net/http"
@@ -379,7 +379,7 @@ func TestRequestWaitsAsLongAsTheLookupTimeoutSays(t *testing.T) {
// clock of the test's own: the wait lasts exactly as long as it should, // clock of the test's own: the wait lasts exactly as long as it should,
// however slowly the test process runs. Nothing in it may wait on the // however slowly the test process runs. Nothing in it may wait on the
// network, which would keep that clock from moving on: the request is // network, which would keep that clock from moving on: the request is
// handed to the proxy's handler, and GeoJS is one that never answers. // handed to servewaf's handler, and GeoJS is one that never answers.
synctest.Test(t, func(t *testing.T) { synctest.Test(t, func(t *testing.T) {
// Not the default. The exclusive list needs the answer, and the app // Not the default. The exclusive list needs the answer, and the app
// is never reached. // is never reached.
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"io" "io"
@@ -12,8 +12,8 @@ import (
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/lookup" "sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
const ( const (
@@ -32,7 +32,7 @@ func TestMetricsAreOffWhileTheTokenIsUnset(t *testing.T) {
// An empty token does not match the unset one either. // An empty token does not match the unset one either.
for _, authorization := range []string{bearer, "Bearer ", ""} { for _, authorization := range []string{bearer, "Bearer ", ""} {
s.adminRequest(authorization, http.MethodGet, proxy.MetricsPath, "", s.adminRequest(authorization, http.MethodGet, servewaf.MetricsPath, "",
http.StatusNotFound) http.StatusNotFound)
} }
} }
@@ -46,18 +46,18 @@ func TestMetricsNeedTheTokenAndOtherPathsAreNotFound(t *testing.T) {
method, path, authorization string method, path, authorization string
status int status int
}{ }{
{http.MethodGet, proxy.MetricsPath, "", http.StatusUnauthorized}, {http.MethodGet, servewaf.MetricsPath, "", http.StatusUnauthorized},
{ {
http.MethodGet, proxy.MetricsPath, "Bearer " + strings.ToUpper(token), http.MethodGet, servewaf.MetricsPath, "Bearer " + strings.ToUpper(token),
http.StatusUnauthorized, http.StatusUnauthorized,
}, },
{http.MethodGet, proxy.MetricsPath, "Basic " + token, http.StatusUnauthorized}, {http.MethodGet, servewaf.MetricsPath, "Basic " + token, http.StatusUnauthorized},
{http.MethodGet, proxy.MetricsPath, bearer, http.StatusOK}, {http.MethodGet, servewaf.MetricsPath, bearer, http.StatusOK},
{http.MethodGet, proxy.MetricsPath, "bearer " + token, http.StatusOK}, {http.MethodGet, servewaf.MetricsPath, "bearer " + token, http.StatusOK},
{http.MethodPost, proxy.MetricsPath, bearer, http.StatusNotFound}, {http.MethodPost, servewaf.MetricsPath, bearer, http.StatusNotFound},
{http.MethodGet, proxy.MetricsPath + "/", bearer, http.StatusNotFound}, {http.MethodGet, servewaf.MetricsPath + "/", bearer, http.StatusNotFound},
{http.MethodGet, "/_smallwebwaf/bans", bearer, http.StatusNotFound}, {http.MethodGet, "/_smallwebwaf/bans", bearer, http.StatusNotFound},
{http.MethodPost, proxy.HealthPath, "", http.StatusNotFound}, {http.MethodPost, servewaf.HealthPath, "", http.StatusNotFound},
} { } {
got := s.adminRequest(tc.authorization, tc.method, tc.path, "", tc.status) got := s.adminRequest(tc.authorization, tc.method, tc.path, "", tc.status)
@@ -180,7 +180,7 @@ func TestMetricsCountTheBansAnAdminMakes(t *testing.T) {
wantMetric(t, s.scrape(), admins, 1) wantMetric(t, s.scrape(), admins, 1)
// And a ban made through the endpoint. // And a ban made through the endpoint.
s.admin(http.MethodPost, proxy.BansPath, banOtherClient, http.StatusOK) s.admin(http.MethodPost, servewaf.BansPath, banOtherClient, http.StatusOK)
wantMetric(t, s.scrape(), admins, 2) wantMetric(t, s.scrape(), admins, 2)
} }
@@ -365,7 +365,7 @@ func keptAnswer(addr, country string) lookup.Answer {
func scrape(t *testing.T, addr string) string { func scrape(t *testing.T, addr string) string {
t.Helper() t.Helper()
req := newRequest(t, http.MethodGet, addr, proxy.MetricsPath, http.NoBody) req := newRequest(t, http.MethodGet, addr, servewaf.MetricsPath, http.NoBody)
req.Header.Set("Authorization", bearer) req.Header.Set("Authorization", bearer)
got := do(t, req) got := do(t, req)
@@ -380,7 +380,7 @@ func scrape(t *testing.T, addr string) string {
func (s *sender) scrape() string { func (s *sender) scrape() string {
s.t.Helper() s.t.Helper()
return string(s.adminRequest(bearer, http.MethodGet, proxy.MetricsPath, "", return string(s.adminRequest(bearer, http.MethodGet, servewaf.MetricsPath, "",
http.StatusOK).body) http.StatusOK).body)
} }
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"net/http" "net/http"
@@ -1,12 +1,10 @@
// Package proxy is servewaf's server. It runs the checks on the request each of // Package servewaf is the server the servewaf command runs. It answers rproxy's
// rproxy's questions carries, and answers rproxy what to do with that request. It // questions, takes rproxy's reports and answers its own endpoints, and writes a
// takes rproxy's report once the visitor's answer has ended, which counts the // log line for each.
// request's bytes and writes its last log line. It answers servewaf's own package servewaf
// endpoints: the health check, the metrics, the bans and the clients. It passes
// nothing to an app: internal/rproxy does that.
package proxy
import ( import (
"fmt"
"io" "io"
"log/slog" "log/slog"
"net/http" "net/http"
@@ -46,7 +44,7 @@ const BansPath = "/_smallwebwaf/bans"
// client's address after it, with SWWAF_ADMIN_TOKEN. // client's address after it, with SWWAF_ADMIN_TOKEN.
const ClientsPath = "/_smallwebwaf/clients/" const ClientsPath = "/_smallwebwaf/clients/"
// Params are what NewServeWAF needs. // Params are what New needs.
type Params struct { type Params struct {
Config *config.Config Config *config.Config
// RequestLog receives two JSON lines per question, one at its decision and // RequestLog receives two JSON lines per question, one at its decision and
@@ -63,6 +61,9 @@ type Params struct {
// LookupFile is the lookup database they are looked up in while // LookupFile is the lookup database they are looked up in while
// SWWAF_LOOKUP_SOURCE is file, and nil otherwise. // SWWAF_LOOKUP_SOURCE is file, and nil otherwise.
LookupFile *lookup.File LookupFile *lookup.File
// CoreRuleSet is what NewCoreRuleSet returns: what inspects requests
// while SWWAF_WAF_MODE is not off, and nil while it is.
CoreRuleSet *waf.CoreRuleSet
// Now tells the time by which requests are counted for the rate // Now tells the time by which requests are counted for the rate
// limits, bans are made and run out, and GeoJS's answers are kept, // limits, bans are made and run out, and GeoJS's answers are kept,
// normally time.Now in UTC, the time the state files give. // normally time.Now in UTC, the time the state files give.
@@ -147,7 +148,7 @@ func newHandler(params Params) *handler {
dnsbl: dnsbl, dnsbl: dnsbl,
abuseIPDB: abuseIPDB, abuseIPDB: abuseIPDB,
rules: params.Rules, rules: params.Rules,
coreRuleSet: newCoreRuleSet(params.Config), coreRuleSet: params.CoreRuleSet,
alerts: params.Alerts, alerts: params.Alerts,
} }
h.geojs = lookup.New(lookup.Params{ h.geojs = lookup.New(lookup.Params{
@@ -205,29 +206,35 @@ func newReputation(
return lists, dnsbl, abuseIPDB return lists, dnsbl, abuseIPDB
} }
// newCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL, // NewCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL,
// without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies // without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies
// up to SWWAF_WAF_BODY_LIMIT, or nil while SWWAF_WAF_MODE is off. It reads at // up to SWWAF_WAF_BODY_LIMIT, with the exclusions in
// most QuestionBodyMaxBytes less one byte: a question's body of // SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, or nil while SWWAF_WAF_MODE is off. It
// reads at most QuestionBodyMaxBytes less one byte: a question's body of
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a // QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
// JSON or XML body over its limit. // JSON or XML body over its limit. An exclusions file that cannot be read, does
func newCoreRuleSet(cfg *config.Config) *waf.CoreRuleSet { // not parse or names an id of smallwebwaf's own rules is an error in every
if cfg.WAFMode == config.WAFModeOff { // mode. While the mode is off, a set file is loaded with the Core Rule Set only
return nil // to check it.
func NewCoreRuleSet(cfg *config.Config) (*waf.CoreRuleSet, error) {
if cfg.WAFMode == config.WAFModeOff && cfg.CoreRuleSetExclusionsFile == "" {
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
} }
coreRuleSet, err := waf.New(waf.Params{ coreRuleSet, err := waf.New(waf.Params{
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules, ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1), BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
}) })
if err != nil { if err != nil {
// The Core Rule Set is built in, and the settings cannot break it: return nil, fmt.Errorf("SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: %w", err)
// the paranoia level is from 1 to 4, and the id of no rule switches
// nothing off.
panic(err)
} }
return coreRuleSet if cfg.WAFMode == config.WAFModeOff {
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
}
return coreRuleSet, nil
} }
// handler is servewaf's handler. It holds what every request shares; what // handler is servewaf's handler. It holds what every request shares; what
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"bufio" "bufio"
@@ -18,10 +18,10 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup" "sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rproxy" "sneak.berlin/go/smallwebwaf/internal/rproxy"
"sneak.berlin/go/smallwebwaf/internal/rules" "sneak.berlin/go/smallwebwaf/internal/rules"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
const ( const (
@@ -300,7 +300,7 @@ func startProxyWithGeoJS(
func startProxyWithClock( func startProxyWithClock(
t *testing.T, appURL, geojsURL string, now func() time.Time, t *testing.T, appURL, geojsURL string, now func() time.Time,
env map[string]string, env map[string]string,
) (string, *output, *proxy.Server) { ) (string, *output, *servewaf.Server) {
t.Helper() t.Helper()
addr, out, server, _ := startProxyWithAlerts(t, appURL, geojsURL, now, env) addr, out, server, _ := startProxyWithAlerts(t, appURL, geojsURL, now, env)
@@ -313,7 +313,7 @@ func startProxyWithClock(
func startProxyWithAlerts( func startProxyWithAlerts(
t *testing.T, appURL, geojsURL string, now func() time.Time, t *testing.T, appURL, geojsURL string, now func() time.Time,
env map[string]string, env map[string]string,
) (string, *output, *proxy.Server, *alerts.Queue) { ) (string, *output, *servewaf.Server, *alerts.Queue) {
t.Helper() t.Helper()
server, out, alertQueue := newServeWAF(t, geojsURL, now, env) server, out, alertQueue := newServeWAF(t, geojsURL, now, env)
@@ -324,7 +324,7 @@ func startProxyWithAlerts(
// serve serves server on a port of its own until the test ends, and sets its // serve serves server on a port of its own until the test ends, and sets its
// Addr to where it listens. // Addr to where it listens.
func serve(t *testing.T, server *proxy.Server) { func serve(t *testing.T, server *servewaf.Server) {
t.Helper() t.Helper()
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0") listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
@@ -401,17 +401,17 @@ func startRProxy(
// inspected by it. // inspected by it.
func newServeWAF( func newServeWAF(
t *testing.T, geojsURL string, now func() time.Time, env map[string]string, t *testing.T, geojsURL string, now func() time.Time, env map[string]string,
) (*proxy.Server, *output, *alerts.Queue) { ) (*servewaf.Server, *output, *alerts.Queue) {
t.Helper() t.Helper()
return newServeWAFWith(t, proxy.NewServeWAF, geojsURL, now, env) return newServeWAFWith(t, servewaf.New, geojsURL, now, env)
} }
// newServeWAFWith is newServeWAF for the server build makes. // newServeWAFWith is newServeWAF for the server build makes.
func newServeWAFWith( func newServeWAFWith(
t *testing.T, build func(proxy.Params) *proxy.Server, t *testing.T, build func(servewaf.Params) *servewaf.Server,
geojsURL string, now func() time.Time, env map[string]string, geojsURL string, now func() time.Time, env map[string]string,
) (*proxy.Server, *output, *alerts.Queue) { ) (*servewaf.Server, *output, *alerts.Queue) {
t.Helper() t.Helper()
settings := map[string]string{ settings := map[string]string{
@@ -463,13 +463,19 @@ func newServeWAFWith(
} }
} }
server := build(proxy.Params{ coreRuleSet, err := servewaf.NewCoreRuleSet(cfg)
if err != nil {
t.Fatalf("Core Rule Set: %v", err)
}
server := build(servewaf.Params{
Config: cfg, Config: cfg,
RequestLog: out, RequestLog: out,
ProcessLog: processLog, ProcessLog: processLog,
GeoJSURL: geojsURL, GeoJSURL: geojsURL,
AbuseIPDBURL: abuseIPDBURL, AbuseIPDBURL: abuseIPDBURL,
LookupFile: lookupFile, LookupFile: lookupFile,
CoreRuleSet: coreRuleSet,
Now: now, Now: now,
Rules: ruleFiles, Rules: ruleFiles,
Alerts: alertQueue, Alerts: alertQueue,
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"net/http" "net/http"
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"context" "context"
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"fmt" "fmt"
@@ -15,10 +15,10 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation" "sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
// The reputation settings. // The reputation settings.
@@ -617,7 +617,7 @@ const (
// AbuseIPDB. // AbuseIPDB.
const abuseipdb = reputation.AbuseIPDBSource const abuseipdb = reputation.AbuseIPDBSource
// abuseIPDBURL is where newProxy has clients checked with AbuseIPDB: at // abuseIPDBURL is where newServeWAF has clients checked with AbuseIPDB: at
// abuseIPDBStandIn, which TestMain registers with Go's default transport, // abuseIPDBStandIn, which TestMain registers with Go's default transport,
// through which AbuseIPDB is asked. // through which AbuseIPDB is asked.
const abuseIPDBURL = "abuseipdb://stand-in/api/v2/check" const abuseIPDBURL = "abuseipdb://stand-in/api/v2/check"
@@ -975,7 +975,7 @@ func verdictsFetched() time.Time {
// verdictsFetched, as reputation.json would at start: one that lists the // verdictsFetched, as reputation.json would at start: one that lists the
// client from each zone listedBy gives for it, and one that does not from // client from each zone listedBy gives for it, and one that does not from
// each other zone. // each other zone.
func loadVerdicts(server *proxy.Server, listedBy map[string][]string) { func loadVerdicts(server *servewaf.Server, listedBy map[string][]string) {
verdicts := make([]reputation.Verdict, 0, len(listedBy)*len(server.DNSBL.Zones())) verdicts := make([]reputation.Verdict, 0, len(listedBy)*len(server.DNSBL.Zones()))
for client, zones := range listedBy { for client, zones := range listedBy {
@@ -993,7 +993,7 @@ func loadVerdicts(server *proxy.Server, listedBy map[string][]string) {
// loadScores puts into server's AbuseIPDB the score scores gives each // loadScores puts into server's AbuseIPDB the score scores gives each
// client, an IPv4 address, fetched at verdictsFetched, as reputation.json // client, an IPv4 address, fetched at verdictsFetched, as reputation.json
// would at start. // would at start.
func loadScores(server *proxy.Server, scores map[string]int64) { func loadScores(server *servewaf.Server, scores map[string]int64) {
kept := make([]reputation.Score, 0, len(scores)) kept := make([]reputation.Score, 0, len(scores))
for client, score := range scores { for client, score := range scores {
kept = append(kept, reputation.Score{ kept = append(kept, reputation.Score{
@@ -1007,7 +1007,7 @@ func loadScores(server *proxy.Server, scores map[string]int64) {
// wantAbuseIPDBChecks checks how many clients server has checked with // wantAbuseIPDBChecks checks how many clients server has checked with
// AbuseIPDB. // AbuseIPDB.
func wantAbuseIPDBChecks(t *testing.T, server *proxy.Server, want int) { func wantAbuseIPDBChecks(t *testing.T, server *servewaf.Server, want int) {
t.Helper() t.Helper()
if got := server.AbuseIPDB.Checked(); got != want { if got := server.AbuseIPDB.Checked(); got != want {
@@ -1017,7 +1017,7 @@ func wantAbuseIPDBChecks(t *testing.T, server *proxy.Server, want int) {
// loadLists puts copies of lists into server's lists, by URL, each with // loadLists puts copies of lists into server's lists, by URL, each with
// its lines, fetched at listsFetched, as reputation.json would at start. // its lines, fetched at listsFetched, as reputation.json would at start.
func loadLists(t *testing.T, server *proxy.Server, copies map[string][]string) { func loadLists(t *testing.T, server *servewaf.Server, copies map[string][]string) {
t.Helper() t.Helper()
lists := make([]reputation.List, 0, len(copies)) lists := make([]reputation.List, 0, len(copies))
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"context" "context"
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"io" "io"
@@ -10,9 +10,9 @@ import (
"testing" "testing"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
const ( const (
@@ -220,7 +220,7 @@ func TestTimingsAreInOrder(t *testing.T) {
wantStatus(t, do(t, req), http.StatusForbidden) wantStatus(t, do(t, req), http.StatusForbidden)
refused := out.endedLines(t, 2)[1] refused := out.endedLines(t, 2)[1]
wantStatus(t, get(t, server.Addr, proxy.HealthPath), http.StatusOK) wantStatus(t, get(t, server.Addr, servewaf.HealthPath), http.StatusOK)
health := out.endedLines(t, 3)[2] health := out.endedLines(t, 3)[2]
// A request passed to the app has every timing; one refused, none of // A request passed to the app has every timing; one refused, none of
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"net/http" "net/http"
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"slices" "slices"
@@ -1,4 +1,4 @@
package proxy package servewaf
import ( import (
"bytes" "bytes"
@@ -103,13 +103,13 @@ type Report struct {
DurationUpstreamTotal float64 `json:"duration_upstream_total"` DurationUpstreamTotal float64 `json:"duration_upstream_total"`
} }
// NewServeWAF returns the server `smallwebwaf servewaf` runs. It answers // New returns the server `smallwebwaf servewaf` runs. It answers
// rproxy's questions, takes rproxy's reports and answers its own endpoints. It // rproxy's questions, takes rproxy's reports and answers its own endpoints. It
// passes nothing to an app. rproxy applies the size and time limits to the // passes nothing to an app. rproxy applies the size and time limits to the
// visitor. Go's default limit on a request's line and headers, 1 MiB, stays: it // visitor. Go's default limit on a request's line and headers, 1 MiB, stays: it
// is far above rproxy's, and a question refused for its size would have rproxy // is far above rproxy's, and a question refused for its size would have rproxy
// let the request through. // let the request through.
func NewServeWAF(params Params) *Server { func New(params Params) *Server {
h := newHandler(params) h := newHandler(params)
return &Server{ return &Server{
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"context" "context"
@@ -19,9 +19,9 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
const ( const (
@@ -57,7 +57,7 @@ type serveWAF struct {
t *testing.T t *testing.T
addr string addr string
out *output out *output
server *proxy.Server server *servewaf.Server
clock *clock clock *clock
queue *alerts.Queue queue *alerts.Queue
} }
@@ -67,13 +67,13 @@ type serveWAF struct {
func startServeWAF(t *testing.T, geojsURL string, env map[string]string) *serveWAF { func startServeWAF(t *testing.T, geojsURL string, env map[string]string) *serveWAF {
t.Helper() t.Helper()
return startServeWAFWith(t, geojsURL, env, func(*proxy.Server) {}) return startServeWAFWith(t, geojsURL, env, func(*servewaf.Server) {})
} }
// startServeWAFWith is startServeWAF with change applied to the server before // startServeWAFWith is startServeWAF with change applied to the server before
// it starts. // it starts.
func startServeWAFWith( func startServeWAFWith(
t *testing.T, geojsURL string, env map[string]string, change func(*proxy.Server), t *testing.T, geojsURL string, env map[string]string, change func(*servewaf.Server),
) *serveWAF { ) *serveWAF {
t.Helper() t.Helper()
@@ -103,8 +103,8 @@ func (s *serveWAF) questionWith(
req := newRequest(s.t, method, s.addr, path, body) req := newRequest(s.t, method, s.addr, path, body)
req.Host = host req.Host = host
req.Header.Set(proxy.ClientHeader, from) req.Header.Set(servewaf.ClientHeader, from)
req.Header.Set(proxy.PeerHeader, rproxyPeer) req.Header.Set(servewaf.PeerHeader, rproxyPeer)
req.Header.Set(requestIDHeader, rand.Text()) req.Header.Set(requestIDHeader, rand.Text())
return req return req
@@ -131,13 +131,13 @@ func (s *serveWAF) allowed(from string) string {
s.t.Helper() s.t.Helper()
req := s.question(siteA, from, "/") req := s.question(siteA, from, "/")
wantDisposition(s.t, do(s.t, req), http.StatusOK, proxy.DispositionAllow) wantDisposition(s.t, do(s.t, req), http.StatusOK, servewaf.DispositionAllow)
return req.Header.Get(requestIDHeader) return req.Header.Get(requestIDHeader)
} }
// report sends rproxy's report and returns the answer. // report sends rproxy's report and returns the answer.
func (s *serveWAF) report(report proxy.Report) answer { func (s *serveWAF) report(report servewaf.Report) answer {
s.t.Helper() s.t.Helper()
body, err := json.Marshal(report) body, err := json.Marshal(report)
@@ -152,7 +152,7 @@ func (s *serveWAF) report(report proxy.Report) answer {
func (s *serveWAF) reportBody(body string) answer { func (s *serveWAF) reportBody(body string) answer {
s.t.Helper() s.t.Helper()
return do(s.t, newRequest(s.t, http.MethodPost, s.addr, proxy.ReportPath, return do(s.t, newRequest(s.t, http.MethodPost, s.addr, servewaf.ReportPath,
strings.NewReader(body))) strings.NewReader(body)))
} }
@@ -161,7 +161,7 @@ func (s *serveWAF) reportBody(body string) answer {
func (s *serveWAF) passedOn(from string) { func (s *serveWAF) passedOn(from string) {
s.t.Helper() s.t.Helper()
got := s.report(proxy.Report{ got := s.report(servewaf.Report{
RequestID: s.allowed(from), RequestID: s.allowed(from),
Status: http.StatusOK, Status: http.StatusOK,
RequestBytes: bodyBytes, RequestBytes: bodyBytes,
@@ -182,7 +182,7 @@ func (s *serveWAF) history() ratelimit.History {
func (s *serveWAF) metrics() string { func (s *serveWAF) metrics() string {
s.t.Helper() s.t.Helper()
req := newRequest(s.t, http.MethodGet, s.addr, proxy.MetricsPath, http.NoBody) req := newRequest(s.t, http.MethodGet, s.addr, servewaf.MetricsPath, http.NoBody)
req.Header.Set("Authorization", bearer) req.Header.Set("Authorization", bearer)
got := do(s.t, req) got := do(s.t, req)
@@ -196,7 +196,7 @@ func wantDisposition(t *testing.T, got answer, status int, disposition string) {
t.Helper() t.Helper()
wantStatus(t, got, status) wantStatus(t, got, status)
if answered := got.header.Get(proxy.DispositionHeader); answered != disposition { if answered := got.header.Get(servewaf.DispositionHeader); answered != disposition {
t.Errorf("answered with disposition %q, want %q", answered, disposition) t.Errorf("answered with disposition %q, want %q", answered, disposition)
} }
} }
@@ -248,10 +248,10 @@ func TestQuestionAllowedIsAnsweredAndLogged(t *testing.T) {
req := s.question(siteA, client, "/page?q=1&token=visible") req := s.question(siteA, client, "/page?q=1&token=visible")
req.Header.Set(requestIDHeader, "the-id") req.Header.Set(requestIDHeader, "the-id")
req.Header.Set("X-Forwarded-Proto", "https") req.Header.Set("X-Forwarded-Proto", "https")
req.Header.Set(proxy.ContentLengthHeader, "1234") req.Header.Set(servewaf.ContentLengthHeader, "1234")
got := do(t, req) got := do(t, req)
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
wantBody(t, got, "") wantBody(t, got, "")
if id := got.header.Get(requestIDHeader); id != "" { if id := got.header.Get(requestIDHeader); id != "" {
@@ -278,7 +278,7 @@ func TestIPv6VisitorIsCountedInItsGroup(t *testing.T) {
s.allowed("2001:db8::1") s.allowed("2001:db8::1")
got := s.ask("2001:db8::2", "/") got := s.ask("2001:db8::2", "/")
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
} }
func TestIPv4VisitorInIPv6FormIsTheIPv4Client(t *testing.T) { func TestIPv4VisitorInIPv6FormIsTheIPv4Client(t *testing.T) {
@@ -287,9 +287,9 @@ func TestIPv4VisitorInIPv6FormIsTheIPv4Client(t *testing.T) {
s := startServeWAF(t, "", map[string]string{denyNets: client}) s := startServeWAF(t, "", map[string]string{denyNets: client})
req := s.question(siteA, "::ffff:"+client, "/") req := s.question(siteA, "::ffff:"+client, "/")
req.Header.Set(proxy.PeerHeader, "::ffff:"+rproxyPeer) req.Header.Set(servewaf.PeerHeader, "::ffff:"+rproxyPeer)
wantDisposition(t, do(t, req), http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, do(t, req), http.StatusForbidden, servewaf.DispositionDeny)
line := s.out.requestLine(t) line := s.out.requestLine(t)
if line.ClientIP != client || line.PeerIP != rproxyPeer { if line.ClientIP != client || line.PeerIP != rproxyPeer {
@@ -305,13 +305,13 @@ func TestMalformedQuestionGets400AndAWarningAndNoLine(t *testing.T) {
change func(*http.Request) change func(*http.Request)
}{ }{
{"client not an address", func(req *http.Request) { {"client not an address", func(req *http.Request) {
req.Header.Set(proxy.ClientHeader, "not an address") req.Header.Set(servewaf.ClientHeader, "not an address")
}}, }},
{"client empty", func(req *http.Request) { {"client empty", func(req *http.Request) {
req.Header.Set(proxy.ClientHeader, "") req.Header.Set(servewaf.ClientHeader, "")
}}, }},
{"client address with a zone", func(req *http.Request) { {"client address with a zone", func(req *http.Request) {
req.Header.Set(proxy.ClientHeader, "2001:db8::1%eth0") req.Header.Set(servewaf.ClientHeader, "2001:db8::1%eth0")
}}, }},
{"id absent", func(req *http.Request) { {"id absent", func(req *http.Request) {
req.Header.Del(requestIDHeader) req.Header.Del(requestIDHeader)
@@ -353,7 +353,7 @@ func TestLongestValidIDIsTaken(t *testing.T) {
req := s.question(siteA, client, "/") req := s.question(siteA, client, "/")
req.Header.Set(requestIDHeader, strings.Repeat("x", 128)) req.Header.Set(requestIDHeader, strings.Repeat("x", 128))
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
} }
func TestQuestionWithABodyOver64KiBGets400(t *testing.T) { func TestQuestionWithABodyOver64KiBGets400(t *testing.T) {
@@ -361,15 +361,15 @@ func TestQuestionWithABodyOver64KiBGets400(t *testing.T) {
s := startServeWAF(t, "", nil) s := startServeWAF(t, "", nil)
over := strings.NewReader(strings.Repeat("x", proxy.QuestionBodyMaxBytes+1)) over := strings.NewReader(strings.Repeat("x", servewaf.QuestionBodyMaxBytes+1))
got := do(t, s.questionWith(http.MethodPost, siteA, client, "/", over)) got := do(t, s.questionWith(http.MethodPost, siteA, client, "/", over))
wantDisposition(t, got, http.StatusBadRequest, "") wantDisposition(t, got, http.StatusBadRequest, "")
wantProcessMessages(t, s, "a question is refused", 1) wantProcessMessages(t, s, "a question is refused", 1)
wantNoRequestLines(t, s) wantNoRequestLines(t, s)
whole := strings.NewReader(strings.Repeat("x", proxy.QuestionBodyMaxBytes)) whole := strings.NewReader(strings.Repeat("x", servewaf.QuestionBodyMaxBytes))
got = do(t, s.questionWith(http.MethodPost, siteA, client, "/", whole)) got = do(t, s.questionWith(http.MethodPost, siteA, client, "/", whole))
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
} }
func TestQuestionWithAHeldIDIsRefused(t *testing.T) { func TestQuestionWithAHeldIDIsRefused(t *testing.T) {
@@ -385,15 +385,15 @@ func TestQuestionWithAHeldIDIsRefused(t *testing.T) {
wantProcessMessages(t, s, "a question is refused", 1) wantProcessMessages(t, s, "a question is refused", 1)
// A denied request's id is held as well. // A denied request's id is held as well.
denied := s.question(siteA, otherClient, proxy.HealthPath) denied := s.question(siteA, otherClient, servewaf.HealthPath)
wantDisposition(t, do(t, denied), http.StatusNotFound, proxy.DispositionDeny) wantDisposition(t, do(t, denied), http.StatusNotFound, servewaf.DispositionDeny)
again = s.question(siteA, client, "/") again = s.question(siteA, client, "/")
again.Header.Set(requestIDHeader, denied.Header.Get(requestIDHeader)) again.Header.Set(requestIDHeader, denied.Header.Get(requestIDHeader))
wantDisposition(t, do(t, again), http.StatusBadRequest, "") wantDisposition(t, do(t, again), http.StatusBadRequest, "")
// The report ends the holding, and the refused questions changed nothing. // The report ends the holding, and the refused questions changed nothing.
got := s.report(proxy.Report{RequestID: id, ResponseBytes: answerBytes}) got := s.report(servewaf.Report{RequestID: id, ResponseBytes: answerBytes})
wantStatus(t, got, http.StatusNoContent) wantStatus(t, got, http.StatusNoContent)
if history := s.history(); history.ResponseBytes != answerBytes { if history := s.history(); history.ResponseBytes != answerBytes {
@@ -402,7 +402,7 @@ func TestQuestionWithAHeldIDIsRefused(t *testing.T) {
again = s.question(siteA, client, "/") again = s.question(siteA, client, "/")
again.Header.Set(requestIDHeader, id) again.Header.Set(requestIDHeader, id)
wantDisposition(t, do(t, again), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, do(t, again), http.StatusOK, servewaf.DispositionAllow)
} }
func TestReportNeverReachesAQuestionStillBeingHandled(t *testing.T) { func TestReportNeverReachesAQuestionStillBeingHandled(t *testing.T) {
@@ -436,7 +436,7 @@ func TestReportNeverReachesAQuestionStillBeingHandled(t *testing.T) {
// A report for a question not yet answered is ignored, and that question's // A report for a question not yet answered is ignored, and that question's
// id is taken. // id is taken.
got := s.report(proxy.Report{RequestID: id, ResponseBytes: 1000}) got := s.report(servewaf.Report{RequestID: id, ResponseBytes: 1000})
wantStatus(t, got, http.StatusNoContent) wantStatus(t, got, http.StatusNoContent)
// This would wait on the same lookup were the id not taken. // This would wait on the same lookup were the id not taken.
@@ -451,7 +451,7 @@ func TestReportNeverReachesAQuestionStillBeingHandled(t *testing.T) {
select { select {
case got := <-answered: case got := <-answered:
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
case <-time.After(waitLimit): case <-time.After(waitLimit):
t.Fatal("the question got no answer") t.Fatal("the question got no answer")
} }
@@ -466,7 +466,7 @@ func TestReportNeverReachesAQuestionStillBeingHandled(t *testing.T) {
} }
// The true report comes after, and is the one that counts. // The true report comes after, and is the one that counts.
got = s.report(proxy.Report{RequestID: id, ResponseBytes: answerBytes}) got = s.report(servewaf.Report{RequestID: id, ResponseBytes: answerBytes})
wantStatus(t, got, http.StatusNoContent) wantStatus(t, got, http.StatusNoContent)
if history := s.history(); history.ResponseBytes != answerBytes { if history := s.history(); history.ResponseBytes != answerBytes {
@@ -484,13 +484,13 @@ func TestPeerAbsentOrUnreadableIsAnsweredAndLoggedInvalid(t *testing.T) {
s := startServeWAF(t, "", nil) s := startServeWAF(t, "", nil)
req := s.question(siteA, client, "/") req := s.question(siteA, client, "/")
req.Header.Del(proxy.PeerHeader) req.Header.Del(servewaf.PeerHeader)
if peer != "" { if peer != "" {
req.Header.Set(proxy.PeerHeader, peer) req.Header.Set(servewaf.PeerHeader, peer)
} }
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
if line := s.out.requestLine(t); line.PeerIP != "invalid IP" { if line := s.out.requestLine(t); line.PeerIP != "invalid IP" {
t.Errorf("log line has peer_ip %q, want invalid IP", line.PeerIP) t.Errorf("log line has peer_ip %q, want invalid IP", line.PeerIP)
@@ -511,8 +511,8 @@ func TestContentLengthUnreadableOrNegativeCountsAsAbsent(t *testing.T) {
}) })
req := s.question(siteA, client, "/") req := s.question(siteA, client, "/")
req.Header.Set(proxy.ContentLengthHeader, announced) req.Header.Set(servewaf.ContentLengthHeader, announced)
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
if line := s.out.requestLine(t); line.ContentLength != 0 { if line := s.out.requestLine(t); line.ContentLength != 0 {
t.Errorf("log line has content_length %d, want none", line.ContentLength) t.Errorf("log line has content_length %d, want none", line.ContentLength)
@@ -527,13 +527,13 @@ func TestHeaderSentTwiceUsesItsFirstValue(t *testing.T) {
s := startServeWAF(t, "", nil) s := startServeWAF(t, "", nil)
req := s.question(siteA, client, "/") req := s.question(siteA, client, "/")
req.Header.Add(proxy.ClientHeader, otherClient) req.Header.Add(servewaf.ClientHeader, otherClient)
req.Header.Add(proxy.PeerHeader, "172.18.0.3") req.Header.Add(servewaf.PeerHeader, "172.18.0.3")
req.Header.Set(requestIDHeader, "first") req.Header.Set(requestIDHeader, "first")
req.Header.Add(requestIDHeader, "second") req.Header.Add(requestIDHeader, "second")
req.Header.Set(proxy.ContentLengthHeader, "1234") req.Header.Set(servewaf.ContentLengthHeader, "1234")
req.Header.Add(proxy.ContentLengthHeader, "5678") req.Header.Add(servewaf.ContentLengthHeader, "5678")
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
line := s.out.requestLine(t) line := s.out.requestLine(t)
if line.ClientIP != client || line.PeerIP != rproxyPeer || if line.ClientIP != client || line.PeerIP != rproxyPeer ||
@@ -562,7 +562,7 @@ func sendQuestionWithPartOfItsBody(t *testing.T, conn net.Conn) {
_, err := fmt.Fprintf(conn, _, err := fmt.Fprintf(conn,
"POST / HTTP/1.1\r\nHost: %s\r\n%s: %s\r\n%s: an-id\r\n"+ "POST / HTTP/1.1\r\nHost: %s\r\n%s: %s\r\n%s: an-id\r\n"+
"Content-Length: 100\r\n\r\npart", "Content-Length: 100\r\n\r\npart",
siteA, proxy.ClientHeader, client, requestIDHeader) siteA, servewaf.ClientHeader, client, requestIDHeader)
if err != nil { if err != nil {
t.Fatalf("send the question: %v", err) t.Fatalf("send the question: %v", err)
} }
@@ -592,7 +592,7 @@ func TestQuestionWhoseBodyStopsArrivingGetsNoAnswer(t *testing.T) {
t.Parallel() t.Parallel()
s := startServeWAFWith(t, "", map[string]string{metricsToken: token}, s := startServeWAFWith(t, "", map[string]string{metricsToken: token},
func(server *proxy.Server) { server.ReadTimeout = readTimeout }) func(server *servewaf.Server) { server.ReadTimeout = readTimeout })
conn := dial(t, s.addr) conn := dial(t, s.addr)
sendQuestionWithPartOfItsBody(t, conn) sendQuestionWithPartOfItsBody(t, conn)
@@ -615,7 +615,7 @@ func TestQuestionWhoseBodyStopsArrivingGetsNoAnswer(t *testing.T) {
// The id was not taken. // The id was not taken.
again := s.question(siteA, client, "/") again := s.question(siteA, client, "/")
again.Header.Set(requestIDHeader, "an-id") again.Header.Set(requestIDHeader, "an-id")
wantDisposition(t, do(t, again), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, do(t, again), http.StatusOK, servewaf.DispositionAllow)
}) })
} }
} }
@@ -632,7 +632,7 @@ func TestQuestionDeniedGetsTheBanResponse(t *testing.T) {
}) })
got := s.ask(client, "/") got := s.ask(client, "/")
wantDisposition(t, got, status, proxy.DispositionDeny) wantDisposition(t, got, status, servewaf.DispositionDeny)
wantBody(t, got, http.StatusText(status)+"\n") wantBody(t, got, http.StatusText(status)+"\n")
wantLine(t, s.out.requestLine(t), status, requestlog.ActionDenied) wantLine(t, s.out.requestLine(t), status, requestlog.ActionDenied)
}) })
@@ -647,7 +647,7 @@ func TestBanResponseCloseIsAnAnswerOfItsOwn(t *testing.T) {
}) })
got := s.ask(client, "/") got := s.ask(client, "/")
wantDisposition(t, got, http.StatusOK, proxy.DispositionClose) wantDisposition(t, got, http.StatusOK, servewaf.DispositionClose)
wantBody(t, got, "") wantBody(t, got, "")
// The visitor is sent nothing. // The visitor is sent nothing.
wantLine(t, s.out.requestLine(t), 0, requestlog.ActionDenied) wantLine(t, s.out.requestLine(t), 0, requestlog.ActionDenied)
@@ -664,12 +664,12 @@ func TestBlockRuleIsDeniedWith403AndABanRuleWithTheBanResponse(t *testing.T) {
}) })
got := s.ask(client, "/blocked") got := s.ask(client, "/blocked")
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
wantLine(t, s.out.requestLine(t), http.StatusForbidden, wantLine(t, s.out.requestLine(t), http.StatusForbidden,
requestlog.ActionRuleBlocked) requestlog.ActionRuleBlocked)
got = s.ask(client, "/.env") got = s.ask(client, "/.env")
wantDisposition(t, got, http.StatusTooManyRequests, proxy.DispositionDeny) wantDisposition(t, got, http.StatusTooManyRequests, servewaf.DispositionDeny)
wantLine(t, s.out.requestLines(t, 2)[1], http.StatusTooManyRequests, wantLine(t, s.out.requestLines(t, 2)[1], http.StatusTooManyRequests,
requestlog.ActionBanned) requestlog.ActionBanned)
} }
@@ -681,15 +681,17 @@ func TestQuestionUnderTheOwnPrefixIsDeniedWith404AfterTheChecks(t *testing.T) {
metricsToken: token, denyNets: otherClient, metricsToken: token, denyNets: otherClient,
}) })
for _, path := range []string{proxy.HealthPath, proxy.MetricsPath, proxy.ReportPath} { for _, path := range []string{
servewaf.HealthPath, servewaf.MetricsPath, servewaf.ReportPath,
} {
got := s.ask(client, path) got := s.ask(client, path)
wantDisposition(t, got, http.StatusNotFound, proxy.DispositionDeny) wantDisposition(t, got, http.StatusNotFound, servewaf.DispositionDeny)
} }
wantLine(t, s.out.requestLine(t), http.StatusNotFound, requestlog.ActionAdmin) wantLine(t, s.out.requestLine(t), http.StatusNotFound, requestlog.ActionAdmin)
got := s.ask(otherClient, proxy.MetricsPath) got := s.ask(otherClient, servewaf.MetricsPath)
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
} }
func TestObserveModeAllowsAndLogsWhatEnforceModeWouldHaveDone(t *testing.T) { func TestObserveModeAllowsAndLogsWhatEnforceModeWouldHaveDone(t *testing.T) {
@@ -698,7 +700,7 @@ func TestObserveModeAllowsAndLogsWhatEnforceModeWouldHaveDone(t *testing.T) {
s := startServeWAF(t, "", map[string]string{mode: observe, denyNets: client}) s := startServeWAF(t, "", map[string]string{mode: observe, denyNets: client})
got := s.ask(client, "/") got := s.ask(client, "/")
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
line := s.out.requestLine(t) line := s.out.requestLine(t)
wantLine(t, line, http.StatusOK, requestlog.ActionForward) wantLine(t, line, http.StatusOK, requestlog.ActionForward)
@@ -714,7 +716,7 @@ func TestObserveModeRefusalAnsweredAllowCountsForTheErrorBurst(t *testing.T) {
for range 2 { for range 2 {
got := s.ask(client, "/blocked") got := s.ask(client, "/blocked")
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
} }
lines := s.out.requestLines(t, 2) lines := s.out.requestLines(t, 2)
@@ -733,7 +735,7 @@ func TestDeniedQuestionCountsForTheErrorBurst(t *testing.T) {
for range 2 { for range 2 {
got := s.ask(client, "/blocked") got := s.ask(client, "/blocked")
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
} }
lines := s.out.requestLines(t, 2) lines := s.out.requestLines(t, 2)
@@ -762,7 +764,7 @@ func TestAllowCarriesTheLookupHeadersWhileTheSettingIsSet(t *testing.T) {
}) })
got := s.ask(fromDE, "/") got := s.ask(fromDE, "/")
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country") asn, country := got.header.Get("X-Client-Asn"), got.header.Get("X-Client-Country")
if asn != tc.wantASN || country != tc.wantCountry { if asn != tc.wantASN || country != tc.wantCountry {
@@ -787,7 +789,7 @@ func TestReportCountsTowardAByteLimitWhichThenBans(t *testing.T) {
} }
// 100 bytes are over the limit of 99. // 100 bytes are over the limit of 99.
got := s.report(proxy.Report{ got := s.report(servewaf.Report{
RequestID: id, Status: http.StatusBadGateway, RequestID: id, Status: http.StatusBadGateway,
RequestBytes: bodyBytes, ResponseBytes: answerBytes, RequestBytes: bodyBytes, ResponseBytes: answerBytes,
}) })
@@ -811,7 +813,7 @@ func TestReportCountsTowardAByteLimitWhichThenBans(t *testing.T) {
} }
got = s.ask(client, "/") got = s.ask(client, "/")
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
} }
func TestReportCountsAtThePercentageItsQuestionWasGiven(t *testing.T) { func TestReportCountsAtThePercentageItsQuestionWasGiven(t *testing.T) {
@@ -825,10 +827,10 @@ func TestReportCountsAtThePercentageItsQuestionWasGiven(t *testing.T) {
}) })
s.passedOn(fromKP) s.passedOn(fromKP)
wantDisposition(t, s.ask(fromKP, "/"), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, s.ask(fromKP, "/"), http.StatusOK, servewaf.DispositionAllow)
s.passedOn(fromDE) s.passedOn(fromDE)
wantDisposition(t, s.ask(fromDE, "/"), http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, s.ask(fromDE, "/"), http.StatusForbidden, servewaf.DispositionDeny)
} }
func TestReportAddsTheRequestToItsClientsHistoryAndTheAnomalyCounters(t *testing.T) { func TestReportAddsTheRequestToItsClientsHistoryAndTheAnomalyCounters(t *testing.T) {
@@ -846,7 +848,7 @@ func TestReportAddsTheRequestToItsClientsHistoryAndTheAnomalyCounters(t *testing
t.Errorf("anomaly counters %v before the report, want none", counters) t.Errorf("anomaly counters %v before the report, want none", counters)
} }
got := s.report(proxy.Report{ got := s.report(servewaf.Report{
RequestID: id, Status: http.StatusBadGateway, RequestID: id, Status: http.StatusBadGateway,
RequestBytes: bodyBytes, ResponseBytes: answerBytes, RequestBytes: bodyBytes, ResponseBytes: answerBytes,
}) })
@@ -874,7 +876,7 @@ func TestDeniedRequestIsFinishedAtItsDecision(t *testing.T) {
s.allowed(client) s.allowed(client)
denied := s.question(siteA, client, "/") denied := s.question(siteA, client, "/")
wantDisposition(t, do(t, denied), http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, do(t, denied), http.StatusForbidden, servewaf.DispositionDeny)
counters := s.server.Anomalies.Snapshot() counters := s.server.Anomalies.Snapshot()
if len(counters) != 1 || counters[0].Minute.Current != 1 { if len(counters) != 1 || counters[0].Minute.Current != 1 {
@@ -891,7 +893,7 @@ func TestDeniedRequestIsFinishedAtItsDecision(t *testing.T) {
wantMetric(t, s.metrics(), series, 1) wantMetric(t, s.metrics(), series, 1)
got := s.report(proxy.Report{ got := s.report(servewaf.Report{
RequestID: denied.Header.Get(requestIDHeader), Status: http.StatusForbidden, RequestID: denied.Header.Get(requestIDHeader), Status: http.StatusForbidden,
ResponseBytes: 1000, DurationTotal: 2, ResponseBytes: 1000, DurationTotal: 2,
}) })
@@ -924,7 +926,7 @@ func TestMetricsCountAnAllowedRequestFromItsReport(t *testing.T) {
wantMetric(t, metrics, `smallwebwaf_requests_in_flight{instance="`+alertInstance+`"}`, wantMetric(t, metrics, `smallwebwaf_requests_in_flight{instance="`+alertInstance+`"}`,
1) // the scrape itself 1) // the scrape itself
got := s.report(proxy.Report{ got := s.report(servewaf.Report{
RequestID: id, Status: http.StatusBadGateway, RequestID: id, Status: http.StatusBadGateway,
RequestBytes: bodyBytes, ResponseBytes: answerBytes, RequestBytes: bodyBytes, ResponseBytes: answerBytes,
DurationTotal: 5, DurationUpstreamTotal: 4, DurationTotal: 5, DurationUpstreamTotal: 4,
@@ -942,7 +944,7 @@ func TestMetricsCountAnAllowedRequestFromItsReport(t *testing.T) {
// check needs a server that has not been scraped yet. // check needs a server that has not been scraped yet.
unscraped := startServeWAF(t, "", map[string]string{metricsToken: token}) unscraped := startServeWAF(t, "", map[string]string{metricsToken: token})
got = unscraped.report(proxy.Report{ got = unscraped.report(servewaf.Report{
RequestID: unscraped.allowed(client), Status: http.StatusBadGateway, DurationTotal: 5, RequestID: unscraped.allowed(client), Status: http.StatusBadGateway, DurationTotal: 5,
}) })
wantStatus(t, got, http.StatusNoContent) wantStatus(t, got, http.StatusNoContent)
@@ -955,7 +957,7 @@ func TestReportForARequestNotHeldIsIgnored(t *testing.T) {
s := startServeWAF(t, "", map[string]string{bytesLimitPerMinute: byteLimit}) s := startServeWAF(t, "", map[string]string{bytesLimitPerMinute: byteLimit})
got := s.report(proxy.Report{RequestID: "no-such-request", ResponseBytes: 1000}) got := s.report(servewaf.Report{RequestID: "no-such-request", ResponseBytes: 1000})
wantStatus(t, got, http.StatusNoContent) wantStatus(t, got, http.StatusNoContent)
if clients := s.server.Limiter.Snapshot(); len(clients) != 0 { if clients := s.server.Limiter.Snapshot(); len(clients) != 0 {
@@ -972,7 +974,7 @@ func TestReportForARequestNotHeldIsIgnored(t *testing.T) {
// A second report of the same request does not count it twice. // A second report of the same request does not count it twice.
id := s.allowed(client) id := s.allowed(client)
for range 2 { for range 2 {
got = s.report(proxy.Report{RequestID: id, ResponseBytes: answerBytes}) got = s.report(servewaf.Report{RequestID: id, ResponseBytes: answerBytes})
wantStatus(t, got, http.StatusNoContent) wantStatus(t, got, http.StatusNoContent)
} }
@@ -1160,7 +1162,7 @@ func TestBytesAtTheirMostEachWayBanRatherThanOverflow(t *testing.T) {
s := startServeWAF(t, "", map[string]string{bytesLimitPerMinute: byteLimit}) s := startServeWAF(t, "", map[string]string{bytesLimitPerMinute: byteLimit})
got := s.report(proxy.Report{ got := s.report(servewaf.Report{
RequestID: s.allowed(client), Status: http.StatusOK, RequestID: s.allowed(client), Status: http.StatusOK,
RequestBytes: reportedBytesLimit, ResponseBytes: reportedBytesLimit, RequestBytes: reportedBytesLimit, ResponseBytes: reportedBytesLimit,
}) })
@@ -1245,9 +1247,9 @@ func TestReportWritesTheLastLineWithTheDecisionsAndTheReportsFields(t *testing.T
req.Header.Set("X-Forwarded-Proto", "https") req.Header.Set("X-Forwarded-Proto", "https")
id := req.Header.Get(requestIDHeader) id := req.Header.Get(requestIDHeader)
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
got := s.report(proxy.Report{ got := s.report(servewaf.Report{
RequestID: id, Status: http.StatusCreated, RequestID: id, Status: http.StatusCreated,
RequestBytes: bodyBytes, ResponseBytes: answerBytes, RequestBytes: bodyBytes, ResponseBytes: answerBytes,
DurationTotal: 12.5, DurationUpstreamTotal: 10.25, DurationTotal: 12.5, DurationUpstreamTotal: 10.25,
@@ -1294,7 +1296,7 @@ func TestEveryDeniedRequestsReportGetsItsLastLine(t *testing.T) {
"close", map[string]string{denyNets: client, banResponse: banResponseClose}, "/", "close", map[string]string{denyNets: client, banResponse: banResponseClose}, "/",
0, requestlog.ActionDenied, 0, requestlog.ActionDenied,
}, },
{"own prefix", nil, proxy.HealthPath, http.StatusNotFound, requestlog.ActionAdmin}, {"own prefix", nil, servewaf.HealthPath, http.StatusNotFound, requestlog.ActionAdmin},
} { } {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1305,7 +1307,7 @@ func TestEveryDeniedRequestsReportGetsItsLastLine(t *testing.T) {
id := req.Header.Get(requestIDHeader) id := req.Header.Get(requestIDHeader)
do(t, req) do(t, req)
got := s.report(proxy.Report{ got := s.report(servewaf.Report{
RequestID: id, Status: tc.status, ResponseBytes: 8, DurationTotal: 3, RequestID: id, Status: tc.status, ResponseBytes: 8, DurationTotal: 3,
}) })
wantStatus(t, got, http.StatusNoContent) wantStatus(t, got, http.StatusNoContent)
@@ -1341,18 +1343,18 @@ func TestOldestAnsweredRequestIsDroppedPastTheBound(t *testing.T) {
t.Fatalf("question: %v", err) t.Fatalf("question: %v", err)
} }
wantDisposition(t, readAnswer(res), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, readAnswer(res), http.StatusOK, servewaf.DispositionAllow)
} }
// The table is full again with next and those, without oldest. // The table is full again with next and those, without oldest.
got := s.report(proxy.Report{RequestID: oldest, ResponseBytes: answerBytes}) got := s.report(servewaf.Report{RequestID: oldest, ResponseBytes: answerBytes})
wantStatus(t, got, http.StatusNoContent) wantStatus(t, got, http.StatusNoContent)
if counted := s.history().Requests; counted != 0 { if counted := s.history().Requests; counted != 0 {
t.Errorf("history holds %d requests, want the dropped one not counted", counted) t.Errorf("history holds %d requests, want the dropped one not counted", counted)
} }
got = s.report(proxy.Report{RequestID: next, ResponseBytes: answerBytes}) got = s.report(servewaf.Report{RequestID: next, ResponseBytes: answerBytes})
wantStatus(t, got, http.StatusNoContent) wantStatus(t, got, http.StatusNoContent)
if counted := s.history().Requests; counted != 1 { if counted := s.history().Requests; counted != 1 {
@@ -1366,10 +1368,10 @@ func TestBanMadeThroughOneHostRefusesTheClientOnAnother(t *testing.T) {
s := startServeWAF(t, "", map[string]string{rulesDir: writeRules(t, testRules)}) s := startServeWAF(t, "", map[string]string{rulesDir: writeRules(t, testRules)})
got := do(t, s.question(siteA, client, "/.env")) got := do(t, s.question(siteA, client, "/.env"))
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
got = do(t, s.question(siteB, client, "/")) got = do(t, s.question(siteB, client, "/"))
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
lines := s.out.requestLines(t, 2) lines := s.out.requestLines(t, 2)
if lines[0].Host != siteA || lines[1].Host != siteB { if lines[0].Host != siteA || lines[1].Host != siteB {
@@ -1395,7 +1397,7 @@ func TestRequestsOnEverySiteCountTowardOneSetOfLimits(t *testing.T) {
s.allowed(client) s.allowed(client)
got := do(t, s.question(siteB, client, "/")) got := do(t, s.question(siteB, client, "/"))
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
} }
func TestOwnEndpointsAnswerOnlyWithoutTheClientHeader(t *testing.T) { func TestOwnEndpointsAnswerOnlyWithoutTheClientHeader(t *testing.T) {
@@ -1410,13 +1412,13 @@ func TestOwnEndpointsAnswerOnlyWithoutTheClientHeader(t *testing.T) {
path, authorization string path, authorization string
want int want int
}{ }{
{proxy.HealthPath, "", http.StatusOK}, {servewaf.HealthPath, "", http.StatusOK},
{proxy.MetricsPath, bearer, http.StatusOK}, {servewaf.MetricsPath, bearer, http.StatusOK},
{proxy.MetricsPath, "", http.StatusUnauthorized}, {servewaf.MetricsPath, "", http.StatusUnauthorized},
{proxy.BansPath, adminBearer, http.StatusOK}, {servewaf.BansPath, adminBearer, http.StatusOK},
{proxy.BansPath, "", http.StatusUnauthorized}, {servewaf.BansPath, "", http.StatusUnauthorized},
{proxy.ClientsPath + client, adminBearer, http.StatusOK}, {servewaf.ClientsPath + client, adminBearer, http.StatusOK},
{proxy.ReportPath, "", http.StatusNotFound}, {servewaf.ReportPath, "", http.StatusNotFound},
{"/_smallwebwaf/other", "", http.StatusNotFound}, {"/_smallwebwaf/other", "", http.StatusNotFound},
{"/", "", http.StatusNotFound}, {"/", "", http.StatusNotFound},
} { } {
@@ -1429,9 +1431,9 @@ func TestOwnEndpointsAnswerOnlyWithoutTheClientHeader(t *testing.T) {
req.Header.Set("Authorization", tc.authorization) req.Header.Set("Authorization", tc.authorization)
if strings.HasPrefix(tc.path, "/_smallwebwaf/") { if strings.HasPrefix(tc.path, "/_smallwebwaf/") {
wantDisposition(t, do(t, req), http.StatusNotFound, proxy.DispositionDeny) wantDisposition(t, do(t, req), http.StatusNotFound, servewaf.DispositionDeny)
} else { } else {
wantDisposition(t, do(t, req), http.StatusOK, proxy.DispositionAllow) wantDisposition(t, do(t, req), http.StatusOK, servewaf.DispositionAllow)
} }
} }
} }
@@ -1441,10 +1443,11 @@ func TestOwnEndpointsLinesDoNotSayWhichOfTheTwoTheyAre(t *testing.T) {
s := startServeWAF(t, "", nil) s := startServeWAF(t, "", nil)
wantStatus(t, get(t, s.addr, proxy.HealthPath), http.StatusOK) wantStatus(t, get(t, s.addr, servewaf.HealthPath), http.StatusOK)
line := s.out.requestLine(t) line := s.out.requestLine(t)
if _, there := line.fields["servewaf_line"]; there || line.Path != proxy.HealthPath { if _, there := line.fields["servewaf_line"]; there ||
line.Path != servewaf.HealthPath {
t.Errorf("the health check's log line is %+v", line.fields) t.Errorf("the health check's log line is %+v", line.fields)
} }
} }
@@ -1456,12 +1459,12 @@ func TestALargeBodyAnnouncedAndLongHeadersAreNoReasonToRefuse(t *testing.T) {
req := s.questionWith(http.MethodPost, siteA, client, "/", req := s.questionWith(http.MethodPost, siteA, client, "/",
strings.NewReader("a body")) strings.NewReader("a body"))
req.Header.Set(proxy.ContentLengthHeader, "1073741824") req.Header.Set(servewaf.ContentLengthHeader, "1073741824")
req.Header.Set(forwardedFor, otherClient) req.Header.Set(forwardedFor, otherClient)
req.Header.Set("X-Long", strings.Repeat("x", 40<<10)) req.Header.Set("X-Long", strings.Repeat("x", 40<<10))
got := do(t, req) got := do(t, req)
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
line := s.out.requestLine(t) line := s.out.requestLine(t)
if line.ClientIP != client || line.ContentLength != 1<<30 { if line.ClientIP != client || line.ContentLength != 1<<30 {
@@ -1529,7 +1532,7 @@ carried header:Content-Length block ^6$
strings.NewReader("a body")) strings.NewReader("a body"))
if tc.announced != "" { if tc.announced != "" {
req.Header.Set(proxy.ContentLengthHeader, tc.announced) req.Header.Set(servewaf.ContentLengthHeader, tc.announced)
} }
wantStatus(t, do(t, req), tc.want) wantStatus(t, do(t, req), tc.want)
@@ -1547,19 +1550,19 @@ func TestCoreRuleSetReadsTheBodyAQuestionCarries(t *testing.T) {
req := s.questionWith(http.MethodPost, siteA, client, "/", req := s.questionWith(http.MethodPost, siteA, client, "/",
strings.NewReader(body)) strings.NewReader(body))
req.Header.Set("Content-Type", contentType) req.Header.Set("Content-Type", contentType)
req.Header.Set(proxy.ContentLengthHeader, "100000") req.Header.Set(servewaf.ContentLengthHeader, "100000")
return do(t, req) return do(t, req)
} }
got := ask(formData, "id=1'%20OR%20'1'='1") got := ask(formData, "id=1'%20OR%20'1'='1")
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
// The first 64 KiB of a longer JSON body cannot be parsed, which is no // The first 64 KiB of a longer JSON body cannot be parsed, which is no
// attack, so the Core Rule Set does not read it. // attack, so the Core Rule Set does not read it.
cut := `{"text":"` + strings.Repeat("x", proxy.QuestionBodyMaxBytes) cut := `{"text":"` + strings.Repeat("x", servewaf.QuestionBodyMaxBytes)
got = ask("application/json", cut[:proxy.QuestionBodyMaxBytes]) got = ask("application/json", cut[:servewaf.QuestionBodyMaxBytes])
wantDisposition(t, got, http.StatusOK, proxy.DispositionAllow) wantDisposition(t, got, http.StatusOK, servewaf.DispositionAllow)
} }
func TestCoreRuleSetAlertNamesTheSite(t *testing.T) { func TestCoreRuleSetAlertNamesTheSite(t *testing.T) {
@@ -1568,7 +1571,7 @@ func TestCoreRuleSetAlertNamesTheSite(t *testing.T) {
s := startServeWAF(t, "", map[string]string{wafMode: block}) s := startServeWAF(t, "", map[string]string{wafMode: block})
got := s.ask(client, sqlInjection) got := s.ask(client, sqlInjection)
wantDisposition(t, got, http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, got, http.StatusForbidden, servewaf.DispositionDeny)
waiting := s.queue.Snapshot().Waiting[alerts.DestinationWebhook] waiting := s.queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || waiting[0].Event != alerts.EventWAFBlock || if len(waiting) != 1 || waiting[0].Event != alerts.EventWAFBlock ||
@@ -1583,7 +1586,7 @@ func TestReputationHitAlertNamesTheSite(t *testing.T) {
s := startServeWAF(t, "", map[string]string{blocklistURLs: dropURL}) s := startServeWAF(t, "", map[string]string{blocklistURLs: dropURL})
loadLists(t, s.server, map[string][]string{dropURL: {client}}) loadLists(t, s.server, map[string][]string{dropURL: {client}})
wantDisposition(t, s.ask(client, "/"), http.StatusForbidden, proxy.DispositionDeny) wantDisposition(t, s.ask(client, "/"), http.StatusForbidden, servewaf.DispositionDeny)
waiting := s.queue.Snapshot().Waiting[alerts.DestinationWebhook] waiting := s.queue.Snapshot().Waiting[alerts.DestinationWebhook]
if len(waiting) != 1 || waiting[0].Event != alerts.EventReputationHit || if len(waiting) != 1 || waiting[0].Event != alerts.EventReputationHit ||
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"net/http" "net/http"
@@ -1,4 +1,4 @@
package proxy_test package servewaf_test
import ( import (
"net/http" "net/http"
+2 -2
View File
@@ -11,7 +11,7 @@ import (
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/proxy" "sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
// healthCheckTimeout bounds the whole health check. // healthCheckTimeout bounds the whole health check.
@@ -107,7 +107,7 @@ func checkRProxyAndApp(
// listenAddr, which the settings have checked has one. // listenAddr, which the settings have checked has one.
func askHealthEndpoint(ctx context.Context, listenAddr string) error { func askHealthEndpoint(ctx context.Context, listenAddr string) error {
_, port, _ := net.SplitHostPort(listenAddr) _, port, _ := net.SplitHostPort(listenAddr)
health := "http://" + net.JoinHostPort("127.0.0.1", port) + proxy.HealthPath health := "http://" + net.JoinHostPort("127.0.0.1", port) + servewaf.HealthPath
req, err := http.NewRequestWithContext(ctx, http.MethodGet, health, http.NoBody) req, err := http.NewRequestWithContext(ctx, http.MethodGet, health, http.NoBody)
if err != nil { if err != nil {
+10 -10
View File
@@ -19,7 +19,7 @@ import (
"testing" "testing"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/proxy" "sneak.berlin/go/smallwebwaf/internal/servewaf"
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf" "sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
) )
@@ -153,7 +153,7 @@ func TestABanThroughOneSiteRefusesTheClientOnTheOther(t *testing.T) {
// One request's three lines carry one id, rproxy's with servewaf's answer. // One request's three lines carry one id, rproxy's with servewaf's answer.
line := firstOut.line(t, "path", bannedProbe) line := firstOut.line(t, "path", bannedProbe)
if line["disposition"] != proxy.DispositionDeny || line["client_ip"] != placed { if line["disposition"] != servewaf.DispositionDeny || line["client_ip"] != placed {
t.Errorf("rproxy's line is %v", line) t.Errorf("rproxy's line is %v", line)
} }
@@ -225,7 +225,7 @@ func TestTheSiteAnswersWhileServeWAFIsStoppedAndStartedAgain(t *testing.T) {
wantStatus(t, site+"/after-stop", placed, http.StatusOK) wantStatus(t, site+"/after-stop", placed, http.StatusOK)
line := out.line(t, "path", "/after-stop") line := out.line(t, "path", "/after-stop")
if line["disposition"] != proxy.DispositionAllow || if line["disposition"] != servewaf.DispositionAllow ||
line["default_disposition"] != true { line["default_disposition"] != true {
t.Errorf("rproxy's line is %v", line) t.Errorf("rproxy's line is %v", line)
} }
@@ -318,7 +318,7 @@ func TestAChunkedBodyStaysABodyAtServeWAF(t *testing.T) {
const smuggledID = "smuggled-request" const smuggledID = "smuggled-request"
smuggled := "GET " + bannedProbe + " HTTP/1.1\r\nHost: site.example\r\n" + smuggled := "GET " + bannedProbe + " HTTP/1.1\r\nHost: site.example\r\n" +
proxy.ClientHeader + ": " + otherClient + "\r\n" + servewaf.ClientHeader + ": " + otherClient + "\r\n" +
"X-Request-ID: " + smuggledID + "\r\n\r\n" "X-Request-ID: " + smuggledID + "\r\n\r\n"
status := sendRaw(t, site, "POST /upload HTTP/1.1\r\nHost: site.example\r\n"+ status := sendRaw(t, site, "POST /upload HTTP/1.1\r\nHost: site.example\r\n"+
@@ -459,10 +459,10 @@ func TestRProxyAsksAServeWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
protocols := make(chan string, 10) protocols := make(chan string, 10)
backend := httptest.NewUnstartedServer(http.HandlerFunc( backend := httptest.NewUnstartedServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) { func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get(proxy.ClientHeader) != "" { if r.Header.Get(servewaf.ClientHeader) != "" {
protocols <- r.Proto protocols <- r.Proto
w.Header().Set(proxy.DispositionHeader, proxy.DispositionDeny) w.Header().Set(servewaf.DispositionHeader, servewaf.DispositionDeny)
w.WriteHeader(http.StatusForbidden) w.WriteHeader(http.StatusForbidden)
} }
})) }))
@@ -532,8 +532,8 @@ func TestRProxyStoppingWaitsForAReportBeingSent(t *testing.T) {
release := make(chan struct{}) release := make(chan struct{})
backend := httptest.NewServer(http.HandlerFunc( backend := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) { func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get(proxy.ClientHeader) != "" { if r.Header.Get(servewaf.ClientHeader) != "" {
w.Header().Set(proxy.DispositionHeader, proxy.DispositionAllow) w.Header().Set(servewaf.DispositionHeader, servewaf.DispositionAllow)
return return
} }
@@ -640,7 +640,7 @@ func TestRProxyStoppingCutsOffARequestAfterFiveSeconds(t *testing.T) {
backend := httptest.NewServer(http.HandlerFunc( backend := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, _ *http.Request) { func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(proxy.DispositionHeader, proxy.DispositionAllow) w.Header().Set(servewaf.DispositionHeader, servewaf.DispositionAllow)
})) }))
t.Cleanup(backend.Close) t.Cleanup(backend.Close)
@@ -757,7 +757,7 @@ func TestMainRunsRProxy(t *testing.T) {
addr, cmd, out := startRProxyProcess(t, wafBackend+"=http://127.0.0.1:9") addr, cmd, out := startRProxyProcess(t, wafBackend+"=http://127.0.0.1:9")
req, err := http.NewRequestWithContext(t.Context(), http.MethodGet, req, err := http.NewRequestWithContext(t.Context(), http.MethodGet,
"http://"+addr+proxy.HealthPath, http.NoBody) "http://"+addr+servewaf.HealthPath, http.NoBody)
if err != nil { if err != nil {
t.Fatalf("new request: %v", err) t.Fatalf("new request: %v", err)
} }
+8 -8
View File
@@ -14,7 +14,7 @@ import (
"testing" "testing"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/proxy" "sneak.berlin/go/smallwebwaf/internal/servewaf"
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf" "sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
) )
@@ -67,7 +67,7 @@ func TestMainRunsServeWAF(t *testing.T) {
}) })
addr, _ := out.line(t, "msg", "starting")["address"].(string) addr, _ := out.line(t, "msg", "starting")["address"].(string)
ask(t, addr, "/", http.StatusOK, proxy.DispositionAllow) ask(t, addr, "/", http.StatusOK, servewaf.DispositionAllow)
err = cmd.Process.Signal(syscall.SIGTERM) err = cmd.Process.Signal(syscall.SIGTERM)
if err != nil { if err != nil {
@@ -132,7 +132,7 @@ func ask(t *testing.T, addr, path string, status int, disposition string) {
t.Fatalf("new request: %v", err) t.Fatalf("new request: %v", err)
} }
req.Header.Set(proxy.ClientHeader, placed) req.Header.Set(servewaf.ClientHeader, placed)
req.Header.Set("X-Request-ID", rand.Text()) req.Header.Set("X-Request-ID", rand.Text())
transport := &http.Transport{} transport := &http.Transport{}
@@ -147,9 +147,9 @@ func ask(t *testing.T, addr, path string, status int, disposition string) {
_ = res.Body.Close() _ = res.Body.Close()
if res.StatusCode != status || if res.StatusCode != status ||
res.Header.Get(proxy.DispositionHeader) != disposition { res.Header.Get(servewaf.DispositionHeader) != disposition {
t.Errorf("answered %d with disposition %q, want %d with %q", res.StatusCode, t.Errorf("answered %d with disposition %q, want %d with %q", res.StatusCode,
res.Header.Get(proxy.DispositionHeader), status, disposition) res.Header.Get(servewaf.DispositionHeader), status, disposition)
} }
} }
@@ -193,12 +193,12 @@ func TestServeWAFKeepsItsBansAcrossARestartAndNeedsNoApp(t *testing.T) {
} }
runServeWAF(t, env, func(addr string) { runServeWAF(t, env, func(addr string) {
ask(t, addr, "/", http.StatusOK, proxy.DispositionAllow) ask(t, addr, "/", http.StatusOK, servewaf.DispositionAllow)
// The default rule file bans a probe for this. // The default rule file bans a probe for this.
ask(t, addr, "/.env", http.StatusForbidden, proxy.DispositionDeny) ask(t, addr, "/.env", http.StatusForbidden, servewaf.DispositionDeny)
}) })
runServeWAF(t, env, func(addr string) { runServeWAF(t, env, func(addr string) {
ask(t, addr, "/", http.StatusForbidden, proxy.DispositionDeny) ask(t, addr, "/", http.StatusForbidden, servewaf.DispositionDeny)
}) })
} }
+17 -8
View File
@@ -23,12 +23,13 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup" "sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/remotelog" "sneak.berlin/go/smallwebwaf/internal/remotelog"
"sneak.berlin/go/smallwebwaf/internal/reputation" "sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rules" "sneak.berlin/go/smallwebwaf/internal/rules"
"sneak.berlin/go/smallwebwaf/internal/servewaf"
"sneak.berlin/go/smallwebwaf/internal/state" "sneak.berlin/go/smallwebwaf/internal/state"
"sneak.berlin/go/smallwebwaf/internal/waf"
) )
// shutdownTimeout is how long requests in progress may take to finish // shutdownTimeout is how long requests in progress may take to finish
@@ -150,7 +151,14 @@ func RunServeWAF(ctx context.Context, params Params) int {
return 1 return 1
} }
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue) crs, err := servewaf.NewCoreRuleSet(cfg)
if err != nil {
processLog.Error("cannot use the Core Rule Set", "error", err.Error())
return 1
}
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue, crs)
if err != nil { if err != nil {
processLog.Error("cannot use the lookup database", "error", err.Error()) processLog.Error("cannot use the lookup database", "error", err.Error())
@@ -170,8 +178,7 @@ func RunServeWAF(ctx context.Context, params Params) int {
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr) listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
if err != nil { if err != nil {
processLog.Error("cannot listen on SWWAF_LISTEN_ADDR", processLog.Error("cannot listen on SWWAF_LISTEN_ADDR", "error", err.Error())
"error", err.Error())
return 1 return 1
} }
@@ -190,7 +197,8 @@ func RunServeWAF(ctx context.Context, params Params) int {
func newServer( func newServer(
cfg *config.Config, stdout io.Writer, processLog *slog.Logger, cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue, now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
) (*proxy.Server, error) { crs *waf.CoreRuleSet,
) (*servewaf.Server, error) {
var lookupFile *lookup.File var lookupFile *lookup.File
if cfg.LookupSource == "file" { if cfg.LookupSource == "file" {
@@ -207,13 +215,14 @@ func newServer(
} }
} }
server := proxy.NewServeWAF(proxy.Params{ server := servewaf.New(servewaf.Params{
Config: cfg, Config: cfg,
RequestLog: stdout, RequestLog: stdout,
ProcessLog: processLog, ProcessLog: processLog,
GeoJSURL: lookup.URL, GeoJSURL: lookup.URL,
AbuseIPDBURL: reputation.AbuseIPDBURL, AbuseIPDBURL: reputation.AbuseIPDBURL,
LookupFile: lookupFile, LookupFile: lookupFile,
CoreRuleSet: crs,
Now: now, Now: now,
Rules: ruleFiles, Rules: ruleFiles,
Alerts: alertQueue, Alerts: alertQueue,
@@ -230,7 +239,7 @@ func newServer(
// loadStateFiles reads the state files into the parts of server and into // loadStateFiles reads the state files into the parts of server and into
// alertQueue, as state.Load does. // alertQueue, as state.Load does.
func loadStateFiles( func loadStateFiles(
cfg *config.Config, server *proxy.Server, alertQueue *alerts.Queue, cfg *config.Config, server *servewaf.Server, alertQueue *alerts.Queue,
now func() time.Time, processLog *slog.Logger, now func() time.Time, processLog *slog.Logger,
) (*state.Files, error) { ) (*state.Files, error) {
return state.Load(state.Params{ return state.Load(state.Params{
@@ -316,7 +325,7 @@ func startSending(
// shutdownTimeout to finish, and writes every state file, alerts.json with // shutdownTimeout to finish, and writes every state file, alerts.json with
// the alerts still waiting. // the alerts still waiting.
func serve( func serve(
ctx context.Context, server *proxy.Server, listener net.Listener, ctx context.Context, server *servewaf.Server, listener net.Listener,
files *state.Files, ruleFiles *rules.Files, alertQueue *alerts.Queue, files *state.Files, ruleFiles *rules.Files, alertQueue *alerts.Queue,
processLog *slog.Logger, processLog *slog.Logger,
) int { ) int {
+56 -5
View File
@@ -19,8 +19,9 @@ import (
"testing" "testing"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest" "sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
"sneak.berlin/go/smallwebwaf/internal/proxy" "sneak.berlin/go/smallwebwaf/internal/servewaf"
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf" "sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
) )
@@ -551,6 +552,56 @@ func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
": no such file or directory") ": no such file or directory")
} }
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "crs-exclusions.conf")
out := &output{}
status := run(t.Context(), map[string]string{
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
"SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
}, out)
if status != 1 {
t.Fatalf("exit status %d, want 1", status)
}
line := out.line(t, "msg", "cannot use the Core Rule Set")
want := "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: read the exclusions file: open " +
path + ": no such file or directory"
if line["error"] != want {
t.Errorf("start refused with %v, want the error %q", line, want)
}
}
func TestCoreRuleSetExclusionsFileThatCannotBeUsedStopsTheStartWhileTheCoreRuleSetIsOff(
t *testing.T,
) {
t.Parallel()
broken := filepath.Join(t.TempDir(), "broken.conf")
err := os.WriteFile(broken, []byte("SecNoSuchDirective On\n"), 0o600)
if err != nil {
t.Fatal(err)
}
for _, path := range []string{filepath.Join(t.TempDir(), "missing.conf"), broken} {
out := &output{}
status := run(t.Context(), map[string]string{
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
"SWWAF_WAF_MODE": config.WAFModeOff, "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
}, out)
if status != 1 {
t.Fatalf("%s: exit status %d, want 1", path, status)
}
out.line(t, "msg", "cannot use the Core Rule Set")
}
}
func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) { func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1225,9 +1276,9 @@ func wantStartingLine(t *testing.T, line map[string]any, dir string) {
"SWWAF_ALLOW_NETS": "", "SWWAF_ALLOW_NETS": "",
rateLimitExemptNets: "", rateLimitExemptNets: "",
"SWWAF_DENY_NETS": "", "SWWAF_DENY_NETS": "",
"SWWAF_RATE_LIMIT_PER_MINUTE": "500", "SWWAF_RATE_LIMIT_PER_MINUTE": "120",
"SWWAF_RATE_LIMIT_PER_HOUR": "20000", "SWWAF_RATE_LIMIT_PER_HOUR": "1500",
rateLimitPerDay: "300000", rateLimitPerDay: "6000",
"SWWAF_RATE_LIMIT_EXEMPT_PATHS": "", "SWWAF_RATE_LIMIT_EXEMPT_PATHS": "",
"SWWAF_DENIED_COUNTRIES": "", "SWWAF_DENIED_COUNTRIES": "",
"SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES": "", "SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES": "",
@@ -1627,7 +1678,7 @@ func statusFrom(t *testing.T, url, from string) int {
} }
req.Header.Set("X-Forwarded-For", from) req.Header.Set("X-Forwarded-For", from)
req.Header.Set(proxy.ClientHeader, from) req.Header.Set(servewaf.ClientHeader, from)
req.Header.Set("X-Request-ID", rand.Text()) req.Header.Set("X-Request-ID", rand.Text())
transport := &http.Transport{} transport := &http.Transport{}
+337
View File
@@ -0,0 +1,337 @@
# Ordinary gitea requests that the Core Rule Set must let through with
# deploy/gitea/crs-exclusions.conf. Each line is a path and query, and may end
# with one header after a space. The tests of internal/waf inspect each as a GET,
# and script/gitea-setup sends each to gitea through the setup as a GET. The Core
# Rule Set reads the query and the headers the same way for any method.
# Searches with ordinary words and symbols.
/owner/repo/search?q=package.json
/owner/repo/search?q=.gitignore
/owner/repo/search?q=docker-compose.yml
/owner/repo/search?q=python3
/owner/repo/search?q=ssh%20key
/owner/repo/search?q=feat%2Fdocker-support
/owner/repo/search?q=ls%20-la
/owner/repo/search?q=sed%20-i
/owner/repo/search?q=%2Fbin%2Fsh
/owner/repo/search?q=%60make%20check%60
/owner/repo/search?q=%60whoami%60
/owner/repo/search?q=fetch%28
/owner/repo/search?q=%24%7BVAR%7D
/owner/repo/search?q=%24%28pwd%29
/owner/repo/search?q=process.env
/owner/repo/search?q=%3Cimg%20src%3D
/owner/repo/search?q=%3Cscript%3E
/owner/repo/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/search?q=union%20select
/owner/repo/search?q=http%3A%2F%2F127.0.0.1%3A8080
/owner/repo/search?q=..%2F
/owner/repo/search?q=%2Fetc%2Fpasswd
/owner/repo/search?q=eval%28
/owner/repo/search?q=chmod%20%2Bx
/owner/repo/search?q=echo%20%24PATH
/owner/repo/search?q=java.lang.Runtime
/owner/repo/search?q=document.cookie
/owner/repo/search?q=alert%281%29
/owner/repo/search?q=__proto__
/owner/repo/search?q=C%3A%5CWindows%5CSystem32
/owner/repo/search?q=Invoke-WebRequest
/owner/repo/search?q=if%20%28a%20%3D%3D%20b%29%20%7B
/owner/repo/search?q=sudo%20apt%20install
/owner/repo/search?q=~%2F.ssh%2Fconfig
/owner/repo/search?q=.htaccess
/owner/repo/search?q=%24_GET%5B
/owner/repo/search?q=%24_POST
/owner/repo/search?q=%24_SERVER%5B%27REQUEST_URI%27%5D
/owner/repo/search?q=%3C%3Fphp%20echo
/owner/repo/search?q=information_schema
/owner/repo/search?q=LIKE%20%27%25foo%25%27
/owner/repo/search?q=onClick%3D%7B
/explore/repos?q=SELECT%20%2A%20FROM%20users%20WHERE
/explore/repos?q=%3Cimg%20src%3D
/explore/code?q=SELECT%20%2A%20FROM%20users%20WHERE
/explore/code?q=%3Cimg%20src%3D
/owner/repo/issues?type=all&state=open&q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/issues?type=all&state=open&q=%3Cimg%20src%3D
/owner/repo/pulls?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/pulls?q=%3Cimg%20src%3D
/issues?type=your_repositories&q=SELECT%20%2A%20FROM%20users%20WHERE
/issues?type=your_repositories&q=%3Cimg%20src%3D
/owner/repo/commits/branch/main/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/commits/branch/main/search?q=%3Cimg%20src%3D
/owner/repo/wiki/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/wiki/search?q=%3Cimg%20src%3D
/owner/repo/branches?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/branches?q=%3Cimg%20src%3D
/owner/repo/releases?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/releases?q=%3Cimg%20src%3D
/owner/-/packages?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/-/packages?q=%3Cimg%20src%3D
/user/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/user/search?q=%3Cimg%20src%3D
/api/v1/repos/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/api/v1/repos/search?q=%3Cimg%20src%3D
/api/v1/repos/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/api/v1/repos/issues/search?q=%3Cimg%20src%3D
/api/v1/repos/owner/repo/issues?q=SELECT%20%2A%20FROM%20users%20WHERE
/api/v1/repos/owner/repo/issues?q=%3Cimg%20src%3D
# Pages, raw files and edit pages of files, and the API's call for their
# contents.
/owner/repo/src/branch/main/package.json
/owner/repo/raw/branch/main/package.json
/owner/repo/_edit/main/package.json
/api/v1/repos/owner/repo/contents/package.json
/owner/repo/src/branch/main/.gitignore
/owner/repo/raw/branch/main/.gitignore
/owner/repo/_edit/main/.gitignore
/api/v1/repos/owner/repo/contents/.gitignore
/owner/repo/src/branch/main/docker-compose.yml
/owner/repo/raw/branch/main/docker-compose.yml
/owner/repo/_edit/main/docker-compose.yml
/api/v1/repos/owner/repo/contents/docker-compose.yml
/owner/repo/src/branch/main/Dockerfile
/owner/repo/raw/branch/main/Dockerfile
/owner/repo/_edit/main/Dockerfile
/api/v1/repos/owner/repo/contents/Dockerfile
/owner/repo/src/branch/main/.htaccess
/owner/repo/raw/branch/main/.htaccess
/owner/repo/_edit/main/.htaccess
/api/v1/repos/owner/repo/contents/.htaccess
/owner/repo/src/branch/main/rootfs/etc/passwd
/owner/repo/raw/branch/main/rootfs/etc/passwd
/owner/repo/_edit/main/rootfs/etc/passwd
/api/v1/repos/owner/repo/contents/rootfs/etc/passwd
/owner/repo/src/branch/main/bin/docker-entrypoint
/owner/repo/raw/branch/main/bin/docker-entrypoint
/owner/repo/_edit/main/bin/docker-entrypoint
/api/v1/repos/owner/repo/contents/bin/docker-entrypoint
/owner/repo/src/branch/main/scripts/python3-setup.sh
/owner/repo/raw/branch/main/scripts/python3-setup.sh
/owner/repo/_edit/main/scripts/python3-setup.sh
/api/v1/repos/owner/repo/contents/scripts/python3-setup.sh
/owner/repo/src/branch/main/src/java.base/share/classes/java/lang/Runtime.java
/owner/repo/raw/branch/main/src/java.base/share/classes/java/lang/Runtime.java
/owner/repo/_edit/main/src/java.base/share/classes/java/lang/Runtime.java
/api/v1/repos/owner/repo/contents/src/java.base/share/classes/java/lang/Runtime.java
/owner/repo/src/branch/main/java.lang.String.md
/owner/repo/raw/branch/main/java.lang.String.md
/owner/repo/_edit/main/java.lang.String.md
/api/v1/repos/owner/repo/contents/java.lang.String.md
/owner/repo/src/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
/owner/repo/raw/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
/owner/repo/_edit/main/META-INF/services/org.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/contents/META-INF/services/org.apache.commons.logging.LogFactory
/owner/repo/src/branch/main/process.env.d.ts
/owner/repo/raw/branch/main/process.env.d.ts
/owner/repo/_edit/main/process.env.d.ts
/api/v1/repos/owner/repo/contents/process.env.d.ts
/owner/repo/src/branch/main/document.write.js
/owner/repo/raw/branch/main/document.write.js
/owner/repo/_edit/main/document.write.js
/api/v1/repos/owner/repo/contents/document.write.js
/owner/repo/src/branch/main/test/fixtures/__proto__.json
/owner/repo/raw/branch/main/test/fixtures/__proto__.json
/owner/repo/_edit/main/test/fixtures/__proto__.json
/api/v1/repos/owner/repo/contents/test/fixtures/__proto__.json
/owner/repo/src/branch/main/install-script.sh
/owner/repo/raw/branch/main/install-script.sh
/owner/repo/_edit/main/install-script.sh
/api/v1/repos/owner/repo/contents/install-script.sh
/owner/repo/src/branch/main/src/backend/catalog/information_schema.sql
/owner/repo/raw/branch/main/src/backend/catalog/information_schema.sql
/owner/repo/_edit/main/src/backend/catalog/information_schema.sql
/api/v1/repos/owner/repo/contents/src/backend/catalog/information_schema.sql
/owner/repo/src/branch/main/docs/a%20b.xhtml
/owner/repo/raw/branch/main/docs/a%20b.xhtml
/owner/repo/_edit/main/docs/a%20b.xhtml
/api/v1/repos/owner/repo/contents/docs/a%20b.xhtml
/owner/repo/src/branch/main/%5Bdraft%5D.md
/owner/repo/raw/branch/main/%5Bdraft%5D.md
/owner/repo/_edit/main/%5Bdraft%5D.md
/api/v1/repos/owner/repo/contents/%5Bdraft%5D.md
/owner/repo/src/branch/main/notes.txt~
/owner/repo/raw/branch/main/notes.txt~
/owner/repo/_edit/main/notes.txt~
/api/v1/repos/owner/repo/contents/notes.txt~
# File names in the query string: the commit graph of a file, the API's lists
# of its commits and of its contents, an issue opened from a template, and the
# sign-in page on the way to its edit page.
/owner/repo/graph?file=package.json
/api/v1/repos/owner/repo/commits?path=package.json
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22package.json%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fpackage.json
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fpackage.json
/owner/repo/graph?file=.gitignore
/api/v1/repos/owner/repo/commits?path=.gitignore
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.gitignore%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.gitignore
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.gitignore
/owner/repo/graph?file=docker-compose.yml
/api/v1/repos/owner/repo/commits?path=docker-compose.yml
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docker-compose.yml%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocker-compose.yml
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocker-compose.yml
/owner/repo/graph?file=.htaccess
/api/v1/repos/owner/repo/commits?path=.htaccess
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.htaccess%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.htaccess
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.htaccess
/owner/repo/graph?file=rootfs%2Fetc%2Fpasswd
/api/v1/repos/owner/repo/commits?path=rootfs%2Fetc%2Fpasswd
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22rootfs%2Fetc%2Fpasswd%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Frootfs%2Fetc%2Fpasswd
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Frootfs%2Fetc%2Fpasswd
/owner/repo/graph?file=bin%2Fdocker-entrypoint
/api/v1/repos/owner/repo/commits?path=bin%2Fdocker-entrypoint
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22bin%2Fdocker-entrypoint%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fbin%2Fdocker-entrypoint
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fbin%2Fdocker-entrypoint
/owner/repo/graph?file=scripts%2Fpython3-setup.sh
/api/v1/repos/owner/repo/commits?path=scripts%2Fpython3-setup.sh
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22scripts%2Fpython3-setup.sh%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fscripts%2Fpython3-setup.sh
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fscripts%2Fpython3-setup.sh
/owner/repo/graph?file=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/api/v1/repos/owner/repo/commits?path=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/owner/repo/graph?file=java.lang.String.md
/api/v1/repos/owner/repo/commits?path=java.lang.String.md
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22java.lang.String.md%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fjava.lang.String.md
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fjava.lang.String.md
/owner/repo/graph?file=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/commits?path=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/owner/repo/graph?file=process.env.d.ts
/api/v1/repos/owner/repo/commits?path=process.env.d.ts
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22process.env.d.ts%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fprocess.env.d.ts
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fprocess.env.d.ts
/owner/repo/graph?file=document.write.js
/api/v1/repos/owner/repo/commits?path=document.write.js
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22document.write.js%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocument.write.js
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocument.write.js
/owner/repo/graph?file=test%2Ffixtures%2F__proto__.json
/api/v1/repos/owner/repo/commits?path=test%2Ffixtures%2F__proto__.json
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22test%2Ffixtures%2F__proto__.json%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Ftest%2Ffixtures%2F__proto__.json
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Ftest%2Ffixtures%2F__proto__.json
/owner/repo/graph?file=install-script.sh
/api/v1/repos/owner/repo/commits?path=install-script.sh
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22install-script.sh%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Finstall-script.sh
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Finstall-script.sh
/owner/repo/graph?file=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
/api/v1/repos/owner/repo/commits?path=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fbackend%2Fcatalog%2Finformation_schema.sql%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
/owner/repo/graph?file=docs%2Fa%20b.xhtml
/api/v1/repos/owner/repo/commits?path=docs%2Fa%20b.xhtml
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docs%2Fa%20b.xhtml%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocs%2Fa%20b.xhtml
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocs%2Fa%20b.xhtml
/owner/repo/graph?file=%5Bdraft%5D.md
/api/v1/repos/owner/repo/commits?path=%5Bdraft%5D.md
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22%5Bdraft%5D.md%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F%5Bdraft%5D.md
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F%5Bdraft%5D.md
# File names of attachments uploaded through the API, to a release, an issue and
# a comment.
/api/v1/repos/owner/repo/releases/1/assets?name=package.json
/api/v1/repos/owner/repo/releases/1/assets?name=.gitignore
/api/v1/repos/owner/repo/releases/1/assets?name=docker-compose.yml
/api/v1/repos/owner/repo/releases/1/assets?name=.htaccess
/api/v1/repos/owner/repo/releases/1/assets?name=docker-entrypoint
/api/v1/repos/owner/repo/releases/1/assets?name=python3-setup.sh
/api/v1/repos/owner/repo/releases/1/assets?name=Runtime.java
/api/v1/repos/owner/repo/releases/1/assets?name=org.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/releases/1/assets?name=%5Bdraft%5D.md
/api/v1/repos/owner/repo/releases/1/assets?name=notes.txt~
/api/v1/repos/owner/repo/releases/1/assets?name=install-script.sh
/api/v1/repos/owner/repo/issues/1/assets?name=install-script.sh
/api/v1/repos/owner/repo/issues/comments/1/assets?name=install-script.sh
/api/v1/repos/owner/repo/releases/1/assets?name=process.env.d.ts
/api/v1/repos/owner/repo/issues/1/assets?name=process.env.d.ts
/api/v1/repos/owner/repo/issues/comments/1/assets?name=process.env.d.ts
/api/v1/repos/owner/repo/releases/1/assets?name=java.lang.String.md
/api/v1/repos/owner/repo/issues/1/assets?name=java.lang.String.md
/api/v1/repos/owner/repo/issues/comments/1/assets?name=java.lang.String.md
/api/v1/repos/owner/repo/releases/1/assets?name=information_schema.sql
/api/v1/repos/owner/repo/issues/1/assets?name=information_schema.sql
/api/v1/repos/owner/repo/issues/comments/1/assets?name=information_schema.sql
/api/v1/repos/owner/repo/releases/1/assets?name=document.write.js
/api/v1/repos/owner/repo/issues/1/assets?name=document.write.js
/api/v1/repos/owner/repo/issues/comments/1/assets?name=document.write.js
/api/v1/repos/owner/repo/releases/1/assets?name=__proto__.json
/api/v1/repos/owner/repo/issues/1/assets?name=__proto__.json
/api/v1/repos/owner/repo/issues/comments/1/assets?name=__proto__.json
/api/v1/repos/owner/repo/releases/1/assets?name=index.xhtml
/api/v1/repos/owner/repo/issues/1/assets?name=index.xhtml
/api/v1/repos/owner/repo/issues/comments/1/assets?name=index.xhtml
# An artifact upload from actions/upload-artifact@v3, which sends Content-Range.
/api/actions_pipeline/_apis/pipelines/workflows/1/artifacts/9b7c1f3a/upload?itemPath=artifact%2Freport.txt Content-Range: bytes 0-9/10
# Branch and tag names in the query string: deleting a branch, starting a
# release from a tag, a branch protection rule, a directory listing's last
# commits, and a container image push that reuses a layer.
/owner/repo/branches?name=docker-build
/owner/repo/releases/new?tag=docker-build
/owner/repo/settings/branches/edit?rule_name=docker-build
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fdocker-build
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fdocker-build
/owner/repo/branches?name=feat%2Fdocker-support
/owner/repo/releases/new?tag=feat%2Fdocker-support
/owner/repo/settings/branches/edit?rule_name=feat%2Fdocker-support
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Ffeat%2Fdocker-support
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Ffeat%2Fdocker-support
/owner/repo/branches?name=renovate%2Fdocker-build-push-action-6.x
/owner/repo/releases/new?tag=renovate%2Fdocker-build-push-action-6.x
/owner/repo/settings/branches/edit?rule_name=renovate%2Fdocker-build-push-action-6.x
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Frenovate%2Fdocker-build-push-action-6.x
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Frenovate%2Fdocker-build-push-action-6.x
/owner/repo/branches?name=python3-migration
/owner/repo/releases/new?tag=python3-migration
/owner/repo/settings/branches/edit?rule_name=python3-migration
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fpython3-migration
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fpython3-migration
/owner/repo/branches?name=ansible-roles
/owner/repo/releases/new?tag=ansible-roles
/owner/repo/settings/branches/edit?rule_name=ansible-roles
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fansible-roles
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fansible-roles
/owner/repo/branches?name=whoami
/owner/repo/releases/new?tag=whoami
/owner/repo/settings/branches/edit?rule_name=whoami
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fwhoami
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fwhoami
# Pages a signed-in user opens.
/
/owner/repo
/owner/repo/issues
/owner/repo/pulls
/owner/repo/releases
/owner/repo/wiki
/owner/repo/activity
/owner/repo/branches
/owner/repo/tags
/owner/repo/settings
/owner/repo/actions
/owner/repo/compare/main...main
/owner/repo/commits/branch/main
/owner/repo/archive/main.zip
/notifications
/user/settings
/user/settings/applications
/explore/repos
/-/admin
+94 -13
View File
@@ -1,8 +1,9 @@
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the // Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
// method, the URL with its query and the headers of a request, and on its // method, the URL with its query and the headers of a request, and on its
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf // body while SWWAF_WAF_BODY_LIMIT is set, with the seven changes smallwebwaf
// makes to it, as "Attack detection" under "Settings" in SPEC.md describes // makes to it and the exclusions in SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, as
// them. It reads no response. // "Attack detection" under "Settings" in SPEC.md describes them. It reads no
// response.
// //
// smallwebwaf writes only to its state directory, so Coraza is built with // smallwebwaf writes only to its state directory, so Coraza is built with
// its no_fs_access tag, as the Dockerfile and script/build build it: of a // its no_fs_access tag, as the Dockerfile and script/build build it: of a
@@ -11,10 +12,13 @@
package waf package waf
import ( import (
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
"net/netip" "net/netip"
"os"
"regexp"
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
@@ -26,12 +30,13 @@ import (
) )
// directives are the Core Rule Set as smallwebwaf runs it, with the // directives are the Core Rule Set as smallwebwaf runs it, with the
// paranoia level for %d, and bodyDirectives for %s while // paranoia level for %d, bodyDirectives for the first %s while
// SWWAF_WAF_BODY_LIMIT is set. Each rule smallwebwaf adds has an id from // SWWAF_WAF_BODY_LIMIT is set, and the exclusions file for the second.
// 900000 to 900999, the ids the Core Rule Set keeps for the rules that set // Each rule smallwebwaf adds has an id from 900000 to 900999, the ids the
// it up, which SWWAF_WAF_DISABLED_RULES refuses, so that no setting // Core Rule Set keeps for the rules that set it up, which
// switches one off. Coraza joins a line ending in \ to the next, without // SWWAF_WAF_DISABLED_RULES and the exclusions file refuse, so that no
// the spaces at the start of the next. // setting switches one off. Coraza joins a line ending in \ to the next,
// without the spaces at the start of the next.
const directives = ` const directives = `
# The engine only detects. smallwebwaf compares the request's anomaly # The engine only detects. smallwebwaf compares the request's anomaly
# score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode # score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
@@ -69,6 +74,11 @@ SecArgumentsLimit 1000
SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\ SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'" severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
# The exclusions file comes before the rules, where the Core Rule Set puts
# its own file of exclusions that run before the rules, so that they run
# before the rules of phase 1 too.
%s
# The sixth: only the rules for requests are loaded, and no response is # The sixth: only the rules for requests are loaded, and no response is
# inspected. # inspected.
Include @owasp_crs/REQUEST-*.conf Include @owasp_crs/REQUEST-*.conf
@@ -100,6 +110,13 @@ SecRuleUpdateTargetById 932260 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
# Inspect. # Inspect.
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer" SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer" SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
# The seventh: a container image push that reuses a layer names the image
# it takes the layer from in the query parameter from, which is not
# checked for command names. 932260 refused the push of a layer from
# owner/python3-app.
SecRule REQUEST_FILENAME "@rx ^/v2/.+/blobs/uploads/$" \
"id:900500,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:from"
` `
// bodyDirectives have the Core Rule Set read the part of a request body // bodyDirectives have the Core Rule Set read the part of a request body
@@ -160,6 +177,51 @@ type Params struct {
// BodyLimit is the most of a request body the Core Rule Set reads // BodyLimit is the most of a request body the Core Rule Set reads
// (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none. // (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
BodyLimit int64 BodyLimit int64
// ExclusionsFile is the path of a file of exclusions loaded before the
// Core Rule Set's rules (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
// there is none.
ExclusionsFile string
}
// The ids of smallwebwaf's own rules, which an exclusions file may not
// name.
const (
firstOwnRuleID = 900000
lastOwnRuleID = 900999
)
// numberOrRange matches a number, or a range of numbers such as
// 900000-900999.
var numberOrRange = regexp.MustCompile(`(\d+)(?:-(\d+))?`)
// errSmallwebwafRuleID is the error of an exclusions file that names an
// id of smallwebwaf's own rules.
var errSmallwebwafRuleID = errors.New("an id of smallwebwaf's own rules")
// ownRuleNamed returns the first number or range in exclusions, the text
// of an exclusions file, that takes in an id of smallwebwaf's own rules,
// or "" if none does. Comment lines are skipped, as Coraza skips them.
func ownRuleNamed(exclusions string) string {
for line := range strings.Lines(exclusions) {
if strings.HasPrefix(strings.TrimSpace(line), "#") {
continue
}
for _, match := range numberOrRange.FindAllStringSubmatch(line, -1) {
first, _ := strconv.Atoi(match[1])
last := first
if match[2] != "" {
last, _ = strconv.Atoi(match[2])
}
if first <= lastOwnRuleID && last >= firstOwnRuleID {
return match[0]
}
}
}
return ""
} }
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe // CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
@@ -169,16 +231,35 @@ type CoreRuleSet struct {
bodyLimit int64 bodyLimit int64
} }
// New returns the Core Rule Set with the six changes, at params' // New returns the Core Rule Set with the seven changes, at params'
// paranoia level, without the rules it switches off, and reading request // paranoia level, without the rules it switches off, reading request
// bodies up to params' limit. // bodies up to params' limit, and with params' exclusions file loaded
// before the Core Rule Set's rules. An exclusions file that cannot be
// read, does not parse or names an id of smallwebwaf's own rules is an
// error.
func New(params Params) (*CoreRuleSet, error) { func New(params Params) (*CoreRuleSet, error) {
body := "" body := ""
if params.BodyLimit > 0 { if params.BodyLimit > 0 {
body = fmt.Sprintf(bodyDirectives, params.BodyLimit) body = fmt.Sprintf(bodyDirectives, params.BodyLimit)
} }
text := fmt.Sprintf(directives, params.ParanoiaLevel, body) exclusions := ""
if params.ExclusionsFile != "" {
file, err := os.ReadFile(params.ExclusionsFile)
if err != nil {
return nil, fmt.Errorf("read the exclusions file: %w", err)
}
exclusions = string(file)
if named := ownRuleNamed(exclusions); named != "" {
return nil, fmt.Errorf("the exclusions file names %s: %w", named,
errSmallwebwafRuleID)
}
}
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions)
if len(params.DisabledRules) > 0 { if len(params.DisabledRules) > 0 {
ids := make([]string, len(params.DisabledRules)) ids := make([]string, len(params.DisabledRules))
+163 -1
View File
@@ -5,6 +5,7 @@ import (
"net/http/httptest" "net/http/httptest"
"net/netip" "net/netip"
"net/url" "net/url"
"os"
"path/filepath" "path/filepath"
"reflect" "reflect"
"strconv" "strconv"
@@ -119,7 +120,7 @@ func atDefaults(t *testing.T) *waf.CoreRuleSet {
} }
// wantChange checks that crs lets through passes, a gitea request one of // wantChange checks that crs lets through passes, a gitea request one of
// the six changes is for, and still finds result in refused, a request // the seven changes is for, and still finds result in refused, a request
// like it that the change is not for. // like it that the change is not for.
func wantChange( func wantChange(
t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result, t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result,
@@ -362,6 +363,167 @@ func TestEachDisabledRuleIsSwitchedOff(t *testing.T) {
wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{}) wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{})
} }
func TestImagePushThatReusesALayerIsNotCheckedForCommandNames(t *testing.T) {
t.Parallel()
crs := atDefaults(t)
reuse := "?mount=sha256:0000&from=owner/python3-app"
wantChange(t, crs, get("/v2/owner/app/blobs/uploads/"+reuse), get("/upload/"+reuse),
matched(932260))
// A command after the image's name is still refused.
wantResult(t, crs,
get("/v2/owner/app/blobs/uploads/?from=owner/app;cat%20/etc/passwd"),
matched(930120, 932160))
}
// withExclusions returns the Core Rule Set at the defaults, with the
// exclusions file at path.
func withExclusions(path string) (*waf.CoreRuleSet, error) {
return waf.New(waf.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, ExclusionsFile: path,
})
}
// exclusionsFile writes text to a file of exclusions and returns its path.
func exclusionsFile(t *testing.T, text string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "exclusions.conf")
err := os.WriteFile(path, []byte(text), 0o600)
if err != nil {
t.Fatal(err)
}
return path
}
// backupFileExclusion is an exclusions file that switches 920500, a rule
// of phase 1, off for the path of a backup file under /files/.
const backupFileExclusion = `SecRule REQUEST_FILENAME "@beginsWith /files/" ` +
`"id:1,phase:1,pass,nolog,ctl:ruleRemoveTargetById=920500;REQUEST_FILENAME"` + "\n"
func TestExclusionsFileIsLoadedBeforeTheRules(t *testing.T) {
t.Parallel()
crs, err := withExclusions(exclusionsFile(t, backupFileExclusion))
if err != nil {
t.Fatalf("load the exclusions: %v", err)
}
wantChange(t, crs, get("/files/notes.txt~"), get("/notes.txt~"), matched(920500))
}
func TestRuleSwitchedOffAndNamedInTheExclusionsFileIsOff(t *testing.T) {
t.Parallel()
crs, err := waf.New(waf.Params{
ParanoiaLevel: 1, DisabledRules: []int{920500},
ExclusionsFile: exclusionsFile(t, backupFileExclusion),
})
if err != nil {
t.Fatalf("load the exclusions: %v", err)
}
wantResult(t, crs, get("/notes.txt~"), waf.Result{})
}
func TestExclusionsFileMayNameAnyNumberInAComment(t *testing.T) {
t.Parallel()
_, err := withExclusions(exclusionsFile(t,
"# 942100: GET /issues/900123\n"+backupFileExclusion))
if err != nil {
t.Errorf("the exclusions did not load: %v", err)
}
}
func TestExclusionsFileThatCannotBeUsedIsAnError(t *testing.T) {
t.Parallel()
for _, path := range []string{
filepath.Join(t.TempDir(), "missing.conf"),
exclusionsFile(t, "SecNoSuchDirective On\n"),
// The Core Rule Set's rules are not loaded yet.
exclusionsFile(t, "SecRuleUpdateTargetById 942100 \"!ARGS:id\"\n"),
// smallwebwaf's own rules may not be switched off or changed, by
// id or in a range.
exclusionsFile(t, "SecRuleRemoveById 900300\n"),
exclusionsFile(t, "SecRuleRemoveById 899999-901000\n"),
exclusionsFile(t, "SecAction \"id:1,phase:1,pass,nolog,"+
"ctl:ruleRemoveById=899999-901000\"\n"),
exclusionsFile(t, "SecAction \"id:1,phase:1,pass,nolog,"+
"ctl:ruleRemoveTargetById=900500;ARGS:from\"\n"),
} {
_, err := withExclusions(path)
if err == nil {
t.Errorf("%s loaded", path)
}
}
}
// giteaRequests returns the recorded ordinary gitea requests in testdata,
// which script/gitea-setup sends too. Each has the Referer of a page that
// names the same, as a page opened from it has.
func giteaRequests(t *testing.T) []request {
t.Helper()
text, err := os.ReadFile(filepath.Join("testdata", "gitea-requests.txt"))
if err != nil {
t.Fatal(err)
}
var requests []request
for line := range strings.Lines(string(text)) {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") {
continue
}
target, header, _ := strings.Cut(line, " ")
headers := []string{"Referer: http://git.example" + target}
if header != "" {
headers = append(headers, header)
}
requests = append(requests, get(target, headers...))
}
return requests
}
func TestGiteaExclusionsLetOrdinaryGiteaRequestsThroughAndRefuseAttacks(t *testing.T) {
t.Parallel()
crs, err := withExclusions(filepath.Join("..", "..", "deploy", "gitea",
"crs-exclusions.conf"))
if err != nil {
t.Fatalf("load gitea's exclusions: %v", err)
}
const threshold = 5 // SWWAF_WAF_ANOMALY_THRESHOLD's default
for _, r := range giteaRequests(t) {
if got := inspect(t, crs, r); got.Score >= threshold {
t.Errorf("%s %q refused: %+v", r.target, r.headers[1:], got)
}
}
for _, attack := range []string{
"/owner/repo?id=1'%20OR%20'1'='1",
"/owner/repo?id=..%2Fcustom%2Fconf%2Fapp.ini",
"/owner/repo?id=%3Cscript%3Ealert(1)%3C%2Fscript%3E",
} {
if got := inspect(t, crs, get(attack)); got.Score < threshold {
t.Errorf("%s let through: %+v", attack, got)
}
}
}
// bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies. // bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies.
const bodyLimit = 8 << 10 const bodyLimit = 8 << 10
+2 -1
View File
@@ -4,7 +4,8 @@
# default but these: SWRP_UPSTREAM_URL names gitea's container, SWRP_WAF_BACKEND # default but these: SWRP_UPSTREAM_URL names gitea's container, SWRP_WAF_BACKEND
# names servewaf's, and SWWAF_LOG_REQUEST_HEADERS adds the header that labels a # names servewaf's, and SWWAF_LOG_REQUEST_HEADERS adds the header that labels a
# request. Then send through rproxy the gitea requests that reading gitea's # request. Then send through rproxy the gitea requests that reading gitea's
# source found the Core Rule Set refuses, which docs/gitea.md records, and a # source found the Core Rule Set refuses without the exclusions in
# deploy/gitea/crs-exclusions.conf, and a
# short sample of gitea's ordinary use: signing in, pages of files and commits, # short sample of gitea's ordinary use: signing in, pages of files and commits,
# a diff, the API, a push, clone and fetch over HTTP, and two container image # a diff, the API, a push, clone and fetch over HTTP, and two container image
# pushes. Last, print each request servewaf logged, with what it did with it and # pushes. Last, print each request servewaf logged, with what it did with it and
+293
View File
@@ -0,0 +1,293 @@
#!/bin/sh
# script/gitea-setup: build the image, bring up deploy/gitea under a project
# name of its own, and check it:
# - that a page load, a push and a clone sent through traefik pass through
# rproxy, and that servewaf checks them;
# - that gitea logs the visitor's address, not rproxy's;
# - that a request from a container on the shared network skips the checks;
# - that the Core Rule Set refuses none of the ordinary gitea requests in
# internal/waf/testdata/gitea-requests.txt;
# - that it refuses none of the API calls with JSON bodies;
# - that it refuses no fetch;
# - that it refuses SQL injection, path traversal and script injection in a
# query string;
# - that with servewaf stopped, gitea still answers through rproxy.
# It publishes no port of traefik's, since this host's ports 80 and 443 may be
# taken. It reaches traefik at its address on the shared network instead, so
# its own requests come from the network's gateway, which servewaf checks like
# any visitor. Two runs at once on one docker host cannot both start, since
# both would use the subnet compose.yaml names. The containers, the network,
# the volumes, the image and the files the script makes are removed however it
# ends; the gitea and traefik images stay, as the base images of a build do.
# It needs network access. script/check does not run this.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
# Named after this run, so that runs in other clones on the same host
# never touch each other's.
NAME="$("$SCRIPT_DIR/projectname")-gitea-setup-$$"
IMAGE="$NAME-image"
NETWORK="${NAME}_shared"
VISITOR="$NAME-visitor"
# The site's host name and gitea's image, as deploy/gitea/compose.yaml names
# them. gitea's image also runs the container on the shared network.
SITE="git.example.org"
# gitea 1.27.3, 2026-08-29
GITEA_IMAGE="gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef"
# gitea's one user, who owns the repository.
PASSWORD="owner-password"
# The files the script makes, gitea's secret key for this run, and traefik's
# and the gateway's addresses on the shared network once the setup runs.
WORK=""
secret_key=""
traefik=""
gateway=""
cleanup() {
# So that a signal cannot stop it partway through.
trap '' HUP INT TERM
docker rm --force "$VISITOR" >/dev/null 2>&1 || true
if [ -n "$WORK" ]; then
compose down --volumes --remove-orphans --timeout 5 >/dev/null 2>&1 || true
rm -rf "$WORK"
fi
docker rmi --force "$IMAGE" >/dev/null 2>&1 || true
}
# compose <argument>...: docker compose on the setup, under this run's project
# name, with the image built here, this run's secret key and no port of
# traefik's published.
compose() {
SMALLWEBWAF_IMAGE="$IMAGE" GITEA_SECRET_KEY="$secret_key" \
docker compose --project-name "$NAME" \
--file "$ROOT/deploy/gitea/compose.yaml" --file "$WORK/no-ports.yaml" "$@"
}
fail() {
echo "gitea-setup: $*; the last lines of each container's output:" >&2
compose logs --no-color --tail 30 >&2 || true
exit 1
}
# wait_for <what fails> <command>...: run the command every second until
# it succeeds, for at most a minute.
wait_for() {
failure="$1"
shift
tries=0
until "$@"; do
tries=$((tries + 1))
[ "$tries" -lt 60 ] || fail "$failure"
sleep 1
done
}
# logged <service> <text>...: a line of the service's output holds every text,
# in any order.
logged() {
lines="$(compose logs --no-color --no-log-prefix "$1" 2>&1)"
shift
for text in "$@"; do
lines="$(printf '%s\n' "$lines" | grep -F -e "$text")" || return 1
done
}
# send <path> <curl option>...: send a request for path to the site, at
# traefik's address, and print the status of the answer, 000 for none. traefik
# answers with a certificate it made itself, since the setup names no
# certificate resolver.
send() {
path="$1"
shift
curl --silent --show-error --insecure --max-time 30 --output /dev/null \
--write-out '%{http_code}' --resolve "$SITE:443:$traefik" "$@" \
"https://$SITE$path" || true
}
# routed: traefik passes a request for the site's front page on to gitea.
routed() {
[ "$(send / 2>/dev/null)" = 200 ]
}
# site_git <git argument>...: run git as the owner, without the host's own git
# settings, reaching the site at traefik's address. git gives up on a transfer
# slower than 1000 bytes a second for 30 seconds.
site_git() {
GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 git \
-c user.name=owner -c user.email=owner@example.invalid \
-c http.sslVerify=false -c "http.curloptResolve=$SITE:443:$traefik" \
-c http.lowSpeedLimit=1000 -c http.lowSpeedTime=30 "$@"
}
# address_of <service>: print the service's address on the shared network.
address_of() {
docker inspect --format "{{(index .NetworkSettings.Networks \"$NETWORK\").IPAddress}}" \
"$(compose ps --quiet "$1")"
}
# passed_through <method> <path>: rproxy passed a request from the gateway for
# path to gitea on servewaf's allow, and servewaf checked it: its report line
# gives the client's counts and the Core Rule Set's score.
passed_through() {
wait_for "rproxy logged no $1 $2 that servewaf allowed" logged rproxy \
"\"method\":\"$1\"" "\"path\":\"$2\"" "\"client_ip\":\"$gateway\"" \
'"disposition":"allow"'
wait_for "servewaf logged no check of $1 $2" logged servewaf \
'"servewaf_line":"report"' "\"method\":\"$1\"" "\"path\":\"$2\"" \
"\"client_ip\":\"$gateway\"" '"action":"forward"' '"counts":' '"waf_score":'
}
# refused: print each request servewaf logged as refused, for any reason.
refused() {
compose logs --no-color --no-log-prefix servewaf 2>&1 |
grep -F '"servewaf_line":"report"' | grep -v -F '"action":"forward"' || true
}
# ordinary_requests: sign in as the owner, save a file and open an issue
# through the API with JSON bodies, then send, signed in, each request in
# internal/waf/testdata/gitea-requests.txt, with the header its line ends with,
# if any, and that request's own address as its Referer, as a page opened from it
# has. One a second keeps them under the default limit of 120 requests a minute,
# since a request refused for a limit is never inspected. Fail if servewaf
# refuses any of them.
ordinary_requests() {
send /user/login --cookie-jar "$WORK/cookies" >/dev/null
status="$(send /user/login --cookie "$WORK/cookies" --cookie-jar "$WORK/cookies" \
--header "Origin: https://$SITE" --data-urlencode user_name=owner \
--data-urlencode "password=$PASSWORD")"
[ "$status" = 303 ] || fail "signing in was answered $status"
content="$(printf 'declare const env: Record<string, string>;\n' | base64)"
status="$(send /api/v1/repos/owner/repo/contents/process.env.d.ts \
--user "owner:$PASSWORD" --header "Content-Type: application/json" \
--data "{\"message\":\"Add process.env.d.ts\",\"content\":\"$content\"}")"
[ "$status" = 201 ] || fail "saving a file through the API was answered $status"
status="$(send /api/v1/repos/owner/repo/issues --user "owner:$PASSWORD" \
--header "Content-Type: application/json" \
--data '{"title":"ls -la fails","body":"SELECT * FROM users; <img src=x>"}')"
[ "$status" = 201 ] || fail "opening an issue through the API was answered $status"
grep -v -e '^#' -e '^$' "$ROOT/internal/waf/testdata/gitea-requests.txt" |
while read -r target header; do
if [ -n "$header" ]; then
send "$target" --globoff --cookie "$WORK/cookies" \
--header "Referer: https://$SITE$target" --header "$header" >/dev/null
else
send "$target" --globoff --cookie "$WORK/cookies" \
--header "Referer: https://$SITE$target" >/dev/null
fi
sleep 1
done
# The last request's report is logged once its answer has ended.
send /api/v1/settings/api >/dev/null
wait_for "servewaf logged no report of the last ordinary request" logged servewaf \
'"servewaf_line":"report"' '"path":"/api/v1/settings/api"'
refusals="$(refused)"
[ -z "$refusals" ] || fail "servewaf refused ordinary requests:
$refusals"
}
# attacks: send SQL injection, path traversal and script injection in a query
# string, and fail unless the Core Rule Set refuses each.
attacks() {
# The traversal climbs one directory: the default rule file refuses two or
# more first, before the Core Rule Set runs.
for query in "id=1'%20OR%20'1'%3D'1" "id=..%2Fcustom%2Fconf%2Fapp.ini" \
"id=%3Cscript%3Ealert(1)%3C%2Fscript%3E"; do
status="$(send "/owner/repo?$query")"
[ "$status" = 403 ] || fail "the attack ?$query was answered $status"
wait_for "servewaf logged no refusal of ?$query" logged servewaf \
'"servewaf_line":"report"' '"action":"waf_blocked"' "\"query\":\"$query\""
done
}
main() {
cd "$ROOT"
trap cleanup EXIT
trap 'exit 1' HUP INT TERM
WORK="$(mktemp -d)"
printf 'services:\n traefik:\n ports: !reset []\n' >"$WORK/no-ports.yaml"
docker build --no-cache -t "$IMAGE" .
secret_key="$(docker run --rm "$GITEA_IMAGE" gitea generate secret SECRET_KEY)"
# rproxy is healthy once gitea takes connections.
compose up --detach --quiet-pull --wait --wait-timeout 120 ||
fail "the setup did not start"
traefik="$(address_of traefik)"
gateway="$(docker network inspect --format '{{(index .IPAM.Config 0).Gateway}}' "$NETWORK")"
wait_for "traefik did not route the site to rproxy" routed
compose exec --user git gitea gitea admin user create --admin \
--username owner --password "$PASSWORD" --email owner@example.invalid \
--must-change-password=false >/dev/null || fail "creating gitea's user failed"
status="$(send /api/v1/user/repos --user "owner:$PASSWORD" \
--header "Content-Type: application/json" --data '{"name":"repo"}')"
[ "$status" = 201 ] || fail "creating the repository was answered $status"
status="$(send /owner/repo)"
[ "$status" = 200 ] || fail "the repository's page was answered $status"
passed_through GET /owner/repo
echo "gitea-setup: a page load passes through rproxy, and servewaf checks it"
wait_for "gitea did not log the visitor's address" logged gitea \
"GET /owner/repo for $gateway:"
echo "gitea-setup: gitea logs the visitor's address, not rproxy's"
site_git init --quiet --initial-branch=main "$WORK/repo"
echo "hello" >"$WORK/repo/README.md"
site_git -C "$WORK/repo" add README.md
site_git -C "$WORK/repo" commit --quiet --message "Add a README"
site_git -C "$WORK/repo" push --quiet \
"https://owner:$PASSWORD@$SITE/owner/repo.git" main ||
fail "the push failed"
passed_through POST /owner/repo.git/git-receive-pack
echo "gitea-setup: a push passes through rproxy, and servewaf checks it"
site_git clone --quiet "https://owner:$PASSWORD@$SITE/owner/repo.git" \
"$WORK/clone" || fail "the clone failed"
[ "$(cat "$WORK/clone/README.md")" = hello ] || fail "the clone lacks the README"
passed_through POST /owner/repo.git/git-upload-pack
echo "gitea-setup: a clone passes through rproxy, and servewaf checks it"
# A container on the shared network, as sandcastle is, asks for the site
# by its name, and so reaches traefik from its own address.
status="$(docker run --rm --name "$VISITOR" --network "$NETWORK" \
--entrypoint curl "$GITEA_IMAGE" --silent --insecure --max-time 30 \
--output /dev/null --write-out '%{http_code}' "https://$SITE/api/v1/version" ||
true)"
[ "$status" = 200 ] || fail "the request from the shared network was answered $status"
wait_for "servewaf logged no request from the shared network" logged servewaf \
'"servewaf_line":"report"' '"path":"/api/v1/version"' '"action":"forward"'
line="$(compose logs --no-color --no-log-prefix servewaf |
grep -F '"servewaf_line":"report"' | grep -F '"path":"/api/v1/version"')"
case "$line" in
*"\"client_ip\":\"$gateway\""*)
fail "the request from the shared network came from the gateway" ;;
*'"counts":'* | *'"waf_score":'*)
fail "servewaf checked the request from the shared network" ;;
esac
echo "gitea-setup: a request from the shared network skips the checks"
site_git -C "$WORK/clone" fetch --quiet || fail "the fetch failed"
ordinary_requests
echo "gitea-setup: the Core Rule Set refuses no ordinary gitea request"
attacks
echo "gitea-setup: the Core Rule Set refuses the attacks"
compose stop servewaf >/dev/null 2>&1 || fail "servewaf did not stop"
status="$(send /owner/repo)"
[ "$status" = 200 ] ||
fail "with servewaf stopped, the repository's page was answered $status"
wait_for "rproxy logged no request SWRP_DEFAULT_DISPOSITION decided" \
logged rproxy '"path":"/owner/repo"' '"default_disposition":true'
echo "gitea-setup: with servewaf stopped, gitea answers through rproxy"
}
main "$@"