Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
14e8ee7413 |
No files matched your search
+3
-3
@@ -153,15 +153,15 @@ Closest existing options, and why each still falls short:
|
|||||||
- What it is: a single-binary reverse proxy that makes browsers solve a
|
- What it is: a single-binary reverse proxy that makes browsers solve a
|
||||||
proof-of-work challenge before passing them to `TARGET`. Aimed at
|
proof-of-work challenge before passing them to `TARGET`. Aimed at
|
||||||
scrapers, which is likely a large share of unwanted traffic on a public
|
scrapers, which is likely a large share of unwanted traffic on a public
|
||||||
code forge.
|
gitea.
|
||||||
- Covers: R10 well (one container, env vars for listener, target,
|
- Covers: R10 well (one container, env vars for listener, target,
|
||||||
difficulty, cookies). Policy rules can match on path, user agent, headers,
|
difficulty, cookies). Policy rules can match on path, user agent, headers,
|
||||||
IP ranges, and with the vendor's hosted data service also AS number and
|
IP ranges, and with the vendor's hosted data service also AS number and
|
||||||
country, and can weigh a request toward a harder challenge.
|
country, and can weigh a request toward a harder challenge.
|
||||||
- Misses: R1, R3, R4, R5, R6, R9. Bot policy needs a YAML file, not env
|
- Misses: R1, R3, R4, R5, R6, R9. Bot policy needs a YAML file, not env
|
||||||
vars. AS number and country matching depend on the vendor's hosted
|
vars. AS number and country matching depend on the vendor's hosted
|
||||||
service. Breaks non-browser clients unless paths are exempted; for a code
|
service. Breaks non-browser clients unless paths are exempted; for gitea,
|
||||||
forge, git-over-HTTP and API paths must be allowed through by rule.
|
git-over-HTTP and API paths must be allowed through by rule.
|
||||||
- Maturity: very active, widely deployed on code forges since 2025.
|
- Maturity: very active, widely deployed on code forges since 2025.
|
||||||
- Value here: complementary. It can be chained (traefik, then the sidecar,
|
- Value here: complementary. It can be chained (traefik, then the sidecar,
|
||||||
then Anubis, then the app) if challenge pages are wanted.
|
then Anubis, then the app) if challenge pages are wanted.
|
||||||
|
|||||||
@@ -1,11 +1,13 @@
|
|||||||
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
|
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
|
||||||
run example-app
|
run example-app gitea-requests gitea-setup
|
||||||
|
|
||||||
# Makefile targets are thin shims; the implementations live in script/
|
# Makefile targets are thin shims; the implementations live in script/
|
||||||
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
|
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
|
||||||
# of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
|
# of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
|
||||||
# which test checks. build and run are for working on the code by hand;
|
# which test checks. build and run are for working on the code by hand;
|
||||||
# example-app checks the image with an app built on it.
|
# example-app checks the image with an app built on it; gitea-requests
|
||||||
|
# reports the gitea requests the Core Rule Set refuses at the defaults;
|
||||||
|
# gitea-setup checks the setup in deploy/gitea.
|
||||||
|
|
||||||
bootstrap:
|
bootstrap:
|
||||||
@script/bootstrap
|
@script/bootstrap
|
||||||
@@ -45,3 +47,9 @@ run:
|
|||||||
|
|
||||||
example-app:
|
example-app:
|
||||||
@script/example-app
|
@script/example-app
|
||||||
|
|
||||||
|
gitea-requests:
|
||||||
|
@script/gitea-requests
|
||||||
|
|
||||||
|
gitea-setup:
|
||||||
|
@script/gitea-setup
|
||||||
@@ -100,8 +100,8 @@ Point traefik's labels at port 8080 of `rproxy`
|
|||||||
of the app or of `servewaf`. Anything on that network can reach the app around
|
of the app or of `servewaf`. Anything on that network can reach the app around
|
||||||
`rproxy`, and can ask `servewaf` questions, so the network holds only what you
|
`rproxy`, and can ask `servewaf` questions, so the network holds only what you
|
||||||
trust. Each further app adds its own container and its own `rproxy`, whose
|
trust. Each further app adds its own container and its own `rproxy`, whose
|
||||||
`SWRP_WAF_BACKEND` names the same `servewaf`. The "Smallest setup" example below
|
`SWRP_WAF_BACKEND` names the same `servewaf`. The "Smallest setup" and "In front
|
||||||
shows this shape.
|
of gitea" examples below show this shape.
|
||||||
|
|
||||||
### `rproxy` in the app's container
|
### `rproxy` in the app's container
|
||||||
|
|
||||||
@@ -189,12 +189,12 @@ restarting the container. `rproxy` reads only the `SWRP_` settings, and
|
|||||||
default, chosen for a service on the open internet. A setting that needs an
|
default, chosen for a service on the open internet. A setting that needs an
|
||||||
account, a key, a token or an alert destination stays off until you give one.
|
account, a key, a token or an alert destination stays off until you give one.
|
||||||
|
|
||||||
- Any `SWWAF_` or `SWRP_` setting can be given as a file instead, for secrets
|
- Any `SWWAF_` setting can be given as a file instead, for secrets and long
|
||||||
and long lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds
|
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
|
||||||
`SWWAF_ADMIN_TOKEN`. The file's contents are the value, less one newline at
|
The file's contents are the value, less one newline at the end. Setting both
|
||||||
the end. Setting both forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE`
|
forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` already names a file, so
|
||||||
and `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` already name a file, so they have no
|
it has no `_FILE` form, and no `SWRP_` setting has one, since `rproxy` never
|
||||||
`_FILE` form.
|
touches disk.
|
||||||
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
|
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
|
||||||
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
|
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
|
||||||
1024: `512K`, `100M`, `5G`.
|
1024: `512K`, `100M`, `5G`.
|
||||||
@@ -272,7 +272,6 @@ Every setting the code reads, with its default:
|
|||||||
| `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
|
| `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
|
||||||
| `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
| `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
||||||
| `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
| `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
||||||
| `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
|
|
||||||
| `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
| `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
||||||
| `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
| `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
||||||
| `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
| `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
||||||
@@ -339,8 +338,9 @@ full, and the limits that are fixed rather than settings.
|
|||||||
Each example is a complete `compose.yaml`. `smallwebwaf` is the tag
|
Each example is a complete `compose.yaml`. `smallwebwaf` is the tag
|
||||||
`make docker` gives the image; on another host, use the name you pushed it
|
`make docker` gives the image; on another host, use the name you pushed it
|
||||||
under, by digest. Docker Compose puts the services on a private network of its
|
under, by digest. Docker Compose puts the services on a private network of its
|
||||||
own, where each reaches the others by service name. Each example publishes
|
own, where each reaches the others by service name. Only "In front of gitea"
|
||||||
`rproxy`'s port on the host, for trying it out.
|
uses traefik. The other three publish `rproxy`'s port on the host, for trying it
|
||||||
|
out.
|
||||||
|
|
||||||
### Smallest setup
|
### Smallest setup
|
||||||
|
|
||||||
@@ -426,6 +426,88 @@ volumes:
|
|||||||
servewaf-state:
|
servewaf-state:
|
||||||
```
|
```
|
||||||
|
|
||||||
|
### In front of gitea
|
||||||
|
|
||||||
|
gitea behind traefik, set up for a code forge:
|
||||||
|
|
||||||
|
- the upload limits raised, in size and in time;
|
||||||
|
- the response size limit raised, for clones over 1 GiB;
|
||||||
|
- the Actions runners exempt from counting;
|
||||||
|
- the default request and byte limits kept.
|
||||||
|
|
||||||
|
`traefik` is the network traefik is on, made outside this file.
|
||||||
|
|
||||||
|
```yaml
|
||||||
|
services:
|
||||||
|
gitea:
|
||||||
|
# gitea 1.27.3, 2026-08-29
|
||||||
|
image: gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef
|
||||||
|
environment:
|
||||||
|
GITEA__server__ROOT_URL: https://git.example/
|
||||||
|
# The host and the port of the SSH clone URLs gitea shows.
|
||||||
|
GITEA__server__SSH_DOMAIN: git.example
|
||||||
|
GITEA__server__SSH_PORT: "2222"
|
||||||
|
# gitea's TCP peer is rproxy, on the private network.
|
||||||
|
GITEA__security__REVERSE_PROXY_TRUSTED_PROXIES: 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
|
||||||
|
volumes:
|
||||||
|
- gitea-data:/data
|
||||||
|
networks:
|
||||||
|
- site
|
||||||
|
ports:
|
||||||
|
# SSH does not pass through smallwebwaf.
|
||||||
|
- "2222:22"
|
||||||
|
|
||||||
|
rproxy:
|
||||||
|
image: smallwebwaf
|
||||||
|
environment:
|
||||||
|
SWRP_WAF_BACKEND: http://servewaf:8080
|
||||||
|
SWRP_UPSTREAM_URL: http://gitea:3000
|
||||||
|
# Pushes, LFS objects and image layers over 100 MiB, or slower
|
||||||
|
# than a minute, are cut off at the defaults.
|
||||||
|
SWRP_MAX_REQUEST_SIZE: 2G
|
||||||
|
SWRP_MAX_REQUEST_TIME_MS: "600000"
|
||||||
|
# A clone larger than 1 GiB is cut off at the default.
|
||||||
|
SWRP_MAX_RESPONSE_SIZE: 4G
|
||||||
|
networks:
|
||||||
|
- site
|
||||||
|
- traefik
|
||||||
|
labels:
|
||||||
|
traefik.enable: "true"
|
||||||
|
# rproxy is on two networks; traefik reaches it on this one.
|
||||||
|
traefik.docker.network: traefik
|
||||||
|
traefik.http.routers.git.rule: Host(`git.example`)
|
||||||
|
traefik.http.routers.git.entrypoints: websecure
|
||||||
|
traefik.http.routers.git.tls.certresolver: letsencrypt
|
||||||
|
traefik.http.services.git.loadbalancer.server.port: "8080"
|
||||||
|
|
||||||
|
servewaf:
|
||||||
|
image: smallwebwaf
|
||||||
|
environment:
|
||||||
|
SMALLWEBWAF_COMMAND: servewaf
|
||||||
|
SWWAF_INSTANCE_NAME: fsn1/servewaf
|
||||||
|
# The Actions runners ask for work all day; never count them.
|
||||||
|
SWWAF_RATE_LIMIT_EXEMPT_NETS: 10.0.4.0/24
|
||||||
|
# gitea never serves these; a request for one bans the client.
|
||||||
|
SWWAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php,/wp-admin/
|
||||||
|
volumes:
|
||||||
|
- servewaf-state:/var/lib/smallwebwaf
|
||||||
|
networks:
|
||||||
|
- site
|
||||||
|
|
||||||
|
networks:
|
||||||
|
site:
|
||||||
|
traefik:
|
||||||
|
external: true
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
gitea-data:
|
||||||
|
servewaf-state:
|
||||||
|
```
|
||||||
|
|
||||||
|
`deploy/gitea` holds a complete setup of gitea, traefik included, as a Docker
|
||||||
|
Compose file. [`docs/gitea.md`](docs/gitea.md) explains it, and records what the
|
||||||
|
defaults do in front of gitea, request by request.
|
||||||
|
|
||||||
### `rproxy` beside the app
|
### `rproxy` beside the app
|
||||||
|
|
||||||
The app is built on the `smallwebwaf` image, with `rproxy` inside its container.
|
The app is built on the `smallwebwaf` image, with `rproxy` inside its container.
|
||||||
@@ -554,13 +636,10 @@ and the status for a size or time limit, such as `413` for a body too large.
|
|||||||
headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In
|
headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In
|
||||||
front of a code forge, bodies carry what people write and publish, which the
|
front of a code forge, bodies carry what people write and publish, which the
|
||||||
rules take for attacks.
|
rules take for attacks.
|
||||||
- `smallwebwaf` changes the Core Rule Set in eight ways so that it lets ordinary
|
- `smallwebwaf` changes the Core Rule Set in six ways so that it lets ordinary
|
||||||
requests to a code forge through, such as git over HTTP and sign-in through
|
requests to a code forge through, such as git over HTTP and sign-in through
|
||||||
Git Credential Manager. On gitea's own paths, its searches, file names and
|
Git Credential Manager. No setting undoes them; "Attack detection" in
|
||||||
branch names get through too. No setting undoes the changes; "Attack
|
[`SPEC.md`](SPEC.md) lists them.
|
||||||
detection" in [`SPEC.md`](SPEC.md) lists them.
|
|
||||||
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's
|
|
||||||
own fields or paths.
|
|
||||||
- Rule files are plain text, one rule to a line, edited by hand and read again
|
- Rule files are plain text, one rule to a line, edited by hand and read again
|
||||||
while `servewaf` runs. The image ships one file of probes no real visitor
|
while `servewaf` runs. The image ships one file of probes no real visitor
|
||||||
sends (see [`docs/rules.md`](docs/rules.md)).
|
sends (see [`docs/rules.md`](docs/rules.md)).
|
||||||
@@ -692,6 +771,18 @@ on the host.
|
|||||||
- orderly stops with `sv stop` and `docker stop`;
|
- orderly stops with `sv stop` and `docker stop`;
|
||||||
- bans surviving a new container on the same volume;
|
- bans surviving a new container on the same volume;
|
||||||
- a setting given as a file.
|
- a setting given as a file.
|
||||||
|
- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind
|
||||||
|
`rproxy` and `servewaf`, three containers on one network. Every setting is at
|
||||||
|
its default, but for those that join the containers and
|
||||||
|
`SWWAF_LOG_REQUEST_HEADERS`, which adds the header that labels each request.
|
||||||
|
It sends through them a sample of gitea's ordinary use and the requests the
|
||||||
|
Core Rule Set is known to refuse, and prints what `servewaf` did with each.
|
||||||
|
Needs network access, and a docker daemon that pushes to a registry on
|
||||||
|
`localhost` over plain HTTP. `make gitea-requests`.
|
||||||
|
- `script/gitea-setup`: builds the image, brings up the setup in `deploy/gitea`,
|
||||||
|
and checks it as [`docs/gitea.md`](docs/gitea.md) describes. Then it removes
|
||||||
|
the containers, the network, the volumes and the image it built. Needs network
|
||||||
|
access. `make gitea-setup`.
|
||||||
|
|
||||||
## TODO
|
## TODO
|
||||||
|
|
||||||
@@ -715,6 +806,8 @@ on the host.
|
|||||||
- [`docs/state-files.md`](docs/state-files.md): the state files and what each
|
- [`docs/state-files.md`](docs/state-files.md): the state files and what each
|
||||||
holds.
|
holds.
|
||||||
- [`docs/image.md`](docs/image.md): what the image holds and how it is pinned.
|
- [`docs/image.md`](docs/image.md): what the image holds and how it is pinned.
|
||||||
|
- [`docs/gitea.md`](docs/gitea.md): `smallwebwaf` in front of gitea: the setup
|
||||||
|
in `deploy/gitea`, and what the defaults do there.
|
||||||
|
|
||||||
## License
|
## License
|
||||||
|
|
||||||
|
|||||||
@@ -31,13 +31,12 @@ to them.
|
|||||||
reputation sources, and send alerts.
|
reputation sources, and send alerts.
|
||||||
- Everything is set by environment variables. The names of `servewaf`'s start
|
- Everything is set by environment variables. The names of `servewaf`'s start
|
||||||
with `SWWAF_`, and those of `rproxy` with `SWRP_`. The only other inputs are
|
with `SWWAF_`, and those of `rproxy` with `SWRP_`. The only other inputs are
|
||||||
files:
|
`servewaf`'s files:
|
||||||
- `servewaf`'s rule files;
|
- the rule files;
|
||||||
- `servewaf`'s state files;
|
- the state files;
|
||||||
- `servewaf`'s lookup database;
|
- a lookup database;
|
||||||
- a `SWWAF_` or `SWRP_` setting given as a file;
|
- a `SWWAF_` setting given as a file;
|
||||||
- the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names;
|
- the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names.
|
||||||
- the file `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names.
|
|
||||||
|
|
||||||
## Non-goals
|
## Non-goals
|
||||||
|
|
||||||
@@ -82,7 +81,7 @@ to them.
|
|||||||
of bans is full.
|
of bans is full.
|
||||||
- `internal/rules`: reads the rule files at start and again when they
|
- `internal/rules`: reads the rule files at start and again when they
|
||||||
change, and tells which rules a request matches.
|
change, and tells which rules a request matches.
|
||||||
- `internal/waf`: the Core Rule Set, with the eight changes listed under
|
- `internal/waf`: the Core Rule Set, with the six changes listed under
|
||||||
"Attack detection", as Coraza directives.
|
"Attack detection", as Coraza directives.
|
||||||
- `internal/lookup`: looks clients up through GeoJS or in the lookup
|
- `internal/lookup`: looks clients up through GeoJS or in the lookup
|
||||||
database.
|
database.
|
||||||
@@ -124,7 +123,9 @@ runs steps 3 to 13, and does the counting of step 15. The first step that
|
|||||||
produces a final answer ends the checks. Step 15 runs for each request
|
produces a final answer ends the checks. Step 15 runs for each request
|
||||||
`servewaf` answers with `allow`, `deny` or `close`. `rproxy`'s size and time
|
`servewaf` answers with `allow`, `deny` or `close`. `rproxy`'s size and time
|
||||||
limits apply to the whole exchange. Each step names the section that holds its
|
limits apply to the whole exchange. Each step names the section that holds its
|
||||||
rules.
|
rules. Each check is a heuristic. Its name, in parentheses below, is the value
|
||||||
|
of the request log's `heuristic` field. `refusal_limit` and `byte_limits` give
|
||||||
|
theirs in `report_heuristic` instead.
|
||||||
|
|
||||||
1. `GET /_smallwebwaf/healthz` is answered at once by `rproxy`, before any
|
1. `GET /_smallwebwaf/healthz` is answered at once by `rproxy`, before any
|
||||||
check, so a health checker never needs an exemption and is never refused.
|
check, so a health checker never needs an exemption and is never refused.
|
||||||
@@ -142,9 +143,9 @@ rules.
|
|||||||
around its address, a /64 by default, since one abuser usually controls a
|
around its address, a /64 by default, since one abuser usually controls a
|
||||||
whole /64. A client is therefore one IPv4 address or one IPv6 group.
|
whole /64. A client is therefore one IPv4 address or one IPv6 group.
|
||||||
3. The static lists. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 12, the
|
3. The static lists. A client in `SWWAF_ALLOW_NETS` skips steps 4 to 12, the
|
||||||
byte limits and the error burst. A client in `SWWAF_DENY_NETS` is refused; a
|
byte limits and the error burst. A client in `SWWAF_DENY_NETS` is refused
|
||||||
client in both lists is let through.
|
(`deny_nets`); a client in both lists is let through.
|
||||||
4. Bans. A client under a ban is refused (see "Bans").
|
4. Bans. A client under a ban is refused (`active_ban`; see "Bans").
|
||||||
5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
|
5. The lookup, unless `SWWAF_LOOKUP_SOURCE` is `off` (see "Lookup of AS number
|
||||||
and country"). The request goes on at once. It waits for the first GeoJS
|
and country"). The request goes on at once. It waits for the first GeoJS
|
||||||
answer, up to `SWWAF_LOOKUP_TIMEOUT`, only when one of these needs it:
|
answer, up to `SWWAF_LOOKUP_TIMEOUT`, only when one of these needs it:
|
||||||
@@ -154,22 +155,26 @@ rules.
|
|||||||
|
|
||||||
A request that ends before the answer comes is logged without it.
|
A request that ends before the answer comes is logged without it.
|
||||||
|
|
||||||
6. The country lists (see "Country lists").
|
6. The country lists (`countries`; see "Country lists").
|
||||||
7. The reputation sources (see "Reputation"). A client on a fetched blocklist
|
7. The reputation sources (see "Reputation"), in this order:
|
||||||
gets `SWWAF_BLOCKLIST_ACTION`. A client on the CrowdSec decision list is
|
- A client on a fetched blocklist gets `SWWAF_BLOCKLIST_ACTION`
|
||||||
banned. A client listed by a DNS blocklist, or scored at least
|
(`blocklists`).
|
||||||
`SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB, gets `SWWAF_REPUTATION_ACTION`.
|
- A client on the CrowdSec decision list is banned (`crowdsec`).
|
||||||
|
- A client listed by a DNS blocklist gets `SWWAF_REPUTATION_ACTION`
|
||||||
|
(`dnsbl`).
|
||||||
|
- A client scored at least `SWWAF_ABUSEIPDB_MIN_SCORE` by AbuseIPDB gets
|
||||||
|
`SWWAF_REPUTATION_ACTION` (`abuseipdb`).
|
||||||
8. The client's limit percentage (see "Biased thresholds").
|
8. The client's limit percentage (see "Biased thresholds").
|
||||||
9. Rate limiting (see "Request rate limits, per client"). A request over a limit
|
9. Rate limiting (`rate_limits`; see "Request rate limits, per client"). A
|
||||||
times the percentage is refused and bans the client.
|
request over a limit times the percentage is refused and bans the client.
|
||||||
10. The trap paths (see "Attack detection"). A request for one is refused and
|
10. The trap paths (`trap_paths`; see "Attack detection"). A request for one is
|
||||||
bans the client.
|
refused and bans the client.
|
||||||
11. The rule files (see "Rule files"). Matching stops at the first rule that
|
11. The rule files (`rule_files`; see "Rule files"). Matching stops at the first
|
||||||
refuses.
|
rule that refuses.
|
||||||
12. The header limits (see "Answering in time"), and then the Core Rule Set (see
|
12. The header limits (`header_limits`; see "Answering in time"), and then the
|
||||||
"Attack detection"). A Core Rule Set match is refused with `403` while
|
Core Rule Set (`core_rule_set`; see "Attack detection"). A Core Rule Set
|
||||||
`SWWAF_WAF_MODE` is `block`, and only logged and alerted while it is
|
match is refused with `403` while `SWWAF_WAF_MODE` is `block`, and only
|
||||||
`detect`.
|
logged and alerted while it is `detect`.
|
||||||
13. A path under `/_smallwebwaf/` is answered `404`, so that the app never sees
|
13. A path under `/_smallwebwaf/` is answered `404`, so that the app never sees
|
||||||
that prefix.
|
that prefix.
|
||||||
14. `rproxy` passes the request to `SWRP_UPSTREAM_URL`, streaming (see "Passing
|
14. `rproxy` passes the request to `SWRP_UPSTREAM_URL`, streaming (see "Passing
|
||||||
@@ -183,14 +188,14 @@ rules.
|
|||||||
are added to the byte counters, as `SWWAF_BYTES_COUNT` says. For a
|
are added to the byte counters, as `SWWAF_BYTES_COUNT` says. For a
|
||||||
WebSocket, what it carried each way counts once it closes. A client whose
|
WebSocket, what it carried each way counts once it closes. A client whose
|
||||||
total passes a byte limit times the percentage has broken that limit and
|
total passes a byte limit times the percentage has broken that limit and
|
||||||
is banned. The response in progress is not cut off.
|
is banned (`byte_limits`). The response in progress is not cut off.
|
||||||
- For a request it allowed, the client's history and the anomaly counters
|
- For a request it allowed, the client's history and the anomaly counters
|
||||||
are updated, and the alert thresholds are evaluated.
|
are updated, and the alert thresholds are evaluated.
|
||||||
|
|
||||||
For a request it refused, `servewaf` adds nothing to the byte limits. It
|
For a request it refused, `servewaf` adds nothing to the byte limits. It
|
||||||
updates the history and the anomaly counters at its decision. It counts the
|
updates the history and the anomaly counters at its decision. It counts the
|
||||||
refusals the error burst counts at the decision too (see "Attack
|
refusals the error burst counts at the decision too (`refusal_limit`; see
|
||||||
detection").
|
"Attack detection").
|
||||||
|
|
||||||
In `SWWAF_MODE=observe`, every refusal above is logged as what would have
|
In `SWWAF_MODE=observe`, every refusal above is logged as what would have
|
||||||
happened, nothing is refused, and `servewaf` makes no ban of its own. A path
|
happened, nothing is refused, and `servewaf` makes no ban of its own. A path
|
||||||
@@ -322,13 +327,13 @@ Conventions:
|
|||||||
No other setting can be switched off with `off`.
|
No other setting can be switched off with `off`.
|
||||||
|
|
||||||
- A list set to an empty value is an empty list, and replaces the default.
|
- A list set to an empty value is an empty list, and replaces the default.
|
||||||
- Any `SWWAF_` or `SWRP_` setting can be read from a file instead, for secrets
|
- Any `SWWAF_` setting can be read from a file instead, for secrets and long
|
||||||
and long lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds
|
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
|
||||||
`SWWAF_ADMIN_TOKEN`. The file's contents are the value, less one newline at
|
The file's contents are the value, less one newline at the end, and are
|
||||||
the end, and are checked as the value itself would be. Setting both forms
|
checked as the value itself would be. Setting both forms stops the start, and
|
||||||
stops the start, and so does naming a file that cannot be read.
|
so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE`
|
||||||
`SWWAF_LOG_REMOTE_TLS_CA_FILE` and `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE`
|
already holds a file name, so it has no `_FILE` form. No `SWRP_` setting has
|
||||||
already hold a file name, so they have no `_FILE` form.
|
one, since `rproxy` never touches disk.
|
||||||
- Settings, files included, are read once at start. Changing one means
|
- Settings, files included, are read once at start. Changing one means
|
||||||
restarting the container. The files `servewaf` watches while it runs are its
|
restarting the container. The files `servewaf` watches while it runs are its
|
||||||
state files, its rule files and the lookup database.
|
state files, its rule files and the lookup database.
|
||||||
@@ -403,10 +408,10 @@ scrapers that send many requests slowly over hours or days. The hour limit is
|
|||||||
the main one.
|
the main one.
|
||||||
|
|
||||||
- Per minute: 120, 2 requests a second. It leaves room for a person opening two
|
- Per minute: 120, 2 requests a second. It leaves room for a person opening two
|
||||||
pages at once with an empty cache. A web app's page opened with an empty cache
|
pages at once with an empty cache. A gitea page opened with an empty cache
|
||||||
loads 20 to 60 files, and a handful once cached.
|
loads 20 to 60 files, and a handful once cached.
|
||||||
- Per hour: 1,500. A person reading a site for a whole hour, a page every 20
|
- Per hour: 1,500. A person reading code in gitea for a whole hour, a page every
|
||||||
seconds, makes about 900 requests. A scraper making one request every 2
|
20 seconds, makes about 900 requests. A scraper making one request every 2
|
||||||
seconds makes 1,800 an hour, more than the limit.
|
seconds makes 1,800 an hour, more than the limit.
|
||||||
- Per day: 6,000, four hours at the hour limit. A scraper making one request
|
- Per day: 6,000, four hours at the hour limit. A scraper making one request
|
||||||
every 10 seconds makes 8,640 a day, more than the limit.
|
every 10 seconds makes 8,640 a day, more than the limit.
|
||||||
@@ -418,7 +423,7 @@ the blocklists. An abuser that trips none of them is not caught.
|
|||||||
|
|
||||||
Some legitimate clients pass the limits:
|
Some legitimate clients pass the limits:
|
||||||
|
|
||||||
- A CI runner asks its code forge for work all day and can pass the limits.
|
- An Actions runner asks gitea for work all day and can pass the limits.
|
||||||
- Everyone behind one shared address, such as an office, shares one client's
|
- Everyone behind one shared address, such as an office, shares one client's
|
||||||
limits.
|
limits.
|
||||||
|
|
||||||
@@ -551,11 +556,12 @@ service's abuse comes from.
|
|||||||
By default the Core Rule Set reads the method, the URL with its query string,
|
By default the Core Rule Set reads the method, the URL with its query string,
|
||||||
and the headers of every request. That is where automated attacks show:
|
and the headers of every request. That is where automated attacks show:
|
||||||
injection in query strings, path traversal, attacks carried in headers,
|
injection in query strings, path traversal, attacks carried in headers,
|
||||||
scanners' user agents. It reads no request body. In front of a code forge,
|
scanners' user agents. It reads no request body. In front of a code forge such
|
||||||
bodies carry what people write and publish, such as issue text and files saved
|
as gitea, bodies carry what people write and publish, such as issue text and
|
||||||
in the web editor. The Core Rule Set takes the shell commands and code in them
|
files saved in the web editor. The Core Rule Set takes the shell commands and
|
||||||
for attacks. The default gives up refusing an attack carried in a body. The rule
|
code in them for attacks. The default gives up refusing an attack carried in a
|
||||||
files, the limits and the bans still apply to the client that sends it.
|
body. The rule files, the limits and the bans still apply to the client that
|
||||||
|
sends it.
|
||||||
|
|
||||||
- The Core Rule Set skips a path under `SWWAF_WAF_EXEMPT_PATHS`.
|
- The Core Rule Set skips a path under `SWWAF_WAF_EXEMPT_PATHS`.
|
||||||
- `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule
|
- `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule
|
||||||
@@ -583,27 +589,6 @@ files, the limits and the bans still apply to the client that sends it.
|
|||||||
probes these rules caught (see "Rule files"). A list you give replaces the
|
probes these rules caught (see "Rule files"). A list you give replaces the
|
||||||
default, so name these six ids in it to keep them off.
|
default, so name these six ids in it to keep them off.
|
||||||
|
|
||||||
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names a file of exclusions, written as
|
|
||||||
Coraza directives. An exclusion switches one rule off for some fields or paths
|
|
||||||
only, where `SWWAF_WAF_DISABLED_RULES` switches it off everywhere. The file
|
|
||||||
holds an app's own exclusions; the eighth change below holds gitea's.
|
|
||||||
- `servewaf` loads the file after `smallwebwaf`'s own setup rules and before
|
|
||||||
the Core Rule Set's rules. The Core Rule Set loads its own
|
|
||||||
`REQUEST-900-EXCLUSION-RULES-BEFORE-CRS.conf` at the same point.
|
|
||||||
- So the file holds only `SecAction` and `SecRule` lines that exclude with
|
|
||||||
`ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. These run with each
|
|
||||||
request, before the rules they exclude, those of phase 1 included.
|
|
||||||
- A rule that `SWWAF_WAF_DISABLED_RULES` switches off and the file names is
|
|
||||||
simply off.
|
|
||||||
- The start stops on a file that cannot be read or does not parse, whatever
|
|
||||||
`SWWAF_WAF_MODE` is.
|
|
||||||
- It stops too on a file that names a number from 900000 to 900999, the ids
|
|
||||||
of `smallwebwaf`'s own rules, alone or in a range. Comment lines are not
|
|
||||||
checked.
|
|
||||||
- It stops too on a directive that removes or changes rules as the file
|
|
||||||
loads, such as `SecRuleRemoveById`, since none of the Core Rule Set's
|
|
||||||
rules is loaded yet.
|
|
||||||
|
|
||||||
- `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. It takes
|
- `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. It takes
|
||||||
at most `64K`, since `servewaf` gets no more of a body than that (see "The
|
at most `64K`, since `servewaf` gets no more of a body than that (see "The
|
||||||
answer"). Form data and multipart bodies are read up to the size. JSON and XML
|
answer"). Form data and multipart bodies are read up to the size. JSON and XML
|
||||||
@@ -612,14 +597,15 @@ files, the limits and the bans still apply to the client that sends it.
|
|||||||
data, where binary content such as a git push trips rules written for text. A
|
data, where binary content such as a git push trips rules written for text. A
|
||||||
body the Core Rule Set cannot parse adds 5 to the score, and so does a
|
body the Core Rule Set cannot parse adds 5 to the score, and so does a
|
||||||
multipart body cut by the limit in the middle of a part's header. Body
|
multipart body cut by the limit in the middle of a part's header. Body
|
||||||
inspection suits apps whose forms carry no code.
|
inspection suits apps whose forms carry no code; `docs/gitea.md` records what
|
||||||
|
it refuses in front of gitea.
|
||||||
- `SWWAF_TRAP_PATHS` names paths the app never serves and only scanners ask for,
|
- `SWWAF_TRAP_PATHS` names paths the app never serves and only scanners ask for,
|
||||||
such as `/wp-login.php` in front of an app that is not WordPress. This is the
|
such as `/wp-login.php` in front of gitea. This is the short form of a `path`
|
||||||
short form of a `path` rule with the `ban` action, for deployments that mount
|
rule with the `ban` action, for deployments that mount no rule files. A trap
|
||||||
no rule files. A trap path is matched whole, without the query, against the
|
path is matched whole, without the query, against the path as sent and once
|
||||||
path as sent and once percent-decoded: `/wp-login.php` matches
|
percent-decoded: `/wp-login.php` matches `/wp-login.php?x=1` and
|
||||||
`/wp-login.php?x=1` and `/%77p-login.php`, and does not match `/wp-login.php/`
|
`/%77p-login.php`, and does not match `/wp-login.php/` or
|
||||||
or `/blog/wp-login.php`.
|
`/blog/wp-login.php`.
|
||||||
- `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute of these
|
- `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute of these
|
||||||
kinds, and of no other:
|
kinds, and of no other:
|
||||||
- a rule file match;
|
- a rule file match;
|
||||||
@@ -629,10 +615,10 @@ files, the limits and the bans still apply to the client that sends it.
|
|||||||
More than this breaks a limit (see "Bans"). A client trying one attack after
|
More than this breaks a limit (see "Bans"). A client trying one attack after
|
||||||
another is refused many times a minute; a person rarely more than a few
|
another is refused many times a minute; a person rarely more than a few
|
||||||
times. Answers the app gives are not counted, since they do not tell a
|
times. Answers the app gives are not counted, since they do not tell a
|
||||||
scanner from an ordinary client. In front of a code forge, registry and git
|
scanner from an ordinary client. In front of gitea, registry and git clients
|
||||||
clients get `404` and `401` by design, hundreds of times a minute.
|
get `404` and `401` by design, hundreds of times a minute.
|
||||||
|
|
||||||
`smallwebwaf` changes the Core Rule Set 4.25.0 in eight ways, since in front of
|
`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of
|
||||||
gitea it would otherwise refuse ordinary requests. The changes hold in front of
|
gitea it would otherwise refuse ordinary requests. The changes hold in front of
|
||||||
every app, and no setting undoes them:
|
every app, and no setting undoes them:
|
||||||
|
|
||||||
@@ -664,10 +650,9 @@ every app, and no setting undoes them:
|
|||||||
or `docker-build`. In these parameters, what only these three rules would
|
or `docker-build`. In these parameters, what only these three rules would
|
||||||
catch reaches the app, such as the command `|cat /etc/passwd`. Path
|
catch reaches the app, such as the command `|cat /etc/passwd`. Path
|
||||||
traversal, SQL and script injection, and PHP, Java and Node.js code are
|
traversal, SQL and script injection, and PHP, Java and Node.js code are
|
||||||
still refused there. Other parameters keep all three rules, except on the
|
still refused there. Every other parameter keeps all three rules. An app
|
||||||
gitea paths the eighth change names. An app that uses one of these
|
that uses one of these parameters as a file on the server, or passes it to a
|
||||||
parameters as a file on the server, or passes it to a shell, must check the
|
shell, must check the value itself. A rule file can refuse it instead.
|
||||||
value itself. A rule file can refuse it instead.
|
|
||||||
|
|
||||||
5. Two cookies are left out, and `Referer` keeps fewer rules:
|
5. Two cookies are left out, and `Referer` keeps fewer rules:
|
||||||
- The Core Rule Set reads the request without the `gitea_flash` and
|
- The Core Rule Set reads the request without the `gitea_flash` and
|
||||||
@@ -681,83 +666,15 @@ every app, and no setting undoes them:
|
|||||||
only to its own pages. Every other cookie is read in full. An app that
|
only to its own pages. Every other cookie is read in full. An app that
|
||||||
reads a cookie of either name must check it itself.
|
reads a cookie of either name must check it itself.
|
||||||
- `Referer` is not checked for a Unix command given without arguments
|
- `Referer` is not checked for a Unix command given without arguments
|
||||||
(932340), for Java starting a process (944110) or for a Java class name
|
(932340) or for Java starting a process (944110). `Referer` names the page
|
||||||
(944130). `Referer` names the page a request comes from, on everything
|
a request comes from, on everything that page loads and on every link
|
||||||
that page loads and on every link followed from it. The Core Rule Set
|
followed from it. The Core Rule Set takes some page addresses for attacks,
|
||||||
takes some page addresses for attacks:
|
such as a search for `env`, or OpenJDK's `Runtime.java`, whose path holds
|
||||||
- a search for `env`;
|
both `runtime` and `java.`. All of those requests would be refused.
|
||||||
- OpenJDK's `Runtime.java`, whose path holds both `runtime` and `java.`;
|
`Referer` keeps the other rules. An app that passes `Referer` to a shell
|
||||||
- a file named `java.lang.String.md`.
|
must check it itself.
|
||||||
|
|
||||||
Every request from such a page would be refused. `Referer` keeps the
|
|
||||||
other rules, except after some gitea searches (the eighth change). An
|
|
||||||
app that passes `Referer` to a shell must check it itself.
|
|
||||||
|
|
||||||
6. Responses are not inspected. A raw file from a repository, such as a shell
|
6. Responses are not inspected. A raw file from a repository, such as a shell
|
||||||
script, looks to the response rules like source code leaking from the server.
|
script, looks to the response rules like source code leaking from the server.
|
||||||
7. The `from` parameter of a container registry's upload path,
|
|
||||||
`/v2/<image>/blobs/uploads/`, is not checked for command names (932260). When
|
|
||||||
a push reuses a layer, `from` names the image the layer comes from. An image
|
|
||||||
such as `owner/python3-app` would have that push refused.
|
|
||||||
8. On gitea's paths, some fields skip the rules that refused ordinary gitea
|
|
||||||
requests there. They are the fields where gitea sends a search, a file name
|
|
||||||
or a branch name. `internal/waf/gitea.conf` lists each rule and the request
|
|
||||||
it refused. The fields are:
|
|
||||||
- `q` on gitea's searches of code, of issues and pull requests, and of
|
|
||||||
commits. A code search holds code, such as `<script>`,
|
|
||||||
`SELECT * FROM users WHERE` or `../`.
|
|
||||||
- `Referer` of a page opened from such a search, for script and SQL
|
|
||||||
injection and Java code (941110, 941120, 941160, 942100, 944100).
|
|
||||||
- `q` on gitea's searches for a name, for the names of system files and
|
|
||||||
commands (930120, 932120, 932260). Names such as `.profile` and
|
|
||||||
`python3-app` were refused. gitea searches for the names of:
|
|
||||||
- repositories;
|
|
||||||
- users and organisations;
|
|
||||||
- branches and tags;
|
|
||||||
- packages.
|
|
||||||
|
|
||||||
A user's or an organisation's page is any path of one segment, so on
|
|
||||||
every such path, on any app, `q` skips these three rules, for example on
|
|
||||||
`/search`.
|
|
||||||
|
|
||||||
Some of these searches also skip the rule for Java class names (944130),
|
|
||||||
which refused `org.apache.commons.logging`.
|
|
||||||
|
|
||||||
- The path of these requests, which names a file:
|
|
||||||
- a file's page;
|
|
||||||
- its raw file;
|
|
||||||
- its download;
|
|
||||||
- its blame page;
|
|
||||||
- its history;
|
|
||||||
- its edit page;
|
|
||||||
- its preview in the editor;
|
|
||||||
- its delete page;
|
|
||||||
- the API's call for its contents;
|
|
||||||
- a release attachment's download.
|
|
||||||
|
|
||||||
Names such as `notes.txt~`, `process.env.d.ts` and `java.lang.String.md`
|
|
||||||
were refused.
|
|
||||||
|
|
||||||
- A file name in a query parameter:
|
|
||||||
- `file` of a commit graph;
|
|
||||||
- `path` of the API's list of commits;
|
|
||||||
- `path` of a diff's hidden lines;
|
|
||||||
- `body` of the API's call for several files;
|
|
||||||
- `template` of a new issue;
|
|
||||||
- `redirect_to` of the sign-in page;
|
|
||||||
- `name` of an attachment uploaded through the API.
|
|
||||||
- A branch or tag name in a query parameter, for command names (932260).
|
|
||||||
gitea sends one:
|
|
||||||
- to delete or restore a branch;
|
|
||||||
- to start a release from a tag;
|
|
||||||
- to edit a branch protection rule;
|
|
||||||
- to list a directory's last commits.
|
|
||||||
- The `Content-Range` header of an artifact upload from
|
|
||||||
`actions/upload-artifact@v3` (920450).
|
|
||||||
|
|
||||||
Elsewhere these fields keep every rule. The paths are those of gitea at the
|
|
||||||
site root. An app other than gitea that serves one of them must check these
|
|
||||||
fields itself.
|
|
||||||
|
|
||||||
The parameter and cookie names in these changes are matched without regard to
|
The parameter and cookie names in these changes are matched without regard to
|
||||||
case, as Coraza matches them.
|
case, as Coraza matches them.
|
||||||
@@ -925,7 +842,7 @@ and the default file.
|
|||||||
runs. It reads the files again once the directory has had no change for 2
|
runs. It reads the files again once the directory has had no change for 2
|
||||||
seconds, so that a file still being written is read whole. A file whose name
|
seconds, so that a file still being written is read whole. A file whose name
|
||||||
starts with `.`, such as an editor's lock file, is not a rule file.
|
starts with `.`, such as an editor's lock file, is not a rule file.
|
||||||
- Files are read in name order (`00-default.rules` before `50-site.rules`),
|
- Files are read in name order (`00-default.rules` before `50-gitea.rules`),
|
||||||
rules in line order.
|
rules in line order.
|
||||||
- A rule's regex is Go's RE2 syntax. It has no backreferences or lookaround; in
|
- A rule's regex is Go's RE2 syntax. It has no backreferences or lookaround; in
|
||||||
exchange matching time is linear in the input, so no rule can be made to stall
|
exchange matching time is linear in the input, so no rule can be made to stall
|
||||||
@@ -954,12 +871,14 @@ and the default file.
|
|||||||
|
|
||||||
- The image ships a default file, kept short and limited to patterns that are
|
- The image ships a default file, kept short and limited to patterns that are
|
||||||
wrong for every app, with its path rules anchored at the site root. In front
|
wrong for every app, with its path rules anchored at the site root. In front
|
||||||
of a code forge, `/.env` is a probe, while
|
of gitea, `/.env` is a probe, while
|
||||||
`/<owner>/<repo>/src/branch/main/.env.example` is a file in a repository that
|
`/<owner>/<repo>/src/branch/main/.env.example` is a file in a repository that
|
||||||
any visitor may open. Anchor the paths of your own `ban` rules at the site
|
any visitor may open. Anchor the paths of your own `ban` rules at the site
|
||||||
root too. A rule has one target, so a `path` rule applies on every site behind
|
root too. Anything else, such as a rule banning WordPress probes in front of
|
||||||
that `servewaf`. A directory mounted over `SWWAF_RULES_DIR` replaces the
|
gitea, belongs in a file the deployer adds to the image `servewaf` runs from,
|
||||||
default file; single files mounted into it add to it.
|
or mounts. A rule has one target, so a `path` rule applies on every site
|
||||||
|
behind that `servewaf`. A directory mounted over `SWWAF_RULES_DIR` replaces
|
||||||
|
the default file; single files mounted into it add to it.
|
||||||
- Rules meant to be edited while `servewaf` runs belong in a mounted directory,
|
- Rules meant to be edited while `servewaf` runs belong in a mounted directory,
|
||||||
not a mounted file. Many editors save a file by replacing it, and a container
|
not a mounted file. Many editors save a file by replacing it, and a container
|
||||||
keeps seeing the old one when the file alone is mounted. The lookup database
|
keeps seeing the old one when the file alone is mounted. The lookup database
|
||||||
@@ -1782,7 +1701,8 @@ with no change to upaas. upaas builds the app's Dockerfile and runs the one
|
|||||||
container as it runs any app. The traefik labels route to port 8080, and the
|
container as it runs any app. The traefik labels route to port 8080, and the
|
||||||
`SWRP_` settings go with the app's environment variables. `servewaf` runs in a
|
`SWRP_` settings go with the app's environment variables. `servewaf` runs in a
|
||||||
container of its own, on a network the app's container can reach, with its state
|
container of its own, on a network the app's container can reach, with its state
|
||||||
volume and the directory of any token file.
|
volume and the directory of any token file. `docs/gitea.md` records what the
|
||||||
|
defaults do in front of gitea, and `script/gitea-requests` checks those notes.
|
||||||
|
|
||||||
## Planned, not built
|
## Planned, not built
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,135 @@
|
|||||||
|
# Gitea behind traefik, with rproxy and servewaf in front of it.
|
||||||
|
# docs/gitea.md says what each setting here does and why. Before starting it:
|
||||||
|
# - set SMALLWEBWAF_IMAGE to the smallwebwaf image, named by digest;
|
||||||
|
# - replace git.example.org with the site's host name;
|
||||||
|
# - replace each 10.99.0 with the first three numbers of a /24 subnet that
|
||||||
|
# no other network on the host uses. The /24 must lie inside 10.0.0.0/8,
|
||||||
|
# 172.16.0.0/12 or 192.168.0.0/16, because rproxy believes
|
||||||
|
# X-Forwarded-For only from those by default. Replace each fd10:99 with
|
||||||
|
# the first two hextets of an unused /64 subnet whose third and fourth
|
||||||
|
# hextets are 0, as in fd10:99::/64;
|
||||||
|
# - set GITEA_SECRET_KEY to the output of
|
||||||
|
# `docker run --rm <gitea's image below> gitea generate secret SECRET_KEY`,
|
||||||
|
# once per install, and keep it.
|
||||||
|
# Then run `docker compose up --detach` in this directory.
|
||||||
|
# script/gitea-setup checks this setup.
|
||||||
|
|
||||||
|
name: gitea
|
||||||
|
|
||||||
|
x-logging: &logging
|
||||||
|
driver: json-file
|
||||||
|
options:
|
||||||
|
max-size: 10m
|
||||||
|
max-file: "5"
|
||||||
|
|
||||||
|
services:
|
||||||
|
traefik:
|
||||||
|
# traefik v3.7.17, 2026-11-03
|
||||||
|
image: traefik@sha256:68ae5fe5b64ca166704b746a9e8afcd385e36b3f569a5691401bf652d57b3748
|
||||||
|
# traefik replaces the X-Forwarded-For header a visitor sends. Keep it
|
||||||
|
# so: otherwise a visitor could name an address on the shared network,
|
||||||
|
# and skip servewaf's checks.
|
||||||
|
command:
|
||||||
|
- --providers.docker.exposedbydefault=false
|
||||||
|
- --entrypoints.web.address=:80
|
||||||
|
- --entrypoints.web.http.redirections.entrypoint.to=websecure
|
||||||
|
- --entrypoints.websecure.address=:443
|
||||||
|
# Until a certificate resolver is added here, as for your other
|
||||||
|
# sites, traefik answers with a certificate it made itself.
|
||||||
|
- --entrypoints.websecure.http.tls=true
|
||||||
|
# traefik's own limit of 60 seconds for a whole request would cut
|
||||||
|
# off an upload long before rproxy's limit of 30 minutes.
|
||||||
|
- --entrypoints.websecure.transport.respondingTimeouts.readTimeout=30m
|
||||||
|
ports:
|
||||||
|
- 80:80
|
||||||
|
- 443:443
|
||||||
|
volumes:
|
||||||
|
- /var/run/docker.sock:/var/run/docker.sock:ro
|
||||||
|
networks:
|
||||||
|
shared:
|
||||||
|
# Containers on the shared network find traefik under the
|
||||||
|
# site's name, and so reach it from their own address.
|
||||||
|
aliases:
|
||||||
|
- git.example.org
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
gitea:
|
||||||
|
# gitea 1.27.3, 2026-08-29
|
||||||
|
image: gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef
|
||||||
|
environment:
|
||||||
|
GITEA__server__ROOT_URL: https://git.example.org/
|
||||||
|
# Gitea starts at its defaults, with SQLite, instead of showing its
|
||||||
|
# install page to whoever opens it first.
|
||||||
|
GITEA__security__INSTALL_LOCK: "true"
|
||||||
|
GITEA__security__SECRET_KEY: ${GITEA_SECRET_KEY:?set GITEA_SECRET_KEY, see docs/gitea.md}
|
||||||
|
# Accounts are made with `gitea admin user create`.
|
||||||
|
GITEA__service__DISABLE_REGISTRATION: "true"
|
||||||
|
# rproxy's addresses. Gitea then takes each visitor's address from
|
||||||
|
# the X-Real-Ip header traefik sets and rproxy passes on.
|
||||||
|
GITEA__security__REVERSE_PROXY_TRUSTED_PROXIES: 10.99.0.2,fd10:99::2
|
||||||
|
volumes:
|
||||||
|
- gitea-data:/data
|
||||||
|
networks:
|
||||||
|
- shared
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
rproxy:
|
||||||
|
image: ${SMALLWEBWAF_IMAGE:?set SMALLWEBWAF_IMAGE to the smallwebwaf image}
|
||||||
|
environment:
|
||||||
|
SWRP_WAF_BACKEND: http://servewaf:8080
|
||||||
|
SWRP_UPSTREAM_URL: http://gitea:3000
|
||||||
|
# Pushes, LFS objects, packages and release attachments of up to
|
||||||
|
# 2 GiB, sent in up to 30 minutes.
|
||||||
|
SWRP_MAX_REQUEST_SIZE: 2G
|
||||||
|
SWRP_MAX_REQUEST_TIME_MS: "1800000"
|
||||||
|
# 15 minutes for a clone or a download, half the default.
|
||||||
|
SWRP_MAX_RESPONSE_TIME_MS: "900000"
|
||||||
|
labels:
|
||||||
|
traefik.enable: "true"
|
||||||
|
traefik.http.routers.gitea.rule: Host(`git.example.org`)
|
||||||
|
traefik.http.routers.gitea.entrypoints: websecure
|
||||||
|
traefik.http.services.gitea.loadbalancer.server.port: "8080"
|
||||||
|
networks:
|
||||||
|
shared:
|
||||||
|
ipv4_address: 10.99.0.2
|
||||||
|
ipv6_address: fd10:99::2
|
||||||
|
logging: *logging
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
servewaf:
|
||||||
|
image: ${SMALLWEBWAF_IMAGE:?set SMALLWEBWAF_IMAGE to the smallwebwaf image}
|
||||||
|
environment:
|
||||||
|
SMALLWEBWAF_COMMAND: servewaf
|
||||||
|
# The containers on the shared network skip the checks. Add the
|
||||||
|
# address of an automation fleet on another host here too; without
|
||||||
|
# that, it is counted like any client.
|
||||||
|
SWWAF_ALLOW_NETS: 10.99.0.128/25,fd10:99:0:0:8000::/65
|
||||||
|
# In block mode the Core Rule Set refuses everyday Gitea searches
|
||||||
|
# and file names, and 31 refusals in a minute ban the address.
|
||||||
|
SWWAF_WAF_MODE: detect
|
||||||
|
volumes:
|
||||||
|
- servewaf-state:/var/lib/smallwebwaf
|
||||||
|
networks:
|
||||||
|
- shared
|
||||||
|
logging: *logging
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
networks:
|
||||||
|
# The network traefik and Gitea share. Its containers get addresses in the
|
||||||
|
# upper half of each subnet, which servewaf lets skip the checks. Each
|
||||||
|
# subnet's gateway, the host's own address on it, stays out of that half:
|
||||||
|
# requests docker passes on from a published port can come from it.
|
||||||
|
shared:
|
||||||
|
enable_ipv6: true
|
||||||
|
ipam:
|
||||||
|
config:
|
||||||
|
- subnet: 10.99.0.0/24
|
||||||
|
ip_range: 10.99.0.128/25
|
||||||
|
gateway: 10.99.0.1
|
||||||
|
- subnet: fd10:99::/64
|
||||||
|
ip_range: fd10:99:0:0:8000::/65
|
||||||
|
gateway: fd10:99::1
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
gitea-data:
|
||||||
|
servewaf-state:
|
||||||
+340
@@ -0,0 +1,340 @@
|
|||||||
|
# smallwebwaf in front of gitea
|
||||||
|
|
||||||
|
`deploy/gitea/compose.yaml` runs gitea behind traefik, with `rproxy` and
|
||||||
|
`servewaf` in front of it. This document says what that setup sets and why, and
|
||||||
|
what the defaults would do without it. [`SPEC.md`](../SPEC.md) says how
|
||||||
|
`smallwebwaf` behaves.
|
||||||
|
|
||||||
|
## The setup
|
||||||
|
|
||||||
|
Four containers share one Docker network, the shared network:
|
||||||
|
|
||||||
|
- traefik, which routes the site's HTTPS requests to `rproxy`, and redirects
|
||||||
|
plain HTTP to HTTPS;
|
||||||
|
- `rproxy`, in front of gitea;
|
||||||
|
- gitea 1.27.3, as its image comes;
|
||||||
|
- `servewaf`, with a volume for its state files.
|
||||||
|
|
||||||
|
To start it:
|
||||||
|
|
||||||
|
1. In `deploy/gitea/compose.yaml`, replace `git.example.org` with the site's
|
||||||
|
host name.
|
||||||
|
2. Replace each `10.99.0` with the first three numbers of a `/24` subnet that no
|
||||||
|
other network on the host uses. The `/24` must lie inside `10.0.0.0/8`,
|
||||||
|
`172.16.0.0/12` or `192.168.0.0/16`, because `rproxy` believes
|
||||||
|
`X-Forwarded-For` only from those by default. Replace each `fd10:99` with the
|
||||||
|
first two hextets of an unused `/64` subnet whose third and fourth hextets
|
||||||
|
are 0, as in `fd10:99::/64`.
|
||||||
|
3. Set `SMALLWEBWAF_IMAGE` to the `smallwebwaf` image, named by digest.
|
||||||
|
4. Set `GITEA_SECRET_KEY` to the output of
|
||||||
|
`docker run --rm <gitea's image> gitea generate secret SECRET_KEY`, with
|
||||||
|
gitea's image as `deploy/gitea/compose.yaml` names it. Do this once per
|
||||||
|
install, and keep the key.
|
||||||
|
5. Run `docker compose up --detach` in `deploy/gitea`.
|
||||||
|
6. Make gitea's first user with `gitea admin user create`, in gitea's container,
|
||||||
|
as the user `git`.
|
||||||
|
|
||||||
|
Sign-up is off: accounts are made with `gitea admin user create`. gitea starts
|
||||||
|
at its defaults, with SQLite. traefik answers with a certificate it made itself
|
||||||
|
until you add a certificate resolver, as for your other sites. SSH to gitea does
|
||||||
|
not pass through `smallwebwaf`; publish gitea's SSH port yourself if you use it.
|
||||||
|
|
||||||
|
`make gitea-setup` checks the setup. It builds the image, brings the setup up
|
||||||
|
under a name of its own, and checks:
|
||||||
|
|
||||||
|
- that a page load, a push and a clone sent through traefik pass through
|
||||||
|
`rproxy`, and that `servewaf` checks them;
|
||||||
|
- that gitea logs the visitor's address, not `rproxy`'s;
|
||||||
|
- that a request from a container on the shared network skips the checks;
|
||||||
|
- that with `servewaf` stopped, gitea still answers through `rproxy`.
|
||||||
|
|
||||||
|
## Uploads and downloads
|
||||||
|
|
||||||
|
An upload is a request:
|
||||||
|
|
||||||
|
- a git push;
|
||||||
|
- an LFS object;
|
||||||
|
- a container image layer;
|
||||||
|
- a package file;
|
||||||
|
- a release attachment.
|
||||||
|
|
||||||
|
The setup lets an upload of up to 2 GiB, sent in up to 30 minutes, through:
|
||||||
|
|
||||||
|
- `SWRP_MAX_REQUEST_SIZE` is `2G`. At the default, an upload over 100 MiB is cut
|
||||||
|
off.
|
||||||
|
- `SWRP_MAX_REQUEST_TIME_MS` is `1800000`, 30 minutes. At the default, an upload
|
||||||
|
that takes over 60 seconds is cut off.
|
||||||
|
- traefik's `readTimeout` is 30 minutes. traefik's own default of 60 seconds for
|
||||||
|
a whole request would cut off an upload first.
|
||||||
|
|
||||||
|
A clone or a download is a response:
|
||||||
|
|
||||||
|
- `SWRP_MAX_RESPONSE_TIME_MS` is `900000`, 15 minutes, half the default. A clone
|
||||||
|
or download that takes longer is cut off.
|
||||||
|
- `SWRP_MAX_RESPONSE_SIZE` stays at its default, 1 GiB. A repository whose clone
|
||||||
|
is larger needs it raised.
|
||||||
|
|
||||||
|
The byte limits stay at their defaults: no minute limit, 50 GiB an hour and 100
|
||||||
|
GiB a day.
|
||||||
|
|
||||||
|
## The checks
|
||||||
|
|
||||||
|
`SWWAF_WAF_MODE` is `detect`. The Core Rule Set then logs a match, with its rule
|
||||||
|
ids, and refuses nothing. In `block` mode, the default, it refuses everyday
|
||||||
|
gitea searches and file names. The section "What the Core Rule Set refuses in
|
||||||
|
`block` mode" below lists them. Each such refusal counts toward the error burst,
|
||||||
|
so 31 in one minute ban the address.
|
||||||
|
|
||||||
|
Every other check stays at its default. A client may make 120 requests a minute,
|
||||||
|
1,500 an hour and 6,000 a day. Archive downloads and blame and history pages are
|
||||||
|
what scrapers hammer, and these limits do most of the work there.
|
||||||
|
|
||||||
|
## The shared network
|
||||||
|
|
||||||
|
Containers on the shared network skip `servewaf`'s checks:
|
||||||
|
|
||||||
|
- The shared network's subnets are `10.99.0.0/24` and `fd10:99::/64`. Docker
|
||||||
|
gives its containers addresses in the upper half of each, `10.99.0.128/25` and
|
||||||
|
`fd10:99:0:0:8000::/65`, which `SWWAF_ALLOW_NETS` names.
|
||||||
|
- traefik is known on the shared network by the site's host name. So a container
|
||||||
|
there that asks for the site reaches traefik from its own address. An
|
||||||
|
automation fleet in such a container skips the checks.
|
||||||
|
- Each subnet's gateway, `10.99.0.1` and `fd10:99::1`, is the host's own address
|
||||||
|
on it. Requests docker passes on from a published port can come from the
|
||||||
|
gateway, so the gateways are left out of `SWWAF_ALLOW_NETS`. The network has
|
||||||
|
IPv6, so a visitor over IPv6 reaches traefik from its own address, and
|
||||||
|
`servewaf` checks it as itself.
|
||||||
|
- A container on the shared network that asks for the site at the host's public
|
||||||
|
address arrives from the gateway. It is checked like any visitor.
|
||||||
|
- A fleet on another host can have its address added to `SWWAF_ALLOW_NETS`.
|
||||||
|
Without that, it is counted like any client.
|
||||||
|
- traefik replaces the `X-Forwarded-For` a visitor sends, by default. Keep it
|
||||||
|
so: otherwise a visitor could name an address on the shared network, and skip
|
||||||
|
the checks.
|
||||||
|
- Anything on the shared network can reach gitea and `servewaf` around `rproxy`.
|
||||||
|
Put only containers you trust on it.
|
||||||
|
|
||||||
|
## gitea's settings
|
||||||
|
|
||||||
|
- `REVERSE_PROXY_TRUSTED_PROXIES` names `rproxy`'s fixed addresses, `10.99.0.2`
|
||||||
|
and `fd10:99::2`. gitea then takes each visitor's address from the `X-Real-Ip`
|
||||||
|
header, which traefik sets and `rproxy` passes on. Without it, gitea logs
|
||||||
|
every visitor as `rproxy`.
|
||||||
|
- `INSTALL_LOCK` is `true`, as "The setup" above says.
|
||||||
|
|
||||||
|
## Methods
|
||||||
|
|
||||||
|
`rproxy` passes on only these methods: `GET`, `HEAD`, `POST`, `OPTIONS`, `PUT`,
|
||||||
|
`DELETE`, `PATCH` and `QUERY`. gitea 1.27.3 needs no other. Its routes use:
|
||||||
|
|
||||||
|
- pages and the API: `GET`, `HEAD`, `POST`, `PUT`, `PATCH` and `DELETE`;
|
||||||
|
- git's push, clone and fetch over HTTP: `GET`, `POST` and `OPTIONS`;
|
||||||
|
- LFS: `GET`, `POST` and `PUT`;
|
||||||
|
- the container registry and the other package registries: `GET`, `HEAD`,
|
||||||
|
`POST`, `PUT`, `PATCH` and `DELETE`;
|
||||||
|
- Actions runners: `POST`.
|
||||||
|
|
||||||
|
## The log
|
||||||
|
|
||||||
|
`rproxy` and `servewaf` write each query string to the log whole. A token sent
|
||||||
|
in the query string, as `?token=`, is then in the log. Send it in the
|
||||||
|
`Authorization` header instead.
|
||||||
|
|
||||||
|
## Actions runners
|
||||||
|
|
||||||
|
A runner on the shared network skips the checks. One on another host is counted
|
||||||
|
like any client. A runner older than version 0.4 (April 2026) asks for work
|
||||||
|
every 2 seconds, 1,800 requests an hour. While it runs a job, it reports the
|
||||||
|
job's log and state every second instead, two requests a second.
|
||||||
|
|
||||||
|
So one idle runner makes 1,800 requests an hour, more than the hour limit of
|
||||||
|
1,500. A busy one, at two requests a second, passes the minute limit of 120
|
||||||
|
within two minutes. Its address is then banned, its running jobs fail, and each
|
||||||
|
repeat within a day makes the ban longer. Newer runners ask for work less often.
|
||||||
|
Put the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||||
|
|
||||||
|
## What the Core Rule Set refuses in `block` mode
|
||||||
|
|
||||||
|
These notes record what the Core Rule Set does in front of gitea in `block`
|
||||||
|
mode, at the other defaults. They are observations, not requirements.
|
||||||
|
`script/gitea-requests` runs gitea behind `rproxy` and `servewaf` at the
|
||||||
|
defaults. It sends through them a sample of gitea's ordinary use and the
|
||||||
|
requests named below. It reports each request the Core Rule Set refuses, so that
|
||||||
|
these notes can be checked again when gitea or the Core Rule Set changes
|
||||||
|
version. In `detect` mode, as the setup runs it, the same requests are logged
|
||||||
|
with their rule ids, and pass.
|
||||||
|
|
||||||
|
Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts
|
||||||
|
toward). The request log names the rule in `waf_rule_ids`, which
|
||||||
|
`SWWAF_WAF_DISABLED_RULES` can switch off.
|
||||||
|
|
||||||
|
### What it lets through
|
||||||
|
|
||||||
|
The Core Rule Set lets gitea's ordinary use through, apart from the refusals in
|
||||||
|
the sections after this one:
|
||||||
|
|
||||||
|
- browsing and views of files in a repository, with their history, blame and the
|
||||||
|
file tree;
|
||||||
|
- diffs, including their hidden lines and large files, and pull request review;
|
||||||
|
- git's clone, fetch and push over HTTP;
|
||||||
|
- signing in, including the return to the page a visitor came from, and sign-in
|
||||||
|
with Git Credential Manager, git-credential-oauth or tea;
|
||||||
|
- the API's calls for a file and its commits;
|
||||||
|
- pushing and pulling container images and packages;
|
||||||
|
- Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`;
|
||||||
|
- posting issues, pull requests, comments, wiki pages and files saved in the web
|
||||||
|
editor, code included, since no body is read;
|
||||||
|
- the page gitea shows after a change, with its message naming what was done.
|
||||||
|
|
||||||
|
### A query string that reads as an attack
|
||||||
|
|
||||||
|
Most often this is a search. A search term is a query parameter, which the Core
|
||||||
|
Rule Set checks against its lists of system files and commands (see the fourth
|
||||||
|
change under "Attack detection" in `SPEC.md`). A release attachment uploaded
|
||||||
|
through the API as `docker-compose.yml` is refused the same way. Examples:
|
||||||
|
|
||||||
|
- a name on those lists, such as `package.json`, `.gitignore` or
|
||||||
|
`docker-compose.yml`;
|
||||||
|
- text that starts with a command name, such as `python3` or `ssh key`, or has
|
||||||
|
one right after a `/` in its first word, such as `feat/docker-support`;
|
||||||
|
- a shell command with its options or a system path (`ls -la`, `sed -i`,
|
||||||
|
`/bin/sh`), or a command in backticks;
|
||||||
|
- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`) or an SQL
|
||||||
|
statement (`SELECT * FROM users WHERE`);
|
||||||
|
- a URL naming an IP address or `localhost`.
|
||||||
|
|
||||||
|
### A branch, tag or file name that starts with a command name
|
||||||
|
|
||||||
|
The Core Rule Set's list of commands (932260) holds names such as `docker-`,
|
||||||
|
`python3`, `ansible`, `base64` and `whoami`. A query parameter holding a branch,
|
||||||
|
tag or file name that starts with one is refused. So is one that has such a name
|
||||||
|
right after a `/`, such as `feat/docker-support`, `fix/docker-build` or
|
||||||
|
`renovate/docker-build-push-action-6.x`. gitea's own pages send the name in a
|
||||||
|
query parameter that 930120, 932160 and 932260 still check:
|
||||||
|
|
||||||
|
- `refSubUrl`, the branch or tag of a directory listing. A listing sends it when
|
||||||
|
it asks for its entries' last commits in a second request. It does so whenever
|
||||||
|
gitea takes more than a second to work them out;
|
||||||
|
- `name`, when a branch is deleted or restored on the branches page;
|
||||||
|
- `tag`, when a release is started from a tag;
|
||||||
|
- `template`, the file of an issue template, when an issue is opened from it;
|
||||||
|
- `rule_name`, when a branch protection rule is opened for editing in the
|
||||||
|
repository's settings or has just been saved.
|
||||||
|
|
||||||
|
What a visitor sees, for a branch named `docker-build`:
|
||||||
|
|
||||||
|
- A directory listing that makes that second request leaves those last commits
|
||||||
|
out and shows an error.
|
||||||
|
- The branches page shows an error instead of deleting or restoring the branch.
|
||||||
|
- A release started from such a tag gets the `403` answer of `smallwebwaf` in
|
||||||
|
place of its form. So does an issue opened from such a template, and such a
|
||||||
|
rule opened for editing.
|
||||||
|
- Saving such a rule stores it. gitea then returns to the branch settings page,
|
||||||
|
which gets the `403` answer, so the save looks as if it failed.
|
||||||
|
|
||||||
|
### A file name in a query parameter that trips the code rules
|
||||||
|
|
||||||
|
The Core Rule Set's rules for Java, script, PowerShell and SQL find what they
|
||||||
|
look for in some file names. This holds too in the ten query parameters that
|
||||||
|
930120, 932160 and 932260 do not check (see the fourth change under "Attack
|
||||||
|
detection" in `SPEC.md`). Examples:
|
||||||
|
|
||||||
|
- `runtime` in OpenJDK's `src/java.base/share/classes/java/lang/Runtime.java`
|
||||||
|
(944110);
|
||||||
|
- a Java class, as in `java.lang.String.md` (944130);
|
||||||
|
- `document.write.js` (941180);
|
||||||
|
- `process.env.d.ts` (934100);
|
||||||
|
- `__proto__` (934130);
|
||||||
|
- the PowerShell command `Install-Script`, as in `install-script.sh` (932120);
|
||||||
|
- a database's name, as in PostgreSQL's `information_schema.sql` (942140).
|
||||||
|
|
||||||
|
For such a file, these requests are refused:
|
||||||
|
|
||||||
|
- the API's list of its commits (`path`);
|
||||||
|
- the API's call for the contents of files that names it in its query (`body`);
|
||||||
|
- the commit graph of that file alone (`file`);
|
||||||
|
- the hidden lines of its diff (`path`), which stay hidden;
|
||||||
|
- the sign-in page, for a visitor who opens a page of the file that needs
|
||||||
|
signing in, such as its edit page (`redirect_to`). The visitor can still sign
|
||||||
|
in by opening the sign-in page itself.
|
||||||
|
|
||||||
|
### A path
|
||||||
|
|
||||||
|
- A file name ending in `~`, or an `.xhtml` file whose path holds a space.
|
||||||
|
- A path that holds what the Core Rule Set takes for a Java class (944130), as
|
||||||
|
`java.lang.String.md` and
|
||||||
|
`META-INF/services/org.apache.commons.logging.LogFactory` do.
|
||||||
|
- A path that holds what it takes for Node.js code (934100), as
|
||||||
|
`process.env.d.ts` does.
|
||||||
|
- A path with a directory named `...` (930100, 930110), such as `.../notes.txt`.
|
||||||
|
|
||||||
|
For such a file, these are refused:
|
||||||
|
|
||||||
|
- the page of the file;
|
||||||
|
- its raw file;
|
||||||
|
- its history and its blame;
|
||||||
|
- its edit page.
|
||||||
|
|
||||||
|
The directory that holds it lists it, and a clone carries it.
|
||||||
|
|
||||||
|
### A container image push that reuses a layer
|
||||||
|
|
||||||
|
A push can reuse a layer of an image docker has pushed to that gitea before.
|
||||||
|
docker then asks gitea to take the layer from that image, naming it in the query
|
||||||
|
parameter `from`, which 932260 checks. The request is refused when the image's
|
||||||
|
owner starts with an entry on the list of commands. It is refused too when the
|
||||||
|
image's name after the owner starts with one. Examples are `owner/python3-app`,
|
||||||
|
`owner/docker-base`, and any image of an owner named `ansible` or `docker-team`.
|
||||||
|
|
||||||
|
What happens next depends on docker's image store. A docker that keeps its
|
||||||
|
images in containerd's image store fails the push, and fails each new try the
|
||||||
|
same way. A new install of Docker Engine 29 does this. A docker that keeps them
|
||||||
|
in its own older image store uploads the layer instead. The push then succeeds
|
||||||
|
after one refusal for each layer it reuses, and each of those refusals counts
|
||||||
|
toward the error burst ("Attack detection" in `SPEC.md`).
|
||||||
|
|
||||||
|
### Artifact uploads from `actions/upload-artifact@v3`
|
||||||
|
|
||||||
|
This action sends the header `Content-Range`, which the Core Rule Set refuses
|
||||||
|
(920450). The action sends two files at a time, does not retry a `403`, and
|
||||||
|
sends no further file once one is refused. So an upload is refused once or
|
||||||
|
twice, however many files it holds, and the step fails. One upload does not ban
|
||||||
|
the runner. The error burst ("Attack detection" in `SPEC.md`) bans it after more
|
||||||
|
than 30 such refusals from its address in a minute, as when many jobs upload at
|
||||||
|
once. `actions/upload-artifact@v4` does not send the header.
|
||||||
|
|
||||||
|
### What body inspection refuses
|
||||||
|
|
||||||
|
With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In
|
||||||
|
front of gitea it then refuses these, until their rule ids, named in the request
|
||||||
|
log, are added to `SWWAF_WAF_DISABLED_RULES`:
|
||||||
|
|
||||||
|
- issue and comment text, wiki pages and files saved in the web editor that hold
|
||||||
|
shell commands or code (932125, 932235, 932250 and others);
|
||||||
|
- package descriptions that show code;
|
||||||
|
- PyPI uploads (922130);
|
||||||
|
- attachments named like `debug.log` or `config.yml` (932180).
|
||||||
|
|
||||||
|
### What `script/gitea-requests` sends
|
||||||
|
|
||||||
|
Of the requests above, the script sends:
|
||||||
|
|
||||||
|
- for each file named under "A file name in a query parameter that trips the
|
||||||
|
code rules", the API's list of its commits;
|
||||||
|
- for `Runtime.java` alone, the commit graph of that file, the hidden lines of
|
||||||
|
its diff, and the API's call for the contents of files that names it;
|
||||||
|
- the sign-in page for a visitor who opens the edit page of `document.write.js`;
|
||||||
|
- the pages of `java.lang.String.md`,
|
||||||
|
`META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
|
||||||
|
and `.../notes.txt`;
|
||||||
|
- one container image push that reuses a layer, which it does not try again.
|
||||||
|
|
||||||
|
It does not send the raw file, history, blame or edit page of those four files,
|
||||||
|
or the pages of the directories `META-INF/services` and `...`. It also sends a
|
||||||
|
short sample of gitea's ordinary use:
|
||||||
|
|
||||||
|
- signing in;
|
||||||
|
- pages of files and commits, and a diff;
|
||||||
|
- the API;
|
||||||
|
- a push, and clone and fetch over HTTP;
|
||||||
|
- two container image pushes.
|
||||||
@@ -107,6 +107,34 @@ Every one of these lines has:
|
|||||||
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
|
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
|
||||||
request it would have refused. `action` then names what was done, normally
|
request it would have refused. `action` then names what was done, normally
|
||||||
`forward`.
|
`forward`.
|
||||||
|
- `heuristic`: the heuristic that refused the request, in `observe` mode too. It
|
||||||
|
is left out when none did.
|
||||||
|
- `heuristic_figures`: an object of the figures that heuristic acted on. Each
|
||||||
|
heuristic and its figures:
|
||||||
|
- `deny_nets`: `netblock`, the entry of `SWWAF_DENY_NETS` the client is in;
|
||||||
|
- `active_ban`: `netblock`, the ban's netblock, and `ban_expires`;
|
||||||
|
- `countries`: `country`, and `setting`, the country list that refused it;
|
||||||
|
- `blocklists`: `lists`, the blocklist URLs that list the client;
|
||||||
|
- `crowdsec`: `scenario`, the decision's scenario, and `expires`, when the
|
||||||
|
decision ends;
|
||||||
|
- `dnsbl`: `zones`, the DNS blocklist zones that list the client, their keys
|
||||||
|
masked;
|
||||||
|
- `abuseipdb`: `score`, and `min_score`, `SWWAF_ABUSEIPDB_MIN_SCORE`;
|
||||||
|
- `rate_limits`: `window`, `count` and `limit`;
|
||||||
|
- `trap_paths`: `path`, the trap path asked for;
|
||||||
|
- `rule_files`: `rule_id` and `rule_action`, of the rule that refused it;
|
||||||
|
- `header_limits`: `fields`, `fields_limit`, `cookies` and `cookies_limit`;
|
||||||
|
- `core_rule_set`: `score`, and `threshold`, `SWWAF_WAF_ANOMALY_THRESHOLD`;
|
||||||
|
- `refusal_limit`: `window`, `count` and `limit`, of the error burst;
|
||||||
|
- `byte_limits`: `window`, `count` and `limit`, the count in bytes.
|
||||||
|
- `report_heuristic`: the heuristic that banned the client after `servewaf`
|
||||||
|
answered the request, in `observe` mode too: `refusal_limit` or `byte_limits`.
|
||||||
|
- `report_heuristic_figures`: an object of the figures that heuristic acted on,
|
||||||
|
as listed above.
|
||||||
|
|
||||||
|
`byte_limits` bans when the report comes, so only the `report` line names
|
||||||
|
it.
|
||||||
|
|
||||||
- `limit_percent` and `limit_percent_setting`: the client's percentage of the
|
- `limit_percent` and `limit_percent_setting`: the client's percentage of the
|
||||||
rate limits and the setting that lowered it, when one did. `bytes_percent` and
|
rate limits and the setting that lowered it, when one did. `bytes_percent` and
|
||||||
`bytes_percent_setting` do the same for the byte limits.
|
`bytes_percent_setting` do the same for the byte limits.
|
||||||
|
|||||||
+3
-3
@@ -55,10 +55,10 @@ empty-agent user_agent log ^$
|
|||||||
## A file of your own
|
## A file of your own
|
||||||
|
|
||||||
`servewaf` reads the rule files in its own container. Rules of your own go in a
|
`servewaf` reads the rule files in its own container. Rules of your own go in a
|
||||||
file named to sort after the default, such as `50-site.rules`:
|
file named to sort after the default, such as `50-gitea.rules`:
|
||||||
|
|
||||||
```
|
```
|
||||||
# 50-site.rules: WordPress probes, which this site never serves
|
# 50-gitea.rules: WordPress probes, which gitea never serves
|
||||||
wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/)
|
wp-probe path ban (?i)^/(wp-login\.php|xmlrpc\.php|wp-admin/)
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -66,7 +66,7 @@ A `Dockerfile` built on the `smallwebwaf` image, for `servewaf` to run from,
|
|||||||
adds it:
|
adds it:
|
||||||
|
|
||||||
```dockerfile
|
```dockerfile
|
||||||
COPY 50-site.rules /etc/smallwebwaf/rules.d/50-site.rules
|
COPY 50-gitea.rules /etc/smallwebwaf/rules.d/50-gitea.rules
|
||||||
```
|
```
|
||||||
|
|
||||||
Or mount it there. A `path` rule applies on every site behind that `servewaf`.
|
Or mount it there. A `path` rule applies on every site behind that `servewaf`.
|
||||||
+10
-17
@@ -101,18 +101,19 @@ chooses between the two (see "The image's command" below).
|
|||||||
|
|
||||||
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `120`, 2 a second): the most requests a
|
- `SWWAF_RATE_LIMIT_PER_MINUTE` (default `120`, 2 a second): the most requests a
|
||||||
client may make in a minute. A count or `off`. The default leaves room for a
|
client may make in a minute. A count or `off`. The default leaves room for a
|
||||||
person opening two pages at once with an empty cache, up to 60 files each.
|
person opening two gitea pages at once with an empty cache, up to 60 files
|
||||||
|
each.
|
||||||
- `SWWAF_RATE_LIMIT_PER_HOUR` (default `1500`): the most requests a client may
|
- `SWWAF_RATE_LIMIT_PER_HOUR` (default `1500`): the most requests a client may
|
||||||
make in an hour. A count or `off`. It is the main limit against scrapers. A
|
make in an hour. A count or `off`. It is the main limit against scrapers. A
|
||||||
person reading a site for an hour, a page every 20 seconds, makes about 900. A
|
person reading code in gitea for an hour, a page every 20 seconds, makes
|
||||||
scraper making one request every 2 seconds makes 1,800 an hour, more than the
|
about 900. A scraper making one request every 2 seconds makes 1,800 an hour,
|
||||||
limit.
|
more than the limit.
|
||||||
- `SWWAF_RATE_LIMIT_PER_DAY` (default `6000`): the most requests a client may
|
- `SWWAF_RATE_LIMIT_PER_DAY` (default `6000`): the most requests a client may
|
||||||
make in a day. A count or `off`. The default is four hours at the hour limit.
|
make in a day. A count or `off`. The default is four hours at the hour limit.
|
||||||
A scraper making one request every 10 seconds makes 8,640 a day, more than the
|
A scraper making one request every 10 seconds makes 8,640 a day, more than the
|
||||||
limit.
|
limit.
|
||||||
- A CI runner asks its code forge for work all day and can pass the limits. Put
|
- An Actions runner asks gitea for work all day and can pass the limits. Put the
|
||||||
the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. Do the same for a
|
runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. Do the same for a
|
||||||
network where many people share one address, such as an office.
|
network where many people share one address, such as an office.
|
||||||
- Counting cannot catch a client that stays under all three limits (see "Request
|
- Counting cannot catch a client that stays under all three limits (see "Request
|
||||||
rate limits, per client" in [`SPEC.md`](../SPEC.md)).
|
rate limits, per client" in [`SPEC.md`](../SPEC.md)).
|
||||||
@@ -191,15 +192,6 @@ chooses between the two (see "The image's command" below).
|
|||||||
the Core Rule Set reads, a size of at most `64K`. A question carries no more
|
the Core Rule Set reads, a size of at most `64K`. A question carries no more
|
||||||
of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one
|
of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one
|
||||||
byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
|
byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
|
||||||
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule Set
|
|
||||||
exclusions, written as Coraza directives. It is loaded before the Core Rule
|
|
||||||
Set's rules, so it holds only `SecAction` and `SecRule` lines that exclude
|
|
||||||
with `ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. A file that cannot be
|
|
||||||
read or does not parse stops the start, whatever `SWWAF_WAF_MODE` is. So does
|
|
||||||
a file that names a number from 900000 to 900999 outside a comment, alone or
|
|
||||||
in a range. So does a directive that removes or changes rules as the file
|
|
||||||
loads, such as `SecRuleRemoveById`, since none of the Core Rule Set's rules is
|
|
||||||
loaded yet. It has no `_FILE` form.
|
|
||||||
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
|
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
|
||||||
in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
|
in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
|
||||||
|
|
||||||
@@ -354,8 +346,9 @@ asked in the background (see "Answering in time" under "`servewaf`" in
|
|||||||
## `rproxy`
|
## `rproxy`
|
||||||
|
|
||||||
`smallwebwaf rproxy` reads the settings below and no others. They are read once
|
`smallwebwaf rproxy` reads the settings below and no others. They are read once
|
||||||
at start, and an invalid value stops the start. Any can be given as a file, as a
|
at start, and an invalid value stops the start. None can be given as a file,
|
||||||
`SWWAF_` setting can. "`rproxy`" in [`SPEC.md`](../SPEC.md) says what they do.
|
since `rproxy` never touches disk. "`rproxy`" in [`SPEC.md`](../SPEC.md) says
|
||||||
|
what they do.
|
||||||
|
|
||||||
A time here is a whole number of milliseconds above zero, such as `2000`.
|
A time here is a whole number of milliseconds above zero, such as `2000`.
|
||||||
|
|
||||||
|
|||||||
@@ -10,14 +10,12 @@ require (
|
|||||||
github.com/maxmind/mmdbwriter v1.2.0
|
github.com/maxmind/mmdbwriter v1.2.0
|
||||||
github.com/oschwald/maxminddb-golang/v2 v2.7.0
|
github.com/oschwald/maxminddb-golang/v2 v2.7.0
|
||||||
github.com/prometheus/client_golang v1.24.1
|
github.com/prometheus/client_golang v1.24.1
|
||||||
github.com/spf13/viper v1.21.0
|
|
||||||
)
|
)
|
||||||
|
|
||||||
require (
|
require (
|
||||||
github.com/beorn7/perks v1.0.1 // indirect
|
github.com/beorn7/perks v1.0.1 // indirect
|
||||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||||
github.com/corazawaf/libinjection-go v0.3.3 // indirect
|
github.com/corazawaf/libinjection-go v0.3.3 // indirect
|
||||||
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
|
|
||||||
github.com/goccy/go-json v0.10.5 // indirect
|
github.com/goccy/go-json v0.10.5 // indirect
|
||||||
github.com/goccy/go-yaml v1.19.2 // indirect
|
github.com/goccy/go-yaml v1.19.2 // indirect
|
||||||
github.com/gotnospirit/makeplural v0.0.0-20180622080156-a5f48d94d976 // indirect
|
github.com/gotnospirit/makeplural v0.0.0-20180622080156-a5f48d94d976 // indirect
|
||||||
@@ -27,22 +25,14 @@ require (
|
|||||||
github.com/kylelemons/godebug v1.1.0 // indirect
|
github.com/kylelemons/godebug v1.1.0 // indirect
|
||||||
github.com/magefile/mage v1.17.0 // indirect
|
github.com/magefile/mage v1.17.0 // indirect
|
||||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
|
||||||
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
|
|
||||||
github.com/petar-dambovaliev/aho-corasick v0.0.0-20250424160509-463d218d4745 // indirect
|
github.com/petar-dambovaliev/aho-corasick v0.0.0-20250424160509-463d218d4745 // indirect
|
||||||
github.com/prometheus/client_model v0.6.2 // indirect
|
github.com/prometheus/client_model v0.6.2 // indirect
|
||||||
github.com/prometheus/common v0.70.1 // indirect
|
github.com/prometheus/common v0.70.1 // indirect
|
||||||
github.com/prometheus/procfs v0.21.1 // indirect
|
github.com/prometheus/procfs v0.21.1 // indirect
|
||||||
github.com/sagikazarmark/locafero v0.11.0 // indirect
|
|
||||||
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
|
|
||||||
github.com/spf13/afero v1.15.0 // indirect
|
|
||||||
github.com/spf13/cast v1.10.0 // indirect
|
|
||||||
github.com/spf13/pflag v1.0.10 // indirect
|
|
||||||
github.com/subosito/gotenv v1.6.0 // indirect
|
|
||||||
github.com/tidwall/gjson v1.18.0 // indirect
|
github.com/tidwall/gjson v1.18.0 // indirect
|
||||||
github.com/tidwall/match v1.1.1 // indirect
|
github.com/tidwall/match v1.1.1 // indirect
|
||||||
github.com/tidwall/pretty v1.2.1 // indirect
|
github.com/tidwall/pretty v1.2.1 // indirect
|
||||||
github.com/valllabh/ocsf-schema-golang v1.0.3 // indirect
|
github.com/valllabh/ocsf-schema-golang v1.0.3 // indirect
|
||||||
go.yaml.in/yaml/v3 v3.0.5 // indirect
|
|
||||||
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
|
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
|
||||||
golang.org/x/net v0.58.0 // indirect
|
golang.org/x/net v0.58.0 // indirect
|
||||||
golang.org/x/sync v0.23.0 // indirect
|
golang.org/x/sync v0.23.0 // indirect
|
||||||
|
|||||||
@@ -12,12 +12,8 @@ github.com/corazawaf/libinjection-go v0.3.3 h1:NhbXKRfRpqKzBMzv8zpCcnjyEw7BCVhBO
|
|||||||
github.com/corazawaf/libinjection-go v0.3.3/go.mod h1:Ik/+w3UmTWH9yn366RgS9D95K3y7Atb5m/H/gXzzPCk=
|
github.com/corazawaf/libinjection-go v0.3.3/go.mod h1:Ik/+w3UmTWH9yn366RgS9D95K3y7Atb5m/H/gXzzPCk=
|
||||||
github.com/foxcpp/go-mockdns v1.2.0 h1:omK3OrHRD1IWJz1FuFBCFquhXslXoF17OvBS6JPzZF0=
|
github.com/foxcpp/go-mockdns v1.2.0 h1:omK3OrHRD1IWJz1FuFBCFquhXslXoF17OvBS6JPzZF0=
|
||||||
github.com/foxcpp/go-mockdns v1.2.0/go.mod h1:IhLeSFGed3mJIAXPH2aiRQB+kqz7oqu8ld2qVbOu7Wk=
|
github.com/foxcpp/go-mockdns v1.2.0/go.mod h1:IhLeSFGed3mJIAXPH2aiRQB+kqz7oqu8ld2qVbOu7Wk=
|
||||||
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
|
|
||||||
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
|
|
||||||
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
|
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
|
||||||
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
|
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
|
||||||
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
|
|
||||||
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
|
|
||||||
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
|
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
|
||||||
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
|
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
|
||||||
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
|
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
|
||||||
@@ -38,10 +34,6 @@ github.com/kaptinlin/jsonschema v0.4.6 h1:vOSFg5tjmfkOdKg+D6Oo4fVOM/pActWu/ntkPs
|
|||||||
github.com/kaptinlin/jsonschema v0.4.6/go.mod h1:1DUd7r5SdyB2ZnMtyB7uLv64dE3zTFTiYytDCd+AEL0=
|
github.com/kaptinlin/jsonschema v0.4.6/go.mod h1:1DUd7r5SdyB2ZnMtyB7uLv64dE3zTFTiYytDCd+AEL0=
|
||||||
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
|
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
|
||||||
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
||||||
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
|
||||||
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
|
|
||||||
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
|
||||||
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
|
||||||
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
|
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
|
||||||
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
|
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
|
||||||
github.com/magefile/mage v1.17.0 h1:dS4tkq997Ism03akafC8509iqDjeE7TNTexI25Y7sXM=
|
github.com/magefile/mage v1.17.0 h1:dS4tkq997Ism03akafC8509iqDjeE7TNTexI25Y7sXM=
|
||||||
@@ -66,24 +58,8 @@ github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi
|
|||||||
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
|
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
|
||||||
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
||||||
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
||||||
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
|
|
||||||
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
|
|
||||||
github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
|
|
||||||
github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
|
|
||||||
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 h1:+jumHNA0Wrelhe64i8F6HNlS8pkoyMv5sreGx2Ry5Rw=
|
|
||||||
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8/go.mod h1:3n1Cwaq1E1/1lhQhtRK2ts/ZwZEhjcQeJQ1RuC6Q/8U=
|
|
||||||
github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
|
|
||||||
github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
|
|
||||||
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
|
|
||||||
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
|
|
||||||
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
|
|
||||||
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
|
||||||
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
|
|
||||||
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
|
|
||||||
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
|
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
|
||||||
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
|
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
|
||||||
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
|
|
||||||
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
|
|
||||||
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
|
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
|
||||||
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||||
|
|||||||
@@ -39,7 +39,7 @@ const (
|
|||||||
ntfyURL = "https://ntfy.example/smallwebwaf-alerts"
|
ntfyURL = "https://ntfy.example/smallwebwaf-alerts"
|
||||||
ntfyToken = "tk_0123456789abcdefghijklmnopq"
|
ntfyToken = "tk_0123456789abcdefghijklmnopq"
|
||||||
// instance is the instance name every alert gives.
|
// instance is the instance name every alert gives.
|
||||||
instance = "fsn1app1/blog"
|
instance = "fsn1app1/gitea"
|
||||||
// started is when each test starts, as an alert gives it, and
|
// started is when each test starts, as an alert gives it, and
|
||||||
// anHourOn an hour later.
|
// anHourOn an hour later.
|
||||||
started = "2000-01-01T00:00:00Z"
|
started = "2000-01-01T00:00:00Z"
|
||||||
@@ -965,8 +965,8 @@ func TestEachDestinationIsSentOnlyTheAlertsWaitingForIt(t *testing.T) {
|
|||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
|
|
||||||
wantBodies(t, standIns[alerts.DestinationSlack],
|
wantBodies(t, standIns[alerts.DestinationSlack],
|
||||||
`{"text":"*fsn1app1/blog: file_error*\nsecond"}`,
|
`{"text":"*fsn1app1/gitea: file_error*\nsecond"}`,
|
||||||
`{"text":"*fsn1app1/blog: file_error*\nthird"}`)
|
`{"text":"*fsn1app1/gitea: file_error*\nthird"}`)
|
||||||
wantBodies(t, standIns[alerts.DestinationNtfy], "fourth")
|
wantBodies(t, standIns[alerts.DestinationNtfy], "fourth")
|
||||||
|
|
||||||
// alerts.json then lists Slack and ntfy alone, with no alert waiting.
|
// alerts.json then lists Slack and ntfy alone, with no alert waiting.
|
||||||
@@ -1265,7 +1265,7 @@ func anAlertForEachEvent() []eventMessage {
|
|||||||
Reason: "requests per hour over the limit of 10000",
|
Reason: "requests per hour over the limit of 10000",
|
||||||
Detail: map[string]any{"mode": "observe"},
|
Detail: map[string]any{"mode": "observe"},
|
||||||
},
|
},
|
||||||
"fsn1app1/blog: ban", "default no_entry",
|
"fsn1app1/gitea: ban", "default no_entry",
|
||||||
"requests per hour over the limit of 10000\nclient: 203.0.113.9\n" +
|
"requests per hour over the limit of 10000\nclient: 203.0.113.9\n" +
|
||||||
"netblock: 203.0.113.0/24\ncountry: DE\nmode: observe",
|
"netblock: 203.0.113.0/24\ncountry: DE\nmode: observe",
|
||||||
},
|
},
|
||||||
@@ -1275,7 +1275,7 @@ func anAlertForEachEvent() []eventMessage {
|
|||||||
Netblock: netip.MustParsePrefix("198.51.100.7/32"), Country: "FR",
|
Netblock: netip.MustParsePrefix("198.51.100.7/32"), Country: "FR",
|
||||||
Reason: "matched the rule env-file",
|
Reason: "matched the rule env-file",
|
||||||
},
|
},
|
||||||
"fsn1app1/blog: permanent_ban", "high no_entry",
|
"fsn1app1/gitea: permanent_ban", "high no_entry",
|
||||||
"matched the rule env-file\nclient: 198.51.100.7\n" +
|
"matched the rule env-file\nclient: 198.51.100.7\n" +
|
||||||
"netblock: 198.51.100.7/32\ncountry: FR",
|
"netblock: 198.51.100.7/32\ncountry: FR",
|
||||||
},
|
},
|
||||||
@@ -1286,7 +1286,7 @@ func anAlertForEachEvent() []eventMessage {
|
|||||||
Reason: "refused by the Core Rule Set",
|
Reason: "refused by the Core Rule Set",
|
||||||
Host: "app.example",
|
Host: "app.example",
|
||||||
},
|
},
|
||||||
"fsn1app1/blog: waf_block", "default shield",
|
"fsn1app1/gitea: waf_block", "default shield",
|
||||||
"refused by the Core Rule Set\nhost: app.example\nclient: 192.0.2.1\n" +
|
"refused by the Core Rule Set\nhost: app.example\nclient: 192.0.2.1\n" +
|
||||||
"netblock: 192.0.2.1/32",
|
"netblock: 192.0.2.1/32",
|
||||||
},
|
},
|
||||||
@@ -1295,7 +1295,7 @@ func anAlertForEachEvent() []eventMessage {
|
|||||||
Event: alerts.EventAnomaly, Netblock: netip.MustParsePrefix("192.0.2.0/24"),
|
Event: alerts.EventAnomaly, Netblock: netip.MustParsePrefix("192.0.2.0/24"),
|
||||||
Reason: "requests per minute over the threshold of 5000",
|
Reason: "requests per minute over the threshold of 5000",
|
||||||
},
|
},
|
||||||
"fsn1app1/blog: anomaly", "high chart_with_upwards_trend",
|
"fsn1app1/gitea: anomaly", "high chart_with_upwards_trend",
|
||||||
"requests per minute over the threshold of 5000\nnetblock: 192.0.2.0/24",
|
"requests per minute over the threshold of 5000\nnetblock: 192.0.2.0/24",
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -1304,7 +1304,7 @@ func anAlertForEachEvent() []eventMessage {
|
|||||||
Netblock: netip.MustParsePrefix("192.0.2.2/32"),
|
Netblock: netip.MustParsePrefix("192.0.2.2/32"),
|
||||||
Reason: "listed by a DNS blocklist",
|
Reason: "listed by a DNS blocklist",
|
||||||
},
|
},
|
||||||
"fsn1app1/blog: reputation_hit", "low label",
|
"fsn1app1/gitea: reputation_hit", "low label",
|
||||||
"listed by a DNS blocklist\nclient: 192.0.2.2\nnetblock: 192.0.2.2/32",
|
"listed by a DNS blocklist\nclient: 192.0.2.2\nnetblock: 192.0.2.2/32",
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -1312,7 +1312,7 @@ func anAlertForEachEvent() []eventMessage {
|
|||||||
Event: alerts.EventSourceFailure, Reason: "asking GeoJS failed",
|
Event: alerts.EventSourceFailure, Reason: "asking GeoJS failed",
|
||||||
Detail: map[string]any{"source": "geojs"},
|
Detail: map[string]any{"source": "geojs"},
|
||||||
},
|
},
|
||||||
"fsn1app1/blog: source_failure", "high warning",
|
"fsn1app1/gitea: source_failure", "high warning",
|
||||||
"asking GeoJS failed\nsource: geojs",
|
"asking GeoJS failed\nsource: geojs",
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -1323,7 +1323,7 @@ func anAlertForEachEvent() []eventMessage {
|
|||||||
"file": "/etc/smallwebwaf/rules.d/50-app.rules", "error": "line 2: no action",
|
"file": "/etc/smallwebwaf/rules.d/50-app.rules", "error": "line 2: no action",
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
"fsn1app1/blog: file_error", "high warning",
|
"fsn1app1/gitea: file_error", "high warning",
|
||||||
"a rule file has an error, and the rules stay as they were\n" +
|
"a rule file has an error, and the rules stay as they were\n" +
|
||||||
"file: /etc/smallwebwaf/rules.d/50-app.rules\nerror: line 2: no action",
|
"file: /etc/smallwebwaf/rules.d/50-app.rules\nerror: line 2: no action",
|
||||||
},
|
},
|
||||||
|
|||||||
+30
-64
@@ -2,8 +2,9 @@
|
|||||||
// read. Every setting is an environment variable. The settings of `smallwebwaf
|
// read. Every setting is an environment variable. The settings of `smallwebwaf
|
||||||
// servewaf` start with SWWAF_, every one of them has a default, and most can be
|
// servewaf` start with SWWAF_, every one of them has a default, and most can be
|
||||||
// given as a file instead. Those of `smallwebwaf rproxy` start with SWRP_. Each
|
// given as a file instead. Those of `smallwebwaf rproxy` start with SWRP_. Each
|
||||||
// of them but SWRP_WAF_BACKEND has a default, and each can be given as a file
|
// of them but SWRP_WAF_BACKEND has a default, and none can be given as a file,
|
||||||
// too. SMALLWEBWAF_COMMAND chooses which of the two the image runs.
|
// since rproxy never touches disk. SMALLWEBWAF_COMMAND chooses which of the two
|
||||||
|
// the image runs.
|
||||||
package config
|
package config
|
||||||
|
|
||||||
import (
|
import (
|
||||||
@@ -25,8 +26,6 @@ import (
|
|||||||
"unicode"
|
"unicode"
|
||||||
"unicode/utf8"
|
"unicode/utf8"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/remotelog"
|
"sneak.berlin/go/smallwebwaf/internal/remotelog"
|
||||||
@@ -222,15 +221,12 @@ type Config struct {
|
|||||||
// WAFExemptPaths the path prefixes it does not inspect
|
// WAFExemptPaths the path prefixes it does not inspect
|
||||||
// (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body
|
// (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body
|
||||||
// it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
// it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
||||||
// CoreRuleSetExclusionsFile is the path of the file of its exclusions
|
WAFMode string
|
||||||
// (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while there is none.
|
WAFParanoiaLevel int
|
||||||
WAFMode string
|
WAFAnomalyThreshold int
|
||||||
WAFParanoiaLevel int
|
WAFDisabledRules []int
|
||||||
WAFAnomalyThreshold int
|
WAFExemptPaths []string
|
||||||
WAFDisabledRules []int
|
WAFBodyLimit int64
|
||||||
WAFExemptPaths []string
|
|
||||||
WAFBodyLimit int64
|
|
||||||
CoreRuleSetExclusionsFile string
|
|
||||||
// TrapPaths are the paths a request for which is a clear sign of
|
// TrapPaths are the paths a request for which is a clear sign of
|
||||||
// attack (SWWAF_TRAP_PATHS), each starting with / and without a ?.
|
// attack (SWWAF_TRAP_PATHS), each starting with / and without a ?.
|
||||||
TrapPaths []string
|
TrapPaths []string
|
||||||
@@ -413,7 +409,7 @@ var (
|
|||||||
errNoCertificate = errors.New("holds no PEM certificate")
|
errNoCertificate = errors.New("holds no PEM certificate")
|
||||||
errNotFacility = errors.New("is not a syslog facility such as local0 or daemon")
|
errNotFacility = errors.New("is not a syslog facility such as local0 or daemon")
|
||||||
errNotAppName = errors.New(
|
errNotAppName = errors.New(
|
||||||
"is not 1 to 48 printable ASCII characters without a space, such as blog")
|
"is not 1 to 48 printable ASCII characters without a space, such as gitea")
|
||||||
errSetTwice = errors.New("set only one of them")
|
errSetTwice = errors.New("set only one of them")
|
||||||
errNotWebhookURL = errors.New(
|
errNotWebhookURL = errors.New(
|
||||||
"is not an http or https URL without a user or a fragment, " +
|
"is not an http or https URL without a user or a fragment, " +
|
||||||
@@ -454,16 +450,15 @@ var (
|
|||||||
"is in SWWAF_BLOCKLIST_URLS or is SWWAF_ASN_LIMIT_PERCENT_URL too")
|
"is in SWWAF_BLOCKLIST_URLS or is SWWAF_ASN_LIMIT_PERCENT_URL too")
|
||||||
)
|
)
|
||||||
|
|
||||||
// FromEnvironment reads the settings through v, normally the viper
|
// FromEnvironment reads the settings with lookupEnv, normally
|
||||||
// NewViper returns, bound to the process's environment. A setting may
|
// os.LookupEnv. A setting may instead be given as a file: the variable
|
||||||
// instead be given as a file: the variable named by the setting's name
|
// named by the setting's name with _FILE added names the file, which is
|
||||||
// with _FILE added names the file, which is read now (see lookup). A
|
// read now (see lookup). A setting that is not set takes its default. A
|
||||||
// setting that is not set takes its default. A setting that is set but
|
// setting that is set but invalid is an error that names it.
|
||||||
// invalid is an error that names it.
|
|
||||||
//
|
//
|
||||||
//nolint:funlen // one line for each setting, a list that grows with them
|
//nolint:funlen // one line for each setting, a list that grows with them
|
||||||
func FromEnvironment(v *viper.Viper) (*Config, error) {
|
func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
|
||||||
env := &environment{v: v, readFiles: true}
|
env := &environment{lookupEnv: lookupEnv}
|
||||||
cfg := &Config{
|
cfg := &Config{
|
||||||
ListenAddr: env.address("SWWAF_LISTEN_ADDR"),
|
ListenAddr: env.address("SWWAF_LISTEN_ADDR"),
|
||||||
InstanceName: env.instanceName(),
|
InstanceName: env.instanceName(),
|
||||||
@@ -528,10 +523,8 @@ func FromEnvironment(v *viper.Viper) (*Config, error) {
|
|||||||
WAFAnomalyThreshold: env.numberOrOff("SWWAF_WAF_ANOMALY_THRESHOLD", "5"),
|
WAFAnomalyThreshold: env.numberOrOff("SWWAF_WAF_ANOMALY_THRESHOLD", "5"),
|
||||||
WAFDisabledRules: env.ruleIDs("SWWAF_WAF_DISABLED_RULES",
|
WAFDisabledRules: env.ruleIDs("SWWAF_WAF_DISABLED_RULES",
|
||||||
"920340,920420,920440,920640,930130,930140"),
|
"920340,920420,920440,920640,930130,930140"),
|
||||||
WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
|
WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
|
||||||
WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
|
WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
|
||||||
CoreRuleSetExclusionsFile: env.filePath(
|
|
||||||
"SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"),
|
|
||||||
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
|
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
|
||||||
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
|
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
|
||||||
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
|
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
|
||||||
@@ -582,8 +575,8 @@ func FromEnvironment(v *viper.Viper) (*Config, error) {
|
|||||||
// Command reads only SMALLWEBWAF_COMMAND, the command the image runs and its
|
// Command reads only SMALLWEBWAF_COMMAND, the command the image runs and its
|
||||||
// health check checks: rproxy when it is unset or empty, or servewaf. It cannot
|
// health check checks: rproxy when it is unset or empty, or servewaf. It cannot
|
||||||
// be given as a file. Any other value is an error that names the setting.
|
// be given as a file. Any other value is an error that names the setting.
|
||||||
func Command(v *viper.Viper) (string, error) {
|
func Command(lookupEnv func(string) (string, bool)) (string, error) {
|
||||||
command, _ := readSetting(v, "SMALLWEBWAF_COMMAND")
|
command, _ := lookupEnv("SMALLWEBWAF_COMMAND")
|
||||||
|
|
||||||
switch command {
|
switch command {
|
||||||
case "", "rproxy":
|
case "", "rproxy":
|
||||||
@@ -598,8 +591,8 @@ func Command(v *viper.Viper) (string, error) {
|
|||||||
// ServeWAFListenAddr reads only SWWAF_LISTEN_ADDR, which may be given as a
|
// ServeWAFListenAddr reads only SWWAF_LISTEN_ADDR, which may be given as a
|
||||||
// file, as FromEnvironment does. The health check in servewaf's container needs
|
// file, as FromEnvironment does. The health check in servewaf's container needs
|
||||||
// no other setting, so it reads no other, nor a file that another names.
|
// no other setting, so it reads no other, nor a file that another names.
|
||||||
func ServeWAFListenAddr(v *viper.Viper) (string, error) {
|
func ServeWAFListenAddr(lookupEnv func(string) (string, bool)) (string, error) {
|
||||||
env := &environment{v: v, readFiles: true}
|
env := &environment{lookupEnv: lookupEnv}
|
||||||
listenAddr := env.address("SWWAF_LISTEN_ADDR")
|
listenAddr := env.address("SWWAF_LISTEN_ADDR")
|
||||||
|
|
||||||
return listenAddr, env.err
|
return listenAddr, env.err
|
||||||
@@ -609,8 +602,8 @@ func ServeWAFListenAddr(v *viper.Viper) (string, error) {
|
|||||||
// file, as FromEnvironment does, so that the line saying a setting is
|
// file, as FromEnvironment does, so that the line saying a setting is
|
||||||
// invalid carries it too. A file that cannot be read gives the default
|
// invalid carries it too. A file that cannot be read gives the default
|
||||||
// here, and FromEnvironment then stops the start over it.
|
// here, and FromEnvironment then stops the start over it.
|
||||||
func InstanceName(v *viper.Viper) string {
|
func InstanceName(lookupEnv func(string) (string, bool)) string {
|
||||||
env := &environment{v: v, readFiles: true}
|
env := &environment{lookupEnv: lookupEnv}
|
||||||
|
|
||||||
return env.instanceName()
|
return env.instanceName()
|
||||||
}
|
}
|
||||||
@@ -626,26 +619,13 @@ func (c *Config) LogValue() slog.Value {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// environment is where FromEnvironment reads the settings: it notes each
|
// environment is where FromEnvironment reads the settings: it notes each
|
||||||
// value for the log, and keeps the first error. It reads every setting
|
// value for the log, and keeps the first error.
|
||||||
// through v, the viper NewViper returns, and, unless readFiles is false,
|
|
||||||
// honours each setting's _FILE form too.
|
|
||||||
type environment struct {
|
type environment struct {
|
||||||
v *viper.Viper
|
lookupEnv func(string) (string, bool)
|
||||||
readFiles bool
|
|
||||||
settings []slog.Attr
|
settings []slog.Attr
|
||||||
err error
|
err error
|
||||||
}
|
}
|
||||||
|
|
||||||
// readSetting returns a setting's value and whether it is set, reading it
|
|
||||||
// through v.
|
|
||||||
func readSetting(v *viper.Viper, name string) (string, bool) {
|
|
||||||
if !v.IsSet(name) {
|
|
||||||
return "", false
|
|
||||||
}
|
|
||||||
|
|
||||||
return v.GetString(name), true
|
|
||||||
}
|
|
||||||
|
|
||||||
// value returns a setting's value, or its default when it is not set,
|
// value returns a setting's value, or its default when it is not set,
|
||||||
// and notes it for the log.
|
// and notes it for the log.
|
||||||
func (e *environment) value(name, defaultValue string) string {
|
func (e *environment) value(name, defaultValue string) string {
|
||||||
@@ -664,14 +644,10 @@ func (e *environment) value(name, defaultValue string) string {
|
|||||||
// names, less one newline at their end. It notes that file's path for the
|
// names, less one newline at their end. It notes that file's path for the
|
||||||
// log. Both variables set, or a file that cannot be read, is an error.
|
// log. Both variables set, or a file that cannot be read, is an error.
|
||||||
func (e *environment) lookup(name string) (string, bool) {
|
func (e *environment) lookup(name string) (string, bool) {
|
||||||
value, set := readSetting(e.v, name)
|
value, set := e.lookupEnv(name)
|
||||||
if !e.readFiles {
|
|
||||||
return value, set
|
|
||||||
}
|
|
||||||
|
|
||||||
fileName := name + "_FILE"
|
fileName := name + "_FILE"
|
||||||
|
|
||||||
path, inFile := readSetting(e.v, fileName)
|
path, inFile := e.lookupEnv(fileName)
|
||||||
if !inFile {
|
if !inFile {
|
||||||
return value, set
|
return value, set
|
||||||
}
|
}
|
||||||
@@ -1213,21 +1189,11 @@ func (e *environment) logRemoteURL(name string) *url.URL {
|
|||||||
return remote
|
return remote
|
||||||
}
|
}
|
||||||
|
|
||||||
// filePath reads a setting that is the path of a file, "" while it is
|
|
||||||
// unset. Its value names a file already, so, unlike the other settings,
|
|
||||||
// it has no _FILE form. Whoever uses the file reads it.
|
|
||||||
func (e *environment) filePath(name string) string {
|
|
||||||
path, _ := readSetting(e.v, name)
|
|
||||||
e.settings = append(e.settings, slog.String(name, path))
|
|
||||||
|
|
||||||
return path
|
|
||||||
}
|
|
||||||
|
|
||||||
// certificates reads a setting that is the path of a file of PEM
|
// certificates reads a setting that is the path of a file of PEM
|
||||||
// certificates. Unset or empty, it is nil. Its value names a file
|
// certificates. Unset or empty, it is nil. Its value names a file
|
||||||
// already, so, unlike the other settings, it has no _FILE form.
|
// already, so, unlike the other settings, it has no _FILE form.
|
||||||
func (e *environment) certificates(name string) *x509.CertPool {
|
func (e *environment) certificates(name string) *x509.CertPool {
|
||||||
path, _ := readSetting(e.v, name)
|
path, _ := e.lookupEnv(name)
|
||||||
e.settings = append(e.settings, slog.String(name, path))
|
e.settings = append(e.settings, slog.String(name, path))
|
||||||
|
|
||||||
if path == "" {
|
if path == "" {
|
||||||
|
|||||||
@@ -17,8 +17,6 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
)
|
)
|
||||||
@@ -89,7 +87,6 @@ const (
|
|||||||
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
|
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
|
||||||
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
|
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
|
||||||
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
|
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
|
||||||
exclusionsFile = "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"
|
|
||||||
trapPaths = "SWWAF_TRAP_PATHS"
|
trapPaths = "SWWAF_TRAP_PATHS"
|
||||||
errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
|
errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
|
||||||
logRemoteURL = "SWWAF_LOG_REMOTE_URL"
|
logRemoteURL = "SWWAF_LOG_REMOTE_URL"
|
||||||
@@ -200,7 +197,7 @@ const (
|
|||||||
// instance is an SWWAF_INSTANCE_NAME that is a valid app name too, and
|
// instance is an SWWAF_INSTANCE_NAME that is a valid app name too, and
|
||||||
// remoteURL an SWWAF_LOG_REMOTE_URL, for the tests that send the lines.
|
// remoteURL an SWWAF_LOG_REMOTE_URL, for the tests that send the lines.
|
||||||
const (
|
const (
|
||||||
instance = "fsn1app1/blog"
|
instance = "fsn1app1/gitea"
|
||||||
remoteURL = "syslog+udp://192.0.2.1:514"
|
remoteURL = "syslog+udp://192.0.2.1:514"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -220,22 +217,18 @@ const (
|
|||||||
// environment is a set of environment variables, for FromEnvironment.
|
// environment is a set of environment variables, for FromEnvironment.
|
||||||
type environment map[string]string
|
type environment map[string]string
|
||||||
|
|
||||||
// viper returns a viper Set with each of the variables, as the viper
|
// lookupEnv reads one of the variables, as os.LookupEnv does.
|
||||||
// NewViper returns one bound to a process's environment.
|
func (e environment) lookupEnv(name string) (string, bool) {
|
||||||
func (e environment) viper() *viper.Viper {
|
value, ok := e[name]
|
||||||
v := viper.New()
|
|
||||||
for name, value := range e {
|
|
||||||
v.Set(name, value)
|
|
||||||
}
|
|
||||||
|
|
||||||
return v
|
return value, ok
|
||||||
}
|
}
|
||||||
|
|
||||||
// fromEnvironment reads the settings from env, which must be valid.
|
// fromEnvironment reads the settings from env, which must be valid.
|
||||||
func fromEnvironment(t *testing.T, env environment) *config.Config {
|
func fromEnvironment(t *testing.T, env environment) *config.Config {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
cfg, err := config.FromEnvironment(env.viper())
|
cfg, err := config.FromEnvironment(env.lookupEnv)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("settings %v: %v", env, err)
|
t.Fatalf("settings %v: %v", env, err)
|
||||||
}
|
}
|
||||||
@@ -462,7 +455,7 @@ func TestPathPrefixNotStartingWithSlashStopsTheStart(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(
|
_, err := config.FromEnvironment(
|
||||||
environment{rateLimitExemptPaths: "/favicon.ico,assets/"}.viper())
|
environment{rateLimitExemptPaths: "/favicon.ico,assets/"}.lookupEnv)
|
||||||
|
|
||||||
want := rateLimitExemptPaths + `: "assets/" is not a path prefix ` +
|
want := rateLimitExemptPaths + `: "assets/" is not a path prefix ` +
|
||||||
`starting with /, such as /assets/`
|
`starting with /, such as /assets/`
|
||||||
@@ -517,7 +510,7 @@ func TestInvalidTrapPathOrErrorBurstThresholdStopsTheStart(t *testing.T) {
|
|||||||
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper())
|
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
|
||||||
|
|
||||||
want := tc.name + ": " + tc.want
|
want := tc.name + ": " + tc.want
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -546,14 +539,13 @@ func TestCoreRuleSetSettings(t *testing.T) {
|
|||||||
environment{
|
environment{
|
||||||
wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10",
|
wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10",
|
||||||
wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/",
|
wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/",
|
||||||
wafBodyLimit: "32K", exclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
|
wafBodyLimit: "32K",
|
||||||
},
|
},
|
||||||
config.Config{
|
config.Config{
|
||||||
WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10,
|
WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10,
|
||||||
WAFDisabledRules: []int{942100, 920350},
|
WAFDisabledRules: []int{942100, 920350},
|
||||||
WAFExemptPaths: []string{"/api/", "/static/"},
|
WAFExemptPaths: []string{"/api/", "/static/"},
|
||||||
WAFBodyLimit: 32 << 10,
|
WAFBodyLimit: 32 << 10,
|
||||||
CoreRuleSetExclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
|
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
@@ -573,8 +565,7 @@ func TestCoreRuleSetSettings(t *testing.T) {
|
|||||||
WAFMode: cfg.WAFMode, WAFParanoiaLevel: cfg.WAFParanoiaLevel,
|
WAFMode: cfg.WAFMode, WAFParanoiaLevel: cfg.WAFParanoiaLevel,
|
||||||
WAFAnomalyThreshold: cfg.WAFAnomalyThreshold,
|
WAFAnomalyThreshold: cfg.WAFAnomalyThreshold,
|
||||||
WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths,
|
WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths,
|
||||||
WAFBodyLimit: cfg.WAFBodyLimit,
|
WAFBodyLimit: cfg.WAFBodyLimit,
|
||||||
CoreRuleSetExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
|
||||||
}
|
}
|
||||||
if !reflect.DeepEqual(got, tc.want) {
|
if !reflect.DeepEqual(got, tc.want) {
|
||||||
t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want)
|
t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want)
|
||||||
@@ -645,7 +636,7 @@ func TestInvalidCoreRuleSetSettingStopsTheStart(t *testing.T) {
|
|||||||
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper())
|
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
|
||||||
|
|
||||||
want := tc.name + ": " + tc.want
|
want := tc.name + ": " + tc.want
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -659,11 +650,11 @@ func TestInstanceNameAndLoggedHeadersAsSet(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
cfg := fromEnvironment(t, environment{
|
cfg := fromEnvironment(t, environment{
|
||||||
instanceName: "fsn1app1/blog",
|
instanceName: "fsn1app1/gitea",
|
||||||
logRequestHeaders: " Accept , X-Custom",
|
logRequestHeaders: " Accept , X-Custom",
|
||||||
})
|
})
|
||||||
|
|
||||||
if cfg.InstanceName != "fsn1app1/blog" ||
|
if cfg.InstanceName != "fsn1app1/gitea" ||
|
||||||
!slices.Equal(cfg.LogRequestHeaders, []string{"accept", "x-custom"}) {
|
!slices.Equal(cfg.LogRequestHeaders, []string{"accept", "x-custom"}) {
|
||||||
t.Errorf("%s is %q and %s gives %v", instanceName, cfg.InstanceName,
|
t.Errorf("%s is %q and %s gives %v", instanceName, cfg.InstanceName,
|
||||||
logRequestHeaders, cfg.LogRequestHeaders)
|
logRequestHeaders, cfg.LogRequestHeaders)
|
||||||
@@ -770,7 +761,7 @@ func TestInvalidRemoteLogSettingStopsTheStart(t *testing.T) {
|
|||||||
{logRemoteAppName, ""}, {logRemoteAppName, "my app"},
|
{logRemoteAppName, ""}, {logRemoteAppName, "my app"},
|
||||||
{logRemoteAppName, "gitéa"}, {logRemoteAppName, strings.Repeat("a", 49)},
|
{logRemoteAppName, "gitéa"}, {logRemoteAppName, strings.Repeat("a", 49)},
|
||||||
} {
|
} {
|
||||||
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper())
|
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
|
||||||
if err == nil || !strings.HasPrefix(err.Error(), tc.name+": ") {
|
if err == nil || !strings.HasPrefix(err.Error(), tc.name+": ") {
|
||||||
t.Errorf("%s=%q: error %v, want one naming it", tc.name, tc.value, err)
|
t.Errorf("%s=%q: error %v, want one naming it", tc.name, tc.value, err)
|
||||||
}
|
}
|
||||||
@@ -787,7 +778,7 @@ func TestRemoteLogCAFileWithoutCertificateStopsTheStart(t *testing.T) {
|
|||||||
t.Fatalf("write %s: %v", caFile, err)
|
t.Fatalf("write %s: %v", caFile, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
_, err = config.FromEnvironment(environment{logRemoteTLSCAFile: caFile}.viper())
|
_, err = config.FromEnvironment(environment{logRemoteTLSCAFile: caFile}.lookupEnv)
|
||||||
|
|
||||||
want := logRemoteTLSCAFile + `: "` + caFile + `" holds no PEM certificate`
|
want := logRemoteTLSCAFile + `: "` + caFile + `" holds no PEM certificate`
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -802,11 +793,11 @@ func TestInstanceNameNotAnAppNameStopsTheStartOnlyWhileSending(t *testing.T) {
|
|||||||
|
|
||||||
sending := environment{logRemoteURL: remoteURL, instanceName: spaced}
|
sending := environment{logRemoteURL: remoteURL, instanceName: spaced}
|
||||||
|
|
||||||
_, err := config.FromEnvironment(sending.viper())
|
_, err := config.FromEnvironment(sending.lookupEnv)
|
||||||
|
|
||||||
want := logRemoteAppName + `: is unset, and ` + instanceName +
|
want := logRemoteAppName + `: is unset, and ` + instanceName +
|
||||||
` "fsn1 app1", its default, is not 1 to 48 printable ASCII characters ` +
|
` "fsn1 app1", its default, is not 1 to 48 printable ASCII characters ` +
|
||||||
`without a space, such as blog`
|
`without a space, such as gitea`
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
t.Errorf("error %v, want %s", err, want)
|
t.Errorf("error %v, want %s", err, want)
|
||||||
}
|
}
|
||||||
@@ -832,10 +823,10 @@ func TestAppNameSetStopsTheStartWhileSending(t *testing.T) {
|
|||||||
logRemoteURL: remoteURL,
|
logRemoteURL: remoteURL,
|
||||||
instanceName: instance,
|
instanceName: instance,
|
||||||
logRemoteAppName: "my app",
|
logRemoteAppName: "my app",
|
||||||
}.viper())
|
}.lookupEnv)
|
||||||
|
|
||||||
want := logRemoteAppName + `: "my app" is not 1 to 48 printable ASCII ` +
|
want := logRemoteAppName + `: "my app" is not 1 to 48 printable ASCII ` +
|
||||||
`characters without a space, such as blog`
|
`characters without a space, such as gitea`
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
t.Errorf("error %v, want %s", err, want)
|
t.Errorf("error %v, want %s", err, want)
|
||||||
}
|
}
|
||||||
@@ -963,7 +954,7 @@ func TestWebhookHeadersAreLoggedMaskedAndNeverShown(t *testing.T) {
|
|||||||
// An item that is not a header is named by its place, not shown.
|
// An item that is not a header is named by its place, not shown.
|
||||||
_, err := config.FromEnvironment(environment{
|
_, err := config.FromEnvironment(environment{
|
||||||
alertWebhookHeaders: "X-Team:ops," + secret,
|
alertWebhookHeaders: "X-Team:ops," + secret,
|
||||||
}.viper())
|
}.lookupEnv)
|
||||||
|
|
||||||
want := alertWebhookHeaders + ": item 2 is not a header name followed by : " +
|
want := alertWebhookHeaders + ": item 2 is not a header name followed by : " +
|
||||||
"and the header's value, such as Authorization:Bearer <token>"
|
"and the header's value, such as Authorization:Bearer <token>"
|
||||||
@@ -1000,7 +991,7 @@ func TestWebhookURLIsLoggedWithoutItsPathOrQueryAndNeverShown(t *testing.T) {
|
|||||||
"ftp://hooks.example/services/" + secret,
|
"ftp://hooks.example/services/" + secret,
|
||||||
"https://hooks.example/services/%zz" + secret,
|
"https://hooks.example/services/%zz" + secret,
|
||||||
} {
|
} {
|
||||||
_, err := config.FromEnvironment(environment{alertWebhookURL: value}.viper())
|
_, err := config.FromEnvironment(environment{alertWebhookURL: value}.lookupEnv)
|
||||||
|
|
||||||
want := alertWebhookURL + ": is not an http or https URL without a user or " +
|
want := alertWebhookURL + ": is not an http or https URL without a user or " +
|
||||||
"a fragment, such as https://alerts.example/smallwebwaf"
|
"a fragment, such as https://alerts.example/smallwebwaf"
|
||||||
@@ -1051,7 +1042,7 @@ func TestNtfyTokenWithAControlCharacterStopsTheStartWithoutShowingIt(t *testing.
|
|||||||
"set": {alertNtfyToken: token + "\r"},
|
"set": {alertNtfyToken: token + "\r"},
|
||||||
"in a file": {alertNtfyToken + "_FILE": writeFile(t, token+"\r\n")},
|
"in a file": {alertNtfyToken + "_FILE": writeFile(t, token+"\r\n")},
|
||||||
} {
|
} {
|
||||||
_, err := config.FromEnvironment(env.viper())
|
_, err := config.FromEnvironment(env.lookupEnv)
|
||||||
|
|
||||||
want := alertNtfyToken + ": holds a control character, such as the " +
|
want := alertNtfyToken + ": holds a control character, such as the " +
|
||||||
"carriage return of a Windows line end"
|
"carriage return of a Windows line end"
|
||||||
@@ -1068,7 +1059,7 @@ func TestInstanceNameWithAControlCharacterStopsTheStartOnlyWithNtfySet(t *testin
|
|||||||
|
|
||||||
_, err := config.FromEnvironment(environment{
|
_, err := config.FromEnvironment(environment{
|
||||||
instanceName: name, alertNtfyURL: "https://ntfy.example/smallwebwaf-alerts",
|
instanceName: name, alertNtfyURL: "https://ntfy.example/smallwebwaf-alerts",
|
||||||
}.viper())
|
}.lookupEnv)
|
||||||
|
|
||||||
want := instanceName + `: "fsn1app1\r" holds a control character, such as the ` +
|
want := instanceName + `: "fsn1app1\r" holds a control character, such as the ` +
|
||||||
`carriage return of a Windows line end, and is sent to ntfy in a header ` +
|
`carriage return of a Windows line end, and is sent to ntfy in a header ` +
|
||||||
@@ -1093,7 +1084,7 @@ func TestInstanceNameNotUTF8StopsTheStart(t *testing.T) {
|
|||||||
"set": {instanceName: latin1},
|
"set": {instanceName: latin1},
|
||||||
"in a file": {instanceName + "_FILE": writeFile(t, latin1+"\n")},
|
"in a file": {instanceName + "_FILE": writeFile(t, latin1+"\n")},
|
||||||
} {
|
} {
|
||||||
_, err := config.FromEnvironment(env.viper())
|
_, err := config.FromEnvironment(env.lookupEnv)
|
||||||
|
|
||||||
want := instanceName + `: "caf\xe9" is not valid UTF-8`
|
want := instanceName + `: "caf\xe9" is not valid UTF-8`
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -1108,7 +1099,7 @@ func TestCodeOnBothCountryListsStopsTheStart(t *testing.T) {
|
|||||||
_, err := config.FromEnvironment(environment{
|
_, err := config.FromEnvironment(environment{
|
||||||
deniedCountries: "cn,ru",
|
deniedCountries: "cn,ru",
|
||||||
allowedCountries: "de,RU",
|
allowedCountries: "de,RU",
|
||||||
}.viper())
|
}.lookupEnv)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatal("ru on both country lists was accepted")
|
t.Fatal("ru on both country lists was accepted")
|
||||||
}
|
}
|
||||||
@@ -1170,7 +1161,7 @@ func TestLookupDBPathGoesWithTheFileSourceAlone(t *testing.T) {
|
|||||||
lookupDBPath + ": is set while " + lookupSource + " is off; only file reads it",
|
lookupDBPath + ": is set while " + lookupSource + " is off; only file reads it",
|
||||||
},
|
},
|
||||||
} {
|
} {
|
||||||
_, err := config.FromEnvironment(tc.env.viper())
|
_, err := config.FromEnvironment(tc.env.lookupEnv)
|
||||||
if err == nil || err.Error() != tc.want {
|
if err == nil || err.Error() != tc.want {
|
||||||
t.Errorf("settings %v: error %v, want %s", tc.env, err, tc.want)
|
t.Errorf("settings %v: error %v, want %s", tc.env, err, tc.want)
|
||||||
}
|
}
|
||||||
@@ -1200,7 +1191,7 @@ func TestSettingNeedingLookupsStopsTheStartWhileTheyAreOff(t *testing.T) {
|
|||||||
|
|
||||||
_, err := config.FromEnvironment(environment{
|
_, err := config.FromEnvironment(environment{
|
||||||
lookupSource: off, name: value,
|
lookupSource: off, name: value,
|
||||||
}.viper())
|
}.lookupEnv)
|
||||||
if err == nil || !strings.HasPrefix(err.Error(), name+": ") ||
|
if err == nil || !strings.HasPrefix(err.Error(), name+": ") ||
|
||||||
!strings.Contains(err.Error(), lookupSource+" is off") {
|
!strings.Contains(err.Error(), lookupSource+" is off") {
|
||||||
t.Errorf("error %v, want one naming %s and %s", err, name, lookupSource)
|
t.Errorf("error %v, want one naming %s and %s", err, name, lookupSource)
|
||||||
@@ -1350,7 +1341,7 @@ func TestInvalidAnomalySettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
|
|||||||
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper())
|
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
|
||||||
|
|
||||||
want := tc.name + ": " + tc.want
|
want := tc.name + ": " + tc.want
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -1429,7 +1420,7 @@ func TestInvalidBiasedThresholdStopsTheStartSayingWhatIsWrong(t *testing.T) {
|
|||||||
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper())
|
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
|
||||||
|
|
||||||
want := tc.name + ": " + tc.want
|
want := tc.name + ": " + tc.want
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -1526,7 +1517,7 @@ func TestInvalidReputationSettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
|
|||||||
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper())
|
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
|
||||||
|
|
||||||
want := tc.name + ": " + tc.want
|
want := tc.name + ": " + tc.want
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -1541,7 +1532,7 @@ func TestASNLimitPercentURLThatIsABlocklistStopsTheStart(t *testing.T) {
|
|||||||
|
|
||||||
_, err := config.FromEnvironment(environment{
|
_, err := config.FromEnvironment(environment{
|
||||||
blocklistURLs: dropURL + "," + asnURL, asnLimitPercentURL: asnURL,
|
blocklistURLs: dropURL + "," + asnURL, asnLimitPercentURL: asnURL,
|
||||||
}.viper())
|
}.lookupEnv)
|
||||||
|
|
||||||
want := asnLimitPercentURL + `: "` + asnURL + `" is in SWWAF_BLOCKLIST_URLS too`
|
want := asnLimitPercentURL + `: "` + asnURL + `" is in SWWAF_BLOCKLIST_URLS too`
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -1668,7 +1659,7 @@ func TestInvalidDNSBLSettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
|
|||||||
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper())
|
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
|
||||||
|
|
||||||
want := tc.name + ": " + tc.want
|
want := tc.name + ": " + tc.want
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -1736,7 +1727,7 @@ func TestDNSBLZoneKeyIsLoggedMaskedAndNeverShown(t *testing.T) {
|
|||||||
{spamhaus + "," + otherZone, `"` + spamhausMasked + `" is listed twice`},
|
{spamhaus + "," + otherZone, `"` + spamhausMasked + `" is listed twice`},
|
||||||
{spamhaus + ",,", "has an empty item in its list"},
|
{spamhaus + ",,", "has an empty item in its list"},
|
||||||
} {
|
} {
|
||||||
_, err := config.FromEnvironment(environment{dnsblZones: tc.value}.viper())
|
_, err := config.FromEnvironment(environment{dnsblZones: tc.value}.lookupEnv)
|
||||||
|
|
||||||
want := dnsblZones + ": " + tc.want
|
want := dnsblZones + ": " + tc.want
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -1789,7 +1780,7 @@ func TestInvalidAbuseIPDBSettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
|
|||||||
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper())
|
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
|
||||||
|
|
||||||
want := tc.name + ": " + tc.want
|
want := tc.name + ": " + tc.want
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -1903,7 +1894,7 @@ func TestInvalidCrowdSecSettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
|
|||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(tc.env.viper())
|
_, err := config.FromEnvironment(tc.env.lookupEnv)
|
||||||
if err == nil || err.Error() != tc.want {
|
if err == nil || err.Error() != tc.want {
|
||||||
t.Errorf("error %v, want %s", err, tc.want)
|
t.Errorf("error %v, want %s", err, tc.want)
|
||||||
}
|
}
|
||||||
@@ -2063,7 +2054,7 @@ func wantStartStopped(t *testing.T, invalid []struct{ name, value string }) {
|
|||||||
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper())
|
_, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
t.Fatalf("%s=%q was accepted", tc.name, tc.value)
|
t.Fatalf("%s=%q was accepted", tc.name, tc.value)
|
||||||
}
|
}
|
||||||
@@ -2088,7 +2079,7 @@ func TestHostOrTransferEncodingStopsTheStart(t *testing.T) {
|
|||||||
t.Run(value, func(t *testing.T) {
|
t.Run(value, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(environment{logRequestHeaders: value}.viper())
|
_, err := config.FromEnvironment(environment{logRequestHeaders: value}.lookupEnv)
|
||||||
if err == nil || err.Error() != logRequestHeaders+": "+want {
|
if err == nil || err.Error() != logRequestHeaders+": "+want {
|
||||||
t.Errorf("error %v, want %s: %s", err, logRequestHeaders, want)
|
t.Errorf("error %v, want %s: %s", err, logRequestHeaders, want)
|
||||||
}
|
}
|
||||||
@@ -2105,7 +2096,7 @@ func TestShortTokenStopsTheStartWithoutShowingIt(t *testing.T) {
|
|||||||
t.Run(name+"="+value, func(t *testing.T) {
|
t.Run(name+"="+value, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := config.FromEnvironment(environment{name: value}.viper())
|
_, err := config.FromEnvironment(environment{name: value}.lookupEnv)
|
||||||
|
|
||||||
want := name + ": is shorter than 32 characters"
|
want := name + ": is shorter than 32 characters"
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -2161,7 +2152,7 @@ func TestSettingFromFileIsCheckedAsTheSettingItself(t *testing.T) {
|
|||||||
|
|
||||||
_, err := config.FromEnvironment(environment{
|
_, err := config.FromEnvironment(environment{
|
||||||
bytesLimitPerMinute + "_FILE": writeFile(t, "lots\n"),
|
bytesLimitPerMinute + "_FILE": writeFile(t, "lots\n"),
|
||||||
}.viper())
|
}.lookupEnv)
|
||||||
|
|
||||||
want := bytesLimitPerMinute + `: "lots" is not a size such as 512K, 100M or 5G, or off`
|
want := bytesLimitPerMinute + `: "lots" is not a size such as 512K, 100M or 5G, or off`
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
@@ -2172,10 +2163,10 @@ func TestSettingFromFileIsCheckedAsTheSettingItself(t *testing.T) {
|
|||||||
logRemoteURL: remoteURL,
|
logRemoteURL: remoteURL,
|
||||||
instanceName: instance,
|
instanceName: instance,
|
||||||
logRemoteAppName + "_FILE": writeFile(t, "my app\n"),
|
logRemoteAppName + "_FILE": writeFile(t, "my app\n"),
|
||||||
}.viper())
|
}.lookupEnv)
|
||||||
|
|
||||||
want = logRemoteAppName + `: "my app" is not 1 to 48 printable ASCII ` +
|
want = logRemoteAppName + `: "my app" is not 1 to 48 printable ASCII ` +
|
||||||
`characters without a space, such as blog`
|
`characters without a space, such as gitea`
|
||||||
if err == nil || err.Error() != want {
|
if err == nil || err.Error() != want {
|
||||||
t.Errorf("error %v, want %s", err, want)
|
t.Errorf("error %v, want %s", err, want)
|
||||||
}
|
}
|
||||||
@@ -2187,7 +2178,7 @@ func TestSettingAndItsFileBothSetStopsTheStart(t *testing.T) {
|
|||||||
_, err := config.FromEnvironment(environment{
|
_, err := config.FromEnvironment(environment{
|
||||||
metricsToken: token,
|
metricsToken: token,
|
||||||
metricsToken + "_FILE": writeFile(t, token),
|
metricsToken + "_FILE": writeFile(t, token),
|
||||||
}.viper())
|
}.lookupEnv)
|
||||||
|
|
||||||
want := metricsToken + ": is set, and so is " + metricsToken +
|
want := metricsToken + ": is set, and so is " + metricsToken +
|
||||||
"_FILE; set only one of them"
|
"_FILE; set only one of them"
|
||||||
@@ -2202,7 +2193,7 @@ func TestUnreadableSettingFileStopsTheStart(t *testing.T) {
|
|||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
|
|
||||||
for _, path := range []string{filepath.Join(dir, "missing"), dir} {
|
for _, path := range []string{filepath.Join(dir, "missing"), dir} {
|
||||||
_, err := config.FromEnvironment(environment{metricsToken + "_FILE": path}.viper())
|
_, err := config.FromEnvironment(environment{metricsToken + "_FILE": path}.lookupEnv)
|
||||||
|
|
||||||
want := metricsToken + "_FILE: cannot be read: "
|
want := metricsToken + "_FILE: cannot be read: "
|
||||||
if err == nil || !strings.HasPrefix(err.Error(), want) {
|
if err == nil || !strings.HasPrefix(err.Error(), want) {
|
||||||
@@ -2336,7 +2327,6 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
|||||||
wafDisabledRules: defaultWAFDisabledRules,
|
wafDisabledRules: defaultWAFDisabledRules,
|
||||||
wafExemptPaths: "",
|
wafExemptPaths: "",
|
||||||
wafBodyLimit: off,
|
wafBodyLimit: off,
|
||||||
exclusionsFile: "",
|
|
||||||
trapPaths: "",
|
trapPaths: "",
|
||||||
errorBurstThreshold: "30",
|
errorBurstThreshold: "30",
|
||||||
logRemoteURL: "",
|
logRemoteURL: "",
|
||||||
|
|||||||
+24
-12
@@ -8,9 +8,8 @@ import (
|
|||||||
"net/netip"
|
"net/netip"
|
||||||
"net/url"
|
"net/url"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// RProxy is the settings of `smallwebwaf rproxy`. A size or time limit of zero
|
// RProxy is the settings of `smallwebwaf rproxy`. A size or time limit of zero
|
||||||
@@ -59,14 +58,13 @@ var (
|
|||||||
"is not a whole number of milliseconds above zero, such as 60000, or off")
|
"is not a whole number of milliseconds above zero, such as 60000, or off")
|
||||||
)
|
)
|
||||||
|
|
||||||
// RProxyFromEnvironment reads rproxy's settings through v, normally the viper
|
// RProxyFromEnvironment reads rproxy's settings with lookupEnv, normally
|
||||||
// NewViper returns, bound to the process's environment. A setting that is not
|
// os.LookupEnv. A setting that is not set takes its default, and
|
||||||
// set takes its default, and SWRP_WAF_BACKEND, which has none, must be set. A
|
// SWRP_WAF_BACKEND, which has none, must be set. A setting that is set but
|
||||||
// setting may instead be given as a file: the variable named by the setting's
|
// invalid is an error that names it. Since rproxy never touches disk, its
|
||||||
// name with _FILE added names the file, which is read now. A setting that is
|
// settings have no _FILE form.
|
||||||
// set but invalid is an error that names it.
|
func RProxyFromEnvironment(lookupEnv func(string) (string, bool)) (*RProxy, error) {
|
||||||
func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
|
env := &environment{lookupEnv: withoutFileForms(lookupEnv)}
|
||||||
env := &environment{v: v, readFiles: true}
|
|
||||||
cfg := &RProxy{
|
cfg := &RProxy{
|
||||||
ListenAddr: env.address("SWRP_LISTEN_ADDR"),
|
ListenAddr: env.address("SWRP_LISTEN_ADDR"),
|
||||||
UpstreamURL: env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL),
|
UpstreamURL: env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL),
|
||||||
@@ -93,9 +91,9 @@ func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
|
|||||||
// SWRP_UPSTREAM_URL. The health check in rproxy's container needs no other
|
// SWRP_UPSTREAM_URL. The health check in rproxy's container needs no other
|
||||||
// setting, so it reads no other.
|
// setting, so it reads no other.
|
||||||
func RProxyListenAddrAndUpstreamURL(
|
func RProxyListenAddrAndUpstreamURL(
|
||||||
v *viper.Viper,
|
lookupEnv func(string) (string, bool),
|
||||||
) (string, *url.URL, error) {
|
) (string, *url.URL, error) {
|
||||||
env := &environment{v: v, readFiles: true}
|
env := &environment{lookupEnv: withoutFileForms(lookupEnv)}
|
||||||
listenAddr := env.address("SWRP_LISTEN_ADDR")
|
listenAddr := env.address("SWRP_LISTEN_ADDR")
|
||||||
upstreamURL := env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL)
|
upstreamURL := env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL)
|
||||||
|
|
||||||
@@ -106,6 +104,20 @@ func RProxyListenAddrAndUpstreamURL(
|
|||||||
return listenAddr, upstreamURL, nil
|
return listenAddr, upstreamURL, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// withoutFileForms returns lookupEnv finding no setting named with _FILE, since
|
||||||
|
// rproxy never touches disk.
|
||||||
|
func withoutFileForms(
|
||||||
|
lookupEnv func(string) (string, bool),
|
||||||
|
) func(string) (string, bool) {
|
||||||
|
return func(name string) (string, bool) {
|
||||||
|
if strings.HasSuffix(name, "_FILE") {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
return lookupEnv(name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// LogValue makes an RProxy log as each setting's name with the value it was
|
// LogValue makes an RProxy log as each setting's name with the value it was
|
||||||
// given, or its default.
|
// given, or its default.
|
||||||
func (c *RProxy) LogValue() slog.Value {
|
func (c *RProxy) LogValue() slog.Value {
|
||||||
|
|||||||
@@ -9,8 +9,6 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -34,12 +32,11 @@ const (
|
|||||||
|
|
||||||
// rproxyConfig reads rproxy's settings from env.
|
// rproxyConfig reads rproxy's settings from env.
|
||||||
func rproxyConfig(env map[string]string) (*config.RProxy, error) {
|
func rproxyConfig(env map[string]string) (*config.RProxy, error) {
|
||||||
v := viper.New()
|
return config.RProxyFromEnvironment(func(name string) (string, bool) {
|
||||||
for name, value := range env {
|
value, ok := env[name]
|
||||||
v.Set(name, value)
|
|
||||||
}
|
|
||||||
|
|
||||||
return config.RProxyFromEnvironment(v)
|
return value, ok
|
||||||
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRProxyDefaults(t *testing.T) {
|
func TestRProxyDefaults(t *testing.T) {
|
||||||
@@ -180,41 +177,31 @@ func TestRProxyNeedsItsBackend(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRProxySettingReadFromFile(t *testing.T) {
|
func TestRProxyReadsNoFile(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
path := filepath.Join(t.TempDir(), "backend")
|
path := filepath.Join(t.TempDir(), "backend")
|
||||||
|
|
||||||
err := os.WriteFile(path, []byte(backendURL+"\n"), 0o600)
|
err := os.WriteFile(path, []byte("http://from-a-file:8080\n"), 0o600)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("write: %v", err)
|
t.Fatalf("write: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
cfg, err := rproxyConfig(map[string]string{wafBackend + "_FILE": path})
|
_, err = rproxyConfig(map[string]string{wafBackend + "_FILE": path})
|
||||||
|
if err == nil || !strings.HasPrefix(err.Error(), wafBackend+": is required") {
|
||||||
|
t.Errorf("error %v, want %s still required", err, wafBackend)
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg, err := rproxyConfig(map[string]string{
|
||||||
|
wafBackend: backendURL,
|
||||||
|
rproxyTimeout + "_FILE": "/nonexistent/timeout",
|
||||||
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("settings: %v", err)
|
t.Fatalf("settings: %v, with the file left unread", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.WAFBackend.String() != backendURL {
|
if cfg.Timeout != 2*time.Second {
|
||||||
t.Errorf("backend %v, want %s", cfg.WAFBackend, backendURL)
|
t.Errorf("timeout %v, want the default", cfg.Timeout)
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRProxyUnreadableSettingFileStopsTheStart(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
dir := t.TempDir()
|
|
||||||
|
|
||||||
for _, path := range []string{filepath.Join(dir, "missing"), dir} {
|
|
||||||
_, err := rproxyConfig(map[string]string{
|
|
||||||
wafBackend: backendURL,
|
|
||||||
rproxyTimeout + "_FILE": path,
|
|
||||||
})
|
|
||||||
|
|
||||||
want := rproxyTimeout + "_FILE: cannot be read: "
|
|
||||||
if err == nil || !strings.HasPrefix(err.Error(), want) {
|
|
||||||
t.Errorf("%s: error %v, want one starting %s", path, err, want)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,16 +0,0 @@
|
|||||||
package config
|
|
||||||
|
|
||||||
import "github.com/spf13/viper"
|
|
||||||
|
|
||||||
// NewViper returns the viper that reads the settings: each setting's
|
|
||||||
// environment variable, bound to its own name. Every setting of servewaf and
|
|
||||||
// rproxy is read through it, and none is read with os.Getenv or os.LookupEnv
|
|
||||||
// outside it. An environment variable that is set but empty is set, as one
|
|
||||||
// that holds a value is, which viper does not take for granted.
|
|
||||||
func NewViper() *viper.Viper {
|
|
||||||
v := viper.New()
|
|
||||||
v.AutomaticEnv()
|
|
||||||
v.AllowEmptyEnv(true)
|
|
||||||
|
|
||||||
return v
|
|
||||||
}
|
|
||||||
@@ -44,7 +44,7 @@ import (
|
|||||||
const (
|
const (
|
||||||
// started is the time a record made as a test starts gives.
|
// started is the time a record made as a test starts gives.
|
||||||
started = "2000-01-01T00:00:00.000Z"
|
started = "2000-01-01T00:00:00.000Z"
|
||||||
appName = "fsn1app1/blog"
|
appName = "fsn1app1/gitea"
|
||||||
// local0 is the number of the default facility, and local0Info the
|
// local0 is the number of the default facility, and local0Info the
|
||||||
// priority of its records.
|
// priority of its records.
|
||||||
local0 = 16
|
local0 = 16
|
||||||
@@ -94,7 +94,7 @@ func TestRecordsOverTCPAreOctetCountedWithTheirFacility(t *testing.T) {
|
|||||||
endpoint := listen(t)
|
endpoint := listen(t)
|
||||||
endpointParams := params(remotelog.SchemeTCP, endpoint.Addr())
|
endpointParams := params(remotelog.SchemeTCP, endpoint.Addr())
|
||||||
endpointParams.Facility = 19 // local3
|
endpointParams.Facility = 19 // local3
|
||||||
endpointParams.AppName = "blog"
|
endpointParams.AppName = "gitea"
|
||||||
sender, _, _ := run(t, endpointParams)
|
sender, _, _ := run(t, endpointParams)
|
||||||
|
|
||||||
_, _ = sender.Write([]byte("first\nsecond\n"))
|
_, _ = sender.Write([]byte("first\nsecond\n"))
|
||||||
@@ -103,8 +103,8 @@ func TestRecordsOverTCPAreOctetCountedWithTheirFacility(t *testing.T) {
|
|||||||
wantCounts(t, sender, 2, 0, 0)
|
wantCounts(t, sender, 2, 0, 0)
|
||||||
|
|
||||||
frames := bufio.NewReader(accept(t, endpoint))
|
frames := bufio.NewReader(accept(t, endpoint))
|
||||||
wantFrame(t, frames, record(t, "<158>", "blog", "first"))
|
wantFrame(t, frames, record(t, "<158>", "gitea", "first"))
|
||||||
wantFrame(t, frames, record(t, "<158>", "blog", "second"))
|
wantFrame(t, frames, record(t, "<158>", "gitea", "second"))
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -131,6 +131,15 @@ type Line struct {
|
|||||||
// ActionRuleBlocked, ActionWAFBlocked, or, in servewaf, ActionTooLarge
|
// ActionRuleBlocked, ActionWAFBlocked, or, in servewaf, ActionTooLarge
|
||||||
// for the header limits.
|
// for the header limits.
|
||||||
WouldAction string `json:"would_action,omitempty"`
|
WouldAction string `json:"would_action,omitempty"`
|
||||||
|
// Heuristic is the name of the heuristic that refused the request, in
|
||||||
|
// observe mode too, and HeuristicFigures are the figures it acted on, by
|
||||||
|
// name.
|
||||||
|
Heuristic string `json:"heuristic,omitempty"`
|
||||||
|
HeuristicFigures map[string]any `json:"heuristic_figures,omitempty"`
|
||||||
|
// ReportHeuristic and ReportHeuristicFigures are the same for the
|
||||||
|
// heuristic that banned the client after servewaf answered the request.
|
||||||
|
ReportHeuristic string `json:"report_heuristic,omitempty"`
|
||||||
|
ReportHeuristicFigures map[string]any `json:"report_heuristic_figures,omitempty"`
|
||||||
// LimitPercent and LimitPercentSetting are, for a request the rate
|
// LimitPercent and LimitPercentSetting are, for a request the rate
|
||||||
// limits counted whose client a biased threshold gives a percentage of
|
// limits counted whose client a biased threshold gives a percentage of
|
||||||
// the rate limits below 100, that percentage and the setting that gave
|
// the rate limits below 100, that percentage and the setting that gave
|
||||||
|
|||||||
@@ -94,7 +94,7 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
|
|||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
|
|
||||||
requestlog.NewProcessLogger(&out, "fsn1app1/blog", slog.LevelInfo).Info("starting",
|
requestlog.NewProcessLogger(&out, "fsn1app1/gitea", slog.LevelInfo).Info("starting",
|
||||||
"version", "v1")
|
"version", "v1")
|
||||||
|
|
||||||
var fields map[string]any
|
var fields map[string]any
|
||||||
@@ -104,7 +104,7 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
|
|||||||
t.Fatalf("decode %q: %v", out.String(), err)
|
t.Fatalf("decode %q: %v", out.String(), err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if fields["type"] != "process" || fields["instance"] != "fsn1app1/blog" ||
|
if fields["type"] != "process" || fields["instance"] != "fsn1app1/gitea" ||
|
||||||
fields["msg"] != "starting" || fields["level"] != "INFO" ||
|
fields["msg"] != "starting" || fields["level"] != "INFO" ||
|
||||||
fields["version"] != "v1" {
|
fields["version"] != "v1" {
|
||||||
t.Errorf("process line %v", fields)
|
t.Errorf("process line %v", fields)
|
||||||
@@ -133,7 +133,7 @@ func TestProcessLoggerWritesTheMessagesAtItsLevelOrMoreSevere(t *testing.T) {
|
|||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
|
|
||||||
processLog := requestlog.NewProcessLogger(&out, "fsn1app1/blog", level)
|
processLog := requestlog.NewProcessLogger(&out, "fsn1app1/gitea", level)
|
||||||
for _, at := range levels {
|
for _, at := range levels {
|
||||||
processLog.Log(t.Context(), at, "message")
|
processLog.Log(t.Context(), at, "message")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -33,9 +33,13 @@ func announcedLength(size int) string {
|
|||||||
func TestServerHasTheFixedLimits(t *testing.T) {
|
func TestServerHasTheFixedLimits(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
cfg, err := config.RProxyFromEnvironment(viperFrom(map[string]string{
|
cfg, err := config.RProxyFromEnvironment(func(name string) (string, bool) {
|
||||||
"SWRP_WAF_BACKEND": nowhere,
|
if name == "SWRP_WAF_BACKEND" {
|
||||||
}))
|
return nowhere, true
|
||||||
|
}
|
||||||
|
|
||||||
|
return "", false
|
||||||
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("settings: %v", err)
|
t.Fatalf("settings: %v", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -18,8 +18,6 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
||||||
@@ -343,17 +341,6 @@ func startServer(
|
|||||||
return addr, out, server
|
return addr, out, server
|
||||||
}
|
}
|
||||||
|
|
||||||
// viperFrom returns a viper Set with each of settings, as the viper
|
|
||||||
// config.NewViper returns one bound to a process's environment.
|
|
||||||
func viperFrom(settings map[string]string) *viper.Viper {
|
|
||||||
v := viper.New()
|
|
||||||
for name, value := range settings {
|
|
||||||
v.Set(name, value)
|
|
||||||
}
|
|
||||||
|
|
||||||
return v
|
|
||||||
}
|
|
||||||
|
|
||||||
// startServerLogging is startServer with rproxy's request lines going to
|
// startServerLogging is startServer with rproxy's request lines going to
|
||||||
// requestLog and its process lines to processLog.
|
// requestLog and its process lines to processLog.
|
||||||
func startServerLogging(
|
func startServerLogging(
|
||||||
@@ -367,7 +354,11 @@ func startServerLogging(
|
|||||||
}
|
}
|
||||||
maps.Copy(settings, env)
|
maps.Copy(settings, env)
|
||||||
|
|
||||||
cfg, err := config.RProxyFromEnvironment(viperFrom(settings))
|
cfg, err := config.RProxyFromEnvironment(func(name string) (string, bool) {
|
||||||
|
value, ok := settings[name]
|
||||||
|
|
||||||
|
return value, ok
|
||||||
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("settings %v: %v", settings, err)
|
t.Fatalf("settings %v: %v", settings, err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -0,0 +1,55 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic abuseipdb. It reads SWWAF_ABUSEIPDB_KEY,
|
||||||
|
// SWWAF_ABUSEIPDB_MIN_SCORE and SWWAF_REPUTATION_ACTION.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// abuseIPDB notes AbuseIPDB, as noteHit does, with the score, when its score
|
||||||
|
// of the client is a hit, and refuses the request when SWWAF_REPUTATION_ACTION
|
||||||
|
// is deny, as dnsbl does for a zone. While SWWAF_ABUSEIPDB_KEY is unset it does
|
||||||
|
// nothing. A client without a score is checked in the background, by the
|
||||||
|
// request's address, if its history counts an offence, and the request does
|
||||||
|
// not wait for the answer. The score is then used for each address of the
|
||||||
|
// client. Its figures are the score and SWWAF_ABUSEIPDB_MIN_SCORE.
|
||||||
|
type abuseIPDB struct{}
|
||||||
|
|
||||||
|
func (abuseIPDB) name() string { return "abuseipdb" }
|
||||||
|
|
||||||
|
func (abuseIPDB) check(ctx context.Context, rq *request, _ time.Time) *verdict {
|
||||||
|
if rq.h.config.AbuseIPDBKey == "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
client := rq.h.clientGroup(rq.client)
|
||||||
|
held, _ := rq.h.limiter.Client(client)
|
||||||
|
offender := held.History.Offences != ratelimit.Offences{}
|
||||||
|
|
||||||
|
score, hit := rq.h.abuseIPDB.Hit(ctx, client, rq.client, offender)
|
||||||
|
if !hit {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.abuseIPDBHit = true
|
||||||
|
rq.noteHit(bans.ReputationHit{Source: reputation.AbuseIPDBSource, Score: &score},
|
||||||
|
"scored by AbuseIPDB at or over SWWAF_ABUSEIPDB_MIN_SCORE")
|
||||||
|
|
||||||
|
if rq.h.config.ReputationAction != deny {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return &verdict{
|
||||||
|
action: requestlog.ActionDenied,
|
||||||
|
figures: map[string]any{
|
||||||
|
figureScore: score, "min_score": rq.h.config.AbuseIPDBMinScore,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -18,7 +18,7 @@ const (
|
|||||||
alertWebhookURL = "SWWAF_ALERT_WEBHOOK_URL"
|
alertWebhookURL = "SWWAF_ALERT_WEBHOOK_URL"
|
||||||
alertMaxPerHour = "SWWAF_ALERT_MAX_PER_HOUR"
|
alertMaxPerHour = "SWWAF_ALERT_MAX_PER_HOUR"
|
||||||
// alertInstance is the instance every alert of these tests gives.
|
// alertInstance is the instance every alert of these tests gives.
|
||||||
alertInstance = "fsn1app1/blog"
|
alertInstance = "fsn1app1/gitea"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
|
func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
|
||||||
|
|||||||
@@ -91,6 +91,18 @@ func TestSixteenThousandCookiesAreRefusedWellWithinTheTimeLimit(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestHeaderLimitRefusalNamesItsHeuristicCountsAndLimits(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s := startServeWAF(t, "", nil)
|
||||||
|
req := s.question(siteA, client, "/")
|
||||||
|
fillHeaders(req, headerFieldLimit+1)
|
||||||
|
do(t, req)
|
||||||
|
|
||||||
|
wantHeuristic(t, s.out.requestLine(t), "header_limits",
|
||||||
|
`{"fields": 101, "fields_limit": 100, "cookies": 0, "cookies_limit": 50}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
|
func TestEachHeaderLimitRefusesARequestOverIt(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
+25
-79
@@ -1,7 +1,10 @@
|
|||||||
package servewaf
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic active_ban, and the code that makes bans, which several
|
||||||
|
// heuristics share. active_ban reads SWWAF_MODE; the bans are the ban ledger's.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"net/http"
|
"context"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -18,11 +21,15 @@ func (rq *request) banResponse(action string) *refusal {
|
|||||||
return &refusal{status: rq.h.config.BanResponse, action: action}
|
return &refusal{status: rq.h.config.BanResponse, action: action}
|
||||||
}
|
}
|
||||||
|
|
||||||
// banned reports whether a ban on a netblock the client is in covers the
|
// activeBan refuses a client a ban covers: a ban on a netblock the client is
|
||||||
// request at now, and notes for the log line when that ban ends. A
|
// in, in force at now. It notes for the log line when that ban ends. A request
|
||||||
// request that makes the ban permanent, or in observe mode would have,
|
// that makes the ban permanent, or in observe mode would have, raises the alert
|
||||||
// raises the alert for it.
|
// for it. Its figures are the ban's netblock and when it ends.
|
||||||
func (rq *request) banned(now time.Time) bool {
|
type activeBan struct{}
|
||||||
|
|
||||||
|
func (activeBan) name() string { return "active_ban" }
|
||||||
|
|
||||||
|
func (activeBan) check(_ context.Context, rq *request, now time.Time) *verdict {
|
||||||
check := rq.h.ledger.Check
|
check := rq.h.ledger.Check
|
||||||
if rq.h.config.Observe {
|
if rq.h.config.Observe {
|
||||||
check = rq.h.ledger.Find // the ban refuses nothing, and stays as it is
|
check = rq.h.ledger.Find // the ban refuses nothing, and stays as it is
|
||||||
@@ -38,85 +45,24 @@ func (rq *request) banned(now time.Time) bool {
|
|||||||
rq.alertBan(ban)
|
rq.alertBan(ban)
|
||||||
}
|
}
|
||||||
|
|
||||||
return banned
|
if !banned {
|
||||||
}
|
return nil
|
||||||
|
|
||||||
// limitBroken counts the request for the rate limits at now, notes the
|
|
||||||
// client's counts for the log line, and reports whether the request takes
|
|
||||||
// the client over a rate limit, as its limit percentage lowers it, which
|
|
||||||
// breaks it.
|
|
||||||
func (rq *request) limitBroken(now time.Time) bool {
|
|
||||||
counts, hit, over := rq.h.limiter.Count(rq.h.clientGroup(rq.client), now,
|
|
||||||
rq.limitPercent.percent)
|
|
||||||
rq.line.Counts = counts
|
|
||||||
|
|
||||||
if over {
|
|
||||||
rq.banForLimit(now, hit, rq.h.config.BanResponse)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
return over
|
return &verdict{
|
||||||
}
|
action: requestlog.ActionBanned,
|
||||||
|
figures: map[string]any{
|
||||||
// countBytes counts the request's bytes, as countedBytes gives them, for
|
figureNetblock: ban.Netblock.String(), "ban_expires": rq.line.BanExpires,
|
||||||
// the byte limits, once its report has come, and notes the client's byte totals
|
},
|
||||||
// for the log line; its requests stay there as the rate limits counted them.
|
|
||||||
// Only an allowed request has them counted, and only one the rate limits
|
|
||||||
// counted; in observe mode, not one that enforce mode would have refused. Bytes
|
|
||||||
// that take the client over a byte limit, as its limit percentage for the byte
|
|
||||||
// limits lowers it, break it; the response was passed on whole.
|
|
||||||
func (rq *request) countBytes() {
|
|
||||||
if !rq.counted || rq.line.WouldAction != "" {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
now := rq.h.now()
|
|
||||||
|
|
||||||
counts, hit, over := rq.h.limiter.CountBytes(rq.h.clientGroup(rq.client), now,
|
|
||||||
rq.countedBytes(), rq.bytesPercent.percent)
|
|
||||||
rq.line.Counts.MinuteBytes = counts.MinuteBytes
|
|
||||||
rq.line.Counts.HourBytes = counts.HourBytes
|
|
||||||
rq.line.Counts.DayBytes = counts.DayBytes
|
|
||||||
|
|
||||||
if over {
|
|
||||||
rq.banForLimit(now, hit, rq.out.status)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// countRefusal counts the request for the error burst once it has been
|
// limitFigures are the figures of a heuristic that broke hit, a limit: its
|
||||||
// answered, if servewaf refused it after a rule file match, a trap path or a
|
// window, the count and the limit.
|
||||||
// Core Rule Set match, and in observe mode if enforce mode would have: more than
|
func limitFigures(hit ratelimit.Hit) map[string]any {
|
||||||
// SWWAF_ERROR_BURST_THRESHOLD such refusals of the client within a minute
|
return map[string]any{
|
||||||
// break a limit. A client in SWWAF_ALLOW_NETS, which the checks skip, is
|
figureWindow: hit.Window, figureCount: hit.Count, figureLimit: hit.Limit,
|
||||||
// not counted, and nothing is while the threshold is off.
|
|
||||||
func (rq *request) countRefusal() {
|
|
||||||
cfg := rq.h.config
|
|
||||||
if cfg.ErrorBurstThreshold == 0 {
|
|
||||||
return
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if !rq.attack && !rq.ruleBlocked && !rq.wafBlocked {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
now := rq.h.now()
|
|
||||||
|
|
||||||
hit, over := rq.h.limiter.CountRefusal(rq.h.clientGroup(rq.client), now,
|
|
||||||
cfg.ErrorBurstThreshold)
|
|
||||||
if !over {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
// What the client was sent, or in observe mode would have been.
|
|
||||||
status := rq.out.status
|
|
||||||
|
|
||||||
switch rq.line.WouldAction {
|
|
||||||
case requestlog.ActionRuleBlocked, requestlog.ActionWAFBlocked:
|
|
||||||
status = http.StatusForbidden
|
|
||||||
case requestlog.ActionBanned:
|
|
||||||
status = cfg.BanResponse
|
|
||||||
}
|
|
||||||
|
|
||||||
rq.banForLimit(now, hit, status)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// countedBytes returns the request's bytes, once it has ended, as the
|
// countedBytes returns the request's bytes, once it has ended, as the
|
||||||
|
|||||||
@@ -228,6 +228,22 @@ func TestBannedClientIsRefusedBeforeItsCountryIsLookedUp(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRefusalUnderABanNamesItsHeuristicNetblockAndEnd(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, clk, _ := startWithClock(t, "", map[string]string{
|
||||||
|
rateLimitPerMinute: "1", banScopeV4Prefix: "24",
|
||||||
|
})
|
||||||
|
expires := requestlog.FormatTime(clk.Now().Add(time.Hour))
|
||||||
|
|
||||||
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
|
||||||
|
line := s.get(otherClient, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
wantHeuristic(t, line, "active_ban",
|
||||||
|
`{"netblock": "`+clientsNet+`", "ban_expires": "`+expires+`"}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestBanResponseAnswersEveryRefusalButTheSizeLimits(t *testing.T) {
|
func TestBanResponseAnswersEveryRefusalButTheSizeLimits(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,41 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic blocklists. It reads SWWAF_BLOCKLIST_ACTION, and the blocklists
|
||||||
|
// through the reputation lists.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// blocklists notes the blocklists that list the client, as noteListed does,
|
||||||
|
// and refuses the request when SWWAF_BLOCKLIST_ACTION is deny. Being limit, it
|
||||||
|
// lowers the client's limits instead (see limitPercentages), and being log, it
|
||||||
|
// does nothing more. Its figure is the blocklists that list the client.
|
||||||
|
type blocklists struct{}
|
||||||
|
|
||||||
|
func (blocklists) name() string { return "blocklists" }
|
||||||
|
|
||||||
|
func (blocklists) check(_ context.Context, rq *request, _ time.Time) *verdict {
|
||||||
|
listedBy := rq.h.lists.ListedBy(rq.client)
|
||||||
|
rq.blocklisted = len(listedBy) > 0
|
||||||
|
rq.noteListed(listedBy, "listed by a blocklist")
|
||||||
|
|
||||||
|
if !rq.refusedByBlocklists(listedBy) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return &verdict{
|
||||||
|
action: requestlog.ActionDenied,
|
||||||
|
figures: map[string]any{"lists": listedBy},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// refusedByBlocklists reports whether the heuristic blocklists refuses a
|
||||||
|
// client, given listedBy, the blocklists that list it: it does when one lists
|
||||||
|
// it and SWWAF_BLOCKLIST_ACTION is deny.
|
||||||
|
func (rq *request) refusedByBlocklists(listedBy []string) bool {
|
||||||
|
return len(listedBy) > 0 && rq.h.config.BlocklistAction == deny
|
||||||
|
}
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic byte_limits. It reads SWWAF_BYTES_COUNT, and the byte limits
|
||||||
|
// and biased thresholds through the limiter and limitPercentages.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// byteLimits counts the request's bytes, as countedBytes gives them, for the
|
||||||
|
// byte limits, once its report has come, and notes the client's byte totals for
|
||||||
|
// the log line; its requests stay there as the rate limits counted them. Only
|
||||||
|
// an allowed request has them counted, and only one the rate limits counted; in
|
||||||
|
// observe mode, not one that enforce mode would have refused. Bytes that take
|
||||||
|
// the client over a byte limit, as its limit percentage for the byte limits
|
||||||
|
// lowers it, break it, which bans the client; the response was passed on whole.
|
||||||
|
// Its figures are the window, the count and the limit.
|
||||||
|
type byteLimits struct{}
|
||||||
|
|
||||||
|
func (byteLimits) name() string { return "byte_limits" }
|
||||||
|
|
||||||
|
func (byteLimits) check(_ context.Context, rq *request, now time.Time) *verdict {
|
||||||
|
if !rq.counted || rq.line.WouldAction != "" {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
counts, hit, over := rq.h.limiter.CountBytes(rq.h.clientGroup(rq.client), now,
|
||||||
|
rq.countedBytes(), rq.bytesPercent.percent)
|
||||||
|
rq.line.Counts.MinuteBytes = counts.MinuteBytes
|
||||||
|
rq.line.Counts.HourBytes = counts.HourBytes
|
||||||
|
rq.line.Counts.DayBytes = counts.DayBytes
|
||||||
|
|
||||||
|
if !over {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.banForLimit(now, hit, rq.out.status)
|
||||||
|
|
||||||
|
return &verdict{action: requestlog.ActionBanned, figures: limitFigures(hit)}
|
||||||
|
}
|
||||||
@@ -98,6 +98,21 @@ func TestEachByteLimitBansOnceTheResponseHasEnded(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestByteLimitBanNamesItsHeuristicWindowCountAndLimit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _ := startWithAnswers(t, map[string]string{bytesLimitPerMinute: byteLimit})
|
||||||
|
|
||||||
|
line, _ := s.download()
|
||||||
|
wantReportHeuristic(t, line, "", "null")
|
||||||
|
|
||||||
|
// Two answers of 70 bytes are over the limit of 99.
|
||||||
|
line, _ = s.download()
|
||||||
|
wantHeuristic(t, line, "", "null")
|
||||||
|
wantReportHeuristic(t, line, "byte_limits",
|
||||||
|
`{"window": "minute", "count": 140, "limit": 99}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestResponseOverAByteLimitByItselfIsPassedOnWhole(t *testing.T) {
|
func TestResponseOverAByteLimitByItselfIsPassedOnWhole(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,10 @@
|
|||||||
package servewaf
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic core_rule_set. It reads SWWAF_WAF_MODE, SWWAF_WAF_EXEMPT_PATHS
|
||||||
|
// and SWWAF_WAF_ANOMALY_THRESHOLD.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
@@ -9,18 +13,21 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||||
)
|
)
|
||||||
|
|
||||||
// checkCoreRuleSet inspects the request with the Core Rule Set, unless
|
// coreRuleSet inspects the request with the Core Rule Set, unless
|
||||||
// SWWAF_WAF_MODE is off or SWWAF_WAF_EXEMPT_PATHS exempts its path, as
|
// SWWAF_WAF_MODE is off or SWWAF_WAF_EXEMPT_PATHS exempts its path, as
|
||||||
// pathExempt decides, and notes the rules it matched and its score in the
|
// pathExempt decides, and notes the rules it matched and its score in the
|
||||||
// log line, and the rules in the metrics. A score at or over
|
// log line, and the rules in the metrics. A score at or over
|
||||||
// SWWAF_WAF_ANOMALY_THRESHOLD is a match: it raises the waf_block alert,
|
// SWWAF_WAF_ANOMALY_THRESHOLD is a match: it raises the waf_block alert,
|
||||||
// and in block mode refuses the request, which is an offence its client's
|
// and in block mode refuses the request with 403, which is an offence its
|
||||||
// history counts, and so returns ActionWAFBlocked. It returns "" for a
|
// client's history counts. Its figures are the score and the threshold.
|
||||||
// request it does not refuse.
|
type coreRuleSet struct{}
|
||||||
func (rq *request) checkCoreRuleSet() string {
|
|
||||||
|
func (coreRuleSet) name() string { return "core_rule_set" }
|
||||||
|
|
||||||
|
func (coreRuleSet) check(_ context.Context, rq *request, _ time.Time) *verdict {
|
||||||
cfg := rq.h.config
|
cfg := rq.h.config
|
||||||
if cfg.WAFMode == config.WAFModeOff || pathExempt(rq.in, cfg.WAFExemptPaths) {
|
if cfg.WAFMode == config.WAFModeOff || pathExempt(rq.in, cfg.WAFExemptPaths) {
|
||||||
return ""
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
start := time.Now()
|
start := time.Now()
|
||||||
@@ -40,18 +47,21 @@ func (rq *request) checkCoreRuleSet() string {
|
|||||||
|
|
||||||
threshold := cfg.WAFAnomalyThreshold
|
threshold := cfg.WAFAnomalyThreshold
|
||||||
if threshold == 0 || result.Score < threshold {
|
if threshold == 0 || result.Score < threshold {
|
||||||
return ""
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
rq.alertWAFBlock(result)
|
rq.alertWAFBlock(result)
|
||||||
|
|
||||||
if cfg.WAFMode == config.WAFModeDetect {
|
if cfg.WAFMode == config.WAFModeDetect {
|
||||||
return ""
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
rq.wafBlocked = true
|
rq.wafBlocked = true
|
||||||
|
|
||||||
return requestlog.ActionWAFBlocked
|
return &verdict{
|
||||||
|
action: requestlog.ActionWAFBlocked,
|
||||||
|
figures: map[string]any{figureScore: result.Score, "threshold": threshold},
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// alertWAFBlock raises the waf_block alert for the request, which the Core
|
// alertWAFBlock raises the waf_block alert for the request, which the Core
|
||||||
|
|||||||
@@ -110,6 +110,16 @@ func TestCoreRuleSetRefusesAttacksInBlockModeAndOnlyLogsThemInDetectMode(t *test
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestCoreRuleSetRefusalNamesItsHeuristicScoreAndThreshold(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
|
||||||
|
|
||||||
|
line := s.request(client, sqlInjection, http.StatusForbidden,
|
||||||
|
requestlog.ActionWAFBlocked)
|
||||||
|
wantHeuristic(t, line, "core_rule_set", `{"score": 5, "threshold": 5}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestOrdinaryRequestIsInspectedAndPassed(t *testing.T) {
|
func TestOrdinaryRequestIsInspectedAndPassed(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -1,21 +1,42 @@
|
|||||||
package servewaf
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic countries. It reads SWWAF_DENIED_COUNTRIES and
|
||||||
|
// SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"slices"
|
"slices"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
)
|
)
|
||||||
|
|
||||||
// countryDenied reports whether the country lists refuse the request, by
|
// countries refuses a request by the client's country as it was looked up. A
|
||||||
// the client's country as it was looked up. A client without a country,
|
// client without a country, or whose country cannot be found, is refused only
|
||||||
// or whose country cannot be found, is refused only by
|
// by SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES. Its figures are the country and the
|
||||||
// SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
// setting that refused it.
|
||||||
func (rq *request) countryDenied() bool {
|
type countries struct{}
|
||||||
|
|
||||||
|
func (countries) name() string { return "countries" }
|
||||||
|
|
||||||
|
func (countries) check(_ context.Context, rq *request, _ time.Time) *verdict {
|
||||||
denied := rq.h.config.DeniedCountries
|
denied := rq.h.config.DeniedCountries
|
||||||
allowed := rq.h.config.ExclusivelyAllowedCountries
|
allowed := rq.h.config.ExclusivelyAllowedCountries
|
||||||
country := rq.line.Country
|
country := rq.line.Country
|
||||||
|
|
||||||
if slices.Contains(denied, country) {
|
var setting string
|
||||||
return true
|
|
||||||
|
switch {
|
||||||
|
case slices.Contains(denied, country):
|
||||||
|
setting = "SWWAF_DENIED_COUNTRIES"
|
||||||
|
case len(allowed) > 0 && !slices.Contains(allowed, country):
|
||||||
|
setting = "SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES"
|
||||||
|
default:
|
||||||
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
return len(allowed) > 0 && !slices.Contains(allowed, country)
|
return &verdict{
|
||||||
|
action: requestlog.ActionCountryDenied,
|
||||||
|
figures: map[string]any{"country": country, "setting": setting},
|
||||||
|
}
|
||||||
}
|
}
|
||||||
@@ -90,6 +90,23 @@ func TestCountryLists(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestCountryRefusalNamesItsHeuristicCountryAndSetting(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
geojsURL, _ := startGeoJS(t)
|
||||||
|
s, _, _ := startWithClock(t, geojsURL, map[string]string{
|
||||||
|
lookupTimeout: "1h", deniedCountries: "kp", allowedCountries: "de",
|
||||||
|
})
|
||||||
|
|
||||||
|
line := s.get(fromKP, http.StatusForbidden, requestlog.ActionCountryDenied)
|
||||||
|
wantHeuristic(t, line, "countries",
|
||||||
|
`{"country": "KP", "setting": "SWWAF_DENIED_COUNTRIES"}`)
|
||||||
|
|
||||||
|
line = s.get(unplaced, http.StatusForbidden, requestlog.ActionCountryDenied)
|
||||||
|
wantHeuristic(t, line, "countries",
|
||||||
|
`{"country": "", "setting": "SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES"}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestCountryRefusalKeepsARequestFromTheApp(t *testing.T) {
|
func TestCountryRefusalKeepsARequestFromTheApp(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,49 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic crowdsec. It reads SWWAF_CROWDSEC_LAPI_URL, and the decisions
|
||||||
|
// through the reputation lists.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// crowdSec refuses a request when a decision of the CrowdSec decision list on
|
||||||
|
// the client is in force at now. It then notes the list, as noteHit does, and
|
||||||
|
// bans the client until that decision ends. Its figures are the decision's
|
||||||
|
// scenario and when it ends.
|
||||||
|
type crowdSec struct{}
|
||||||
|
|
||||||
|
func (crowdSec) name() string { return "crowdsec" }
|
||||||
|
|
||||||
|
func (crowdSec) check(_ context.Context, rq *request, now time.Time) *verdict {
|
||||||
|
decision, listed := rq.crowdSecDecisionInForce(now)
|
||||||
|
if !listed {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.noteHit(bans.ReputationHit{Source: rq.h.config.CrowdSecDecisionsURL},
|
||||||
|
"listed by the CrowdSec decision list")
|
||||||
|
rq.banForCrowdSec(now, decision)
|
||||||
|
|
||||||
|
return &verdict{
|
||||||
|
action: requestlog.ActionBanned,
|
||||||
|
figures: map[string]any{
|
||||||
|
"scenario": decision.Scenario,
|
||||||
|
"expires": requestlog.FormatTime(decision.Expires),
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// crowdSecDecisionInForce returns the decision of the CrowdSec decision list on
|
||||||
|
// the client that is in force at now, and reports whether there is one, which
|
||||||
|
// the heuristic crowdsec refuses the request for.
|
||||||
|
func (rq *request) crowdSecDecisionInForce(
|
||||||
|
now time.Time,
|
||||||
|
) (reputation.Decision, bool) {
|
||||||
|
return rq.h.lists.CrowdSecDecision(rq.client, now)
|
||||||
|
}
|
||||||
@@ -95,6 +95,21 @@ func TestClientTheCrowdSecDecisionListListsIsBannedUntilTheDecisionEnds(t *testi
|
|||||||
float64(fetched.Unix()))
|
float64(fetched.Unix()))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestCrowdSecBanNamesItsHeuristicScenarioAndEnd(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, clk, server, _ := startWithAlerts(t, map[string]string{
|
||||||
|
crowdSecURL: lapi, crowdSecKey: bouncerKey,
|
||||||
|
})
|
||||||
|
loadDecisions(t, server, clk.Now(), `[{"duration": "4h0m0s", `+
|
||||||
|
`"scenario": "crowdsecurity/ssh-bf", "scope": "Ip", "type": "ban", `+
|
||||||
|
`"value": "`+client+`"}]`)
|
||||||
|
|
||||||
|
line := s.get(client, http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
wantHeuristic(t, line, "crowdsec", `{"scenario": "crowdsecurity/ssh-bf", `+
|
||||||
|
`"expires": "`+requestlog.FormatTime(clk.Now().Add(4*time.Hour))+`"}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt(t *testing.T) {
|
func TestEndedCrowdSecDecisionNoLongerBansThoughTheCopyStillHoldsIt(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,29 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic deny_nets. It reads SWWAF_DENY_NETS.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// denyNets refuses a client in SWWAF_DENY_NETS. Its figure is the netblock of
|
||||||
|
// the setting the client is in.
|
||||||
|
type denyNets struct{}
|
||||||
|
|
||||||
|
func (denyNets) name() string { return "deny_nets" }
|
||||||
|
|
||||||
|
func (denyNets) check(_ context.Context, rq *request, _ time.Time) *verdict {
|
||||||
|
for _, netblock := range rq.h.config.DenyNets {
|
||||||
|
if netblock.Contains(rq.client) {
|
||||||
|
return &verdict{
|
||||||
|
action: requestlog.ActionDenied,
|
||||||
|
figures: map[string]any{figureNetblock: netblock.String()},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return nil
|
||||||
|
}
|
||||||
@@ -0,0 +1,36 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic dnsbl. It reads SWWAF_REPUTATION_ACTION, and the DNSBL zones'
|
||||||
|
// verdicts through the DNSBL checker.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// dnsbl notes the DNSBL zones whose verdict lists the client, as noteListed
|
||||||
|
// does, and refuses the request when SWWAF_REPUTATION_ACTION is deny. Being
|
||||||
|
// limit, it lowers the client's limits instead (see limitPercentages), and
|
||||||
|
// being log, it does nothing more. A zone without a verdict on the client is
|
||||||
|
// asked about it in the background, and the request does not wait for the
|
||||||
|
// answer. Its figure is the zones that list the client, their keys masked.
|
||||||
|
type dnsbl struct{}
|
||||||
|
|
||||||
|
func (dnsbl) name() string { return "dnsbl" }
|
||||||
|
|
||||||
|
func (dnsbl) check(ctx context.Context, rq *request, _ time.Time) *verdict {
|
||||||
|
listedBy := rq.h.dnsbl.ListedBy(ctx, rq.client)
|
||||||
|
rq.dnsblListed = len(listedBy) > 0
|
||||||
|
rq.noteListed(listedBy, "listed by a DNSBL zone")
|
||||||
|
|
||||||
|
if !rq.dnsblListed || rq.h.config.ReputationAction != deny {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return &verdict{
|
||||||
|
action: requestlog.ActionDenied,
|
||||||
|
figures: map[string]any{"zones": listedBy},
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,73 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic header_limits. It reads no setting: its limits are fixed.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"strings"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The header limits: the most cookies and header fields a request may have
|
||||||
|
// when the checks reach the Core Rule Set. Real clients never reach them.
|
||||||
|
const (
|
||||||
|
maxCookies = 50
|
||||||
|
maxHeaderFields = 100
|
||||||
|
)
|
||||||
|
|
||||||
|
// headerLimits refuses a request over the header limits, with 431. Its figures
|
||||||
|
// are the request's header fields and cookies, and the limit of each.
|
||||||
|
type headerLimits struct{}
|
||||||
|
|
||||||
|
func (headerLimits) name() string { return "header_limits" }
|
||||||
|
|
||||||
|
func (headerLimits) check(_ context.Context, rq *request, _ time.Time) *verdict {
|
||||||
|
fields, cookies := rq.headerCounts()
|
||||||
|
if fields <= maxHeaderFields && cookies <= maxCookies {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return &verdict{
|
||||||
|
action: requestlog.ActionTooLarge,
|
||||||
|
figures: map[string]any{
|
||||||
|
"fields": fields, "fields_limit": maxHeaderFields,
|
||||||
|
"cookies": cookies, "cookies_limit": maxCookies,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// overHeaderLimits reports whether the request is over the header limits: it
|
||||||
|
// has more than maxHeaderFields fields or more than maxCookies cookies, as
|
||||||
|
// headerCounts counts them.
|
||||||
|
func (rq *request) overHeaderLimits() bool {
|
||||||
|
fields, cookies := rq.headerCounts()
|
||||||
|
|
||||||
|
return fields > maxHeaderFields || cookies > maxCookies
|
||||||
|
}
|
||||||
|
|
||||||
|
// headerCounts returns the request's header fields, each value of a header sent
|
||||||
|
// twice being one, and its cookies, each part of a Cookie header between
|
||||||
|
// semicolons being one unless it is empty or only spaces and tabs. The Core
|
||||||
|
// Rule Set runs each of its rules on each cookie, so that thousands of small
|
||||||
|
// cookies keep it busy for over a second. It trims only spaces and control
|
||||||
|
// characters, of which a header value can hold only tabs, so a part of Unicode
|
||||||
|
// spaces is a cookie to it.
|
||||||
|
func (rq *request) headerCounts() (int, int) {
|
||||||
|
fields, cookies := 0, 0
|
||||||
|
|
||||||
|
for _, values := range rq.in.Header {
|
||||||
|
fields += len(values)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, value := range rq.in.Header.Values("Cookie") {
|
||||||
|
for part := range strings.SplitSeq(value, ";") {
|
||||||
|
if strings.Trim(part, " \t") != "" {
|
||||||
|
cookies++
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return fields, cookies
|
||||||
|
}
|
||||||
@@ -0,0 +1,133 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// To add a heuristic, write one file with a type that implements heuristic,
|
||||||
|
// and add one line for it to one of the lists below. Its settings go in
|
||||||
|
// config.Config, and the comment at the top of its file names them.
|
||||||
|
|
||||||
|
// heuristic is one of the checks that decide what servewaf does with a request.
|
||||||
|
type heuristic interface {
|
||||||
|
// name is the heuristic's name, which the log line gives.
|
||||||
|
name() string
|
||||||
|
// check looks at the request at now, and returns nil to let it through.
|
||||||
|
// ctx is the request's own context.
|
||||||
|
check(ctx context.Context, rq *request, now time.Time) *verdict
|
||||||
|
}
|
||||||
|
|
||||||
|
// verdict is a heuristic refusing a request, or banning its client: the action
|
||||||
|
// the log line names, one of requestlog's Action constants, and the figures the
|
||||||
|
// heuristic acted on, by name, such as the count and the limit.
|
||||||
|
type verdict struct {
|
||||||
|
action string
|
||||||
|
figures map[string]any
|
||||||
|
}
|
||||||
|
|
||||||
|
// The names of the figures that more than one heuristic gives.
|
||||||
|
const (
|
||||||
|
figureWindow = "window"
|
||||||
|
figureCount = "count"
|
||||||
|
figureLimit = "limit"
|
||||||
|
figureScore = "score"
|
||||||
|
figureNetblock = "netblock"
|
||||||
|
)
|
||||||
|
|
||||||
|
// The lists of heuristics, each in the order its heuristics run. A client in
|
||||||
|
// SWWAF_ALLOW_NETS skips them all.
|
||||||
|
|
||||||
|
// beforeLookup are the heuristics that run before the client's AS number and
|
||||||
|
// country are looked up, so that a client they refuse is not looked up.
|
||||||
|
func beforeLookup() []heuristic {
|
||||||
|
return []heuristic{
|
||||||
|
denyNets{},
|
||||||
|
activeBan{},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// afterLookup are the heuristics that run once the lookup has been made.
|
||||||
|
func afterLookup() []heuristic {
|
||||||
|
return []heuristic{
|
||||||
|
countries{},
|
||||||
|
blocklists{},
|
||||||
|
crowdSec{},
|
||||||
|
dnsbl{},
|
||||||
|
abuseIPDB{},
|
||||||
|
rateLimits{},
|
||||||
|
trapPaths{},
|
||||||
|
ruleFiles{},
|
||||||
|
headerLimits{},
|
||||||
|
coreRuleSet{},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// afterAnswer are the heuristics that run on a request once servewaf has
|
||||||
|
// answered it, before its decision line is written. They ban the client, and do
|
||||||
|
// not change what the request got: their verdict's action is not used.
|
||||||
|
func afterAnswer() []heuristic {
|
||||||
|
return []heuristic{
|
||||||
|
refusalLimit{},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// onReport are the heuristics that run on an allowed request when rproxy's
|
||||||
|
// report of it comes, before its report line is written. Like afterAnswer, they
|
||||||
|
// only ban.
|
||||||
|
func onReport() []heuristic {
|
||||||
|
return []heuristic{
|
||||||
|
byteLimits{},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// runAfterAnswer runs the heuristics of afterAnswer on the request, at the time
|
||||||
|
// it is called.
|
||||||
|
func (rq *request) runAfterAnswer(ctx context.Context) {
|
||||||
|
rq.runBanningHeuristics(ctx, afterAnswer())
|
||||||
|
}
|
||||||
|
|
||||||
|
// runHeuristics runs list, beforeLookup or afterLookup, on the request at now,
|
||||||
|
// and returns the first verdict, or nil when none gives one. The log line's
|
||||||
|
// heuristic and heuristic_figures name the heuristic that gave it, and give its
|
||||||
|
// figures. ctx is the request's own context.
|
||||||
|
func (rq *request) runHeuristics(
|
||||||
|
ctx context.Context, list []heuristic, now time.Time,
|
||||||
|
) *verdict {
|
||||||
|
name, v := rq.firstVerdict(ctx, list, now)
|
||||||
|
if v != nil {
|
||||||
|
rq.line.Heuristic = name
|
||||||
|
rq.line.HeuristicFigures = v.figures
|
||||||
|
}
|
||||||
|
|
||||||
|
return v
|
||||||
|
}
|
||||||
|
|
||||||
|
// runBanningHeuristics runs list, afterAnswer or onReport, on the request at
|
||||||
|
// the time it is called. The log line's report_heuristic and
|
||||||
|
// report_heuristic_figures name the heuristic that gave a verdict, and give its
|
||||||
|
// figures, so that heuristic and heuristic_figures still name the one that
|
||||||
|
// refused the request. ctx is the request's own context.
|
||||||
|
func (rq *request) runBanningHeuristics(ctx context.Context, list []heuristic) {
|
||||||
|
name, v := rq.firstVerdict(ctx, list, rq.h.now())
|
||||||
|
if v != nil {
|
||||||
|
rq.line.ReportHeuristic = name
|
||||||
|
rq.line.ReportHeuristicFigures = v.figures
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// firstVerdict runs list on the request at now, in order, until a heuristic
|
||||||
|
// gives a verdict, and returns that heuristic's name and its verdict, or "" and
|
||||||
|
// nil when none gives one. ctx is the request's own context.
|
||||||
|
func (rq *request) firstVerdict(
|
||||||
|
ctx context.Context, list []heuristic, now time.Time,
|
||||||
|
) (string, *verdict) {
|
||||||
|
for _, each := range list {
|
||||||
|
v := each.check(ctx, rq, now)
|
||||||
|
if v != nil {
|
||||||
|
return each.name(), v
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return "", nil
|
||||||
|
}
|
||||||
@@ -14,6 +14,18 @@ import (
|
|||||||
// observe is the value of SWWAF_MODE for observe mode.
|
// observe is the value of SWWAF_MODE for observe mode.
|
||||||
const observe = "observe"
|
const observe = "observe"
|
||||||
|
|
||||||
|
func TestObserveModeNamesTheHeuristicThatWouldHaveRefused(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _, _ := startWithClock(t, "", map[string]string{
|
||||||
|
mode: observe, denyNets: client,
|
||||||
|
})
|
||||||
|
|
||||||
|
line := s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
wantWouldAction(t, line, requestlog.ActionDenied)
|
||||||
|
wantHeuristic(t, line, "deny_nets", `{"netblock": "`+client+`/32"}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestObserveModeForwardsWhatEnforceModeRefuses(t *testing.T) {
|
func TestObserveModeForwardsWhatEnforceModeRefuses(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
+14
-23
@@ -4,7 +4,6 @@
|
|||||||
package servewaf
|
package servewaf
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"fmt"
|
|
||||||
"io"
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -61,9 +60,6 @@ type Params struct {
|
|||||||
// LookupFile is the lookup database they are looked up in while
|
// LookupFile is the lookup database they are looked up in while
|
||||||
// SWWAF_LOOKUP_SOURCE is file, and nil otherwise.
|
// SWWAF_LOOKUP_SOURCE is file, and nil otherwise.
|
||||||
LookupFile *lookup.File
|
LookupFile *lookup.File
|
||||||
// CoreRuleSet is what NewCoreRuleSet returns: what inspects requests
|
|
||||||
// while SWWAF_WAF_MODE is not off, and nil while it is.
|
|
||||||
CoreRuleSet *waf.CoreRuleSet
|
|
||||||
// Now tells the time by which requests are counted for the rate
|
// Now tells the time by which requests are counted for the rate
|
||||||
// limits, bans are made and run out, and GeoJS's answers are kept,
|
// limits, bans are made and run out, and GeoJS's answers are kept,
|
||||||
// normally time.Now in UTC, the time the state files give.
|
// normally time.Now in UTC, the time the state files give.
|
||||||
@@ -148,7 +144,7 @@ func newHandler(params Params) *handler {
|
|||||||
dnsbl: dnsbl,
|
dnsbl: dnsbl,
|
||||||
abuseIPDB: abuseIPDB,
|
abuseIPDB: abuseIPDB,
|
||||||
rules: params.Rules,
|
rules: params.Rules,
|
||||||
coreRuleSet: params.CoreRuleSet,
|
coreRuleSet: newCoreRuleSet(params.Config),
|
||||||
alerts: params.Alerts,
|
alerts: params.Alerts,
|
||||||
}
|
}
|
||||||
h.geojs = lookup.New(lookup.Params{
|
h.geojs = lookup.New(lookup.Params{
|
||||||
@@ -206,34 +202,29 @@ func newReputation(
|
|||||||
return lists, dnsbl, abuseIPDB
|
return lists, dnsbl, abuseIPDB
|
||||||
}
|
}
|
||||||
|
|
||||||
// NewCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL,
|
// newCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL,
|
||||||
// without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies
|
// without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies
|
||||||
// up to SWWAF_WAF_BODY_LIMIT, with the exclusions in
|
// up to SWWAF_WAF_BODY_LIMIT, or nil while SWWAF_WAF_MODE is off. It reads at
|
||||||
// SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, or nil while SWWAF_WAF_MODE is off. It
|
// most QuestionBodyMaxBytes less one byte: a question's body of
|
||||||
// reads at most QuestionBodyMaxBytes less one byte: a question's body of
|
|
||||||
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
|
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
|
||||||
// JSON or XML body over its limit. An exclusions file that waf.New refuses is
|
// JSON or XML body over its limit.
|
||||||
// an error in every mode. While the mode is off, a set file is loaded with the
|
func newCoreRuleSet(cfg *config.Config) *waf.CoreRuleSet {
|
||||||
// Core Rule Set only to check it.
|
if cfg.WAFMode == config.WAFModeOff {
|
||||||
func NewCoreRuleSet(cfg *config.Config) (*waf.CoreRuleSet, error) {
|
return nil
|
||||||
if cfg.WAFMode == config.WAFModeOff && cfg.CoreRuleSetExclusionsFile == "" {
|
|
||||||
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
|
|
||||||
}
|
}
|
||||||
|
|
||||||
coreRuleSet, err := waf.New(waf.Params{
|
coreRuleSet, err := waf.New(waf.Params{
|
||||||
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
|
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
|
||||||
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
|
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
|
||||||
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: %w", err)
|
// The Core Rule Set is built in, and the settings cannot break it:
|
||||||
|
// the paranoia level is from 1 to 4, and the id of no rule switches
|
||||||
|
// nothing off.
|
||||||
|
panic(err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.WAFMode == config.WAFModeOff {
|
return coreRuleSet
|
||||||
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
|
|
||||||
}
|
|
||||||
|
|
||||||
return coreRuleSet, nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// handler is servewaf's handler. It holds what every request shares; what
|
// handler is servewaf's handler. It holds what every request shares; what
|
||||||
|
|||||||
@@ -15,8 +15,6 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||||
@@ -349,17 +347,6 @@ func serve(t *testing.T, server *servewaf.Server) {
|
|||||||
// at serveWAFAddr, with the settings in env on top of the defaults, and returns
|
// at serveWAFAddr, with the settings in env on top of the defaults, and returns
|
||||||
// where it listens. What it writes is dropped. SWRP_TIMEOUT_MS is a minute unless
|
// where it listens. What it writes is dropped. SWRP_TIMEOUT_MS is a minute unless
|
||||||
// env sets it, so that on a busy host SWRP_DEFAULT_DISPOSITION does not decide.
|
// env sets it, so that on a busy host SWRP_DEFAULT_DISPOSITION does not decide.
|
||||||
// viperFrom returns a viper Set with each of settings, as the viper
|
|
||||||
// config.NewViper returns one bound to a process's environment.
|
|
||||||
func viperFrom(settings map[string]string) *viper.Viper {
|
|
||||||
v := viper.New()
|
|
||||||
for name, value := range settings {
|
|
||||||
v.Set(name, value)
|
|
||||||
}
|
|
||||||
|
|
||||||
return v
|
|
||||||
}
|
|
||||||
|
|
||||||
func startRProxy(
|
func startRProxy(
|
||||||
t *testing.T, serveWAFAddr, appURL string, env map[string]string,
|
t *testing.T, serveWAFAddr, appURL string, env map[string]string,
|
||||||
) string {
|
) string {
|
||||||
@@ -372,7 +359,11 @@ func startRProxy(
|
|||||||
}
|
}
|
||||||
maps.Copy(settings, env)
|
maps.Copy(settings, env)
|
||||||
|
|
||||||
cfg, err := config.RProxyFromEnvironment(viperFrom(settings))
|
cfg, err := config.RProxyFromEnvironment(func(name string) (string, bool) {
|
||||||
|
value, ok := settings[name]
|
||||||
|
|
||||||
|
return value, ok
|
||||||
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("settings %v: %v", settings, err)
|
t.Fatalf("settings %v: %v", settings, err)
|
||||||
}
|
}
|
||||||
@@ -432,7 +423,11 @@ func newServeWAFWith(
|
|||||||
|
|
||||||
maps.Copy(settings, env)
|
maps.Copy(settings, env)
|
||||||
|
|
||||||
cfg, err := config.FromEnvironment(viperFrom(settings))
|
cfg, err := config.FromEnvironment(func(name string) (string, bool) {
|
||||||
|
value, ok := settings[name]
|
||||||
|
|
||||||
|
return value, ok
|
||||||
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("settings %v: %v", settings, err)
|
t.Fatalf("settings %v: %v", settings, err)
|
||||||
}
|
}
|
||||||
@@ -468,11 +463,6 @@ func newServeWAFWith(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
coreRuleSet, err := servewaf.NewCoreRuleSet(cfg)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("Core Rule Set: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
server := build(servewaf.Params{
|
server := build(servewaf.Params{
|
||||||
Config: cfg,
|
Config: cfg,
|
||||||
RequestLog: out,
|
RequestLog: out,
|
||||||
@@ -480,7 +470,6 @@ func newServeWAFWith(
|
|||||||
GeoJSURL: geojsURL,
|
GeoJSURL: geojsURL,
|
||||||
AbuseIPDBURL: abuseIPDBURL,
|
AbuseIPDBURL: abuseIPDBURL,
|
||||||
LookupFile: lookupFile,
|
LookupFile: lookupFile,
|
||||||
CoreRuleSet: coreRuleSet,
|
|
||||||
Now: now,
|
Now: now,
|
||||||
Rules: ruleFiles,
|
Rules: ruleFiles,
|
||||||
Alerts: alertQueue,
|
Alerts: alertQueue,
|
||||||
|
|||||||
@@ -0,0 +1,65 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic rate_limits. It reads SWWAF_RATE_LIMIT_EXEMPT_NETS,
|
||||||
|
// SWWAF_RATE_LIMIT_EXEMPT_PATHS and SWWAF_BAN_RESPONSE, and the rate limits and
|
||||||
|
// biased thresholds through the limiter and limitPercentages.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// rateLimits counts the request for the rate limits, unless
|
||||||
|
// countedByRateLimits exempts it, by the client's limit percentages, which it
|
||||||
|
// notes for the log line with the client's counts. A request that takes the
|
||||||
|
// client over a rate limit, as its limit percentage lowers it, breaks it: it is
|
||||||
|
// refused, and bans the client. A request exempt from the rate limits is exempt
|
||||||
|
// from the byte limits too. Its figures are the window, the count and the
|
||||||
|
// limit.
|
||||||
|
type rateLimits struct{}
|
||||||
|
|
||||||
|
func (rateLimits) name() string { return "rate_limits" }
|
||||||
|
|
||||||
|
func (rateLimits) check(_ context.Context, rq *request, now time.Time) *verdict {
|
||||||
|
rq.counted = rq.countedByRateLimits()
|
||||||
|
if !rq.counted {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.limitPercent, rq.bytesPercent = rq.limitPercentages()
|
||||||
|
rq.line.LimitPercent, rq.line.LimitPercentSetting = rq.limitPercent.logged()
|
||||||
|
rq.line.BytesPercent, rq.line.BytesPercentSetting = rq.bytesPercent.logged()
|
||||||
|
|
||||||
|
counts, hit, over := rq.h.limiter.Count(rq.h.clientGroup(rq.client), now,
|
||||||
|
rq.limitPercent.percent)
|
||||||
|
rq.line.Counts = counts
|
||||||
|
|
||||||
|
if !over {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.banForLimit(now, hit, rq.h.config.BanResponse)
|
||||||
|
|
||||||
|
return &verdict{action: requestlog.ActionRateLimited, figures: limitFigures(hit)}
|
||||||
|
}
|
||||||
|
|
||||||
|
// overAWholeRateLimit reports whether the heuristic rate_limits will refuse the
|
||||||
|
// request at now, whatever its client's limit percentages: the rate limits
|
||||||
|
// count it, and one more request from its client takes it over a whole rate
|
||||||
|
// limit. It counts nothing.
|
||||||
|
func (rq *request) overAWholeRateLimit(now time.Time) bool {
|
||||||
|
return rq.countedByRateLimits() &&
|
||||||
|
rq.h.limiter.OverARateLimit(rq.h.clientGroup(rq.client), now)
|
||||||
|
}
|
||||||
|
|
||||||
|
// countedByRateLimits reports whether the rate limits count the request: its
|
||||||
|
// client is not in SWWAF_RATE_LIMIT_EXEMPT_NETS, and its path is not exempt
|
||||||
|
// under SWWAF_RATE_LIMIT_EXEMPT_PATHS.
|
||||||
|
func (rq *request) countedByRateLimits() bool {
|
||||||
|
cfg := rq.h.config
|
||||||
|
|
||||||
|
return !isInside(rq.client, cfg.RateLimitExemptNets) &&
|
||||||
|
!pathExempt(rq.in, cfg.RateLimitExemptPaths)
|
||||||
|
}
|
||||||
@@ -72,6 +72,17 @@ func TestRateLimitRefusesBeforeTheApp(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRateLimitRefusalNamesItsHeuristicWindowCountAndLimit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _, _ := startWithClock(t, "", map[string]string{rateLimitPerMinute: "1"})
|
||||||
|
|
||||||
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
|
|
||||||
|
line := s.get(client, http.StatusForbidden, requestlog.ActionRateLimited)
|
||||||
|
wantHeuristic(t, line, "rate_limits", `{"window": "minute", "count": 2, "limit": 1}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestIPv6GroupPrefixSetsTheClientTheLimitsCount(t *testing.T) {
|
func TestIPv6GroupPrefixSetsTheClientTheLimitsCount(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,54 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic refusal_limit. It reads SWWAF_ERROR_BURST_THRESHOLD and
|
||||||
|
// SWWAF_BAN_RESPONSE.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
)
|
||||||
|
|
||||||
|
// refusalLimit counts the request once it has been answered, if servewaf
|
||||||
|
// refused it after a rule file match, a trap path or a Core Rule Set match, and
|
||||||
|
// in observe mode if enforce mode would have: more than
|
||||||
|
// SWWAF_ERROR_BURST_THRESHOLD such refusals of the client within a minute break
|
||||||
|
// a limit, which bans the client. A client in SWWAF_ALLOW_NETS, which the
|
||||||
|
// heuristics skip, is not counted, and nothing is while the threshold is off.
|
||||||
|
// Its figures are the window, the count and the limit.
|
||||||
|
type refusalLimit struct{}
|
||||||
|
|
||||||
|
func (refusalLimit) name() string { return "refusal_limit" }
|
||||||
|
|
||||||
|
func (refusalLimit) check(_ context.Context, rq *request, now time.Time) *verdict {
|
||||||
|
cfg := rq.h.config
|
||||||
|
if cfg.ErrorBurstThreshold == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
if !rq.attack && !rq.ruleBlocked && !rq.wafBlocked {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
hit, over := rq.h.limiter.CountRefusal(rq.h.clientGroup(rq.client), now,
|
||||||
|
cfg.ErrorBurstThreshold)
|
||||||
|
if !over {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// What the client was sent, or in observe mode would have been.
|
||||||
|
status := rq.out.status
|
||||||
|
|
||||||
|
switch rq.line.WouldAction {
|
||||||
|
case requestlog.ActionRuleBlocked, requestlog.ActionWAFBlocked:
|
||||||
|
status = http.StatusForbidden
|
||||||
|
case requestlog.ActionBanned:
|
||||||
|
status = cfg.BanResponse
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.banForLimit(now, hit, status)
|
||||||
|
|
||||||
|
return &verdict{action: requestlog.ActionBanned, figures: limitFigures(hit)}
|
||||||
|
}
|
||||||
@@ -106,6 +106,25 @@ func TestErrorBurstBreaksAtOneOverTheThreshold(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRefusalLimitBanNamesItsHeuristicWindowCountAndLimit(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _, _ := startForErrorBurst(t, "1", nil)
|
||||||
|
|
||||||
|
const ruleFigures = `{"rule_id": "blocked", "rule_action": "block"}`
|
||||||
|
|
||||||
|
line := blockRule.send(s, client)
|
||||||
|
wantHeuristic(t, line, "rule_files", ruleFigures)
|
||||||
|
wantReportHeuristic(t, line, "", "null")
|
||||||
|
|
||||||
|
// The second refusal is the request refusal_limit bans for, and the line
|
||||||
|
// names both.
|
||||||
|
line = blockRule.send(s, client)
|
||||||
|
wantHeuristic(t, line, "rule_files", ruleFigures)
|
||||||
|
wantReportHeuristic(t, line, "refusal_limit",
|
||||||
|
`{"window": "minute", "count": 2, "limit": 1}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestErrorBurstBanNotesHistoryAndMetrics(t *testing.T) {
|
func TestErrorBurstBanNotesHistoryAndMetrics(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -1,91 +1,17 @@
|
|||||||
package servewaf
|
package servewaf
|
||||||
|
|
||||||
import (
|
// What the reputation heuristics share: blocklists, crowdsec, dnsbl and
|
||||||
"context"
|
// abuseipdb.
|
||||||
"time"
|
|
||||||
|
|
||||||
|
import (
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// deny is the SWWAF_BLOCKLIST_ACTION and the SWWAF_REPUTATION_ACTION that
|
// deny is the SWWAF_BLOCKLIST_ACTION and the SWWAF_REPUTATION_ACTION that
|
||||||
// refuses the requests of a client a source lists.
|
// refuses the requests of a client a source lists.
|
||||||
const deny = "deny"
|
const deny = "deny"
|
||||||
|
|
||||||
// blocklistDenied notes the blocklists that list the client, as
|
|
||||||
// noteListed does, and reports whether SWWAF_BLOCKLIST_ACTION, being deny,
|
|
||||||
// refuses the request. Being limit, it lowers the client's limits instead
|
|
||||||
// (see limitPercentages), and being log, it does nothing more.
|
|
||||||
func (rq *request) blocklistDenied() bool {
|
|
||||||
listedBy := rq.h.lists.ListedBy(rq.client)
|
|
||||||
rq.blocklisted = len(listedBy) > 0
|
|
||||||
rq.noteListed(listedBy, "listed by a blocklist")
|
|
||||||
|
|
||||||
return rq.blocklisted && rq.h.config.BlocklistAction == deny
|
|
||||||
}
|
|
||||||
|
|
||||||
// crowdSecBanned reports whether a decision of the CrowdSec decision list
|
|
||||||
// on the client is in force at now. If one is, it notes the list, as
|
|
||||||
// noteHit does, and bans the client until that decision ends.
|
|
||||||
func (rq *request) crowdSecBanned(now time.Time) bool {
|
|
||||||
decision, listed := rq.h.lists.CrowdSecDecision(rq.client, now)
|
|
||||||
if !listed {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
rq.noteHit(bans.ReputationHit{Source: rq.h.config.CrowdSecDecisionsURL},
|
|
||||||
"listed by the CrowdSec decision list")
|
|
||||||
rq.banForCrowdSec(now, decision)
|
|
||||||
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
// dnsblDenied notes the DNSBL zones whose verdict lists the client, as
|
|
||||||
// noteListed does, and reports whether SWWAF_REPUTATION_ACTION, being
|
|
||||||
// deny, refuses the request. Being limit, it lowers the client's limits
|
|
||||||
// instead (see limitPercentages), and being log, it does nothing more. A
|
|
||||||
// zone without a verdict on the client is asked about it in the
|
|
||||||
// background, and the request does not wait for the answer. ctx is the
|
|
||||||
// request's own context.
|
|
||||||
func (rq *request) dnsblDenied(ctx context.Context) bool {
|
|
||||||
listedBy := rq.h.dnsbl.ListedBy(ctx, rq.client)
|
|
||||||
rq.dnsblListed = len(listedBy) > 0
|
|
||||||
rq.noteListed(listedBy, "listed by a DNSBL zone")
|
|
||||||
|
|
||||||
return rq.dnsblListed && rq.h.config.ReputationAction == deny
|
|
||||||
}
|
|
||||||
|
|
||||||
// abuseIPDBDenied notes AbuseIPDB, as noteHit does, with the score, when
|
|
||||||
// its score of the client is a hit, and reports whether
|
|
||||||
// SWWAF_REPUTATION_ACTION, being deny, refuses the request, as dnsblDenied
|
|
||||||
// does for a zone. While SWWAF_ABUSEIPDB_KEY is unset it does nothing. A
|
|
||||||
// client without a score is checked in the background, by the request's
|
|
||||||
// address, if its history counts an offence, and the request does not
|
|
||||||
// wait for the answer. The score is then used for each address of the
|
|
||||||
// client. ctx is the request's own context.
|
|
||||||
func (rq *request) abuseIPDBDenied(ctx context.Context) bool {
|
|
||||||
if rq.h.config.AbuseIPDBKey == "" {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
client := rq.h.clientGroup(rq.client)
|
|
||||||
held, _ := rq.h.limiter.Client(client)
|
|
||||||
offender := held.History.Offences != ratelimit.Offences{}
|
|
||||||
|
|
||||||
score, hit := rq.h.abuseIPDB.Hit(ctx, client, rq.client, offender)
|
|
||||||
if !hit {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
rq.abuseIPDBHit = true
|
|
||||||
rq.noteHit(bans.ReputationHit{Source: reputation.AbuseIPDBSource, Score: &score},
|
|
||||||
"scored by AbuseIPDB at or over SWWAF_ABUSEIPDB_MIN_SCORE")
|
|
||||||
|
|
||||||
return rq.h.config.ReputationAction == deny
|
|
||||||
}
|
|
||||||
|
|
||||||
// noteListed notes each of sources, the URLs of the blocklists or the
|
// noteListed notes each of sources, the URLs of the blocklists or the
|
||||||
// DNSBL zones, their keys masked, that list the client, as noteHit does,
|
// DNSBL zones, their keys masked, that list the client, as noteHit does,
|
||||||
// with reason.
|
// with reason.
|
||||||
|
|||||||
@@ -118,6 +118,56 @@ func TestEachBlocklistActionForAListedAddressAndAListedNetblock(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestReputationRefusalNamesItsHeuristicAndWhatListedTheClient(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, tc := range []struct {
|
||||||
|
heuristic string
|
||||||
|
env map[string]string
|
||||||
|
load func(*testing.T, *servewaf.Server)
|
||||||
|
figures string
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
"blocklists",
|
||||||
|
map[string]string{blocklistURLs: dropURL, blocklistAction: actionDeny},
|
||||||
|
func(t *testing.T, server *servewaf.Server) {
|
||||||
|
t.Helper()
|
||||||
|
loadLists(t, server, map[string][]string{dropURL: {client}})
|
||||||
|
},
|
||||||
|
`{"lists": ["` + dropURL + `"]}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"dnsbl",
|
||||||
|
map[string]string{
|
||||||
|
dnsblZones: dnsblZone, dnsblResolver: noResolver,
|
||||||
|
reputationAction: actionDeny,
|
||||||
|
},
|
||||||
|
func(_ *testing.T, server *servewaf.Server) {
|
||||||
|
loadVerdicts(server, map[string][]string{client: {dnsblZone}})
|
||||||
|
},
|
||||||
|
`{"zones": ["` + dnsblZone + `"]}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"abuseipdb",
|
||||||
|
map[string]string{abuseIPDBKey: accountKey, reputationAction: actionDeny},
|
||||||
|
func(_ *testing.T, server *servewaf.Server) {
|
||||||
|
loadScores(server, map[string]int64{client: 80})
|
||||||
|
},
|
||||||
|
`{"score": 80, "min_score": 75}`,
|
||||||
|
},
|
||||||
|
} {
|
||||||
|
t.Run(tc.heuristic, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, server, _ := startWithLookups(t, tc.env)
|
||||||
|
tc.load(t, server)
|
||||||
|
|
||||||
|
line := s.get(client, http.StatusForbidden, requestlog.ActionDenied)
|
||||||
|
wantHeuristic(t, line, tc.heuristic, tc.figures)
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestBlocklistsComeAfterTheCountryListsAndSkipAllowNets(t *testing.T) {
|
func TestBlocklistsComeAfterTheCountryListsAndSkipAllowNets(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -160,16 +160,20 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// check is the one place where a request can be refused. It returns nil to let
|
// check is the one place where a request can be refused. It returns nil to let
|
||||||
// the request through. The checks of checkClient decide: a request they refuse
|
// the request through. The heuristics of checkClient decide: a request they
|
||||||
// is answered with SWWAF_BAN_RESPONSE, 403 for a block rule or the Core Rule Set,
|
// refuse is answered with SWWAF_BAN_RESPONSE, 403 for a block rule or the Core
|
||||||
// or 431 for the header limits. In observe mode nothing is refused, and the log
|
// Rule Set, or 431 for the header limits. In observe mode nothing is refused,
|
||||||
// line names what enforce mode would have done. ctx is the request's own context.
|
// and the log line names what enforce mode would have done. ctx is the
|
||||||
|
// request's own context.
|
||||||
func (rq *request) check(ctx context.Context) *refusal {
|
func (rq *request) check(ctx context.Context) *refusal {
|
||||||
action := rq.checkClient(ctx)
|
v := rq.checkClient(ctx)
|
||||||
|
if v == nil {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
action := v.action
|
||||||
|
|
||||||
switch {
|
switch {
|
||||||
case action == "":
|
|
||||||
return nil
|
|
||||||
case rq.h.config.Observe:
|
case rq.h.config.Observe:
|
||||||
rq.line.WouldAction = action
|
rq.line.WouldAction = action
|
||||||
|
|
||||||
@@ -183,35 +187,21 @@ func (rq *request) check(ctx context.Context) *refusal {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// checkClient runs the checks on the request's client, and returns the
|
// checkClient runs the heuristics of beforeLookup on the request, then looks
|
||||||
// action of the first that refuses the request, or "" when none does. A
|
// up the client's AS number and country, then runs those of afterLookup. It
|
||||||
// client in SWWAF_ALLOW_NETS skips them, and is not looked up. For any
|
// returns the first verdict, or nil when none gives one. A client in
|
||||||
// other client, SWWAF_DENY_NETS comes first, then a ban on its netblock,
|
// SWWAF_ALLOW_NETS skips them all, and is not looked up. ctx is the request's
|
||||||
// so that a client either refuses is not looked up, then the lookup of
|
// own context.
|
||||||
// its AS number and country, then the country lists, then the blocklists,
|
func (rq *request) checkClient(ctx context.Context) *verdict {
|
||||||
// then the CrowdSec decision list, which bans the client it lists, then
|
if isInside(rq.client, rq.h.config.AllowNets) {
|
||||||
// the DNSBL zones' verdicts, and then AbuseIPDB's score; a request any of
|
return nil
|
||||||
// them refuses is not counted for the rate limits. Then come the
|
|
||||||
// rate limits, unless the client is in SWWAF_RATE_LIMIT_EXEMPT_NETS or the
|
|
||||||
// request's path is exempt under SWWAF_RATE_LIMIT_EXEMPT_PATHS, so that
|
|
||||||
// every other request is counted, each of them by the client's limit
|
|
||||||
// percentages, then SWWAF_TRAP_PATHS, then the rule files, then the header
|
|
||||||
// limits, and last the Core Rule Set. A request exempt from the rate limits
|
|
||||||
// is exempt from the byte limits too. ctx is the request's own context.
|
|
||||||
func (rq *request) checkClient(ctx context.Context) string {
|
|
||||||
cfg := rq.h.config
|
|
||||||
if isInside(rq.client, cfg.AllowNets) {
|
|
||||||
return ""
|
|
||||||
}
|
}
|
||||||
|
|
||||||
now := rq.h.now()
|
now := rq.h.now()
|
||||||
|
|
||||||
if isInside(rq.client, cfg.DenyNets) {
|
v := rq.runHeuristics(ctx, beforeLookup(), now)
|
||||||
return requestlog.ActionDenied
|
if v != nil {
|
||||||
}
|
return v
|
||||||
|
|
||||||
if rq.banned(now) {
|
|
||||||
return requestlog.ActionBanned
|
|
||||||
}
|
}
|
||||||
|
|
||||||
rq.lookUp(ctx, now)
|
rq.lookUp(ctx, now)
|
||||||
@@ -220,59 +210,7 @@ func (rq *request) checkClient(ctx context.Context) string {
|
|||||||
// again: a request counted at the time it arrived could be counted
|
// again: a request counted at the time it arrived could be counted
|
||||||
// after a later one of its client's, which the rate limits would take
|
// after a later one of its client's, which the rate limits would take
|
||||||
// for the clock set back.
|
// for the clock set back.
|
||||||
now = rq.h.now()
|
return rq.runHeuristics(ctx, afterLookup(), rq.h.now())
|
||||||
|
|
||||||
if rq.countryDenied() {
|
|
||||||
return requestlog.ActionCountryDenied
|
|
||||||
}
|
|
||||||
|
|
||||||
if rq.blocklistDenied() {
|
|
||||||
return requestlog.ActionDenied
|
|
||||||
}
|
|
||||||
|
|
||||||
if rq.crowdSecBanned(now) {
|
|
||||||
return requestlog.ActionBanned
|
|
||||||
}
|
|
||||||
|
|
||||||
if rq.dnsblDenied(ctx) || rq.abuseIPDBDenied(ctx) {
|
|
||||||
return requestlog.ActionDenied
|
|
||||||
}
|
|
||||||
|
|
||||||
rq.counted = rq.countedByRateLimits()
|
|
||||||
if rq.counted {
|
|
||||||
rq.limitPercent, rq.bytesPercent = rq.limitPercentages()
|
|
||||||
rq.line.LimitPercent, rq.line.LimitPercentSetting = rq.limitPercent.logged()
|
|
||||||
rq.line.BytesPercent, rq.line.BytesPercentSetting = rq.bytesPercent.logged()
|
|
||||||
}
|
|
||||||
|
|
||||||
if rq.counted && rq.limitBroken(now) {
|
|
||||||
return requestlog.ActionRateLimited
|
|
||||||
}
|
|
||||||
|
|
||||||
if rq.trapPath(now) {
|
|
||||||
return requestlog.ActionBanned
|
|
||||||
}
|
|
||||||
|
|
||||||
action := rq.checkRules(now)
|
|
||||||
if action != "" {
|
|
||||||
return action
|
|
||||||
}
|
|
||||||
|
|
||||||
if rq.overHeaderLimits() {
|
|
||||||
return requestlog.ActionTooLarge
|
|
||||||
}
|
|
||||||
|
|
||||||
return rq.checkCoreRuleSet()
|
|
||||||
}
|
|
||||||
|
|
||||||
// countedByRateLimits reports whether the rate limits count the request: its
|
|
||||||
// client is not in SWWAF_RATE_LIMIT_EXEMPT_NETS, and its path is not exempt
|
|
||||||
// under SWWAF_RATE_LIMIT_EXEMPT_PATHS.
|
|
||||||
func (rq *request) countedByRateLimits() bool {
|
|
||||||
cfg := rq.h.config
|
|
||||||
|
|
||||||
return !isInside(rq.client, cfg.RateLimitExemptNets) &&
|
|
||||||
!pathExempt(rq.in, cfg.RateLimitExemptPaths)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// pathExempt reports whether r is exempt under prefixes,
|
// pathExempt reports whether r is exempt under prefixes,
|
||||||
|
|||||||
@@ -1,6 +1,7 @@
|
|||||||
package servewaf_test
|
package servewaf_test
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"encoding/json"
|
||||||
"io"
|
"io"
|
||||||
"math"
|
"math"
|
||||||
"net/http"
|
"net/http"
|
||||||
@@ -19,7 +20,7 @@ const (
|
|||||||
// requestIDHeader carries the request's id.
|
// requestIDHeader carries the request's id.
|
||||||
requestIDHeader = "X-Request-ID"
|
requestIDHeader = "X-Request-ID"
|
||||||
// instance is the SWWAF_INSTANCE_NAME a test sets.
|
// instance is the SWWAF_INSTANCE_NAME a test sets.
|
||||||
instance = "fsn1app1/blog"
|
instance = "fsn1app1/gitea"
|
||||||
// ipv6Client is a client on IPv6, and ipv6Group the netblock the rate
|
// ipv6Client is a client on IPv6, and ipv6Group the netblock the rate
|
||||||
// limits count it as.
|
// limits count it as.
|
||||||
ipv6Client = "2001:db8::7"
|
ipv6Client = "2001:db8::7"
|
||||||
@@ -272,6 +273,47 @@ func wantTimings(t *testing.T, line logLine, want ...string) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// wantHeuristic checks that line names heuristic as the one that refused the
|
||||||
|
// request, with figures, a JSON object. A heuristic of "" with figures "null" is
|
||||||
|
// a line that names none.
|
||||||
|
func wantHeuristic(t *testing.T, line logLine, heuristic, figures string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
if line.Heuristic != heuristic ||
|
||||||
|
!reflect.DeepEqual(line.HeuristicFigures, decodeFigures(t, figures)) {
|
||||||
|
t.Errorf("log line has heuristic %q with heuristic_figures %v, want %q with %s",
|
||||||
|
line.Heuristic, line.HeuristicFigures, heuristic, figures)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// wantReportHeuristic checks report_heuristic and report_heuristic_figures as
|
||||||
|
// wantHeuristic checks heuristic and heuristic_figures.
|
||||||
|
func wantReportHeuristic(t *testing.T, line logLine, heuristic, figures string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
if line.ReportHeuristic != heuristic ||
|
||||||
|
!reflect.DeepEqual(line.ReportHeuristicFigures, decodeFigures(t, figures)) {
|
||||||
|
t.Errorf("log line has report_heuristic %q with report_heuristic_figures %v, "+
|
||||||
|
"want %q with %s",
|
||||||
|
line.ReportHeuristic, line.ReportHeuristicFigures, heuristic, figures)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// decodeFigures decodes figures, a JSON object of a heuristic's figures, as
|
||||||
|
// the log line's figures are decoded.
|
||||||
|
func decodeFigures(t *testing.T, figures string) map[string]any {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var decoded map[string]any
|
||||||
|
|
||||||
|
err := json.Unmarshal([]byte(figures), &decoded)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("decode %s: %v", figures, err)
|
||||||
|
}
|
||||||
|
|
||||||
|
return decoded
|
||||||
|
}
|
||||||
|
|
||||||
// microseconds is a timing in whole microseconds.
|
// microseconds is a timing in whole microseconds.
|
||||||
func microseconds(milliseconds float64) int64 {
|
func microseconds(milliseconds float64) int64 {
|
||||||
return int64(math.Round(milliseconds * 1000))
|
return int64(math.Round(milliseconds * 1000))
|
||||||
|
|||||||
@@ -0,0 +1,55 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic rule_files. It reads the rules of the rule files, which
|
||||||
|
// SWWAF_RULES_DIR and SWWAF_RULES_ENABLED name.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
|
)
|
||||||
|
|
||||||
|
// ruleFiles checks the request against the rules of the rule files at now,
|
||||||
|
// notes the ids of those it matches in the log line, and refuses the request
|
||||||
|
// when the last rule it matches refuses it: with ActionRuleBlocked for a block
|
||||||
|
// rule, and ActionBanned for a ban rule. A ban rule bans the client's netblock
|
||||||
|
// for a clear sign of attack, or in observe mode raises the alert for the ban
|
||||||
|
// it would have made. Either rule's match is noted as an offence, for the
|
||||||
|
// client's history. Its figures are the rule's id and action.
|
||||||
|
type ruleFiles struct{}
|
||||||
|
|
||||||
|
func (ruleFiles) name() string { return "rule_files" }
|
||||||
|
|
||||||
|
func (ruleFiles) check(_ context.Context, rq *request, now time.Time) *verdict {
|
||||||
|
matched := rq.h.rules.Match(rq.in)
|
||||||
|
|
||||||
|
for _, rule := range matched {
|
||||||
|
rq.line.RuleIDs = append(rq.line.RuleIDs, rule.ID)
|
||||||
|
rq.h.metrics.RuleMatched(rule.ID, rule.Action)
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(matched) == 0 {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
// Only the last rule matched can refuse the request.
|
||||||
|
last := matched[len(matched)-1]
|
||||||
|
figures := map[string]any{"rule_id": last.ID, "rule_action": last.Action}
|
||||||
|
|
||||||
|
switch last.Action {
|
||||||
|
case rules.ActionBlock:
|
||||||
|
rq.ruleBlocked = true
|
||||||
|
|
||||||
|
return &verdict{action: requestlog.ActionRuleBlocked, figures: figures}
|
||||||
|
case rules.ActionBan:
|
||||||
|
rq.attack = true
|
||||||
|
rq.banForAttack(now, bans.Notes{RuleID: last.ID, Target: last.Target})
|
||||||
|
|
||||||
|
return &verdict{action: requestlog.ActionBanned, figures: figures}
|
||||||
|
default:
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -93,6 +93,21 @@ func TestEachRuleAction(t *testing.T) {
|
|||||||
s.get(client, http.StatusTooManyRequests, requestlog.ActionBanned)
|
s.get(client, http.StatusTooManyRequests, requestlog.ActionBanned)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestRuleRefusalNamesItsHeuristicAndTheRule(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _, _ := startWithClock(t, "", map[string]string{
|
||||||
|
rulesDir: writeRules(t, "noted path log ^/\n"+testRules),
|
||||||
|
})
|
||||||
|
|
||||||
|
line := s.request(client, "/blocked", http.StatusForbidden,
|
||||||
|
requestlog.ActionRuleBlocked)
|
||||||
|
wantHeuristic(t, line, "rule_files", `{"rule_id": "blocked", "rule_action": "block"}`)
|
||||||
|
|
||||||
|
line = s.request(client, "/.env", http.StatusForbidden, requestlog.ActionBanned)
|
||||||
|
wantHeuristic(t, line, "rule_files", `{"rule_id": "probe", "rule_action": "ban"}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestNextClearSignOfAttackAfterABanBansPermanently(t *testing.T) {
|
func TestNextClearSignOfAttackAfterABanBansPermanently(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -1,64 +0,0 @@
|
|||||||
package servewaf
|
|
||||||
|
|
||||||
import (
|
|
||||||
"slices"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
|
||||||
)
|
|
||||||
|
|
||||||
// trapPath reports whether the request asks for a path in
|
|
||||||
// SWWAF_TRAP_PATHS: its path as a path rule sees it, without the query,
|
|
||||||
// as sent or once percent-decoded, is one of them. Such a request is a
|
|
||||||
// clear sign of attack, as a ban rule's match is: it bans the client's
|
|
||||||
// netblock, or in observe mode raises the alert for the ban it would have
|
|
||||||
// made. The ban's notes give the path as sent.
|
|
||||||
func (rq *request) trapPath(now time.Time) bool {
|
|
||||||
path := rules.Path(rq.in)
|
|
||||||
if !slices.Contains(rq.h.config.TrapPaths, path) &&
|
|
||||||
!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
rq.attack = true
|
|
||||||
rq.banForAttack(now, bans.Notes{TrapPath: path})
|
|
||||||
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
|
|
||||||
// checkRules checks the request against the rules of the rule files at
|
|
||||||
// now, notes the ids of those it matches in the log line, and returns the
|
|
||||||
// action of the rule that refuses it, ActionRuleBlocked for a block rule
|
|
||||||
// and ActionBanned for a ban rule, or "" when none does. A ban rule bans
|
|
||||||
// the client's netblock for a clear sign of attack, or in observe mode
|
|
||||||
// raises the alert for the ban it would have made. Either rule's match
|
|
||||||
// is noted as an offence, for the client's history.
|
|
||||||
func (rq *request) checkRules(now time.Time) string {
|
|
||||||
matched := rq.h.rules.Match(rq.in)
|
|
||||||
|
|
||||||
for _, rule := range matched {
|
|
||||||
rq.line.RuleIDs = append(rq.line.RuleIDs, rule.ID)
|
|
||||||
rq.h.metrics.RuleMatched(rule.ID, rule.Action)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(matched) == 0 {
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
|
|
||||||
// Only the last rule matched can refuse the request.
|
|
||||||
switch last := matched[len(matched)-1]; last.Action {
|
|
||||||
case rules.ActionBlock:
|
|
||||||
rq.ruleBlocked = true
|
|
||||||
|
|
||||||
return requestlog.ActionRuleBlocked
|
|
||||||
case rules.ActionBan:
|
|
||||||
rq.attack = true
|
|
||||||
rq.banForAttack(now, bans.Notes{RuleID: last.ID, Target: last.Target})
|
|
||||||
|
|
||||||
return requestlog.ActionBanned
|
|
||||||
default:
|
|
||||||
return ""
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -2,6 +2,7 @@ package servewaf
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"bytes"
|
"bytes"
|
||||||
|
"context"
|
||||||
"encoding/json"
|
"encoding/json"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
@@ -52,13 +53,6 @@ const ReportPath = "/_smallwebwaf/report"
|
|||||||
// QuestionBodyMaxBytes is the most of a visitor's body a question carries.
|
// QuestionBodyMaxBytes is the most of a visitor's body a question carries.
|
||||||
const QuestionBodyMaxBytes = 64 << 10
|
const QuestionBodyMaxBytes = 64 << 10
|
||||||
|
|
||||||
// The header limits: the most cookies and header fields a request may have
|
|
||||||
// when the checks reach the Core Rule Set. Real clients never reach them.
|
|
||||||
const (
|
|
||||||
maxCookies = 50
|
|
||||||
maxHeaderFields = 100
|
|
||||||
)
|
|
||||||
|
|
||||||
const (
|
const (
|
||||||
// maxAwaitingReport is the most answered requests held until their
|
// maxAwaitingReport is the most answered requests held until their
|
||||||
// reports come. Past it, the request answered longest ago is dropped.
|
// reports come. Past it, the request answered longest ago is dropped.
|
||||||
@@ -147,48 +141,20 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
|
|
||||||
// refusedWhateverTheAnswer reports whether the request is one that does not wait
|
// refusedWhateverTheAnswer reports whether the request is one that does not wait
|
||||||
// for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
|
// for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
|
||||||
// its client: it is over the header limits, a blocklist lists its client while
|
// its client: the heuristic crowdsec, header_limits, blocklists or rate_limits
|
||||||
// SWWAF_BLOCKLIST_ACTION is deny, the CrowdSec decision list lists it, or it is
|
// will refuse it, as a function in that heuristic's file says. In observe mode,
|
||||||
// over a whole rate limit. In observe mode, which refuses nothing, none is.
|
// which refuses nothing, none is. rproxy lets a request through when servewaf
|
||||||
// rproxy lets a request through when servewaf answers too late.
|
// answers too late.
|
||||||
func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
|
func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
|
||||||
if rq.h.config.Observe {
|
if rq.h.config.Observe {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
lists := rq.h.lists
|
_, decided := rq.crowdSecDecisionInForce(now)
|
||||||
_, decided := lists.CrowdSecDecision(rq.client, now)
|
|
||||||
|
|
||||||
return decided || rq.overHeaderLimits() ||
|
return decided || rq.overHeaderLimits() ||
|
||||||
(rq.h.config.BlocklistAction == deny && len(lists.ListedBy(rq.client)) > 0) ||
|
rq.refusedByBlocklists(rq.h.lists.ListedBy(rq.client)) ||
|
||||||
(rq.countedByRateLimits() &&
|
rq.overAWholeRateLimit(now)
|
||||||
rq.h.limiter.OverARateLimit(rq.h.clientGroup(rq.client), now))
|
|
||||||
}
|
|
||||||
|
|
||||||
// overHeaderLimits reports whether servewaf refuses the request for its
|
|
||||||
// headers: it has more than maxHeaderFields fields, each value of a header sent
|
|
||||||
// twice being one, or more than maxCookies cookies, each part of a Cookie header
|
|
||||||
// between semicolons being one unless it is empty or only spaces and tabs. The
|
|
||||||
// Core Rule Set runs each of its rules on each cookie, so that thousands of small
|
|
||||||
// cookies keep it busy for over a second. It trims only spaces and control
|
|
||||||
// characters, of which a header value can hold only tabs, so a part of Unicode
|
|
||||||
// spaces is a cookie to it.
|
|
||||||
func (rq *request) overHeaderLimits() bool {
|
|
||||||
fields, cookies := 0, 0
|
|
||||||
|
|
||||||
for _, values := range rq.in.Header {
|
|
||||||
fields += len(values)
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, value := range rq.in.Header.Values("Cookie") {
|
|
||||||
for part := range strings.SplitSeq(value, ";") {
|
|
||||||
if strings.Trim(part, " \t") != "" {
|
|
||||||
cookies++
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return fields > maxHeaderFields || cookies > maxCookies
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// answerQuestion runs the checks on the visitor's request a question
|
// answerQuestion runs the checks on the visitor's request a question
|
||||||
@@ -219,18 +185,18 @@ func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
|
|||||||
|
|
||||||
if refused == nil && !underOwnPrefix {
|
if refused == nil && !underOwnPrefix {
|
||||||
// In observe mode, what enforce mode would have refused counts here.
|
// In observe mode, what enforce mode would have refused counts here.
|
||||||
defer rq.countRefusal()
|
defer rq.runAfterAnswer(r.Context())
|
||||||
|
|
||||||
rq.allow()
|
rq.allow()
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// The last deferred runs first: countRefusal before addToHistory, so that a
|
// The last deferred runs first: runAfterAnswer before addToHistory, so that
|
||||||
// broken error burst is in the client's history.
|
// a broken refusal limit is in the client's history.
|
||||||
defer rq.addToHistory()
|
defer rq.addToHistory()
|
||||||
defer rq.countAnomalies()
|
defer rq.countAnomalies()
|
||||||
defer rq.countRefusal()
|
defer rq.runAfterAnswer(r.Context())
|
||||||
|
|
||||||
if refused != nil {
|
if refused != nil {
|
||||||
rq.deny(*refused)
|
rq.deny(*refused)
|
||||||
@@ -412,7 +378,7 @@ func (h *handler) takeReport(w http.ResponseWriter, r *http.Request) {
|
|||||||
// Remove reports true to one report only, of several that name the id. A
|
// Remove reports true to one report only, of several that name the id. A
|
||||||
// question still being handled is held as nil.
|
// question still being handled is held as nil.
|
||||||
if found && held != nil && h.awaitingReport.Remove(report.RequestID) {
|
if found && held != nil && h.awaitingReport.Remove(report.RequestID) {
|
||||||
h.finishWith(held, report)
|
h.finishWith(r.Context(), held, report)
|
||||||
}
|
}
|
||||||
|
|
||||||
w.WriteHeader(http.StatusNoContent)
|
w.WriteHeader(http.StatusNoContent)
|
||||||
@@ -475,11 +441,11 @@ func (report *Report) checkRanges() error {
|
|||||||
|
|
||||||
// finishWith finishes a request with what its report says, and writes the
|
// finishWith finishes a request with what its report says, and writes the
|
||||||
// request's last log line, which holds its decision's fields and the report's.
|
// request's last log line, which holds its decision's fields and the report's.
|
||||||
// For an allowed request it first counts the request's bytes, and adds it to its
|
// For an allowed request it first runs the heuristics of onReport, and adds the
|
||||||
// client's history, the anomaly counters and the metrics, with the status and
|
// request to its client's history, the anomaly counters and the metrics, with
|
||||||
// bytes the report gives. A denied request's counting was finished at its
|
// the status and bytes the report gives. A denied request's counting was
|
||||||
// decision.
|
// finished at its decision. ctx is the report's own context.
|
||||||
func (h *handler) finishWith(held *decision, report Report) {
|
func (h *handler) finishWith(ctx context.Context, held *decision, report Report) {
|
||||||
rq := &request{
|
rq := &request{
|
||||||
decision: *held,
|
decision: *held,
|
||||||
h: h,
|
h: h,
|
||||||
@@ -496,7 +462,7 @@ func (h *handler) finishWith(held *decision, report Report) {
|
|||||||
if rq.allowed {
|
if rq.allowed {
|
||||||
rq.line.DurationUpstreamTotal = &report.DurationUpstreamTotal
|
rq.line.DurationUpstreamTotal = &report.DurationUpstreamTotal
|
||||||
|
|
||||||
rq.countBytes()
|
rq.runBanningHeuristics(ctx, onReport())
|
||||||
rq.countAnomalies()
|
rq.countAnomalies()
|
||||||
rq.addToHistory()
|
rq.addToHistory()
|
||||||
|
|
||||||
|
|||||||
@@ -104,6 +104,15 @@ func TestDenyNetsRefuseBeforeTheLookup(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestDenyNetsRefusalNamesItsHeuristicAndNetblock(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _, _ := startWithClock(t, "", map[string]string{denyNets: clientsNet})
|
||||||
|
|
||||||
|
line := s.get(client, http.StatusForbidden, requestlog.ActionDenied)
|
||||||
|
wantHeuristic(t, line, "deny_nets", `{"netblock": "`+clientsNet+`"}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestRequestRefusedByDenyNetsIsNotCounted(t *testing.T) {
|
func TestRequestRefusedByDenyNetsIsNotCounted(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,39 @@
|
|||||||
|
package servewaf
|
||||||
|
|
||||||
|
// The heuristic trap_paths. It reads SWWAF_TRAP_PATHS.
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"slices"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
|
)
|
||||||
|
|
||||||
|
// trapPaths refuses a request for a path in SWWAF_TRAP_PATHS: its path as a
|
||||||
|
// path rule sees it, without the query, as sent or once percent-decoded, is one
|
||||||
|
// of them. Such a request is a clear sign of attack, as a ban rule's match is:
|
||||||
|
// it bans the client's netblock, or in observe mode raises the alert for the
|
||||||
|
// ban it would have made. The ban's notes give the path as sent, and so does
|
||||||
|
// its figure.
|
||||||
|
type trapPaths struct{}
|
||||||
|
|
||||||
|
func (trapPaths) name() string { return "trap_paths" }
|
||||||
|
|
||||||
|
func (trapPaths) check(_ context.Context, rq *request, now time.Time) *verdict {
|
||||||
|
path := rules.Path(rq.in)
|
||||||
|
if !slices.Contains(rq.h.config.TrapPaths, path) &&
|
||||||
|
!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.attack = true
|
||||||
|
rq.banForAttack(now, bans.Notes{TrapPath: path})
|
||||||
|
|
||||||
|
return &verdict{
|
||||||
|
action: requestlog.ActionBanned,
|
||||||
|
figures: map[string]any{"path": path},
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -83,6 +83,16 @@ func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestTrapPathRefusalNamesItsHeuristicAndPath(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
s, _, _ := startWithClock(t, "", map[string]string{trapPaths: trapPathList})
|
||||||
|
|
||||||
|
line := s.request(client, "/wp-login.php?redirect_to=x", http.StatusForbidden,
|
||||||
|
requestlog.ActionBanned)
|
||||||
|
wantHeuristic(t, line, "trap_paths", `{"path": "/wp-login.php"}`)
|
||||||
|
}
|
||||||
|
|
||||||
func TestTrapPathMatchesThePathDecodedOnce(t *testing.T) {
|
func TestTrapPathMatchesThePathDecodedOnce(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
@@ -10,8 +10,6 @@ import (
|
|||||||
"net/url"
|
"net/url"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
||||||
)
|
)
|
||||||
@@ -33,7 +31,7 @@ var errHealthEndpoint = errors.New("the health endpoint answered")
|
|||||||
// args are the arguments after `healthcheck`; it takes none, and given
|
// args are the arguments after `healthcheck`; it takes none, and given
|
||||||
// one it names it on stderr and returns 1 without checking anything.
|
// one it names it on stderr and returns 1 without checking anything.
|
||||||
func HealthCheck(
|
func HealthCheck(
|
||||||
ctx context.Context, args []string, v *viper.Viper,
|
ctx context.Context, args []string, lookupEnv func(string) (string, bool),
|
||||||
stderr io.Writer,
|
stderr io.Writer,
|
||||||
) int {
|
) int {
|
||||||
if len(args) > 0 {
|
if len(args) > 0 {
|
||||||
@@ -46,15 +44,15 @@ func HealthCheck(
|
|||||||
ctx, cancel := context.WithTimeout(ctx, healthCheckTimeout)
|
ctx, cancel := context.WithTimeout(ctx, healthCheckTimeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
command, err := config.Command(v)
|
command, err := config.Command(lookupEnv)
|
||||||
|
|
||||||
switch {
|
switch {
|
||||||
case err != nil:
|
case err != nil:
|
||||||
err = fmt.Errorf("invalid setting: %w", err)
|
err = fmt.Errorf("invalid setting: %w", err)
|
||||||
case command == "servewaf":
|
case command == "servewaf":
|
||||||
err = checkServeWAF(ctx, v)
|
err = checkServeWAF(ctx, lookupEnv)
|
||||||
default:
|
default:
|
||||||
err = checkRProxyAndApp(ctx, v)
|
err = checkRProxyAndApp(ctx, lookupEnv)
|
||||||
}
|
}
|
||||||
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -67,8 +65,8 @@ func HealthCheck(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// checkServeWAF asks servewaf's health endpoint.
|
// checkServeWAF asks servewaf's health endpoint.
|
||||||
func checkServeWAF(ctx context.Context, v *viper.Viper) error {
|
func checkServeWAF(ctx context.Context, lookupEnv func(string) (string, bool)) error {
|
||||||
listenAddr, err := config.ServeWAFListenAddr(v)
|
listenAddr, err := config.ServeWAFListenAddr(lookupEnv)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("invalid setting: %w", err)
|
return fmt.Errorf("invalid setting: %w", err)
|
||||||
}
|
}
|
||||||
@@ -83,9 +81,9 @@ func checkServeWAF(ctx context.Context, v *viper.Viper) error {
|
|||||||
|
|
||||||
// checkRProxyAndApp asks rproxy's health endpoint, and connects to the app.
|
// checkRProxyAndApp asks rproxy's health endpoint, and connects to the app.
|
||||||
func checkRProxyAndApp(
|
func checkRProxyAndApp(
|
||||||
ctx context.Context, v *viper.Viper,
|
ctx context.Context, lookupEnv func(string) (string, bool),
|
||||||
) error {
|
) error {
|
||||||
listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL(v)
|
listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL(lookupEnv)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("invalid setting: %w", err)
|
return fmt.Errorf("invalid setting: %w", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -125,12 +125,11 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
|
|||||||
wantHealthCheck(t, settings(rproxyListenAddr, "8080"), 1,
|
wantHealthCheck(t, settings(rproxyListenAddr, "8080"), 1,
|
||||||
"unhealthy: invalid setting: SWRP_LISTEN_ADDR: ")
|
"unhealthy: invalid setting: SWRP_LISTEN_ADDR: ")
|
||||||
|
|
||||||
// A setting of rproxy's can be given as a file too, here the listen
|
// No setting of rproxy's has a form given as a file.
|
||||||
// address, which the health check reads.
|
|
||||||
wantHealthCheck(t, map[string]string{
|
wantHealthCheck(t, map[string]string{
|
||||||
rproxyListenAddr + "_FILE": writeFile(t, ":"+port+"\n"),
|
rproxyListenAddr + "_FILE": writeFile(t, ":"+port+"\n"),
|
||||||
rproxyUpstreamURL: app.URL,
|
rproxyUpstreamURL: app.URL,
|
||||||
}, 0, "")
|
}, 1, "unhealthy: ask rproxy: ")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestHealthCheckFailsWithAnyOtherCommand(t *testing.T) {
|
func TestHealthCheckFailsWithAnyOtherCommand(t *testing.T) {
|
||||||
@@ -146,7 +145,11 @@ func TestHealthCheckRefusesAnArgument(t *testing.T) {
|
|||||||
|
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
|
|
||||||
got := smallwebwaf.HealthCheck(t.Context(), []string{"now"}, viperFrom(nil), &stderr)
|
noSettings := func(string) (string, bool) {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
got := smallwebwaf.HealthCheck(t.Context(), []string{"now"}, noSettings, &stderr)
|
||||||
|
|
||||||
want := "smallwebwaf healthcheck: unexpected argument \"now\"\n"
|
want := "smallwebwaf healthcheck: unexpected argument \"now\"\n"
|
||||||
if got != 1 || stderr.String() != want {
|
if got != 1 || stderr.String() != want {
|
||||||
@@ -163,7 +166,11 @@ func wantHealthCheck(t *testing.T, env map[string]string, status int, message st
|
|||||||
|
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
|
|
||||||
got := smallwebwaf.HealthCheck(t.Context(), nil, viperFrom(env), &stderr)
|
got := smallwebwaf.HealthCheck(t.Context(), nil, func(name string) (string, bool) {
|
||||||
|
value, ok := env[name]
|
||||||
|
|
||||||
|
return value, ok
|
||||||
|
}, &stderr)
|
||||||
|
|
||||||
wrote := stderr.String()
|
wrote := stderr.String()
|
||||||
if got != status || !strings.HasPrefix(wrote, message) ||
|
if got != status || !strings.HasPrefix(wrote, message) ||
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ func RunRProxy(ctx context.Context, params Params) int {
|
|||||||
hostname, _ := os.Hostname() // "" when the host has no name to give
|
hostname, _ := os.Hostname() // "" when the host has no name to give
|
||||||
processLog := requestlog.NewProcessLogger(params.Stdout, hostname, slog.LevelInfo)
|
processLog := requestlog.NewProcessLogger(params.Stdout, hostname, slog.LevelInfo)
|
||||||
|
|
||||||
cfg, err := config.RProxyFromEnvironment(params.Viper)
|
cfg, err := config.RProxyFromEnvironment(params.LookupEnv)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("invalid setting", "error", err.Error())
|
processLog.Error("invalid setting", "error", err.Error())
|
||||||
|
|
||||||
|
|||||||
@@ -43,8 +43,12 @@ const (
|
|||||||
func runRProxy(ctx context.Context, env map[string]string, out *output) int {
|
func runRProxy(ctx context.Context, env map[string]string, out *output) int {
|
||||||
return smallwebwaf.RunRProxy(ctx, smallwebwaf.Params{
|
return smallwebwaf.RunRProxy(ctx, smallwebwaf.Params{
|
||||||
Version: testVersion,
|
Version: testVersion,
|
||||||
Viper: viperFrom(env),
|
LookupEnv: func(name string) (string, bool) {
|
||||||
Stdout: out,
|
value, ok := env[name]
|
||||||
|
|
||||||
|
return value, ok
|
||||||
|
},
|
||||||
|
Stdout: out,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -20,8 +20,6 @@ import (
|
|||||||
"syscall"
|
"syscall"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||||
@@ -31,7 +29,6 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/state"
|
"sneak.berlin/go/smallwebwaf/internal/state"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// shutdownTimeout is how long requests in progress may take to finish
|
// shutdownTimeout is how long requests in progress may take to finish
|
||||||
@@ -61,9 +58,8 @@ commands:
|
|||||||
type Params struct {
|
type Params struct {
|
||||||
// Version is the version of the binary, set when it is built.
|
// Version is the version of the binary, set when it is built.
|
||||||
Version string
|
Version string
|
||||||
// Viper reads the settings, normally the viper config.NewViper returns
|
// LookupEnv reads an environment variable, normally os.LookupEnv.
|
||||||
// for the process's environment.
|
LookupEnv func(string) (string, bool)
|
||||||
Viper *viper.Viper
|
|
||||||
// Stdout receives the request log and the process's own messages.
|
// Stdout receives the request log and the process's own messages.
|
||||||
Stdout io.Writer
|
Stdout io.Writer
|
||||||
}
|
}
|
||||||
@@ -82,7 +78,7 @@ func Main(version string) int {
|
|||||||
syscall.SIGTERM, os.Interrupt)
|
syscall.SIGTERM, os.Interrupt)
|
||||||
defer stop()
|
defer stop()
|
||||||
|
|
||||||
params := Params{Version: version, Viper: config.NewViper(), Stdout: os.Stdout}
|
params := Params{Version: version, LookupEnv: os.LookupEnv, Stdout: os.Stdout}
|
||||||
|
|
||||||
switch args[0] {
|
switch args[0] {
|
||||||
case "rproxy":
|
case "rproxy":
|
||||||
@@ -90,7 +86,7 @@ func Main(version string) int {
|
|||||||
case "servewaf":
|
case "servewaf":
|
||||||
return RunServeWAF(ctx, params)
|
return RunServeWAF(ctx, params)
|
||||||
case "healthcheck":
|
case "healthcheck":
|
||||||
return HealthCheck(ctx, args[1:], params.Viper, os.Stderr)
|
return HealthCheck(ctx, args[1:], os.LookupEnv, os.Stderr)
|
||||||
default:
|
default:
|
||||||
return printUsage(os.Stderr)
|
return printUsage(os.Stderr)
|
||||||
}
|
}
|
||||||
@@ -110,9 +106,9 @@ func RunServeWAF(ctx context.Context, params Params) int {
|
|||||||
// Until the settings are read, the one message is an invalid setting's
|
// Until the settings are read, the one message is an invalid setting's
|
||||||
// error, which every SWWAF_LOG_LEVEL lets through.
|
// error, which every SWWAF_LOG_LEVEL lets through.
|
||||||
processLog := requestlog.NewProcessLogger(params.Stdout,
|
processLog := requestlog.NewProcessLogger(params.Stdout,
|
||||||
config.InstanceName(params.Viper), slog.LevelError)
|
config.InstanceName(params.LookupEnv), slog.LevelError)
|
||||||
|
|
||||||
cfg, err := config.FromEnvironment(params.Viper)
|
cfg, err := config.FromEnvironment(params.LookupEnv)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("invalid setting", "error", err.Error())
|
processLog.Error("invalid setting", "error", err.Error())
|
||||||
|
|
||||||
@@ -154,14 +150,7 @@ func RunServeWAF(ctx context.Context, params Params) int {
|
|||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
|
|
||||||
crs, err := servewaf.NewCoreRuleSet(cfg)
|
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue)
|
||||||
if err != nil {
|
|
||||||
processLog.Error("cannot use the Core Rule Set", "error", err.Error())
|
|
||||||
|
|
||||||
return 1
|
|
||||||
}
|
|
||||||
|
|
||||||
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue, crs)
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("cannot use the lookup database", "error", err.Error())
|
processLog.Error("cannot use the lookup database", "error", err.Error())
|
||||||
|
|
||||||
@@ -181,7 +170,8 @@ func RunServeWAF(ctx context.Context, params Params) int {
|
|||||||
|
|
||||||
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
|
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("cannot listen on SWWAF_LISTEN_ADDR", "error", err.Error())
|
processLog.Error("cannot listen on SWWAF_LISTEN_ADDR",
|
||||||
|
"error", err.Error())
|
||||||
|
|
||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
@@ -200,7 +190,6 @@ func RunServeWAF(ctx context.Context, params Params) int {
|
|||||||
func newServer(
|
func newServer(
|
||||||
cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
|
cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
|
||||||
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
|
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
|
||||||
crs *waf.CoreRuleSet,
|
|
||||||
) (*servewaf.Server, error) {
|
) (*servewaf.Server, error) {
|
||||||
var lookupFile *lookup.File
|
var lookupFile *lookup.File
|
||||||
|
|
||||||
@@ -225,7 +214,6 @@ func newServer(
|
|||||||
GeoJSURL: lookup.URL,
|
GeoJSURL: lookup.URL,
|
||||||
AbuseIPDBURL: reputation.AbuseIPDBURL,
|
AbuseIPDBURL: reputation.AbuseIPDBURL,
|
||||||
LookupFile: lookupFile,
|
LookupFile: lookupFile,
|
||||||
CoreRuleSet: crs,
|
|
||||||
Now: now,
|
Now: now,
|
||||||
Rules: ruleFiles,
|
Rules: ruleFiles,
|
||||||
Alerts: alertQueue,
|
Alerts: alertQueue,
|
||||||
|
|||||||
@@ -19,9 +19,6 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
|
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
|
||||||
@@ -52,7 +49,7 @@ const (
|
|||||||
adminSecret = "fedcba9876543210fedcba9876543210"
|
adminSecret = "fedcba9876543210fedcba9876543210"
|
||||||
// instance is the SWWAF_INSTANCE_NAME the tests set where they look at
|
// instance is the SWWAF_INSTANCE_NAME the tests set where they look at
|
||||||
// it.
|
// it.
|
||||||
instance = "fsn1app1/blog"
|
instance = "fsn1app1/gitea"
|
||||||
// greeting is what the tests' app answers.
|
// greeting is what the tests' app answers.
|
||||||
greeting = "hello from the app"
|
greeting = "hello from the app"
|
||||||
// placed is the client the tests' lookup databases place.
|
// placed is the client the tests' lookup databases place.
|
||||||
@@ -125,30 +122,21 @@ func (o *output) text() string {
|
|||||||
return o.buf.String()
|
return o.buf.String()
|
||||||
}
|
}
|
||||||
|
|
||||||
// viperFrom returns a viper Set with each of settings, as the viper
|
|
||||||
// config.NewViper returns one bound to a process's environment.
|
|
||||||
func viperFrom(settings map[string]string) *viper.Viper {
|
|
||||||
v := viper.New()
|
|
||||||
for name, value := range settings {
|
|
||||||
v.Set(name, value)
|
|
||||||
}
|
|
||||||
|
|
||||||
return v
|
|
||||||
}
|
|
||||||
|
|
||||||
// run runs `smallwebwaf servewaf` with the settings in env until ctx is done,
|
// run runs `smallwebwaf servewaf` with the settings in env until ctx is done,
|
||||||
// and returns its exit status. SWWAF_LOOKUP_SOURCE is off unless env sets it,
|
// and returns its exit status. SWWAF_LOOKUP_SOURCE is off unless env sets it,
|
||||||
// so that no test sends GeoJS its clients' addresses.
|
// so that no test sends GeoJS its clients' addresses.
|
||||||
func run(ctx context.Context, env map[string]string, out *output) int {
|
func run(ctx context.Context, env map[string]string, out *output) int {
|
||||||
v := viperFrom(env)
|
|
||||||
if _, set := env["SWWAF_LOOKUP_SOURCE"]; !set {
|
|
||||||
v.Set("SWWAF_LOOKUP_SOURCE", "off")
|
|
||||||
}
|
|
||||||
|
|
||||||
return smallwebwaf.RunServeWAF(ctx, smallwebwaf.Params{
|
return smallwebwaf.RunServeWAF(ctx, smallwebwaf.Params{
|
||||||
Version: testVersion,
|
Version: testVersion,
|
||||||
Viper: v,
|
LookupEnv: func(name string) (string, bool) {
|
||||||
Stdout: out,
|
value, ok := env[name]
|
||||||
|
if !ok && name == "SWWAF_LOOKUP_SOURCE" {
|
||||||
|
return "off", true
|
||||||
|
}
|
||||||
|
|
||||||
|
return value, ok
|
||||||
|
},
|
||||||
|
Stdout: out,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -353,8 +341,8 @@ func TestEveryLogLineAndMetricCarriesTheInstanceName(t *testing.T) {
|
|||||||
// the comments.
|
// the comments.
|
||||||
for line := range strings.Lines(metrics) {
|
for line := range strings.Lines(metrics) {
|
||||||
if !strings.HasPrefix(line, "#") &&
|
if !strings.HasPrefix(line, "#") &&
|
||||||
!strings.Contains(line, `instance="fsn1app1/blog"`) {
|
!strings.Contains(line, `instance="fsn1app1/gitea"`) {
|
||||||
t.Errorf("series %q, without instance=\"fsn1app1/blog\"", line)
|
t.Errorf("series %q, without instance=\"fsn1app1/gitea\"", line)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -563,56 +551,6 @@ func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
|
|||||||
": no such file or directory")
|
": no such file or directory")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
path := filepath.Join(t.TempDir(), "crs-exclusions.conf")
|
|
||||||
out := &output{}
|
|
||||||
|
|
||||||
status := run(t.Context(), map[string]string{
|
|
||||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
|
||||||
"SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
|
||||||
}, out)
|
|
||||||
if status != 1 {
|
|
||||||
t.Fatalf("exit status %d, want 1", status)
|
|
||||||
}
|
|
||||||
|
|
||||||
line := out.line(t, "msg", "cannot use the Core Rule Set")
|
|
||||||
want := "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: read the exclusions file: open " +
|
|
||||||
path + ": no such file or directory"
|
|
||||||
|
|
||||||
if line["error"] != want {
|
|
||||||
t.Errorf("start refused with %v, want the error %q", line, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCoreRuleSetExclusionsFileThatCannotBeUsedStopsTheStartWhileTheCoreRuleSetIsOff(
|
|
||||||
t *testing.T,
|
|
||||||
) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
broken := filepath.Join(t.TempDir(), "broken.conf")
|
|
||||||
|
|
||||||
err := os.WriteFile(broken, []byte("SecNoSuchDirective On\n"), 0o600)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, path := range []string{filepath.Join(t.TempDir(), "missing.conf"), broken} {
|
|
||||||
out := &output{}
|
|
||||||
|
|
||||||
status := run(t.Context(), map[string]string{
|
|
||||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
|
||||||
"SWWAF_WAF_MODE": config.WAFModeOff, "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
|
||||||
}, out)
|
|
||||||
if status != 1 {
|
|
||||||
t.Fatalf("%s: exit status %d, want 1", path, status)
|
|
||||||
}
|
|
||||||
|
|
||||||
out.line(t, "msg", "cannot use the Core Rule Set")
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
|
func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -672,9 +610,9 @@ func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
|
|||||||
rename(t, replacement, path)
|
rename(t, replacement, path)
|
||||||
|
|
||||||
metrics := metricsWith(t, url+"_smallwebwaf/metrics", token,
|
metrics := metricsWith(t, url+"_smallwebwaf/metrics", token,
|
||||||
`smallwebwaf_lookup_database_read_failures_total{instance="fsn1app1/blog"} 1`)
|
`smallwebwaf_lookup_database_read_failures_total{instance="fsn1app1/gitea"} 1`)
|
||||||
lastRead = seriesValue(t, metrics,
|
lastRead = seriesValue(t, metrics,
|
||||||
`smallwebwaf_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/blog"}`)
|
`smallwebwaf_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/gitea"}`)
|
||||||
|
|
||||||
wantStatus(t, url, placed, http.StatusOK)
|
wantStatus(t, url, placed, http.StatusOK)
|
||||||
})
|
})
|
||||||
@@ -730,7 +668,7 @@ func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T)
|
|||||||
// too.
|
// too.
|
||||||
rateLimitExemptNets: placed + "," + localhost,
|
rateLimitExemptNets: placed + "," + localhost,
|
||||||
}
|
}
|
||||||
failures := `smallwebwaf_reputation_failures_total{instance="fsn1app1/blog",` +
|
failures := `smallwebwaf_reputation_failures_total{instance="fsn1app1/gitea",` +
|
||||||
`source="` + torURL + `"} `
|
`source="` + torURL + `"} `
|
||||||
|
|
||||||
// tor.txt cannot be fetched at first, which is counted.
|
// tor.txt cannot be fetched at first, which is counted.
|
||||||
@@ -971,8 +909,8 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
|
|||||||
// last.
|
// last.
|
||||||
metrics := metricsText(t, url+"_smallwebwaf/metrics", token)
|
metrics := metricsText(t, url+"_smallwebwaf/metrics", token)
|
||||||
for _, series := range []string{
|
for _, series := range []string{
|
||||||
`smallwebwaf_remote_log_lines_sent_total{instance="fsn1app1/blog"} 0`,
|
`smallwebwaf_remote_log_lines_sent_total{instance="fsn1app1/gitea"} 0`,
|
||||||
`smallwebwaf_remote_log_buffer_depth{instance="fsn1app1/blog"} 1`,
|
`smallwebwaf_remote_log_buffer_depth{instance="fsn1app1/gitea"} 1`,
|
||||||
} {
|
} {
|
||||||
if !strings.Contains(metrics, "\n"+series+"\n") {
|
if !strings.Contains(metrics, "\n"+series+"\n") {
|
||||||
t.Errorf("no %q in the metrics:\n%s", series, metrics)
|
t.Errorf("no %q in the metrics:\n%s", series, metrics)
|
||||||
@@ -980,7 +918,7 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const dropped = "\nsmallwebwaf_remote_log_lines_dropped_total" +
|
const dropped = "\nsmallwebwaf_remote_log_lines_dropped_total" +
|
||||||
`{instance="fsn1app1/blog"} `
|
`{instance="fsn1app1/gitea"} `
|
||||||
if strings.Contains(metrics, dropped+"0\n") || !strings.Contains(metrics, dropped) {
|
if strings.Contains(metrics, dropped+"0\n") || !strings.Contains(metrics, dropped) {
|
||||||
t.Errorf("no line dropped in the metrics:\n%s", metrics)
|
t.Errorf("no line dropped in the metrics:\n%s", metrics)
|
||||||
}
|
}
|
||||||
@@ -1073,7 +1011,7 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
|
|||||||
runUntilStopped(t, env, func(url string) {
|
runUntilStopped(t, env, func(url string) {
|
||||||
webhook.waitFor(t, "permanent_ban", true)
|
webhook.waitFor(t, "permanent_ban", true)
|
||||||
|
|
||||||
const ofWebhook = `{destination="webhook",instance="fsn1app1/blog"}`
|
const ofWebhook = `{destination="webhook",instance="fsn1app1/gitea"}`
|
||||||
|
|
||||||
metrics := metricsWith(t, url+"_smallwebwaf/metrics", token,
|
metrics := metricsWith(t, url+"_smallwebwaf/metrics", token,
|
||||||
"\nsmallwebwaf_alerts_sent_total"+ofWebhook+" 1\n")
|
"\nsmallwebwaf_alerts_sent_total"+ofWebhook+" 1\n")
|
||||||
@@ -1124,13 +1062,13 @@ func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
|
|||||||
slackPost := slack.firstPost(t)
|
slackPost := slack.firstPost(t)
|
||||||
err := json.Unmarshal([]byte(slackPost.body), &message)
|
err := json.Unmarshal([]byte(slackPost.body), &message)
|
||||||
|
|
||||||
if err != nil || !strings.HasPrefix(message.Text, "*fsn1app1/blog: ban*\n") ||
|
if err != nil || !strings.HasPrefix(message.Text, "*fsn1app1/gitea: ban*\n") ||
|
||||||
!strings.Contains(message.Text, "\nclient: "+client+"\n") {
|
!strings.Contains(message.Text, "\nclient: "+client+"\n") {
|
||||||
t.Errorf("Slack was sent %s", slackPost.body)
|
t.Errorf("Slack was sent %s", slackPost.body)
|
||||||
}
|
}
|
||||||
|
|
||||||
ntfyPost := ntfy.firstPost(t)
|
ntfyPost := ntfy.firstPost(t)
|
||||||
if ntfyPost.header.Get("Title") != "fsn1app1/blog: ban" ||
|
if ntfyPost.header.Get("Title") != "fsn1app1/gitea: ban" ||
|
||||||
ntfyPost.header.Get("Authorization") != "Bearer "+ntfyToken ||
|
ntfyPost.header.Get("Authorization") != "Bearer "+ntfyToken ||
|
||||||
!strings.Contains(ntfyPost.body, "\nclient: "+client+"\n") {
|
!strings.Contains(ntfyPost.body, "\nclient: "+client+"\n") {
|
||||||
t.Errorf("ntfy was sent %s, with the headers %v", ntfyPost.body,
|
t.Errorf("ntfy was sent %s, with the headers %v", ntfyPost.body,
|
||||||
@@ -1142,9 +1080,9 @@ func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
|
|||||||
// test process cannot fail the test.
|
// test process cannot fail the test.
|
||||||
sent := []string{
|
sent := []string{
|
||||||
"\nsmallwebwaf_alerts_sent_total{destination=\"slack\"," +
|
"\nsmallwebwaf_alerts_sent_total{destination=\"slack\"," +
|
||||||
"instance=\"fsn1app1/blog\"} 1\n",
|
"instance=\"fsn1app1/gitea\"} 1\n",
|
||||||
"\nsmallwebwaf_alerts_sent_total{destination=\"ntfy\"," +
|
"\nsmallwebwaf_alerts_sent_total{destination=\"ntfy\"," +
|
||||||
"instance=\"fsn1app1/blog\"} 1\n",
|
"instance=\"fsn1app1/gitea\"} 1\n",
|
||||||
}
|
}
|
||||||
|
|
||||||
metrics := metricsWith(t, url+"_smallwebwaf/metrics", token, sent...)
|
metrics := metricsWith(t, url+"_smallwebwaf/metrics", token, sent...)
|
||||||
|
|||||||
@@ -143,7 +143,7 @@ const filledAlertsJSON = `{
|
|||||||
"waiting": {
|
"waiting": {
|
||||||
"webhook": [
|
"webhook": [
|
||||||
{
|
{
|
||||||
"instance": "fsn1app1/blog",
|
"instance": "fsn1app1/gitea",
|
||||||
"time": "2026-10-06T00:00:00Z",
|
"time": "2026-10-06T00:00:00Z",
|
||||||
"event": "ban",
|
"event": "ban",
|
||||||
"client": "203.0.113.9",
|
"client": "203.0.113.9",
|
||||||
@@ -158,7 +158,7 @@ const filledAlertsJSON = `{
|
|||||||
"suppressed_repeats": 0
|
"suppressed_repeats": 0
|
||||||
},
|
},
|
||||||
{
|
{
|
||||||
"instance": "fsn1app1/blog",
|
"instance": "fsn1app1/gitea",
|
||||||
"time": "2026-10-06T00:00:00Z",
|
"time": "2026-10-06T00:00:00Z",
|
||||||
"event": "file_error",
|
"event": "file_error",
|
||||||
"client": "",
|
"client": "",
|
||||||
@@ -1721,7 +1721,7 @@ func newParams(dir string) state.Params {
|
|||||||
Events: alerts.Events(),
|
Events: alerts.Events(),
|
||||||
Cooldown: 15 * time.Minute,
|
Cooldown: 15 * time.Minute,
|
||||||
MaxPerHour: 2,
|
MaxPerHour: 2,
|
||||||
Instance: "fsn1app1/blog",
|
Instance: "fsn1app1/gitea",
|
||||||
Now: midnight,
|
Now: midnight,
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|||||||
@@ -1,283 +0,0 @@
|
|||||||
# The eighth change: on gitea's paths, some fields skip the rules that refused
|
|
||||||
# ordinary gitea requests there. They are the fields where gitea sends a search,
|
|
||||||
# a file name or a branch name. Each exclusion runs with the request, before the
|
|
||||||
# rules it switches off, those of phase 1 included, so this file is loaded before
|
|
||||||
# the Core Rule Set's rules. Each comment names a request the rule refused.
|
|
||||||
|
|
||||||
# gitea's searches of code, of issues and pull requests, and of commits send the
|
|
||||||
# text searched for in q, which skips these rules there:
|
|
||||||
# 930110: ../
|
|
||||||
# 930120: .gitignore
|
|
||||||
# 931100: http://127.0.0.1:8080
|
|
||||||
# 932120: Invoke-WebRequest
|
|
||||||
# 932130: $(pwd)
|
|
||||||
# 932140: if (a == b) {
|
|
||||||
# 932160: /bin/sh
|
|
||||||
# 932235: `make check`
|
|
||||||
# 932250: ls -la
|
|
||||||
# 932260: python3
|
|
||||||
# 932380: `whoami`
|
|
||||||
# 933100: <?php echo
|
|
||||||
# 933130: $_GET[
|
|
||||||
# 933135: ${VAR}
|
|
||||||
# 934100: process.env
|
|
||||||
# 934110: http://127.0.0.1:8080
|
|
||||||
# 934130: __proto__
|
|
||||||
# 941100: <script>
|
|
||||||
# 941110: <script>
|
|
||||||
# 941160: <img src=
|
|
||||||
# 941180: document.cookie
|
|
||||||
# 941390: fetch(
|
|
||||||
# 942100: SELECT * FROM users WHERE
|
|
||||||
# 942140: information_schema
|
|
||||||
# 942190: union select
|
|
||||||
# 942230: if (a == b) {
|
|
||||||
# 944100: java.lang.Runtime
|
|
||||||
# 944110: java.lang.Runtime
|
|
||||||
# 944130: java.lang.Runtime
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/(?:explore/code|issues(?:/search)?|pulls|\
|
|
||||||
[^/]+/-/code|\
|
|
||||||
[^/]+/[^/]+/(?:search|issues(?:/search)?|pulls|\
|
|
||||||
commits/(?:branch|tag|commit)/.+/search)|\
|
|
||||||
api/v1/repos/(?:issues/search|[^/]+/[^/]+/issues))$" \
|
|
||||||
"id:900600,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=930110;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=930120;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=931100;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932120;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932130;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932140;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932160;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932235;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932250;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932260;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932380;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=933100;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=933130;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=933135;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=934100;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=934110;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=934130;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=941100;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=941110;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=941160;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=941180;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=941390;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=942100;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=942140;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=942190;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=942230;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=944100;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=944110;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;ARGS:q"
|
|
||||||
|
|
||||||
# A page opened from the results of one of those searches has the search's
|
|
||||||
# address in Referer, which skips these rules then:
|
|
||||||
# 941110: <script>
|
|
||||||
# 941120: onClick={
|
|
||||||
# 941160: <img src=
|
|
||||||
# 942100: LIKE '%foo%'
|
|
||||||
# 944100: java.lang.Runtime
|
|
||||||
SecRule REQUEST_HEADERS:Referer "@rx ^https?://[^/]+/(?:explore/code|\
|
|
||||||
issues(?:/search)?|pulls|[^/]+/-/code|\
|
|
||||||
[^/]+/[^/]+/(?:search|issues(?:/search)?|pulls|\
|
|
||||||
commits/(?:branch|tag|commit)/.+/search)|\
|
|
||||||
api/v1/repos/(?:issues/search|[^/]+/[^/]+/issues))\?" \
|
|
||||||
"id:900601,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=941110;REQUEST_HEADERS:Referer,\
|
|
||||||
ctl:ruleRemoveTargetById=941120;REQUEST_HEADERS:Referer,\
|
|
||||||
ctl:ruleRemoveTargetById=941160;REQUEST_HEADERS:Referer,\
|
|
||||||
ctl:ruleRemoveTargetById=942100;REQUEST_HEADERS:Referer,\
|
|
||||||
ctl:ruleRemoveTargetById=944100;REQUEST_HEADERS:Referer"
|
|
||||||
|
|
||||||
# gitea's searches of repositories, users, branches, tags and packages send a
|
|
||||||
# name in q, which skips these rules there:
|
|
||||||
# 930120: .profile
|
|
||||||
# 932120: powershell-scripts
|
|
||||||
# 932260: python3-app
|
|
||||||
# 944130: org.apache.commons.logging
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/(?:explore/repos|user/search_candidates|\
|
|
||||||
[^/]+/-/packages|[^/]+/[^/]+/(?:branches|tags)|api/v1/repos/search)$" \
|
|
||||||
"id:900602,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=930120;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932120;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932260;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;ARGS:q"
|
|
||||||
|
|
||||||
# gitea's other searches for a name send it in q too, which skips only these
|
|
||||||
# rules there:
|
|
||||||
# 930120: .profile
|
|
||||||
# 932120: powershell-scripts
|
|
||||||
# 932260: python3-app
|
|
||||||
# The searches:
|
|
||||||
# - the dashboard's filter of repositories;
|
|
||||||
# - the repositories on a user's or an organisation's page, whose path has
|
|
||||||
# one segment;
|
|
||||||
# - the author filter of the issue and pull request lists;
|
|
||||||
# - the lists of users and organisations, and the API's search for users.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/(?:[^/]+|repo/search|\
|
|
||||||
explore/(?:users|organizations)|api/v1/users/search|\
|
|
||||||
[^/]+/[^/]+/(?:issues|pulls)/posters)$" \
|
|
||||||
"id:900603,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=930120;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932120;ARGS:q,\
|
|
||||||
ctl:ruleRemoveTargetById=932260;ARGS:q"
|
|
||||||
|
|
||||||
# These requests name a file in the path, which skips these rules there:
|
|
||||||
# 920500: notes.txt~
|
|
||||||
# 934100: process.env.d.ts
|
|
||||||
# 941130: docs/a b.xhtml
|
|
||||||
# 944130: java.lang.String.md
|
|
||||||
# The requests:
|
|
||||||
# - a file's page, raw file and download;
|
|
||||||
# - its blame, history, edit, preview and delete pages;
|
|
||||||
# - the API's call for its contents;
|
|
||||||
# - a release attachment's download.
|
|
||||||
SecRule REQUEST_FILENAME \
|
|
||||||
"@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_preview|_delete|contents|releases/download)/" \
|
|
||||||
"id:900610,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=920500;REQUEST_FILENAME,\
|
|
||||||
ctl:ruleRemoveTargetById=934100;REQUEST_FILENAME,\
|
|
||||||
ctl:ruleRemoveTargetById=941130;REQUEST_FILENAME,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;REQUEST_FILENAME"
|
|
||||||
|
|
||||||
# The requests below name a file in a query parameter. These rules refused
|
|
||||||
# them for the file named:
|
|
||||||
# 930120: .gitignore
|
|
||||||
# 932120: install-script.sh
|
|
||||||
# 932130: [draft].md
|
|
||||||
# 932160: bin/docker-entrypoint
|
|
||||||
# 932260: docker-compose.yml
|
|
||||||
# 934100: process.env.d.ts
|
|
||||||
# 934130: __proto__.json
|
|
||||||
# 941130: index.xhtml
|
|
||||||
# 941180: document.write.js
|
|
||||||
# 942140: information_schema.sql
|
|
||||||
# 944110: java/lang/Runtime.java
|
|
||||||
# 944130: java.lang.String.md
|
|
||||||
# Each request skips the rules that refused it. The fourth change already has
|
|
||||||
# path and redirect_to skip 930120, 932160 and 932260 on every path.
|
|
||||||
|
|
||||||
# A file's commit graph, in file.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/graph$" \
|
|
||||||
"id:900620,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=930120;ARGS:file,\
|
|
||||||
ctl:ruleRemoveTargetById=932120;ARGS:file,\
|
|
||||||
ctl:ruleRemoveTargetById=932160;ARGS:file,\
|
|
||||||
ctl:ruleRemoveTargetById=932260;ARGS:file,\
|
|
||||||
ctl:ruleRemoveTargetById=934100;ARGS:file,\
|
|
||||||
ctl:ruleRemoveTargetById=934130;ARGS:file,\
|
|
||||||
ctl:ruleRemoveTargetById=941130;ARGS:file,\
|
|
||||||
ctl:ruleRemoveTargetById=941180;ARGS:file,\
|
|
||||||
ctl:ruleRemoveTargetById=942140;ARGS:file,\
|
|
||||||
ctl:ruleRemoveTargetById=944110;ARGS:file,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;ARGS:file"
|
|
||||||
|
|
||||||
# The API's list of a file's commits, in path.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/commits$" \
|
|
||||||
"id:900621,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=932120;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=934100;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=934130;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=941130;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=941180;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=942140;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=944110;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;ARGS:path"
|
|
||||||
|
|
||||||
# The API's call for the contents of several files, in body, as JSON.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/file-contents$" \
|
|
||||||
"id:900622,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=930120;ARGS:body,\
|
|
||||||
ctl:ruleRemoveTargetById=932120;ARGS:body,\
|
|
||||||
ctl:ruleRemoveTargetById=932160;ARGS:body,\
|
|
||||||
ctl:ruleRemoveTargetById=934100;ARGS:body,\
|
|
||||||
ctl:ruleRemoveTargetById=934130;ARGS:body,\
|
|
||||||
ctl:ruleRemoveTargetById=941130;ARGS:body,\
|
|
||||||
ctl:ruleRemoveTargetById=941180;ARGS:body,\
|
|
||||||
ctl:ruleRemoveTargetById=942140;ARGS:body,\
|
|
||||||
ctl:ruleRemoveTargetById=944110;ARGS:body,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;ARGS:body"
|
|
||||||
|
|
||||||
# An issue opened from a template, in template.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/issues/new$" \
|
|
||||||
"id:900623,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=930120;ARGS:template,\
|
|
||||||
ctl:ruleRemoveTargetById=932120;ARGS:template,\
|
|
||||||
ctl:ruleRemoveTargetById=932130;ARGS:template,\
|
|
||||||
ctl:ruleRemoveTargetById=932160;ARGS:template,\
|
|
||||||
ctl:ruleRemoveTargetById=932260;ARGS:template,\
|
|
||||||
ctl:ruleRemoveTargetById=934100;ARGS:template,\
|
|
||||||
ctl:ruleRemoveTargetById=934130;ARGS:template,\
|
|
||||||
ctl:ruleRemoveTargetById=941130;ARGS:template,\
|
|
||||||
ctl:ruleRemoveTargetById=941180;ARGS:template,\
|
|
||||||
ctl:ruleRemoveTargetById=942140;ARGS:template,\
|
|
||||||
ctl:ruleRemoveTargetById=944110;ARGS:template,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;ARGS:template"
|
|
||||||
|
|
||||||
# The sign-in page on the way to a file's edit page, in redirect_to.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/user/login$" \
|
|
||||||
"id:900624,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=932120;ARGS:redirect_to,\
|
|
||||||
ctl:ruleRemoveTargetById=932130;ARGS:redirect_to,\
|
|
||||||
ctl:ruleRemoveTargetById=934100;ARGS:redirect_to,\
|
|
||||||
ctl:ruleRemoveTargetById=934130;ARGS:redirect_to,\
|
|
||||||
ctl:ruleRemoveTargetById=941130;ARGS:redirect_to,\
|
|
||||||
ctl:ruleRemoveTargetById=941180;ARGS:redirect_to,\
|
|
||||||
ctl:ruleRemoveTargetById=942140;ARGS:redirect_to,\
|
|
||||||
ctl:ruleRemoveTargetById=944110;ARGS:redirect_to,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;ARGS:redirect_to"
|
|
||||||
|
|
||||||
# An attachment uploaded through the API to a release, an issue or a comment,
|
|
||||||
# in name.
|
|
||||||
SecRule REQUEST_FILENAME \
|
|
||||||
"@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
|
|
||||||
"id:900625,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=930120;ARGS:name,\
|
|
||||||
ctl:ruleRemoveTargetById=932120;ARGS:name,\
|
|
||||||
ctl:ruleRemoveTargetById=932260;ARGS:name,\
|
|
||||||
ctl:ruleRemoveTargetById=934100;ARGS:name,\
|
|
||||||
ctl:ruleRemoveTargetById=934130;ARGS:name,\
|
|
||||||
ctl:ruleRemoveTargetById=941130;ARGS:name,\
|
|
||||||
ctl:ruleRemoveTargetById=941180;ARGS:name,\
|
|
||||||
ctl:ruleRemoveTargetById=942140;ARGS:name,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;ARGS:name"
|
|
||||||
|
|
||||||
# A diff's hidden lines, fetched when shown, in path.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/blob_excerpt/[0-9a-f]+$" \
|
|
||||||
"id:900626,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=932120;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=934100;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=934130;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=941130;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=941180;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=942140;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=944110;ARGS:path,\
|
|
||||||
ctl:ruleRemoveTargetById=944130;ARGS:path"
|
|
||||||
|
|
||||||
# The requests below name a branch or tag in a query parameter, which skips
|
|
||||||
# the list of command names (932260) there. It refused names such as
|
|
||||||
# docker-build, python3-migration and feat/docker-support.
|
|
||||||
|
|
||||||
# Deleting or restoring a branch, in name.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/branches/(?:delete|restore)$" \
|
|
||||||
"id:900630,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:name"
|
|
||||||
|
|
||||||
# A release started from a tag, in tag.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/releases/new$" \
|
|
||||||
"id:900631,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:tag"
|
|
||||||
|
|
||||||
# A branch protection rule opened for editing, in rule_name.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/settings/branches/edit$" \
|
|
||||||
"id:900632,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:rule_name"
|
|
||||||
|
|
||||||
# A directory listing's last commits, in refSubUrl.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/lastcommit/" \
|
|
||||||
"id:900633,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:refSubUrl"
|
|
||||||
|
|
||||||
# An artifact upload from actions/upload-artifact@v3 sends the header
|
|
||||||
# Content-Range, which the Core Rule Set refused (920450).
|
|
||||||
SecRule REQUEST_FILENAME \
|
|
||||||
"@rx ^/api/actions_pipeline/_apis/pipelines/workflows/[0-9]+/artifacts/[^/]+/upload$" \
|
|
||||||
"id:900640,phase:1,pass,nolog,\
|
|
||||||
ctl:ruleRemoveTargetById=920450;REQUEST_HEADERS_NAMES:Content-Range"
|
|
||||||
-393
@@ -1,393 +0,0 @@
|
|||||||
# Ordinary gitea 1.27 requests that the Core Rule Set must let through at the
|
|
||||||
# defaults. Each line is a path and query, and may end with one header after a
|
|
||||||
# space. The tests of internal/waf inspect each as a GET. The Core Rule Set reads
|
|
||||||
# the query and the headers the same way for any method.
|
|
||||||
|
|
||||||
# Searches with ordinary words and symbols.
|
|
||||||
/owner/repo/search?q=package.json
|
|
||||||
/owner/repo/search?q=.gitignore
|
|
||||||
/owner/repo/search?q=docker-compose.yml
|
|
||||||
/owner/repo/search?q=python3
|
|
||||||
/owner/repo/search?q=ssh%20key
|
|
||||||
/owner/repo/search?q=feat%2Fdocker-support
|
|
||||||
/owner/repo/search?q=ls%20-la
|
|
||||||
/owner/repo/search?q=sed%20-i
|
|
||||||
/owner/repo/search?q=%2Fbin%2Fsh
|
|
||||||
/owner/repo/search?q=%60make%20check%60
|
|
||||||
/owner/repo/search?q=%60whoami%60
|
|
||||||
/owner/repo/search?q=fetch%28
|
|
||||||
/owner/repo/search?q=%24%7BVAR%7D
|
|
||||||
/owner/repo/search?q=%24%28pwd%29
|
|
||||||
/owner/repo/search?q=process.env
|
|
||||||
/owner/repo/search?q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/search?q=%3Cscript%3E
|
|
||||||
/owner/repo/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/search?q=union%20select
|
|
||||||
/owner/repo/search?q=http%3A%2F%2F127.0.0.1%3A8080
|
|
||||||
/owner/repo/search?q=..%2F
|
|
||||||
/owner/repo/search?q=%2Fetc%2Fpasswd
|
|
||||||
/owner/repo/search?q=eval%28
|
|
||||||
/owner/repo/search?q=chmod%20%2Bx
|
|
||||||
/owner/repo/search?q=echo%20%24PATH
|
|
||||||
/owner/repo/search?q=java.lang.Runtime
|
|
||||||
/owner/repo/search?q=document.cookie
|
|
||||||
/owner/repo/search?q=alert%281%29
|
|
||||||
/owner/repo/search?q=__proto__
|
|
||||||
/owner/repo/search?q=C%3A%5CWindows%5CSystem32
|
|
||||||
/owner/repo/search?q=Invoke-WebRequest
|
|
||||||
/owner/repo/search?q=if%20%28a%20%3D%3D%20b%29%20%7B
|
|
||||||
/owner/repo/search?q=sudo%20apt%20install
|
|
||||||
/owner/repo/search?q=~%2F.ssh%2Fconfig
|
|
||||||
/owner/repo/search?q=.htaccess
|
|
||||||
/owner/repo/search?q=%24_GET%5B
|
|
||||||
/owner/repo/search?q=%24_POST
|
|
||||||
/owner/repo/search?q=%24_SERVER%5B%27REQUEST_URI%27%5D
|
|
||||||
/owner/repo/search?q=%3C%3Fphp%20echo
|
|
||||||
/owner/repo/search?q=information_schema
|
|
||||||
/owner/repo/search?q=LIKE%20%27%25foo%25%27
|
|
||||||
/owner/repo/search?q=onClick%3D%7B
|
|
||||||
/explore/code?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/explore/code?q=%3Cimg%20src%3D
|
|
||||||
/owner/-/code?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/-/code?q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/issues?type=all&state=open&q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/issues?type=all&state=open&q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/issues/search?q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/pulls?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/pulls?q=%3Cimg%20src%3D
|
|
||||||
/issues?type=your_repositories&q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/issues?type=your_repositories&q=%3Cimg%20src%3D
|
|
||||||
/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/issues/search?q=%3Cimg%20src%3D
|
|
||||||
/pulls?type=your_repositories&q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/pulls?type=your_repositories&q=%3Cimg%20src%3D
|
|
||||||
/owner/repo/commits/branch/main/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/owner/repo/commits/branch/main/search?q=%3Cimg%20src%3D
|
|
||||||
/api/v1/repos/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/api/v1/repos/issues/search?q=%3Cimg%20src%3D
|
|
||||||
/api/v1/repos/owner/repo/issues?q=SELECT%20%2A%20FROM%20users%20WHERE
|
|
||||||
/api/v1/repos/owner/repo/issues?q=%3Cimg%20src%3D
|
|
||||||
|
|
||||||
# Searches for the names of repositories, users, branches, tags and packages.
|
|
||||||
/explore/repos?q=python3-app
|
|
||||||
/explore/repos?q=.profile
|
|
||||||
/explore/repos?q=powershell-scripts
|
|
||||||
/api/v1/repos/search?q=docker-build
|
|
||||||
/user/search_candidates?q=docker-ci
|
|
||||||
/owner/repo/branches?q=feat%2Fdocker-support
|
|
||||||
/owner/repo/tags?q=docker-v1.0
|
|
||||||
/owner/-/packages?q=python3-requests
|
|
||||||
/owner/-/packages?q=org.apache.commons.logging
|
|
||||||
/repo/search?q=python3-app
|
|
||||||
/repo/search?q=docker-build
|
|
||||||
/repo/search?q=powershell-scripts
|
|
||||||
/repo/search?q=.profile
|
|
||||||
/owner?tab=repositories&q=python3-app
|
|
||||||
/org?q=python3-app
|
|
||||||
/owner/repo/issues/posters?q=docker-ci
|
|
||||||
/owner/repo/pulls/posters?q=docker-ci
|
|
||||||
/explore/users?q=docker-ci
|
|
||||||
/explore/organizations?q=docker-ci
|
|
||||||
/api/v1/users/search?q=docker-ci
|
|
||||||
|
|
||||||
# Pages, raw files and edit pages of files, and the API's call for their
|
|
||||||
# contents.
|
|
||||||
/owner/repo/src/branch/main/package.json
|
|
||||||
/owner/repo/raw/branch/main/package.json
|
|
||||||
/owner/repo/_edit/main/package.json
|
|
||||||
/api/v1/repos/owner/repo/contents/package.json
|
|
||||||
/owner/repo/src/branch/main/.gitignore
|
|
||||||
/owner/repo/raw/branch/main/.gitignore
|
|
||||||
/owner/repo/_edit/main/.gitignore
|
|
||||||
/api/v1/repos/owner/repo/contents/.gitignore
|
|
||||||
/owner/repo/src/branch/main/docker-compose.yml
|
|
||||||
/owner/repo/raw/branch/main/docker-compose.yml
|
|
||||||
/owner/repo/_edit/main/docker-compose.yml
|
|
||||||
/api/v1/repos/owner/repo/contents/docker-compose.yml
|
|
||||||
/owner/repo/src/branch/main/Dockerfile
|
|
||||||
/owner/repo/raw/branch/main/Dockerfile
|
|
||||||
/owner/repo/_edit/main/Dockerfile
|
|
||||||
/api/v1/repos/owner/repo/contents/Dockerfile
|
|
||||||
/owner/repo/src/branch/main/.htaccess
|
|
||||||
/owner/repo/raw/branch/main/.htaccess
|
|
||||||
/owner/repo/_edit/main/.htaccess
|
|
||||||
/api/v1/repos/owner/repo/contents/.htaccess
|
|
||||||
/owner/repo/src/branch/main/rootfs/etc/passwd
|
|
||||||
/owner/repo/raw/branch/main/rootfs/etc/passwd
|
|
||||||
/owner/repo/_edit/main/rootfs/etc/passwd
|
|
||||||
/api/v1/repos/owner/repo/contents/rootfs/etc/passwd
|
|
||||||
/owner/repo/src/branch/main/bin/docker-entrypoint
|
|
||||||
/owner/repo/raw/branch/main/bin/docker-entrypoint
|
|
||||||
/owner/repo/_edit/main/bin/docker-entrypoint
|
|
||||||
/api/v1/repos/owner/repo/contents/bin/docker-entrypoint
|
|
||||||
/owner/repo/src/branch/main/scripts/python3-setup.sh
|
|
||||||
/owner/repo/raw/branch/main/scripts/python3-setup.sh
|
|
||||||
/owner/repo/_edit/main/scripts/python3-setup.sh
|
|
||||||
/api/v1/repos/owner/repo/contents/scripts/python3-setup.sh
|
|
||||||
/owner/repo/src/branch/main/src/java.base/share/classes/java/lang/Runtime.java
|
|
||||||
/owner/repo/raw/branch/main/src/java.base/share/classes/java/lang/Runtime.java
|
|
||||||
/owner/repo/_edit/main/src/java.base/share/classes/java/lang/Runtime.java
|
|
||||||
/api/v1/repos/owner/repo/contents/src/java.base/share/classes/java/lang/Runtime.java
|
|
||||||
/owner/repo/src/branch/main/java.lang.String.md
|
|
||||||
/owner/repo/raw/branch/main/java.lang.String.md
|
|
||||||
/owner/repo/_edit/main/java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/contents/java.lang.String.md
|
|
||||||
/owner/repo/src/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
|
|
||||||
/owner/repo/raw/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
|
|
||||||
/owner/repo/_edit/main/META-INF/services/org.apache.commons.logging.LogFactory
|
|
||||||
/api/v1/repos/owner/repo/contents/META-INF/services/org.apache.commons.logging.LogFactory
|
|
||||||
/owner/repo/src/branch/main/process.env.d.ts
|
|
||||||
/owner/repo/raw/branch/main/process.env.d.ts
|
|
||||||
/owner/repo/_edit/main/process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/contents/process.env.d.ts
|
|
||||||
/owner/repo/src/branch/main/document.write.js
|
|
||||||
/owner/repo/raw/branch/main/document.write.js
|
|
||||||
/owner/repo/_edit/main/document.write.js
|
|
||||||
/api/v1/repos/owner/repo/contents/document.write.js
|
|
||||||
/owner/repo/src/branch/main/test/fixtures/__proto__.json
|
|
||||||
/owner/repo/raw/branch/main/test/fixtures/__proto__.json
|
|
||||||
/owner/repo/_edit/main/test/fixtures/__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/contents/test/fixtures/__proto__.json
|
|
||||||
/owner/repo/src/branch/main/install-script.sh
|
|
||||||
/owner/repo/raw/branch/main/install-script.sh
|
|
||||||
/owner/repo/_edit/main/install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/contents/install-script.sh
|
|
||||||
/owner/repo/src/branch/main/src/backend/catalog/information_schema.sql
|
|
||||||
/owner/repo/raw/branch/main/src/backend/catalog/information_schema.sql
|
|
||||||
/owner/repo/_edit/main/src/backend/catalog/information_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/contents/src/backend/catalog/information_schema.sql
|
|
||||||
/owner/repo/src/branch/main/docs/a%20b.xhtml
|
|
||||||
/owner/repo/raw/branch/main/docs/a%20b.xhtml
|
|
||||||
/owner/repo/_edit/main/docs/a%20b.xhtml
|
|
||||||
/api/v1/repos/owner/repo/contents/docs/a%20b.xhtml
|
|
||||||
/owner/repo/src/branch/main/%5Bdraft%5D.md
|
|
||||||
/owner/repo/raw/branch/main/%5Bdraft%5D.md
|
|
||||||
/owner/repo/_edit/main/%5Bdraft%5D.md
|
|
||||||
/api/v1/repos/owner/repo/contents/%5Bdraft%5D.md
|
|
||||||
/owner/repo/src/branch/main/notes.txt~
|
|
||||||
/owner/repo/raw/branch/main/notes.txt~
|
|
||||||
/owner/repo/_edit/main/notes.txt~
|
|
||||||
/api/v1/repos/owner/repo/contents/notes.txt~
|
|
||||||
|
|
||||||
# The download, blame, history, delete and preview pages of the files above
|
|
||||||
# whose names the Core Rule Set checks in the path, and their downloads as
|
|
||||||
# release attachments.
|
|
||||||
/owner/repo/media/branch/main/notes.txt~
|
|
||||||
/owner/repo/blame/branch/main/notes.txt~
|
|
||||||
/owner/repo/commits/branch/main/notes.txt~
|
|
||||||
/owner/repo/_delete/main/notes.txt~
|
|
||||||
/owner/repo/media/branch/main/process.env.d.ts
|
|
||||||
/owner/repo/blame/branch/main/process.env.d.ts
|
|
||||||
/owner/repo/commits/branch/main/process.env.d.ts
|
|
||||||
/owner/repo/_delete/main/process.env.d.ts
|
|
||||||
/owner/repo/media/branch/main/docs/a%20b.xhtml
|
|
||||||
/owner/repo/blame/branch/main/docs/a%20b.xhtml
|
|
||||||
/owner/repo/commits/branch/main/docs/a%20b.xhtml
|
|
||||||
/owner/repo/_delete/main/docs/a%20b.xhtml
|
|
||||||
/owner/repo/media/branch/main/java.lang.String.md
|
|
||||||
/owner/repo/blame/branch/main/java.lang.String.md
|
|
||||||
/owner/repo/commits/branch/main/java.lang.String.md
|
|
||||||
/owner/repo/_delete/main/java.lang.String.md
|
|
||||||
/owner/repo/_preview/main/notes.txt~
|
|
||||||
/owner/repo/_preview/main/process.env.d.ts
|
|
||||||
/owner/repo/_preview/main/java.lang.String.md
|
|
||||||
/owner/repo/releases/download/v1.0/notes.txt~
|
|
||||||
/owner/repo/releases/download/v1.0/process.env.d.ts
|
|
||||||
/owner/repo/releases/download/v1.0/java.lang.String.md
|
|
||||||
/owner/repo/releases/download/v1.0/a%20b.xhtml
|
|
||||||
|
|
||||||
# File names in the query string: the commit graph of a file, the API's lists
|
|
||||||
# of its commits and of its contents, an issue opened from a template, the
|
|
||||||
# sign-in page on the way to its edit page, and a diff's hidden lines.
|
|
||||||
/owner/repo/graph?file=package.json
|
|
||||||
/api/v1/repos/owner/repo/commits?path=package.json
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22package.json%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fpackage.json
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fpackage.json
|
|
||||||
/owner/repo/graph?file=.gitignore
|
|
||||||
/api/v1/repos/owner/repo/commits?path=.gitignore
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.gitignore%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.gitignore
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.gitignore
|
|
||||||
/owner/repo/graph?file=docker-compose.yml
|
|
||||||
/api/v1/repos/owner/repo/commits?path=docker-compose.yml
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docker-compose.yml%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocker-compose.yml
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocker-compose.yml
|
|
||||||
/owner/repo/graph?file=.htaccess
|
|
||||||
/api/v1/repos/owner/repo/commits?path=.htaccess
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.htaccess%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.htaccess
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.htaccess
|
|
||||||
/owner/repo/graph?file=rootfs%2Fetc%2Fpasswd
|
|
||||||
/api/v1/repos/owner/repo/commits?path=rootfs%2Fetc%2Fpasswd
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22rootfs%2Fetc%2Fpasswd%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Frootfs%2Fetc%2Fpasswd
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Frootfs%2Fetc%2Fpasswd
|
|
||||||
/owner/repo/graph?file=bin%2Fdocker-entrypoint
|
|
||||||
/api/v1/repos/owner/repo/commits?path=bin%2Fdocker-entrypoint
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22bin%2Fdocker-entrypoint%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fbin%2Fdocker-entrypoint
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fbin%2Fdocker-entrypoint
|
|
||||||
/owner/repo/graph?file=scripts%2Fpython3-setup.sh
|
|
||||||
/api/v1/repos/owner/repo/commits?path=scripts%2Fpython3-setup.sh
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22scripts%2Fpython3-setup.sh%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fscripts%2Fpython3-setup.sh
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fscripts%2Fpython3-setup.sh
|
|
||||||
/owner/repo/graph?file=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
|
||||||
/api/v1/repos/owner/repo/commits?path=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
|
||||||
/owner/repo/graph?file=java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/commits?path=java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22java.lang.String.md%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fjava.lang.String.md
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fjava.lang.String.md
|
|
||||||
/owner/repo/graph?file=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
|
||||||
/api/v1/repos/owner/repo/commits?path=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
|
||||||
/owner/repo/graph?file=process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/commits?path=process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22process.env.d.ts%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fprocess.env.d.ts
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fprocess.env.d.ts
|
|
||||||
/owner/repo/graph?file=document.write.js
|
|
||||||
/api/v1/repos/owner/repo/commits?path=document.write.js
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22document.write.js%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocument.write.js
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocument.write.js
|
|
||||||
/owner/repo/graph?file=test%2Ffixtures%2F__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/commits?path=test%2Ffixtures%2F__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22test%2Ffixtures%2F__proto__.json%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Ftest%2Ffixtures%2F__proto__.json
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Ftest%2Ffixtures%2F__proto__.json
|
|
||||||
/owner/repo/graph?file=install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/commits?path=install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22install-script.sh%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Finstall-script.sh
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Finstall-script.sh
|
|
||||||
/owner/repo/graph?file=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/commits?path=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fbackend%2Fcatalog%2Finformation_schema.sql%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
|
||||||
/owner/repo/graph?file=docs%2Fa%20b.xhtml
|
|
||||||
/api/v1/repos/owner/repo/commits?path=docs%2Fa%20b.xhtml
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docs%2Fa%20b.xhtml%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocs%2Fa%20b.xhtml
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocs%2Fa%20b.xhtml
|
|
||||||
/owner/repo/graph?file=%5Bdraft%5D.md
|
|
||||||
/api/v1/repos/owner/repo/commits?path=%5Bdraft%5D.md
|
|
||||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22%5Bdraft%5D.md%22%5D%7D
|
|
||||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F%5Bdraft%5D.md
|
|
||||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F%5Bdraft%5D.md
|
|
||||||
/owner/repo/blob_excerpt/0123456789abcdef0123456789abcdef01234567?last_left=10&last_right=10&left=20&right=20&left_hunk_size=5&right_hunk_size=5&path=process.env.d.ts&style=unified&direction=down&wiki=false
|
|
||||||
/owner/repo/blob_excerpt/0123456789abcdef0123456789abcdef01234567?last_left=10&last_right=10&left=20&right=20&left_hunk_size=5&right_hunk_size=5&path=java.lang.String.md&style=unified&direction=down&wiki=false
|
|
||||||
/owner/repo/blob_excerpt/0123456789abcdef0123456789abcdef01234567?last_left=10&last_right=10&left=20&right=20&left_hunk_size=5&right_hunk_size=5&path=install-script.sh&style=unified&direction=down&wiki=false
|
|
||||||
/owner/repo/blob_excerpt/0123456789abcdef0123456789abcdef01234567?last_left=10&last_right=10&left=20&right=20&left_hunk_size=5&right_hunk_size=5&path=information_schema.sql&style=unified&direction=down&wiki=false
|
|
||||||
/owner/repo/blob_excerpt/0123456789abcdef0123456789abcdef01234567?last_left=10&last_right=10&left=20&right=20&left_hunk_size=5&right_hunk_size=5&path=document.write.js&style=unified&direction=down&wiki=false
|
|
||||||
/owner/repo/blob_excerpt/0123456789abcdef0123456789abcdef01234567?last_left=10&last_right=10&left=20&right=20&left_hunk_size=5&right_hunk_size=5&path=__proto__.json&style=unified&direction=down&wiki=false
|
|
||||||
/owner/repo/blob_excerpt/0123456789abcdef0123456789abcdef01234567?last_left=10&last_right=10&left=20&right=20&left_hunk_size=5&right_hunk_size=5&path=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java&style=unified&direction=down&wiki=false
|
|
||||||
/owner/repo/blob_excerpt/0123456789abcdef0123456789abcdef01234567?last_left=10&last_right=10&left=20&right=20&left_hunk_size=5&right_hunk_size=5&path=index.xhtml&style=unified&direction=down&wiki=false
|
|
||||||
|
|
||||||
# File names of attachments uploaded through the API, to a release, an issue and
|
|
||||||
# a comment.
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=package.json
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=.gitignore
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=docker-compose.yml
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=.htaccess
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=docker-entrypoint
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=python3-setup.sh
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=Runtime.java
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=org.apache.commons.logging.LogFactory
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=%5Bdraft%5D.md
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=notes.txt~
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=install-script.sh
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=process.env.d.ts
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=java.lang.String.md
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=information_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=information_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=information_schema.sql
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=document.write.js
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=document.write.js
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=document.write.js
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=__proto__.json
|
|
||||||
/api/v1/repos/owner/repo/releases/1/assets?name=index.xhtml
|
|
||||||
/api/v1/repos/owner/repo/issues/1/assets?name=index.xhtml
|
|
||||||
/api/v1/repos/owner/repo/issues/comments/1/assets?name=index.xhtml
|
|
||||||
|
|
||||||
# An artifact upload from actions/upload-artifact@v3, which sends Content-Range.
|
|
||||||
/api/actions_pipeline/_apis/pipelines/workflows/1/artifacts/9b7c1f3a/upload?itemPath=artifact%2Freport.txt Content-Range: bytes 0-9/10
|
|
||||||
|
|
||||||
# Branch and tag names in the query string: deleting and restoring a branch,
|
|
||||||
# starting a release from a tag, a branch protection rule, a directory listing's
|
|
||||||
# last commits, and a container image push that reuses a layer.
|
|
||||||
/owner/repo/branches/delete?name=docker-build&page=1
|
|
||||||
/owner/repo/branches/restore?branch_id=1&name=docker-build&page=1
|
|
||||||
/owner/repo/releases/new?tag=docker-build
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=docker-build
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fdocker-build
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fdocker-build
|
|
||||||
/owner/repo/branches/delete?name=feat%2Fdocker-support&page=1
|
|
||||||
/owner/repo/branches/restore?branch_id=1&name=feat%2Fdocker-support&page=1
|
|
||||||
/owner/repo/releases/new?tag=feat%2Fdocker-support
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=feat%2Fdocker-support
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Ffeat%2Fdocker-support
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Ffeat%2Fdocker-support
|
|
||||||
/owner/repo/branches/delete?name=renovate%2Fdocker-build-push-action-6.x&page=1
|
|
||||||
/owner/repo/branches/restore?branch_id=1&name=renovate%2Fdocker-build-push-action-6.x&page=1
|
|
||||||
/owner/repo/releases/new?tag=renovate%2Fdocker-build-push-action-6.x
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=renovate%2Fdocker-build-push-action-6.x
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Frenovate%2Fdocker-build-push-action-6.x
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Frenovate%2Fdocker-build-push-action-6.x
|
|
||||||
/owner/repo/branches/delete?name=python3-migration&page=1
|
|
||||||
/owner/repo/branches/restore?branch_id=1&name=python3-migration&page=1
|
|
||||||
/owner/repo/releases/new?tag=python3-migration
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=python3-migration
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fpython3-migration
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fpython3-migration
|
|
||||||
/owner/repo/branches/delete?name=ansible-roles&page=1
|
|
||||||
/owner/repo/branches/restore?branch_id=1&name=ansible-roles&page=1
|
|
||||||
/owner/repo/releases/new?tag=ansible-roles
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=ansible-roles
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fansible-roles
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fansible-roles
|
|
||||||
/owner/repo/branches/delete?name=whoami&page=1
|
|
||||||
/owner/repo/branches/restore?branch_id=1&name=whoami&page=1
|
|
||||||
/owner/repo/releases/new?tag=whoami
|
|
||||||
/owner/repo/settings/branches/edit?rule_name=whoami
|
|
||||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fwhoami
|
|
||||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fwhoami
|
|
||||||
|
|
||||||
# Pages a signed-in user opens.
|
|
||||||
/
|
|
||||||
/owner/repo
|
|
||||||
/owner/repo/issues
|
|
||||||
/owner/repo/pulls
|
|
||||||
/owner/repo/releases
|
|
||||||
/owner/repo/wiki
|
|
||||||
/owner/repo/activity
|
|
||||||
/owner/repo/branches
|
|
||||||
/owner/repo/tags
|
|
||||||
/owner/repo/settings
|
|
||||||
/owner/repo/actions
|
|
||||||
/owner/repo/compare/main...main
|
|
||||||
/owner/repo/commits/branch/main
|
|
||||||
/owner/repo/archive/main.zip
|
|
||||||
/notifications
|
|
||||||
/user/settings
|
|
||||||
/user/settings/applications
|
|
||||||
/explore/repos
|
|
||||||
/-/admin
|
|
||||||
+15
-153
@@ -1,9 +1,8 @@
|
|||||||
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
||||||
// method, the URL with its query and the headers of a request, and on its
|
// method, the URL with its query and the headers of a request, and on its
|
||||||
// body while SWWAF_WAF_BODY_LIMIT is set, with the eight changes smallwebwaf
|
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
|
||||||
// makes to it and the exclusions in SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, as
|
// makes to it, as "Attack detection" under "Settings" in SPEC.md describes
|
||||||
// "Attack detection" under "Settings" in SPEC.md describes them. It reads no
|
// them. It reads no response.
|
||||||
// response.
|
|
||||||
//
|
//
|
||||||
// smallwebwaf writes only to its state directory, so Coraza is built with
|
// smallwebwaf writes only to its state directory, so Coraza is built with
|
||||||
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
||||||
@@ -12,14 +11,10 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
import (
|
import (
|
||||||
_ "embed"
|
|
||||||
"errors"
|
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"os"
|
|
||||||
"regexp"
|
|
||||||
"slices"
|
"slices"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -31,14 +26,12 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// directives are the Core Rule Set as smallwebwaf runs it, with the
|
// directives are the Core Rule Set as smallwebwaf runs it, with the
|
||||||
// paranoia level for %d, bodyDirectives for the first %s while
|
// paranoia level for %d, and bodyDirectives for %s while
|
||||||
// SWWAF_WAF_BODY_LIMIT is set, the exclusions file for the second, and
|
// SWWAF_WAF_BODY_LIMIT is set. Each rule smallwebwaf adds has an id from
|
||||||
// giteaExclusions for the third.
|
// 900000 to 900999, the ids the Core Rule Set keeps for the rules that set
|
||||||
// Each rule smallwebwaf adds has an id from 900000 to 900999, the ids the
|
// it up, which SWWAF_WAF_DISABLED_RULES refuses, so that no setting
|
||||||
// Core Rule Set keeps for the rules that set it up, which
|
// switches one off. Coraza joins a line ending in \ to the next, without
|
||||||
// SWWAF_WAF_DISABLED_RULES and the exclusions file refuse, so that no
|
// the spaces at the start of the next.
|
||||||
// setting switches one off. Coraza joins a line ending in \ to the next,
|
|
||||||
// without the spaces at the start of the next.
|
|
||||||
const directives = `
|
const directives = `
|
||||||
# The engine only detects. smallwebwaf compares the request's anomaly
|
# The engine only detects. smallwebwaf compares the request's anomaly
|
||||||
# score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
|
# score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
|
||||||
@@ -76,14 +69,6 @@ SecArgumentsLimit 1000
|
|||||||
SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
|
SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
|
||||||
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
||||||
|
|
||||||
# The exclusions file comes before the rules, where the Core Rule Set puts
|
|
||||||
# its own file of exclusions that run before the rules, so that they run
|
|
||||||
# before the rules of phase 1 too.
|
|
||||||
%s
|
|
||||||
|
|
||||||
# The eighth change comes before the rules for the same reason.
|
|
||||||
%s
|
|
||||||
|
|
||||||
# The sixth: only the rules for requests are loaded, and no response is
|
# The sixth: only the rules for requests are loaded, and no response is
|
||||||
# inspected.
|
# inspected.
|
||||||
Include @owasp_crs/REQUEST-*.conf
|
Include @owasp_crs/REQUEST-*.conf
|
||||||
@@ -111,19 +96,10 @@ SecRuleUpdateTargetById 932260 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
|
|||||||
!ARGS:artifactName|!ARGS:redirect_to"
|
!ARGS:artifactName|!ARGS:redirect_to"
|
||||||
|
|
||||||
# The fifth, for Referer: it is not checked for a Unix command without
|
# The fifth, for Referer: it is not checked for a Unix command without
|
||||||
# arguments, for Java starting a process, or for a Java class name. 944130
|
# arguments, or for Java starting a process. The cookies are left out in
|
||||||
# refused every page opened from the page of java.lang.String.md. The
|
# Inspect.
|
||||||
# cookies are left out in Inspect.
|
|
||||||
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
|
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
|
||||||
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
|
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
|
||||||
SecRuleUpdateTargetById 944130 "!REQUEST_HEADERS:Referer"
|
|
||||||
|
|
||||||
# The seventh: a container image push that reuses a layer names the image
|
|
||||||
# it takes the layer from in the query parameter from, which is not
|
|
||||||
# checked for command names. 932260 refused the push of a layer from
|
|
||||||
# owner/python3-app.
|
|
||||||
SecRule REQUEST_FILENAME "@rx ^/v2/.+/blobs/uploads/$" \
|
|
||||||
"id:900500,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:from"
|
|
||||||
`
|
`
|
||||||
|
|
||||||
// bodyDirectives have the Core Rule Set read the part of a request body
|
// bodyDirectives have the Core Rule Set read the part of a request body
|
||||||
@@ -168,13 +144,6 @@ SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
|
|||||||
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
||||||
`
|
`
|
||||||
|
|
||||||
// giteaExclusions are the eighth change: the fields of gitea's own
|
|
||||||
// requests that skip the rules that refused them there, as Coraza
|
|
||||||
// directives.
|
|
||||||
//
|
|
||||||
//go:embed gitea.conf
|
|
||||||
var giteaExclusions string
|
|
||||||
|
|
||||||
// cookiesNotRead are the cookies the Core Rule Set reads a request
|
// cookiesNotRead are the cookies the Core Rule Set reads a request
|
||||||
// without, the rest of the fifth change.
|
// without, the rest of the fifth change.
|
||||||
//
|
//
|
||||||
@@ -191,94 +160,6 @@ type Params struct {
|
|||||||
// BodyLimit is the most of a request body the Core Rule Set reads
|
// BodyLimit is the most of a request body the Core Rule Set reads
|
||||||
// (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
// (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
||||||
BodyLimit int64
|
BodyLimit int64
|
||||||
// ExclusionsFile is the path of a file of exclusions loaded before the
|
|
||||||
// Core Rule Set's rules (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
|
|
||||||
// there is none.
|
|
||||||
ExclusionsFile string
|
|
||||||
}
|
|
||||||
|
|
||||||
// The ids of smallwebwaf's own rules, which an exclusions file may not
|
|
||||||
// name.
|
|
||||||
const (
|
|
||||||
firstOwnRuleID = 900000
|
|
||||||
lastOwnRuleID = 900999
|
|
||||||
)
|
|
||||||
|
|
||||||
// numberOrRange matches a number, or a range of numbers such as
|
|
||||||
// 900000-900999. Coraza reads a number with a + before it too, so the
|
|
||||||
// end of a range may have one.
|
|
||||||
var numberOrRange = regexp.MustCompile(`(\d+)(?:-\+?(\d+))?`)
|
|
||||||
|
|
||||||
// errSmallwebwafRuleID is the error of an exclusions file that names an
|
|
||||||
// id of smallwebwaf's own rules.
|
|
||||||
var errSmallwebwafRuleID = errors.New("an id of smallwebwaf's own rules")
|
|
||||||
|
|
||||||
// errChangesNoRule is the error of an exclusions file that holds a
|
|
||||||
// directive that removes or changes rules as it loads, such as
|
|
||||||
// SecRuleRemoveById. The Core Rule Set's rules load after the file, so
|
|
||||||
// the directive finds none of them.
|
|
||||||
var errChangesNoRule = errors.New("the Core Rule Set's rules load after the file; " +
|
|
||||||
"switch a rule off with SWWAF_WAF_DISABLED_RULES, or on some requests " +
|
|
||||||
"with a ctl exclusion")
|
|
||||||
|
|
||||||
// directivesIn returns the directives in text as Coraza reads them: it
|
|
||||||
// skips blank lines and comment lines, and joins a line ending in \ to
|
|
||||||
// the next, without the spaces around either.
|
|
||||||
func directivesIn(text string) []string {
|
|
||||||
var directives []string
|
|
||||||
|
|
||||||
joined := ""
|
|
||||||
|
|
||||||
for line := range strings.Lines(text) {
|
|
||||||
line = strings.TrimSpace(line)
|
|
||||||
if line == "" || strings.HasPrefix(line, "#") {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
if start, found := strings.CutSuffix(line, `\`); found {
|
|
||||||
joined += start
|
|
||||||
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
directives = append(directives, joined+line)
|
|
||||||
joined = ""
|
|
||||||
}
|
|
||||||
|
|
||||||
return directives
|
|
||||||
}
|
|
||||||
|
|
||||||
// checkExclusions returns an error if exclusions, the text of an
|
|
||||||
// exclusions file, names an id of smallwebwaf's own rules, alone or in a
|
|
||||||
// range, or holds a directive whose name starts with SecRuleRemove or
|
|
||||||
// SecRuleUpdate.
|
|
||||||
func checkExclusions(exclusions string) error {
|
|
||||||
for _, directive := range directivesIn(exclusions) {
|
|
||||||
for _, match := range numberOrRange.FindAllStringSubmatch(directive, -1) {
|
|
||||||
first, _ := strconv.Atoi(match[1])
|
|
||||||
|
|
||||||
last := first
|
|
||||||
if match[2] != "" {
|
|
||||||
last, _ = strconv.Atoi(match[2])
|
|
||||||
}
|
|
||||||
|
|
||||||
if first <= lastOwnRuleID && last >= firstOwnRuleID {
|
|
||||||
return fmt.Errorf("the exclusions file names %s: %w", match[0],
|
|
||||||
errSmallwebwafRuleID)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Coraza takes the directive's name up to the first space, without
|
|
||||||
// regard to case.
|
|
||||||
name, _, _ := strings.Cut(directive, " ")
|
|
||||||
if lower := strings.ToLower(name); strings.HasPrefix(lower, "secruleremove") ||
|
|
||||||
strings.HasPrefix(lower, "secruleupdate") {
|
|
||||||
return fmt.Errorf("%s in the exclusions file changes no rule: %w", name,
|
|
||||||
errChangesNoRule)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return nil
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
|
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
|
||||||
@@ -288,35 +169,16 @@ type CoreRuleSet struct {
|
|||||||
bodyLimit int64
|
bodyLimit int64
|
||||||
}
|
}
|
||||||
|
|
||||||
// New returns the Core Rule Set with the eight changes, at params'
|
// New returns the Core Rule Set with the six changes, at params'
|
||||||
// paranoia level, without the rules it switches off, reading request
|
// paranoia level, without the rules it switches off, and reading request
|
||||||
// bodies up to params' limit, and with params' exclusions file loaded
|
// bodies up to params' limit.
|
||||||
// before the Core Rule Set's rules. An exclusions file is an error if it
|
|
||||||
// cannot be read or does not parse, or if checkExclusions refuses it.
|
|
||||||
func New(params Params) (*CoreRuleSet, error) {
|
func New(params Params) (*CoreRuleSet, error) {
|
||||||
body := ""
|
body := ""
|
||||||
if params.BodyLimit > 0 {
|
if params.BodyLimit > 0 {
|
||||||
body = fmt.Sprintf(bodyDirectives, params.BodyLimit)
|
body = fmt.Sprintf(bodyDirectives, params.BodyLimit)
|
||||||
}
|
}
|
||||||
|
|
||||||
exclusions := ""
|
text := fmt.Sprintf(directives, params.ParanoiaLevel, body)
|
||||||
|
|
||||||
if params.ExclusionsFile != "" {
|
|
||||||
file, err := os.ReadFile(params.ExclusionsFile)
|
|
||||||
if err != nil {
|
|
||||||
return nil, fmt.Errorf("read the exclusions file: %w", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
exclusions = string(file)
|
|
||||||
|
|
||||||
err = checkExclusions(exclusions)
|
|
||||||
if err != nil {
|
|
||||||
return nil, err
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions,
|
|
||||||
giteaExclusions)
|
|
||||||
|
|
||||||
if len(params.DisabledRules) > 0 {
|
if len(params.DisabledRules) > 0 {
|
||||||
ids := make([]string, len(params.DisabledRules))
|
ids := make([]string, len(params.DisabledRules))
|
||||||
|
|||||||
+2
-190
@@ -5,7 +5,6 @@ import (
|
|||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"net/url"
|
"net/url"
|
||||||
"os"
|
|
||||||
"path/filepath"
|
"path/filepath"
|
||||||
"reflect"
|
"reflect"
|
||||||
"strconv"
|
"strconv"
|
||||||
@@ -120,7 +119,7 @@ func atDefaults(t *testing.T) *waf.CoreRuleSet {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// wantChange checks that crs lets through passes, a gitea request one of
|
// wantChange checks that crs lets through passes, a gitea request one of
|
||||||
// the eight changes is for, and still finds result in refused, a request
|
// the six changes is for, and still finds result in refused, a request
|
||||||
// like it that the change is not for.
|
// like it that the change is not for.
|
||||||
func wantChange(
|
func wantChange(
|
||||||
t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result,
|
t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result,
|
||||||
@@ -309,14 +308,13 @@ func TestCookiesGiteaFlashAndRedirectToAreNotRead(t *testing.T) {
|
|||||||
matched(930120))
|
matched(930120))
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRefererIsNotCheckedForACommandOrJava(t *testing.T) {
|
func TestRefererIsNotCheckedForACommandOrJavaStartingAProcess(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const (
|
const (
|
||||||
search = "https://git.example/explore/repos?q=env"
|
search = "https://git.example/explore/repos?q=env"
|
||||||
runtimeJava = "https://git.example/openjdk/jdk/src/branch/master/src/" +
|
runtimeJava = "https://git.example/openjdk/jdk/src/branch/master/src/" +
|
||||||
"java.base/share/classes/java/lang/Runtime.java"
|
"java.base/share/classes/java/lang/Runtime.java"
|
||||||
stringMD = "https://git.example/owner/repo/src/branch/main/java.lang.String.md"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
crs := atDefaults(t)
|
crs := atDefaults(t)
|
||||||
@@ -325,8 +323,6 @@ func TestRefererIsNotCheckedForACommandOrJava(t *testing.T) {
|
|||||||
matched(932340))
|
matched(932340))
|
||||||
wantChange(t, crs, get("/", "Referer: "+runtimeJava),
|
wantChange(t, crs, get("/", "Referer: "+runtimeJava),
|
||||||
get("/", "X-Page: "+runtimeJava), matched(944110))
|
get("/", "X-Page: "+runtimeJava), matched(944110))
|
||||||
wantChange(t, crs, get("/", "Referer: "+stringMD), get("/", "X-Page: "+stringMD),
|
|
||||||
matched(944130))
|
|
||||||
|
|
||||||
// It is still checked for script and SQL injection.
|
// It is still checked for script and SQL injection.
|
||||||
wantResult(t, crs,
|
wantResult(t, crs,
|
||||||
@@ -366,190 +362,6 @@ func TestEachDisabledRuleIsSwitchedOff(t *testing.T) {
|
|||||||
wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{})
|
wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestImagePushThatReusesALayerIsNotCheckedForCommandNames(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
crs := atDefaults(t)
|
|
||||||
reuse := "?mount=sha256:0000&from=owner/python3-app"
|
|
||||||
|
|
||||||
wantChange(t, crs, get("/v2/owner/app/blobs/uploads/"+reuse), get("/upload/"+reuse),
|
|
||||||
matched(932260))
|
|
||||||
|
|
||||||
// A command after the image's name is still refused.
|
|
||||||
wantResult(t, crs,
|
|
||||||
get("/v2/owner/app/blobs/uploads/?from=owner/app;cat%20/etc/passwd"),
|
|
||||||
matched(930120, 932160))
|
|
||||||
}
|
|
||||||
|
|
||||||
// withExclusions returns the Core Rule Set at the defaults, with the
|
|
||||||
// exclusions file at path.
|
|
||||||
func withExclusions(path string) (*waf.CoreRuleSet, error) {
|
|
||||||
return waf.New(waf.Params{
|
|
||||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, ExclusionsFile: path,
|
|
||||||
})
|
|
||||||
}
|
|
||||||
|
|
||||||
// exclusionsFile writes text to a file of exclusions and returns its path.
|
|
||||||
func exclusionsFile(t *testing.T, text string) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
path := filepath.Join(t.TempDir(), "exclusions.conf")
|
|
||||||
|
|
||||||
err := os.WriteFile(path, []byte(text), 0o600)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return path
|
|
||||||
}
|
|
||||||
|
|
||||||
// backupFileExclusion is an exclusions file that switches 920500, a rule
|
|
||||||
// of phase 1, off for the path of a backup file under /files/.
|
|
||||||
const backupFileExclusion = `SecRule REQUEST_FILENAME "@beginsWith /files/" ` +
|
|
||||||
`"id:1,phase:1,pass,nolog,ctl:ruleRemoveTargetById=920500;REQUEST_FILENAME"` + "\n"
|
|
||||||
|
|
||||||
func TestExclusionsFileIsLoadedBeforeTheRules(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
crs, err := withExclusions(exclusionsFile(t, backupFileExclusion))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("load the exclusions: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
wantChange(t, crs, get("/files/notes.txt~"), get("/notes.txt~"), matched(920500))
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRuleSwitchedOffAndNamedInTheExclusionsFileIsOff(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
crs, err := waf.New(waf.Params{
|
|
||||||
ParanoiaLevel: 1, DisabledRules: []int{920500},
|
|
||||||
ExclusionsFile: exclusionsFile(t, backupFileExclusion),
|
|
||||||
})
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("load the exclusions: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
wantResult(t, crs, get("/notes.txt~"), waf.Result{})
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestExclusionsFileMayNameAnyNumberInAComment(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
_, err := withExclusions(exclusionsFile(t,
|
|
||||||
"# 942100: GET /issues/900123\n"+backupFileExclusion))
|
|
||||||
if err != nil {
|
|
||||||
t.Errorf("the exclusions did not load: %v", err)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestExclusionsFileThatCannotBeUsedIsAnError(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
missing := filepath.Join(t.TempDir(), "missing.conf")
|
|
||||||
|
|
||||||
_, err := withExclusions(missing)
|
|
||||||
if err == nil {
|
|
||||||
t.Errorf("%s loaded", missing)
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, tc := range []struct{ text, want string }{
|
|
||||||
{"SecNoSuchDirective On\n", "unknown directive"},
|
|
||||||
// smallwebwaf's own rules may not be switched off or changed, by
|
|
||||||
// id or in a range, on one line or split over several.
|
|
||||||
{"SecRuleRemoveById 899999-901000\n", "names 899999-901000"},
|
|
||||||
{
|
|
||||||
"SecAction \"id:1,phase:1,pass,nolog,ctl:ruleRemoveById=899999-901000\"\n",
|
|
||||||
"names 899999-901000",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"SecAction \"id:1,phase:1,pass,nolog,ctl:ruleRemoveById=899999-+901000\"\n",
|
|
||||||
"names 899999-+901000",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"SecAction \"id:1,phase:1,pass,nolog,ctl:ruleRemoveTargetById=900500;ARGS:from\"\n",
|
|
||||||
"names 900500",
|
|
||||||
},
|
|
||||||
{"SecRuleRemoveById 899000-\\\n 901000\n", "names 899000-901000"},
|
|
||||||
{
|
|
||||||
"SecAction \"id:1,phase:1,pass,nolog,\\\n" +
|
|
||||||
" ctl:ruleRemoveById=9005\\\n 00\"\n",
|
|
||||||
"names 900500",
|
|
||||||
},
|
|
||||||
// The Core Rule Set's rules load after the file.
|
|
||||||
{"SecRuleRemoveById 942100\n", "SecRuleRemoveById in the exclusions file"},
|
|
||||||
{
|
|
||||||
"SecRuleUpdateTargetById 942100 \"!ARGS:id\"\n",
|
|
||||||
"SecRuleUpdateTargetById in the exclusions file",
|
|
||||||
},
|
|
||||||
} {
|
|
||||||
_, err := withExclusions(exclusionsFile(t, tc.text))
|
|
||||||
if err == nil || !strings.Contains(err.Error(), tc.want) {
|
|
||||||
t.Errorf("%q gave the error %v, want one with %q", tc.text, err, tc.want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// giteaRequests returns the recorded ordinary gitea requests in testdata.
|
|
||||||
// Each has the Referer of a page that names the same, as a page opened
|
|
||||||
// from it has.
|
|
||||||
func giteaRequests(t *testing.T) []request {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
text, err := os.ReadFile(filepath.Join("testdata", "gitea-requests.txt"))
|
|
||||||
if err != nil {
|
|
||||||
t.Fatal(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
var requests []request
|
|
||||||
|
|
||||||
for line := range strings.Lines(string(text)) {
|
|
||||||
line = strings.TrimSpace(line)
|
|
||||||
if line == "" || strings.HasPrefix(line, "#") {
|
|
||||||
continue
|
|
||||||
}
|
|
||||||
|
|
||||||
target, header, _ := strings.Cut(line, " ")
|
|
||||||
|
|
||||||
headers := []string{"Referer: http://git.example" + target}
|
|
||||||
if header != "" {
|
|
||||||
headers = append(headers, header)
|
|
||||||
}
|
|
||||||
|
|
||||||
requests = append(requests, get(target, headers...))
|
|
||||||
}
|
|
||||||
|
|
||||||
return requests
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestOrdinaryGiteaRequestsPassAndAttacksAreRefused(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
const threshold = 5 // SWWAF_WAF_ANOMALY_THRESHOLD's default
|
|
||||||
|
|
||||||
crs := atDefaults(t)
|
|
||||||
|
|
||||||
for _, r := range giteaRequests(t) {
|
|
||||||
if got := inspect(t, crs, r); got.Score >= threshold {
|
|
||||||
t.Errorf("%s %q refused: %+v", r.target, r.headers[1:], got)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// In q too, off gitea's search pages.
|
|
||||||
for _, parameter := range []string{"id", "q"} {
|
|
||||||
for _, attack := range []string{
|
|
||||||
"1'%20OR%20'1'='1",
|
|
||||||
"..%2Fcustom%2Fconf%2Fapp.ini",
|
|
||||||
"%3Cscript%3Ealert(1)%3C%2Fscript%3E",
|
|
||||||
} {
|
|
||||||
target := "/owner/repo?" + parameter + "=" + attack
|
|
||||||
if got := inspect(t, crs, get(target)); got.Score < threshold {
|
|
||||||
t.Errorf("%s let through: %+v", target, got)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies.
|
// bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies.
|
||||||
const bodyLimit = 8 << 10
|
const bodyLimit = 8 << 10
|
||||||
|
|
||||||
|
|||||||
Executable
+350
@@ -0,0 +1,350 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/gitea-requests: build the image and run gitea 1.27.3 behind rproxy
|
||||||
|
# and servewaf, three containers on one network. Every setting is at its
|
||||||
|
# default but these: SWRP_UPSTREAM_URL names gitea's container, SWRP_WAF_BACKEND
|
||||||
|
# names servewaf's, and SWWAF_LOG_REQUEST_HEADERS adds the header that labels a
|
||||||
|
# request. Then send through rproxy the gitea requests that reading gitea's
|
||||||
|
# source found the Core Rule Set refuses, which docs/gitea.md records, and a
|
||||||
|
# short sample of gitea's ordinary use: signing in, pages of files and commits,
|
||||||
|
# a diff, the API, a push, clone and fetch over HTTP, and two container image
|
||||||
|
# pushes. Last, print each request servewaf logged, with what it did with it and
|
||||||
|
# the ids of the Core Rule Set's rules that matched it.
|
||||||
|
# Every request comes from one client, docker's gateway, so more than
|
||||||
|
# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
|
||||||
|
# the requests after would show as banned. The containers, the network,
|
||||||
|
# the images and the files the script makes are removed however it ends;
|
||||||
|
# gitea's own image stays, as the base images of a build do. It needs
|
||||||
|
# network access, and a docker daemon that pushes to a registry on
|
||||||
|
# localhost over plain HTTP, as docker does by default. script/check does
|
||||||
|
# not run this.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
|
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||||
|
|
||||||
|
# gitea 1.27.3, 2026-08-29
|
||||||
|
GITEA_IMAGE="gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef"
|
||||||
|
|
||||||
|
# Named after this run, so that runs in other clones on the same host
|
||||||
|
# never touch each other's.
|
||||||
|
NAME="$("$SCRIPT_DIR/projectname")-gitea-$$"
|
||||||
|
IMAGE="$NAME-image"
|
||||||
|
NETWORK="$NAME"
|
||||||
|
RPROXY="$NAME-rproxy"
|
||||||
|
SERVEWAF="$NAME-servewaf"
|
||||||
|
GITEA="$NAME-gitea"
|
||||||
|
|
||||||
|
# gitea's one user, who owns the repository and the container images.
|
||||||
|
PASSWORD="owner-password"
|
||||||
|
|
||||||
|
BROWSER="Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
|
||||||
|
|
||||||
|
# The header that labels a request in the report. rproxy makes every request's
|
||||||
|
# id itself, so a label cannot be the X-Request-ID.
|
||||||
|
LABEL_HEADER="X-Case"
|
||||||
|
|
||||||
|
# The default of SWWAF_LOG_REQUEST_HEADERS, and the header above.
|
||||||
|
LOGGED_HEADERS="accept,accept-language,accept-encoding,content-type,origin,range,x-case"
|
||||||
|
|
||||||
|
# The files the script makes, and rproxy's address once it runs, localhost and
|
||||||
|
# the port docker publishes it on: a host name, since the Core Rule Set takes a
|
||||||
|
# Host that is an IP address for a warning (920350).
|
||||||
|
WORK=""
|
||||||
|
address=""
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
# So that a signal cannot stop it partway through.
|
||||||
|
trap '' HUP INT TERM
|
||||||
|
docker rm --force --volumes "$RPROXY" "$SERVEWAF" "$GITEA" >/dev/null 2>&1 || true
|
||||||
|
docker network rm "$NETWORK" >/dev/null 2>&1 || true
|
||||||
|
docker rmi --force "$IMAGE" >/dev/null 2>&1 || true
|
||||||
|
if [ -n "$address" ]; then
|
||||||
|
docker rmi --force "$address/owner/python3-app:1" "$address/owner/app:1" \
|
||||||
|
>/dev/null 2>&1 || true
|
||||||
|
fi
|
||||||
|
rm -rf "$WORK"
|
||||||
|
}
|
||||||
|
|
||||||
|
fail() {
|
||||||
|
echo "gitea-requests: $*" >&2
|
||||||
|
report
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# wait_for <what fails> <command>...: run the command every second until
|
||||||
|
# it succeeds, for at most a minute.
|
||||||
|
wait_for() {
|
||||||
|
failure="$1"
|
||||||
|
shift
|
||||||
|
tries=0
|
||||||
|
until "$@"; do
|
||||||
|
tries=$((tries + 1))
|
||||||
|
[ "$tries" -lt 60 ] || fail "$failure"
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# healthy <container>: docker says the container's health check passes.
|
||||||
|
healthy() {
|
||||||
|
status="$(docker inspect --format '{{.State.Health.Status}}' "$1")"
|
||||||
|
[ "$status" = healthy ]
|
||||||
|
}
|
||||||
|
|
||||||
|
# logged <text>...: a line of servewaf's output holds every text, in any order.
|
||||||
|
logged() {
|
||||||
|
lines="$(docker logs "$SERVEWAF" 2>&1)"
|
||||||
|
for text in "$@"; do
|
||||||
|
lines="$(printf '%s\n' "$lines" | grep -F "$text")" || return 1
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# send <label> <path> <curl option>...: send a request for path to rproxy, with
|
||||||
|
# label in LABEL_HEADER, which the report gives, and keep the body of the answer
|
||||||
|
# in $WORK/answer. The Core Rule Set reads that header too, so a label names no
|
||||||
|
# file the cases name.
|
||||||
|
send() {
|
||||||
|
label="$1"
|
||||||
|
path="$2"
|
||||||
|
shift 2
|
||||||
|
curl --silent --show-error --max-time 30 --output "$WORK/answer" \
|
||||||
|
--header "$LABEL_HEADER: $label" "$@" "http://$address$path"
|
||||||
|
}
|
||||||
|
|
||||||
|
# page <label> <path> <curl option>...: send the request as a browser
|
||||||
|
# does, with the cookies gitea has set. Its Referer is the repository's
|
||||||
|
# page, which links to the others, and never a page the Core Rule Set
|
||||||
|
# refused: its address would have the next request refused too.
|
||||||
|
page() {
|
||||||
|
label="$1"
|
||||||
|
path="$2"
|
||||||
|
shift 2
|
||||||
|
send "$label" "$path" --compressed --user-agent "$BROWSER" \
|
||||||
|
--header "Accept: text/html,application/xhtml+xml,*/*;q=0.8" \
|
||||||
|
--header "Accept-Language: en-US,en;q=0.5" \
|
||||||
|
--cookie "$WORK/cookies" --cookie-jar "$WORK/cookies" \
|
||||||
|
--referer "http://$address/owner/repo" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
# api <label> <path> <curl option>...: send the request as a client of
|
||||||
|
# gitea's API does, as the owner.
|
||||||
|
api() {
|
||||||
|
label="$1"
|
||||||
|
path="$2"
|
||||||
|
shift 2
|
||||||
|
send "$label" "$path" --user "owner:$PASSWORD" \
|
||||||
|
--header "Accept: application/json" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
# repo_git <git argument>...: run git as the owner, without the host's
|
||||||
|
# own git settings, so that nothing it does is kept outside $WORK.
|
||||||
|
repo_git() {
|
||||||
|
GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 git \
|
||||||
|
-c user.name=owner -c user.email=owner@example.invalid "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
# lines: forty lines of text, enough for a diff to hide some of them.
|
||||||
|
lines() {
|
||||||
|
i=1
|
||||||
|
while [ "$i" -le 40 ]; do
|
||||||
|
echo "line $i"
|
||||||
|
i=$((i + 1))
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# make_repository: create the repository owner/repo, holding a file of
|
||||||
|
# each name the cases name, and push it, which is the push over HTTP of
|
||||||
|
# the sample. The second commit adds a line at the end of Runtime.java,
|
||||||
|
# so that its diff hides the lines above.
|
||||||
|
make_repository() {
|
||||||
|
status="$(api create-repository /api/v1/user/repos \
|
||||||
|
--header "Content-Type: application/json" --data '{"name":"repo"}' \
|
||||||
|
--write-out '%{http_code}')"
|
||||||
|
[ "$status" = 201 ] || fail "creating the repository was answered $status"
|
||||||
|
|
||||||
|
dir="$WORK/repo"
|
||||||
|
repo_git init --quiet --initial-branch=main "$dir"
|
||||||
|
for file in \
|
||||||
|
src/java.base/share/classes/java/lang/Runtime.java \
|
||||||
|
java.lang.String.md \
|
||||||
|
META-INF/services/org.apache.commons.logging.LogFactory \
|
||||||
|
document.write.js \
|
||||||
|
process.env.d.ts \
|
||||||
|
test/fixtures/__proto__.json \
|
||||||
|
install-script.sh \
|
||||||
|
src/backend/catalog/information_schema.sql \
|
||||||
|
.../notes.txt \
|
||||||
|
README.md; do
|
||||||
|
mkdir -p "$dir/$(dirname "$file")"
|
||||||
|
lines >"$dir/$file"
|
||||||
|
repo_git -C "$dir" add "$file"
|
||||||
|
done
|
||||||
|
repo_git -C "$dir" commit --quiet --message "Add the files"
|
||||||
|
echo "line 41" >>"$dir/src/java.base/share/classes/java/lang/Runtime.java"
|
||||||
|
repo_git -C "$dir" commit --quiet --all --message "Add a line"
|
||||||
|
|
||||||
|
repo_git -C "$dir" -c "http.extraHeader=$LABEL_HEADER: git-push" push --quiet \
|
||||||
|
"http://owner:$PASSWORD@$address/owner/repo.git" main ||
|
||||||
|
fail "the push of the repository failed"
|
||||||
|
}
|
||||||
|
|
||||||
|
# browse: sign in, then open pages as a person does. The cases are those
|
||||||
|
# of https://git.eeqj.de/sneak/smallwebwaf/issues/30.
|
||||||
|
browse() {
|
||||||
|
# Case: gitea sends a visitor who is not signed in to the sign-in
|
||||||
|
# page, with the page asked for in redirect_to, when that page needs
|
||||||
|
# signing in, as a file's edit page does.
|
||||||
|
page issue-30-case /owner/repo/_edit/main/document.write.js --location
|
||||||
|
|
||||||
|
page sign-in /user/login
|
||||||
|
status="$(page sign-in /user/login --header "Origin: http://$address" \
|
||||||
|
--data-urlencode user_name=owner --data-urlencode "password=$PASSWORD" \
|
||||||
|
--write-out '%{http_code}')"
|
||||||
|
[ "$status" = 303 ] || fail "signing in was answered $status"
|
||||||
|
|
||||||
|
page browse /
|
||||||
|
page browse /owner/repo
|
||||||
|
page browse /owner/repo/src/branch/main/README.md
|
||||||
|
page browse /owner/repo/commits/branch/main
|
||||||
|
page browse "/owner/repo/commit/$(repo_git -C "$WORK/repo" rev-parse HEAD)"
|
||||||
|
|
||||||
|
# Case: expanding the lines the diff hides, by the link the diff's
|
||||||
|
# page gives for it.
|
||||||
|
excerpt="$(sed -n 's/.*data-fetch-url="\([^"]*blob_excerpt[^"]*\)".*/\1/p' \
|
||||||
|
"$WORK/answer" | head -n 1 | sed 's/&/\&/g')"
|
||||||
|
[ -n "$excerpt" ] || fail "the diff's page has no link to its hidden lines"
|
||||||
|
page issue-30-case "$excerpt"
|
||||||
|
|
||||||
|
# Cases: the commit graph of one file, and the pages of files whose
|
||||||
|
# path the Core Rule Set takes for an attack.
|
||||||
|
page issue-30-case "/owner/repo/graph?file=src/java.base/share/classes/java/lang/Runtime.java"
|
||||||
|
page issue-30-case /owner/repo/src/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
|
||||||
|
page issue-30-case /owner/repo/src/branch/main/java.lang.String.md
|
||||||
|
page issue-30-case /owner/repo/src/branch/main/process.env.d.ts
|
||||||
|
page issue-30-case /owner/repo/src/branch/main/.../notes.txt
|
||||||
|
}
|
||||||
|
|
||||||
|
# call_api: the API's calls the cases name: the commits of a file, for a
|
||||||
|
# file of each name the Core Rule Set takes for an attack in the query
|
||||||
|
# parameter path, and the file-contents call, which takes its list of
|
||||||
|
# files as JSON in the query parameter body.
|
||||||
|
call_api() {
|
||||||
|
for file in \
|
||||||
|
src/java.base/share/classes/java/lang/Runtime.java \
|
||||||
|
java.lang.String.md \
|
||||||
|
document.write.js \
|
||||||
|
process.env.d.ts \
|
||||||
|
test/fixtures/__proto__.json \
|
||||||
|
install-script.sh \
|
||||||
|
src/backend/catalog/information_schema.sql; do
|
||||||
|
api issue-30-case /api/v1/repos/owner/repo/commits --get --data-urlencode "path=$file"
|
||||||
|
done
|
||||||
|
api issue-30-case /api/v1/repos/owner/repo/file-contents --get \
|
||||||
|
--data-urlencode 'body={"files":["src/java.base/share/classes/java/lang/Runtime.java"]}'
|
||||||
|
}
|
||||||
|
|
||||||
|
# push_images: push a container image to owner/python3-app, then the same
|
||||||
|
# image to owner/app. For the second, the case, docker asks gitea to take
|
||||||
|
# its layer from owner/python3-app, naming it in the query parameter
|
||||||
|
# from. docker sends no label, so the report gives its requests none. The image
|
||||||
|
# is one small file, imported rather than built.
|
||||||
|
push_images() {
|
||||||
|
echo "$NAME" >"$WORK/layer.txt"
|
||||||
|
tar -cf - -C "$WORK" layer.txt |
|
||||||
|
docker import - "$address/owner/python3-app:1" >/dev/null
|
||||||
|
docker tag "$address/owner/python3-app:1" "$address/owner/app:1"
|
||||||
|
|
||||||
|
# The login is written here rather than made with docker login, which
|
||||||
|
# would keep it in the host's credential helper when it has one.
|
||||||
|
mkdir "$WORK/docker"
|
||||||
|
printf '{"auths":{"%s":{"auth":"%s"}}}\n' "$address" \
|
||||||
|
"$(printf 'owner:%s' "$PASSWORD" | base64)" >"$WORK/docker/config.json"
|
||||||
|
for image in "$address/owner/python3-app:1" "$address/owner/app:1"; do
|
||||||
|
DOCKER_CONFIG="$WORK/docker" docker push --quiet "$image" >/dev/null ||
|
||||||
|
echo "gitea-requests: the push of $image failed" >&2
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# clone_and_fetch: the clone and fetch over HTTP of the sample.
|
||||||
|
clone_and_fetch() {
|
||||||
|
repo_git -c "http.extraHeader=$LABEL_HEADER: git-clone" clone --quiet \
|
||||||
|
"http://owner:$PASSWORD@$address/owner/repo.git" "$WORK/clone" ||
|
||||||
|
fail "the clone failed"
|
||||||
|
repo_git -C "$WORK/clone" -c "http.extraHeader=$LABEL_HEADER: git-fetch" fetch --quiet ||
|
||||||
|
fail "the fetch failed"
|
||||||
|
}
|
||||||
|
|
||||||
|
# field <name> <line>: the text of the field name in line, a line of the
|
||||||
|
# request log, or nothing when it has none.
|
||||||
|
field() {
|
||||||
|
printf '%s\n' "$2" | sed -n "s/.*\"$1\":\"\([^\"]*\)\".*/\1/p"
|
||||||
|
}
|
||||||
|
|
||||||
|
# report: print a line for each request rproxy reported to servewaf: what
|
||||||
|
# servewaf did with the request, the ids of the Core Rule Set's rules that
|
||||||
|
# matched it, its label, and the request. waf_blocked is a refusal by the Core
|
||||||
|
# Rule Set, and forward a request passed on to gitea, whatever gitea answered.
|
||||||
|
report() {
|
||||||
|
docker logs "$SERVEWAF" 2>/dev/null | grep '"servewaf_line":"report"' |
|
||||||
|
sed 's/\\u0026/\&/g' >"$WORK/requests"
|
||||||
|
while IFS= read -r line; do
|
||||||
|
ids="$(printf '%s\n' "$line" | sed -n 's/.*"waf_rule_ids":\[\([0-9,]*\)\].*/\1/p')"
|
||||||
|
query="$(field query "$line")"
|
||||||
|
label="$(field x-case "$line")"
|
||||||
|
printf '%-11s %-13s %-17s %s %s%s\n' "$(field action "$line")" "${ids:--}" \
|
||||||
|
"${label:--}" "$(field method "$line")" \
|
||||||
|
"$(field path "$line")" "${query:+?$query}"
|
||||||
|
done <"$WORK/requests"
|
||||||
|
echo "gitea-requests: $(grep -c . "$WORK/requests") requests," \
|
||||||
|
"$(grep -c '"action":"waf_blocked"' "$WORK/requests") refused by the Core Rule Set"
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT"
|
||||||
|
trap cleanup EXIT
|
||||||
|
trap 'exit 1' HUP INT TERM
|
||||||
|
WORK="$(mktemp -d)"
|
||||||
|
|
||||||
|
docker build --no-cache -t "$IMAGE" .
|
||||||
|
|
||||||
|
# rproxy first, so that gitea can be given the address it is reached at,
|
||||||
|
# and gitea's image pulled before, so that gitea starts within the minute
|
||||||
|
# rproxy's health check waits for it.
|
||||||
|
docker pull --quiet "$GITEA_IMAGE" >/dev/null
|
||||||
|
docker network create "$NETWORK" >/dev/null
|
||||||
|
docker run --detach --name "$SERVEWAF" --network "$NETWORK" \
|
||||||
|
--env SMALLWEBWAF_COMMAND=servewaf \
|
||||||
|
--env SWWAF_LOG_REQUEST_HEADERS="$LOGGED_HEADERS" \
|
||||||
|
"$IMAGE" >/dev/null
|
||||||
|
wait_for "servewaf's health check did not pass" healthy "$SERVEWAF"
|
||||||
|
docker run --detach --name "$RPROXY" --network "$NETWORK" \
|
||||||
|
--publish 127.0.0.1::8080 \
|
||||||
|
--env SWRP_UPSTREAM_URL="http://$GITEA:3000" \
|
||||||
|
--env SWRP_WAF_BACKEND="http://$SERVEWAF:8080" \
|
||||||
|
"$IMAGE" >/dev/null
|
||||||
|
published="$(docker port "$RPROXY" 8080/tcp)"
|
||||||
|
address="localhost:${published##*:}"
|
||||||
|
|
||||||
|
docker run --detach --name "$GITEA" --network "$NETWORK" \
|
||||||
|
--env GITEA__security__INSTALL_LOCK=true \
|
||||||
|
--env GITEA__server__ROOT_URL="http://$address/" \
|
||||||
|
"$GITEA_IMAGE" >/dev/null
|
||||||
|
# rproxy's health check passes once gitea takes connections.
|
||||||
|
wait_for "rproxy's health check did not pass" healthy "$RPROXY"
|
||||||
|
docker exec --user git "$GITEA" gitea admin user create --admin \
|
||||||
|
--username owner --password "$PASSWORD" --email owner@example.invalid \
|
||||||
|
--must-change-password=false >/dev/null
|
||||||
|
|
||||||
|
make_repository
|
||||||
|
browse
|
||||||
|
call_api
|
||||||
|
push_images
|
||||||
|
clone_and_fetch
|
||||||
|
|
||||||
|
# The fetch's last request is the last of all; once it is logged, so
|
||||||
|
# is every other.
|
||||||
|
wait_for "servewaf logged no report of the fetch" \
|
||||||
|
logged '"x-case":"git-fetch"' '"method":"POST"' '"servewaf_line":"report"'
|
||||||
|
report
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
Executable
+216
@@ -0,0 +1,216 @@
|
|||||||
|
#!/bin/sh
|
||||||
|
# script/gitea-setup: build the image, bring up deploy/gitea under a project
|
||||||
|
# name of its own, and check it:
|
||||||
|
# - that a page load, a push and a clone sent through traefik pass through
|
||||||
|
# rproxy, and that servewaf checks them;
|
||||||
|
# - that gitea logs the visitor's address, not rproxy's;
|
||||||
|
# - that a request from a container on the shared network skips the checks;
|
||||||
|
# - that with servewaf stopped, gitea still answers through rproxy.
|
||||||
|
# It publishes no port of traefik's, since this host's ports 80 and 443 may be
|
||||||
|
# taken. It reaches traefik at its address on the shared network instead, so
|
||||||
|
# its own requests come from the network's gateway, which servewaf checks like
|
||||||
|
# any visitor. Two runs at once on one docker host cannot both start, since
|
||||||
|
# both would use the subnet compose.yaml names. The containers, the network,
|
||||||
|
# the volumes, the image and the files the script makes are removed however it
|
||||||
|
# ends; the gitea and traefik images stay, as the base images of a build do.
|
||||||
|
# It needs network access. script/check does not run this.
|
||||||
|
set -eu
|
||||||
|
|
||||||
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
|
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||||
|
|
||||||
|
# Named after this run, so that runs in other clones on the same host
|
||||||
|
# never touch each other's.
|
||||||
|
NAME="$("$SCRIPT_DIR/projectname")-gitea-setup-$$"
|
||||||
|
IMAGE="$NAME-image"
|
||||||
|
NETWORK="${NAME}_shared"
|
||||||
|
VISITOR="$NAME-visitor"
|
||||||
|
|
||||||
|
# The site's host name and gitea's image, as deploy/gitea/compose.yaml names
|
||||||
|
# them. gitea's image also runs the container on the shared network.
|
||||||
|
SITE="git.example.org"
|
||||||
|
# gitea 1.27.3, 2026-08-29
|
||||||
|
GITEA_IMAGE="gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef"
|
||||||
|
|
||||||
|
# gitea's one user, who owns the repository.
|
||||||
|
PASSWORD="owner-password"
|
||||||
|
|
||||||
|
# The files the script makes, gitea's secret key for this run, and traefik's
|
||||||
|
# and the gateway's addresses on the shared network once the setup runs.
|
||||||
|
WORK=""
|
||||||
|
secret_key=""
|
||||||
|
traefik=""
|
||||||
|
gateway=""
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
# So that a signal cannot stop it partway through.
|
||||||
|
trap '' HUP INT TERM
|
||||||
|
docker rm --force "$VISITOR" >/dev/null 2>&1 || true
|
||||||
|
if [ -n "$WORK" ]; then
|
||||||
|
compose down --volumes --remove-orphans --timeout 5 >/dev/null 2>&1 || true
|
||||||
|
rm -rf "$WORK"
|
||||||
|
fi
|
||||||
|
docker rmi --force "$IMAGE" >/dev/null 2>&1 || true
|
||||||
|
}
|
||||||
|
|
||||||
|
# compose <argument>...: docker compose on the setup, under this run's project
|
||||||
|
# name, with the image built here, this run's secret key and no port of
|
||||||
|
# traefik's published.
|
||||||
|
compose() {
|
||||||
|
SMALLWEBWAF_IMAGE="$IMAGE" GITEA_SECRET_KEY="$secret_key" \
|
||||||
|
docker compose --project-name "$NAME" \
|
||||||
|
--file "$ROOT/deploy/gitea/compose.yaml" --file "$WORK/no-ports.yaml" "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
fail() {
|
||||||
|
echo "gitea-setup: $*; the last lines of each container's output:" >&2
|
||||||
|
compose logs --no-color --tail 30 >&2 || true
|
||||||
|
exit 1
|
||||||
|
}
|
||||||
|
|
||||||
|
# wait_for <what fails> <command>...: run the command every second until
|
||||||
|
# it succeeds, for at most a minute.
|
||||||
|
wait_for() {
|
||||||
|
failure="$1"
|
||||||
|
shift
|
||||||
|
tries=0
|
||||||
|
until "$@"; do
|
||||||
|
tries=$((tries + 1))
|
||||||
|
[ "$tries" -lt 60 ] || fail "$failure"
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# logged <service> <text>...: a line of the service's output holds every text,
|
||||||
|
# in any order.
|
||||||
|
logged() {
|
||||||
|
lines="$(compose logs --no-color --no-log-prefix "$1" 2>&1)"
|
||||||
|
shift
|
||||||
|
for text in "$@"; do
|
||||||
|
lines="$(printf '%s\n' "$lines" | grep -F -e "$text")" || return 1
|
||||||
|
done
|
||||||
|
}
|
||||||
|
|
||||||
|
# send <path> <curl option>...: send a request for path to the site, at
|
||||||
|
# traefik's address, and print the status of the answer, 000 for none. traefik
|
||||||
|
# answers with a certificate it made itself, since the setup names no
|
||||||
|
# certificate resolver.
|
||||||
|
send() {
|
||||||
|
path="$1"
|
||||||
|
shift
|
||||||
|
curl --silent --show-error --insecure --max-time 30 --output /dev/null \
|
||||||
|
--write-out '%{http_code}' --resolve "$SITE:443:$traefik" "$@" \
|
||||||
|
"https://$SITE$path" || true
|
||||||
|
}
|
||||||
|
|
||||||
|
# routed: traefik passes a request for the site's front page on to gitea.
|
||||||
|
routed() {
|
||||||
|
[ "$(send / 2>/dev/null)" = 200 ]
|
||||||
|
}
|
||||||
|
|
||||||
|
# site_git <git argument>...: run git as the owner, without the host's own git
|
||||||
|
# settings, reaching the site at traefik's address. git gives up on a transfer
|
||||||
|
# slower than 1000 bytes a second for 30 seconds.
|
||||||
|
site_git() {
|
||||||
|
GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 git \
|
||||||
|
-c user.name=owner -c user.email=owner@example.invalid \
|
||||||
|
-c http.sslVerify=false -c "http.curloptResolve=$SITE:443:$traefik" \
|
||||||
|
-c http.lowSpeedLimit=1000 -c http.lowSpeedTime=30 "$@"
|
||||||
|
}
|
||||||
|
|
||||||
|
# address_of <service>: print the service's address on the shared network.
|
||||||
|
address_of() {
|
||||||
|
docker inspect --format "{{(index .NetworkSettings.Networks \"$NETWORK\").IPAddress}}" \
|
||||||
|
"$(compose ps --quiet "$1")"
|
||||||
|
}
|
||||||
|
|
||||||
|
# passed_through <method> <path>: rproxy passed a request from the gateway for
|
||||||
|
# path to gitea on servewaf's allow, and servewaf checked it: its report line
|
||||||
|
# gives the client's counts and the Core Rule Set's score.
|
||||||
|
passed_through() {
|
||||||
|
wait_for "rproxy logged no $1 $2 that servewaf allowed" logged rproxy \
|
||||||
|
"\"method\":\"$1\"" "\"path\":\"$2\"" "\"client_ip\":\"$gateway\"" \
|
||||||
|
'"disposition":"allow"'
|
||||||
|
wait_for "servewaf logged no check of $1 $2" logged servewaf \
|
||||||
|
'"servewaf_line":"report"' "\"method\":\"$1\"" "\"path\":\"$2\"" \
|
||||||
|
"\"client_ip\":\"$gateway\"" '"action":"forward"' '"counts":' '"waf_score":'
|
||||||
|
}
|
||||||
|
|
||||||
|
main() {
|
||||||
|
cd "$ROOT"
|
||||||
|
trap cleanup EXIT
|
||||||
|
trap 'exit 1' HUP INT TERM
|
||||||
|
WORK="$(mktemp -d)"
|
||||||
|
printf 'services:\n traefik:\n ports: !reset []\n' >"$WORK/no-ports.yaml"
|
||||||
|
|
||||||
|
docker build --no-cache -t "$IMAGE" .
|
||||||
|
secret_key="$(docker run --rm "$GITEA_IMAGE" gitea generate secret SECRET_KEY)"
|
||||||
|
|
||||||
|
# rproxy is healthy once gitea takes connections.
|
||||||
|
compose up --detach --quiet-pull --wait --wait-timeout 120 ||
|
||||||
|
fail "the setup did not start"
|
||||||
|
traefik="$(address_of traefik)"
|
||||||
|
gateway="$(docker network inspect --format '{{(index .IPAM.Config 0).Gateway}}' "$NETWORK")"
|
||||||
|
wait_for "traefik did not route the site to rproxy" routed
|
||||||
|
|
||||||
|
compose exec --user git gitea gitea admin user create --admin \
|
||||||
|
--username owner --password "$PASSWORD" --email owner@example.invalid \
|
||||||
|
--must-change-password=false >/dev/null || fail "creating gitea's user failed"
|
||||||
|
status="$(send /api/v1/user/repos --user "owner:$PASSWORD" \
|
||||||
|
--header "Content-Type: application/json" --data '{"name":"repo"}')"
|
||||||
|
[ "$status" = 201 ] || fail "creating the repository was answered $status"
|
||||||
|
|
||||||
|
status="$(send /owner/repo)"
|
||||||
|
[ "$status" = 200 ] || fail "the repository's page was answered $status"
|
||||||
|
passed_through GET /owner/repo
|
||||||
|
echo "gitea-setup: a page load passes through rproxy, and servewaf checks it"
|
||||||
|
|
||||||
|
wait_for "gitea did not log the visitor's address" logged gitea \
|
||||||
|
"GET /owner/repo for $gateway:"
|
||||||
|
echo "gitea-setup: gitea logs the visitor's address, not rproxy's"
|
||||||
|
|
||||||
|
site_git init --quiet --initial-branch=main "$WORK/repo"
|
||||||
|
echo "hello" >"$WORK/repo/README.md"
|
||||||
|
site_git -C "$WORK/repo" add README.md
|
||||||
|
site_git -C "$WORK/repo" commit --quiet --message "Add a README"
|
||||||
|
site_git -C "$WORK/repo" push --quiet \
|
||||||
|
"https://owner:$PASSWORD@$SITE/owner/repo.git" main ||
|
||||||
|
fail "the push failed"
|
||||||
|
passed_through POST /owner/repo.git/git-receive-pack
|
||||||
|
echo "gitea-setup: a push passes through rproxy, and servewaf checks it"
|
||||||
|
|
||||||
|
site_git clone --quiet "https://owner:$PASSWORD@$SITE/owner/repo.git" \
|
||||||
|
"$WORK/clone" || fail "the clone failed"
|
||||||
|
[ "$(cat "$WORK/clone/README.md")" = hello ] || fail "the clone lacks the README"
|
||||||
|
passed_through POST /owner/repo.git/git-upload-pack
|
||||||
|
echo "gitea-setup: a clone passes through rproxy, and servewaf checks it"
|
||||||
|
|
||||||
|
# A container on the shared network, as sandcastle is, asks for the site
|
||||||
|
# by its name, and so reaches traefik from its own address.
|
||||||
|
status="$(docker run --rm --name "$VISITOR" --network "$NETWORK" \
|
||||||
|
--entrypoint curl "$GITEA_IMAGE" --silent --insecure --max-time 30 \
|
||||||
|
--output /dev/null --write-out '%{http_code}' "https://$SITE/api/v1/version" ||
|
||||||
|
true)"
|
||||||
|
[ "$status" = 200 ] || fail "the request from the shared network was answered $status"
|
||||||
|
wait_for "servewaf logged no request from the shared network" logged servewaf \
|
||||||
|
'"servewaf_line":"report"' '"path":"/api/v1/version"' '"action":"forward"'
|
||||||
|
line="$(compose logs --no-color --no-log-prefix servewaf |
|
||||||
|
grep -F '"servewaf_line":"report"' | grep -F '"path":"/api/v1/version"')"
|
||||||
|
case "$line" in
|
||||||
|
*"\"client_ip\":\"$gateway\""*)
|
||||||
|
fail "the request from the shared network came from the gateway" ;;
|
||||||
|
*'"counts":'* | *'"waf_score":'*)
|
||||||
|
fail "servewaf checked the request from the shared network" ;;
|
||||||
|
esac
|
||||||
|
echo "gitea-setup: a request from the shared network skips the checks"
|
||||||
|
|
||||||
|
compose stop servewaf >/dev/null 2>&1 || fail "servewaf did not stop"
|
||||||
|
status="$(send /owner/repo)"
|
||||||
|
[ "$status" = 200 ] ||
|
||||||
|
fail "with servewaf stopped, the repository's page was answered $status"
|
||||||
|
wait_for "rproxy logged no request SWRP_DEFAULT_DISPOSITION decided" \
|
||||||
|
logged rproxy '"path":"/owner/repo"' '"default_disposition":true'
|
||||||
|
echo "gitea-setup: with servewaf stopped, gitea answers through rproxy"
|
||||||
|
}
|
||||||
|
|
||||||
|
main "$@"
|
||||||
Reference in new issue
Block a user