Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3669abe761 |
No files matched your search
@@ -308,7 +308,6 @@ Every setting the code reads, with its default:
|
|||||||
| `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
|
| `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
|
||||||
| `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
|
| `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
|
||||||
| `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
| `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
||||||
| `SWW_RPROXY_CONTACT_EMAIL` | unset | The email address the refusal pages link to. |
|
|
||||||
| `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. |
|
| `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. |
|
||||||
| `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `host1/waf`. |
|
| `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `host1/waf`. |
|
||||||
| `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
|
| `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
|
||||||
|
|||||||
@@ -828,8 +828,8 @@ The supported apps are:
|
|||||||
- `vaultwarden`;
|
- `vaultwarden`;
|
||||||
- `wordpress`.
|
- `wordpress`.
|
||||||
|
|
||||||
Each app's fixes are in one file named for the app, such as
|
Each app's fixes are in one file, such as `internal/coreruleset/gitea.conf`, and
|
||||||
`internal/coreruleset/apps/gitea.conf`. gitea's fixes are:
|
one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||||
|
|
||||||
1. Ten parameters are not checked against the Core Rule Set's lists of system
|
1. Ten parameters are not checked against the Core Rule Set's lists of system
|
||||||
files (930120), shell paths (932160) and command names (932260), on any path:
|
files (930120), shell paths (932160) and command names (932260), on any path:
|
||||||
@@ -862,8 +862,8 @@ Each app's fixes are in one file named for the app, such as
|
|||||||
3. On gitea's paths, some fields skip the rules that refused ordinary gitea
|
3. On gitea's paths, some fields skip the rules that refused ordinary gitea
|
||||||
requests there. They are the fields where gitea sends a search, a file's name
|
requests there. They are the fields where gitea sends a search, a file's name
|
||||||
or text, a commit message, a branch name or a repository's name.
|
or text, a commit message, a branch name or a repository's name.
|
||||||
`internal/coreruleset/apps/gitea.conf` lists each rule and the request it
|
`internal/coreruleset/gitea.conf` lists each rule and the request it refused.
|
||||||
refused. The fields are:
|
The fields are:
|
||||||
- `q` on gitea's searches of code, of issues and pull requests, and of
|
- `q` on gitea's searches of code, of issues and pull requests, and of
|
||||||
commits. A code search holds code, such as `<script>`,
|
commits. A code search holds code, such as `<script>`,
|
||||||
`SELECT * FROM users WHERE` or `../`. The searches of issues and pull
|
`SELECT * FROM users WHERE` or `../`. The searches of issues and pull
|
||||||
@@ -1048,7 +1048,7 @@ Each app's fixes are in one file named for the app, such as
|
|||||||
- The `Content-Range` header of an artifact upload from
|
- The `Content-Range` header of an artifact upload from
|
||||||
`actions/upload-artifact@v3` (920450).
|
`actions/upload-artifact@v3` (920450).
|
||||||
- The passwords in a body, for each rule that random passwords set off. The
|
- The passwords in a body, for each rule that random passwords set off. The
|
||||||
numbers are in `internal/coreruleset/apps/gitea.conf`.
|
numbers are in `internal/coreruleset/gitea.conf`.
|
||||||
- `password` on these paths:
|
- `password` on these paths:
|
||||||
- `/user/login`, `/user/sign_up`, `/user/activate`;
|
- `/user/login`, `/user/sign_up`, `/user/activate`;
|
||||||
- `/user/link_account_signin` and `/user/link_account_signup`;
|
- `/user/link_account_signin` and `/user/link_account_signup`;
|
||||||
@@ -1080,8 +1080,8 @@ Each app's fixes are in one file named for the app, such as
|
|||||||
such as `owner/python3-app` would have that push refused.
|
such as `owner/python3-app` would have that push refused.
|
||||||
|
|
||||||
WordPress's fixes: on WordPress's paths, some fields skip the rules that refused
|
WordPress's fixes: on WordPress's paths, some fields skip the rules that refused
|
||||||
ordinary WordPress requests there. `internal/coreruleset/apps/wordpress.conf`
|
ordinary WordPress requests there. `internal/coreruleset/wordpress.conf` lists
|
||||||
lists each rule and the request it refused. The REST API has four addresses:
|
each rule and the request it refused. The REST API has four addresses:
|
||||||
|
|
||||||
- `/wp-json/...`
|
- `/wp-json/...`
|
||||||
- `/index.php/wp-json/...`
|
- `/index.php/wp-json/...`
|
||||||
@@ -1173,9 +1173,9 @@ Elsewhere WordPress's fixes do not apply. The paths are those of WordPress at
|
|||||||
the site root.
|
the site root.
|
||||||
|
|
||||||
Drupal's fixes: on Drupal's paths, some fields skip the rules that refused
|
Drupal's fixes: on Drupal's paths, some fields skip the rules that refused
|
||||||
ordinary Drupal requests there. `internal/coreruleset/apps/drupal.conf` lists
|
ordinary Drupal requests there. `internal/coreruleset/drupal.conf` lists each
|
||||||
each rule and the request it refused. These query fields skip the lists of
|
rule and the request it refused. These query fields skip the lists of system
|
||||||
system files and command names (930120, 932120, 932250, 932260):
|
files and command names (930120, 932120, 932250, 932260):
|
||||||
|
|
||||||
- On every page under `/admin`: `title`, `name`, `user`, `filename`, `filemime`,
|
- On every page under `/admin`: `title`, `name`, `user`, `filename`, `filemime`,
|
||||||
`info`, `subject`, `author_name`, `search`, `q` and `destination`.
|
`info`, `subject`, `author_name`, `search`, `q` and `destination`.
|
||||||
@@ -1265,10 +1265,10 @@ first path segment of two or three lowercase letters, optionally followed by a
|
|||||||
hyphen and more letters, so `/api` counts too.
|
hyphen and more letters, so `/api` counts too.
|
||||||
|
|
||||||
MediaWiki's fixes: on MediaWiki's paths, some fields skip the rules that refused
|
MediaWiki's fixes: on MediaWiki's paths, some fields skip the rules that refused
|
||||||
ordinary MediaWiki requests there. `internal/coreruleset/apps/mediawiki.conf`
|
ordinary MediaWiki requests there. `internal/coreruleset/mediawiki.conf` lists
|
||||||
lists each rule and the request it refused. These fields skip the lists of
|
each rule and the request it refused. These fields skip the lists of system
|
||||||
system files and command names (930120, 932120, 932250, 932260), in the query or
|
files and command names (930120, 932120, 932250, 932260), in the query or the
|
||||||
the body:
|
body:
|
||||||
|
|
||||||
- On MediaWiki's pages, which are `/index.php` and the paths under `/index.php/`
|
- On MediaWiki's pages, which are `/index.php` and the paths under `/index.php/`
|
||||||
and `/wiki/`:
|
and `/wiki/`:
|
||||||
@@ -1374,9 +1374,9 @@ Elsewhere these fields keep every rule. The paths are those of MediaWiki at the
|
|||||||
site root or under `/w/`.
|
site root or under `/w/`.
|
||||||
|
|
||||||
Mastodon's fixes: on Mastodon's paths, some fields skip the rules that refused
|
Mastodon's fixes: on Mastodon's paths, some fields skip the rules that refused
|
||||||
ordinary Mastodon requests there. `internal/coreruleset/apps/mastodon.conf`
|
ordinary Mastodon requests there. `internal/coreruleset/mastodon.conf` lists
|
||||||
lists each rule and the request it refused. These fields skip the lists of
|
each rule and the request it refused. These fields skip the lists of system
|
||||||
system files and command names (930120, 932120, 932250, 932260), in the query:
|
files and command names (930120, 932120, 932250, 932260), in the query:
|
||||||
|
|
||||||
- On the API's search, accounts and admin accounts, under `/api/v1/` and
|
- On the API's search, accounts and admin accounts, under `/api/v1/` and
|
||||||
`/api/v2/`, the searches and names `q`, `acct`, `username`, `display_name`,
|
`/api/v2/`, the searches and names `q`, `acct`, `username`, `display_name`,
|
||||||
@@ -1537,7 +1537,7 @@ every rule.
|
|||||||
|
|
||||||
Vaultwarden's fixes: on Vaultwarden's paths, some fields skip the rules that
|
Vaultwarden's fixes: on Vaultwarden's paths, some fields skip the rules that
|
||||||
refused ordinary Vaultwarden requests there.
|
refused ordinary Vaultwarden requests there.
|
||||||
`internal/coreruleset/apps/vaultwarden.conf` lists each rule and the request it
|
`internal/coreruleset/vaultwarden.conf` lists each rule and the request it
|
||||||
refused. The apps encrypt vault items, folders, collections and sends before
|
refused. The apps encrypt vault items, folders, collections and sends before
|
||||||
sending them, so those fields skip no rules. The fields below hold plain text.
|
sending them, so those fields skip no rules. The fields below hold plain text.
|
||||||
|
|
||||||
@@ -1588,8 +1588,8 @@ has a parameter of the same name.
|
|||||||
|
|
||||||
`token` in the body of `/admin`, the admin token typed on the admin page, skips
|
`token` in the body of `/admin`, the admin token typed on the admin page, skips
|
||||||
each rule that random passwords set off. The numbers are in
|
each rule that random passwords set off. The numbers are in
|
||||||
`internal/coreruleset/apps/vaultwarden.conf`. It applies only when the query has
|
`internal/coreruleset/vaultwarden.conf`. It applies only when the query has no
|
||||||
no `token` parameter. `' OR '1'='1`, `../../etc/passwd`, `; cat /etc/passwd` and
|
`token` parameter. `' OR '1'='1`, `../../etc/passwd`, `; cat /etc/passwd` and
|
||||||
`<img src=x onerror=x()>` sent as the token reach Vaultwarden;
|
`<img src=x onerror=x()>` sent as the token reach Vaultwarden;
|
||||||
`<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
|
`<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
|
||||||
|
|
||||||
@@ -1611,7 +1611,7 @@ Elsewhere these fields keep every rule.
|
|||||||
|
|
||||||
Nextcloud's fixes: on Nextcloud's paths, some fields skip the rules that refused
|
Nextcloud's fixes: on Nextcloud's paths, some fields skip the rules that refused
|
||||||
ordinary Nextcloud requests there, and its WebDAV methods are allowed.
|
ordinary Nextcloud requests there, and its WebDAV methods are allowed.
|
||||||
`internal/coreruleset/apps/nextcloud.conf` lists each rule and the request it
|
`internal/coreruleset/nextcloud.conf` lists each rule and the request it
|
||||||
refused.
|
refused.
|
||||||
|
|
||||||
The sync client and the Files app use WebDAV on these paths:
|
The sync client and the Files app use WebDAV on these paths:
|
||||||
@@ -1621,8 +1621,7 @@ The sync client and the Files app use WebDAV on these paths:
|
|||||||
- `/remote.php/webdav/`;
|
- `/remote.php/webdav/`;
|
||||||
- `/public.php/webdav/`.
|
- `/public.php/webdav/`.
|
||||||
|
|
||||||
These methods are allowed there, besides `PUT`, `PATCH` and `DELETE`, which
|
These methods are allowed there, besides those of the first change:
|
||||||
every app gets:
|
|
||||||
|
|
||||||
- `PROPFIND` and `PROPPATCH`;
|
- `PROPFIND` and `PROPPATCH`;
|
||||||
- `MKCOL`;
|
- `MKCOL`;
|
||||||
@@ -1766,7 +1765,7 @@ gets gitea's fixes too. Forgejo's own fixes hold only what Forgejo sends and
|
|||||||
gitea does not.
|
gitea does not.
|
||||||
|
|
||||||
On Forgejo's paths, some fields skip the rules that refused ordinary Forgejo
|
On Forgejo's paths, some fields skip the rules that refused ordinary Forgejo
|
||||||
requests there. `internal/coreruleset/apps/forgejo.conf` lists each rule and the
|
requests there. `internal/coreruleset/forgejo.conf` lists each rule and the
|
||||||
request it refused.
|
request it refused.
|
||||||
|
|
||||||
`q` skips rules on some of Forgejo's searches. The searches are:
|
`q` skips rules on some of Forgejo's searches. The searches are:
|
||||||
@@ -1812,10 +1811,10 @@ The Core Rule Set reads the request without the `flash` cookie, in which Forgejo
|
|||||||
puts what gitea puts in `gitea_flash` (gitea's second fix).
|
puts what gitea puts in `gitea_flash` (gitea's second fix).
|
||||||
|
|
||||||
Ghost's fixes: on Ghost's paths, some fields skip the rules that refused
|
Ghost's fixes: on Ghost's paths, some fields skip the rules that refused
|
||||||
ordinary Ghost requests there. `internal/coreruleset/apps/ghost.conf` lists each
|
ordinary Ghost requests there. `internal/coreruleset/ghost.conf` lists each rule
|
||||||
rule and the request it refused. Ghost's admin and integrations call the Admin
|
and the request it refused. Ghost's admin and integrations call the Admin API
|
||||||
API under `/ghost/api/admin/`. Themes and Ghost's scripts call the Content API
|
under `/ghost/api/admin/`. Themes and Ghost's scripts call the Content API under
|
||||||
under `/ghost/api/content/`, and members call `/members/api/`. The admin calls
|
`/ghost/api/content/`, and members call `/members/api/`. The admin calls
|
||||||
ActivityPub, a service of its own, under `/.ghost/activitypub/`.
|
ActivityPub, a service of its own, under `/.ghost/activitypub/`.
|
||||||
|
|
||||||
Eight rules refused words that name commands and files, such as `python3`,
|
Eight rules refused words that name commands and files, such as `python3`,
|
||||||
@@ -1953,9 +1952,9 @@ That name holds `.profile`, a file on the list of system files, so the name
|
|||||||
skips 930120 when a profile is saved.
|
skips 930120 when a profile is saved.
|
||||||
|
|
||||||
The passwords skip each rule that random passwords set off. The numbers are in
|
The passwords skip each rule that random passwords set off. The numbers are in
|
||||||
`internal/coreruleset/apps/ghost.conf`. They are `password` when signing in, and
|
`internal/coreruleset/ghost.conf`. They are `password` when signing in, and the
|
||||||
the passwords sent when creating the first staff user, accepting an invitation,
|
passwords sent when creating the first staff user, accepting an invitation, and
|
||||||
and resetting or changing a password. `' OR '1'='1`, `../../etc/passwd`,
|
resetting or changing a password. `' OR '1'='1`, `../../etc/passwd`,
|
||||||
`; cat /etc/passwd` and `<img src=x onerror=x()>` sent as a password reach
|
`; cat /etc/passwd` and `<img src=x onerror=x()>` sent as a password reach
|
||||||
Ghost; `<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
|
Ghost; `<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
|
||||||
|
|
||||||
@@ -2431,9 +2430,8 @@ but the last below carries `X-Sww-Disposition`.
|
|||||||
carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the
|
carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the
|
||||||
request it passes on. A header whose value is unknown is left out.
|
request it passes on. A header whose value is unknown is left out.
|
||||||
- `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to
|
- `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to
|
||||||
get. `rproxy` gives them to the visitor, less this header; a `403` or `429`
|
get. `rproxy` gives them to the visitor, less this header. The status is one
|
||||||
gets `rproxy`'s own page as its body, linking `SWW_RPROXY_CONTACT_EMAIL` when
|
of:
|
||||||
set. The status is one of:
|
|
||||||
- `SWW_WAF_BAN_RESPONSE`, for a refusal;
|
- `SWW_WAF_BAN_RESPONSE`, for a refusal;
|
||||||
- `403`, for a `block` rule or the Core Rule Set;
|
- `403`, for a `block` rule or the Core Rule Set;
|
||||||
- `414` or `431`, for a request over the header limits;
|
- `414` or `431`, for a request over the header limits;
|
||||||
|
|||||||
@@ -1,217 +0,0 @@
|
|||||||
services:
|
|
||||||
exampleapp:
|
|
||||||
# Your app, built from the Dockerfile in this directory, listening
|
|
||||||
# on port 8081. Publish no port of it: it is reached through rproxy.
|
|
||||||
build: .
|
|
||||||
|
|
||||||
rproxy:
|
|
||||||
image: git.eeqj.de/sneak/smallwebwaf:latest
|
|
||||||
environment:
|
|
||||||
SWW_RPROXY_WAF_BACKEND: http://waf:8080
|
|
||||||
SWW_RPROXY_UPSTREAM_URL: http://exampleapp:8081
|
|
||||||
|
|
||||||
# The address the refusal pages link to.
|
|
||||||
# SWW_RPROXY_CONTACT_EMAIL: ops@example.com
|
|
||||||
|
|
||||||
# The netblocks whose X-Forwarded-For rproxy believes. By default
|
|
||||||
# the private ranges, which cover a TLS-terminating proxy on the
|
|
||||||
# Docker network. A proxy elsewhere needs its address added; a
|
|
||||||
# list you give replaces the default, so repeat the private
|
|
||||||
# ranges if they are still wanted. Set but empty, rproxy believes
|
|
||||||
# no forwarded header.
|
|
||||||
# SWW_RPROXY_TRUSTED_PROXIES: 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,203.0.113.5,2001:db8:1::/64
|
|
||||||
|
|
||||||
# By default rproxy passes a request to the app while waf gives
|
|
||||||
# no answer. Uncomment this to answer 503 instead.
|
|
||||||
# SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION: deny
|
|
||||||
|
|
||||||
# The largest request body (default 100M) and response body
|
|
||||||
# (default 1G): a size with K, M or G, or `off`. Raise them for
|
|
||||||
# uploads and downloads larger than that, such as git pushes and
|
|
||||||
# clones or image layers.
|
|
||||||
# SWW_RPROXY_MAX_REQUEST_SIZE: 2G
|
|
||||||
# SWW_RPROXY_MAX_RESPONSE_SIZE: 4G
|
|
||||||
|
|
||||||
# How long a request may take to reach the app (default 60000,
|
|
||||||
# one minute) and how long its answer may take to reach the
|
|
||||||
# visitor (default 1800000, 30 minutes), in milliseconds or
|
|
||||||
# `off`. Raise the first for slow uploads, the second for long
|
|
||||||
# downloads.
|
|
||||||
# SWW_RPROXY_MAX_REQUEST_TIME_MS: "600000"
|
|
||||||
# SWW_RPROXY_MAX_RESPONSE_TIME_MS: "7200000"
|
|
||||||
|
|
||||||
# traefik terminates TLS and forwards to rproxy:8080 over its own
|
|
||||||
# network, which the `networks` block at the end of this file names.
|
|
||||||
# No port is published.
|
|
||||||
networks:
|
|
||||||
- default
|
|
||||||
- traefik
|
|
||||||
labels:
|
|
||||||
traefik.enable: "true"
|
|
||||||
# rproxy is on two networks; traefik reaches it on this one.
|
|
||||||
traefik.docker.network: traefik
|
|
||||||
traefik.http.routers.exampleapp.rule: Host(`app.example.com`)
|
|
||||||
traefik.http.routers.exampleapp.entrypoints: websecure
|
|
||||||
traefik.http.routers.exampleapp.tls.certresolver: letsencrypt
|
|
||||||
traefik.http.services.exampleapp.loadbalancer.server.port: "8080"
|
|
||||||
|
|
||||||
waf:
|
|
||||||
image: git.eeqj.de/sneak/smallwebwaf:latest
|
|
||||||
environment:
|
|
||||||
SWW_COMMAND: waf
|
|
||||||
|
|
||||||
# `observe` refuses nothing and makes no ban; each log line says
|
|
||||||
# what `enforce` (the default) would have done. Start here to
|
|
||||||
# see what the defaults would refuse on your site.
|
|
||||||
# SWW_WAF_MODE: observe
|
|
||||||
|
|
||||||
# waf's own messages below `info` (the default), while finding
|
|
||||||
# out why something is refused.
|
|
||||||
# SWW_WAF_LOG_LEVEL: debug
|
|
||||||
|
|
||||||
# Netblocks whose clients skip every check, such as monitoring.
|
|
||||||
# SWW_WAF_ALLOW_NETS: 192.0.2.10,2001:db8:2::/48
|
|
||||||
|
|
||||||
# Netblocks whose clients are refused.
|
|
||||||
# SWW_WAF_DENY_NETS: 198.51.100.128/25,2001:db8:3::/48
|
|
||||||
|
|
||||||
# Netblocks exempt from rate limiting and the byte limits, such
|
|
||||||
# as the office and an automation host that calls an API. Their
|
|
||||||
# requests are still checked for attacks.
|
|
||||||
# SWW_WAF_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7
|
|
||||||
|
|
||||||
# Refuse every request from a country, by its two-letter code.
|
|
||||||
# This refuses North Korea, Cuba, Russia and Iran. A client the
|
|
||||||
# lookup cannot place gets through; see SWW_WAF_LOOKUP_SOURCE.
|
|
||||||
# SWW_WAF_TAG_DENY: country=KP,country=CU,country=RU,country=IR
|
|
||||||
|
|
||||||
# Or from a network, by its AS number: here two hosting networks
|
|
||||||
# that send only scrapers, M247 (AS9009) and WorldStream
|
|
||||||
# (AS49981). Countries and networks go in the one list together.
|
|
||||||
# SWW_WAF_TAG_DENY: asn=9009,asn=49981,country=KP,country=RU
|
|
||||||
|
|
||||||
# Or let only these countries through; a client the lookup
|
|
||||||
# cannot place is refused too.
|
|
||||||
# SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES: us,de
|
|
||||||
|
|
||||||
# Address lists that each tag their clients' requests, as
|
|
||||||
# name=URL pairs; this one tags Tor exit nodes. A tag is refused
|
|
||||||
# by adding its name to SWW_WAF_TAG_DENY, such as
|
|
||||||
# `torexit,country=KP`, or given a share of the limits in
|
|
||||||
# SWW_WAF_TAG_LIMIT_PERCENT, such as `torexit:25`.
|
|
||||||
# SWW_WAF_TAG_LISTS: torexit=https://check.torproject.org/torbulkexitlist
|
|
||||||
|
|
||||||
# A percentage, from 1 to 99, of every limit for a tag: here
|
|
||||||
# half for DigitalOcean (AS14061), a quarter for OVH (AS16276)
|
|
||||||
# and a quarter for China. The lowest of a client's tags applies.
|
|
||||||
# SWW_WAF_TAG_LIMIT_PERCENT: asn=14061:50,asn=16276:25,country=CN:25
|
|
||||||
|
|
||||||
# The most requests a client may make in a minute, an hour and a
|
|
||||||
# day (defaults 120, 1500 and 6000), a count or `off`; the next
|
|
||||||
# one bans it. Lower them for a small site, raise them for a
|
|
||||||
# busy API.
|
|
||||||
# SWW_WAF_RATE_LIMIT_PER_MINUTE: "60"
|
|
||||||
# SWW_WAF_RATE_LIMIT_PER_HOUR: "600"
|
|
||||||
# SWW_WAF_RATE_LIMIT_PER_DAY: "3000"
|
|
||||||
|
|
||||||
# Path prefixes exempt from rate limiting and the byte limits,
|
|
||||||
# such as static assets that one page loads many of.
|
|
||||||
# SWW_WAF_RATE_LIMIT_EXEMPT_PATHS: /assets/,/static/
|
|
||||||
|
|
||||||
# Paths only scanners ask for; one request bans the client. Pick
|
|
||||||
# paths your app never serves.
|
|
||||||
# SWW_WAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php,/wp-admin/
|
|
||||||
|
|
||||||
# How a refused client is answered: 403 (the default), 429 or
|
|
||||||
# `close`, which drops the connection.
|
|
||||||
# SWW_WAF_BAN_RESPONSE: "429"
|
|
||||||
|
|
||||||
# The ban for a first broken limit (default 1h) and for a first
|
|
||||||
# clear sign of attack (default 7d): a duration with m, h or d.
|
|
||||||
# A repeat within SWW_WAF_LIMIT_BAN_REPEAT_WINDOW (default 24h)
|
|
||||||
# gets a ban three times as long.
|
|
||||||
# SWW_WAF_LIMIT_BAN_DURATION: 30m
|
|
||||||
# SWW_WAF_ATTACK_BAN_DURATION: 30d
|
|
||||||
|
|
||||||
# The Core Rule Set, which finds attacks in requests. `block`
|
|
||||||
# (the default) refuses a match, `detect` only logs and alerts
|
|
||||||
# it, `off` runs no rule. Use `detect` while finding out what it
|
|
||||||
# would refuse of your app's ordinary requests.
|
|
||||||
# SWW_WAF_CORE_RULE_SET_MODE: detect
|
|
||||||
|
|
||||||
# Path prefixes the Core Rule Set does not inspect, for a part of
|
|
||||||
# the app that it keeps matching, such as an editor that posts
|
|
||||||
# HTML or code.
|
|
||||||
# SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS: /api/editor/,/admin/
|
|
||||||
|
|
||||||
# The apps each hostname serves, as hostname:app pairs separated
|
|
||||||
# by semicolons, so that each hostname gets its own apps' fixes
|
|
||||||
# to the attack detection and no other's. "Attack detection" in
|
|
||||||
# the README lists the apps.
|
|
||||||
# SWW_WAF_APPS: git.example.com:gitea;blog.example.com:wordpress
|
|
||||||
|
|
||||||
# Blocklists of addresses and netblocks, text files fetched once
|
|
||||||
# a day, whose clients are refused.
|
|
||||||
# SWW_WAF_BLOCKLIST_URLS: https://www.spamhaus.org/drop/drop.txt,https://www.spamhaus.org/drop/dropv6.txt
|
|
||||||
|
|
||||||
# With the key of an AbuseIPDB account, a client that has
|
|
||||||
# committed an offence is checked there in the background; one
|
|
||||||
# scored 75 or more (SWW_WAF_ABUSEIPDB_MIN_SCORE) gets what
|
|
||||||
# SWW_WAF_DNSBL_ABUSEIPDB_ACTION says: a quarter of every limit
|
|
||||||
# by default, or `deny`. At most 900 checks a day
|
|
||||||
# (SWW_WAF_ABUSEIPDB_DAILY_BUDGET), under the free plan's 1000.
|
|
||||||
# SWW_WAF_ABUSEIPDB_KEY: api-key-.....
|
|
||||||
# SWW_WAF_DNSBL_ABUSEIPDB_ACTION: deny
|
|
||||||
|
|
||||||
# By default waf looks each client's address up at GeoJS, a web
|
|
||||||
# service, for its country and AS number. Set `off`, it makes no
|
|
||||||
# external requests for lookups. Every setting at its default
|
|
||||||
# works without them; one that needs a lookup, such as a country
|
|
||||||
# or ASN setting above, then stops the start. Set `file`, it
|
|
||||||
# reads an IPinfo Lite database instead, mounted by its
|
|
||||||
# directory as under `volumes` below, so that a replaced file is
|
|
||||||
# seen.
|
|
||||||
# SWW_WAF_LOOKUP_SOURCE: "off"
|
|
||||||
# SWW_WAF_LOOKUP_SOURCE: file
|
|
||||||
# SWW_WAF_LOOKUP_DB_PATH: /var/lib/ipinfo/ipinfo_lite.mmdb
|
|
||||||
|
|
||||||
# Alerts about bans, attacks and failures, to an ntfy topic, a
|
|
||||||
# Slack incoming webhook or any webhook. The ntfy token is the
|
|
||||||
# access token ntfy issued for the topic.
|
|
||||||
# SWW_WAF_ALERT_NTFY_URL: https://ntfy.sh/my-alerts
|
|
||||||
# SWW_WAF_ALERT_NTFY_TOKEN: hunter2
|
|
||||||
# SWW_WAF_ALERT_SLACK_WEBHOOK_URL: https://hooks.slack.com/services/...
|
|
||||||
|
|
||||||
# Bearer tokens you make up, at least 32 characters each. The
|
|
||||||
# admin token switches on the ban and client endpoints, through
|
|
||||||
# which a ban is lifted, your own included, as
|
|
||||||
# docs/admin-endpoints.md describes. The metrics token switches
|
|
||||||
# on /_sww/metrics.
|
|
||||||
# SWW_WAF_ADMIN_TOKEN: hunter2
|
|
||||||
# SWW_WAF_METRICS_TOKEN: hunter2
|
|
||||||
|
|
||||||
# Any setting can be read from a file instead, by adding _FILE
|
|
||||||
# to its name: the file's contents are the value, less one
|
|
||||||
# newline at the end. That keeps a secret out of this file and
|
|
||||||
# out of `docker inspect`. Mount the files as under `volumes`
|
|
||||||
# below, and set each setting in one form only.
|
|
||||||
# SWW_WAF_ALERT_NTFY_TOKEN_FILE: /etc/sww/tokens/ntfy
|
|
||||||
# SWW_WAF_ADMIN_TOKEN_FILE: /etc/sww/tokens/admin
|
|
||||||
# SWW_WAF_METRICS_TOKEN_FILE: /etc/sww/tokens/metrics
|
|
||||||
# SWW_WAF_ABUSEIPDB_KEY_FILE: /etc/sww/tokens/abuseipdb
|
|
||||||
volumes:
|
|
||||||
# waf's state files: its clients and bans, kept across restarts.
|
|
||||||
- waf-state:/var/lib/sww
|
|
||||||
# Token files, each owned by uid 65532 (the sww user) with mode
|
|
||||||
# 0400, for the _FILE settings above.
|
|
||||||
# - /srv/sww/tokens:/etc/sww/tokens:ro
|
|
||||||
# The IPinfo Lite database, for SWW_WAF_LOOKUP_SOURCE=file.
|
|
||||||
# - /srv/ipinfo:/var/lib/ipinfo:ro
|
|
||||||
|
|
||||||
volumes:
|
|
||||||
waf-state:
|
|
||||||
|
|
||||||
# traefik's network, made outside this file.
|
|
||||||
networks:
|
|
||||||
traefik:
|
|
||||||
external: true
|
|
||||||
@@ -483,8 +483,6 @@ start, and an invalid value stops the start. Any can be given as a file, as a
|
|||||||
answer `rproxy` gives itself always has at least 10 seconds to be sent, and so
|
answer `rproxy` gives itself always has at least 10 seconds to be sent, and so
|
||||||
do the status and headers of the app's answer. A time in milliseconds or
|
do the status and headers of the app's answer. A time in milliseconds or
|
||||||
`off`.
|
`off`.
|
||||||
- `SWW_RPROXY_CONTACT_EMAIL` (default unset): the email address the refusal
|
|
||||||
pages link to.
|
|
||||||
- `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as
|
- `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as
|
||||||
`:9000`.
|
`:9000`.
|
||||||
- `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An
|
- `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An
|
||||||
|
|||||||
@@ -46,9 +46,6 @@ type RProxy struct {
|
|||||||
// (SWW_RPROXY_MAX_RESPONSE_TIME_MS).
|
// (SWW_RPROXY_MAX_RESPONSE_TIME_MS).
|
||||||
MaxRequestTime time.Duration
|
MaxRequestTime time.Duration
|
||||||
MaxResponseTime time.Duration
|
MaxResponseTime time.Duration
|
||||||
// ContactEmail is the address the pages a refused visitor gets link to,
|
|
||||||
// or "" for no link (SWW_RPROXY_CONTACT_EMAIL). It is taken as given.
|
|
||||||
ContactEmail string
|
|
||||||
|
|
||||||
// settings are the values read, as given or by default, for the log line
|
// settings are the values read, as given or by default, for the log line
|
||||||
// at start.
|
// at start.
|
||||||
@@ -111,7 +108,6 @@ func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
|
|||||||
"SWW_RPROXY_MAX_RESPONSE_TIME_MS",
|
"SWW_RPROXY_MAX_RESPONSE_TIME_MS",
|
||||||
"1800000",
|
"1800000",
|
||||||
),
|
),
|
||||||
ContactEmail: env.value("SWW_RPROXY_CONTACT_EMAIL", ""),
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if env.err != nil {
|
if env.err != nil {
|
||||||
|
|||||||
@@ -25,7 +25,6 @@ const (
|
|||||||
rproxyMaxResponse = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
rproxyMaxResponse = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
||||||
rproxyRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
rproxyRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
||||||
rproxyAnswerTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
rproxyAnswerTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
||||||
rproxyContactEmail = "SWW_RPROXY_CONTACT_EMAIL"
|
|
||||||
backendURL = "http://waf:8080"
|
backendURL = "http://waf:8080"
|
||||||
rproxyNoAnswerAddr = ":8080"
|
rproxyNoAnswerAddr = ":8080"
|
||||||
rproxyNoAnswerApp = "http://127.0.0.1:8081"
|
rproxyNoAnswerApp = "http://127.0.0.1:8081"
|
||||||
@@ -251,7 +250,6 @@ func TestRProxySettingsAreLoggedByName(t *testing.T) {
|
|||||||
rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, wafBackendTimeout,
|
rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, wafBackendTimeout,
|
||||||
rproxyNoAnswer,
|
rproxyNoAnswer,
|
||||||
rproxyMaxRequest, rproxyMaxResponse, rproxyRequestTime, rproxyAnswerTime,
|
rproxyMaxRequest, rproxyMaxResponse, rproxyRequestTime, rproxyAnswerTime,
|
||||||
rproxyContactEmail,
|
|
||||||
} {
|
} {
|
||||||
if !slices.Contains(names, name) {
|
if !slices.Contains(names, name) {
|
||||||
t.Errorf("%s is not logged, only %v", name, names)
|
t.Errorf("%s is not logged, only %v", name, names)
|
||||||
|
|||||||
@@ -12,10 +12,11 @@
|
|||||||
package coreruleset
|
package coreruleset
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"embed"
|
_ "embed"
|
||||||
"errors"
|
"errors"
|
||||||
"fmt"
|
"fmt"
|
||||||
"io"
|
"io"
|
||||||
|
"maps"
|
||||||
"net"
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
@@ -34,7 +35,7 @@ import (
|
|||||||
// directives are the Core Rule Set as sww runs it, with the
|
// directives are the Core Rule Set as sww runs it, with the
|
||||||
// paranoia level for %d, bodyDirectives for the first %s while
|
// paranoia level for %d, bodyDirectives for the first %s while
|
||||||
// SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set, the exclusions file for
|
// SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set, the exclusions file for
|
||||||
// the second, and the fixes of every app in appFixes for the third.
|
// the second, and the directives of every app in apps for the third.
|
||||||
// Each rule sww adds has an id from 900000 to 900999, the ids the
|
// Each rule sww adds has an id from 900000 to 900999, the ids the
|
||||||
// Core Rule Set keeps for the rules that set it up, which
|
// Core Rule Set keeps for the rules that set it up, which
|
||||||
// SWW_WAF_CORE_RULE_SET_DISABLED_RULES and the exclusions file refuse, so that no
|
// SWW_WAF_CORE_RULE_SET_DISABLED_RULES and the exclusions file refuse, so that no
|
||||||
@@ -53,12 +54,13 @@ Include @crs-setup.conf.example
|
|||||||
SecAction "id:900000,phase:1,pass,nolog,\
|
SecAction "id:900000,phase:1,pass,nolog,\
|
||||||
setvar:tx.blocking_paranoia_level=%d"
|
setvar:tx.blocking_paranoia_level=%d"
|
||||||
|
|
||||||
# PUT, PATCH and DELETE are allowed besides GET, HEAD, POST and OPTIONS.
|
# The first change: PUT, PATCH and DELETE are allowed besides GET, HEAD,
|
||||||
|
# POST and OPTIONS.
|
||||||
SecAction "id:900200,phase:1,pass,nolog,\
|
SecAction "id:900200,phase:1,pass,nolog,\
|
||||||
setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE'"
|
setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE'"
|
||||||
|
|
||||||
# Expect and Content-Encoding are taken off the Core Rule Set's list of the
|
# The second: Expect and Content-Encoding are taken off the Core Rule Set's
|
||||||
# headers it refuses. Content-Encoding goes back on it for a
|
# list of the headers it refuses. Content-Encoding goes back on it for a
|
||||||
# body the Core Rule Set reads (900260 in bodyDirectives).
|
# body the Core Rule Set reads (900260 in bodyDirectives).
|
||||||
SecAction "id:900250,phase:1,pass,nolog,\
|
SecAction "id:900250,phase:1,pass,nolog,\
|
||||||
setvar:'tx.restricted_headers_basic=/proxy/ /lock-token/ /content-range/ \
|
setvar:'tx.restricted_headers_basic=/proxy/ /lock-token/ /content-range/ \
|
||||||
@@ -84,17 +86,18 @@ SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
|
|||||||
# The apps' fixes come before the rules for the same reason.
|
# The apps' fixes come before the rules for the same reason.
|
||||||
%s
|
%s
|
||||||
|
|
||||||
# Only the rules for requests are loaded, and no response is inspected.
|
# The fifth: only the rules for requests are loaded, and no response is
|
||||||
|
# inspected.
|
||||||
Include @owasp_crs/REQUEST-*.conf
|
Include @owasp_crs/REQUEST-*.conf
|
||||||
|
|
||||||
# redirect_uri is not checked for a URL naming an IP address or
|
# The third: redirect_uri is not checked for a URL naming an IP address or
|
||||||
# localhost. Coraza matches a parameter name here without regard to case.
|
# localhost. Coraza matches a parameter name here without regard to case.
|
||||||
# ARGS holds the fields of a form data or multipart body Coraza reads as
|
# ARGS holds the fields of a form data or multipart body Coraza reads as
|
||||||
# well as the query parameters, so a field of this name is left out too.
|
# well as the query parameters, so a field of this name is left out too.
|
||||||
SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri"
|
SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri"
|
||||||
SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri"
|
SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri"
|
||||||
|
|
||||||
# Referer is not checked for a Unix command without arguments
|
# The fourth: Referer is not checked for a Unix command without arguments
|
||||||
# or for Java starting a process.
|
# or for Java starting a process.
|
||||||
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
|
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
|
||||||
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
|
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
|
||||||
@@ -119,9 +122,9 @@ SecRule REQUEST_HEADERS:Content-Type \
|
|||||||
"@rx ^(?:application|text)/(?:[a-z0-9.-]+[+])?json" \
|
"@rx ^(?:application|text)/(?:[a-z0-9.-]+[+])?json" \
|
||||||
"id:900420,phase:1,pass,nolog,t:none,t:lowercase,ctl:requestBodyProcessor=JSON"
|
"id:900420,phase:1,pass,nolog,t:none,t:lowercase,ctl:requestBodyProcessor=JSON"
|
||||||
|
|
||||||
# Content-Encoding goes back on the list of the headers the Core Rule Set
|
# The rest of the second change: Content-Encoding is refused again on a
|
||||||
# refuses (900250 in directives) for a body of a kind it reads, since a
|
# body of a kind the Core Rule Set reads, since a compressed body cannot be
|
||||||
# compressed body cannot be inspected.
|
# inspected.
|
||||||
SecRule REQBODY_PROCESSOR "@rx ^(?:URLENCODED|MULTIPART|JSON|XML)$" \
|
SecRule REQBODY_PROCESSOR "@rx ^(?:URLENCODED|MULTIPART|JSON|XML)$" \
|
||||||
"id:900260,phase:1,pass,nolog,\
|
"id:900260,phase:1,pass,nolog,\
|
||||||
setvar:'tx.restricted_headers_basic=%%{tx.restricted_headers_basic} \
|
setvar:'tx.restricted_headers_basic=%%{tx.restricted_headers_basic} \
|
||||||
@@ -142,19 +145,15 @@ SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
|
|||||||
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
||||||
`
|
`
|
||||||
|
|
||||||
// appFixes holds the fixes of each app whose ordinary requests the Core
|
// app is an app whose ordinary requests the Core Rule Set would refuse,
|
||||||
// Rule Set would refuse, in a file named for the app by the name
|
// with the fixes that let them pass.
|
||||||
// SWW_WAF_APPS gives it, such as apps/gitea.conf. The fixes are Coraza
|
|
||||||
// exclusions, loaded before the Core Rule Set's rules. Their first rule
|
|
||||||
// skips to the marker they end with, unless Inspect sets the transaction
|
|
||||||
// variable named for the app, such as app_gitea. The skip reaches only
|
|
||||||
// phase 1, so a rule of a later phase must check that variable itself.
|
|
||||||
//
|
|
||||||
//go:embed apps/*.conf
|
|
||||||
var appFixes embed.FS
|
|
||||||
|
|
||||||
// app is what an app's file in appFixes cannot say about it.
|
|
||||||
type app struct {
|
type app struct {
|
||||||
|
// directives are Coraza exclusions, loaded before the Core Rule Set's
|
||||||
|
// rules. Their first rule skips to the marker they end with, unless
|
||||||
|
// Inspect sets the transaction variable named for the app, such as
|
||||||
|
// app_gitea. The skip reaches only phase 1, so a rule of a later phase
|
||||||
|
// must check that variable itself.
|
||||||
|
directives string
|
||||||
// cookies are the cookies the Core Rule Set reads a request without.
|
// cookies are the cookies the Core Rule Set reads a request without.
|
||||||
cookies []string
|
cookies []string
|
||||||
// forkOf names the app this one is a fork of, "" for none. A hostname
|
// forkOf names the app this one is a fork of, "" for none. A hostname
|
||||||
@@ -162,34 +161,60 @@ type app struct {
|
|||||||
forkOf string
|
forkOf string
|
||||||
}
|
}
|
||||||
|
|
||||||
// apps are the apps that need more than their file in appFixes, by the
|
//go:embed gitea.conf
|
||||||
// name SWW_WAF_APPS gives each. flash is Forgejo's name for gitea_flash.
|
var giteaDirectives string
|
||||||
|
|
||||||
|
//go:embed wordpress.conf
|
||||||
|
var wordpressDirectives string
|
||||||
|
|
||||||
|
//go:embed drupal.conf
|
||||||
|
var drupalDirectives string
|
||||||
|
|
||||||
|
//go:embed mediawiki.conf
|
||||||
|
var mediawikiDirectives string
|
||||||
|
|
||||||
|
//go:embed mastodon.conf
|
||||||
|
var mastodonDirectives string
|
||||||
|
|
||||||
|
//go:embed vaultwarden.conf
|
||||||
|
var vaultwardenDirectives string
|
||||||
|
|
||||||
|
//go:embed nextcloud.conf
|
||||||
|
var nextcloudDirectives string
|
||||||
|
|
||||||
|
//go:embed forgejo.conf
|
||||||
|
var forgejoDirectives string
|
||||||
|
|
||||||
|
//go:embed ghost.conf
|
||||||
|
var ghostDirectives string
|
||||||
|
|
||||||
|
// apps are the apps whose fixes sww has, by the name SWW_WAF_APPS gives each.
|
||||||
|
// flash is Forgejo's name for gitea_flash.
|
||||||
//
|
//
|
||||||
//nolint:gochecknoglobals // a constant cannot be a map
|
//nolint:gochecknoglobals // a constant cannot be a map
|
||||||
var apps = map[string]app{
|
var apps = map[string]app{
|
||||||
|
"drupal": {directives: drupalDirectives},
|
||||||
"forgejo": {
|
"forgejo": {
|
||||||
cookies: []string{"flash"},
|
directives: forgejoDirectives,
|
||||||
forkOf: "gitea",
|
cookies: []string{"flash"},
|
||||||
|
forkOf: "gitea",
|
||||||
},
|
},
|
||||||
|
"ghost": {directives: ghostDirectives},
|
||||||
"gitea": {
|
"gitea": {
|
||||||
cookies: []string{"gitea_flash", "redirect_to"},
|
directives: giteaDirectives,
|
||||||
|
cookies: []string{"gitea_flash", "redirect_to"},
|
||||||
},
|
},
|
||||||
|
"mastodon": {directives: mastodonDirectives},
|
||||||
|
"mediawiki": {directives: mediawikiDirectives},
|
||||||
|
"nextcloud": {directives: nextcloudDirectives},
|
||||||
|
"vaultwarden": {directives: vaultwardenDirectives},
|
||||||
|
"wordpress": {directives: wordpressDirectives},
|
||||||
}
|
}
|
||||||
|
|
||||||
// AppNames returns the names of the apps whose fixes sww has, in
|
// AppNames returns the names of the apps whose fixes sww has, in
|
||||||
// alphabetical order.
|
// alphabetical order.
|
||||||
func AppNames() []string {
|
func AppNames() []string {
|
||||||
// An embedded directory always reads.
|
return slices.Sorted(maps.Keys(apps))
|
||||||
files, _ := appFixes.ReadDir("apps")
|
|
||||||
|
|
||||||
names := make([]string, len(files))
|
|
||||||
for i, file := range files {
|
|
||||||
names[i] = strings.TrimSuffix(file.Name(), ".conf")
|
|
||||||
}
|
|
||||||
|
|
||||||
slices.Sort(names)
|
|
||||||
|
|
||||||
return names
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// Params are what New needs.
|
// Params are what New needs.
|
||||||
@@ -348,11 +373,8 @@ func New(params Params) (*CoreRuleSet, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
var fixes strings.Builder
|
var fixes strings.Builder
|
||||||
|
|
||||||
for _, name := range AppNames() {
|
for _, name := range AppNames() {
|
||||||
// An embedded file always reads.
|
fixes.WriteString(apps[name].directives + "\n")
|
||||||
file, _ := appFixes.ReadFile("apps/" + name + ".conf")
|
|
||||||
fixes.WriteString(string(file) + "\n")
|
|
||||||
}
|
}
|
||||||
|
|
||||||
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions,
|
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions,
|
||||||
|
|||||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -17,7 +17,7 @@ const (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// giteaSearches are searches of the pages gitea and Forgejo share whose q
|
// giteaSearches are searches of the pages gitea and Forgejo share whose q
|
||||||
// skips the lists of system files and command names, in gitea's fixes.
|
// skips the lists of system files and command names, in the eighth change.
|
||||||
// gitea serves its admin pages under /-/admin/ and Forgejo under /admin/.
|
// gitea serves its admin pages under /-/admin/ and Forgejo under /admin/.
|
||||||
//
|
//
|
||||||
//nolint:gochecknoglobals // a constant cannot be a list
|
//nolint:gochecknoglobals // a constant cannot be a list
|
||||||
@@ -222,8 +222,8 @@ func TestGiteaSearchExclusionsAreNarrow(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Under /admin/, where Forgejo serves its admin pages, only the hard
|
// Under /admin/, where Forgejo serves its admin pages, only the hard
|
||||||
// values are left out: Drupal's fixes let them through in q on every path
|
// values are left out: the tenth change lets them through in q on every
|
||||||
// there.
|
// path there, for Drupal.
|
||||||
for _, target := range []string{
|
for _, target := range []string{
|
||||||
"/admin/packages/delete?q=",
|
"/admin/packages/delete?q=",
|
||||||
"/admin/packages?other=",
|
"/admin/packages?other=",
|
||||||
|
|||||||
File renamed without changes.
File renamed without changes.
@@ -37,10 +37,9 @@
|
|||||||
SecRule &TX:app_nextcloud "@eq 0" \
|
SecRule &TX:app_nextcloud "@eq 0" \
|
||||||
"id:900090,phase:1,pass,nolog,skipAfter:END_NEXTCLOUD"
|
"id:900090,phase:1,pass,nolog,skipAfter:END_NEXTCLOUD"
|
||||||
|
|
||||||
# The sync client and the Files app use these WebDAV methods besides PUT,
|
# The sync client and the Files app use these WebDAV methods besides those
|
||||||
# PATCH and DELETE, which sww allows for every app, on /remote.php/dav/ and
|
# the first change allows, on /remote.php/dav/ and /public.php/dav/ and on the
|
||||||
# /public.php/dav/ and on the older /remote.php/webdav/ and /public.php/webdav/.
|
# older /remote.php/webdav/ and /public.php/webdav/. 911100 refused each:
|
||||||
# 911100 refused each:
|
|
||||||
# PROPFIND, PROPPATCH: listing a folder, marking a file a favourite
|
# PROPFIND, PROPPATCH: listing a folder, marking a file a favourite
|
||||||
# MKCOL: making a folder, and starting a chunked upload
|
# MKCOL: making a folder, and starting a chunked upload
|
||||||
# COPY, MOVE: copying, moving and renaming, and ending a chunked upload
|
# COPY, MOVE: copying, moving and renaming, and ending a chunked upload
|
||||||
File renamed without changes.
File renamed without changes.
@@ -1,71 +0,0 @@
|
|||||||
package rproxy
|
|
||||||
|
|
||||||
import (
|
|
||||||
"embed"
|
|
||||||
"html"
|
|
||||||
"net/http"
|
|
||||||
"strings"
|
|
||||||
)
|
|
||||||
|
|
||||||
// pageFiles are the pages a refused visitor gets, as HTML with nothing
|
|
||||||
// external: refused.html for a 403, ratelimited.html for a 429.
|
|
||||||
//
|
|
||||||
//go:embed refused.html ratelimited.html
|
|
||||||
var pageFiles embed.FS
|
|
||||||
|
|
||||||
// contactSentence is the sentence of each page that becomes a link to
|
|
||||||
// SWW_RPROXY_CONTACT_EMAIL when that is set.
|
|
||||||
const contactSentence = "contact the site's operator"
|
|
||||||
|
|
||||||
// htmlContentType is the Content-Type of the pages.
|
|
||||||
const htmlContentType = "text/html; charset=utf-8"
|
|
||||||
|
|
||||||
// blockPages are the pages a refused visitor gets, ready to send, made once
|
|
||||||
// at start.
|
|
||||||
type blockPages struct {
|
|
||||||
refused []byte
|
|
||||||
rateLimited []byte
|
|
||||||
}
|
|
||||||
|
|
||||||
// newBlockPages makes the pages. With contactEmail set, each page's
|
|
||||||
// contactSentence is a mailto link to it.
|
|
||||||
func newBlockPages(contactEmail string) blockPages {
|
|
||||||
return blockPages{
|
|
||||||
refused: readPage("refused.html", contactEmail),
|
|
||||||
rateLimited: readPage("ratelimited.html", contactEmail),
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// readPage reads the page in name from pageFiles, linking its
|
|
||||||
// contactSentence to contactEmail unless that is "". The files are part of
|
|
||||||
// the binary, so a missing one is a bug, and a panic.
|
|
||||||
func readPage(name, contactEmail string) []byte {
|
|
||||||
contents, err := pageFiles.ReadFile(name)
|
|
||||||
if err != nil {
|
|
||||||
panic(err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if contactEmail == "" {
|
|
||||||
return contents
|
|
||||||
}
|
|
||||||
|
|
||||||
link := `<a href="mailto:` + html.EscapeString(contactEmail) + `">` +
|
|
||||||
contactSentence + `</a>`
|
|
||||||
|
|
||||||
return []byte(
|
|
||||||
strings.Replace(string(contents), contactSentence, link, 1),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
// forStatus returns the page for a refusal with status, and whether there is
|
|
||||||
// one: the rate-limited page for a 429, the refused page for a 403.
|
|
||||||
func (p blockPages) forStatus(status int) ([]byte, bool) {
|
|
||||||
switch status {
|
|
||||||
case http.StatusForbidden:
|
|
||||||
return p.refused, true
|
|
||||||
case http.StatusTooManyRequests:
|
|
||||||
return p.rateLimited, true
|
|
||||||
default:
|
|
||||||
return nil, false
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -1,14 +1,6 @@
|
|||||||
package rproxy
|
package rproxy
|
||||||
|
|
||||||
import (
|
import "time"
|
||||||
"net/http"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
// HTTPServer is Go's server s holds, for tests that read its settings.
|
|
||||||
func (s *Server) HTTPServer() *http.Server {
|
|
||||||
return s.httpServer
|
|
||||||
}
|
|
||||||
|
|
||||||
// StopBreakerClock has the circuit breaker of s read at as the time, so that how
|
// StopBreakerClock has the circuit breaker of s read at as the time, so that how
|
||||||
// long requests take cannot matter to it. It is called before s serves.
|
// long requests take cannot matter to it. It is called before s serves.
|
||||||
|
|||||||
@@ -56,7 +56,7 @@ func TestServerHasTheFixedLimits(t *testing.T) {
|
|||||||
InstanceName: "test",
|
InstanceName: "test",
|
||||||
Level: slog.LevelInfo,
|
Level: slog.LevelInfo,
|
||||||
}),
|
}),
|
||||||
}).HTTPServer()
|
})
|
||||||
|
|
||||||
if server.Addr != ":8080" || server.MaxHeaderBytes != 28<<10 ||
|
if server.Addr != ":8080" || server.MaxHeaderBytes != 28<<10 ||
|
||||||
server.IdleTimeout != 2*time.Minute || server.ReadHeaderTimeout != time.Minute {
|
server.IdleTimeout != 2*time.Minute || server.ReadHeaderTimeout != time.Minute {
|
||||||
@@ -709,7 +709,7 @@ func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
|
|||||||
|
|
||||||
before := time.Now()
|
before := time.Now()
|
||||||
|
|
||||||
server.ServeHTTP(w, req)
|
server.Handler.ServeHTTP(w, req)
|
||||||
|
|
||||||
if w.Code != http.StatusNoContent {
|
if w.Code != http.StatusNoContent {
|
||||||
t.Errorf("status %d, want the app's 204", w.Code)
|
t.Errorf("status %d, want the app's 204", w.Code)
|
||||||
@@ -762,7 +762,7 @@ func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(
|
|||||||
|
|
||||||
before := time.Now()
|
before := time.Now()
|
||||||
|
|
||||||
server.ServeHTTP(w, req)
|
server.Handler.ServeHTTP(w, req)
|
||||||
|
|
||||||
if w.Code != http.StatusOK {
|
if w.Code != http.StatusOK {
|
||||||
t.Errorf("status %d, want the app's 200", w.Code)
|
t.Errorf("status %d, want the app's 200", w.Code)
|
||||||
@@ -815,7 +815,7 @@ func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
|
|||||||
|
|
||||||
before := time.Now()
|
before := time.Now()
|
||||||
|
|
||||||
server.ServeHTTP(w, req)
|
server.Handler.ServeHTTP(w, req)
|
||||||
|
|
||||||
if w.Code != tc.status {
|
if w.Code != tc.status {
|
||||||
t.Errorf("status %d, want %d", w.Code, tc.status)
|
t.Errorf("status %d, want %d", w.Code, tc.status)
|
||||||
@@ -856,7 +856,7 @@ func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(
|
|||||||
|
|
||||||
before := time.Now()
|
before := time.Now()
|
||||||
|
|
||||||
server.ServeHTTP(w, req)
|
server.Handler.ServeHTTP(w, req)
|
||||||
|
|
||||||
if w.writeDeadline.Before(before.Add(least)) {
|
if w.writeDeadline.Before(before.Add(least)) {
|
||||||
t.Errorf(
|
t.Errorf(
|
||||||
|
|||||||
@@ -583,7 +583,7 @@ func TestAQuestionNamesNoZone(t *testing.T) {
|
|||||||
req.RemoteAddr = tc.remoteAddr
|
req.RemoteAddr = tc.remoteAddr
|
||||||
req.Header.Set(forwardedForHeader, tc.forwardedFor)
|
req.Header.Set(forwardedForHeader, tc.forwardedFor)
|
||||||
|
|
||||||
server.ServeHTTP(httptest.NewRecorder(), req)
|
server.Handler.ServeHTTP(httptest.NewRecorder(), req)
|
||||||
|
|
||||||
asked := standIn.question(t)
|
asked := standIn.question(t)
|
||||||
if asked.header.Get(wafprotocol.ClientHeader) != tc.client ||
|
if asked.header.Get(wafprotocol.ClientHeader) != tc.client ||
|
||||||
|
|||||||
@@ -1,27 +0,0 @@
|
|||||||
<!doctype html>
|
|
||||||
<html lang="en">
|
|
||||||
<head>
|
|
||||||
<meta charset="utf-8">
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
||||||
<meta name="robots" content="noindex">
|
|
||||||
<title>Too many requests</title>
|
|
||||||
<style>
|
|
||||||
:root{color-scheme:light dark;--fg:#1a1a1a;--muted:#5c5c5c;--bg:#f6f6f4}
|
|
||||||
@media(prefers-color-scheme:dark){:root{--fg:#ebebe8;--muted:#9a9a96;--bg:#161616}}
|
|
||||||
html{background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif}
|
|
||||||
body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;padding:1.5rem}
|
|
||||||
main{max-width:34rem;width:100%}
|
|
||||||
p.code{margin:0 0 .5rem;color:var(--muted);font-size:.8rem;letter-spacing:.08em}
|
|
||||||
h1{margin:0 0 1rem;font-size:1.6rem;font-weight:600;line-height:1.25}
|
|
||||||
p{margin:0 0 1rem}
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<main>
|
|
||||||
<p class="code">429</p>
|
|
||||||
<h1>Too many requests</h1>
|
|
||||||
<p>Your address has sent more requests than this site allows in a short time. Wait a while, then try again.</p>
|
|
||||||
<p>If you think this is a mistake, contact the site's operator.</p>
|
|
||||||
</main>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
@@ -1,27 +0,0 @@
|
|||||||
<!doctype html>
|
|
||||||
<html lang="en">
|
|
||||||
<head>
|
|
||||||
<meta charset="utf-8">
|
|
||||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
|
||||||
<meta name="robots" content="noindex">
|
|
||||||
<title>Request refused</title>
|
|
||||||
<style>
|
|
||||||
:root{color-scheme:light dark;--fg:#1a1a1a;--muted:#5c5c5c;--bg:#f6f6f4}
|
|
||||||
@media(prefers-color-scheme:dark){:root{--fg:#ebebe8;--muted:#9a9a96;--bg:#161616}}
|
|
||||||
html{background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif}
|
|
||||||
body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;padding:1.5rem}
|
|
||||||
main{max-width:34rem;width:100%}
|
|
||||||
p.code{margin:0 0 .5rem;color:var(--muted);font-size:.8rem;letter-spacing:.08em}
|
|
||||||
h1{margin:0 0 1rem;font-size:1.6rem;font-weight:600;line-height:1.25}
|
|
||||||
p{margin:0 0 1rem}
|
|
||||||
</style>
|
|
||||||
</head>
|
|
||||||
<body>
|
|
||||||
<main>
|
|
||||||
<p class="code">403</p>
|
|
||||||
<h1>Request refused</h1>
|
|
||||||
<p>This site did not accept your request.</p>
|
|
||||||
<p>If you think this is a mistake, contact the site's operator.</p>
|
|
||||||
</main>
|
|
||||||
</body>
|
|
||||||
</html>
|
|
||||||
+24
-42
@@ -16,11 +16,11 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"log"
|
"log"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net"
|
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/url"
|
"net/url"
|
||||||
"slices"
|
"slices"
|
||||||
"strings"
|
"strings"
|
||||||
|
"sync"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
@@ -64,8 +64,9 @@ type Params struct {
|
|||||||
|
|
||||||
// Server is the server `sww rproxy` runs.
|
// Server is the server `sww rproxy` runs.
|
||||||
type Server struct {
|
type Server struct {
|
||||||
httpServer *http.Server
|
*http.Server
|
||||||
h *handler
|
|
||||||
|
h *handler
|
||||||
}
|
}
|
||||||
|
|
||||||
// New returns the server `sww rproxy` runs. Go's server itself refuses
|
// New returns the server `sww rproxy` runs. Go's server itself refuses
|
||||||
@@ -92,50 +93,30 @@ func New(params Params) *Server {
|
|||||||
Path: wafprotocol.ReportPath,
|
Path: wafprotocol.ReportPath,
|
||||||
}).String(),
|
}).String(),
|
||||||
breakerClock: time.Now,
|
breakerClock: time.Now,
|
||||||
pages: newBlockPages(params.Config.ContactEmail),
|
|
||||||
}
|
}
|
||||||
|
|
||||||
s := &Server{h: h}
|
return &Server{
|
||||||
s.httpServer = &http.Server{
|
Server: &http.Server{
|
||||||
Addr: params.Config.ListenAddr,
|
Addr: params.Config.ListenAddr,
|
||||||
Handler: s,
|
Handler: h,
|
||||||
ErrorLog: errorLog,
|
ErrorLog: errorLog,
|
||||||
ReadHeaderTimeout: params.Config.MaxRequestTime,
|
ReadHeaderTimeout: params.Config.MaxRequestTime,
|
||||||
IdleTimeout: idleTimeout,
|
IdleTimeout: idleTimeout,
|
||||||
MaxHeaderBytes: requestHeaderMaxBytes - goServerExtraHeaderBytes,
|
MaxHeaderBytes: requestHeaderMaxBytes - goServerExtraHeaderBytes,
|
||||||
|
},
|
||||||
|
h: h,
|
||||||
}
|
}
|
||||||
|
|
||||||
return s
|
|
||||||
}
|
|
||||||
|
|
||||||
// Serve serves requests on listener until Shutdown or Close, as Go's
|
|
||||||
// http.Server.Serve does.
|
|
||||||
func (s *Server) Serve(listener net.Listener) error {
|
|
||||||
return s.httpServer.Serve(listener)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Shutdown stops serving once the requests in progress have finished, or
|
|
||||||
// when ctx is done, as Go's http.Server.Shutdown does.
|
|
||||||
func (s *Server) Shutdown(ctx context.Context) error {
|
|
||||||
return s.httpServer.Shutdown(ctx)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Close stops serving at once and closes every connection, as Go's
|
|
||||||
// http.Server.Close does.
|
|
||||||
func (s *Server) Close() error {
|
|
||||||
return s.httpServer.Close()
|
|
||||||
}
|
|
||||||
|
|
||||||
// ServeHTTP handles one request from a visitor, as handler.ServeHTTP says.
|
|
||||||
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|
||||||
s.h.ServeHTTP(w, r)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// WaitForReports returns once the reports being sent have gone or failed,
|
// WaitForReports returns once the reports being sent have gone or failed,
|
||||||
// which SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS bounds. Any report after this call
|
// which SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS bounds. Any report after this call
|
||||||
// is dropped: that of a WebSocket that closes while rproxy stops, for one.
|
// is dropped: that of a WebSocket that closes while rproxy stops, for one.
|
||||||
func (s *Server) WaitForReports() {
|
func (s *Server) WaitForReports() {
|
||||||
s.h.reports.stopAndWait()
|
s.h.reportsMu.Lock()
|
||||||
|
s.h.reportsStopped = true
|
||||||
|
s.h.reportsMu.Unlock()
|
||||||
|
|
||||||
|
s.h.reports.Wait()
|
||||||
}
|
}
|
||||||
|
|
||||||
// handler is the proxy. It holds what every request shares; what belongs to
|
// handler is the proxy. It holds what every request shares; what belongs to
|
||||||
@@ -155,10 +136,11 @@ type handler struct {
|
|||||||
// the time.
|
// the time.
|
||||||
breaker breaker
|
breaker breaker
|
||||||
breakerClock func() time.Time
|
breakerClock func() time.Time
|
||||||
// reports are the goroutines sending reports to waf.
|
// reports waits for the reports being sent. reportsMu guards starting one
|
||||||
reports reportGoroutines
|
// against reportsStopped, which is set once nothing is to wait for another.
|
||||||
// pages are the pages a refused visitor gets.
|
reports sync.WaitGroup
|
||||||
pages blockPages
|
reportsMu sync.Mutex
|
||||||
|
reportsStopped bool
|
||||||
}
|
}
|
||||||
|
|
||||||
// newTransport returns what carries requests to the app. It never goes through
|
// newTransport returns what carries requests to the app. It never goes through
|
||||||
|
|||||||
+31
-182
@@ -11,7 +11,6 @@ import (
|
|||||||
"net"
|
"net"
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/http/httptest"
|
"net/http/httptest"
|
||||||
"os"
|
|
||||||
"slices"
|
"slices"
|
||||||
"strconv"
|
"strconv"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -53,56 +52,8 @@ const (
|
|||||||
maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
||||||
maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
||||||
maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
||||||
contactEmail = "SWW_RPROXY_CONTACT_EMAIL"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// The pages rproxy gives a refused visitor in place of waf's body.
|
|
||||||
const (
|
|
||||||
// refusedPageFile and rateLimitedPageFile are the pages' files, beside
|
|
||||||
// the code.
|
|
||||||
refusedPageFile = "refused.html"
|
|
||||||
rateLimitedPageFile = "ratelimited.html"
|
|
||||||
// contactSentence is the sentence of each page that becomes a link to
|
|
||||||
// SWW_RPROXY_CONTACT_EMAIL.
|
|
||||||
contactSentence = "contact the site's operator"
|
|
||||||
htmlContentType = "text/html; charset=utf-8"
|
|
||||||
)
|
|
||||||
|
|
||||||
// page is the page in file, as rproxy serves it without a contact email.
|
|
||||||
func page(t *testing.T, file string) string {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
//nolint:gosec // a page beside the code
|
|
||||||
contents, err := os.ReadFile(file)
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("read the page: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
return string(contents)
|
|
||||||
}
|
|
||||||
|
|
||||||
// wantPage checks that got is status with the page in file as its body, and
|
|
||||||
// the headers of a page.
|
|
||||||
func wantPage(t *testing.T, got answered, status int, file string) {
|
|
||||||
t.Helper()
|
|
||||||
|
|
||||||
if got.status != status || got.body != page(t, file) {
|
|
||||||
t.Errorf(
|
|
||||||
"answered %d %q, want %d and %s",
|
|
||||||
got.status,
|
|
||||||
got.body,
|
|
||||||
status,
|
|
||||||
file,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
if got.header.Get("Content-Type") != htmlContentType ||
|
|
||||||
got.header.Get("X-Content-Type-Options") != "nosniff" ||
|
|
||||||
got.header.Get("Cache-Control") != "no-store" {
|
|
||||||
t.Errorf("the page's headers are %v", got.header)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// The headers the tests set and look at.
|
// The headers the tests set and look at.
|
||||||
const (
|
const (
|
||||||
dispositionHeader = wafprotocol.DispositionHeader
|
dispositionHeader = wafprotocol.DispositionHeader
|
||||||
@@ -355,20 +306,6 @@ func deny(w http.ResponseWriter, _ *http.Request) {
|
|||||||
_, _ = io.WriteString(w, deniedBody)
|
_, _ = io.WriteString(w, deniedBody)
|
||||||
}
|
}
|
||||||
|
|
||||||
// answerAs answers a question with status and body, with disposition unless
|
|
||||||
// it is "", and a header of its own.
|
|
||||||
func answerAs(disposition string, status int, body string) http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, _ *http.Request) {
|
|
||||||
if disposition != "" {
|
|
||||||
w.Header().Set(dispositionHeader, disposition)
|
|
||||||
}
|
|
||||||
|
|
||||||
w.Header().Set(fromWAFHeader, addedHeaderValue)
|
|
||||||
w.WriteHeader(status)
|
|
||||||
_, _ = io.WriteString(w, body)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// startApp starts app as the app rproxy passes requests to.
|
// startApp starts app as the app rproxy passes requests to.
|
||||||
func startApp(t *testing.T, app http.HandlerFunc) *httptest.Server {
|
func startApp(t *testing.T, app http.HandlerFunc) *httptest.Server {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
@@ -773,7 +710,7 @@ func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
|
|||||||
)
|
)
|
||||||
w := httptest.NewRecorder()
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
server.ServeHTTP(w, req)
|
server.Handler.ServeHTTP(w, req)
|
||||||
|
|
||||||
// Go's server guesses a type when the first part of the body goes before the
|
// Go's server guesses a type when the first part of the body goes before the
|
||||||
// headers are flushed, which is a matter of timing here. So this looks for what
|
// headers are flushed, which is a matter of timing here. So this looks for what
|
||||||
@@ -830,7 +767,7 @@ func TestRemovesTheVisitorsLookupHeadersWhenWAFGivesNone(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
|
func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startWAF(t, deny)
|
standIn := startWAF(t, deny)
|
||||||
@@ -840,10 +777,9 @@ func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
|
|||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
||||||
|
|
||||||
got := get(t, addr, "/")
|
got := get(t, addr, "/")
|
||||||
wantPage(t, got, http.StatusForbidden, refusedPageFile)
|
if got.status != http.StatusForbidden || got.body != deniedBody ||
|
||||||
|
got.header.Get(fromWAFHeader) != addedHeaderValue {
|
||||||
if got.header.Get(fromWAFHeader) != addedHeaderValue {
|
t.Errorf("answered %+v, want waf's answer", got)
|
||||||
t.Errorf("answered %+v, want waf's headers", got)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(got.header.Values(dispositionHeader)) != 0 {
|
if len(got.header.Values(dispositionHeader)) != 0 {
|
||||||
@@ -852,7 +788,7 @@ func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
|
|||||||
|
|
||||||
report := standIn.report(t)
|
report := standIn.report(t)
|
||||||
if report.Status != http.StatusForbidden ||
|
if report.Status != http.StatusForbidden ||
|
||||||
report.ResponseBytes != int64(len(page(t, refusedPageFile))) ||
|
report.ResponseBytes != int64(len(deniedBody)) ||
|
||||||
report.DurationUpstreamTotalMS != 0 {
|
report.DurationUpstreamTotalMS != 0 {
|
||||||
t.Errorf("report %+v, of a request denied", report)
|
t.Errorf("report %+v, of a request denied", report)
|
||||||
}
|
}
|
||||||
@@ -864,7 +800,7 @@ func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestADeniedHeadGivesThePagesContentLength(t *testing.T) {
|
func TestADeniedHeadKeepsWAFsContentLength(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startWAF(t, deny)
|
standIn := startWAF(t, deny)
|
||||||
@@ -872,13 +808,13 @@ func TestADeniedHeadGivesThePagesContentLength(t *testing.T) {
|
|||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
||||||
|
|
||||||
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
|
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
|
||||||
if got.status != http.StatusForbidden || got.body != "" ||
|
if got.status != http.StatusForbidden ||
|
||||||
got.header.Get(
|
got.header.Get(
|
||||||
contentLengthHeader,
|
contentLengthHeader,
|
||||||
) != strconv.Itoa(
|
) != strconv.Itoa(
|
||||||
len(page(t, refusedPageFile)),
|
len(deniedBody),
|
||||||
) {
|
) {
|
||||||
t.Errorf("answered %+v, want waf's 403 and the page's length", got)
|
t.Errorf("answered %+v, want waf's 403 and its length", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -887,13 +823,11 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
|
|||||||
|
|
||||||
const host = " HTTP/1.1\r\nHost: site.example\r\n\r\n"
|
const host = " HTTP/1.1\r\nHost: site.example\r\n\r\n"
|
||||||
|
|
||||||
refusedPage := page(t, refusedPageFile)
|
|
||||||
|
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
name, request, body string
|
name, request, body string
|
||||||
status int
|
status int
|
||||||
}{
|
}{
|
||||||
{"a deny", "GET /" + host, refusedPage, http.StatusForbidden},
|
{"a deny", "GET /" + host, deniedBody, http.StatusForbidden},
|
||||||
{"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest},
|
{"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest},
|
||||||
{"the health check", "GET " + wafprotocol.HealthPath + host, "ok\n", http.StatusOK},
|
{"the health check", "GET " + wafprotocol.HealthPath + host, "ok\n", http.StatusOK},
|
||||||
} {
|
} {
|
||||||
@@ -965,83 +899,6 @@ func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestADenyOf429GivesTheVisitorTheRateLimitedPage(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
standIn := startWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
|
||||||
w.Header().Set(dispositionHeader, dispositionDeny)
|
|
||||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
|
||||||
w.WriteHeader(http.StatusTooManyRequests)
|
|
||||||
_, _ = io.WriteString(w, deniedBody)
|
|
||||||
})
|
|
||||||
app := startApp(t, greet)
|
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
|
||||||
|
|
||||||
wantPage(
|
|
||||||
t,
|
|
||||||
get(t, addr, "/"),
|
|
||||||
http.StatusTooManyRequests,
|
|
||||||
rateLimitedPageFile,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestThePagesLinkTheContactEmail(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
const email = "ops@example.com"
|
|
||||||
|
|
||||||
link := `<a href="mailto:` + email + `">` + contactSentence + `</a>`
|
|
||||||
|
|
||||||
for _, tc := range []struct {
|
|
||||||
file string
|
|
||||||
status int
|
|
||||||
}{
|
|
||||||
{refusedPageFile, http.StatusForbidden},
|
|
||||||
{rateLimitedPageFile, http.StatusTooManyRequests},
|
|
||||||
} {
|
|
||||||
t.Run(tc.file, func(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
standIn := startWAF(
|
|
||||||
t,
|
|
||||||
func(w http.ResponseWriter, _ *http.Request) {
|
|
||||||
w.Header().Set(dispositionHeader, dispositionDeny)
|
|
||||||
w.WriteHeader(tc.status)
|
|
||||||
},
|
|
||||||
)
|
|
||||||
app := startApp(t, greet)
|
|
||||||
addr, _ := startRProxy(
|
|
||||||
t,
|
|
||||||
standIn.URL,
|
|
||||||
app.URL,
|
|
||||||
map[string]string{contactEmail: email},
|
|
||||||
)
|
|
||||||
|
|
||||||
got := get(t, addr, "/")
|
|
||||||
want := strings.Replace(
|
|
||||||
page(t, tc.file),
|
|
||||||
contactSentence,
|
|
||||||
link,
|
|
||||||
1,
|
|
||||||
)
|
|
||||||
|
|
||||||
if got.status != tc.status || got.body != want {
|
|
||||||
t.Errorf(
|
|
||||||
"answered %d %q, want %d %q",
|
|
||||||
got.status,
|
|
||||||
got.body,
|
|
||||||
tc.status,
|
|
||||||
want,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
if !strings.Contains(page(t, tc.file), contactSentence) {
|
|
||||||
t.Errorf("%s has no %q to link", tc.file, contactSentence)
|
|
||||||
}
|
|
||||||
})
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -1160,28 +1017,17 @@ func noAnswers() map[string]func(t *testing.T) string {
|
|||||||
func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const unknownBody = "who knows\n"
|
|
||||||
|
|
||||||
refusedPage := page(t, refusedPageFile)
|
|
||||||
rateLimitedPage := page(t, rateLimitedPageFile)
|
|
||||||
|
|
||||||
// want is the body the visitor gets: waf's, but for a refusal, which gets
|
|
||||||
// rproxy's page for its status.
|
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
name, disposition, body, want string
|
name, disposition, body string
|
||||||
status int
|
status int
|
||||||
}{
|
}{
|
||||||
{"a refusal", dispositionDeny, deniedBody, refusedPage, http.StatusForbidden},
|
{"a refusal", dispositionDeny, deniedBody, http.StatusForbidden},
|
||||||
{
|
{"a refusal, 429", dispositionDeny, deniedBody, http.StatusTooManyRequests},
|
||||||
"a refusal, 429", dispositionDeny, deniedBody, rateLimitedPage,
|
{"waf's 400", "", "refused\n", http.StatusBadRequest},
|
||||||
http.StatusTooManyRequests,
|
{"a 500", "", "broken\n", http.StatusInternalServerError},
|
||||||
},
|
{"a 503 denied", dispositionDeny, "busy\n", http.StatusServiceUnavailable},
|
||||||
{"waf's 400", "", "refused\n", "refused\n", http.StatusBadRequest},
|
{"no header", "", "who knows\n", http.StatusOK},
|
||||||
{"a 500", "", "broken\n", "broken\n", http.StatusInternalServerError},
|
{"an unknown disposition", "maybe", "who knows\n", http.StatusOK},
|
||||||
{"a 503 denied", dispositionDeny, "busy\n", "busy\n", http.StatusServiceUnavailable},
|
|
||||||
{"a 403 without a disposition", "", "nope\n", "nope\n", http.StatusForbidden},
|
|
||||||
{"no header", "", unknownBody, unknownBody, http.StatusOK},
|
|
||||||
{"an unknown disposition", "maybe", unknownBody, unknownBody, http.StatusOK},
|
|
||||||
} {
|
} {
|
||||||
for _, setting := range []string{dispositionAllow, dispositionDeny} {
|
for _, setting := range []string{dispositionAllow, dispositionDeny} {
|
||||||
subtest := tc.name + " with " + noAnswerSetting + "=" + setting
|
subtest := tc.name + " with " + noAnswerSetting + "=" + setting
|
||||||
@@ -1190,7 +1036,15 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
|||||||
|
|
||||||
standIn := startWAF(
|
standIn := startWAF(
|
||||||
t,
|
t,
|
||||||
answerAs(tc.disposition, tc.status, tc.body),
|
func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
if tc.disposition != "" {
|
||||||
|
w.Header().Set(dispositionHeader, tc.disposition)
|
||||||
|
}
|
||||||
|
|
||||||
|
w.Header().Set(fromWAFHeader, addedHeaderValue)
|
||||||
|
w.WriteHeader(tc.status)
|
||||||
|
_, _ = io.WriteString(w, tc.body)
|
||||||
|
},
|
||||||
)
|
)
|
||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
@@ -1205,14 +1059,9 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
got := get(t, addr, "/")
|
got := get(t, addr, "/")
|
||||||
if got.status != tc.status || got.body != tc.want ||
|
if got.status != tc.status || got.body != tc.body ||
|
||||||
got.header.Get(fromWAFHeader) != addedHeaderValue {
|
got.header.Get(fromWAFHeader) != addedHeaderValue {
|
||||||
t.Errorf(
|
t.Errorf("answered %+v, want waf's answer", got)
|
||||||
"answered %+v, want %d %q",
|
|
||||||
got,
|
|
||||||
tc.status,
|
|
||||||
tc.want,
|
|
||||||
)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(got.header.Values(dispositionHeader)) != 0 {
|
if len(got.header.Values(dispositionHeader)) != 0 {
|
||||||
|
|||||||
+9
-57
@@ -7,9 +7,7 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"maps"
|
"maps"
|
||||||
"net/http"
|
"net/http"
|
||||||
"strconv"
|
|
||||||
"strings"
|
"strings"
|
||||||
"sync"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/wafprotocol"
|
"sneak.berlin/go/smallwebwaf/internal/wafprotocol"
|
||||||
@@ -188,9 +186,7 @@ func canBeAsked(r *http.Request) bool {
|
|||||||
len(r.Header.Values("User-Agent")) <= 1
|
len(r.Header.Values("User-Agent")) <= 1
|
||||||
}
|
}
|
||||||
|
|
||||||
// answerWith gives the visitor waf's answer, less its disposition. A deny of
|
// answerWith gives the visitor waf's answer, less its disposition.
|
||||||
// 403 or 429 gets rproxy's page for that status as its body, in place of
|
|
||||||
// waf's.
|
|
||||||
func (rq *request) answerWith(answer *wafAnswer) {
|
func (rq *request) answerWith(answer *wafAnswer) {
|
||||||
rq.prepareOwnAnswer(rq.limits.requestDeadline())
|
rq.prepareOwnAnswer(rq.limits.requestDeadline())
|
||||||
|
|
||||||
@@ -198,15 +194,6 @@ func (rq *request) answerWith(answer *wafAnswer) {
|
|||||||
maps.Copy(header, answer.header)
|
maps.Copy(header, answer.header)
|
||||||
header.Del(wafprotocol.DispositionHeader)
|
header.Del(wafprotocol.DispositionHeader)
|
||||||
|
|
||||||
if rq.disposition == wafprotocol.DispositionDeny {
|
|
||||||
page, found := rq.h.pages.forStatus(answer.status)
|
|
||||||
if found {
|
|
||||||
rq.sendPage(answer.status, page)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// Go's server adds a Content-Type it guesses from the body to a response whose
|
// Go's server adds a Content-Type it guesses from the body to a response whose
|
||||||
// headers have no Content-Type key. The key with no value sends none.
|
// headers have no Content-Type key. The key with no value sends none.
|
||||||
if header.Get("Content-Type") == "" {
|
if header.Get("Content-Type") == "" {
|
||||||
@@ -216,17 +203,6 @@ func (rq *request) answerWith(answer *wafAnswer) {
|
|||||||
rq.send(answer.status, answer.body)
|
rq.send(answer.status, answer.body)
|
||||||
}
|
}
|
||||||
|
|
||||||
// sendPage sends page as the body of a refusal with status. The length is set
|
|
||||||
// here too, since send leaves a HEAD's as waf gave it, which is of waf's body.
|
|
||||||
func (rq *request) sendPage(status int, page []byte) {
|
|
||||||
header := rq.out.Header()
|
|
||||||
header.Set("Content-Type", htmlContentType)
|
|
||||||
header.Set("X-Content-Type-Options", "nosniff")
|
|
||||||
header.Set("Cache-Control", "no-store")
|
|
||||||
header.Set("Content-Length", strconv.Itoa(len(page)))
|
|
||||||
rq.send(status, page)
|
|
||||||
}
|
|
||||||
|
|
||||||
// sendReport sends waf report on a goroutine of its own, so that the
|
// sendReport sends waf report on a goroutine of its own, so that the
|
||||||
// visitor does not wait for it. The report goes on after ctx, the request's own
|
// visitor does not wait for it. The report goes on after ctx, the request's own
|
||||||
// context, has ended.
|
// context, has ended.
|
||||||
@@ -234,7 +210,14 @@ func (h *handler) sendReport(
|
|||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
report *wafprotocol.Report,
|
report *wafprotocol.Report,
|
||||||
) {
|
) {
|
||||||
h.reports.startUnlessStopped(func() {
|
h.reportsMu.Lock()
|
||||||
|
defer h.reportsMu.Unlock()
|
||||||
|
|
||||||
|
if h.reportsStopped {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
h.reports.Go(func() {
|
||||||
err := h.postReport(context.WithoutCancel(ctx), report)
|
err := h.postReport(context.WithoutCancel(ctx), report)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
h.processLog.Warn("a report is dropped", "error", err.Error())
|
h.processLog.Warn("a report is dropped", "error", err.Error())
|
||||||
@@ -242,37 +225,6 @@ func (h *handler) sendReport(
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
// reportGoroutines are the goroutines sending reports to waf. None starts once
|
|
||||||
// stopAndWait has been called.
|
|
||||||
type reportGoroutines struct {
|
|
||||||
mu sync.Mutex
|
|
||||||
stopped bool
|
|
||||||
running sync.WaitGroup
|
|
||||||
}
|
|
||||||
|
|
||||||
// startUnlessStopped runs send on a goroutine of its own, or drops it once
|
|
||||||
// stopAndWait has been called.
|
|
||||||
func (g *reportGoroutines) startUnlessStopped(send func()) {
|
|
||||||
g.mu.Lock()
|
|
||||||
defer g.mu.Unlock()
|
|
||||||
|
|
||||||
if g.stopped {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
g.running.Go(send)
|
|
||||||
}
|
|
||||||
|
|
||||||
// stopAndWait stops any more goroutines from starting, and returns once those
|
|
||||||
// running have ended.
|
|
||||||
func (g *reportGoroutines) stopAndWait() {
|
|
||||||
g.mu.Lock()
|
|
||||||
g.stopped = true
|
|
||||||
g.mu.Unlock()
|
|
||||||
|
|
||||||
g.running.Wait()
|
|
||||||
}
|
|
||||||
|
|
||||||
// postReport posts report to waf, and waits for its answer for up to
|
// postReport posts report to waf, and waits for its answer for up to
|
||||||
// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS.
|
// SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS.
|
||||||
func (h *handler) postReport(
|
func (h *handler) postReport(
|
||||||
|
|||||||
@@ -95,20 +95,12 @@ func background(loop func(context.Context)) fx.Hook {
|
|||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// daemonServer is the server a daemon runs, waf.Server or rproxy.Server: the
|
|
||||||
// methods of Go's http.Server that serve starts and stops it with.
|
|
||||||
type daemonServer interface {
|
|
||||||
Serve(listener net.Listener) error
|
|
||||||
Shutdown(ctx context.Context) error
|
|
||||||
Close() error
|
|
||||||
}
|
|
||||||
|
|
||||||
// serve serves requests on listener from the start. At the stop it gives the
|
// serve serves requests on listener from the start. At the stop it gives the
|
||||||
// requests in progress shutdownTimeout to finish, then closes their
|
// requests in progress shutdownTimeout to finish, then closes their
|
||||||
// connections. When serving fails, it logs why and sends the error to
|
// connections. When serving fails, it logs why and sends the error to
|
||||||
// failed.
|
// failed.
|
||||||
func serve(
|
func serve(
|
||||||
lc fx.Lifecycle, server daemonServer, listener net.Listener,
|
lc fx.Lifecycle, server *http.Server, listener net.Listener,
|
||||||
processLog *slog.Logger, failed servingFailed,
|
processLog *slog.Logger, failed servingFailed,
|
||||||
) {
|
) {
|
||||||
served := make(chan error, 1)
|
served := make(chan error, 1)
|
||||||
|
|||||||
@@ -4,6 +4,7 @@ import (
|
|||||||
"context"
|
"context"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net"
|
"net"
|
||||||
|
"net/http"
|
||||||
"os"
|
"os"
|
||||||
|
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
@@ -43,13 +44,8 @@ func RunRProxy(ctx context.Context, params Params) int {
|
|||||||
app := fx.New(
|
app := fx.New(
|
||||||
fx.Supply(cfg, processLog, requestLog{params.Stdout}, failed),
|
fx.Supply(cfg, processLog, requestLog{params.Stdout}, failed),
|
||||||
fx.Provide(
|
fx.Provide(
|
||||||
// run waits for rproxy.Server's reports, and serve takes it as a
|
newRProxyServer,
|
||||||
// daemonServer.
|
func(server *rproxy.Server) *http.Server { return server.Server },
|
||||||
fx.Annotate(
|
|
||||||
newRProxyServer,
|
|
||||||
fx.As(fx.Self()),
|
|
||||||
fx.As(new(daemonServer)),
|
|
||||||
),
|
|
||||||
newRProxyListener,
|
newRProxyListener,
|
||||||
),
|
),
|
||||||
fx.Invoke(serve),
|
fx.Invoke(serve),
|
||||||
|
|||||||
+3
-2
@@ -6,6 +6,7 @@ import (
|
|||||||
"io"
|
"io"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net"
|
"net"
|
||||||
|
"net/http"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
@@ -96,8 +97,8 @@ func RunWAF(ctx context.Context, params Params) int {
|
|||||||
newAlertQueue, newRuleFiles, newCoreRuleSet, newLookupFile,
|
newAlertQueue, newRuleFiles, newCoreRuleSet, newLookupFile,
|
||||||
newLimiter, newLedger, newAnomalyCounters,
|
newLimiter, newLedger, newAnomalyCounters,
|
||||||
newLists, newDNSBL, newAbuseIPDB, newRouteWatch,
|
newLists, newDNSBL, newAbuseIPDB, newRouteWatch,
|
||||||
newMetrics, newGeoJS,
|
newMetrics, newGeoJS, newWAFServer,
|
||||||
fx.Annotate(newWAFServer, fx.As(new(daemonServer))),
|
func(server *waf.Server) *http.Server { return server.Server },
|
||||||
newWAFListener, newStateFiles,
|
newWAFListener, newStateFiles,
|
||||||
),
|
),
|
||||||
// fx stops hooks in the reverse of the order they were added in. The
|
// fx stops hooks in the reverse of the order they were added in. The
|
||||||
|
|||||||
@@ -1,8 +1,7 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
// The heuristic abuseipdb. It reads SWW_WAF_ABUSEIPDB_KEY,
|
// The heuristic abuseipdb. It reads SWW_WAF_ABUSEIPDB_KEY,
|
||||||
// SWW_WAF_ABUSEIPDB_MIN_SCORE, SWW_WAF_DNSBL_ABUSEIPDB_ACTION and
|
// SWW_WAF_ABUSEIPDB_MIN_SCORE and SWW_WAF_DNSBL_ABUSEIPDB_ACTION.
|
||||||
// SWW_WAF_BAN_RESPONSE.
|
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -17,11 +16,10 @@ import (
|
|||||||
|
|
||||||
// abuseIPDB adds the tag abuseipdb and notes AbuseIPDB, as noteHit does, with
|
// abuseIPDB adds the tag abuseipdb and notes AbuseIPDB, as noteHit does, with
|
||||||
// the score, when its score of the client is a hit, and refuses the request
|
// the score, when its score of the client is a hit, and refuses the request
|
||||||
// with SWW_WAF_BAN_RESPONSE when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is deny, as
|
// when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is deny, as dnsbl does for a zone. While
|
||||||
// dnsbl does for a zone. While SWW_WAF_ABUSEIPDB_KEY is unset it does nothing.
|
// SWW_WAF_ABUSEIPDB_KEY is unset it does nothing. A client without a score is
|
||||||
// A client without a score is checked in the background, by the request's
|
// checked in the background, by the request's address, if its history counts
|
||||||
// address, if its history counts an offence, and the request does not wait for
|
// an offence, and the request does not wait for the answer. The score is then
|
||||||
// the answer. The score is then
|
|
||||||
// used for each address of the client. Its figures are the score and
|
// used for each address of the client. Its figures are the score and
|
||||||
// SWW_WAF_ABUSEIPDB_MIN_SCORE.
|
// SWW_WAF_ABUSEIPDB_MIN_SCORE.
|
||||||
type abuseIPDB struct {
|
type abuseIPDB struct {
|
||||||
@@ -32,7 +30,6 @@ type abuseIPDB struct {
|
|||||||
keySet bool // SWW_WAF_ABUSEIPDB_KEY is set
|
keySet bool // SWW_WAF_ABUSEIPDB_KEY is set
|
||||||
minScore int64 // SWW_WAF_ABUSEIPDB_MIN_SCORE
|
minScore int64 // SWW_WAF_ABUSEIPDB_MIN_SCORE
|
||||||
action config.ReputationAction // SWW_WAF_DNSBL_ABUSEIPDB_ACTION
|
action config.ReputationAction // SWW_WAF_DNSBL_ABUSEIPDB_ACTION
|
||||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
|
||||||
ipv6GroupPrefix int // SWW_WAF_IPV6_GROUP_PREFIX
|
ipv6GroupPrefix int // SWW_WAF_IPV6_GROUP_PREFIX
|
||||||
hitNoter reputationHitNoter
|
hitNoter reputationHitNoter
|
||||||
}
|
}
|
||||||
@@ -74,7 +71,6 @@ func (a abuseIPDB) check(
|
|||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionClientDenied,
|
action: requestlog.ActionClientDenied,
|
||||||
status: a.banResponse,
|
|
||||||
figures: map[string]any{
|
figures: map[string]any{
|
||||||
figureScore: score, "min_score": a.minScore,
|
figureScore: score, "min_score": a.minScore,
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -1,58 +0,0 @@
|
|||||||
package waf
|
|
||||||
|
|
||||||
// The heuristic active_ban. It reads SWW_WAF_MODE and SWW_WAF_BAN_RESPONSE; the
|
|
||||||
// bans are the ban ledger's.
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"time"
|
|
||||||
)
|
|
||||||
|
|
||||||
// activeBan refuses a client a ban covers, with SWW_WAF_BAN_RESPONSE: a ban on
|
|
||||||
// a netblock the client is in, in force at now. It notes for the log line when
|
|
||||||
// that ban ends. A request that makes the ban permanent, or in observe mode
|
|
||||||
// would have, raises the alert for it. Its figures are the ban's netblock and
|
|
||||||
// when it ends.
|
|
||||||
type activeBan struct {
|
|
||||||
banMaker banMaker
|
|
||||||
}
|
|
||||||
|
|
||||||
func (activeBan) name() string { return "active_ban" }
|
|
||||||
|
|
||||||
func (a activeBan) check(
|
|
||||||
_ context.Context,
|
|
||||||
rq *request,
|
|
||||||
now time.Time,
|
|
||||||
) *verdict {
|
|
||||||
ledger := a.banMaker.ledger
|
|
||||||
|
|
||||||
check := ledger.Check
|
|
||||||
if a.banMaker.observe {
|
|
||||||
check = ledger.Find // the ban refuses nothing, and stays as it is
|
|
||||||
}
|
|
||||||
|
|
||||||
ban, banned, madePermanent := check(rq.client, now)
|
|
||||||
|
|
||||||
var expires string
|
|
||||||
if banned {
|
|
||||||
expires = banExpires(ban)
|
|
||||||
rq.line.BanExpires = expires
|
|
||||||
}
|
|
||||||
|
|
||||||
if madePermanent {
|
|
||||||
ban.Expires = time.Time{} // the ban made permanent, which Find leaves as it is
|
|
||||||
a.banMaker.alertBan(rq, ban)
|
|
||||||
}
|
|
||||||
|
|
||||||
if !banned {
|
|
||||||
return nil
|
|
||||||
}
|
|
||||||
|
|
||||||
return &verdict{
|
|
||||||
action: banAction(ban.Permanent()),
|
|
||||||
status: a.banMaker.banResponse,
|
|
||||||
figures: map[string]any{
|
|
||||||
figureNetblock: ban.Netblock.String(), "ban_expires": expires,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
}
|
|
||||||
@@ -717,7 +717,7 @@ func askDirectly(
|
|||||||
recorder := httptest.NewRecorder()
|
recorder := httptest.NewRecorder()
|
||||||
began := time.Now()
|
began := time.Now()
|
||||||
|
|
||||||
server.ServeHTTP(recorder, req)
|
server.Handler.ServeHTTP(recorder, req)
|
||||||
|
|
||||||
return recorder.Code, time.Since(began)
|
return recorder.Code, time.Since(began)
|
||||||
}
|
}
|
||||||
|
|||||||
+58
-5
@@ -1,9 +1,10 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
// banMaker, which makes the bans of several heuristics. The bans are the ban
|
// The heuristic active_ban, and banMaker, which makes the bans of several
|
||||||
// ledger's.
|
// heuristics. active_ban reads SWW_WAF_MODE; the bans are the ban ledger's.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
|
"context"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -15,6 +16,59 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
)
|
)
|
||||||
|
|
||||||
|
// banResponse is a refusal answered with SWW_WAF_BAN_RESPONSE, and logged
|
||||||
|
// with action.
|
||||||
|
func (rq *request) banResponse(action requestlog.Action) *refusal {
|
||||||
|
return &refusal{status: rq.h.config.BanResponse, action: action}
|
||||||
|
}
|
||||||
|
|
||||||
|
// activeBan refuses a client a ban covers: a ban on a netblock the client is
|
||||||
|
// in, in force at now. It notes for the log line when that ban ends. A request
|
||||||
|
// that makes the ban permanent, or in observe mode would have, raises the alert
|
||||||
|
// for it. Its figures are the ban's netblock and when it ends.
|
||||||
|
type activeBan struct {
|
||||||
|
banMaker banMaker
|
||||||
|
}
|
||||||
|
|
||||||
|
func (activeBan) name() string { return "active_ban" }
|
||||||
|
|
||||||
|
func (a activeBan) check(
|
||||||
|
_ context.Context,
|
||||||
|
rq *request,
|
||||||
|
now time.Time,
|
||||||
|
) *verdict {
|
||||||
|
ledger := a.banMaker.ledger
|
||||||
|
|
||||||
|
check := ledger.Check
|
||||||
|
if a.banMaker.observe {
|
||||||
|
check = ledger.Find // the ban refuses nothing, and stays as it is
|
||||||
|
}
|
||||||
|
|
||||||
|
ban, banned, madePermanent := check(rq.client, now)
|
||||||
|
|
||||||
|
var expires string
|
||||||
|
if banned {
|
||||||
|
expires = banExpires(ban)
|
||||||
|
rq.line.BanExpires = expires
|
||||||
|
}
|
||||||
|
|
||||||
|
if madePermanent {
|
||||||
|
ban.Expires = time.Time{} // the ban made permanent, which Find leaves as it is
|
||||||
|
a.banMaker.alertBan(rq, ban)
|
||||||
|
}
|
||||||
|
|
||||||
|
if !banned {
|
||||||
|
return nil
|
||||||
|
}
|
||||||
|
|
||||||
|
return &verdict{
|
||||||
|
action: banAction(ban.Permanent()),
|
||||||
|
figures: map[string]any{
|
||||||
|
figureNetblock: ban.Netblock.String(), "ban_expires": expires,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// banAction is the action of a request refused under a ban or for making one:
|
// banAction is the action of a request refused under a ban or for making one:
|
||||||
// permanent says the ban never runs out.
|
// permanent says the ban never runs out.
|
||||||
func banAction(permanent bool) requestlog.Action {
|
func banAction(permanent bool) requestlog.Action {
|
||||||
@@ -64,9 +118,8 @@ type banMaker struct {
|
|||||||
alerts *alerts.Queue
|
alerts *alerts.Queue
|
||||||
// observe is true in observe mode (SWW_WAF_MODE).
|
// observe is true in observe mode (SWW_WAF_MODE).
|
||||||
observe bool
|
observe bool
|
||||||
// banResponse is SWW_WAF_BAN_RESPONSE: the status of a request refused
|
// banResponse is SWW_WAF_BAN_RESPONSE, the status the notes of a ban for a
|
||||||
// under a ban or for a clear sign of attack, which the notes of a ban for
|
// clear sign of attack give.
|
||||||
// a clear sign of attack give.
|
|
||||||
banResponse int
|
banResponse int
|
||||||
// banScopeV4Prefix is SWW_WAF_BAN_SCOPE_V4_PREFIX and ipv6GroupPrefix
|
// banScopeV4Prefix is SWW_WAF_BAN_SCOPE_V4_PREFIX and ipv6GroupPrefix
|
||||||
// SWW_WAF_IPV6_GROUP_PREFIX, which set the netblock a ban covers.
|
// SWW_WAF_IPV6_GROUP_PREFIX, which set the netblock a ban covers.
|
||||||
|
|||||||
@@ -4,7 +4,6 @@ package waf
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"net/http"
|
|
||||||
"sync/atomic"
|
"sync/atomic"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -162,7 +161,6 @@ func (rq *request) applyFindings(
|
|||||||
func (rq *request) timedOut(running string) *verdict {
|
func (rq *request) timedOut(running string) *verdict {
|
||||||
v := &verdict{
|
v := &verdict{
|
||||||
action: requestlog.ActionChecksTimedOut,
|
action: requestlog.ActionChecksTimedOut,
|
||||||
status: http.StatusServiceUnavailable,
|
|
||||||
figures: map[string]any{
|
figures: map[string]any{
|
||||||
"time_limit_ms": rq.h.besideLookupLimit.Milliseconds(),
|
"time_limit_ms": rq.h.besideLookupLimit.Milliseconds(),
|
||||||
},
|
},
|
||||||
|
|||||||
@@ -592,7 +592,7 @@ func serveFromDE(
|
|||||||
req.Header.Set(wafprotocol.ClientHeader, fromDE)
|
req.Header.Set(wafprotocol.ClientHeader, fromDE)
|
||||||
req.Header.Set(requestIDHeader, "the-id")
|
req.Header.Set(requestIDHeader, "the-id")
|
||||||
|
|
||||||
server.ServeHTTP(httptest.NewRecorder(), req)
|
server.Handler.ServeHTTP(httptest.NewRecorder(), req)
|
||||||
}
|
}
|
||||||
|
|
||||||
// wantPercent checks a limit percentage that a log line or a ban's notes
|
// wantPercent checks a limit percentage that a log line or a ban's notes
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
// The heuristic blocklists. It reads SWW_WAF_BLOCKLIST_ACTION and
|
// The heuristic blocklists. It reads SWW_WAF_BLOCKLIST_ACTION, and the blocklists
|
||||||
// SWW_WAF_BAN_RESPONSE, and the blocklists through the reputation lists.
|
// through the reputation lists.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -13,15 +13,13 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// blocklists notes the blocklists that list the client, as noteListed does,
|
// blocklists notes the blocklists that list the client, as noteListed does,
|
||||||
// and refuses the request with SWW_WAF_BAN_RESPONSE when
|
// and refuses the request when SWW_WAF_BLOCKLIST_ACTION is deny. Being limit, it
|
||||||
// SWW_WAF_BLOCKLIST_ACTION is deny. Being limit, it lowers the client's limits
|
// lowers the client's limits instead (see limitPercentages), and being log, it
|
||||||
// instead (see limitPercentages), and being log, it does nothing more. Its
|
// does nothing more. Its figure is the blocklists that list the client.
|
||||||
// figure is the blocklists that list the client.
|
|
||||||
type blocklists struct {
|
type blocklists struct {
|
||||||
lists *reputation.Lists
|
lists *reputation.Lists
|
||||||
action config.ReputationAction // SWW_WAF_BLOCKLIST_ACTION
|
action config.ReputationAction // SWW_WAF_BLOCKLIST_ACTION
|
||||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
hitNoter reputationHitNoter
|
||||||
hitNoter reputationHitNoter
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (blocklists) name() string { return "blocklists" }
|
func (blocklists) name() string { return "blocklists" }
|
||||||
@@ -41,7 +39,6 @@ func (b blocklists) check(
|
|||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionClientDenied,
|
action: requestlog.ActionClientDenied,
|
||||||
status: b.banResponse,
|
|
||||||
figures: map[string]any{"lists": listedBy},
|
figures: map[string]any{"lists": listedBy},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,7 +5,6 @@ package waf
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"net/http"
|
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
@@ -78,7 +77,6 @@ func (c coreRuleSet) check(
|
|||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionBlockedByCoreRuleSet,
|
action: requestlog.ActionBlockedByCoreRuleSet,
|
||||||
status: http.StatusForbidden,
|
|
||||||
figures: map[string]any{
|
figures: map[string]any{
|
||||||
figureScore: result.Score,
|
figureScore: result.Score,
|
||||||
"threshold": threshold,
|
"threshold": threshold,
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
// The heuristic countries. It reads SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES and
|
// The heuristic countries. It reads SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
||||||
// SWW_WAF_BAN_RESPONSE.
|
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -14,12 +13,10 @@ import (
|
|||||||
|
|
||||||
// countries refuses a request when SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES is set and
|
// countries refuses a request when SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES is set and
|
||||||
// does not hold the client's country as it was looked up. A client without a
|
// does not hold the client's country as it was looked up. A client without a
|
||||||
// country, or whose country cannot be found, is refused. It refuses with
|
// country, or whose country cannot be found, is refused. Its figures are the
|
||||||
// SWW_WAF_BAN_RESPONSE. Its figures are the country and the setting that
|
// country and the setting that refused it.
|
||||||
// refused it.
|
|
||||||
type countries struct {
|
type countries struct {
|
||||||
allowed []string // SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES
|
allowed []string // SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES
|
||||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (countries) name() string { return "countries" }
|
func (countries) name() string { return "countries" }
|
||||||
@@ -38,7 +35,6 @@ func (c countries) check(
|
|||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionCountryDenied,
|
action: requestlog.ActionCountryDenied,
|
||||||
status: c.banResponse,
|
|
||||||
figures: map[string]any{
|
figures: map[string]any{
|
||||||
"country": country,
|
"country": country,
|
||||||
"setting": config.ExclusivelyAllowedCountriesSetting,
|
"setting": config.ExclusivelyAllowedCountriesSetting,
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
// The heuristic deny_nets. It reads SWW_WAF_DENY_NETS and SWW_WAF_BAN_RESPONSE.
|
// The heuristic deny_nets. It reads SWW_WAF_DENY_NETS.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -10,11 +10,10 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
)
|
)
|
||||||
|
|
||||||
// denyNets refuses a client in SWW_WAF_DENY_NETS, with SWW_WAF_BAN_RESPONSE.
|
// denyNets refuses a client in SWW_WAF_DENY_NETS. Its figure is the netblock of
|
||||||
// Its figure is the netblock of the setting the client is in.
|
// the setting the client is in.
|
||||||
type denyNets struct {
|
type denyNets struct {
|
||||||
netblocks []netip.Prefix // SWW_WAF_DENY_NETS
|
netblocks []netip.Prefix // SWW_WAF_DENY_NETS
|
||||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (denyNets) name() string { return "deny_nets" }
|
func (denyNets) name() string { return "deny_nets" }
|
||||||
@@ -28,7 +27,6 @@ func (d denyNets) check(
|
|||||||
if netblock.Contains(rq.client) {
|
if netblock.Contains(rq.client) {
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionClientDenied,
|
action: requestlog.ActionClientDenied,
|
||||||
status: d.banResponse,
|
|
||||||
figures: map[string]any{figureNetblock: netblock.String()},
|
figures: map[string]any{figureNetblock: netblock.String()},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+10
-14
@@ -1,8 +1,7 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
// The heuristic dnsbl. It reads SWW_WAF_DNSBL_ABUSEIPDB_ACTION and
|
// The heuristic dnsbl. It reads SWW_WAF_DNSBL_ABUSEIPDB_ACTION, and the DNSBL zones'
|
||||||
// SWW_WAF_BAN_RESPONSE, and the DNSBL zones' verdicts through the DNSBL
|
// verdicts through the DNSBL checker.
|
||||||
// checker.
|
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -16,17 +15,15 @@ import (
|
|||||||
|
|
||||||
// dnsbl notes the DNSBL zones whose verdict lists the client, as noteListed
|
// dnsbl notes the DNSBL zones whose verdict lists the client, as noteListed
|
||||||
// does, adds the tag dnsbl=<zone> for each, the zone's key masked, and refuses
|
// does, adds the tag dnsbl=<zone> for each, the zone's key masked, and refuses
|
||||||
// the request with SWW_WAF_BAN_RESPONSE when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is
|
// the request when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is deny. Being
|
||||||
// deny. Being limit, it lowers the client's limits instead (see
|
// limit, it lowers the client's limits instead (see limitPercentages), and
|
||||||
// limitPercentages), and being log, it does nothing more. A zone without a
|
// being log, it does nothing more. A zone without a verdict on the client is
|
||||||
// verdict on the client is asked about it in the background, and the request
|
// asked about it in the background, and the request does not wait for the
|
||||||
// does not wait for the answer. Its figure is the zones that list the client,
|
// answer. Its figure is the zones that list the client, their keys masked.
|
||||||
// their keys masked.
|
|
||||||
type dnsbl struct {
|
type dnsbl struct {
|
||||||
dnsbl *reputation.DNSBL
|
dnsbl *reputation.DNSBL
|
||||||
action config.ReputationAction // SWW_WAF_DNSBL_ABUSEIPDB_ACTION
|
action config.ReputationAction // SWW_WAF_DNSBL_ABUSEIPDB_ACTION
|
||||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
hitNoter reputationHitNoter
|
||||||
hitNoter reputationHitNoter
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (dnsbl) name() string { return "dnsbl" }
|
func (dnsbl) name() string { return "dnsbl" }
|
||||||
@@ -50,7 +47,6 @@ func (d dnsbl) check(
|
|||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionClientDenied,
|
action: requestlog.ActionClientDenied,
|
||||||
status: d.banResponse,
|
|
||||||
figures: map[string]any{"zones": listedBy},
|
figures: map[string]any{"zones": listedBy},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1,11 +1,4 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
import "net/http"
|
|
||||||
|
|
||||||
// BesideLookupTimeLimit is besideLookupTimeLimit.
|
// BesideLookupTimeLimit is besideLookupTimeLimit.
|
||||||
const BesideLookupTimeLimit = besideLookupTimeLimit
|
const BesideLookupTimeLimit = besideLookupTimeLimit
|
||||||
|
|
||||||
// HTTPServer is Go's server s holds, for tests that read its settings.
|
|
||||||
func (s *Server) HTTPServer() *http.Server {
|
|
||||||
return s.httpServer
|
|
||||||
}
|
|
||||||
@@ -4,7 +4,6 @@ package waf
|
|||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"net/http"
|
|
||||||
"strings"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
@@ -40,7 +39,6 @@ func (headerLimits) check(
|
|||||||
if requestLine > maxLineBytes {
|
if requestLine > maxLineBytes {
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionRequestLineTooLong,
|
action: requestlog.ActionRequestLineTooLong,
|
||||||
status: http.StatusRequestURITooLong,
|
|
||||||
figures: map[string]any{
|
figures: map[string]any{
|
||||||
"request_line_bytes": requestLine,
|
"request_line_bytes": requestLine,
|
||||||
"request_line_bytes_limit": maxLineBytes,
|
"request_line_bytes_limit": maxLineBytes,
|
||||||
@@ -58,7 +56,6 @@ func (headerLimits) check(
|
|||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionRequestHeaderTooLarge,
|
action: requestlog.ActionRequestHeaderTooLarge,
|
||||||
status: http.StatusRequestHeaderFieldsTooLarge,
|
|
||||||
figures: map[string]any{
|
figures: map[string]any{
|
||||||
"longest_header_line_bytes": longest,
|
"longest_header_line_bytes": longest,
|
||||||
"header_line_bytes_limit": maxLineBytes,
|
"header_line_bytes_limit": maxLineBytes,
|
||||||
|
|||||||
@@ -42,13 +42,10 @@ type refusesWithoutLookup interface {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// verdict is a heuristic refusing a request, or banning its client: the action
|
// verdict is a heuristic refusing a request, or banning its client: the action
|
||||||
// the log line names, the status the visitor is answered with, and the figures
|
// the log line names, and the figures the heuristic acted on, by name, such as
|
||||||
// the heuristic acted on, by name, such as the count and the limit. A status of
|
// the count and the limit.
|
||||||
// 0 closes the visitor's connection without an answer. A verdict of afterAnswer
|
|
||||||
// or onReport refuses nothing, and has neither action nor status.
|
|
||||||
type verdict struct {
|
type verdict struct {
|
||||||
action requestlog.Action
|
action requestlog.Action
|
||||||
status int
|
|
||||||
figures map[string]any
|
figures map[string]any
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -80,26 +80,6 @@ func (rq *request) startLookUp(
|
|||||||
return result
|
return result
|
||||||
}
|
}
|
||||||
|
|
||||||
// refusedWhateverTheAnswer reports whether the request is one that does not wait
|
|
||||||
// for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
|
|
||||||
// its client: a heuristic of afterLookup that implements refusesWithoutLookup
|
|
||||||
// says it will refuse it. In observe mode, which refuses nothing, none is.
|
|
||||||
// rproxy lets a request through when waf answers too late.
|
|
||||||
func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
|
|
||||||
if rq.h.config.Observe {
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
for _, each := range rq.h.afterLookup {
|
|
||||||
knows, ok := each.(refusesWithoutLookup)
|
|
||||||
if ok && knows.refusesWhateverTheLookupAnswers(rq, now) {
|
|
||||||
return true
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
return false
|
|
||||||
}
|
|
||||||
|
|
||||||
// noteLookup notes result for the log line and as the request's asn= and
|
// noteLookup notes result for the log line and as the request's asn= and
|
||||||
// country= tags.
|
// country= tags.
|
||||||
func (rq *request) noteLookup(result lookupResult) {
|
func (rq *request) noteLookup(result lookupResult) {
|
||||||
|
|||||||
@@ -0,0 +1,321 @@
|
|||||||
|
// Package waf is the server the waf command runs. It answers
|
||||||
|
// rproxy's questions, takes rproxy's reports and answers its own endpoints, and
|
||||||
|
// writes a log line for each. It also bans the clients on AbuseIPDB's
|
||||||
|
// blacklist each time the reputation package fetches it.
|
||||||
|
package waf
|
||||||
|
|
||||||
|
import (
|
||||||
|
"cmp"
|
||||||
|
"io"
|
||||||
|
"log/slog"
|
||||||
|
"net/http"
|
||||||
|
"net/netip"
|
||||||
|
"time"
|
||||||
|
|
||||||
|
lru "github.com/hashicorp/golang-lru/v2"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/metrics"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/routewatch"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
|
)
|
||||||
|
|
||||||
|
// MetricsPath is where the metrics are, for a request that carries
|
||||||
|
// SWW_WAF_METRICS_TOKEN.
|
||||||
|
const MetricsPath = "/_sww/metrics"
|
||||||
|
|
||||||
|
// BansPath is where an admin lists and adds bans, and, followed by / and
|
||||||
|
// a client's address, lifts them, with SWW_WAF_ADMIN_TOKEN.
|
||||||
|
const BansPath = "/_sww/bans"
|
||||||
|
|
||||||
|
// ClientsPath is where an admin asks what waf knows of a client, by
|
||||||
|
// the client's address after it, with SWW_WAF_ADMIN_TOKEN.
|
||||||
|
const ClientsPath = "/_sww/clients/"
|
||||||
|
|
||||||
|
// Params are what New needs.
|
||||||
|
type Params struct {
|
||||||
|
Config *config.WAF
|
||||||
|
// RequestLog receives two JSON lines per question, one at its decision and
|
||||||
|
// one when its report comes, and one per request to waf's own
|
||||||
|
// endpoints.
|
||||||
|
RequestLog io.Writer
|
||||||
|
// ProcessLog receives the process's own messages.
|
||||||
|
ProcessLog *slog.Logger
|
||||||
|
// Metrics count what waf does. Whoever builds them adds the metrics
|
||||||
|
// read from the other parts.
|
||||||
|
Metrics *metrics.Metrics
|
||||||
|
// Limiter is the table of clients the rate limits count, with each
|
||||||
|
// client's history.
|
||||||
|
Limiter *ratelimit.Limiter
|
||||||
|
// Ledger holds the bans.
|
||||||
|
Ledger *bans.Ledger
|
||||||
|
// Anomalies are the counters of the anomaly thresholds.
|
||||||
|
Anomalies *anomaly.Counters
|
||||||
|
// Lists are the lists fetched from URLs, DNSBL the DNSBL zones'
|
||||||
|
// verdicts, and AbuseIPDB AbuseIPDB's scores and the checks spent.
|
||||||
|
Lists *reputation.Lists
|
||||||
|
DNSBL *reputation.DNSBL
|
||||||
|
AbuseIPDB *reputation.AbuseIPDB
|
||||||
|
// RouteWatch asks routewatch about clients, and is nil while
|
||||||
|
// SWW_WAF_ROUTEWATCH_URL is unset.
|
||||||
|
RouteWatch *routewatch.Client
|
||||||
|
// GeoJS looks clients' AS numbers and countries up while
|
||||||
|
// SWW_WAF_LOOKUP_SOURCE is geojs. It is given AddLookup as its Answered
|
||||||
|
// and GeoJSFailed as its Failed.
|
||||||
|
GeoJS *geojs.GeoJS
|
||||||
|
// LookupFile is the lookup database they are looked up in while
|
||||||
|
// SWW_WAF_LOOKUP_SOURCE is file, and nil otherwise.
|
||||||
|
LookupFile *lookup.File
|
||||||
|
// CoreRuleSet is what inspects requests while
|
||||||
|
// SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
|
||||||
|
CoreRuleSet CoreRuleSetInspector
|
||||||
|
// Now tells the time by which requests are counted for the rate
|
||||||
|
// limits, bans are made and run out, and GeoJS's answers are kept,
|
||||||
|
// normally time.Now in UTC, the time the state files give.
|
||||||
|
Now func() time.Time
|
||||||
|
// Rules are the rule files' rules, which each request is checked
|
||||||
|
// against.
|
||||||
|
Rules *rules.Files
|
||||||
|
// Alerts receive the alert for each ban waf makes or makes
|
||||||
|
// permanent, for each count over an anomaly threshold, for each request
|
||||||
|
// whose client a blocklist, a DNSBL zone or AbuseIPDB lists, for each
|
||||||
|
// request the Core Rule Set scores at or over
|
||||||
|
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD, and for GeoJS failing, a fetch of a list
|
||||||
|
// failing, a query to a DNSBL zone or a check with AbuseIPDB failing,
|
||||||
|
// or the day's AbuseIPDB checks used up.
|
||||||
|
Alerts *alerts.Queue
|
||||||
|
// BesideLookupTimeLimit is how long the heuristics of besideLookup have
|
||||||
|
// for a request. Zero means besideLookupTimeLimit. Tests that run the
|
||||||
|
// Core Rule Set lengthen it, since the race detector and a busy host can
|
||||||
|
// slow an inspection past besideLookupTimeLimit.
|
||||||
|
BesideLookupTimeLimit time.Duration
|
||||||
|
}
|
||||||
|
|
||||||
|
// CoreRuleSetInspector inspects a request with the Core Rule Set, as
|
||||||
|
// coreruleset.CoreRuleSet's Inspect says. Tests give waf a stand-in for it.
|
||||||
|
type CoreRuleSetInspector interface {
|
||||||
|
Inspect(
|
||||||
|
r *http.Request, client netip.Addr, body io.Reader,
|
||||||
|
) (coreruleset.Result, []byte, error)
|
||||||
|
}
|
||||||
|
|
||||||
|
// Server is the server waf runs.
|
||||||
|
type Server struct {
|
||||||
|
*http.Server
|
||||||
|
}
|
||||||
|
|
||||||
|
// newHandler returns the handler with what every request shares, the parts
|
||||||
|
// in params among it. It builds the heuristics, each with the settings and
|
||||||
|
// parts it reads, and the lists they run in.
|
||||||
|
//
|
||||||
|
//nolint:funlen // a few lines for each heuristic, a list that grows with them
|
||||||
|
func newHandler(params Params) *handler {
|
||||||
|
awaitingReport, err := lru.New[string, *decision](maxAwaitingReport)
|
||||||
|
if err != nil {
|
||||||
|
panic(err) // New fails only for a size below one
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg := params.Config
|
||||||
|
h := &handler{
|
||||||
|
config: cfg,
|
||||||
|
requestLog: params.RequestLog,
|
||||||
|
processLog: params.ProcessLog,
|
||||||
|
now: params.Now,
|
||||||
|
metrics: params.Metrics,
|
||||||
|
awaitingReport: awaitingReport,
|
||||||
|
limiter: params.Limiter,
|
||||||
|
ledger: params.Ledger,
|
||||||
|
geojs: params.GeoJS,
|
||||||
|
anomalies: params.Anomalies,
|
||||||
|
lookupFile: params.LookupFile,
|
||||||
|
// The country list, the headers, the tag settings that name an asn=
|
||||||
|
// or country= tag and the biased thresholds act on the answer before
|
||||||
|
// the request goes on.
|
||||||
|
waitForGeoJS: len(cfg.ExclusivelyAllowedCountries) > 0 ||
|
||||||
|
cfg.AddLookupHeaders ||
|
||||||
|
cfg.TagsNeedLookup() ||
|
||||||
|
biasedThresholdsSet(cfg),
|
||||||
|
besideLookupLimit: cmp.Or(
|
||||||
|
params.BesideLookupTimeLimit,
|
||||||
|
besideLookupTimeLimit,
|
||||||
|
),
|
||||||
|
}
|
||||||
|
|
||||||
|
sharedBanMaker := banMaker{
|
||||||
|
ledger: params.Ledger,
|
||||||
|
limiter: params.Limiter,
|
||||||
|
metrics: params.Metrics,
|
||||||
|
alerts: params.Alerts,
|
||||||
|
observe: cfg.Observe,
|
||||||
|
banResponse: cfg.BanResponse,
|
||||||
|
banScopeV4Prefix: cfg.BanScopeV4Prefix,
|
||||||
|
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||||
|
}
|
||||||
|
hitNoter := reputationHitNoter{
|
||||||
|
metrics: params.Metrics,
|
||||||
|
alerts: params.Alerts,
|
||||||
|
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||||
|
}
|
||||||
|
|
||||||
|
h.trapPaths = trapPaths{paths: cfg.TrapPaths, banMaker: sharedBanMaker}
|
||||||
|
h.ruleFiles = ruleFiles{
|
||||||
|
rules: params.Rules,
|
||||||
|
metrics: params.Metrics,
|
||||||
|
banMaker: sharedBanMaker,
|
||||||
|
}
|
||||||
|
h.coreRuleSet = coreRuleSet{
|
||||||
|
inspector: params.CoreRuleSet,
|
||||||
|
metrics: params.Metrics,
|
||||||
|
alerts: params.Alerts,
|
||||||
|
mode: cfg.CoreRuleSetMode,
|
||||||
|
exemptPaths: cfg.CoreRuleSetExemptPaths,
|
||||||
|
scoreThreshold: cfg.CoreRuleSetScoreThreshold,
|
||||||
|
observe: cfg.Observe,
|
||||||
|
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||||
|
}
|
||||||
|
|
||||||
|
h.beforeLookup = []heuristic{
|
||||||
|
headerLimits{},
|
||||||
|
tagLists{lists: params.Lists},
|
||||||
|
routeWatchTags{
|
||||||
|
routeWatch: params.RouteWatch,
|
||||||
|
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||||
|
},
|
||||||
|
denyNets{netblocks: cfg.DenyNets},
|
||||||
|
activeBan{banMaker: sharedBanMaker},
|
||||||
|
}
|
||||||
|
h.afterLookup = []heuristic{
|
||||||
|
countries{allowed: cfg.ExclusivelyAllowedCountries},
|
||||||
|
blocklists{
|
||||||
|
lists: params.Lists,
|
||||||
|
action: cfg.BlocklistAction,
|
||||||
|
hitNoter: hitNoter,
|
||||||
|
},
|
||||||
|
dnsbl{
|
||||||
|
dnsbl: params.DNSBL,
|
||||||
|
action: cfg.DNSBLAbuseIPDBAction,
|
||||||
|
hitNoter: hitNoter,
|
||||||
|
},
|
||||||
|
abuseIPDB{
|
||||||
|
abuseIPDB: params.AbuseIPDB,
|
||||||
|
limiter: params.Limiter,
|
||||||
|
keySet: cfg.AbuseIPDBKey != "",
|
||||||
|
minScore: cfg.AbuseIPDBMinScore,
|
||||||
|
action: cfg.DNSBLAbuseIPDBAction,
|
||||||
|
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||||
|
hitNoter: hitNoter,
|
||||||
|
},
|
||||||
|
tagDeny{tags: cfg.TagDeny},
|
||||||
|
rateLimits{
|
||||||
|
limiter: params.Limiter,
|
||||||
|
exemptNets: cfg.RateLimitExemptNets,
|
||||||
|
exemptPaths: cfg.RateLimitExemptPaths,
|
||||||
|
banResponse: cfg.BanResponse,
|
||||||
|
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||||
|
biasedThresholds: biasedThresholds{
|
||||||
|
lists: params.Lists,
|
||||||
|
tagLimitPercent: cfg.TagLimitPercent,
|
||||||
|
asnBytesLimitPercent: cfg.ASNBytesLimitPercent,
|
||||||
|
countryBytesLimitPercent: cfg.CountryBytesLimitPercent,
|
||||||
|
unknownCountryLimitPercent: cfg.UnknownCountryLimitPercent,
|
||||||
|
blocklistAction: cfg.BlocklistAction,
|
||||||
|
blocklistLimitPercent: cfg.BlocklistLimitPercent,
|
||||||
|
dnsblAbuseIPDBAction: cfg.DNSBLAbuseIPDBAction,
|
||||||
|
dnsblAbuseIPDBLimitPercent: cfg.DNSBLAbuseIPDBLimitPercent,
|
||||||
|
},
|
||||||
|
banMaker: sharedBanMaker,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
h.besideLookup = []heuristic{h.trapPaths, h.ruleFiles, h.coreRuleSet}
|
||||||
|
h.afterBesideLookup = []heuristic{tagDeny{tags: cfg.TagDeny}}
|
||||||
|
h.afterAnswer = []heuristic{
|
||||||
|
refusalLimit{
|
||||||
|
limiter: params.Limiter,
|
||||||
|
perMinute: cfg.RefusalsLimitPerMinute,
|
||||||
|
banResponse: cfg.BanResponse,
|
||||||
|
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||||
|
banMaker: sharedBanMaker,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
h.onReport = []heuristic{
|
||||||
|
byteLimits{
|
||||||
|
limiter: params.Limiter,
|
||||||
|
bytesCounted: cfg.BytesCounted,
|
||||||
|
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||||
|
banMaker: sharedBanMaker,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
h.ownEndpoints = h.routes()
|
||||||
|
|
||||||
|
return h
|
||||||
|
}
|
||||||
|
|
||||||
|
// handler is waf's handler. It holds what every request shares; what
|
||||||
|
// belongs to one request is in a request.
|
||||||
|
type handler struct {
|
||||||
|
config *config.WAF
|
||||||
|
requestLog io.Writer
|
||||||
|
processLog *slog.Logger
|
||||||
|
now func() time.Time
|
||||||
|
metrics *metrics.Metrics
|
||||||
|
limiter *ratelimit.Limiter
|
||||||
|
ledger *bans.Ledger
|
||||||
|
geojs *geojs.GeoJS
|
||||||
|
anomalies *anomaly.Counters
|
||||||
|
lookupFile *lookup.File
|
||||||
|
// waitForGeoJS is true when a setting needs each request's answer from
|
||||||
|
// GeoJS before the request goes on.
|
||||||
|
waitForGeoJS bool
|
||||||
|
// besideLookupLimit is how long the heuristics of besideLookup have for a
|
||||||
|
// request, as Params.BesideLookupTimeLimit says.
|
||||||
|
besideLookupLimit time.Duration
|
||||||
|
// awaitingReport holds the decision on each answered question by its id
|
||||||
|
// until its report comes, and nil for a question still being handled.
|
||||||
|
awaitingReport *lru.Cache[string, *decision]
|
||||||
|
// ownEndpoints answers every request that is not a question, as routes
|
||||||
|
// lists the endpoints.
|
||||||
|
ownEndpoints http.Handler
|
||||||
|
|
||||||
|
// The lists of heuristics, each in the order its heuristics run. A client
|
||||||
|
// in SWW_WAF_ALLOW_NETS skips them all.
|
||||||
|
|
||||||
|
// beforeLookup are the heuristics that run before the client's AS number
|
||||||
|
// and country are looked up, so that a client they refuse is not looked
|
||||||
|
// up. header_limits comes first: it only measures and counts, and a
|
||||||
|
// request it refuses causes no other work.
|
||||||
|
beforeLookup []heuristic
|
||||||
|
// afterLookup are the heuristics that run once the lookup has been made.
|
||||||
|
afterLookup []heuristic
|
||||||
|
// besideLookup are the heuristics that only read the request. Their work
|
||||||
|
// runs beside the lookup and afterLookup, in findBesideLookup, which
|
||||||
|
// changes nothing of the request. Their check then applies what that work
|
||||||
|
// found, and runs only once waf takes it.
|
||||||
|
besideLookup []heuristic
|
||||||
|
// afterBesideLookup are the heuristics that run once afterLookup and
|
||||||
|
// besideLookup have both let the request through. tag_deny runs a second
|
||||||
|
// time, on the tag rule_files and core_rule_set can add.
|
||||||
|
afterBesideLookup []heuristic
|
||||||
|
// afterAnswer are the heuristics that run on a request once waf has
|
||||||
|
// answered it, before its decision line is written. They ban the client,
|
||||||
|
// and do not change what the request got: their verdict's action is not
|
||||||
|
// used.
|
||||||
|
afterAnswer []heuristic
|
||||||
|
// onReport are the heuristics that run on an allowed request when
|
||||||
|
// rproxy's report of it comes, before its report line is written. Like
|
||||||
|
// afterAnswer, they only ban.
|
||||||
|
onReport []heuristic
|
||||||
|
|
||||||
|
// The heuristics of besideLookup, whose work findBesideLookup does.
|
||||||
|
trapPaths trapPaths
|
||||||
|
ruleFiles ruleFiles
|
||||||
|
coreRuleSet coreRuleSet
|
||||||
|
}
|
||||||
@@ -344,8 +344,8 @@ func startProxyWithAlerts(
|
|||||||
return startRProxy(t, server.Addr, appURL, env), out, server, alertQueue
|
return startRProxy(t, server.Addr, appURL, env), out, server, alertQueue
|
||||||
}
|
}
|
||||||
|
|
||||||
// serve serves server on a port of its own until the test ends, and sets
|
// serve serves server on a port of its own until the test ends, and sets its
|
||||||
// server.Addr to where it listens.
|
// Addr to where it listens.
|
||||||
func serve(t *testing.T, server *wafAndParts) {
|
func serve(t *testing.T, server *wafAndParts) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -423,13 +423,10 @@ func startRProxy(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// wafAndParts is waf's server with the Params it was built from, so that a
|
// wafAndParts is waf's server with the Params it was built from, so that a
|
||||||
// test can read the parts the server uses. Addr is where it listens once serve
|
// test can read the parts the server uses.
|
||||||
// has it serving.
|
|
||||||
type wafAndParts struct {
|
type wafAndParts struct {
|
||||||
*waf.Server
|
*waf.Server
|
||||||
waf.Params
|
waf.Params
|
||||||
|
|
||||||
Addr string
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// banAbuseIPDBBlacklist does on w what a fetch of AbuseIPDB's blacklist with
|
// banAbuseIPDBBlacklist does on w what a fetch of AbuseIPDB's blacklist with
|
||||||
|
|||||||
@@ -1,235 +0,0 @@
|
|||||||
package waf
|
|
||||||
|
|
||||||
import (
|
|
||||||
"context"
|
|
||||||
"errors"
|
|
||||||
"net/http"
|
|
||||||
"strings"
|
|
||||||
"time"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/wafprotocol"
|
|
||||||
)
|
|
||||||
|
|
||||||
// answerQuestion runs the checks on the visitor's request a question
|
|
||||||
// carries, and answers rproxy what to do with it. A denied request's counting
|
|
||||||
// is finished here, and an allowed one's by its report.
|
|
||||||
func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
|
|
||||||
question, ok := h.readQuestion(w, r)
|
|
||||||
if !ok {
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
rq := h.startRequest(
|
|
||||||
w,
|
|
||||||
question.Visitor,
|
|
||||||
question.Client,
|
|
||||||
question.Peer,
|
|
||||||
true,
|
|
||||||
)
|
|
||||||
// These run even when a check panics. finish runs first, so that what is
|
|
||||||
// held has the finished log line. Go's server sends the answer only once
|
|
||||||
// this function has returned, so the report cannot come before it is held.
|
|
||||||
defer func() {
|
|
||||||
rq.finish()
|
|
||||||
rq.holdForReport()
|
|
||||||
}()
|
|
||||||
|
|
||||||
rq.site = r.Host
|
|
||||||
rq.line.WAFLine = requestlog.WAFLineDecision
|
|
||||||
refused := rq.check(r.Context())
|
|
||||||
rq.checked = time.Now()
|
|
||||||
underOwnPrefix := strings.HasPrefix(
|
|
||||||
r.URL.Path,
|
|
||||||
wafprotocol.OwnEndpointPrefix,
|
|
||||||
)
|
|
||||||
|
|
||||||
if refused == nil && !underOwnPrefix {
|
|
||||||
rq.allow()
|
|
||||||
// In observe mode, what enforce mode would have refused counts here.
|
|
||||||
rq.runBanningHeuristics(r.Context(), rq.h.afterAnswer)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
if refused != nil {
|
|
||||||
rq.deny(*refused)
|
|
||||||
} else {
|
|
||||||
// The app never sees this prefix. waf's own endpoints are at its
|
|
||||||
// own address, so this answers as a path with no endpoint does.
|
|
||||||
rq.line.Action = requestlog.ActionOwnEndpoint
|
|
||||||
wafprotocol.SetAnswerHeaders(
|
|
||||||
rq.out.Header(),
|
|
||||||
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
|
|
||||||
)
|
|
||||||
http.Error(
|
|
||||||
rq.out,
|
|
||||||
http.StatusText(http.StatusNotFound),
|
|
||||||
http.StatusNotFound,
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
rq.banAndCount(r.Context(), rq.h.afterAnswer)
|
|
||||||
}
|
|
||||||
|
|
||||||
// readQuestion reads a question, and reports whether waf takes it. It answers
|
|
||||||
// a question it does not take with 400 and no disposition. rproxy passes that
|
|
||||||
// 400 to the visitor, and SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION does
|
|
||||||
// not apply. The id of a question waf takes is held from here on, without a
|
|
||||||
// decision yet, so that another question with that id is refused while this
|
|
||||||
// one is being handled.
|
|
||||||
//
|
|
||||||
// A question whose body stops arriving gets no answer, and is not looked at.
|
|
||||||
func (h *handler) readQuestion(
|
|
||||||
w http.ResponseWriter, r *http.Request,
|
|
||||||
) (wafprotocol.Question, bool) {
|
|
||||||
question, err := wafprotocol.ReadQuestion(w, r)
|
|
||||||
if errors.Is(err, wafprotocol.ErrBodyStoppedArriving) {
|
|
||||||
h.processLog.Warn(
|
|
||||||
"a question's body stopped arriving",
|
|
||||||
"error",
|
|
||||||
err.Error(),
|
|
||||||
)
|
|
||||||
// This panic has Go's server close the connection without an answer,
|
|
||||||
// and log nothing.
|
|
||||||
panic(http.ErrAbortHandler)
|
|
||||||
}
|
|
||||||
|
|
||||||
if err != nil {
|
|
||||||
h.refuseQuestion(w, err.Error())
|
|
||||||
|
|
||||||
return question, false
|
|
||||||
}
|
|
||||||
|
|
||||||
held, _ := h.awaitingReport.ContainsOrAdd(question.RequestID, nil)
|
|
||||||
if held {
|
|
||||||
h.refuseQuestion(
|
|
||||||
w,
|
|
||||||
wafprotocol.RequestIDHeader+" is the id of a request waf holds",
|
|
||||||
)
|
|
||||||
|
|
||||||
return question, false
|
|
||||||
}
|
|
||||||
|
|
||||||
return question, true
|
|
||||||
}
|
|
||||||
|
|
||||||
// refuseQuestion answers a question waf does not take, for reason.
|
|
||||||
func (h *handler) refuseQuestion(w http.ResponseWriter, reason string) {
|
|
||||||
h.processLog.Warn("a question is refused", "reason", reason)
|
|
||||||
http.Error(w, reason, http.StatusBadRequest)
|
|
||||||
}
|
|
||||||
|
|
||||||
// allow answers a question with wafprotocol.DispositionAllow and, while
|
|
||||||
// SWW_WAF_ADD_LOOKUP_HEADERS is true, the client's AS number and country.
|
|
||||||
func (rq *request) allow() {
|
|
||||||
rq.allowed = true
|
|
||||||
|
|
||||||
answer := wafprotocol.Answer{Disposition: wafprotocol.DispositionAllow}
|
|
||||||
if rq.h.config.AddLookupHeaders {
|
|
||||||
answer.ASN = rq.lookupAnswer.ASN
|
|
||||||
answer.Country = rq.lookupAnswer.Country
|
|
||||||
}
|
|
||||||
|
|
||||||
wafprotocol.SetAnswerHeaders(rq.out.Header(), answer)
|
|
||||||
rq.out.WriteHeader(http.StatusOK)
|
|
||||||
}
|
|
||||||
|
|
||||||
// holdForReport holds a copy of what the report's steps need, by the request's
|
|
||||||
// id. A report only ever reaches that copy, never the request being handled.
|
|
||||||
func (rq *request) holdForReport() {
|
|
||||||
held := rq.decision
|
|
||||||
rq.h.awaitingReport.Add(rq.line.RequestID, &held)
|
|
||||||
}
|
|
||||||
|
|
||||||
// deny answers a question with what the visitor is to get for r.
|
|
||||||
func (rq *request) deny(r refusal) {
|
|
||||||
rq.refused = &r
|
|
||||||
|
|
||||||
if r.status == 0 {
|
|
||||||
// SWW_WAF_BAN_RESPONSE is close. Written past rq.out, since the log line's
|
|
||||||
// status stays 0: the visitor is sent nothing.
|
|
||||||
wafprotocol.SetAnswerHeaders(
|
|
||||||
rq.out.Header(),
|
|
||||||
wafprotocol.Answer{Disposition: wafprotocol.DispositionClose},
|
|
||||||
)
|
|
||||||
rq.out.ResponseWriter.WriteHeader(http.StatusOK)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
wafprotocol.SetAnswerHeaders(
|
|
||||||
rq.out.Header(),
|
|
||||||
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
|
|
||||||
)
|
|
||||||
http.Error(rq.out, http.StatusText(r.status), r.status)
|
|
||||||
}
|
|
||||||
|
|
||||||
// handleReport answers POST wafprotocol.ReportPath: it finishes the request
|
|
||||||
// the report names, if it holds that request's decision, and ignores the
|
|
||||||
// report otherwise.
|
|
||||||
func (h *handler) handleReport() http.HandlerFunc {
|
|
||||||
return func(w http.ResponseWriter, r *http.Request) {
|
|
||||||
report, err := wafprotocol.ReadReport(w, r)
|
|
||||||
if err != nil {
|
|
||||||
h.processLog.Warn("a report is refused", "error", err.Error())
|
|
||||||
http.Error(w, "the body is not a report", http.StatusBadRequest)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
held, found := h.awaitingReport.Peek(report.RequestID)
|
|
||||||
// Remove reports true to one report only, of several that name the
|
|
||||||
// id. A question still being handled is held as nil.
|
|
||||||
if found && held != nil &&
|
|
||||||
h.awaitingReport.Remove(report.RequestID) {
|
|
||||||
h.finishWith(r.Context(), held, report)
|
|
||||||
}
|
|
||||||
|
|
||||||
w.WriteHeader(http.StatusNoContent)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
// finishWith finishes a request with what its report says, and writes the
|
|
||||||
// request's last log line, which holds its decision's fields and the report's.
|
|
||||||
// For an allowed request it first runs the heuristics of onReport, and adds the
|
|
||||||
// request to its client's history, the anomaly counters and the metrics, with
|
|
||||||
// the status and bytes the report gives. A denied request's counting was
|
|
||||||
// finished at its decision. ctx is the report's own context.
|
|
||||||
func (h *handler) finishWith(
|
|
||||||
ctx context.Context,
|
|
||||||
held *decision,
|
|
||||||
report wafprotocol.Report,
|
|
||||||
) {
|
|
||||||
rq := &request{
|
|
||||||
decision: *held,
|
|
||||||
h: h,
|
|
||||||
out: &responseWriter{
|
|
||||||
status: report.Status,
|
|
||||||
bytes: report.ResponseBytes,
|
|
||||||
},
|
|
||||||
requestBytes: report.RequestBytes,
|
|
||||||
}
|
|
||||||
|
|
||||||
rq.line.WAFLine = requestlog.WAFLineReport
|
|
||||||
rq.line.Status = report.Status
|
|
||||||
rq.line.RequestBytes = report.RequestBytes
|
|
||||||
rq.line.ResponseBytes = report.ResponseBytes
|
|
||||||
rq.line.DurationTotalMS = report.DurationTotalMS
|
|
||||||
|
|
||||||
if rq.allowed {
|
|
||||||
rq.line.DurationUpstreamTotalMS = &report.DurationUpstreamTotalMS
|
|
||||||
|
|
||||||
rq.banAndCount(ctx, rq.h.onReport)
|
|
||||||
|
|
||||||
rq.h.metrics.RequestReported(
|
|
||||||
&rq.line,
|
|
||||||
time.Duration(report.DurationTotalMS*float64(time.Millisecond)),
|
|
||||||
time.Duration(
|
|
||||||
report.DurationUpstreamTotalMS*float64(time.Millisecond),
|
|
||||||
),
|
|
||||||
)
|
|
||||||
}
|
|
||||||
|
|
||||||
rq.writeLine()
|
|
||||||
}
|
|
||||||
@@ -17,9 +17,9 @@ import (
|
|||||||
// countedByRateLimits exempts it, by the client's limit percentages, which it
|
// countedByRateLimits exempts it, by the client's limit percentages, which it
|
||||||
// notes for the log line with the client's counts. A request that takes the
|
// notes for the log line with the client's counts. A request that takes the
|
||||||
// client over a rate limit, as its limit percentage lowers it, breaks it: it is
|
// client over a rate limit, as its limit percentage lowers it, breaks it: it is
|
||||||
// refused with SWW_WAF_BAN_RESPONSE, and bans the client. A request exempt
|
// refused, and bans the client. A request exempt from the rate limits is exempt
|
||||||
// from the rate limits is exempt from the byte limits too. Its figures are the
|
// from the byte limits too. Its figures are the window, the count and the
|
||||||
// window, the count and the limit.
|
// limit.
|
||||||
type rateLimits struct {
|
type rateLimits struct {
|
||||||
limiter *ratelimit.Limiter
|
limiter *ratelimit.Limiter
|
||||||
exemptNets []netip.Prefix // SWW_WAF_RATE_LIMIT_EXEMPT_NETS
|
exemptNets []netip.Prefix // SWW_WAF_RATE_LIMIT_EXEMPT_NETS
|
||||||
@@ -63,9 +63,7 @@ func (r rateLimits) check(
|
|||||||
r.banMaker.banForLimit(rq, now, hit, r.banResponse)
|
r.banMaker.banForLimit(rq, now, hit, r.banResponse)
|
||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionClientIsRateLimited,
|
action: requestlog.ActionClientIsRateLimited, figures: limitFigures(hit),
|
||||||
status: r.banResponse,
|
|
||||||
figures: limitFigures(hit),
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,12 +1,15 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
// The heuristic refusal_limit. It reads SWW_WAF_REFUSALS_LIMIT_PER_MINUTE.
|
// The heuristic refusal_limit. It reads SWW_WAF_REFUSALS_LIMIT_PER_MINUTE and
|
||||||
|
// SWW_WAF_BAN_RESPONSE.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
|
"net/http"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
)
|
)
|
||||||
|
|
||||||
// refusalLimit counts the request once it has been answered, if waf
|
// refusalLimit counts the request once it has been answered, if waf
|
||||||
@@ -15,11 +18,11 @@ import (
|
|||||||
// SWW_WAF_REFUSALS_LIMIT_PER_MINUTE such refusals of the client within a minute break
|
// SWW_WAF_REFUSALS_LIMIT_PER_MINUTE such refusals of the client within a minute break
|
||||||
// a limit, which bans the client. A client in SWW_WAF_ALLOW_NETS, which the
|
// a limit, which bans the client. A client in SWW_WAF_ALLOW_NETS, which the
|
||||||
// heuristics skip, is not counted, and nothing is while the limit is off.
|
// heuristics skip, is not counted, and nothing is while the limit is off.
|
||||||
// The ban's notes give the status the client was sent, or in observe mode
|
// Its figures are the window, the count and the limit.
|
||||||
// would have been. Its figures are the window, the count and the limit.
|
|
||||||
type refusalLimit struct {
|
type refusalLimit struct {
|
||||||
limiter *ratelimit.Limiter
|
limiter *ratelimit.Limiter
|
||||||
perMinute int64 // SWW_WAF_REFUSALS_LIMIT_PER_MINUTE
|
perMinute int64 // SWW_WAF_REFUSALS_LIMIT_PER_MINUTE
|
||||||
|
banResponse int // SWW_WAF_BAN_RESPONSE
|
||||||
ipv6GroupPrefix int // SWW_WAF_IPV6_GROUP_PREFIX
|
ipv6GroupPrefix int // SWW_WAF_IPV6_GROUP_PREFIX
|
||||||
banMaker banMaker
|
banMaker banMaker
|
||||||
}
|
}
|
||||||
@@ -48,7 +51,27 @@ func (r refusalLimit) check(
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
r.banMaker.banForLimit(rq, now, hit, rq.refusalStatus)
|
// What the client was sent, or in observe mode would have been.
|
||||||
|
status := rq.out.status
|
||||||
|
|
||||||
|
switch rq.wouldAction {
|
||||||
|
case requestlog.ActionBlockedByBlockRule,
|
||||||
|
requestlog.ActionBlockedByCoreRuleSet:
|
||||||
|
status = http.StatusForbidden
|
||||||
|
case requestlog.ActionTemporarilyBanned,
|
||||||
|
requestlog.ActionPermanentlyBanned:
|
||||||
|
status = r.banResponse
|
||||||
|
case requestlog.ActionForward,
|
||||||
|
requestlog.ActionRequestLineTooLong,
|
||||||
|
requestlog.ActionRequestHeaderTooLarge,
|
||||||
|
requestlog.ActionClientIsRateLimited,
|
||||||
|
requestlog.ActionClientDenied,
|
||||||
|
requestlog.ActionCountryDenied,
|
||||||
|
requestlog.ActionChecksTimedOut,
|
||||||
|
requestlog.ActionOwnEndpoint:
|
||||||
|
}
|
||||||
|
|
||||||
|
r.banMaker.banForLimit(rq, now, hit, status)
|
||||||
|
|
||||||
return &verdict{figures: limitFigures(hit)}
|
return &verdict{figures: limitFigures(hit)}
|
||||||
}
|
}
|
||||||
+25
-12
@@ -69,10 +69,6 @@ type decision struct {
|
|||||||
// its client's history counts. trapPaths and ruleFiles set attack,
|
// its client's history counts. trapPaths and ruleFiles set attack,
|
||||||
// ruleFiles blockedByBlockRule, and coreRuleSet blockedByCoreRuleSet.
|
// ruleFiles blockedByBlockRule, and coreRuleSet blockedByCoreRuleSet.
|
||||||
attack, blockedByBlockRule, blockedByCoreRuleSet bool
|
attack, blockedByBlockRule, blockedByCoreRuleSet bool
|
||||||
// refusalStatus is the status of the verdict that refused the request, or
|
|
||||||
// in observe mode would have: what the visitor was answered with, or would
|
|
||||||
// have been. check sets it.
|
|
||||||
refusalStatus int
|
|
||||||
// wouldAction is, in observe mode, the action enforce mode would have
|
// wouldAction is, in observe mode, the action enforce mode would have
|
||||||
// refused the request with, one of requestlog's Action constants, and ""
|
// refused the request with, one of requestlog's Action constants, and ""
|
||||||
// otherwise. check sets it.
|
// otherwise. check sets it.
|
||||||
@@ -207,8 +203,9 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
|
|||||||
|
|
||||||
// check is the one place where a request can be refused. It returns nil to let
|
// check is the one place where a request can be refused. It returns nil to let
|
||||||
// the request through. The heuristics of checkClient decide: a request they
|
// the request through. The heuristics of checkClient decide: a request they
|
||||||
// refuse is answered with the status and logged with the action of the verdict
|
// refuse is answered with SWW_WAF_BAN_RESPONSE, 403 for a block rule or the Core
|
||||||
// that refused it. In observe mode nothing is refused, and the log line names
|
// Rule Set, 503 for heuristics that ran out of time, or 414 or 431 for the
|
||||||
|
// header limits. In observe mode nothing is refused, and the log line names
|
||||||
// what enforce mode would have done. ctx is the request's own context.
|
// what enforce mode would have done. ctx is the request's own context.
|
||||||
func (rq *request) check(ctx context.Context) *refusal {
|
func (rq *request) check(ctx context.Context) *refusal {
|
||||||
v := rq.checkClient(ctx)
|
v := rq.checkClient(ctx)
|
||||||
@@ -216,16 +213,32 @@ func (rq *request) check(ctx context.Context) *refusal {
|
|||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
|
|
||||||
rq.refusalStatus = v.status
|
action := v.action
|
||||||
|
|
||||||
if rq.h.config.Observe {
|
switch {
|
||||||
rq.wouldAction = v.action
|
case rq.h.config.Observe:
|
||||||
rq.line.WouldAction = v.action
|
rq.wouldAction = action
|
||||||
|
rq.line.WouldAction = action
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
|
case action == requestlog.ActionBlockedByBlockRule ||
|
||||||
|
action == requestlog.ActionBlockedByCoreRuleSet:
|
||||||
|
return &refusal{status: http.StatusForbidden, action: action}
|
||||||
|
case action == requestlog.ActionChecksTimedOut:
|
||||||
|
return &refusal{
|
||||||
|
status: http.StatusServiceUnavailable,
|
||||||
|
action: action,
|
||||||
|
}
|
||||||
|
case action == requestlog.ActionRequestLineTooLong:
|
||||||
|
return &refusal{status: http.StatusRequestURITooLong, action: action}
|
||||||
|
case action == requestlog.ActionRequestHeaderTooLarge:
|
||||||
|
return &refusal{
|
||||||
|
status: http.StatusRequestHeaderFieldsTooLarge,
|
||||||
|
action: action,
|
||||||
|
}
|
||||||
|
default:
|
||||||
|
return rq.banResponse(action)
|
||||||
}
|
}
|
||||||
|
|
||||||
return &refusal{status: v.status, action: v.action}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// checkClient runs the heuristics on the request, and returns the first
|
// checkClient runs the heuristics on the request, and returns the first
|
||||||
|
|||||||
@@ -1,12 +1,10 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
// The heuristic rule_files. It reads the rules of the rule files, which
|
// The heuristic rule_files. It reads the rules of the rule files, which
|
||||||
// SWW_WAF_RULE_FILES_DIR and SWW_WAF_RULE_FILES_ENABLED name, and
|
// SWW_WAF_RULE_FILES_DIR and SWW_WAF_RULE_FILES_ENABLED name.
|
||||||
// SWW_WAF_BAN_RESPONSE.
|
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
"net/http"
|
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||||
@@ -19,9 +17,8 @@ import (
|
|||||||
// ruleFiles takes the rules of the rule files the request matched, which
|
// ruleFiles takes the rules of the rule files the request matched, which
|
||||||
// findBesideLookup found, notes their ids in the log line, adds the tag
|
// findBesideLookup found, notes their ids in the log line, adds the tag
|
||||||
// exploitsignature when it matches any, and refuses the request
|
// exploitsignature when it matches any, and refuses the request
|
||||||
// when the last rule it matches refuses it: with 403 and
|
// when the last rule it matches refuses it: with ActionBlockedByBlockRule for a
|
||||||
// ActionBlockedByBlockRule for a block rule, and for a ban rule with
|
// block rule, and for a ban rule with the action for the ban it makes. A ban
|
||||||
// SWW_WAF_BAN_RESPONSE and the action for the ban it makes. A ban
|
|
||||||
// rule bans the client's netblock for a clear sign of attack, or in observe
|
// rule bans the client's netblock for a clear sign of attack, or in observe
|
||||||
// mode raises the alert for the ban
|
// mode raises the alert for the ban
|
||||||
// it would have made. Either rule's match is noted as an offence, for the
|
// it would have made. Either rule's match is noted as an offence, for the
|
||||||
@@ -62,7 +59,6 @@ func (r ruleFiles) check(
|
|||||||
|
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionBlockedByBlockRule,
|
action: requestlog.ActionBlockedByBlockRule,
|
||||||
status: http.StatusForbidden,
|
|
||||||
figures: figures,
|
figures: figures,
|
||||||
}
|
}
|
||||||
case rules.ActionBan:
|
case rules.ActionBan:
|
||||||
@@ -74,7 +70,6 @@ func (r ruleFiles) check(
|
|||||||
now,
|
now,
|
||||||
bans.Notes{RuleID: last.ID, Target: last.Target},
|
bans.Notes{RuleID: last.ID, Target: last.Target},
|
||||||
),
|
),
|
||||||
status: r.banMaker.banResponse,
|
|
||||||
figures: figures,
|
figures: figures,
|
||||||
}
|
}
|
||||||
case rules.ActionLog:
|
case rules.ActionLog:
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
// The heuristic tag_deny. It reads SWW_WAF_TAG_DENY and SWW_WAF_BAN_RESPONSE.
|
// The heuristic tag_deny. It reads SWW_WAF_TAG_DENY.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -10,11 +10,10 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
)
|
)
|
||||||
|
|
||||||
// tagDeny refuses a request that carries a tag in SWW_WAF_TAG_DENY, with
|
// tagDeny refuses a request that carries a tag in SWW_WAF_TAG_DENY. Its figure is
|
||||||
// SWW_WAF_BAN_RESPONSE. Its figure is the first such tag, in sorted order.
|
// the first such tag, in sorted order.
|
||||||
type tagDeny struct {
|
type tagDeny struct {
|
||||||
tags []string // SWW_WAF_TAG_DENY
|
tags []string // SWW_WAF_TAG_DENY
|
||||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (tagDeny) name() string { return "tag_deny" }
|
func (tagDeny) name() string { return "tag_deny" }
|
||||||
@@ -28,7 +27,6 @@ func (t tagDeny) check(
|
|||||||
if slices.Contains(t.tags, tag) {
|
if slices.Contains(t.tags, tag) {
|
||||||
return &verdict{
|
return &verdict{
|
||||||
action: requestlog.ActionClientDenied,
|
action: requestlog.ActionClientDenied,
|
||||||
status: t.banResponse,
|
|
||||||
figures: map[string]any{"tag": tag},
|
figures: map[string]any{"tag": tag},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,7 +1,6 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
// The heuristic trap_paths. It reads SWW_WAF_TRAP_PATHS and
|
// The heuristic trap_paths. It reads SWW_WAF_TRAP_PATHS.
|
||||||
// SWW_WAF_BAN_RESPONSE.
|
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -13,10 +12,10 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// trapPaths refuses a request for a path in SWW_WAF_TRAP_PATHS, as trapPath
|
// trapPaths refuses a request for a path in SWW_WAF_TRAP_PATHS, as trapPath
|
||||||
// finds it, with SWW_WAF_BAN_RESPONSE. Such a request is a clear sign of
|
// finds it. Such a request is a clear sign of attack, as a ban rule's match is:
|
||||||
// attack, as a ban rule's match is: it bans the client's netblock, or in
|
// it bans the client's netblock, or in observe mode raises the alert for the
|
||||||
// observe mode raises the alert for the ban it would have made. The ban's
|
// ban it would have made. The ban's notes give the path as sent, and so does
|
||||||
// notes give the path as sent, and so does its figure.
|
// its figure.
|
||||||
type trapPaths struct {
|
type trapPaths struct {
|
||||||
paths []string // SWW_WAF_TRAP_PATHS
|
paths []string // SWW_WAF_TRAP_PATHS
|
||||||
banMaker banMaker
|
banMaker banMaker
|
||||||
@@ -42,7 +41,6 @@ func (t trapPaths) check(
|
|||||||
now,
|
now,
|
||||||
bans.Notes{TrapPath: path},
|
bans.Notes{TrapPath: path},
|
||||||
),
|
),
|
||||||
status: t.banMaker.banResponse,
|
|
||||||
figures: map[string]any{"path": path},
|
figures: map[string]any{"path": path},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+257
-357
@@ -1,48 +1,17 @@
|
|||||||
// Package waf is the server the waf command runs. It answers
|
|
||||||
// rproxy's questions, takes rproxy's reports and answers its own endpoints, and
|
|
||||||
// writes a log line for each. It also bans the clients on AbuseIPDB's
|
|
||||||
// blacklist each time the reputation package fetches it.
|
|
||||||
package waf
|
package waf
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"cmp"
|
|
||||||
"context"
|
"context"
|
||||||
"io"
|
"errors"
|
||||||
"log/slog"
|
"log/slog"
|
||||||
"net"
|
|
||||||
"net/http"
|
"net/http"
|
||||||
"net/netip"
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
lru "github.com/hashicorp/golang-lru/v2"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/metrics"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/routewatch"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/wafprotocol"
|
"sneak.berlin/go/smallwebwaf/internal/wafprotocol"
|
||||||
)
|
)
|
||||||
|
|
||||||
// MetricsPath is where the metrics are, for a request that carries
|
|
||||||
// SWW_WAF_METRICS_TOKEN.
|
|
||||||
const MetricsPath = "/_sww/metrics"
|
|
||||||
|
|
||||||
// BansPath is where an admin lists and adds bans, and, followed by / and
|
|
||||||
// a client's address, lifts them, with SWW_WAF_ADMIN_TOKEN.
|
|
||||||
const BansPath = "/_sww/bans"
|
|
||||||
|
|
||||||
// ClientsPath is where an admin asks what waf knows of a client, by
|
|
||||||
// the client's address after it, with SWW_WAF_ADMIN_TOKEN.
|
|
||||||
const ClientsPath = "/_sww/clients/"
|
|
||||||
|
|
||||||
const (
|
const (
|
||||||
// maxAwaitingReport is the most answered requests held until their
|
// maxAwaitingReport is the most answered requests held until their
|
||||||
// reports come. Past it, the request answered longest ago is dropped.
|
// reports come. Past it, the request answered longest ago is dropped.
|
||||||
@@ -54,79 +23,6 @@ const (
|
|||||||
wafIdleTimeout = 120 * time.Second
|
wafIdleTimeout = 120 * time.Second
|
||||||
)
|
)
|
||||||
|
|
||||||
// Params are what New needs.
|
|
||||||
type Params struct {
|
|
||||||
Config *config.WAF
|
|
||||||
// RequestLog receives two JSON lines per question, one at its decision and
|
|
||||||
// one when its report comes, and one per request to waf's own
|
|
||||||
// endpoints.
|
|
||||||
RequestLog io.Writer
|
|
||||||
// ProcessLog receives the process's own messages.
|
|
||||||
ProcessLog *slog.Logger
|
|
||||||
// Metrics count what waf does. Whoever builds them adds the metrics
|
|
||||||
// read from the other parts.
|
|
||||||
Metrics *metrics.Metrics
|
|
||||||
// Limiter is the table of clients the rate limits count, with each
|
|
||||||
// client's history.
|
|
||||||
Limiter *ratelimit.Limiter
|
|
||||||
// Ledger holds the bans.
|
|
||||||
Ledger *bans.Ledger
|
|
||||||
// Anomalies are the counters of the anomaly thresholds.
|
|
||||||
Anomalies *anomaly.Counters
|
|
||||||
// Lists are the lists fetched from URLs, DNSBL the DNSBL zones'
|
|
||||||
// verdicts, and AbuseIPDB AbuseIPDB's scores and the checks spent.
|
|
||||||
Lists *reputation.Lists
|
|
||||||
DNSBL *reputation.DNSBL
|
|
||||||
AbuseIPDB *reputation.AbuseIPDB
|
|
||||||
// RouteWatch asks routewatch about clients, and is nil while
|
|
||||||
// SWW_WAF_ROUTEWATCH_URL is unset.
|
|
||||||
RouteWatch *routewatch.Client
|
|
||||||
// GeoJS looks clients' AS numbers and countries up while
|
|
||||||
// SWW_WAF_LOOKUP_SOURCE is geojs. It is given AddLookup as its Answered
|
|
||||||
// and GeoJSFailed as its Failed.
|
|
||||||
GeoJS *geojs.GeoJS
|
|
||||||
// LookupFile is the lookup database they are looked up in while
|
|
||||||
// SWW_WAF_LOOKUP_SOURCE is file, and nil otherwise.
|
|
||||||
LookupFile *lookup.File
|
|
||||||
// CoreRuleSet is what inspects requests while
|
|
||||||
// SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
|
|
||||||
CoreRuleSet CoreRuleSetInspector
|
|
||||||
// Now tells the time by which requests are counted for the rate
|
|
||||||
// limits, bans are made and run out, and GeoJS's answers are kept,
|
|
||||||
// normally time.Now in UTC, the time the state files give.
|
|
||||||
Now func() time.Time
|
|
||||||
// Rules are the rule files' rules, which each request is checked
|
|
||||||
// against.
|
|
||||||
Rules *rules.Files
|
|
||||||
// Alerts receive the alert for each ban waf makes or makes
|
|
||||||
// permanent, for each count over an anomaly threshold, for each request
|
|
||||||
// whose client a blocklist, a DNSBL zone or AbuseIPDB lists, for each
|
|
||||||
// request the Core Rule Set scores at or over
|
|
||||||
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD, and for GeoJS failing, a fetch of a list
|
|
||||||
// failing, a query to a DNSBL zone or a check with AbuseIPDB failing,
|
|
||||||
// or the day's AbuseIPDB checks used up.
|
|
||||||
Alerts *alerts.Queue
|
|
||||||
// BesideLookupTimeLimit is how long the heuristics of besideLookup have
|
|
||||||
// for a request. Zero means besideLookupTimeLimit. Tests that run the
|
|
||||||
// Core Rule Set lengthen it, since the race detector and a busy host can
|
|
||||||
// slow an inspection past besideLookupTimeLimit.
|
|
||||||
BesideLookupTimeLimit time.Duration
|
|
||||||
}
|
|
||||||
|
|
||||||
// CoreRuleSetInspector inspects a request with the Core Rule Set, as
|
|
||||||
// coreruleset.CoreRuleSet's Inspect says. Tests give waf a stand-in for it.
|
|
||||||
type CoreRuleSetInspector interface {
|
|
||||||
Inspect(
|
|
||||||
r *http.Request, client netip.Addr, body io.Reader,
|
|
||||||
) (coreruleset.Result, []byte, error)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Server is the server waf runs.
|
|
||||||
type Server struct {
|
|
||||||
httpServer *http.Server
|
|
||||||
h *handler
|
|
||||||
}
|
|
||||||
|
|
||||||
// New returns the server `sww waf` runs. It answers
|
// New returns the server `sww waf` runs. It answers
|
||||||
// rproxy's questions, takes rproxy's reports and answers its own endpoints. It
|
// rproxy's questions, takes rproxy's reports and answers its own endpoints. It
|
||||||
// passes nothing to an app. rproxy applies the size and time limits to the
|
// passes nothing to an app. rproxy applies the size and time limits to the
|
||||||
@@ -134,258 +30,19 @@ type Server struct {
|
|||||||
// is far above rproxy's, and a question refused for its size would have rproxy
|
// is far above rproxy's, and a question refused for its size would have rproxy
|
||||||
// let the request through.
|
// let the request through.
|
||||||
func New(params Params) *Server {
|
func New(params Params) *Server {
|
||||||
s := &Server{h: newHandler(params)}
|
return &Server{
|
||||||
s.httpServer = &http.Server{
|
Server: &http.Server{
|
||||||
Addr: params.Config.ListenAddr,
|
Addr: params.Config.ListenAddr,
|
||||||
Handler: s,
|
Handler: newHandler(params),
|
||||||
ErrorLog: slog.NewLogLogger(
|
ErrorLog: slog.NewLogLogger(
|
||||||
params.ProcessLog.Handler(),
|
params.ProcessLog.Handler(),
|
||||||
slog.LevelWarn,
|
slog.LevelWarn,
|
||||||
),
|
),
|
||||||
ReadTimeout: wafTimeout,
|
ReadTimeout: wafTimeout,
|
||||||
WriteTimeout: wafTimeout,
|
WriteTimeout: wafTimeout,
|
||||||
IdleTimeout: wafIdleTimeout,
|
IdleTimeout: wafIdleTimeout,
|
||||||
}
|
|
||||||
|
|
||||||
return s
|
|
||||||
}
|
|
||||||
|
|
||||||
// Serve serves requests on listener until Shutdown or Close, as Go's
|
|
||||||
// http.Server.Serve does.
|
|
||||||
func (s *Server) Serve(listener net.Listener) error {
|
|
||||||
return s.httpServer.Serve(listener)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Shutdown stops serving once the requests in progress have finished, or
|
|
||||||
// when ctx is done, as Go's http.Server.Shutdown does.
|
|
||||||
func (s *Server) Shutdown(ctx context.Context) error {
|
|
||||||
return s.httpServer.Shutdown(ctx)
|
|
||||||
}
|
|
||||||
|
|
||||||
// Close stops serving at once and closes every connection, as Go's
|
|
||||||
// http.Server.Close does.
|
|
||||||
func (s *Server) Close() error {
|
|
||||||
return s.httpServer.Close()
|
|
||||||
}
|
|
||||||
|
|
||||||
// ServeHTTP handles one request to waf, as handler.ServeHTTP says.
|
|
||||||
func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|
||||||
s.h.ServeHTTP(w, r)
|
|
||||||
}
|
|
||||||
|
|
||||||
// newHandler returns the handler with what every request shares, the parts
|
|
||||||
// in params among it. It builds the heuristics, each with the settings and
|
|
||||||
// parts it reads, and the lists they run in.
|
|
||||||
//
|
|
||||||
//nolint:funlen // a few lines for each heuristic, a list that grows with them
|
|
||||||
func newHandler(params Params) *handler {
|
|
||||||
awaitingReport, err := lru.New[string, *decision](maxAwaitingReport)
|
|
||||||
if err != nil {
|
|
||||||
panic(err) // New fails only for a size below one
|
|
||||||
}
|
|
||||||
|
|
||||||
cfg := params.Config
|
|
||||||
h := &handler{
|
|
||||||
config: cfg,
|
|
||||||
requestLog: params.RequestLog,
|
|
||||||
processLog: params.ProcessLog,
|
|
||||||
now: params.Now,
|
|
||||||
metrics: params.Metrics,
|
|
||||||
awaitingReport: awaitingReport,
|
|
||||||
limiter: params.Limiter,
|
|
||||||
ledger: params.Ledger,
|
|
||||||
geojs: params.GeoJS,
|
|
||||||
anomalies: params.Anomalies,
|
|
||||||
lookupFile: params.LookupFile,
|
|
||||||
// The country list, the headers, the tag settings that name an asn=
|
|
||||||
// or country= tag and the biased thresholds act on the answer before
|
|
||||||
// the request goes on.
|
|
||||||
waitForGeoJS: len(cfg.ExclusivelyAllowedCountries) > 0 ||
|
|
||||||
cfg.AddLookupHeaders ||
|
|
||||||
cfg.TagsNeedLookup() ||
|
|
||||||
biasedThresholdsSet(cfg),
|
|
||||||
besideLookupLimit: cmp.Or(
|
|
||||||
params.BesideLookupTimeLimit,
|
|
||||||
besideLookupTimeLimit,
|
|
||||||
),
|
|
||||||
}
|
|
||||||
|
|
||||||
sharedBanMaker := banMaker{
|
|
||||||
ledger: params.Ledger,
|
|
||||||
limiter: params.Limiter,
|
|
||||||
metrics: params.Metrics,
|
|
||||||
alerts: params.Alerts,
|
|
||||||
observe: cfg.Observe,
|
|
||||||
banResponse: cfg.BanResponse,
|
|
||||||
banScopeV4Prefix: cfg.BanScopeV4Prefix,
|
|
||||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
|
||||||
}
|
|
||||||
hitNoter := reputationHitNoter{
|
|
||||||
metrics: params.Metrics,
|
|
||||||
alerts: params.Alerts,
|
|
||||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
|
||||||
}
|
|
||||||
|
|
||||||
h.trapPaths = trapPaths{paths: cfg.TrapPaths, banMaker: sharedBanMaker}
|
|
||||||
h.ruleFiles = ruleFiles{
|
|
||||||
rules: params.Rules,
|
|
||||||
metrics: params.Metrics,
|
|
||||||
banMaker: sharedBanMaker,
|
|
||||||
}
|
|
||||||
h.coreRuleSet = coreRuleSet{
|
|
||||||
inspector: params.CoreRuleSet,
|
|
||||||
metrics: params.Metrics,
|
|
||||||
alerts: params.Alerts,
|
|
||||||
mode: cfg.CoreRuleSetMode,
|
|
||||||
exemptPaths: cfg.CoreRuleSetExemptPaths,
|
|
||||||
scoreThreshold: cfg.CoreRuleSetScoreThreshold,
|
|
||||||
observe: cfg.Observe,
|
|
||||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
|
||||||
}
|
|
||||||
|
|
||||||
h.beforeLookup = []heuristic{
|
|
||||||
headerLimits{},
|
|
||||||
tagLists{lists: params.Lists},
|
|
||||||
routeWatchTags{
|
|
||||||
routeWatch: params.RouteWatch,
|
|
||||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
|
||||||
},
|
|
||||||
denyNets{netblocks: cfg.DenyNets, banResponse: cfg.BanResponse},
|
|
||||||
activeBan{banMaker: sharedBanMaker},
|
|
||||||
}
|
|
||||||
h.afterLookup = []heuristic{
|
|
||||||
countries{
|
|
||||||
allowed: cfg.ExclusivelyAllowedCountries,
|
|
||||||
banResponse: cfg.BanResponse,
|
|
||||||
},
|
|
||||||
blocklists{
|
|
||||||
lists: params.Lists,
|
|
||||||
action: cfg.BlocklistAction,
|
|
||||||
banResponse: cfg.BanResponse,
|
|
||||||
hitNoter: hitNoter,
|
|
||||||
},
|
|
||||||
dnsbl{
|
|
||||||
dnsbl: params.DNSBL,
|
|
||||||
action: cfg.DNSBLAbuseIPDBAction,
|
|
||||||
banResponse: cfg.BanResponse,
|
|
||||||
hitNoter: hitNoter,
|
|
||||||
},
|
|
||||||
abuseIPDB{
|
|
||||||
abuseIPDB: params.AbuseIPDB,
|
|
||||||
limiter: params.Limiter,
|
|
||||||
keySet: cfg.AbuseIPDBKey != "",
|
|
||||||
minScore: cfg.AbuseIPDBMinScore,
|
|
||||||
action: cfg.DNSBLAbuseIPDBAction,
|
|
||||||
banResponse: cfg.BanResponse,
|
|
||||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
|
||||||
hitNoter: hitNoter,
|
|
||||||
},
|
|
||||||
tagDeny{tags: cfg.TagDeny, banResponse: cfg.BanResponse},
|
|
||||||
rateLimits{
|
|
||||||
limiter: params.Limiter,
|
|
||||||
exemptNets: cfg.RateLimitExemptNets,
|
|
||||||
exemptPaths: cfg.RateLimitExemptPaths,
|
|
||||||
banResponse: cfg.BanResponse,
|
|
||||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
|
||||||
biasedThresholds: biasedThresholds{
|
|
||||||
lists: params.Lists,
|
|
||||||
tagLimitPercent: cfg.TagLimitPercent,
|
|
||||||
asnBytesLimitPercent: cfg.ASNBytesLimitPercent,
|
|
||||||
countryBytesLimitPercent: cfg.CountryBytesLimitPercent,
|
|
||||||
unknownCountryLimitPercent: cfg.UnknownCountryLimitPercent,
|
|
||||||
blocklistAction: cfg.BlocklistAction,
|
|
||||||
blocklistLimitPercent: cfg.BlocklistLimitPercent,
|
|
||||||
dnsblAbuseIPDBAction: cfg.DNSBLAbuseIPDBAction,
|
|
||||||
dnsblAbuseIPDBLimitPercent: cfg.DNSBLAbuseIPDBLimitPercent,
|
|
||||||
},
|
|
||||||
banMaker: sharedBanMaker,
|
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
h.besideLookup = []heuristic{h.trapPaths, h.ruleFiles, h.coreRuleSet}
|
|
||||||
h.afterBesideLookup = []heuristic{
|
|
||||||
tagDeny{tags: cfg.TagDeny, banResponse: cfg.BanResponse},
|
|
||||||
}
|
|
||||||
h.afterAnswer = []heuristic{
|
|
||||||
refusalLimit{
|
|
||||||
limiter: params.Limiter,
|
|
||||||
perMinute: cfg.RefusalsLimitPerMinute,
|
|
||||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
|
||||||
banMaker: sharedBanMaker,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
h.onReport = []heuristic{
|
|
||||||
byteLimits{
|
|
||||||
limiter: params.Limiter,
|
|
||||||
bytesCounted: cfg.BytesCounted,
|
|
||||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
|
||||||
banMaker: sharedBanMaker,
|
|
||||||
},
|
|
||||||
}
|
|
||||||
|
|
||||||
h.ownEndpoints = h.routes()
|
|
||||||
|
|
||||||
return h
|
|
||||||
}
|
|
||||||
|
|
||||||
// handler is waf's handler. It holds what every request shares; what
|
|
||||||
// belongs to one request is in a request.
|
|
||||||
type handler struct {
|
|
||||||
config *config.WAF
|
|
||||||
requestLog io.Writer
|
|
||||||
processLog *slog.Logger
|
|
||||||
now func() time.Time
|
|
||||||
metrics *metrics.Metrics
|
|
||||||
limiter *ratelimit.Limiter
|
|
||||||
ledger *bans.Ledger
|
|
||||||
geojs *geojs.GeoJS
|
|
||||||
anomalies *anomaly.Counters
|
|
||||||
lookupFile *lookup.File
|
|
||||||
// waitForGeoJS is true when a setting needs each request's answer from
|
|
||||||
// GeoJS before the request goes on.
|
|
||||||
waitForGeoJS bool
|
|
||||||
// besideLookupLimit is how long the heuristics of besideLookup have for a
|
|
||||||
// request, as Params.BesideLookupTimeLimit says.
|
|
||||||
besideLookupLimit time.Duration
|
|
||||||
// awaitingReport holds the decision on each answered question by its id
|
|
||||||
// until its report comes, and nil for a question still being handled.
|
|
||||||
awaitingReport *lru.Cache[string, *decision]
|
|
||||||
// ownEndpoints answers every request that is not a question, as routes
|
|
||||||
// lists the endpoints.
|
|
||||||
ownEndpoints http.Handler
|
|
||||||
|
|
||||||
// The lists of heuristics, each in the order its heuristics run. A client
|
|
||||||
// in SWW_WAF_ALLOW_NETS skips them all.
|
|
||||||
|
|
||||||
// beforeLookup are the heuristics that run before the client's AS number
|
|
||||||
// and country are looked up, so that a client they refuse is not looked
|
|
||||||
// up. header_limits comes first: it only measures and counts, and a
|
|
||||||
// request it refuses causes no other work.
|
|
||||||
beforeLookup []heuristic
|
|
||||||
// afterLookup are the heuristics that run once the lookup has been made.
|
|
||||||
afterLookup []heuristic
|
|
||||||
// besideLookup are the heuristics that only read the request. Their work
|
|
||||||
// runs beside the lookup and afterLookup, in findBesideLookup, which
|
|
||||||
// changes nothing of the request. Their check then applies what that work
|
|
||||||
// found, and runs only once waf takes it.
|
|
||||||
besideLookup []heuristic
|
|
||||||
// afterBesideLookup are the heuristics that run once afterLookup and
|
|
||||||
// besideLookup have both let the request through. tag_deny runs a second
|
|
||||||
// time, on the tag rule_files and core_rule_set can add.
|
|
||||||
afterBesideLookup []heuristic
|
|
||||||
// afterAnswer are the heuristics that run on a request once waf has
|
|
||||||
// answered it, before its decision line is written. They ban the client,
|
|
||||||
// and do not change what the request got: their verdict's action is not
|
|
||||||
// used.
|
|
||||||
afterAnswer []heuristic
|
|
||||||
// onReport are the heuristics that run on an allowed request when
|
|
||||||
// rproxy's report of it comes, before its report line is written. Like
|
|
||||||
// afterAnswer, they only ban.
|
|
||||||
onReport []heuristic
|
|
||||||
|
|
||||||
// The heuristics of besideLookup, whose work findBesideLookup does.
|
|
||||||
trapPaths trapPaths
|
|
||||||
ruleFiles ruleFiles
|
|
||||||
coreRuleSet coreRuleSet
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// ServeHTTP handles one request to waf: a request carrying
|
// ServeHTTP handles one request to waf: a request carrying
|
||||||
@@ -400,3 +57,246 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
|
|
||||||
h.ownEndpoints.ServeHTTP(w, r)
|
h.ownEndpoints.ServeHTTP(w, r)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// refusedWhateverTheAnswer reports whether the request is one that does not wait
|
||||||
|
// for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
|
||||||
|
// its client: a heuristic of afterLookup that implements refusesWithoutLookup
|
||||||
|
// says it will refuse it. In observe mode, which refuses nothing, none is.
|
||||||
|
// rproxy lets a request through when waf answers too late.
|
||||||
|
func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
|
||||||
|
if rq.h.config.Observe {
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, each := range rq.h.afterLookup {
|
||||||
|
knows, ok := each.(refusesWithoutLookup)
|
||||||
|
if ok && knows.refusesWhateverTheLookupAnswers(rq, now) {
|
||||||
|
return true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return false
|
||||||
|
}
|
||||||
|
|
||||||
|
// answerQuestion runs the checks on the visitor's request a question
|
||||||
|
// carries, and answers rproxy what to do with it. A denied request's counting
|
||||||
|
// is finished here, and an allowed one's by its report.
|
||||||
|
func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
|
||||||
|
question, ok := h.readQuestion(w, r)
|
||||||
|
if !ok {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
rq := h.startRequest(
|
||||||
|
w,
|
||||||
|
question.Visitor,
|
||||||
|
question.Client,
|
||||||
|
question.Peer,
|
||||||
|
true,
|
||||||
|
)
|
||||||
|
// These run even when a check panics. finish runs first, so that what is
|
||||||
|
// held has the finished log line. Go's server sends the answer only once
|
||||||
|
// this function has returned, so the report cannot come before it is held.
|
||||||
|
defer func() {
|
||||||
|
rq.finish()
|
||||||
|
rq.holdForReport()
|
||||||
|
}()
|
||||||
|
|
||||||
|
rq.site = r.Host
|
||||||
|
rq.line.WAFLine = requestlog.WAFLineDecision
|
||||||
|
refused := rq.check(r.Context())
|
||||||
|
rq.checked = time.Now()
|
||||||
|
underOwnPrefix := strings.HasPrefix(
|
||||||
|
r.URL.Path,
|
||||||
|
wafprotocol.OwnEndpointPrefix,
|
||||||
|
)
|
||||||
|
|
||||||
|
if refused == nil && !underOwnPrefix {
|
||||||
|
rq.allow()
|
||||||
|
// In observe mode, what enforce mode would have refused counts here.
|
||||||
|
rq.runBanningHeuristics(r.Context(), rq.h.afterAnswer)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if refused != nil {
|
||||||
|
rq.deny(*refused)
|
||||||
|
} else {
|
||||||
|
// The app never sees this prefix. waf's own endpoints are at its
|
||||||
|
// own address, so this answers as a path with no endpoint does.
|
||||||
|
rq.line.Action = requestlog.ActionOwnEndpoint
|
||||||
|
wafprotocol.SetAnswerHeaders(
|
||||||
|
rq.out.Header(),
|
||||||
|
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
|
||||||
|
)
|
||||||
|
http.Error(
|
||||||
|
rq.out,
|
||||||
|
http.StatusText(http.StatusNotFound),
|
||||||
|
http.StatusNotFound,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.banAndCount(r.Context(), rq.h.afterAnswer)
|
||||||
|
}
|
||||||
|
|
||||||
|
// readQuestion reads a question, and reports whether waf takes it. It answers
|
||||||
|
// a question it does not take with 400 and no disposition. rproxy passes that
|
||||||
|
// 400 to the visitor, and SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION does
|
||||||
|
// not apply. The id of a question waf takes is held from here on, without a
|
||||||
|
// decision yet, so that another question with that id is refused while this
|
||||||
|
// one is being handled.
|
||||||
|
//
|
||||||
|
// A question whose body stops arriving gets no answer, and is not looked at.
|
||||||
|
func (h *handler) readQuestion(
|
||||||
|
w http.ResponseWriter, r *http.Request,
|
||||||
|
) (wafprotocol.Question, bool) {
|
||||||
|
question, err := wafprotocol.ReadQuestion(w, r)
|
||||||
|
if errors.Is(err, wafprotocol.ErrBodyStoppedArriving) {
|
||||||
|
h.processLog.Warn(
|
||||||
|
"a question's body stopped arriving",
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
|
// This panic has Go's server close the connection without an answer,
|
||||||
|
// and log nothing.
|
||||||
|
panic(http.ErrAbortHandler)
|
||||||
|
}
|
||||||
|
|
||||||
|
if err != nil {
|
||||||
|
h.refuseQuestion(w, err.Error())
|
||||||
|
|
||||||
|
return question, false
|
||||||
|
}
|
||||||
|
|
||||||
|
held, _ := h.awaitingReport.ContainsOrAdd(question.RequestID, nil)
|
||||||
|
if held {
|
||||||
|
h.refuseQuestion(
|
||||||
|
w,
|
||||||
|
wafprotocol.RequestIDHeader+" is the id of a request waf holds",
|
||||||
|
)
|
||||||
|
|
||||||
|
return question, false
|
||||||
|
}
|
||||||
|
|
||||||
|
return question, true
|
||||||
|
}
|
||||||
|
|
||||||
|
// refuseQuestion answers a question waf does not take, for reason.
|
||||||
|
func (h *handler) refuseQuestion(w http.ResponseWriter, reason string) {
|
||||||
|
h.processLog.Warn("a question is refused", "reason", reason)
|
||||||
|
http.Error(w, reason, http.StatusBadRequest)
|
||||||
|
}
|
||||||
|
|
||||||
|
// allow answers a question with wafprotocol.DispositionAllow and, while
|
||||||
|
// SWW_WAF_ADD_LOOKUP_HEADERS is true, the client's AS number and country.
|
||||||
|
func (rq *request) allow() {
|
||||||
|
rq.allowed = true
|
||||||
|
|
||||||
|
answer := wafprotocol.Answer{Disposition: wafprotocol.DispositionAllow}
|
||||||
|
if rq.h.config.AddLookupHeaders {
|
||||||
|
answer.ASN = rq.lookupAnswer.ASN
|
||||||
|
answer.Country = rq.lookupAnswer.Country
|
||||||
|
}
|
||||||
|
|
||||||
|
wafprotocol.SetAnswerHeaders(rq.out.Header(), answer)
|
||||||
|
rq.out.WriteHeader(http.StatusOK)
|
||||||
|
}
|
||||||
|
|
||||||
|
// holdForReport holds a copy of what the report's steps need, by the request's
|
||||||
|
// id. A report only ever reaches that copy, never the request being handled.
|
||||||
|
func (rq *request) holdForReport() {
|
||||||
|
held := rq.decision
|
||||||
|
rq.h.awaitingReport.Add(rq.line.RequestID, &held)
|
||||||
|
}
|
||||||
|
|
||||||
|
// deny answers a question with what the visitor is to get for r.
|
||||||
|
func (rq *request) deny(r refusal) {
|
||||||
|
rq.refused = &r
|
||||||
|
|
||||||
|
if r.status == 0 {
|
||||||
|
// SWW_WAF_BAN_RESPONSE is close. Written past rq.out, since the log line's
|
||||||
|
// status stays 0: the visitor is sent nothing.
|
||||||
|
wafprotocol.SetAnswerHeaders(
|
||||||
|
rq.out.Header(),
|
||||||
|
wafprotocol.Answer{Disposition: wafprotocol.DispositionClose},
|
||||||
|
)
|
||||||
|
rq.out.ResponseWriter.WriteHeader(http.StatusOK)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
wafprotocol.SetAnswerHeaders(
|
||||||
|
rq.out.Header(),
|
||||||
|
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
|
||||||
|
)
|
||||||
|
http.Error(rq.out, http.StatusText(r.status), r.status)
|
||||||
|
}
|
||||||
|
|
||||||
|
// handleReport answers POST wafprotocol.ReportPath: it finishes the request
|
||||||
|
// the report names, if it holds that request's decision, and ignores the
|
||||||
|
// report otherwise.
|
||||||
|
func (h *handler) handleReport() http.HandlerFunc {
|
||||||
|
return func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
report, err := wafprotocol.ReadReport(w, r)
|
||||||
|
if err != nil {
|
||||||
|
h.processLog.Warn("a report is refused", "error", err.Error())
|
||||||
|
http.Error(w, "the body is not a report", http.StatusBadRequest)
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
held, found := h.awaitingReport.Peek(report.RequestID)
|
||||||
|
// Remove reports true to one report only, of several that name the
|
||||||
|
// id. A question still being handled is held as nil.
|
||||||
|
if found && held != nil &&
|
||||||
|
h.awaitingReport.Remove(report.RequestID) {
|
||||||
|
h.finishWith(r.Context(), held, report)
|
||||||
|
}
|
||||||
|
|
||||||
|
w.WriteHeader(http.StatusNoContent)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// finishWith finishes a request with what its report says, and writes the
|
||||||
|
// request's last log line, which holds its decision's fields and the report's.
|
||||||
|
// For an allowed request it first runs the heuristics of onReport, and adds the
|
||||||
|
// request to its client's history, the anomaly counters and the metrics, with
|
||||||
|
// the status and bytes the report gives. A denied request's counting was
|
||||||
|
// finished at its decision. ctx is the report's own context.
|
||||||
|
func (h *handler) finishWith(
|
||||||
|
ctx context.Context,
|
||||||
|
held *decision,
|
||||||
|
report wafprotocol.Report,
|
||||||
|
) {
|
||||||
|
rq := &request{
|
||||||
|
decision: *held,
|
||||||
|
h: h,
|
||||||
|
out: &responseWriter{
|
||||||
|
status: report.Status,
|
||||||
|
bytes: report.ResponseBytes,
|
||||||
|
},
|
||||||
|
requestBytes: report.RequestBytes,
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.line.WAFLine = requestlog.WAFLineReport
|
||||||
|
rq.line.Status = report.Status
|
||||||
|
rq.line.RequestBytes = report.RequestBytes
|
||||||
|
rq.line.ResponseBytes = report.ResponseBytes
|
||||||
|
rq.line.DurationTotalMS = report.DurationTotalMS
|
||||||
|
|
||||||
|
if rq.allowed {
|
||||||
|
rq.line.DurationUpstreamTotalMS = &report.DurationUpstreamTotalMS
|
||||||
|
|
||||||
|
rq.banAndCount(ctx, rq.h.onReport)
|
||||||
|
|
||||||
|
rq.h.metrics.RequestReported(
|
||||||
|
&rq.line,
|
||||||
|
time.Duration(report.DurationTotalMS*float64(time.Millisecond)),
|
||||||
|
time.Duration(
|
||||||
|
report.DurationUpstreamTotalMS*float64(time.Millisecond),
|
||||||
|
),
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
rq.writeLine()
|
||||||
|
}
|
||||||
@@ -703,9 +703,7 @@ func TestQuestionWhoseBodyStopsArrivingGetsNoAnswer(t *testing.T) {
|
|||||||
t,
|
t,
|
||||||
"",
|
"",
|
||||||
map[string]string{metricsToken: token},
|
map[string]string{metricsToken: token},
|
||||||
func(server *wafAndParts) {
|
func(server *wafAndParts) { server.ReadTimeout = readTimeout },
|
||||||
server.HTTPServer().ReadTimeout = readTimeout
|
|
||||||
},
|
|
||||||
)
|
)
|
||||||
|
|
||||||
conn := dial(t, s.addr)
|
conn := dial(t, s.addr)
|
||||||
@@ -1836,16 +1834,16 @@ func TestLogLevelHoldsBackNoRequestLine(t *testing.T) {
|
|||||||
func TestServerHasTheTimeLimitsSPECGives(t *testing.T) {
|
func TestServerHasTheTimeLimitsSPECGives(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
server := startWAF(t, "", nil).server.HTTPServer()
|
s := startWAF(t, "", nil)
|
||||||
|
|
||||||
if server.ReadTimeout != time.Minute ||
|
if s.server.ReadTimeout != time.Minute ||
|
||||||
server.WriteTimeout != time.Minute ||
|
s.server.WriteTimeout != time.Minute ||
|
||||||
server.IdleTimeout != 2*time.Minute {
|
s.server.IdleTimeout != 2*time.Minute {
|
||||||
t.Errorf(
|
t.Errorf(
|
||||||
"the read, write and idle limits are %v, %v and %v, want 1m, 1m and 2m",
|
"the read, write and idle limits are %v, %v and %v, want 1m, 1m and 2m",
|
||||||
server.ReadTimeout,
|
s.server.ReadTimeout,
|
||||||
server.WriteTimeout,
|
s.server.WriteTimeout,
|
||||||
server.IdleTimeout,
|
s.server.IdleTimeout,
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
+3
-8
@@ -8,8 +8,6 @@ set -eu
|
|||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||||
|
|
||||||
set -x
|
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
# Own line: a failing command substitution inside an argument does
|
# Own line: a failing command substitution inside an argument does
|
||||||
@@ -20,15 +18,12 @@ main() {
|
|||||||
[ -n "$version" ] || version="unknown"
|
[ -n "$version" ] || version="unknown"
|
||||||
revision="$(git rev-parse HEAD 2>/dev/null || true)"
|
revision="$(git rev-parse HEAD 2>/dev/null || true)"
|
||||||
[ -n "$revision" ] || revision="unknown"
|
[ -n "$revision" ] || revision="unknown"
|
||||||
created="$(date -u +%Y-%m-%d)"
|
created="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||||
docker build \
|
docker build --no-cache \
|
||||||
--build-arg VERSION="$version" \
|
--build-arg VERSION="$version" \
|
||||||
--build-arg REVISION="$revision" \
|
--build-arg REVISION="$revision" \
|
||||||
--build-arg CREATED="$created" \
|
--build-arg CREATED="$created" \
|
||||||
-t "git.eeqj.de/sneak/$("$SCRIPT_DIR/projectname")" \
|
-t "$("$SCRIPT_DIR/projectname")" .
|
||||||
-t "git.eeqj.de/sneak/$("$SCRIPT_DIR/projectname"):$version" \
|
|
||||||
-t "git.eeqj.de/sneak/$("$SCRIPT_DIR/projectname"):$(date -u +%Y-%m-%d)" \
|
|
||||||
.
|
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
+9
-2
@@ -14,9 +14,16 @@ set -eu
|
|||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||||
|
|
||||||
PUBLISH_IMAGE="git.eeqj.de/sneak/smallwebwaf"
|
|
||||||
|
|
||||||
main() {
|
main() {
|
||||||
|
if [ -z "${DOCKER_HOST:-}" ]; then
|
||||||
|
echo "script/publish: DOCKER_HOST is not set; set it to the Docker host that builds the image, such as ssh://user@build.example.com" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
if [ -z "${PUBLISH_IMAGE:-}" ]; then
|
||||||
|
echo "script/publish: PUBLISH_IMAGE is not set; set it to the image name in the registry, such as registry.example.com/smallwebwaf" >&2
|
||||||
|
exit 1
|
||||||
|
fi
|
||||||
|
|
||||||
cd "$ROOT"
|
cd "$ROOT"
|
||||||
|
|
||||||
# Own line: a failing command substitution inside an argument does
|
# Own line: a failing command substitution inside an argument does
|
||||||
|
|||||||
Reference in new issue
Block a user