Compare commits
13
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
b157359910 | ||
|
|
5dfa47c97b | ||
|
|
8219664ad1 | ||
|
|
58b9985b6d | ||
|
|
677df6a235 | ||
|
|
8188602a09 | ||
|
|
ee550be738 | ||
|
|
493aac02a1 | ||
|
|
4774518cb3 | ||
|
|
5d5e9542f8 | ||
|
|
1b9ca94774 | ||
|
|
9afbd1b683 | ||
|
|
51245804d0 |
No files matched your search
@@ -308,6 +308,7 @@ Every setting the code reads, with its default:
|
||||
| `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
|
||||
| `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
|
||||
| `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
||||
| `SWW_RPROXY_CONTACT_EMAIL` | unset | The email address the refusal pages link to. |
|
||||
| `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. |
|
||||
| `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `host1/waf`. |
|
||||
| `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
|
||||
|
||||
@@ -828,8 +828,8 @@ The supported apps are:
|
||||
- `vaultwarden`;
|
||||
- `wordpress`.
|
||||
|
||||
Each app's fixes are in one file, such as `internal/coreruleset/gitea.conf`, and
|
||||
one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||
Each app's fixes are in one file named for the app, such as
|
||||
`internal/coreruleset/apps/gitea.conf`. gitea's fixes are:
|
||||
|
||||
1. Ten parameters are not checked against the Core Rule Set's lists of system
|
||||
files (930120), shell paths (932160) and command names (932260), on any path:
|
||||
@@ -862,8 +862,8 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||
3. On gitea's paths, some fields skip the rules that refused ordinary gitea
|
||||
requests there. They are the fields where gitea sends a search, a file's name
|
||||
or text, a commit message, a branch name or a repository's name.
|
||||
`internal/coreruleset/gitea.conf` lists each rule and the request it refused.
|
||||
The fields are:
|
||||
`internal/coreruleset/apps/gitea.conf` lists each rule and the request it
|
||||
refused. The fields are:
|
||||
- `q` on gitea's searches of code, of issues and pull requests, and of
|
||||
commits. A code search holds code, such as `<script>`,
|
||||
`SELECT * FROM users WHERE` or `../`. The searches of issues and pull
|
||||
@@ -1048,7 +1048,7 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||
- The `Content-Range` header of an artifact upload from
|
||||
`actions/upload-artifact@v3` (920450).
|
||||
- The passwords in a body, for each rule that random passwords set off. The
|
||||
numbers are in `internal/coreruleset/gitea.conf`.
|
||||
numbers are in `internal/coreruleset/apps/gitea.conf`.
|
||||
- `password` on these paths:
|
||||
- `/user/login`, `/user/sign_up`, `/user/activate`;
|
||||
- `/user/link_account_signin` and `/user/link_account_signup`;
|
||||
@@ -1080,8 +1080,8 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
|
||||
such as `owner/python3-app` would have that push refused.
|
||||
|
||||
WordPress's fixes: on WordPress's paths, some fields skip the rules that refused
|
||||
ordinary WordPress requests there. `internal/coreruleset/wordpress.conf` lists
|
||||
each rule and the request it refused. The REST API has four addresses:
|
||||
ordinary WordPress requests there. `internal/coreruleset/apps/wordpress.conf`
|
||||
lists each rule and the request it refused. The REST API has four addresses:
|
||||
|
||||
- `/wp-json/...`
|
||||
- `/index.php/wp-json/...`
|
||||
@@ -1173,9 +1173,9 @@ Elsewhere WordPress's fixes do not apply. The paths are those of WordPress at
|
||||
the site root.
|
||||
|
||||
Drupal's fixes: on Drupal's paths, some fields skip the rules that refused
|
||||
ordinary Drupal requests there. `internal/coreruleset/drupal.conf` lists each
|
||||
rule and the request it refused. These query fields skip the lists of system
|
||||
files and command names (930120, 932120, 932250, 932260):
|
||||
ordinary Drupal requests there. `internal/coreruleset/apps/drupal.conf` lists
|
||||
each rule and the request it refused. These query fields skip the lists of
|
||||
system files and command names (930120, 932120, 932250, 932260):
|
||||
|
||||
- On every page under `/admin`: `title`, `name`, `user`, `filename`, `filemime`,
|
||||
`info`, `subject`, `author_name`, `search`, `q` and `destination`.
|
||||
@@ -1265,10 +1265,10 @@ first path segment of two or three lowercase letters, optionally followed by a
|
||||
hyphen and more letters, so `/api` counts too.
|
||||
|
||||
MediaWiki's fixes: on MediaWiki's paths, some fields skip the rules that refused
|
||||
ordinary MediaWiki requests there. `internal/coreruleset/mediawiki.conf` lists
|
||||
each rule and the request it refused. These fields skip the lists of system
|
||||
files and command names (930120, 932120, 932250, 932260), in the query or the
|
||||
body:
|
||||
ordinary MediaWiki requests there. `internal/coreruleset/apps/mediawiki.conf`
|
||||
lists each rule and the request it refused. These fields skip the lists of
|
||||
system files and command names (930120, 932120, 932250, 932260), in the query or
|
||||
the body:
|
||||
|
||||
- On MediaWiki's pages, which are `/index.php` and the paths under `/index.php/`
|
||||
and `/wiki/`:
|
||||
@@ -1374,9 +1374,9 @@ Elsewhere these fields keep every rule. The paths are those of MediaWiki at the
|
||||
site root or under `/w/`.
|
||||
|
||||
Mastodon's fixes: on Mastodon's paths, some fields skip the rules that refused
|
||||
ordinary Mastodon requests there. `internal/coreruleset/mastodon.conf` lists
|
||||
each rule and the request it refused. These fields skip the lists of system
|
||||
files and command names (930120, 932120, 932250, 932260), in the query:
|
||||
ordinary Mastodon requests there. `internal/coreruleset/apps/mastodon.conf`
|
||||
lists each rule and the request it refused. These fields skip the lists of
|
||||
system files and command names (930120, 932120, 932250, 932260), in the query:
|
||||
|
||||
- On the API's search, accounts and admin accounts, under `/api/v1/` and
|
||||
`/api/v2/`, the searches and names `q`, `acct`, `username`, `display_name`,
|
||||
@@ -1537,7 +1537,7 @@ every rule.
|
||||
|
||||
Vaultwarden's fixes: on Vaultwarden's paths, some fields skip the rules that
|
||||
refused ordinary Vaultwarden requests there.
|
||||
`internal/coreruleset/vaultwarden.conf` lists each rule and the request it
|
||||
`internal/coreruleset/apps/vaultwarden.conf` lists each rule and the request it
|
||||
refused. The apps encrypt vault items, folders, collections and sends before
|
||||
sending them, so those fields skip no rules. The fields below hold plain text.
|
||||
|
||||
@@ -1588,8 +1588,8 @@ has a parameter of the same name.
|
||||
|
||||
`token` in the body of `/admin`, the admin token typed on the admin page, skips
|
||||
each rule that random passwords set off. The numbers are in
|
||||
`internal/coreruleset/vaultwarden.conf`. It applies only when the query has no
|
||||
`token` parameter. `' OR '1'='1`, `../../etc/passwd`, `; cat /etc/passwd` and
|
||||
`internal/coreruleset/apps/vaultwarden.conf`. It applies only when the query has
|
||||
no `token` parameter. `' OR '1'='1`, `../../etc/passwd`, `; cat /etc/passwd` and
|
||||
`<img src=x onerror=x()>` sent as the token reach Vaultwarden;
|
||||
`<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
|
||||
|
||||
@@ -1611,7 +1611,7 @@ Elsewhere these fields keep every rule.
|
||||
|
||||
Nextcloud's fixes: on Nextcloud's paths, some fields skip the rules that refused
|
||||
ordinary Nextcloud requests there, and its WebDAV methods are allowed.
|
||||
`internal/coreruleset/nextcloud.conf` lists each rule and the request it
|
||||
`internal/coreruleset/apps/nextcloud.conf` lists each rule and the request it
|
||||
refused.
|
||||
|
||||
The sync client and the Files app use WebDAV on these paths:
|
||||
@@ -1621,7 +1621,8 @@ The sync client and the Files app use WebDAV on these paths:
|
||||
- `/remote.php/webdav/`;
|
||||
- `/public.php/webdav/`.
|
||||
|
||||
These methods are allowed there, besides those of the first change:
|
||||
These methods are allowed there, besides `PUT`, `PATCH` and `DELETE`, which
|
||||
every app gets:
|
||||
|
||||
- `PROPFIND` and `PROPPATCH`;
|
||||
- `MKCOL`;
|
||||
@@ -1765,7 +1766,7 @@ gets gitea's fixes too. Forgejo's own fixes hold only what Forgejo sends and
|
||||
gitea does not.
|
||||
|
||||
On Forgejo's paths, some fields skip the rules that refused ordinary Forgejo
|
||||
requests there. `internal/coreruleset/forgejo.conf` lists each rule and the
|
||||
requests there. `internal/coreruleset/apps/forgejo.conf` lists each rule and the
|
||||
request it refused.
|
||||
|
||||
`q` skips rules on some of Forgejo's searches. The searches are:
|
||||
@@ -1811,10 +1812,10 @@ The Core Rule Set reads the request without the `flash` cookie, in which Forgejo
|
||||
puts what gitea puts in `gitea_flash` (gitea's second fix).
|
||||
|
||||
Ghost's fixes: on Ghost's paths, some fields skip the rules that refused
|
||||
ordinary Ghost requests there. `internal/coreruleset/ghost.conf` lists each rule
|
||||
and the request it refused. Ghost's admin and integrations call the Admin API
|
||||
under `/ghost/api/admin/`. Themes and Ghost's scripts call the Content API under
|
||||
`/ghost/api/content/`, and members call `/members/api/`. The admin calls
|
||||
ordinary Ghost requests there. `internal/coreruleset/apps/ghost.conf` lists each
|
||||
rule and the request it refused. Ghost's admin and integrations call the Admin
|
||||
API under `/ghost/api/admin/`. Themes and Ghost's scripts call the Content API
|
||||
under `/ghost/api/content/`, and members call `/members/api/`. The admin calls
|
||||
ActivityPub, a service of its own, under `/.ghost/activitypub/`.
|
||||
|
||||
Eight rules refused words that name commands and files, such as `python3`,
|
||||
@@ -1952,9 +1953,9 @@ That name holds `.profile`, a file on the list of system files, so the name
|
||||
skips 930120 when a profile is saved.
|
||||
|
||||
The passwords skip each rule that random passwords set off. The numbers are in
|
||||
`internal/coreruleset/ghost.conf`. They are `password` when signing in, and the
|
||||
passwords sent when creating the first staff user, accepting an invitation, and
|
||||
resetting or changing a password. `' OR '1'='1`, `../../etc/passwd`,
|
||||
`internal/coreruleset/apps/ghost.conf`. They are `password` when signing in, and
|
||||
the passwords sent when creating the first staff user, accepting an invitation,
|
||||
and resetting or changing a password. `' OR '1'='1`, `../../etc/passwd`,
|
||||
`; cat /etc/passwd` and `<img src=x onerror=x()>` sent as a password reach
|
||||
Ghost; `<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
|
||||
|
||||
@@ -2430,8 +2431,9 @@ but the last below carries `X-Sww-Disposition`.
|
||||
carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the
|
||||
request it passes on. A header whose value is unknown is left out.
|
||||
- `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to
|
||||
get. `rproxy` gives them to the visitor, less this header. The status is one
|
||||
of:
|
||||
get. `rproxy` gives them to the visitor, less this header; a `403` or `429`
|
||||
gets `rproxy`'s own page as its body, linking `SWW_RPROXY_CONTACT_EMAIL` when
|
||||
set. The status is one of:
|
||||
- `SWW_WAF_BAN_RESPONSE`, for a refusal;
|
||||
- `403`, for a `block` rule or the Core Rule Set;
|
||||
- `414` or `431`, for a request over the header limits;
|
||||
|
||||
@@ -0,0 +1,217 @@
|
||||
services:
|
||||
exampleapp:
|
||||
# Your app, built from the Dockerfile in this directory, listening
|
||||
# on port 8081. Publish no port of it: it is reached through rproxy.
|
||||
build: .
|
||||
|
||||
rproxy:
|
||||
image: git.eeqj.de/sneak/smallwebwaf:latest
|
||||
environment:
|
||||
SWW_RPROXY_WAF_BACKEND: http://waf:8080
|
||||
SWW_RPROXY_UPSTREAM_URL: http://exampleapp:8081
|
||||
|
||||
# The address the refusal pages link to.
|
||||
# SWW_RPROXY_CONTACT_EMAIL: ops@example.com
|
||||
|
||||
# The netblocks whose X-Forwarded-For rproxy believes. By default
|
||||
# the private ranges, which cover a TLS-terminating proxy on the
|
||||
# Docker network. A proxy elsewhere needs its address added; a
|
||||
# list you give replaces the default, so repeat the private
|
||||
# ranges if they are still wanted. Set but empty, rproxy believes
|
||||
# no forwarded header.
|
||||
# SWW_RPROXY_TRUSTED_PROXIES: 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,203.0.113.5,2001:db8:1::/64
|
||||
|
||||
# By default rproxy passes a request to the app while waf gives
|
||||
# no answer. Uncomment this to answer 503 instead.
|
||||
# SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION: deny
|
||||
|
||||
# The largest request body (default 100M) and response body
|
||||
# (default 1G): a size with K, M or G, or `off`. Raise them for
|
||||
# uploads and downloads larger than that, such as git pushes and
|
||||
# clones or image layers.
|
||||
# SWW_RPROXY_MAX_REQUEST_SIZE: 2G
|
||||
# SWW_RPROXY_MAX_RESPONSE_SIZE: 4G
|
||||
|
||||
# How long a request may take to reach the app (default 60000,
|
||||
# one minute) and how long its answer may take to reach the
|
||||
# visitor (default 1800000, 30 minutes), in milliseconds or
|
||||
# `off`. Raise the first for slow uploads, the second for long
|
||||
# downloads.
|
||||
# SWW_RPROXY_MAX_REQUEST_TIME_MS: "600000"
|
||||
# SWW_RPROXY_MAX_RESPONSE_TIME_MS: "7200000"
|
||||
|
||||
# traefik terminates TLS and forwards to rproxy:8080 over its own
|
||||
# network, which the `networks` block at the end of this file names.
|
||||
# No port is published.
|
||||
networks:
|
||||
- default
|
||||
- traefik
|
||||
labels:
|
||||
traefik.enable: "true"
|
||||
# rproxy is on two networks; traefik reaches it on this one.
|
||||
traefik.docker.network: traefik
|
||||
traefik.http.routers.exampleapp.rule: Host(`app.example.com`)
|
||||
traefik.http.routers.exampleapp.entrypoints: websecure
|
||||
traefik.http.routers.exampleapp.tls.certresolver: letsencrypt
|
||||
traefik.http.services.exampleapp.loadbalancer.server.port: "8080"
|
||||
|
||||
waf:
|
||||
image: git.eeqj.de/sneak/smallwebwaf:latest
|
||||
environment:
|
||||
SWW_COMMAND: waf
|
||||
|
||||
# `observe` refuses nothing and makes no ban; each log line says
|
||||
# what `enforce` (the default) would have done. Start here to
|
||||
# see what the defaults would refuse on your site.
|
||||
# SWW_WAF_MODE: observe
|
||||
|
||||
# waf's own messages below `info` (the default), while finding
|
||||
# out why something is refused.
|
||||
# SWW_WAF_LOG_LEVEL: debug
|
||||
|
||||
# Netblocks whose clients skip every check, such as monitoring.
|
||||
# SWW_WAF_ALLOW_NETS: 192.0.2.10,2001:db8:2::/48
|
||||
|
||||
# Netblocks whose clients are refused.
|
||||
# SWW_WAF_DENY_NETS: 198.51.100.128/25,2001:db8:3::/48
|
||||
|
||||
# Netblocks exempt from rate limiting and the byte limits, such
|
||||
# as the office and an automation host that calls an API. Their
|
||||
# requests are still checked for attacks.
|
||||
# SWW_WAF_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7
|
||||
|
||||
# Refuse every request from a country, by its two-letter code.
|
||||
# This refuses North Korea, Cuba, Russia and Iran. A client the
|
||||
# lookup cannot place gets through; see SWW_WAF_LOOKUP_SOURCE.
|
||||
# SWW_WAF_TAG_DENY: country=KP,country=CU,country=RU,country=IR
|
||||
|
||||
# Or from a network, by its AS number: here two hosting networks
|
||||
# that send only scrapers, M247 (AS9009) and WorldStream
|
||||
# (AS49981). Countries and networks go in the one list together.
|
||||
# SWW_WAF_TAG_DENY: asn=9009,asn=49981,country=KP,country=RU
|
||||
|
||||
# Or let only these countries through; a client the lookup
|
||||
# cannot place is refused too.
|
||||
# SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES: us,de
|
||||
|
||||
# Address lists that each tag their clients' requests, as
|
||||
# name=URL pairs; this one tags Tor exit nodes. A tag is refused
|
||||
# by adding its name to SWW_WAF_TAG_DENY, such as
|
||||
# `torexit,country=KP`, or given a share of the limits in
|
||||
# SWW_WAF_TAG_LIMIT_PERCENT, such as `torexit:25`.
|
||||
# SWW_WAF_TAG_LISTS: torexit=https://check.torproject.org/torbulkexitlist
|
||||
|
||||
# A percentage, from 1 to 99, of every limit for a tag: here
|
||||
# half for DigitalOcean (AS14061), a quarter for OVH (AS16276)
|
||||
# and a quarter for China. The lowest of a client's tags applies.
|
||||
# SWW_WAF_TAG_LIMIT_PERCENT: asn=14061:50,asn=16276:25,country=CN:25
|
||||
|
||||
# The most requests a client may make in a minute, an hour and a
|
||||
# day (defaults 120, 1500 and 6000), a count or `off`; the next
|
||||
# one bans it. Lower them for a small site, raise them for a
|
||||
# busy API.
|
||||
# SWW_WAF_RATE_LIMIT_PER_MINUTE: "60"
|
||||
# SWW_WAF_RATE_LIMIT_PER_HOUR: "600"
|
||||
# SWW_WAF_RATE_LIMIT_PER_DAY: "3000"
|
||||
|
||||
# Path prefixes exempt from rate limiting and the byte limits,
|
||||
# such as static assets that one page loads many of.
|
||||
# SWW_WAF_RATE_LIMIT_EXEMPT_PATHS: /assets/,/static/
|
||||
|
||||
# Paths only scanners ask for; one request bans the client. Pick
|
||||
# paths your app never serves.
|
||||
# SWW_WAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php,/wp-admin/
|
||||
|
||||
# How a refused client is answered: 403 (the default), 429 or
|
||||
# `close`, which drops the connection.
|
||||
# SWW_WAF_BAN_RESPONSE: "429"
|
||||
|
||||
# The ban for a first broken limit (default 1h) and for a first
|
||||
# clear sign of attack (default 7d): a duration with m, h or d.
|
||||
# A repeat within SWW_WAF_LIMIT_BAN_REPEAT_WINDOW (default 24h)
|
||||
# gets a ban three times as long.
|
||||
# SWW_WAF_LIMIT_BAN_DURATION: 30m
|
||||
# SWW_WAF_ATTACK_BAN_DURATION: 30d
|
||||
|
||||
# The Core Rule Set, which finds attacks in requests. `block`
|
||||
# (the default) refuses a match, `detect` only logs and alerts
|
||||
# it, `off` runs no rule. Use `detect` while finding out what it
|
||||
# would refuse of your app's ordinary requests.
|
||||
# SWW_WAF_CORE_RULE_SET_MODE: detect
|
||||
|
||||
# Path prefixes the Core Rule Set does not inspect, for a part of
|
||||
# the app that it keeps matching, such as an editor that posts
|
||||
# HTML or code.
|
||||
# SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS: /api/editor/,/admin/
|
||||
|
||||
# The apps each hostname serves, as hostname:app pairs separated
|
||||
# by semicolons, so that each hostname gets its own apps' fixes
|
||||
# to the attack detection and no other's. "Attack detection" in
|
||||
# the README lists the apps.
|
||||
# SWW_WAF_APPS: git.example.com:gitea;blog.example.com:wordpress
|
||||
|
||||
# Blocklists of addresses and netblocks, text files fetched once
|
||||
# a day, whose clients are refused.
|
||||
# SWW_WAF_BLOCKLIST_URLS: https://www.spamhaus.org/drop/drop.txt,https://www.spamhaus.org/drop/dropv6.txt
|
||||
|
||||
# With the key of an AbuseIPDB account, a client that has
|
||||
# committed an offence is checked there in the background; one
|
||||
# scored 75 or more (SWW_WAF_ABUSEIPDB_MIN_SCORE) gets what
|
||||
# SWW_WAF_DNSBL_ABUSEIPDB_ACTION says: a quarter of every limit
|
||||
# by default, or `deny`. At most 900 checks a day
|
||||
# (SWW_WAF_ABUSEIPDB_DAILY_BUDGET), under the free plan's 1000.
|
||||
# SWW_WAF_ABUSEIPDB_KEY: api-key-.....
|
||||
# SWW_WAF_DNSBL_ABUSEIPDB_ACTION: deny
|
||||
|
||||
# By default waf looks each client's address up at GeoJS, a web
|
||||
# service, for its country and AS number. Set `off`, it makes no
|
||||
# external requests for lookups. Every setting at its default
|
||||
# works without them; one that needs a lookup, such as a country
|
||||
# or ASN setting above, then stops the start. Set `file`, it
|
||||
# reads an IPinfo Lite database instead, mounted by its
|
||||
# directory as under `volumes` below, so that a replaced file is
|
||||
# seen.
|
||||
# SWW_WAF_LOOKUP_SOURCE: "off"
|
||||
# SWW_WAF_LOOKUP_SOURCE: file
|
||||
# SWW_WAF_LOOKUP_DB_PATH: /var/lib/ipinfo/ipinfo_lite.mmdb
|
||||
|
||||
# Alerts about bans, attacks and failures, to an ntfy topic, a
|
||||
# Slack incoming webhook or any webhook. The ntfy token is the
|
||||
# access token ntfy issued for the topic.
|
||||
# SWW_WAF_ALERT_NTFY_URL: https://ntfy.sh/my-alerts
|
||||
# SWW_WAF_ALERT_NTFY_TOKEN: hunter2
|
||||
# SWW_WAF_ALERT_SLACK_WEBHOOK_URL: https://hooks.slack.com/services/...
|
||||
|
||||
# Bearer tokens you make up, at least 32 characters each. The
|
||||
# admin token switches on the ban and client endpoints, through
|
||||
# which a ban is lifted, your own included, as
|
||||
# docs/admin-endpoints.md describes. The metrics token switches
|
||||
# on /_sww/metrics.
|
||||
# SWW_WAF_ADMIN_TOKEN: hunter2
|
||||
# SWW_WAF_METRICS_TOKEN: hunter2
|
||||
|
||||
# Any setting can be read from a file instead, by adding _FILE
|
||||
# to its name: the file's contents are the value, less one
|
||||
# newline at the end. That keeps a secret out of this file and
|
||||
# out of `docker inspect`. Mount the files as under `volumes`
|
||||
# below, and set each setting in one form only.
|
||||
# SWW_WAF_ALERT_NTFY_TOKEN_FILE: /etc/sww/tokens/ntfy
|
||||
# SWW_WAF_ADMIN_TOKEN_FILE: /etc/sww/tokens/admin
|
||||
# SWW_WAF_METRICS_TOKEN_FILE: /etc/sww/tokens/metrics
|
||||
# SWW_WAF_ABUSEIPDB_KEY_FILE: /etc/sww/tokens/abuseipdb
|
||||
volumes:
|
||||
# waf's state files: its clients and bans, kept across restarts.
|
||||
- waf-state:/var/lib/sww
|
||||
# Token files, each owned by uid 65532 (the sww user) with mode
|
||||
# 0400, for the _FILE settings above.
|
||||
# - /srv/sww/tokens:/etc/sww/tokens:ro
|
||||
# The IPinfo Lite database, for SWW_WAF_LOOKUP_SOURCE=file.
|
||||
# - /srv/ipinfo:/var/lib/ipinfo:ro
|
||||
|
||||
volumes:
|
||||
waf-state:
|
||||
|
||||
# traefik's network, made outside this file.
|
||||
networks:
|
||||
traefik:
|
||||
external: true
|
||||
@@ -483,6 +483,8 @@ start, and an invalid value stops the start. Any can be given as a file, as a
|
||||
answer `rproxy` gives itself always has at least 10 seconds to be sent, and so
|
||||
do the status and headers of the app's answer. A time in milliseconds or
|
||||
`off`.
|
||||
- `SWW_RPROXY_CONTACT_EMAIL` (default unset): the email address the refusal
|
||||
pages link to.
|
||||
- `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as
|
||||
`:9000`.
|
||||
- `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An
|
||||
|
||||
@@ -46,6 +46,9 @@ type RProxy struct {
|
||||
// (SWW_RPROXY_MAX_RESPONSE_TIME_MS).
|
||||
MaxRequestTime time.Duration
|
||||
MaxResponseTime time.Duration
|
||||
// ContactEmail is the address the pages a refused visitor gets link to,
|
||||
// or "" for no link (SWW_RPROXY_CONTACT_EMAIL). It is taken as given.
|
||||
ContactEmail string
|
||||
|
||||
// settings are the values read, as given or by default, for the log line
|
||||
// at start.
|
||||
@@ -108,6 +111,7 @@ func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
|
||||
"SWW_RPROXY_MAX_RESPONSE_TIME_MS",
|
||||
"1800000",
|
||||
),
|
||||
ContactEmail: env.value("SWW_RPROXY_CONTACT_EMAIL", ""),
|
||||
}
|
||||
|
||||
if env.err != nil {
|
||||
|
||||
@@ -25,6 +25,7 @@ const (
|
||||
rproxyMaxResponse = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
||||
rproxyRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
||||
rproxyAnswerTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
||||
rproxyContactEmail = "SWW_RPROXY_CONTACT_EMAIL"
|
||||
backendURL = "http://waf:8080"
|
||||
rproxyNoAnswerAddr = ":8080"
|
||||
rproxyNoAnswerApp = "http://127.0.0.1:8081"
|
||||
@@ -250,6 +251,7 @@ func TestRProxySettingsAreLoggedByName(t *testing.T) {
|
||||
rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, wafBackendTimeout,
|
||||
rproxyNoAnswer,
|
||||
rproxyMaxRequest, rproxyMaxResponse, rproxyRequestTime, rproxyAnswerTime,
|
||||
rproxyContactEmail,
|
||||
} {
|
||||
if !slices.Contains(names, name) {
|
||||
t.Errorf("%s is not logged, only %v", name, names)
|
||||
|
||||
+1621
-1444
File diff suppressed because it is too large.
Load diff
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -37,9 +37,10 @@
|
||||
SecRule &TX:app_nextcloud "@eq 0" \
|
||||
"id:900090,phase:1,pass,nolog,skipAfter:END_NEXTCLOUD"
|
||||
|
||||
# The sync client and the Files app use these WebDAV methods besides those
|
||||
# the first change allows, on /remote.php/dav/ and /public.php/dav/ and on the
|
||||
# older /remote.php/webdav/ and /public.php/webdav/. 911100 refused each:
|
||||
# The sync client and the Files app use these WebDAV methods besides PUT,
|
||||
# PATCH and DELETE, which sww allows for every app, on /remote.php/dav/ and
|
||||
# /public.php/dav/ and on the older /remote.php/webdav/ and /public.php/webdav/.
|
||||
# 911100 refused each:
|
||||
# PROPFIND, PROPPATCH: listing a folder, marking a file a favourite
|
||||
# MKCOL: making a folder, and starting a chunked upload
|
||||
# COPY, MOVE: copying, moving and renaming, and ending a chunked upload
|
||||
File renamed without changes.
File renamed without changes.
@@ -12,11 +12,10 @@
|
||||
package coreruleset
|
||||
|
||||
import (
|
||||
_ "embed"
|
||||
"embed"
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"maps"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
@@ -35,7 +34,7 @@ import (
|
||||
// directives are the Core Rule Set as sww runs it, with the
|
||||
// paranoia level for %d, bodyDirectives for the first %s while
|
||||
// SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set, the exclusions file for
|
||||
// the second, and the directives of every app in apps for the third.
|
||||
// the second, and the fixes of every app in appFixes for the third.
|
||||
// Each rule sww adds has an id from 900000 to 900999, the ids the
|
||||
// Core Rule Set keeps for the rules that set it up, which
|
||||
// SWW_WAF_CORE_RULE_SET_DISABLED_RULES and the exclusions file refuse, so that no
|
||||
@@ -54,13 +53,12 @@ Include @crs-setup.conf.example
|
||||
SecAction "id:900000,phase:1,pass,nolog,\
|
||||
setvar:tx.blocking_paranoia_level=%d"
|
||||
|
||||
# The first change: PUT, PATCH and DELETE are allowed besides GET, HEAD,
|
||||
# POST and OPTIONS.
|
||||
# PUT, PATCH and DELETE are allowed besides GET, HEAD, POST and OPTIONS.
|
||||
SecAction "id:900200,phase:1,pass,nolog,\
|
||||
setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE'"
|
||||
|
||||
# The second: Expect and Content-Encoding are taken off the Core Rule Set's
|
||||
# list of the headers it refuses. Content-Encoding goes back on it for a
|
||||
# Expect and Content-Encoding are taken off the Core Rule Set's list of the
|
||||
# headers it refuses. Content-Encoding goes back on it for a
|
||||
# body the Core Rule Set reads (900260 in bodyDirectives).
|
||||
SecAction "id:900250,phase:1,pass,nolog,\
|
||||
setvar:'tx.restricted_headers_basic=/proxy/ /lock-token/ /content-range/ \
|
||||
@@ -86,18 +84,17 @@ SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
|
||||
# The apps' fixes come before the rules for the same reason.
|
||||
%s
|
||||
|
||||
# The fifth: only the rules for requests are loaded, and no response is
|
||||
# inspected.
|
||||
# Only the rules for requests are loaded, and no response is inspected.
|
||||
Include @owasp_crs/REQUEST-*.conf
|
||||
|
||||
# The third: redirect_uri is not checked for a URL naming an IP address or
|
||||
# redirect_uri is not checked for a URL naming an IP address or
|
||||
# localhost. Coraza matches a parameter name here without regard to case.
|
||||
# ARGS holds the fields of a form data or multipart body Coraza reads as
|
||||
# well as the query parameters, so a field of this name is left out too.
|
||||
SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri"
|
||||
SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri"
|
||||
|
||||
# The fourth: Referer is not checked for a Unix command without arguments
|
||||
# Referer is not checked for a Unix command without arguments
|
||||
# or for Java starting a process.
|
||||
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
|
||||
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
|
||||
@@ -122,9 +119,9 @@ SecRule REQUEST_HEADERS:Content-Type \
|
||||
"@rx ^(?:application|text)/(?:[a-z0-9.-]+[+])?json" \
|
||||
"id:900420,phase:1,pass,nolog,t:none,t:lowercase,ctl:requestBodyProcessor=JSON"
|
||||
|
||||
# The rest of the second change: Content-Encoding is refused again on a
|
||||
# body of a kind the Core Rule Set reads, since a compressed body cannot be
|
||||
# inspected.
|
||||
# Content-Encoding goes back on the list of the headers the Core Rule Set
|
||||
# refuses (900250 in directives) for a body of a kind it reads, since a
|
||||
# compressed body cannot be inspected.
|
||||
SecRule REQBODY_PROCESSOR "@rx ^(?:URLENCODED|MULTIPART|JSON|XML)$" \
|
||||
"id:900260,phase:1,pass,nolog,\
|
||||
setvar:'tx.restricted_headers_basic=%%{tx.restricted_headers_basic} \
|
||||
@@ -145,15 +142,19 @@ SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
|
||||
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
||||
`
|
||||
|
||||
// app is an app whose ordinary requests the Core Rule Set would refuse,
|
||||
// with the fixes that let them pass.
|
||||
// appFixes holds the fixes of each app whose ordinary requests the Core
|
||||
// Rule Set would refuse, in a file named for the app by the name
|
||||
// SWW_WAF_APPS gives it, such as apps/gitea.conf. The fixes are Coraza
|
||||
// exclusions, loaded before the Core Rule Set's rules. Their first rule
|
||||
// skips to the marker they end with, unless Inspect sets the transaction
|
||||
// variable named for the app, such as app_gitea. The skip reaches only
|
||||
// phase 1, so a rule of a later phase must check that variable itself.
|
||||
//
|
||||
//go:embed apps/*.conf
|
||||
var appFixes embed.FS
|
||||
|
||||
// app is what an app's file in appFixes cannot say about it.
|
||||
type app struct {
|
||||
// directives are Coraza exclusions, loaded before the Core Rule Set's
|
||||
// rules. Their first rule skips to the marker they end with, unless
|
||||
// Inspect sets the transaction variable named for the app, such as
|
||||
// app_gitea. The skip reaches only phase 1, so a rule of a later phase
|
||||
// must check that variable itself.
|
||||
directives string
|
||||
// cookies are the cookies the Core Rule Set reads a request without.
|
||||
cookies []string
|
||||
// forkOf names the app this one is a fork of, "" for none. A hostname
|
||||
@@ -161,60 +162,34 @@ type app struct {
|
||||
forkOf string
|
||||
}
|
||||
|
||||
//go:embed gitea.conf
|
||||
var giteaDirectives string
|
||||
|
||||
//go:embed wordpress.conf
|
||||
var wordpressDirectives string
|
||||
|
||||
//go:embed drupal.conf
|
||||
var drupalDirectives string
|
||||
|
||||
//go:embed mediawiki.conf
|
||||
var mediawikiDirectives string
|
||||
|
||||
//go:embed mastodon.conf
|
||||
var mastodonDirectives string
|
||||
|
||||
//go:embed vaultwarden.conf
|
||||
var vaultwardenDirectives string
|
||||
|
||||
//go:embed nextcloud.conf
|
||||
var nextcloudDirectives string
|
||||
|
||||
//go:embed forgejo.conf
|
||||
var forgejoDirectives string
|
||||
|
||||
//go:embed ghost.conf
|
||||
var ghostDirectives string
|
||||
|
||||
// apps are the apps whose fixes sww has, by the name SWW_WAF_APPS gives each.
|
||||
// flash is Forgejo's name for gitea_flash.
|
||||
// apps are the apps that need more than their file in appFixes, by the
|
||||
// name SWW_WAF_APPS gives each. flash is Forgejo's name for gitea_flash.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a map
|
||||
var apps = map[string]app{
|
||||
"drupal": {directives: drupalDirectives},
|
||||
"forgejo": {
|
||||
directives: forgejoDirectives,
|
||||
cookies: []string{"flash"},
|
||||
forkOf: "gitea",
|
||||
cookies: []string{"flash"},
|
||||
forkOf: "gitea",
|
||||
},
|
||||
"ghost": {directives: ghostDirectives},
|
||||
"gitea": {
|
||||
directives: giteaDirectives,
|
||||
cookies: []string{"gitea_flash", "redirect_to"},
|
||||
cookies: []string{"gitea_flash", "redirect_to"},
|
||||
},
|
||||
"mastodon": {directives: mastodonDirectives},
|
||||
"mediawiki": {directives: mediawikiDirectives},
|
||||
"nextcloud": {directives: nextcloudDirectives},
|
||||
"vaultwarden": {directives: vaultwardenDirectives},
|
||||
"wordpress": {directives: wordpressDirectives},
|
||||
}
|
||||
|
||||
// AppNames returns the names of the apps whose fixes sww has, in
|
||||
// alphabetical order.
|
||||
func AppNames() []string {
|
||||
return slices.Sorted(maps.Keys(apps))
|
||||
// An embedded directory always reads.
|
||||
files, _ := appFixes.ReadDir("apps")
|
||||
|
||||
names := make([]string, len(files))
|
||||
for i, file := range files {
|
||||
names[i] = strings.TrimSuffix(file.Name(), ".conf")
|
||||
}
|
||||
|
||||
slices.Sort(names)
|
||||
|
||||
return names
|
||||
}
|
||||
|
||||
// Params are what New needs.
|
||||
@@ -373,8 +348,11 @@ func New(params Params) (*CoreRuleSet, error) {
|
||||
}
|
||||
|
||||
var fixes strings.Builder
|
||||
|
||||
for _, name := range AppNames() {
|
||||
fixes.WriteString(apps[name].directives + "\n")
|
||||
// An embedded file always reads.
|
||||
file, _ := appFixes.ReadFile("apps/" + name + ".conf")
|
||||
fixes.WriteString(string(file) + "\n")
|
||||
}
|
||||
|
||||
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions,
|
||||
|
||||
@@ -17,7 +17,7 @@ const (
|
||||
)
|
||||
|
||||
// giteaSearches are searches of the pages gitea and Forgejo share whose q
|
||||
// skips the lists of system files and command names, in the eighth change.
|
||||
// skips the lists of system files and command names, in gitea's fixes.
|
||||
// gitea serves its admin pages under /-/admin/ and Forgejo under /admin/.
|
||||
//
|
||||
//nolint:gochecknoglobals // a constant cannot be a list
|
||||
@@ -222,8 +222,8 @@ func TestGiteaSearchExclusionsAreNarrow(t *testing.T) {
|
||||
}
|
||||
|
||||
// Under /admin/, where Forgejo serves its admin pages, only the hard
|
||||
// values are left out: the tenth change lets them through in q on every
|
||||
// path there, for Drupal.
|
||||
// values are left out: Drupal's fixes let them through in q on every path
|
||||
// there.
|
||||
for _, target := range []string{
|
||||
"/admin/packages/delete?q=",
|
||||
"/admin/packages?other=",
|
||||
|
||||
@@ -0,0 +1,71 @@
|
||||
package rproxy
|
||||
|
||||
import (
|
||||
"embed"
|
||||
"html"
|
||||
"net/http"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// pageFiles are the pages a refused visitor gets, as HTML with nothing
|
||||
// external: refused.html for a 403, ratelimited.html for a 429.
|
||||
//
|
||||
//go:embed refused.html ratelimited.html
|
||||
var pageFiles embed.FS
|
||||
|
||||
// contactSentence is the sentence of each page that becomes a link to
|
||||
// SWW_RPROXY_CONTACT_EMAIL when that is set.
|
||||
const contactSentence = "contact the site's operator"
|
||||
|
||||
// htmlContentType is the Content-Type of the pages.
|
||||
const htmlContentType = "text/html; charset=utf-8"
|
||||
|
||||
// blockPages are the pages a refused visitor gets, ready to send, made once
|
||||
// at start.
|
||||
type blockPages struct {
|
||||
refused []byte
|
||||
rateLimited []byte
|
||||
}
|
||||
|
||||
// newBlockPages makes the pages. With contactEmail set, each page's
|
||||
// contactSentence is a mailto link to it.
|
||||
func newBlockPages(contactEmail string) blockPages {
|
||||
return blockPages{
|
||||
refused: readPage("refused.html", contactEmail),
|
||||
rateLimited: readPage("ratelimited.html", contactEmail),
|
||||
}
|
||||
}
|
||||
|
||||
// readPage reads the page in name from pageFiles, linking its
|
||||
// contactSentence to contactEmail unless that is "". The files are part of
|
||||
// the binary, so a missing one is a bug, and a panic.
|
||||
func readPage(name, contactEmail string) []byte {
|
||||
contents, err := pageFiles.ReadFile(name)
|
||||
if err != nil {
|
||||
panic(err)
|
||||
}
|
||||
|
||||
if contactEmail == "" {
|
||||
return contents
|
||||
}
|
||||
|
||||
link := `<a href="mailto:` + html.EscapeString(contactEmail) + `">` +
|
||||
contactSentence + `</a>`
|
||||
|
||||
return []byte(
|
||||
strings.Replace(string(contents), contactSentence, link, 1),
|
||||
)
|
||||
}
|
||||
|
||||
// forStatus returns the page for a refusal with status, and whether there is
|
||||
// one: the rate-limited page for a 429, the refused page for a 403.
|
||||
func (p blockPages) forStatus(status int) ([]byte, bool) {
|
||||
switch status {
|
||||
case http.StatusForbidden:
|
||||
return p.refused, true
|
||||
case http.StatusTooManyRequests:
|
||||
return p.rateLimited, true
|
||||
default:
|
||||
return nil, false
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="robots" content="noindex">
|
||||
<title>Too many requests</title>
|
||||
<style>
|
||||
:root{color-scheme:light dark;--fg:#1a1a1a;--muted:#5c5c5c;--bg:#f6f6f4}
|
||||
@media(prefers-color-scheme:dark){:root{--fg:#ebebe8;--muted:#9a9a96;--bg:#161616}}
|
||||
html{background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif}
|
||||
body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;padding:1.5rem}
|
||||
main{max-width:34rem;width:100%}
|
||||
p.code{margin:0 0 .5rem;color:var(--muted);font-size:.8rem;letter-spacing:.08em}
|
||||
h1{margin:0 0 1rem;font-size:1.6rem;font-weight:600;line-height:1.25}
|
||||
p{margin:0 0 1rem}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<main>
|
||||
<p class="code">429</p>
|
||||
<h1>Too many requests</h1>
|
||||
<p>Your address has sent more requests than this site allows in a short time. Wait a while, then try again.</p>
|
||||
<p>If you think this is a mistake, contact the site's operator.</p>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
@@ -0,0 +1,27 @@
|
||||
<!doctype html>
|
||||
<html lang="en">
|
||||
<head>
|
||||
<meta charset="utf-8">
|
||||
<meta name="viewport" content="width=device-width, initial-scale=1">
|
||||
<meta name="robots" content="noindex">
|
||||
<title>Request refused</title>
|
||||
<style>
|
||||
:root{color-scheme:light dark;--fg:#1a1a1a;--muted:#5c5c5c;--bg:#f6f6f4}
|
||||
@media(prefers-color-scheme:dark){:root{--fg:#ebebe8;--muted:#9a9a96;--bg:#161616}}
|
||||
html{background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif}
|
||||
body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;padding:1.5rem}
|
||||
main{max-width:34rem;width:100%}
|
||||
p.code{margin:0 0 .5rem;color:var(--muted);font-size:.8rem;letter-spacing:.08em}
|
||||
h1{margin:0 0 1rem;font-size:1.6rem;font-weight:600;line-height:1.25}
|
||||
p{margin:0 0 1rem}
|
||||
</style>
|
||||
</head>
|
||||
<body>
|
||||
<main>
|
||||
<p class="code">403</p>
|
||||
<h1>Request refused</h1>
|
||||
<p>This site did not accept your request.</p>
|
||||
<p>If you think this is a mistake, contact the site's operator.</p>
|
||||
</main>
|
||||
</body>
|
||||
</html>
|
||||
@@ -92,6 +92,7 @@ func New(params Params) *Server {
|
||||
Path: wafprotocol.ReportPath,
|
||||
}).String(),
|
||||
breakerClock: time.Now,
|
||||
pages: newBlockPages(params.Config.ContactEmail),
|
||||
}
|
||||
|
||||
s := &Server{h: h}
|
||||
@@ -156,6 +157,8 @@ type handler struct {
|
||||
breakerClock func() time.Time
|
||||
// reports are the goroutines sending reports to waf.
|
||||
reports reportGoroutines
|
||||
// pages are the pages a refused visitor gets.
|
||||
pages blockPages
|
||||
}
|
||||
|
||||
// newTransport returns what carries requests to the app. It never goes through
|
||||
|
||||
+181
-30
@@ -11,6 +11,7 @@ import (
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -52,8 +53,56 @@ const (
|
||||
maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
||||
maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
||||
maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
||||
contactEmail = "SWW_RPROXY_CONTACT_EMAIL"
|
||||
)
|
||||
|
||||
// The pages rproxy gives a refused visitor in place of waf's body.
|
||||
const (
|
||||
// refusedPageFile and rateLimitedPageFile are the pages' files, beside
|
||||
// the code.
|
||||
refusedPageFile = "refused.html"
|
||||
rateLimitedPageFile = "ratelimited.html"
|
||||
// contactSentence is the sentence of each page that becomes a link to
|
||||
// SWW_RPROXY_CONTACT_EMAIL.
|
||||
contactSentence = "contact the site's operator"
|
||||
htmlContentType = "text/html; charset=utf-8"
|
||||
)
|
||||
|
||||
// page is the page in file, as rproxy serves it without a contact email.
|
||||
func page(t *testing.T, file string) string {
|
||||
t.Helper()
|
||||
|
||||
//nolint:gosec // a page beside the code
|
||||
contents, err := os.ReadFile(file)
|
||||
if err != nil {
|
||||
t.Fatalf("read the page: %v", err)
|
||||
}
|
||||
|
||||
return string(contents)
|
||||
}
|
||||
|
||||
// wantPage checks that got is status with the page in file as its body, and
|
||||
// the headers of a page.
|
||||
func wantPage(t *testing.T, got answered, status int, file string) {
|
||||
t.Helper()
|
||||
|
||||
if got.status != status || got.body != page(t, file) {
|
||||
t.Errorf(
|
||||
"answered %d %q, want %d and %s",
|
||||
got.status,
|
||||
got.body,
|
||||
status,
|
||||
file,
|
||||
)
|
||||
}
|
||||
|
||||
if got.header.Get("Content-Type") != htmlContentType ||
|
||||
got.header.Get("X-Content-Type-Options") != "nosniff" ||
|
||||
got.header.Get("Cache-Control") != "no-store" {
|
||||
t.Errorf("the page's headers are %v", got.header)
|
||||
}
|
||||
}
|
||||
|
||||
// The headers the tests set and look at.
|
||||
const (
|
||||
dispositionHeader = wafprotocol.DispositionHeader
|
||||
@@ -306,6 +355,20 @@ func deny(w http.ResponseWriter, _ *http.Request) {
|
||||
_, _ = io.WriteString(w, deniedBody)
|
||||
}
|
||||
|
||||
// answerAs answers a question with status and body, with disposition unless
|
||||
// it is "", and a header of its own.
|
||||
func answerAs(disposition string, status int, body string) http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, _ *http.Request) {
|
||||
if disposition != "" {
|
||||
w.Header().Set(dispositionHeader, disposition)
|
||||
}
|
||||
|
||||
w.Header().Set(fromWAFHeader, addedHeaderValue)
|
||||
w.WriteHeader(status)
|
||||
_, _ = io.WriteString(w, body)
|
||||
}
|
||||
}
|
||||
|
||||
// startApp starts app as the app rproxy passes requests to.
|
||||
func startApp(t *testing.T, app http.HandlerFunc) *httptest.Server {
|
||||
t.Helper()
|
||||
@@ -767,7 +830,7 @@ func TestRemovesTheVisitorsLookupHeadersWhenWAFGivesNone(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
|
||||
func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
standIn := startWAF(t, deny)
|
||||
@@ -777,9 +840,10 @@ func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
|
||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
||||
|
||||
got := get(t, addr, "/")
|
||||
if got.status != http.StatusForbidden || got.body != deniedBody ||
|
||||
got.header.Get(fromWAFHeader) != addedHeaderValue {
|
||||
t.Errorf("answered %+v, want waf's answer", got)
|
||||
wantPage(t, got, http.StatusForbidden, refusedPageFile)
|
||||
|
||||
if got.header.Get(fromWAFHeader) != addedHeaderValue {
|
||||
t.Errorf("answered %+v, want waf's headers", got)
|
||||
}
|
||||
|
||||
if len(got.header.Values(dispositionHeader)) != 0 {
|
||||
@@ -788,7 +852,7 @@ func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
|
||||
|
||||
report := standIn.report(t)
|
||||
if report.Status != http.StatusForbidden ||
|
||||
report.ResponseBytes != int64(len(deniedBody)) ||
|
||||
report.ResponseBytes != int64(len(page(t, refusedPageFile))) ||
|
||||
report.DurationUpstreamTotalMS != 0 {
|
||||
t.Errorf("report %+v, of a request denied", report)
|
||||
}
|
||||
@@ -800,7 +864,7 @@ func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestADeniedHeadKeepsWAFsContentLength(t *testing.T) {
|
||||
func TestADeniedHeadGivesThePagesContentLength(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
standIn := startWAF(t, deny)
|
||||
@@ -808,13 +872,13 @@ func TestADeniedHeadKeepsWAFsContentLength(t *testing.T) {
|
||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
||||
|
||||
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
|
||||
if got.status != http.StatusForbidden ||
|
||||
if got.status != http.StatusForbidden || got.body != "" ||
|
||||
got.header.Get(
|
||||
contentLengthHeader,
|
||||
) != strconv.Itoa(
|
||||
len(deniedBody),
|
||||
len(page(t, refusedPageFile)),
|
||||
) {
|
||||
t.Errorf("answered %+v, want waf's 403 and its length", got)
|
||||
t.Errorf("answered %+v, want waf's 403 and the page's length", got)
|
||||
}
|
||||
}
|
||||
|
||||
@@ -823,11 +887,13 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
|
||||
|
||||
const host = " HTTP/1.1\r\nHost: site.example\r\n\r\n"
|
||||
|
||||
refusedPage := page(t, refusedPageFile)
|
||||
|
||||
for _, tc := range []struct {
|
||||
name, request, body string
|
||||
status int
|
||||
}{
|
||||
{"a deny", "GET /" + host, deniedBody, http.StatusForbidden},
|
||||
{"a deny", "GET /" + host, refusedPage, http.StatusForbidden},
|
||||
{"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest},
|
||||
{"the health check", "GET " + wafprotocol.HealthPath + host, "ok\n", http.StatusOK},
|
||||
} {
|
||||
@@ -899,6 +965,83 @@ func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestADenyOf429GivesTheVisitorTheRateLimitedPage(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
standIn := startWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set(dispositionHeader, dispositionDeny)
|
||||
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
|
||||
w.WriteHeader(http.StatusTooManyRequests)
|
||||
_, _ = io.WriteString(w, deniedBody)
|
||||
})
|
||||
app := startApp(t, greet)
|
||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
||||
|
||||
wantPage(
|
||||
t,
|
||||
get(t, addr, "/"),
|
||||
http.StatusTooManyRequests,
|
||||
rateLimitedPageFile,
|
||||
)
|
||||
}
|
||||
|
||||
func TestThePagesLinkTheContactEmail(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const email = "ops@example.com"
|
||||
|
||||
link := `<a href="mailto:` + email + `">` + contactSentence + `</a>`
|
||||
|
||||
for _, tc := range []struct {
|
||||
file string
|
||||
status int
|
||||
}{
|
||||
{refusedPageFile, http.StatusForbidden},
|
||||
{rateLimitedPageFile, http.StatusTooManyRequests},
|
||||
} {
|
||||
t.Run(tc.file, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
standIn := startWAF(
|
||||
t,
|
||||
func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header().Set(dispositionHeader, dispositionDeny)
|
||||
w.WriteHeader(tc.status)
|
||||
},
|
||||
)
|
||||
app := startApp(t, greet)
|
||||
addr, _ := startRProxy(
|
||||
t,
|
||||
standIn.URL,
|
||||
app.URL,
|
||||
map[string]string{contactEmail: email},
|
||||
)
|
||||
|
||||
got := get(t, addr, "/")
|
||||
want := strings.Replace(
|
||||
page(t, tc.file),
|
||||
contactSentence,
|
||||
link,
|
||||
1,
|
||||
)
|
||||
|
||||
if got.status != tc.status || got.body != want {
|
||||
t.Errorf(
|
||||
"answered %d %q, want %d %q",
|
||||
got.status,
|
||||
got.body,
|
||||
tc.status,
|
||||
want,
|
||||
)
|
||||
}
|
||||
|
||||
if !strings.Contains(page(t, tc.file), contactSentence) {
|
||||
t.Errorf("%s has no %q to link", tc.file, contactSentence)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -1017,17 +1160,28 @@ func noAnswers() map[string]func(t *testing.T) string {
|
||||
func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const unknownBody = "who knows\n"
|
||||
|
||||
refusedPage := page(t, refusedPageFile)
|
||||
rateLimitedPage := page(t, rateLimitedPageFile)
|
||||
|
||||
// want is the body the visitor gets: waf's, but for a refusal, which gets
|
||||
// rproxy's page for its status.
|
||||
for _, tc := range []struct {
|
||||
name, disposition, body string
|
||||
status int
|
||||
name, disposition, body, want string
|
||||
status int
|
||||
}{
|
||||
{"a refusal", dispositionDeny, deniedBody, http.StatusForbidden},
|
||||
{"a refusal, 429", dispositionDeny, deniedBody, http.StatusTooManyRequests},
|
||||
{"waf's 400", "", "refused\n", http.StatusBadRequest},
|
||||
{"a 500", "", "broken\n", http.StatusInternalServerError},
|
||||
{"a 503 denied", dispositionDeny, "busy\n", http.StatusServiceUnavailable},
|
||||
{"no header", "", "who knows\n", http.StatusOK},
|
||||
{"an unknown disposition", "maybe", "who knows\n", http.StatusOK},
|
||||
{"a refusal", dispositionDeny, deniedBody, refusedPage, http.StatusForbidden},
|
||||
{
|
||||
"a refusal, 429", dispositionDeny, deniedBody, rateLimitedPage,
|
||||
http.StatusTooManyRequests,
|
||||
},
|
||||
{"waf's 400", "", "refused\n", "refused\n", http.StatusBadRequest},
|
||||
{"a 500", "", "broken\n", "broken\n", http.StatusInternalServerError},
|
||||
{"a 503 denied", dispositionDeny, "busy\n", "busy\n", http.StatusServiceUnavailable},
|
||||
{"a 403 without a disposition", "", "nope\n", "nope\n", http.StatusForbidden},
|
||||
{"no header", "", unknownBody, unknownBody, http.StatusOK},
|
||||
{"an unknown disposition", "maybe", unknownBody, unknownBody, http.StatusOK},
|
||||
} {
|
||||
for _, setting := range []string{dispositionAllow, dispositionDeny} {
|
||||
subtest := tc.name + " with " + noAnswerSetting + "=" + setting
|
||||
@@ -1036,15 +1190,7 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
||||
|
||||
standIn := startWAF(
|
||||
t,
|
||||
func(w http.ResponseWriter, _ *http.Request) {
|
||||
if tc.disposition != "" {
|
||||
w.Header().Set(dispositionHeader, tc.disposition)
|
||||
}
|
||||
|
||||
w.Header().Set(fromWAFHeader, addedHeaderValue)
|
||||
w.WriteHeader(tc.status)
|
||||
_, _ = io.WriteString(w, tc.body)
|
||||
},
|
||||
answerAs(tc.disposition, tc.status, tc.body),
|
||||
)
|
||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||
t.Error("the app was asked")
|
||||
@@ -1059,9 +1205,14 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
||||
)
|
||||
|
||||
got := get(t, addr, "/")
|
||||
if got.status != tc.status || got.body != tc.body ||
|
||||
if got.status != tc.status || got.body != tc.want ||
|
||||
got.header.Get(fromWAFHeader) != addedHeaderValue {
|
||||
t.Errorf("answered %+v, want waf's answer", got)
|
||||
t.Errorf(
|
||||
"answered %+v, want %d %q",
|
||||
got,
|
||||
tc.status,
|
||||
tc.want,
|
||||
)
|
||||
}
|
||||
|
||||
if len(got.header.Values(dispositionHeader)) != 0 {
|
||||
|
||||
+24
-1
@@ -7,6 +7,7 @@ import (
|
||||
"io"
|
||||
"maps"
|
||||
"net/http"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync"
|
||||
|
||||
@@ -187,7 +188,9 @@ func canBeAsked(r *http.Request) bool {
|
||||
len(r.Header.Values("User-Agent")) <= 1
|
||||
}
|
||||
|
||||
// answerWith gives the visitor waf's answer, less its disposition.
|
||||
// answerWith gives the visitor waf's answer, less its disposition. A deny of
|
||||
// 403 or 429 gets rproxy's page for that status as its body, in place of
|
||||
// waf's.
|
||||
func (rq *request) answerWith(answer *wafAnswer) {
|
||||
rq.prepareOwnAnswer(rq.limits.requestDeadline())
|
||||
|
||||
@@ -195,6 +198,15 @@ func (rq *request) answerWith(answer *wafAnswer) {
|
||||
maps.Copy(header, answer.header)
|
||||
header.Del(wafprotocol.DispositionHeader)
|
||||
|
||||
if rq.disposition == wafprotocol.DispositionDeny {
|
||||
page, found := rq.h.pages.forStatus(answer.status)
|
||||
if found {
|
||||
rq.sendPage(answer.status, page)
|
||||
|
||||
return
|
||||
}
|
||||
}
|
||||
|
||||
// Go's server adds a Content-Type it guesses from the body to a response whose
|
||||
// headers have no Content-Type key. The key with no value sends none.
|
||||
if header.Get("Content-Type") == "" {
|
||||
@@ -204,6 +216,17 @@ func (rq *request) answerWith(answer *wafAnswer) {
|
||||
rq.send(answer.status, answer.body)
|
||||
}
|
||||
|
||||
// sendPage sends page as the body of a refusal with status. The length is set
|
||||
// here too, since send leaves a HEAD's as waf gave it, which is of waf's body.
|
||||
func (rq *request) sendPage(status int, page []byte) {
|
||||
header := rq.out.Header()
|
||||
header.Set("Content-Type", htmlContentType)
|
||||
header.Set("X-Content-Type-Options", "nosniff")
|
||||
header.Set("Cache-Control", "no-store")
|
||||
header.Set("Content-Length", strconv.Itoa(len(page)))
|
||||
rq.send(status, page)
|
||||
}
|
||||
|
||||
// sendReport sends waf report on a goroutine of its own, so that the
|
||||
// visitor does not wait for it. The report goes on after ctx, the request's own
|
||||
// context, has ended.
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
package waf
|
||||
|
||||
// The heuristic abuseipdb. It reads SWW_WAF_ABUSEIPDB_KEY,
|
||||
// SWW_WAF_ABUSEIPDB_MIN_SCORE and SWW_WAF_DNSBL_ABUSEIPDB_ACTION.
|
||||
// SWW_WAF_ABUSEIPDB_MIN_SCORE, SWW_WAF_DNSBL_ABUSEIPDB_ACTION and
|
||||
// SWW_WAF_BAN_RESPONSE.
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -16,10 +17,11 @@ import (
|
||||
|
||||
// abuseIPDB adds the tag abuseipdb and notes AbuseIPDB, as noteHit does, with
|
||||
// the score, when its score of the client is a hit, and refuses the request
|
||||
// when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is deny, as dnsbl does for a zone. While
|
||||
// SWW_WAF_ABUSEIPDB_KEY is unset it does nothing. A client without a score is
|
||||
// checked in the background, by the request's address, if its history counts
|
||||
// an offence, and the request does not wait for the answer. The score is then
|
||||
// with SWW_WAF_BAN_RESPONSE when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is deny, as
|
||||
// dnsbl does for a zone. While SWW_WAF_ABUSEIPDB_KEY is unset it does nothing.
|
||||
// A client without a score is checked in the background, by the request's
|
||||
// address, if its history counts an offence, and the request does not wait for
|
||||
// the answer. The score is then
|
||||
// used for each address of the client. Its figures are the score and
|
||||
// SWW_WAF_ABUSEIPDB_MIN_SCORE.
|
||||
type abuseIPDB struct {
|
||||
@@ -30,6 +32,7 @@ type abuseIPDB struct {
|
||||
keySet bool // SWW_WAF_ABUSEIPDB_KEY is set
|
||||
minScore int64 // SWW_WAF_ABUSEIPDB_MIN_SCORE
|
||||
action config.ReputationAction // SWW_WAF_DNSBL_ABUSEIPDB_ACTION
|
||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
||||
ipv6GroupPrefix int // SWW_WAF_IPV6_GROUP_PREFIX
|
||||
hitNoter reputationHitNoter
|
||||
}
|
||||
@@ -71,6 +74,7 @@ func (a abuseIPDB) check(
|
||||
|
||||
return &verdict{
|
||||
action: requestlog.ActionClientDenied,
|
||||
status: a.banResponse,
|
||||
figures: map[string]any{
|
||||
figureScore: score, "min_score": a.minScore,
|
||||
},
|
||||
|
||||
@@ -0,0 +1,58 @@
|
||||
package waf
|
||||
|
||||
// The heuristic active_ban. It reads SWW_WAF_MODE and SWW_WAF_BAN_RESPONSE; the
|
||||
// bans are the ban ledger's.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
)
|
||||
|
||||
// activeBan refuses a client a ban covers, with SWW_WAF_BAN_RESPONSE: a ban on
|
||||
// a netblock the client is in, in force at now. It notes for the log line when
|
||||
// that ban ends. A request that makes the ban permanent, or in observe mode
|
||||
// would have, raises the alert for it. Its figures are the ban's netblock and
|
||||
// when it ends.
|
||||
type activeBan struct {
|
||||
banMaker banMaker
|
||||
}
|
||||
|
||||
func (activeBan) name() string { return "active_ban" }
|
||||
|
||||
func (a activeBan) check(
|
||||
_ context.Context,
|
||||
rq *request,
|
||||
now time.Time,
|
||||
) *verdict {
|
||||
ledger := a.banMaker.ledger
|
||||
|
||||
check := ledger.Check
|
||||
if a.banMaker.observe {
|
||||
check = ledger.Find // the ban refuses nothing, and stays as it is
|
||||
}
|
||||
|
||||
ban, banned, madePermanent := check(rq.client, now)
|
||||
|
||||
var expires string
|
||||
if banned {
|
||||
expires = banExpires(ban)
|
||||
rq.line.BanExpires = expires
|
||||
}
|
||||
|
||||
if madePermanent {
|
||||
ban.Expires = time.Time{} // the ban made permanent, which Find leaves as it is
|
||||
a.banMaker.alertBan(rq, ban)
|
||||
}
|
||||
|
||||
if !banned {
|
||||
return nil
|
||||
}
|
||||
|
||||
return &verdict{
|
||||
action: banAction(ban.Permanent()),
|
||||
status: a.banMaker.banResponse,
|
||||
figures: map[string]any{
|
||||
figureNetblock: ban.Netblock.String(), "ban_expires": expires,
|
||||
},
|
||||
}
|
||||
}
|
||||
+5
-58
@@ -1,10 +1,9 @@
|
||||
package waf
|
||||
|
||||
// The heuristic active_ban, and banMaker, which makes the bans of several
|
||||
// heuristics. active_ban reads SWW_WAF_MODE; the bans are the ban ledger's.
|
||||
// banMaker, which makes the bans of several heuristics. The bans are the ban
|
||||
// ledger's.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/netip"
|
||||
"time"
|
||||
|
||||
@@ -16,59 +15,6 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
|
||||
// banResponse is a refusal answered with SWW_WAF_BAN_RESPONSE, and logged
|
||||
// with action.
|
||||
func (rq *request) banResponse(action requestlog.Action) *refusal {
|
||||
return &refusal{status: rq.h.config.BanResponse, action: action}
|
||||
}
|
||||
|
||||
// activeBan refuses a client a ban covers: a ban on a netblock the client is
|
||||
// in, in force at now. It notes for the log line when that ban ends. A request
|
||||
// that makes the ban permanent, or in observe mode would have, raises the alert
|
||||
// for it. Its figures are the ban's netblock and when it ends.
|
||||
type activeBan struct {
|
||||
banMaker banMaker
|
||||
}
|
||||
|
||||
func (activeBan) name() string { return "active_ban" }
|
||||
|
||||
func (a activeBan) check(
|
||||
_ context.Context,
|
||||
rq *request,
|
||||
now time.Time,
|
||||
) *verdict {
|
||||
ledger := a.banMaker.ledger
|
||||
|
||||
check := ledger.Check
|
||||
if a.banMaker.observe {
|
||||
check = ledger.Find // the ban refuses nothing, and stays as it is
|
||||
}
|
||||
|
||||
ban, banned, madePermanent := check(rq.client, now)
|
||||
|
||||
var expires string
|
||||
if banned {
|
||||
expires = banExpires(ban)
|
||||
rq.line.BanExpires = expires
|
||||
}
|
||||
|
||||
if madePermanent {
|
||||
ban.Expires = time.Time{} // the ban made permanent, which Find leaves as it is
|
||||
a.banMaker.alertBan(rq, ban)
|
||||
}
|
||||
|
||||
if !banned {
|
||||
return nil
|
||||
}
|
||||
|
||||
return &verdict{
|
||||
action: banAction(ban.Permanent()),
|
||||
figures: map[string]any{
|
||||
figureNetblock: ban.Netblock.String(), "ban_expires": expires,
|
||||
},
|
||||
}
|
||||
}
|
||||
|
||||
// banAction is the action of a request refused under a ban or for making one:
|
||||
// permanent says the ban never runs out.
|
||||
func banAction(permanent bool) requestlog.Action {
|
||||
@@ -118,8 +64,9 @@ type banMaker struct {
|
||||
alerts *alerts.Queue
|
||||
// observe is true in observe mode (SWW_WAF_MODE).
|
||||
observe bool
|
||||
// banResponse is SWW_WAF_BAN_RESPONSE, the status the notes of a ban for a
|
||||
// clear sign of attack give.
|
||||
// banResponse is SWW_WAF_BAN_RESPONSE: the status of a request refused
|
||||
// under a ban or for a clear sign of attack, which the notes of a ban for
|
||||
// a clear sign of attack give.
|
||||
banResponse int
|
||||
// banScopeV4Prefix is SWW_WAF_BAN_SCOPE_V4_PREFIX and ipv6GroupPrefix
|
||||
// SWW_WAF_IPV6_GROUP_PREFIX, which set the netblock a ban covers.
|
||||
|
||||
@@ -4,6 +4,7 @@ package waf
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"sync/atomic"
|
||||
"time"
|
||||
|
||||
@@ -161,6 +162,7 @@ func (rq *request) applyFindings(
|
||||
func (rq *request) timedOut(running string) *verdict {
|
||||
v := &verdict{
|
||||
action: requestlog.ActionChecksTimedOut,
|
||||
status: http.StatusServiceUnavailable,
|
||||
figures: map[string]any{
|
||||
"time_limit_ms": rq.h.besideLookupLimit.Milliseconds(),
|
||||
},
|
||||
|
||||
@@ -1,7 +1,7 @@
|
||||
package waf
|
||||
|
||||
// The heuristic blocklists. It reads SWW_WAF_BLOCKLIST_ACTION, and the blocklists
|
||||
// through the reputation lists.
|
||||
// The heuristic blocklists. It reads SWW_WAF_BLOCKLIST_ACTION and
|
||||
// SWW_WAF_BAN_RESPONSE, and the blocklists through the reputation lists.
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -13,13 +13,15 @@ import (
|
||||
)
|
||||
|
||||
// blocklists notes the blocklists that list the client, as noteListed does,
|
||||
// and refuses the request when SWW_WAF_BLOCKLIST_ACTION is deny. Being limit, it
|
||||
// lowers the client's limits instead (see limitPercentages), and being log, it
|
||||
// does nothing more. Its figure is the blocklists that list the client.
|
||||
// and refuses the request with SWW_WAF_BAN_RESPONSE when
|
||||
// SWW_WAF_BLOCKLIST_ACTION is deny. Being limit, it lowers the client's limits
|
||||
// instead (see limitPercentages), and being log, it does nothing more. Its
|
||||
// figure is the blocklists that list the client.
|
||||
type blocklists struct {
|
||||
lists *reputation.Lists
|
||||
action config.ReputationAction // SWW_WAF_BLOCKLIST_ACTION
|
||||
hitNoter reputationHitNoter
|
||||
lists *reputation.Lists
|
||||
action config.ReputationAction // SWW_WAF_BLOCKLIST_ACTION
|
||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
||||
hitNoter reputationHitNoter
|
||||
}
|
||||
|
||||
func (blocklists) name() string { return "blocklists" }
|
||||
@@ -39,6 +41,7 @@ func (b blocklists) check(
|
||||
|
||||
return &verdict{
|
||||
action: requestlog.ActionClientDenied,
|
||||
status: b.banResponse,
|
||||
figures: map[string]any{"lists": listedBy},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -5,6 +5,7 @@ package waf
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
@@ -77,6 +78,7 @@ func (c coreRuleSet) check(
|
||||
|
||||
return &verdict{
|
||||
action: requestlog.ActionBlockedByCoreRuleSet,
|
||||
status: http.StatusForbidden,
|
||||
figures: map[string]any{
|
||||
figureScore: result.Score,
|
||||
"threshold": threshold,
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package waf
|
||||
|
||||
// The heuristic countries. It reads SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
||||
// The heuristic countries. It reads SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES and
|
||||
// SWW_WAF_BAN_RESPONSE.
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -13,10 +14,12 @@ import (
|
||||
|
||||
// countries refuses a request when SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES is set and
|
||||
// does not hold the client's country as it was looked up. A client without a
|
||||
// country, or whose country cannot be found, is refused. Its figures are the
|
||||
// country and the setting that refused it.
|
||||
// country, or whose country cannot be found, is refused. It refuses with
|
||||
// SWW_WAF_BAN_RESPONSE. Its figures are the country and the setting that
|
||||
// refused it.
|
||||
type countries struct {
|
||||
allowed []string // SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES
|
||||
allowed []string // SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES
|
||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
||||
}
|
||||
|
||||
func (countries) name() string { return "countries" }
|
||||
@@ -35,6 +38,7 @@ func (c countries) check(
|
||||
|
||||
return &verdict{
|
||||
action: requestlog.ActionCountryDenied,
|
||||
status: c.banResponse,
|
||||
figures: map[string]any{
|
||||
"country": country,
|
||||
"setting": config.ExclusivelyAllowedCountriesSetting,
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
package waf
|
||||
|
||||
// The heuristic deny_nets. It reads SWW_WAF_DENY_NETS.
|
||||
// The heuristic deny_nets. It reads SWW_WAF_DENY_NETS and SWW_WAF_BAN_RESPONSE.
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -10,10 +10,11 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
|
||||
// denyNets refuses a client in SWW_WAF_DENY_NETS. Its figure is the netblock of
|
||||
// the setting the client is in.
|
||||
// denyNets refuses a client in SWW_WAF_DENY_NETS, with SWW_WAF_BAN_RESPONSE.
|
||||
// Its figure is the netblock of the setting the client is in.
|
||||
type denyNets struct {
|
||||
netblocks []netip.Prefix // SWW_WAF_DENY_NETS
|
||||
netblocks []netip.Prefix // SWW_WAF_DENY_NETS
|
||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
||||
}
|
||||
|
||||
func (denyNets) name() string { return "deny_nets" }
|
||||
@@ -27,6 +28,7 @@ func (d denyNets) check(
|
||||
if netblock.Contains(rq.client) {
|
||||
return &verdict{
|
||||
action: requestlog.ActionClientDenied,
|
||||
status: d.banResponse,
|
||||
figures: map[string]any{figureNetblock: netblock.String()},
|
||||
}
|
||||
}
|
||||
|
||||
+14
-10
@@ -1,7 +1,8 @@
|
||||
package waf
|
||||
|
||||
// The heuristic dnsbl. It reads SWW_WAF_DNSBL_ABUSEIPDB_ACTION, and the DNSBL zones'
|
||||
// verdicts through the DNSBL checker.
|
||||
// The heuristic dnsbl. It reads SWW_WAF_DNSBL_ABUSEIPDB_ACTION and
|
||||
// SWW_WAF_BAN_RESPONSE, and the DNSBL zones' verdicts through the DNSBL
|
||||
// checker.
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -15,15 +16,17 @@ import (
|
||||
|
||||
// dnsbl notes the DNSBL zones whose verdict lists the client, as noteListed
|
||||
// does, adds the tag dnsbl=<zone> for each, the zone's key masked, and refuses
|
||||
// the request when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is deny. Being
|
||||
// limit, it lowers the client's limits instead (see limitPercentages), and
|
||||
// being log, it does nothing more. A zone without a verdict on the client is
|
||||
// asked about it in the background, and the request does not wait for the
|
||||
// answer. Its figure is the zones that list the client, their keys masked.
|
||||
// the request with SWW_WAF_BAN_RESPONSE when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is
|
||||
// deny. Being limit, it lowers the client's limits instead (see
|
||||
// limitPercentages), and being log, it does nothing more. A zone without a
|
||||
// verdict on the client is asked about it in the background, and the request
|
||||
// does not wait for the answer. Its figure is the zones that list the client,
|
||||
// their keys masked.
|
||||
type dnsbl struct {
|
||||
dnsbl *reputation.DNSBL
|
||||
action config.ReputationAction // SWW_WAF_DNSBL_ABUSEIPDB_ACTION
|
||||
hitNoter reputationHitNoter
|
||||
dnsbl *reputation.DNSBL
|
||||
action config.ReputationAction // SWW_WAF_DNSBL_ABUSEIPDB_ACTION
|
||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
||||
hitNoter reputationHitNoter
|
||||
}
|
||||
|
||||
func (dnsbl) name() string { return "dnsbl" }
|
||||
@@ -47,6 +50,7 @@ func (d dnsbl) check(
|
||||
|
||||
return &verdict{
|
||||
action: requestlog.ActionClientDenied,
|
||||
status: d.banResponse,
|
||||
figures: map[string]any{"zones": listedBy},
|
||||
}
|
||||
}
|
||||
@@ -4,6 +4,7 @@ package waf
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
@@ -39,6 +40,7 @@ func (headerLimits) check(
|
||||
if requestLine > maxLineBytes {
|
||||
return &verdict{
|
||||
action: requestlog.ActionRequestLineTooLong,
|
||||
status: http.StatusRequestURITooLong,
|
||||
figures: map[string]any{
|
||||
"request_line_bytes": requestLine,
|
||||
"request_line_bytes_limit": maxLineBytes,
|
||||
@@ -56,6 +58,7 @@ func (headerLimits) check(
|
||||
|
||||
return &verdict{
|
||||
action: requestlog.ActionRequestHeaderTooLarge,
|
||||
status: http.StatusRequestHeaderFieldsTooLarge,
|
||||
figures: map[string]any{
|
||||
"longest_header_line_bytes": longest,
|
||||
"header_line_bytes_limit": maxLineBytes,
|
||||
|
||||
@@ -42,10 +42,13 @@ type refusesWithoutLookup interface {
|
||||
}
|
||||
|
||||
// verdict is a heuristic refusing a request, or banning its client: the action
|
||||
// the log line names, and the figures the heuristic acted on, by name, such as
|
||||
// the count and the limit.
|
||||
// the log line names, the status the visitor is answered with, and the figures
|
||||
// the heuristic acted on, by name, such as the count and the limit. A status of
|
||||
// 0 closes the visitor's connection without an answer. A verdict of afterAnswer
|
||||
// or onReport refuses nothing, and has neither action nor status.
|
||||
type verdict struct {
|
||||
action requestlog.Action
|
||||
status int
|
||||
figures map[string]any
|
||||
}
|
||||
|
||||
|
||||
@@ -80,6 +80,26 @@ func (rq *request) startLookUp(
|
||||
return result
|
||||
}
|
||||
|
||||
// refusedWhateverTheAnswer reports whether the request is one that does not wait
|
||||
// for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
|
||||
// its client: a heuristic of afterLookup that implements refusesWithoutLookup
|
||||
// says it will refuse it. In observe mode, which refuses nothing, none is.
|
||||
// rproxy lets a request through when waf answers too late.
|
||||
func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
|
||||
if rq.h.config.Observe {
|
||||
return false
|
||||
}
|
||||
|
||||
for _, each := range rq.h.afterLookup {
|
||||
knows, ok := each.(refusesWithoutLookup)
|
||||
if ok && knows.refusesWhateverTheLookupAnswers(rq, now) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
// noteLookup notes result for the log line and as the request's asn= and
|
||||
// country= tags.
|
||||
func (rq *request) noteLookup(result lookupResult) {
|
||||
|
||||
@@ -1,322 +0,0 @@
|
||||
// Package waf is the server the waf command runs. It answers
|
||||
// rproxy's questions, takes rproxy's reports and answers its own endpoints, and
|
||||
// writes a log line for each. It also bans the clients on AbuseIPDB's
|
||||
// blacklist each time the reputation package fetches it.
|
||||
package waf
|
||||
|
||||
import (
|
||||
"cmp"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"time"
|
||||
|
||||
lru "github.com/hashicorp/golang-lru/v2"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/metrics"
|
||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||
"sneak.berlin/go/smallwebwaf/internal/routewatch"
|
||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||
)
|
||||
|
||||
// MetricsPath is where the metrics are, for a request that carries
|
||||
// SWW_WAF_METRICS_TOKEN.
|
||||
const MetricsPath = "/_sww/metrics"
|
||||
|
||||
// BansPath is where an admin lists and adds bans, and, followed by / and
|
||||
// a client's address, lifts them, with SWW_WAF_ADMIN_TOKEN.
|
||||
const BansPath = "/_sww/bans"
|
||||
|
||||
// ClientsPath is where an admin asks what waf knows of a client, by
|
||||
// the client's address after it, with SWW_WAF_ADMIN_TOKEN.
|
||||
const ClientsPath = "/_sww/clients/"
|
||||
|
||||
// Params are what New needs.
|
||||
type Params struct {
|
||||
Config *config.WAF
|
||||
// RequestLog receives two JSON lines per question, one at its decision and
|
||||
// one when its report comes, and one per request to waf's own
|
||||
// endpoints.
|
||||
RequestLog io.Writer
|
||||
// ProcessLog receives the process's own messages.
|
||||
ProcessLog *slog.Logger
|
||||
// Metrics count what waf does. Whoever builds them adds the metrics
|
||||
// read from the other parts.
|
||||
Metrics *metrics.Metrics
|
||||
// Limiter is the table of clients the rate limits count, with each
|
||||
// client's history.
|
||||
Limiter *ratelimit.Limiter
|
||||
// Ledger holds the bans.
|
||||
Ledger *bans.Ledger
|
||||
// Anomalies are the counters of the anomaly thresholds.
|
||||
Anomalies *anomaly.Counters
|
||||
// Lists are the lists fetched from URLs, DNSBL the DNSBL zones'
|
||||
// verdicts, and AbuseIPDB AbuseIPDB's scores and the checks spent.
|
||||
Lists *reputation.Lists
|
||||
DNSBL *reputation.DNSBL
|
||||
AbuseIPDB *reputation.AbuseIPDB
|
||||
// RouteWatch asks routewatch about clients, and is nil while
|
||||
// SWW_WAF_ROUTEWATCH_URL is unset.
|
||||
RouteWatch *routewatch.Client
|
||||
// GeoJS looks clients' AS numbers and countries up while
|
||||
// SWW_WAF_LOOKUP_SOURCE is geojs. It is given AddLookup as its Answered
|
||||
// and GeoJSFailed as its Failed.
|
||||
GeoJS *geojs.GeoJS
|
||||
// LookupFile is the lookup database they are looked up in while
|
||||
// SWW_WAF_LOOKUP_SOURCE is file, and nil otherwise.
|
||||
LookupFile *lookup.File
|
||||
// CoreRuleSet is what inspects requests while
|
||||
// SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
|
||||
CoreRuleSet CoreRuleSetInspector
|
||||
// Now tells the time by which requests are counted for the rate
|
||||
// limits, bans are made and run out, and GeoJS's answers are kept,
|
||||
// normally time.Now in UTC, the time the state files give.
|
||||
Now func() time.Time
|
||||
// Rules are the rule files' rules, which each request is checked
|
||||
// against.
|
||||
Rules *rules.Files
|
||||
// Alerts receive the alert for each ban waf makes or makes
|
||||
// permanent, for each count over an anomaly threshold, for each request
|
||||
// whose client a blocklist, a DNSBL zone or AbuseIPDB lists, for each
|
||||
// request the Core Rule Set scores at or over
|
||||
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD, and for GeoJS failing, a fetch of a list
|
||||
// failing, a query to a DNSBL zone or a check with AbuseIPDB failing,
|
||||
// or the day's AbuseIPDB checks used up.
|
||||
Alerts *alerts.Queue
|
||||
// BesideLookupTimeLimit is how long the heuristics of besideLookup have
|
||||
// for a request. Zero means besideLookupTimeLimit. Tests that run the
|
||||
// Core Rule Set lengthen it, since the race detector and a busy host can
|
||||
// slow an inspection past besideLookupTimeLimit.
|
||||
BesideLookupTimeLimit time.Duration
|
||||
}
|
||||
|
||||
// CoreRuleSetInspector inspects a request with the Core Rule Set, as
|
||||
// coreruleset.CoreRuleSet's Inspect says. Tests give waf a stand-in for it.
|
||||
type CoreRuleSetInspector interface {
|
||||
Inspect(
|
||||
r *http.Request, client netip.Addr, body io.Reader,
|
||||
) (coreruleset.Result, []byte, error)
|
||||
}
|
||||
|
||||
// Server is the server waf runs.
|
||||
type Server struct {
|
||||
httpServer *http.Server
|
||||
h *handler
|
||||
}
|
||||
|
||||
// newHandler returns the handler with what every request shares, the parts
|
||||
// in params among it. It builds the heuristics, each with the settings and
|
||||
// parts it reads, and the lists they run in.
|
||||
//
|
||||
//nolint:funlen // a few lines for each heuristic, a list that grows with them
|
||||
func newHandler(params Params) *handler {
|
||||
awaitingReport, err := lru.New[string, *decision](maxAwaitingReport)
|
||||
if err != nil {
|
||||
panic(err) // New fails only for a size below one
|
||||
}
|
||||
|
||||
cfg := params.Config
|
||||
h := &handler{
|
||||
config: cfg,
|
||||
requestLog: params.RequestLog,
|
||||
processLog: params.ProcessLog,
|
||||
now: params.Now,
|
||||
metrics: params.Metrics,
|
||||
awaitingReport: awaitingReport,
|
||||
limiter: params.Limiter,
|
||||
ledger: params.Ledger,
|
||||
geojs: params.GeoJS,
|
||||
anomalies: params.Anomalies,
|
||||
lookupFile: params.LookupFile,
|
||||
// The country list, the headers, the tag settings that name an asn=
|
||||
// or country= tag and the biased thresholds act on the answer before
|
||||
// the request goes on.
|
||||
waitForGeoJS: len(cfg.ExclusivelyAllowedCountries) > 0 ||
|
||||
cfg.AddLookupHeaders ||
|
||||
cfg.TagsNeedLookup() ||
|
||||
biasedThresholdsSet(cfg),
|
||||
besideLookupLimit: cmp.Or(
|
||||
params.BesideLookupTimeLimit,
|
||||
besideLookupTimeLimit,
|
||||
),
|
||||
}
|
||||
|
||||
sharedBanMaker := banMaker{
|
||||
ledger: params.Ledger,
|
||||
limiter: params.Limiter,
|
||||
metrics: params.Metrics,
|
||||
alerts: params.Alerts,
|
||||
observe: cfg.Observe,
|
||||
banResponse: cfg.BanResponse,
|
||||
banScopeV4Prefix: cfg.BanScopeV4Prefix,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
}
|
||||
hitNoter := reputationHitNoter{
|
||||
metrics: params.Metrics,
|
||||
alerts: params.Alerts,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
}
|
||||
|
||||
h.trapPaths = trapPaths{paths: cfg.TrapPaths, banMaker: sharedBanMaker}
|
||||
h.ruleFiles = ruleFiles{
|
||||
rules: params.Rules,
|
||||
metrics: params.Metrics,
|
||||
banMaker: sharedBanMaker,
|
||||
}
|
||||
h.coreRuleSet = coreRuleSet{
|
||||
inspector: params.CoreRuleSet,
|
||||
metrics: params.Metrics,
|
||||
alerts: params.Alerts,
|
||||
mode: cfg.CoreRuleSetMode,
|
||||
exemptPaths: cfg.CoreRuleSetExemptPaths,
|
||||
scoreThreshold: cfg.CoreRuleSetScoreThreshold,
|
||||
observe: cfg.Observe,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
}
|
||||
|
||||
h.beforeLookup = []heuristic{
|
||||
headerLimits{},
|
||||
tagLists{lists: params.Lists},
|
||||
routeWatchTags{
|
||||
routeWatch: params.RouteWatch,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
},
|
||||
denyNets{netblocks: cfg.DenyNets},
|
||||
activeBan{banMaker: sharedBanMaker},
|
||||
}
|
||||
h.afterLookup = []heuristic{
|
||||
countries{allowed: cfg.ExclusivelyAllowedCountries},
|
||||
blocklists{
|
||||
lists: params.Lists,
|
||||
action: cfg.BlocklistAction,
|
||||
hitNoter: hitNoter,
|
||||
},
|
||||
dnsbl{
|
||||
dnsbl: params.DNSBL,
|
||||
action: cfg.DNSBLAbuseIPDBAction,
|
||||
hitNoter: hitNoter,
|
||||
},
|
||||
abuseIPDB{
|
||||
abuseIPDB: params.AbuseIPDB,
|
||||
limiter: params.Limiter,
|
||||
keySet: cfg.AbuseIPDBKey != "",
|
||||
minScore: cfg.AbuseIPDBMinScore,
|
||||
action: cfg.DNSBLAbuseIPDBAction,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
hitNoter: hitNoter,
|
||||
},
|
||||
tagDeny{tags: cfg.TagDeny},
|
||||
rateLimits{
|
||||
limiter: params.Limiter,
|
||||
exemptNets: cfg.RateLimitExemptNets,
|
||||
exemptPaths: cfg.RateLimitExemptPaths,
|
||||
banResponse: cfg.BanResponse,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
biasedThresholds: biasedThresholds{
|
||||
lists: params.Lists,
|
||||
tagLimitPercent: cfg.TagLimitPercent,
|
||||
asnBytesLimitPercent: cfg.ASNBytesLimitPercent,
|
||||
countryBytesLimitPercent: cfg.CountryBytesLimitPercent,
|
||||
unknownCountryLimitPercent: cfg.UnknownCountryLimitPercent,
|
||||
blocklistAction: cfg.BlocklistAction,
|
||||
blocklistLimitPercent: cfg.BlocklistLimitPercent,
|
||||
dnsblAbuseIPDBAction: cfg.DNSBLAbuseIPDBAction,
|
||||
dnsblAbuseIPDBLimitPercent: cfg.DNSBLAbuseIPDBLimitPercent,
|
||||
},
|
||||
banMaker: sharedBanMaker,
|
||||
},
|
||||
}
|
||||
h.besideLookup = []heuristic{h.trapPaths, h.ruleFiles, h.coreRuleSet}
|
||||
h.afterBesideLookup = []heuristic{tagDeny{tags: cfg.TagDeny}}
|
||||
h.afterAnswer = []heuristic{
|
||||
refusalLimit{
|
||||
limiter: params.Limiter,
|
||||
perMinute: cfg.RefusalsLimitPerMinute,
|
||||
banResponse: cfg.BanResponse,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
banMaker: sharedBanMaker,
|
||||
},
|
||||
}
|
||||
h.onReport = []heuristic{
|
||||
byteLimits{
|
||||
limiter: params.Limiter,
|
||||
bytesCounted: cfg.BytesCounted,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
banMaker: sharedBanMaker,
|
||||
},
|
||||
}
|
||||
|
||||
h.ownEndpoints = h.routes()
|
||||
|
||||
return h
|
||||
}
|
||||
|
||||
// handler is waf's handler. It holds what every request shares; what
|
||||
// belongs to one request is in a request.
|
||||
type handler struct {
|
||||
config *config.WAF
|
||||
requestLog io.Writer
|
||||
processLog *slog.Logger
|
||||
now func() time.Time
|
||||
metrics *metrics.Metrics
|
||||
limiter *ratelimit.Limiter
|
||||
ledger *bans.Ledger
|
||||
geojs *geojs.GeoJS
|
||||
anomalies *anomaly.Counters
|
||||
lookupFile *lookup.File
|
||||
// waitForGeoJS is true when a setting needs each request's answer from
|
||||
// GeoJS before the request goes on.
|
||||
waitForGeoJS bool
|
||||
// besideLookupLimit is how long the heuristics of besideLookup have for a
|
||||
// request, as Params.BesideLookupTimeLimit says.
|
||||
besideLookupLimit time.Duration
|
||||
// awaitingReport holds the decision on each answered question by its id
|
||||
// until its report comes, and nil for a question still being handled.
|
||||
awaitingReport *lru.Cache[string, *decision]
|
||||
// ownEndpoints answers every request that is not a question, as routes
|
||||
// lists the endpoints.
|
||||
ownEndpoints http.Handler
|
||||
|
||||
// The lists of heuristics, each in the order its heuristics run. A client
|
||||
// in SWW_WAF_ALLOW_NETS skips them all.
|
||||
|
||||
// beforeLookup are the heuristics that run before the client's AS number
|
||||
// and country are looked up, so that a client they refuse is not looked
|
||||
// up. header_limits comes first: it only measures and counts, and a
|
||||
// request it refuses causes no other work.
|
||||
beforeLookup []heuristic
|
||||
// afterLookup are the heuristics that run once the lookup has been made.
|
||||
afterLookup []heuristic
|
||||
// besideLookup are the heuristics that only read the request. Their work
|
||||
// runs beside the lookup and afterLookup, in findBesideLookup, which
|
||||
// changes nothing of the request. Their check then applies what that work
|
||||
// found, and runs only once waf takes it.
|
||||
besideLookup []heuristic
|
||||
// afterBesideLookup are the heuristics that run once afterLookup and
|
||||
// besideLookup have both let the request through. tag_deny runs a second
|
||||
// time, on the tag rule_files and core_rule_set can add.
|
||||
afterBesideLookup []heuristic
|
||||
// afterAnswer are the heuristics that run on a request once waf has
|
||||
// answered it, before its decision line is written. They ban the client,
|
||||
// and do not change what the request got: their verdict's action is not
|
||||
// used.
|
||||
afterAnswer []heuristic
|
||||
// onReport are the heuristics that run on an allowed request when
|
||||
// rproxy's report of it comes, before its report line is written. Like
|
||||
// afterAnswer, they only ban.
|
||||
onReport []heuristic
|
||||
|
||||
// The heuristics of besideLookup, whose work findBesideLookup does.
|
||||
trapPaths trapPaths
|
||||
ruleFiles ruleFiles
|
||||
coreRuleSet coreRuleSet
|
||||
}
|
||||
@@ -0,0 +1,235 @@
|
||||
package waf
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"net/http"
|
||||
"strings"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
"sneak.berlin/go/smallwebwaf/internal/wafprotocol"
|
||||
)
|
||||
|
||||
// answerQuestion runs the checks on the visitor's request a question
|
||||
// carries, and answers rproxy what to do with it. A denied request's counting
|
||||
// is finished here, and an allowed one's by its report.
|
||||
func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
|
||||
question, ok := h.readQuestion(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
|
||||
rq := h.startRequest(
|
||||
w,
|
||||
question.Visitor,
|
||||
question.Client,
|
||||
question.Peer,
|
||||
true,
|
||||
)
|
||||
// These run even when a check panics. finish runs first, so that what is
|
||||
// held has the finished log line. Go's server sends the answer only once
|
||||
// this function has returned, so the report cannot come before it is held.
|
||||
defer func() {
|
||||
rq.finish()
|
||||
rq.holdForReport()
|
||||
}()
|
||||
|
||||
rq.site = r.Host
|
||||
rq.line.WAFLine = requestlog.WAFLineDecision
|
||||
refused := rq.check(r.Context())
|
||||
rq.checked = time.Now()
|
||||
underOwnPrefix := strings.HasPrefix(
|
||||
r.URL.Path,
|
||||
wafprotocol.OwnEndpointPrefix,
|
||||
)
|
||||
|
||||
if refused == nil && !underOwnPrefix {
|
||||
rq.allow()
|
||||
// In observe mode, what enforce mode would have refused counts here.
|
||||
rq.runBanningHeuristics(r.Context(), rq.h.afterAnswer)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
if refused != nil {
|
||||
rq.deny(*refused)
|
||||
} else {
|
||||
// The app never sees this prefix. waf's own endpoints are at its
|
||||
// own address, so this answers as a path with no endpoint does.
|
||||
rq.line.Action = requestlog.ActionOwnEndpoint
|
||||
wafprotocol.SetAnswerHeaders(
|
||||
rq.out.Header(),
|
||||
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
|
||||
)
|
||||
http.Error(
|
||||
rq.out,
|
||||
http.StatusText(http.StatusNotFound),
|
||||
http.StatusNotFound,
|
||||
)
|
||||
}
|
||||
|
||||
rq.banAndCount(r.Context(), rq.h.afterAnswer)
|
||||
}
|
||||
|
||||
// readQuestion reads a question, and reports whether waf takes it. It answers
|
||||
// a question it does not take with 400 and no disposition. rproxy passes that
|
||||
// 400 to the visitor, and SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION does
|
||||
// not apply. The id of a question waf takes is held from here on, without a
|
||||
// decision yet, so that another question with that id is refused while this
|
||||
// one is being handled.
|
||||
//
|
||||
// A question whose body stops arriving gets no answer, and is not looked at.
|
||||
func (h *handler) readQuestion(
|
||||
w http.ResponseWriter, r *http.Request,
|
||||
) (wafprotocol.Question, bool) {
|
||||
question, err := wafprotocol.ReadQuestion(w, r)
|
||||
if errors.Is(err, wafprotocol.ErrBodyStoppedArriving) {
|
||||
h.processLog.Warn(
|
||||
"a question's body stopped arriving",
|
||||
"error",
|
||||
err.Error(),
|
||||
)
|
||||
// This panic has Go's server close the connection without an answer,
|
||||
// and log nothing.
|
||||
panic(http.ErrAbortHandler)
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
h.refuseQuestion(w, err.Error())
|
||||
|
||||
return question, false
|
||||
}
|
||||
|
||||
held, _ := h.awaitingReport.ContainsOrAdd(question.RequestID, nil)
|
||||
if held {
|
||||
h.refuseQuestion(
|
||||
w,
|
||||
wafprotocol.RequestIDHeader+" is the id of a request waf holds",
|
||||
)
|
||||
|
||||
return question, false
|
||||
}
|
||||
|
||||
return question, true
|
||||
}
|
||||
|
||||
// refuseQuestion answers a question waf does not take, for reason.
|
||||
func (h *handler) refuseQuestion(w http.ResponseWriter, reason string) {
|
||||
h.processLog.Warn("a question is refused", "reason", reason)
|
||||
http.Error(w, reason, http.StatusBadRequest)
|
||||
}
|
||||
|
||||
// allow answers a question with wafprotocol.DispositionAllow and, while
|
||||
// SWW_WAF_ADD_LOOKUP_HEADERS is true, the client's AS number and country.
|
||||
func (rq *request) allow() {
|
||||
rq.allowed = true
|
||||
|
||||
answer := wafprotocol.Answer{Disposition: wafprotocol.DispositionAllow}
|
||||
if rq.h.config.AddLookupHeaders {
|
||||
answer.ASN = rq.lookupAnswer.ASN
|
||||
answer.Country = rq.lookupAnswer.Country
|
||||
}
|
||||
|
||||
wafprotocol.SetAnswerHeaders(rq.out.Header(), answer)
|
||||
rq.out.WriteHeader(http.StatusOK)
|
||||
}
|
||||
|
||||
// holdForReport holds a copy of what the report's steps need, by the request's
|
||||
// id. A report only ever reaches that copy, never the request being handled.
|
||||
func (rq *request) holdForReport() {
|
||||
held := rq.decision
|
||||
rq.h.awaitingReport.Add(rq.line.RequestID, &held)
|
||||
}
|
||||
|
||||
// deny answers a question with what the visitor is to get for r.
|
||||
func (rq *request) deny(r refusal) {
|
||||
rq.refused = &r
|
||||
|
||||
if r.status == 0 {
|
||||
// SWW_WAF_BAN_RESPONSE is close. Written past rq.out, since the log line's
|
||||
// status stays 0: the visitor is sent nothing.
|
||||
wafprotocol.SetAnswerHeaders(
|
||||
rq.out.Header(),
|
||||
wafprotocol.Answer{Disposition: wafprotocol.DispositionClose},
|
||||
)
|
||||
rq.out.ResponseWriter.WriteHeader(http.StatusOK)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
wafprotocol.SetAnswerHeaders(
|
||||
rq.out.Header(),
|
||||
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
|
||||
)
|
||||
http.Error(rq.out, http.StatusText(r.status), r.status)
|
||||
}
|
||||
|
||||
// handleReport answers POST wafprotocol.ReportPath: it finishes the request
|
||||
// the report names, if it holds that request's decision, and ignores the
|
||||
// report otherwise.
|
||||
func (h *handler) handleReport() http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
report, err := wafprotocol.ReadReport(w, r)
|
||||
if err != nil {
|
||||
h.processLog.Warn("a report is refused", "error", err.Error())
|
||||
http.Error(w, "the body is not a report", http.StatusBadRequest)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
held, found := h.awaitingReport.Peek(report.RequestID)
|
||||
// Remove reports true to one report only, of several that name the
|
||||
// id. A question still being handled is held as nil.
|
||||
if found && held != nil &&
|
||||
h.awaitingReport.Remove(report.RequestID) {
|
||||
h.finishWith(r.Context(), held, report)
|
||||
}
|
||||
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
}
|
||||
|
||||
// finishWith finishes a request with what its report says, and writes the
|
||||
// request's last log line, which holds its decision's fields and the report's.
|
||||
// For an allowed request it first runs the heuristics of onReport, and adds the
|
||||
// request to its client's history, the anomaly counters and the metrics, with
|
||||
// the status and bytes the report gives. A denied request's counting was
|
||||
// finished at its decision. ctx is the report's own context.
|
||||
func (h *handler) finishWith(
|
||||
ctx context.Context,
|
||||
held *decision,
|
||||
report wafprotocol.Report,
|
||||
) {
|
||||
rq := &request{
|
||||
decision: *held,
|
||||
h: h,
|
||||
out: &responseWriter{
|
||||
status: report.Status,
|
||||
bytes: report.ResponseBytes,
|
||||
},
|
||||
requestBytes: report.RequestBytes,
|
||||
}
|
||||
|
||||
rq.line.WAFLine = requestlog.WAFLineReport
|
||||
rq.line.Status = report.Status
|
||||
rq.line.RequestBytes = report.RequestBytes
|
||||
rq.line.ResponseBytes = report.ResponseBytes
|
||||
rq.line.DurationTotalMS = report.DurationTotalMS
|
||||
|
||||
if rq.allowed {
|
||||
rq.line.DurationUpstreamTotalMS = &report.DurationUpstreamTotalMS
|
||||
|
||||
rq.banAndCount(ctx, rq.h.onReport)
|
||||
|
||||
rq.h.metrics.RequestReported(
|
||||
&rq.line,
|
||||
time.Duration(report.DurationTotalMS*float64(time.Millisecond)),
|
||||
time.Duration(
|
||||
report.DurationUpstreamTotalMS*float64(time.Millisecond),
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
rq.writeLine()
|
||||
}
|
||||
@@ -17,9 +17,9 @@ import (
|
||||
// countedByRateLimits exempts it, by the client's limit percentages, which it
|
||||
// notes for the log line with the client's counts. A request that takes the
|
||||
// client over a rate limit, as its limit percentage lowers it, breaks it: it is
|
||||
// refused, and bans the client. A request exempt from the rate limits is exempt
|
||||
// from the byte limits too. Its figures are the window, the count and the
|
||||
// limit.
|
||||
// refused with SWW_WAF_BAN_RESPONSE, and bans the client. A request exempt
|
||||
// from the rate limits is exempt from the byte limits too. Its figures are the
|
||||
// window, the count and the limit.
|
||||
type rateLimits struct {
|
||||
limiter *ratelimit.Limiter
|
||||
exemptNets []netip.Prefix // SWW_WAF_RATE_LIMIT_EXEMPT_NETS
|
||||
@@ -63,7 +63,9 @@ func (r rateLimits) check(
|
||||
r.banMaker.banForLimit(rq, now, hit, r.banResponse)
|
||||
|
||||
return &verdict{
|
||||
action: requestlog.ActionClientIsRateLimited, figures: limitFigures(hit),
|
||||
action: requestlog.ActionClientIsRateLimited,
|
||||
status: r.banResponse,
|
||||
figures: limitFigures(hit),
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -1,15 +1,12 @@
|
||||
package waf
|
||||
|
||||
// The heuristic refusal_limit. It reads SWW_WAF_REFUSALS_LIMIT_PER_MINUTE and
|
||||
// SWW_WAF_BAN_RESPONSE.
|
||||
// The heuristic refusal_limit. It reads SWW_WAF_REFUSALS_LIMIT_PER_MINUTE.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
|
||||
// refusalLimit counts the request once it has been answered, if waf
|
||||
@@ -18,11 +15,11 @@ import (
|
||||
// SWW_WAF_REFUSALS_LIMIT_PER_MINUTE such refusals of the client within a minute break
|
||||
// a limit, which bans the client. A client in SWW_WAF_ALLOW_NETS, which the
|
||||
// heuristics skip, is not counted, and nothing is while the limit is off.
|
||||
// Its figures are the window, the count and the limit.
|
||||
// The ban's notes give the status the client was sent, or in observe mode
|
||||
// would have been. Its figures are the window, the count and the limit.
|
||||
type refusalLimit struct {
|
||||
limiter *ratelimit.Limiter
|
||||
perMinute int64 // SWW_WAF_REFUSALS_LIMIT_PER_MINUTE
|
||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
||||
ipv6GroupPrefix int // SWW_WAF_IPV6_GROUP_PREFIX
|
||||
banMaker banMaker
|
||||
}
|
||||
@@ -51,27 +48,7 @@ func (r refusalLimit) check(
|
||||
return nil
|
||||
}
|
||||
|
||||
// What the client was sent, or in observe mode would have been.
|
||||
status := rq.out.status
|
||||
|
||||
switch rq.wouldAction {
|
||||
case requestlog.ActionBlockedByBlockRule,
|
||||
requestlog.ActionBlockedByCoreRuleSet:
|
||||
status = http.StatusForbidden
|
||||
case requestlog.ActionTemporarilyBanned,
|
||||
requestlog.ActionPermanentlyBanned:
|
||||
status = r.banResponse
|
||||
case requestlog.ActionForward,
|
||||
requestlog.ActionRequestLineTooLong,
|
||||
requestlog.ActionRequestHeaderTooLarge,
|
||||
requestlog.ActionClientIsRateLimited,
|
||||
requestlog.ActionClientDenied,
|
||||
requestlog.ActionCountryDenied,
|
||||
requestlog.ActionChecksTimedOut,
|
||||
requestlog.ActionOwnEndpoint:
|
||||
}
|
||||
|
||||
r.banMaker.banForLimit(rq, now, hit, status)
|
||||
r.banMaker.banForLimit(rq, now, hit, rq.refusalStatus)
|
||||
|
||||
return &verdict{figures: limitFigures(hit)}
|
||||
}
|
||||
+12
-25
@@ -69,6 +69,10 @@ type decision struct {
|
||||
// its client's history counts. trapPaths and ruleFiles set attack,
|
||||
// ruleFiles blockedByBlockRule, and coreRuleSet blockedByCoreRuleSet.
|
||||
attack, blockedByBlockRule, blockedByCoreRuleSet bool
|
||||
// refusalStatus is the status of the verdict that refused the request, or
|
||||
// in observe mode would have: what the visitor was answered with, or would
|
||||
// have been. check sets it.
|
||||
refusalStatus int
|
||||
// wouldAction is, in observe mode, the action enforce mode would have
|
||||
// refused the request with, one of requestlog's Action constants, and ""
|
||||
// otherwise. check sets it.
|
||||
@@ -203,9 +207,8 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
|
||||
|
||||
// check is the one place where a request can be refused. It returns nil to let
|
||||
// the request through. The heuristics of checkClient decide: a request they
|
||||
// refuse is answered with SWW_WAF_BAN_RESPONSE, 403 for a block rule or the Core
|
||||
// Rule Set, 503 for heuristics that ran out of time, or 414 or 431 for the
|
||||
// header limits. In observe mode nothing is refused, and the log line names
|
||||
// refuse is answered with the status and logged with the action of the verdict
|
||||
// that refused it. In observe mode nothing is refused, and the log line names
|
||||
// what enforce mode would have done. ctx is the request's own context.
|
||||
func (rq *request) check(ctx context.Context) *refusal {
|
||||
v := rq.checkClient(ctx)
|
||||
@@ -213,32 +216,16 @@ func (rq *request) check(ctx context.Context) *refusal {
|
||||
return nil
|
||||
}
|
||||
|
||||
action := v.action
|
||||
rq.refusalStatus = v.status
|
||||
|
||||
switch {
|
||||
case rq.h.config.Observe:
|
||||
rq.wouldAction = action
|
||||
rq.line.WouldAction = action
|
||||
if rq.h.config.Observe {
|
||||
rq.wouldAction = v.action
|
||||
rq.line.WouldAction = v.action
|
||||
|
||||
return nil
|
||||
case action == requestlog.ActionBlockedByBlockRule ||
|
||||
action == requestlog.ActionBlockedByCoreRuleSet:
|
||||
return &refusal{status: http.StatusForbidden, action: action}
|
||||
case action == requestlog.ActionChecksTimedOut:
|
||||
return &refusal{
|
||||
status: http.StatusServiceUnavailable,
|
||||
action: action,
|
||||
}
|
||||
case action == requestlog.ActionRequestLineTooLong:
|
||||
return &refusal{status: http.StatusRequestURITooLong, action: action}
|
||||
case action == requestlog.ActionRequestHeaderTooLarge:
|
||||
return &refusal{
|
||||
status: http.StatusRequestHeaderFieldsTooLarge,
|
||||
action: action,
|
||||
}
|
||||
default:
|
||||
return rq.banResponse(action)
|
||||
}
|
||||
|
||||
return &refusal{status: v.status, action: v.action}
|
||||
}
|
||||
|
||||
// checkClient runs the heuristics on the request, and returns the first
|
||||
|
||||
@@ -1,10 +1,12 @@
|
||||
package waf
|
||||
|
||||
// The heuristic rule_files. It reads the rules of the rule files, which
|
||||
// SWW_WAF_RULE_FILES_DIR and SWW_WAF_RULE_FILES_ENABLED name.
|
||||
// SWW_WAF_RULE_FILES_DIR and SWW_WAF_RULE_FILES_ENABLED name, and
|
||||
// SWW_WAF_BAN_RESPONSE.
|
||||
|
||||
import (
|
||||
"context"
|
||||
"net/http"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
@@ -17,8 +19,9 @@ import (
|
||||
// ruleFiles takes the rules of the rule files the request matched, which
|
||||
// findBesideLookup found, notes their ids in the log line, adds the tag
|
||||
// exploitsignature when it matches any, and refuses the request
|
||||
// when the last rule it matches refuses it: with ActionBlockedByBlockRule for a
|
||||
// block rule, and for a ban rule with the action for the ban it makes. A ban
|
||||
// when the last rule it matches refuses it: with 403 and
|
||||
// ActionBlockedByBlockRule for a block rule, and for a ban rule with
|
||||
// SWW_WAF_BAN_RESPONSE and the action for the ban it makes. A ban
|
||||
// rule bans the client's netblock for a clear sign of attack, or in observe
|
||||
// mode raises the alert for the ban
|
||||
// it would have made. Either rule's match is noted as an offence, for the
|
||||
@@ -59,6 +62,7 @@ func (r ruleFiles) check(
|
||||
|
||||
return &verdict{
|
||||
action: requestlog.ActionBlockedByBlockRule,
|
||||
status: http.StatusForbidden,
|
||||
figures: figures,
|
||||
}
|
||||
case rules.ActionBan:
|
||||
@@ -70,6 +74,7 @@ func (r ruleFiles) check(
|
||||
now,
|
||||
bans.Notes{RuleID: last.ID, Target: last.Target},
|
||||
),
|
||||
status: r.banMaker.banResponse,
|
||||
figures: figures,
|
||||
}
|
||||
case rules.ActionLog:
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
package waf
|
||||
|
||||
// The heuristic tag_deny. It reads SWW_WAF_TAG_DENY.
|
||||
// The heuristic tag_deny. It reads SWW_WAF_TAG_DENY and SWW_WAF_BAN_RESPONSE.
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -10,10 +10,11 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
)
|
||||
|
||||
// tagDeny refuses a request that carries a tag in SWW_WAF_TAG_DENY. Its figure is
|
||||
// the first such tag, in sorted order.
|
||||
// tagDeny refuses a request that carries a tag in SWW_WAF_TAG_DENY, with
|
||||
// SWW_WAF_BAN_RESPONSE. Its figure is the first such tag, in sorted order.
|
||||
type tagDeny struct {
|
||||
tags []string // SWW_WAF_TAG_DENY
|
||||
tags []string // SWW_WAF_TAG_DENY
|
||||
banResponse int // SWW_WAF_BAN_RESPONSE
|
||||
}
|
||||
|
||||
func (tagDeny) name() string { return "tag_deny" }
|
||||
@@ -27,6 +28,7 @@ func (t tagDeny) check(
|
||||
if slices.Contains(t.tags, tag) {
|
||||
return &verdict{
|
||||
action: requestlog.ActionClientDenied,
|
||||
status: t.banResponse,
|
||||
figures: map[string]any{"tag": tag},
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
package waf
|
||||
|
||||
// The heuristic trap_paths. It reads SWW_WAF_TRAP_PATHS.
|
||||
// The heuristic trap_paths. It reads SWW_WAF_TRAP_PATHS and
|
||||
// SWW_WAF_BAN_RESPONSE.
|
||||
|
||||
import (
|
||||
"context"
|
||||
@@ -12,10 +13,10 @@ import (
|
||||
)
|
||||
|
||||
// trapPaths refuses a request for a path in SWW_WAF_TRAP_PATHS, as trapPath
|
||||
// finds it. Such a request is a clear sign of attack, as a ban rule's match is:
|
||||
// it bans the client's netblock, or in observe mode raises the alert for the
|
||||
// ban it would have made. The ban's notes give the path as sent, and so does
|
||||
// its figure.
|
||||
// finds it, with SWW_WAF_BAN_RESPONSE. Such a request is a clear sign of
|
||||
// attack, as a ban rule's match is: it bans the client's netblock, or in
|
||||
// observe mode raises the alert for the ban it would have made. The ban's
|
||||
// notes give the path as sent, and so does its figure.
|
||||
type trapPaths struct {
|
||||
paths []string // SWW_WAF_TRAP_PATHS
|
||||
banMaker banMaker
|
||||
@@ -41,6 +42,7 @@ func (t trapPaths) check(
|
||||
now,
|
||||
bans.Notes{TrapPath: path},
|
||||
),
|
||||
status: t.banMaker.banResponse,
|
||||
figures: map[string]any{"path": path},
|
||||
}
|
||||
}
|
||||
|
||||
+321
-246
@@ -1,18 +1,48 @@
|
||||
// Package waf is the server the waf command runs. It answers
|
||||
// rproxy's questions, takes rproxy's reports and answers its own endpoints, and
|
||||
// writes a log line for each. It also bans the clients on AbuseIPDB's
|
||||
// blacklist each time the reputation package fetches it.
|
||||
package waf
|
||||
|
||||
import (
|
||||
"cmp"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net"
|
||||
"net/http"
|
||||
"strings"
|
||||
"net/netip"
|
||||
"time"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||
lru "github.com/hashicorp/golang-lru/v2"
|
||||
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
"sneak.berlin/go/smallwebwaf/internal/anomaly"
|
||||
"sneak.berlin/go/smallwebwaf/internal/bans"
|
||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||
"sneak.berlin/go/smallwebwaf/internal/geojs"
|
||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||
"sneak.berlin/go/smallwebwaf/internal/metrics"
|
||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||
"sneak.berlin/go/smallwebwaf/internal/reputation"
|
||||
"sneak.berlin/go/smallwebwaf/internal/routewatch"
|
||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||
"sneak.berlin/go/smallwebwaf/internal/wafprotocol"
|
||||
)
|
||||
|
||||
// MetricsPath is where the metrics are, for a request that carries
|
||||
// SWW_WAF_METRICS_TOKEN.
|
||||
const MetricsPath = "/_sww/metrics"
|
||||
|
||||
// BansPath is where an admin lists and adds bans, and, followed by / and
|
||||
// a client's address, lifts them, with SWW_WAF_ADMIN_TOKEN.
|
||||
const BansPath = "/_sww/bans"
|
||||
|
||||
// ClientsPath is where an admin asks what waf knows of a client, by
|
||||
// the client's address after it, with SWW_WAF_ADMIN_TOKEN.
|
||||
const ClientsPath = "/_sww/clients/"
|
||||
|
||||
const (
|
||||
// maxAwaitingReport is the most answered requests held until their
|
||||
// reports come. Past it, the request answered longest ago is dropped.
|
||||
@@ -24,6 +54,79 @@ const (
|
||||
wafIdleTimeout = 120 * time.Second
|
||||
)
|
||||
|
||||
// Params are what New needs.
|
||||
type Params struct {
|
||||
Config *config.WAF
|
||||
// RequestLog receives two JSON lines per question, one at its decision and
|
||||
// one when its report comes, and one per request to waf's own
|
||||
// endpoints.
|
||||
RequestLog io.Writer
|
||||
// ProcessLog receives the process's own messages.
|
||||
ProcessLog *slog.Logger
|
||||
// Metrics count what waf does. Whoever builds them adds the metrics
|
||||
// read from the other parts.
|
||||
Metrics *metrics.Metrics
|
||||
// Limiter is the table of clients the rate limits count, with each
|
||||
// client's history.
|
||||
Limiter *ratelimit.Limiter
|
||||
// Ledger holds the bans.
|
||||
Ledger *bans.Ledger
|
||||
// Anomalies are the counters of the anomaly thresholds.
|
||||
Anomalies *anomaly.Counters
|
||||
// Lists are the lists fetched from URLs, DNSBL the DNSBL zones'
|
||||
// verdicts, and AbuseIPDB AbuseIPDB's scores and the checks spent.
|
||||
Lists *reputation.Lists
|
||||
DNSBL *reputation.DNSBL
|
||||
AbuseIPDB *reputation.AbuseIPDB
|
||||
// RouteWatch asks routewatch about clients, and is nil while
|
||||
// SWW_WAF_ROUTEWATCH_URL is unset.
|
||||
RouteWatch *routewatch.Client
|
||||
// GeoJS looks clients' AS numbers and countries up while
|
||||
// SWW_WAF_LOOKUP_SOURCE is geojs. It is given AddLookup as its Answered
|
||||
// and GeoJSFailed as its Failed.
|
||||
GeoJS *geojs.GeoJS
|
||||
// LookupFile is the lookup database they are looked up in while
|
||||
// SWW_WAF_LOOKUP_SOURCE is file, and nil otherwise.
|
||||
LookupFile *lookup.File
|
||||
// CoreRuleSet is what inspects requests while
|
||||
// SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
|
||||
CoreRuleSet CoreRuleSetInspector
|
||||
// Now tells the time by which requests are counted for the rate
|
||||
// limits, bans are made and run out, and GeoJS's answers are kept,
|
||||
// normally time.Now in UTC, the time the state files give.
|
||||
Now func() time.Time
|
||||
// Rules are the rule files' rules, which each request is checked
|
||||
// against.
|
||||
Rules *rules.Files
|
||||
// Alerts receive the alert for each ban waf makes or makes
|
||||
// permanent, for each count over an anomaly threshold, for each request
|
||||
// whose client a blocklist, a DNSBL zone or AbuseIPDB lists, for each
|
||||
// request the Core Rule Set scores at or over
|
||||
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD, and for GeoJS failing, a fetch of a list
|
||||
// failing, a query to a DNSBL zone or a check with AbuseIPDB failing,
|
||||
// or the day's AbuseIPDB checks used up.
|
||||
Alerts *alerts.Queue
|
||||
// BesideLookupTimeLimit is how long the heuristics of besideLookup have
|
||||
// for a request. Zero means besideLookupTimeLimit. Tests that run the
|
||||
// Core Rule Set lengthen it, since the race detector and a busy host can
|
||||
// slow an inspection past besideLookupTimeLimit.
|
||||
BesideLookupTimeLimit time.Duration
|
||||
}
|
||||
|
||||
// CoreRuleSetInspector inspects a request with the Core Rule Set, as
|
||||
// coreruleset.CoreRuleSet's Inspect says. Tests give waf a stand-in for it.
|
||||
type CoreRuleSetInspector interface {
|
||||
Inspect(
|
||||
r *http.Request, client netip.Addr, body io.Reader,
|
||||
) (coreruleset.Result, []byte, error)
|
||||
}
|
||||
|
||||
// Server is the server waf runs.
|
||||
type Server struct {
|
||||
httpServer *http.Server
|
||||
h *handler
|
||||
}
|
||||
|
||||
// New returns the server `sww waf` runs. It answers
|
||||
// rproxy's questions, takes rproxy's reports and answers its own endpoints. It
|
||||
// passes nothing to an app. rproxy applies the size and time limits to the
|
||||
@@ -70,6 +173,221 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
s.h.ServeHTTP(w, r)
|
||||
}
|
||||
|
||||
// newHandler returns the handler with what every request shares, the parts
|
||||
// in params among it. It builds the heuristics, each with the settings and
|
||||
// parts it reads, and the lists they run in.
|
||||
//
|
||||
//nolint:funlen // a few lines for each heuristic, a list that grows with them
|
||||
func newHandler(params Params) *handler {
|
||||
awaitingReport, err := lru.New[string, *decision](maxAwaitingReport)
|
||||
if err != nil {
|
||||
panic(err) // New fails only for a size below one
|
||||
}
|
||||
|
||||
cfg := params.Config
|
||||
h := &handler{
|
||||
config: cfg,
|
||||
requestLog: params.RequestLog,
|
||||
processLog: params.ProcessLog,
|
||||
now: params.Now,
|
||||
metrics: params.Metrics,
|
||||
awaitingReport: awaitingReport,
|
||||
limiter: params.Limiter,
|
||||
ledger: params.Ledger,
|
||||
geojs: params.GeoJS,
|
||||
anomalies: params.Anomalies,
|
||||
lookupFile: params.LookupFile,
|
||||
// The country list, the headers, the tag settings that name an asn=
|
||||
// or country= tag and the biased thresholds act on the answer before
|
||||
// the request goes on.
|
||||
waitForGeoJS: len(cfg.ExclusivelyAllowedCountries) > 0 ||
|
||||
cfg.AddLookupHeaders ||
|
||||
cfg.TagsNeedLookup() ||
|
||||
biasedThresholdsSet(cfg),
|
||||
besideLookupLimit: cmp.Or(
|
||||
params.BesideLookupTimeLimit,
|
||||
besideLookupTimeLimit,
|
||||
),
|
||||
}
|
||||
|
||||
sharedBanMaker := banMaker{
|
||||
ledger: params.Ledger,
|
||||
limiter: params.Limiter,
|
||||
metrics: params.Metrics,
|
||||
alerts: params.Alerts,
|
||||
observe: cfg.Observe,
|
||||
banResponse: cfg.BanResponse,
|
||||
banScopeV4Prefix: cfg.BanScopeV4Prefix,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
}
|
||||
hitNoter := reputationHitNoter{
|
||||
metrics: params.Metrics,
|
||||
alerts: params.Alerts,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
}
|
||||
|
||||
h.trapPaths = trapPaths{paths: cfg.TrapPaths, banMaker: sharedBanMaker}
|
||||
h.ruleFiles = ruleFiles{
|
||||
rules: params.Rules,
|
||||
metrics: params.Metrics,
|
||||
banMaker: sharedBanMaker,
|
||||
}
|
||||
h.coreRuleSet = coreRuleSet{
|
||||
inspector: params.CoreRuleSet,
|
||||
metrics: params.Metrics,
|
||||
alerts: params.Alerts,
|
||||
mode: cfg.CoreRuleSetMode,
|
||||
exemptPaths: cfg.CoreRuleSetExemptPaths,
|
||||
scoreThreshold: cfg.CoreRuleSetScoreThreshold,
|
||||
observe: cfg.Observe,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
}
|
||||
|
||||
h.beforeLookup = []heuristic{
|
||||
headerLimits{},
|
||||
tagLists{lists: params.Lists},
|
||||
routeWatchTags{
|
||||
routeWatch: params.RouteWatch,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
},
|
||||
denyNets{netblocks: cfg.DenyNets, banResponse: cfg.BanResponse},
|
||||
activeBan{banMaker: sharedBanMaker},
|
||||
}
|
||||
h.afterLookup = []heuristic{
|
||||
countries{
|
||||
allowed: cfg.ExclusivelyAllowedCountries,
|
||||
banResponse: cfg.BanResponse,
|
||||
},
|
||||
blocklists{
|
||||
lists: params.Lists,
|
||||
action: cfg.BlocklistAction,
|
||||
banResponse: cfg.BanResponse,
|
||||
hitNoter: hitNoter,
|
||||
},
|
||||
dnsbl{
|
||||
dnsbl: params.DNSBL,
|
||||
action: cfg.DNSBLAbuseIPDBAction,
|
||||
banResponse: cfg.BanResponse,
|
||||
hitNoter: hitNoter,
|
||||
},
|
||||
abuseIPDB{
|
||||
abuseIPDB: params.AbuseIPDB,
|
||||
limiter: params.Limiter,
|
||||
keySet: cfg.AbuseIPDBKey != "",
|
||||
minScore: cfg.AbuseIPDBMinScore,
|
||||
action: cfg.DNSBLAbuseIPDBAction,
|
||||
banResponse: cfg.BanResponse,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
hitNoter: hitNoter,
|
||||
},
|
||||
tagDeny{tags: cfg.TagDeny, banResponse: cfg.BanResponse},
|
||||
rateLimits{
|
||||
limiter: params.Limiter,
|
||||
exemptNets: cfg.RateLimitExemptNets,
|
||||
exemptPaths: cfg.RateLimitExemptPaths,
|
||||
banResponse: cfg.BanResponse,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
biasedThresholds: biasedThresholds{
|
||||
lists: params.Lists,
|
||||
tagLimitPercent: cfg.TagLimitPercent,
|
||||
asnBytesLimitPercent: cfg.ASNBytesLimitPercent,
|
||||
countryBytesLimitPercent: cfg.CountryBytesLimitPercent,
|
||||
unknownCountryLimitPercent: cfg.UnknownCountryLimitPercent,
|
||||
blocklistAction: cfg.BlocklistAction,
|
||||
blocklistLimitPercent: cfg.BlocklistLimitPercent,
|
||||
dnsblAbuseIPDBAction: cfg.DNSBLAbuseIPDBAction,
|
||||
dnsblAbuseIPDBLimitPercent: cfg.DNSBLAbuseIPDBLimitPercent,
|
||||
},
|
||||
banMaker: sharedBanMaker,
|
||||
},
|
||||
}
|
||||
h.besideLookup = []heuristic{h.trapPaths, h.ruleFiles, h.coreRuleSet}
|
||||
h.afterBesideLookup = []heuristic{
|
||||
tagDeny{tags: cfg.TagDeny, banResponse: cfg.BanResponse},
|
||||
}
|
||||
h.afterAnswer = []heuristic{
|
||||
refusalLimit{
|
||||
limiter: params.Limiter,
|
||||
perMinute: cfg.RefusalsLimitPerMinute,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
banMaker: sharedBanMaker,
|
||||
},
|
||||
}
|
||||
h.onReport = []heuristic{
|
||||
byteLimits{
|
||||
limiter: params.Limiter,
|
||||
bytesCounted: cfg.BytesCounted,
|
||||
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
|
||||
banMaker: sharedBanMaker,
|
||||
},
|
||||
}
|
||||
|
||||
h.ownEndpoints = h.routes()
|
||||
|
||||
return h
|
||||
}
|
||||
|
||||
// handler is waf's handler. It holds what every request shares; what
|
||||
// belongs to one request is in a request.
|
||||
type handler struct {
|
||||
config *config.WAF
|
||||
requestLog io.Writer
|
||||
processLog *slog.Logger
|
||||
now func() time.Time
|
||||
metrics *metrics.Metrics
|
||||
limiter *ratelimit.Limiter
|
||||
ledger *bans.Ledger
|
||||
geojs *geojs.GeoJS
|
||||
anomalies *anomaly.Counters
|
||||
lookupFile *lookup.File
|
||||
// waitForGeoJS is true when a setting needs each request's answer from
|
||||
// GeoJS before the request goes on.
|
||||
waitForGeoJS bool
|
||||
// besideLookupLimit is how long the heuristics of besideLookup have for a
|
||||
// request, as Params.BesideLookupTimeLimit says.
|
||||
besideLookupLimit time.Duration
|
||||
// awaitingReport holds the decision on each answered question by its id
|
||||
// until its report comes, and nil for a question still being handled.
|
||||
awaitingReport *lru.Cache[string, *decision]
|
||||
// ownEndpoints answers every request that is not a question, as routes
|
||||
// lists the endpoints.
|
||||
ownEndpoints http.Handler
|
||||
|
||||
// The lists of heuristics, each in the order its heuristics run. A client
|
||||
// in SWW_WAF_ALLOW_NETS skips them all.
|
||||
|
||||
// beforeLookup are the heuristics that run before the client's AS number
|
||||
// and country are looked up, so that a client they refuse is not looked
|
||||
// up. header_limits comes first: it only measures and counts, and a
|
||||
// request it refuses causes no other work.
|
||||
beforeLookup []heuristic
|
||||
// afterLookup are the heuristics that run once the lookup has been made.
|
||||
afterLookup []heuristic
|
||||
// besideLookup are the heuristics that only read the request. Their work
|
||||
// runs beside the lookup and afterLookup, in findBesideLookup, which
|
||||
// changes nothing of the request. Their check then applies what that work
|
||||
// found, and runs only once waf takes it.
|
||||
besideLookup []heuristic
|
||||
// afterBesideLookup are the heuristics that run once afterLookup and
|
||||
// besideLookup have both let the request through. tag_deny runs a second
|
||||
// time, on the tag rule_files and core_rule_set can add.
|
||||
afterBesideLookup []heuristic
|
||||
// afterAnswer are the heuristics that run on a request once waf has
|
||||
// answered it, before its decision line is written. They ban the client,
|
||||
// and do not change what the request got: their verdict's action is not
|
||||
// used.
|
||||
afterAnswer []heuristic
|
||||
// onReport are the heuristics that run on an allowed request when
|
||||
// rproxy's report of it comes, before its report line is written. Like
|
||||
// afterAnswer, they only ban.
|
||||
onReport []heuristic
|
||||
|
||||
// The heuristics of besideLookup, whose work findBesideLookup does.
|
||||
trapPaths trapPaths
|
||||
ruleFiles ruleFiles
|
||||
coreRuleSet coreRuleSet
|
||||
}
|
||||
|
||||
// ServeHTTP handles one request to waf: a request carrying
|
||||
// wafprotocol.ClientHeader is rproxy's question, and any other is for one of
|
||||
// waf's own endpoints.
|
||||
@@ -82,246 +400,3 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
|
||||
h.ownEndpoints.ServeHTTP(w, r)
|
||||
}
|
||||
|
||||
// refusedWhateverTheAnswer reports whether the request is one that does not wait
|
||||
// for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
|
||||
// its client: a heuristic of afterLookup that implements refusesWithoutLookup
|
||||
// says it will refuse it. In observe mode, which refuses nothing, none is.
|
||||
// rproxy lets a request through when waf answers too late.
|
||||
func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
|
||||
if rq.h.config.Observe {
|
||||
return false
|
||||
}
|
||||
|
||||
for _, each := range rq.h.afterLookup {
|
||||
knows, ok := each.(refusesWithoutLookup)
|
||||
if ok && knows.refusesWhateverTheLookupAnswers(rq, now) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
|
||||
return false
|
||||
}
|
||||
|
||||
// answerQuestion runs the checks on the visitor's request a question
|
||||
// carries, and answers rproxy what to do with it. A denied request's counting
|
||||
// is finished here, and an allowed one's by its report.
|
||||
func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
|
||||
question, ok := h.readQuestion(w, r)
|
||||
if !ok {
|
||||
return
|
||||
}
|
||||
|
||||
rq := h.startRequest(
|
||||
w,
|
||||
question.Visitor,
|
||||
question.Client,
|
||||
question.Peer,
|
||||
true,
|
||||
)
|
||||
// These run even when a check panics. finish runs first, so that what is
|
||||
// held has the finished log line. Go's server sends the answer only once
|
||||
// this function has returned, so the report cannot come before it is held.
|
||||
defer func() {
|
||||
rq.finish()
|
||||
rq.holdForReport()
|
||||
}()
|
||||
|
||||
rq.site = r.Host
|
||||
rq.line.WAFLine = requestlog.WAFLineDecision
|
||||
refused := rq.check(r.Context())
|
||||
rq.checked = time.Now()
|
||||
underOwnPrefix := strings.HasPrefix(
|
||||
r.URL.Path,
|
||||
wafprotocol.OwnEndpointPrefix,
|
||||
)
|
||||
|
||||
if refused == nil && !underOwnPrefix {
|
||||
rq.allow()
|
||||
// In observe mode, what enforce mode would have refused counts here.
|
||||
rq.runBanningHeuristics(r.Context(), rq.h.afterAnswer)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
if refused != nil {
|
||||
rq.deny(*refused)
|
||||
} else {
|
||||
// The app never sees this prefix. waf's own endpoints are at its
|
||||
// own address, so this answers as a path with no endpoint does.
|
||||
rq.line.Action = requestlog.ActionOwnEndpoint
|
||||
wafprotocol.SetAnswerHeaders(
|
||||
rq.out.Header(),
|
||||
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
|
||||
)
|
||||
http.Error(
|
||||
rq.out,
|
||||
http.StatusText(http.StatusNotFound),
|
||||
http.StatusNotFound,
|
||||
)
|
||||
}
|
||||
|
||||
rq.banAndCount(r.Context(), rq.h.afterAnswer)
|
||||
}
|
||||
|
||||
// readQuestion reads a question, and reports whether waf takes it. It answers
|
||||
// a question it does not take with 400 and no disposition. rproxy passes that
|
||||
// 400 to the visitor, and SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION does
|
||||
// not apply. The id of a question waf takes is held from here on, without a
|
||||
// decision yet, so that another question with that id is refused while this
|
||||
// one is being handled.
|
||||
//
|
||||
// A question whose body stops arriving gets no answer, and is not looked at.
|
||||
func (h *handler) readQuestion(
|
||||
w http.ResponseWriter, r *http.Request,
|
||||
) (wafprotocol.Question, bool) {
|
||||
question, err := wafprotocol.ReadQuestion(w, r)
|
||||
if errors.Is(err, wafprotocol.ErrBodyStoppedArriving) {
|
||||
h.processLog.Warn(
|
||||
"a question's body stopped arriving",
|
||||
"error",
|
||||
err.Error(),
|
||||
)
|
||||
// This panic has Go's server close the connection without an answer,
|
||||
// and log nothing.
|
||||
panic(http.ErrAbortHandler)
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
h.refuseQuestion(w, err.Error())
|
||||
|
||||
return question, false
|
||||
}
|
||||
|
||||
held, _ := h.awaitingReport.ContainsOrAdd(question.RequestID, nil)
|
||||
if held {
|
||||
h.refuseQuestion(
|
||||
w,
|
||||
wafprotocol.RequestIDHeader+" is the id of a request waf holds",
|
||||
)
|
||||
|
||||
return question, false
|
||||
}
|
||||
|
||||
return question, true
|
||||
}
|
||||
|
||||
// refuseQuestion answers a question waf does not take, for reason.
|
||||
func (h *handler) refuseQuestion(w http.ResponseWriter, reason string) {
|
||||
h.processLog.Warn("a question is refused", "reason", reason)
|
||||
http.Error(w, reason, http.StatusBadRequest)
|
||||
}
|
||||
|
||||
// allow answers a question with wafprotocol.DispositionAllow and, while
|
||||
// SWW_WAF_ADD_LOOKUP_HEADERS is true, the client's AS number and country.
|
||||
func (rq *request) allow() {
|
||||
rq.allowed = true
|
||||
|
||||
answer := wafprotocol.Answer{Disposition: wafprotocol.DispositionAllow}
|
||||
if rq.h.config.AddLookupHeaders {
|
||||
answer.ASN = rq.lookupAnswer.ASN
|
||||
answer.Country = rq.lookupAnswer.Country
|
||||
}
|
||||
|
||||
wafprotocol.SetAnswerHeaders(rq.out.Header(), answer)
|
||||
rq.out.WriteHeader(http.StatusOK)
|
||||
}
|
||||
|
||||
// holdForReport holds a copy of what the report's steps need, by the request's
|
||||
// id. A report only ever reaches that copy, never the request being handled.
|
||||
func (rq *request) holdForReport() {
|
||||
held := rq.decision
|
||||
rq.h.awaitingReport.Add(rq.line.RequestID, &held)
|
||||
}
|
||||
|
||||
// deny answers a question with what the visitor is to get for r.
|
||||
func (rq *request) deny(r refusal) {
|
||||
rq.refused = &r
|
||||
|
||||
if r.status == 0 {
|
||||
// SWW_WAF_BAN_RESPONSE is close. Written past rq.out, since the log line's
|
||||
// status stays 0: the visitor is sent nothing.
|
||||
wafprotocol.SetAnswerHeaders(
|
||||
rq.out.Header(),
|
||||
wafprotocol.Answer{Disposition: wafprotocol.DispositionClose},
|
||||
)
|
||||
rq.out.ResponseWriter.WriteHeader(http.StatusOK)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
wafprotocol.SetAnswerHeaders(
|
||||
rq.out.Header(),
|
||||
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
|
||||
)
|
||||
http.Error(rq.out, http.StatusText(r.status), r.status)
|
||||
}
|
||||
|
||||
// handleReport answers POST wafprotocol.ReportPath: it finishes the request
|
||||
// the report names, if it holds that request's decision, and ignores the
|
||||
// report otherwise.
|
||||
func (h *handler) handleReport() http.HandlerFunc {
|
||||
return func(w http.ResponseWriter, r *http.Request) {
|
||||
report, err := wafprotocol.ReadReport(w, r)
|
||||
if err != nil {
|
||||
h.processLog.Warn("a report is refused", "error", err.Error())
|
||||
http.Error(w, "the body is not a report", http.StatusBadRequest)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
held, found := h.awaitingReport.Peek(report.RequestID)
|
||||
// Remove reports true to one report only, of several that name the
|
||||
// id. A question still being handled is held as nil.
|
||||
if found && held != nil &&
|
||||
h.awaitingReport.Remove(report.RequestID) {
|
||||
h.finishWith(r.Context(), held, report)
|
||||
}
|
||||
|
||||
w.WriteHeader(http.StatusNoContent)
|
||||
}
|
||||
}
|
||||
|
||||
// finishWith finishes a request with what its report says, and writes the
|
||||
// request's last log line, which holds its decision's fields and the report's.
|
||||
// For an allowed request it first runs the heuristics of onReport, and adds the
|
||||
// request to its client's history, the anomaly counters and the metrics, with
|
||||
// the status and bytes the report gives. A denied request's counting was
|
||||
// finished at its decision. ctx is the report's own context.
|
||||
func (h *handler) finishWith(
|
||||
ctx context.Context,
|
||||
held *decision,
|
||||
report wafprotocol.Report,
|
||||
) {
|
||||
rq := &request{
|
||||
decision: *held,
|
||||
h: h,
|
||||
out: &responseWriter{
|
||||
status: report.Status,
|
||||
bytes: report.ResponseBytes,
|
||||
},
|
||||
requestBytes: report.RequestBytes,
|
||||
}
|
||||
|
||||
rq.line.WAFLine = requestlog.WAFLineReport
|
||||
rq.line.Status = report.Status
|
||||
rq.line.RequestBytes = report.RequestBytes
|
||||
rq.line.ResponseBytes = report.ResponseBytes
|
||||
rq.line.DurationTotalMS = report.DurationTotalMS
|
||||
|
||||
if rq.allowed {
|
||||
rq.line.DurationUpstreamTotalMS = &report.DurationUpstreamTotalMS
|
||||
|
||||
rq.banAndCount(ctx, rq.h.onReport)
|
||||
|
||||
rq.h.metrics.RequestReported(
|
||||
&rq.line,
|
||||
time.Duration(report.DurationTotalMS*float64(time.Millisecond)),
|
||||
time.Duration(
|
||||
report.DurationUpstreamTotalMS*float64(time.Millisecond),
|
||||
),
|
||||
)
|
||||
}
|
||||
|
||||
rq.writeLine()
|
||||
}
|
||||
+8
-3
@@ -8,6 +8,8 @@ set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||
|
||||
set -x
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
# Own line: a failing command substitution inside an argument does
|
||||
@@ -18,12 +20,15 @@ main() {
|
||||
[ -n "$version" ] || version="unknown"
|
||||
revision="$(git rev-parse HEAD 2>/dev/null || true)"
|
||||
[ -n "$revision" ] || revision="unknown"
|
||||
created="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
|
||||
docker build --no-cache \
|
||||
created="$(date -u +%Y-%m-%d)"
|
||||
docker build \
|
||||
--build-arg VERSION="$version" \
|
||||
--build-arg REVISION="$revision" \
|
||||
--build-arg CREATED="$created" \
|
||||
-t "$("$SCRIPT_DIR/projectname")" .
|
||||
-t "git.eeqj.de/sneak/$("$SCRIPT_DIR/projectname")" \
|
||||
-t "git.eeqj.de/sneak/$("$SCRIPT_DIR/projectname"):$version" \
|
||||
-t "git.eeqj.de/sneak/$("$SCRIPT_DIR/projectname"):$(date -u +%Y-%m-%d)" \
|
||||
.
|
||||
}
|
||||
|
||||
main "$@"
|
||||
+2
-9
@@ -14,16 +14,9 @@ set -eu
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||
|
||||
main() {
|
||||
if [ -z "${DOCKER_HOST:-}" ]; then
|
||||
echo "script/publish: DOCKER_HOST is not set; set it to the Docker host that builds the image, such as ssh://user@build.example.com" >&2
|
||||
exit 1
|
||||
fi
|
||||
if [ -z "${PUBLISH_IMAGE:-}" ]; then
|
||||
echo "script/publish: PUBLISH_IMAGE is not set; set it to the image name in the registry, such as registry.example.com/smallwebwaf" >&2
|
||||
exit 1
|
||||
fi
|
||||
PUBLISH_IMAGE="git.eeqj.de/sneak/smallwebwaf"
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
|
||||
# Own line: a failing command substitution inside an argument does
|
||||
|
||||
Reference in new issue
Block a user