Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fef6b2f5ce |
@@ -38,13 +38,11 @@ It protects against:
|
||||
traversal. The OWASP Core Rule Set, a published set of rules that recognise
|
||||
common web attacks, runs on every request through Coraza, a Go engine. A
|
||||
request the rules take for an attack is refused.
|
||||
- Abuse from networks and countries you do not want. You can:
|
||||
- refuse whole netblocks and countries;
|
||||
- give the clients of chosen networks and countries a fraction of every
|
||||
limit, so that they are banned sooner;
|
||||
- refuse or limit clients on a blocklist, a DNS blocklist, or with a high
|
||||
AbuseIPDB score;
|
||||
- ban clients named by your own CrowdSec engine.
|
||||
- Abuse from networks and countries you do not want. You can refuse whole
|
||||
netblocks and countries; give the clients of chosen networks and countries a
|
||||
fraction of every limit, so that they are banned sooner; refuse or limit
|
||||
clients on a blocklist, a DNS blocklist, or with a high AbuseIPDB score; and
|
||||
ban clients named by your own CrowdSec engine.
|
||||
- Slow and oversized requests. Every request and every answer has a time limit
|
||||
and a size limit.
|
||||
|
||||
@@ -174,7 +172,8 @@ does.
|
||||
- To see a refusal without banning yourself, run `servewaf` with
|
||||
`SWWAF_MODE=observe` and ask for `/.env`. The request reaches the app.
|
||||
`servewaf`'s log lines have `"would_action":"banned"` and
|
||||
`"rule_ids":["env-file"]`. A ban you get in `enforce` mode is lifted as
|
||||
`"rule_ids":["env-file"]`. In `enforce` mode the same request bans your
|
||||
address, and your next request makes the ban permanent. Lift the ban as
|
||||
"Clients and bans" below describes.
|
||||
- With `SWWAF_METRICS_TOKEN` set, this answers with the metrics, from a host on
|
||||
`servewaf`'s network:
|
||||
@@ -339,9 +338,9 @@ full, and the limits that are fixed rather than settings.
|
||||
Each example is a complete `compose.yaml`. `smallwebwaf` is the tag
|
||||
`make docker` gives the image; on another host, use the name you pushed it
|
||||
under, by digest. Docker Compose puts the services on a private network of its
|
||||
own, where each reaches the others by service name. Only "In front of gitea"
|
||||
uses traefik. The other three publish `rproxy`'s port on the host, for trying it
|
||||
out.
|
||||
own, where each reaches the others by service name. The first two examples
|
||||
publish `rproxy`'s port on the host, for trying it out; the others leave that to
|
||||
traefik.
|
||||
|
||||
### Smallest setup
|
||||
|
||||
@@ -375,15 +374,10 @@ volumes:
|
||||
|
||||
### Custom per-address limits
|
||||
|
||||
For a site with no API:
|
||||
|
||||
- tighter limits for everyone;
|
||||
- no limits for the office and the automation host;
|
||||
- no checks for monitoring;
|
||||
- a refused netblock;
|
||||
- a smaller share of the limits for two hosting networks and one country;
|
||||
- trap paths;
|
||||
- `429` for a refused client.
|
||||
A site with no API: tighter limits for everyone, no limits for the office and
|
||||
the automation host, no checks for monitoring, a refused netblock, a smaller
|
||||
share of the limits for two hosting networks and one country, trap paths, and
|
||||
`429` for a refused client.
|
||||
|
||||
```yaml
|
||||
services:
|
||||
@@ -429,15 +423,10 @@ volumes:
|
||||
|
||||
### In front of gitea
|
||||
|
||||
gitea behind traefik, set up for a code forge:
|
||||
|
||||
- the upload limits raised, in size and in time;
|
||||
- the response size limit and the byte limits raised, for clones over 1 GiB;
|
||||
- the Actions runners exempt from counting;
|
||||
- the default request limits kept, since they already leave room for API
|
||||
clients.
|
||||
|
||||
`traefik` is the network traefik is on, made outside this file.
|
||||
gitea behind traefik, with the upload and byte limits raised for a code forge,
|
||||
the Actions runners exempt from counting, and the default request limits, which
|
||||
already leave room for API clients, kept. `traefik` is the network traefik is
|
||||
on, made outside this file.
|
||||
|
||||
```yaml
|
||||
services:
|
||||
@@ -446,9 +435,6 @@ services:
|
||||
image: gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef
|
||||
environment:
|
||||
GITEA__server__ROOT_URL: https://git.example/
|
||||
# The host and the port of the SSH clone URLs gitea shows.
|
||||
GITEA__server__SSH_DOMAIN: git.example
|
||||
GITEA__server__SSH_PORT: "2222"
|
||||
# gitea's TCP peer is rproxy, on the private network.
|
||||
GITEA__security__REVERSE_PROXY_TRUSTED_PROXIES: 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16
|
||||
volumes:
|
||||
@@ -468,15 +454,11 @@ services:
|
||||
# than a minute, are cut off at the defaults.
|
||||
SWRP_MAX_REQUEST_SIZE: 2G
|
||||
SWRP_MAX_REQUEST_TIME_MS: "600000"
|
||||
# A clone larger than 1 GiB is cut off at the default.
|
||||
SWRP_MAX_RESPONSE_SIZE: 4G
|
||||
networks:
|
||||
- site
|
||||
- traefik
|
||||
labels:
|
||||
traefik.enable: "true"
|
||||
# rproxy is on two networks; traefik reaches it on this one.
|
||||
traefik.docker.network: traefik
|
||||
traefik.http.routers.git.rule: Host(`git.example`)
|
||||
traefik.http.routers.git.entrypoints: websecure
|
||||
traefik.http.routers.git.tls.certresolver: letsencrypt
|
||||
@@ -489,8 +471,8 @@ services:
|
||||
SWWAF_INSTANCE_NAME: fsn1/servewaf
|
||||
# The Actions runners ask for work all day; never count them.
|
||||
SWWAF_RATE_LIMIT_EXEMPT_NETS: 10.0.4.0/24
|
||||
# Raised with rproxy's response limit, to four times the default
|
||||
# each, so that one clone of that size breaks no byte limit.
|
||||
# A clone larger than 1 GiB needs the response limit and the
|
||||
# byte limits raised together.
|
||||
SWWAF_BYTES_LIMIT_PER_MINUTE: 16G
|
||||
SWWAF_BYTES_LIMIT_PER_HOUR: 80G
|
||||
SWWAF_BYTES_LIMIT_PER_DAY: 800G
|
||||
@@ -516,15 +498,11 @@ request by request.
|
||||
|
||||
### `rproxy` beside the app
|
||||
|
||||
The app is built on the `smallwebwaf` image, with `rproxy` inside its container.
|
||||
`app/` holds the `Dockerfile` and `app.run` from "`rproxy` in the app's
|
||||
container". `rproxy` answers `503` to every request while `servewaf` is down,
|
||||
and `servewaf`:
|
||||
|
||||
- observes before it enforces;
|
||||
- looks addresses up in a local database rather than GeoJS;
|
||||
- sends alerts to ntfy;
|
||||
- reads its ntfy token from a file.
|
||||
The app built on the `smallwebwaf` image, with `rproxy` inside its container and
|
||||
`503` for every request while `servewaf` is down; and a `servewaf` that observes
|
||||
before it enforces, looks addresses up in a local database rather than GeoJS,
|
||||
sends alerts to ntfy, and reads its token from a file. `app/` holds the
|
||||
`Dockerfile` and `app.run` from "`rproxy` in the app's container".
|
||||
|
||||
```yaml
|
||||
services:
|
||||
@@ -561,15 +539,13 @@ volumes:
|
||||
servewaf-state:
|
||||
```
|
||||
|
||||
The database is mounted by its directory, not by file, so that a replaced
|
||||
database is seen. A token file belongs to uid 65532, the `smallwebwaf` user,
|
||||
with mode `0400`. This one holds an access token the ntfy server issued, which
|
||||
`ntfy token add <user>` prints there:
|
||||
The database and the token are mounted by their directories, not by file, so
|
||||
that a replaced file is seen. A token file belongs to uid 65532, the
|
||||
`smallwebwaf` user, with mode `0400`:
|
||||
|
||||
```sh
|
||||
mkdir -p /srv/smallwebwaf/tokens
|
||||
# <token> is the access token that ntfy printed.
|
||||
echo '<token>' > /srv/smallwebwaf/tokens/ntfy
|
||||
openssl rand -hex 32 > /srv/smallwebwaf/tokens/ntfy
|
||||
chown 65532:65532 /srv/smallwebwaf/tokens/ntfy
|
||||
chmod 0400 /srv/smallwebwaf/tokens/ntfy
|
||||
```
|
||||
@@ -617,9 +593,8 @@ the request reaches the app.
|
||||
A refused visitor is answered with `SWWAF_BAN_RESPONSE`. A `block` rule or a
|
||||
Core Rule Set match answers `403` and bans nobody, since the Core Rule Set has
|
||||
false positives; such refusals count toward `SWWAF_ERROR_BURST_THRESHOLD`
|
||||
instead. A request over the header limits of step 12 is answered `431`, and bans
|
||||
nobody either. `rproxy` itself answers `405` to a method it does not pass on,
|
||||
and the status for a size or time limit, such as `413` for a body too large.
|
||||
instead. `rproxy` itself answers `405` to a method it does not pass on, and the
|
||||
status for a size or time limit, such as `413` for a body too large.
|
||||
|
||||
### Clients and bans
|
||||
|
||||
@@ -667,9 +642,10 @@ aside as `<name>.bad`, and never stops the running process (see
|
||||
`Dockerfile` installs a package with `nix-env -iA nixpkgs.<name>` and gets the
|
||||
same package each time it is built (see [`docs/image.md`](docs/image.md)).
|
||||
- runit, a service supervisor, runs `smallwebwaf` as the `smallwebwaf` user (uid
|
||||
and gid 65532) and starts it again when it exits. The `run` script of
|
||||
`servewaf` gives the state directory to the `smallwebwaf` user at each start,
|
||||
so a volume needs no change of owner.
|
||||
and gid 65532) and starts it again when it exits. In an app's image built on
|
||||
the `smallwebwaf` image, it runs `rproxy` and the app side by side, each as
|
||||
its own user. The `run` script of `servewaf` gives the state directory to the
|
||||
`smallwebwaf` user at each start, so a volume needs no change of owner.
|
||||
- `rproxy` and `servewaf` use no token between them. `servewaf`'s own endpoints,
|
||||
under `/_smallwebwaf/`, are for your admin tools and your metrics collector,
|
||||
at its own address on the private network; nothing routes the internet there.
|
||||
@@ -700,15 +676,12 @@ aside as `<name>.bad`, and never stops the running process (see
|
||||
## Rationale
|
||||
|
||||
Small self-hosted sites receive a great deal of traffic nobody asked for:
|
||||
|
||||
- scrapers that ignore `robots.txt` and crawl every commit of every repository
|
||||
on a public git server;
|
||||
- scanners walking through lists of WordPress and `.env` paths;
|
||||
- credential-guessing bots.
|
||||
|
||||
Most of it comes from a small number of hosting networks and countries. A
|
||||
single-person operation has no abuse desk and no CDN contract. It needs
|
||||
something small that can be put in front of its services and left alone.
|
||||
scrapers that ignore `robots.txt` and crawl every commit of every repository on
|
||||
a public git server, scanners walking through lists of WordPress and `.env`
|
||||
paths, and credential-guessing bots. Most of it comes from a small number of
|
||||
hosting networks and countries. A single-person operation has no abuse desk and
|
||||
no CDN contract. It needs something small that can be put in front of its
|
||||
services and left alone.
|
||||
|
||||
The existing tools each solve part of this. Rule-based firewalls catch attack
|
||||
payloads but do not limit request rates. Rate limiters count requests but cannot
|
||||
@@ -727,10 +700,8 @@ The tests and the linter run in Docker, as phases of the `Dockerfile`, and never
|
||||
on the host.
|
||||
|
||||
- `script/bootstrap`: installs on the host what the other scripts need, with
|
||||
`make bootstrap`:
|
||||
- `make`, `git` and `curl`;
|
||||
- Go, for `gofmt`;
|
||||
- node, yarn and prettier.
|
||||
`make bootstrap`: `make`, `git` and `curl`; Go, for `gofmt`; node, yarn and
|
||||
prettier.
|
||||
- `script/setup`: readies a fresh clone: `script/bootstrap`, then
|
||||
`script/install-precommit`. `make setup`.
|
||||
- `script/projectname`: prints `smallwebwaf`, which the other scripts tag their
|
||||
@@ -757,34 +728,26 @@ on the host.
|
||||
`make build`.
|
||||
- `script/run`: builds `bin/smallwebwaf` and runs `servewaf` and `rproxy` on the
|
||||
host, with the settings in your environment. `make run`. Unless your settings
|
||||
say otherwise:
|
||||
- `rproxy` listens on port 8080, in front of an app at
|
||||
`http://127.0.0.1:8081`;
|
||||
- `servewaf` listens on port 8090;
|
||||
- its state files go in `bin/state`;
|
||||
- its rule files are those of `share/rules.d`.
|
||||
say otherwise, `rproxy` listens on port 8080, in front of an app at
|
||||
`http://127.0.0.1:8081`; `servewaf` listens on port 8090; its state files go
|
||||
in `bin/state`; its rule files are those of `share/rules.d`.
|
||||
- `script/example-app`: builds the image and the example app
|
||||
(`deploy/example-app`) on it, runs the app with `rproxy` in its container and
|
||||
`servewaf` in another, and checks them. Then it removes the containers, the
|
||||
network, the volumes and the images. Needs network access. `make example-app`.
|
||||
It checks:
|
||||
- the health checks;
|
||||
- a request passed to the app;
|
||||
- a ban for a broken limit, and a ban for a probe made permanent;
|
||||
- that only `servewaf`'s container changes the state directory's owner;
|
||||
- with `servewaf` stopped, the app still answering through `rproxy`;
|
||||
- with `servewaf` started again, bans applying again;
|
||||
- orderly stops with `sv stop` and `docker stop`;
|
||||
- bans surviving a new container on the same volume;
|
||||
- a setting given as a file.
|
||||
`servewaf` in another, and checks them: the health checks; a request passed to
|
||||
the app; a ban for a broken limit, and a ban for a probe made permanent; that
|
||||
only `servewaf`'s container changes the state directory's owner; with
|
||||
`servewaf` stopped, the app still answering through `rproxy`; with `servewaf`
|
||||
started again, bans applying again; orderly stops with `sv stop` and
|
||||
`docker stop`; bans surviving a new container on the same volume; a setting
|
||||
given as a file. Then it removes the containers, the network, the volumes and
|
||||
the images. Needs network access. `make example-app`.
|
||||
- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind
|
||||
`rproxy` and `servewaf`, three containers on one network. Every setting is at
|
||||
its default, but for those that join the containers and
|
||||
`SWWAF_LOG_REQUEST_HEADERS`, which adds the header that labels each request.
|
||||
It sends through them a sample of gitea's ordinary use and the requests the
|
||||
Core Rule Set is known to refuse, and prints what `servewaf` did with each.
|
||||
Needs network access, and a docker daemon that pushes to a registry on
|
||||
`localhost` over plain HTTP. `make gitea-requests`.
|
||||
`rproxy` and `servewaf`, three containers on one network, every setting at its
|
||||
default but those that join the containers. It sends through them a sample of
|
||||
gitea's ordinary use and the requests the Core Rule Set is known to refuse,
|
||||
and prints what `servewaf` did with each. Needs network access, and a docker
|
||||
daemon that pushes to a registry on `localhost` over plain HTTP.
|
||||
`make gitea-requests`.
|
||||
|
||||
## TODO
|
||||
|
||||
|
||||
Reference in new issue
Block a user