Caches of outside APIs live in /var/cache/sww (closes #314) #336

Merged
clawbot merged 1 commits from issue-314-cache-dir into next 2026-11-11 10:56:43 +01:00
30 changed files with 1677 additions and 585 deletions

No files matched your search

+30 -14
View File
@@ -293,7 +293,8 @@ Every setting the code reads, with its default:
| `SWW_REPUTATION_TIMEOUT` | `2s` | How long a DNS blocklist query or an AbuseIPDB check may take. | | `SWW_REPUTATION_TIMEOUT` | `2s` | How long a DNS blocklist query or an AbuseIPDB check may take. |
| `SWW_STATE_DIR` | `/var/lib/sww` | The directory of the state files, an absolute path. | | `SWW_STATE_DIR` | `/var/lib/sww` | The directory of the state files, an absolute path. |
| `SWW_STATE_WRITE_DELAY` | `10s` | How long `bans.json` waits to be written after a ban is made, lifted or made permanent. | | `SWW_STATE_WRITE_DELAY` | `10s` | How long `bans.json` waits to be written after a ban is made, lifted or made permanent. |
| `SWW_STATE_COUNTER_INTERVAL` | `15m` | How often every state file is written. | | `SWW_STATE_COUNTER_INTERVAL` | `15m` | How often every state and cache file is written. |
| `SWW_CACHE_DIR` | `/var/cache/sww` | The directory of the cache files, an absolute path. |
| `SWW_LOG_LEVEL` | `info` | The least severe of `waf`'s own messages that are written: `debug`, `info`, `warn` or `error`; request lines are always written. | | `SWW_LOG_LEVEL` | `info` | The least severe of `waf`'s own messages that are written: `debug`, `info`, `warn` or `error`; request lines are always written. |
| `SWW_LOG_REQUEST_HEADERS` | `accept,accept-language,accept-encoding,content-type,origin,range` | The request headers whose values the request log records; `Authorization` and `Cookie` are never recorded. | | `SWW_LOG_REQUEST_HEADERS` | `accept,accept-language,accept-encoding,content-type,origin,range` | The request headers whose values the request log records; `Authorization` and `Cookie` are never recorded. |
| `SWW_LOG_REMOTE_URL` | unset | A syslog server that also gets every line `waf` writes: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` with a host and a port. | | `SWW_LOG_REMOTE_URL` | unset | A syslog server that also gets every line `waf` writes: `syslog+udp://`, `syslog+tcp://` or `syslog+tls://` with a host and a port. |
@@ -614,15 +615,29 @@ client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
while `waf` runs. The image ships one file of probes no real visitor sends while `waf` runs. The image ships one file of probes no real visitor sends
(see [`docs/rules.md`](docs/rules.md)). (see [`docs/rules.md`](docs/rules.md)).
### State files ### State and cache files
`waf` keeps everything in memory, and the five JSON files in `SWW_STATE_DIR` `waf` keeps everything in memory, and JSON files hold a copy:
hold a copy: `bans.json`, `clients.json`, `lookups.json`, `reputation.json` and
`alerts.json`. Each is written whole, to a temporary file renamed over the real - The state files, in `SWW_STATE_DIR`, hold what cannot be fetched again:
one, so a crash leaves a complete file. `waf` notices a file edited on disk and - `bans.json`;
takes the edit in; an edit that does not parse is set aside as `<name>.bad`, and - `clients.json`;
never stops the running process (see - `abuseipdb-checks-spent.json`;
[`docs/state-files.md`](docs/state-files.md)). - `alerts.json`.
- The cache files, in `SWW_CACHE_DIR`, hold answers from outside services:
- `lookups.json`, from GeoJS;
- `lists.json`, the lists fetched by URL;
- `dnsbl.json`, from the DNS blocklists;
- `abuseipdb-scores.json`, from AbuseIPDB;
- `routewatch.json`, from routewatch.
Back up the state directory. The cache directory can be left out of backups: a
lost cache file starts empty and fills again.
Each file is written whole, to a temporary file renamed over the real one, so a
crash leaves a complete file. `waf` notices a file edited on disk and takes the
edit in. An edit that does not parse is set aside as `<name>.bad`, and never
stops the running process (see [`docs/state-files.md`](docs/state-files.md)).
### The image and its processes ### The image and its processes
@@ -633,7 +648,7 @@ never stops the running process (see
- runit, a service supervisor, runs `sww` as the `sww` user (uid and gid 65532) - runit, a service supervisor, runs `sww` as the `sww` user (uid and gid 65532)
and starts it again when it exits. The `run` script of `waf` gives the state and starts it again when it exits. The `run` script of `waf` gives the state
directory to the `sww` user at each start, so a volume needs no change of directory to the `sww` user at each start, so a volume needs no change of
owner. owner. It makes the cache directory for that user too.
- `rproxy` and `waf` use no token between them. `waf`'s own endpoints, under - `rproxy` and `waf` use no token between them. `waf`'s own endpoints, under
`/_sww/`, are for your admin tools and your metrics collector, at its own `/_sww/`, are for your admin tools and your metrics collector, at its own
address on the private network; nothing routes the internet there. address on the private network; nothing routes the internet there.
@@ -658,8 +673,8 @@ never stops the running process (see
`internal/routewatch`, `internal/anomaly` and `internal/alerts`: one check or `internal/routewatch`, `internal/anomaly` and `internal/alerts`: one check or
source each. source each.
- `internal/state`, `internal/requestlog`, `internal/remotelog` and - `internal/state`, `internal/requestlog`, `internal/remotelog` and
`internal/metrics`: the state files, the log lines, the syslog sender and the `internal/metrics`: the state and cache files, the log lines, the syslog
metrics. sender and the metrics.
## Rationale ## Rationale
@@ -739,6 +754,7 @@ on the host.
`http://127.0.0.1:8081`; `http://127.0.0.1:8081`;
- `waf` listens on port 8090; - `waf` listens on port 8090;
- its state files go in `bin/state`; - its state files go in `bin/state`;
- its cache files go in `bin/cache`;
- its rule files are those of `share/rules.d`. - its rule files are those of `share/rules.d`.
- `script/example-app`: builds the image and the example app - `script/example-app`: builds the image and the example app
(`deploy/example-app`) on it, runs the app with `rproxy` in its container and (`deploy/example-app`) on it, runs the app with `rproxy` in its container and
@@ -775,8 +791,8 @@ on the host.
- [`docs/alerts.md`](docs/alerts.md): the alert events and what each destination - [`docs/alerts.md`](docs/alerts.md): the alert events and what each destination
gets. gets.
- [`docs/metrics.md`](docs/metrics.md): the metrics. - [`docs/metrics.md`](docs/metrics.md): the metrics.
- [`docs/state-files.md`](docs/state-files.md): the state files and what each - [`docs/state-files.md`](docs/state-files.md): the state and cache files and
holds. what each holds.
- [`docs/image.md`](docs/image.md): what the image holds and how it is pinned. - [`docs/image.md`](docs/image.md): what the image holds and how it is pinned.
## License ## License
+80 -47
View File
@@ -37,7 +37,7 @@ never as a time.
`SWW_`, and those of `rproxy` with `SWRP_`. The only other inputs are `waf`'s `SWW_`, and those of `rproxy` with `SWRP_`. The only other inputs are `waf`'s
files: files:
- the rule files; - the rule files;
- the state files; - the state files and the cache files;
- a lookup database; - a lookup database;
- a `SWW_` setting given as a file; - a `SWW_` setting given as a file;
- the file `SWW_LOG_REMOTE_TLS_CA_FILE` names; - the file `SWW_LOG_REMOTE_TLS_CA_FILE` names;
@@ -56,19 +56,19 @@ never as a time.
## Architecture ## Architecture
- One statically linked Go binary. Only `waf` writes to disk, and only to its - One statically linked Go binary. Only `waf` writes to disk, and only to its
state directory. It ships in one image, which runs either command (see state directory and its cache directory. It ships in one image, which runs
"Deployment"). either command (see "Deployment").
- No database of any kind. Nothing is read from disk while serving a request. - No database of any kind. Nothing is read from disk while serving a request.
- Packages: - Packages:
- `cmd/sww`: the binary, which only calls `internal/sww`. - `cmd/sww`: the binary, which only calls `internal/sww`.
- `internal/sww`: the process, as the command it is given. `sww waf` reads - `internal/sww`: the process, as the command it is given. `sww waf` reads
the settings, the rule files, the lookup database and the state files, the settings, the rule files, the lookup database and the state and cache
listens, and serves until `SIGTERM` or `SIGINT`. Then it stops, writing files, listens, and serves until `SIGTERM` or `SIGINT`. Then it stops,
the state files (see "`waf`"). `sww rproxy` reads only that command's writing the state and cache files (see "`waf`"). `sww rproxy` reads only
settings, and runs `internal/rproxy` until `SIGTERM` or `SIGINT` (see that command's settings, and runs `internal/rproxy` until `SIGTERM` or
"`rproxy`"). `sww healthcheck` is the image's health check. With no `SIGINT` (see "`rproxy`"). `sww healthcheck` is the image's health check.
command, or one it does not know, the binary prints the commands and exits With no command, or one it does not know, the binary prints the commands
with status 2. and exits with status 2.
- `internal/config`: reads the settings, the one place they are read. - `internal/config`: reads the settings, the one place they are read.
- `internal/waf`: `waf`'s server. It runs the checks in `check`, and holds - `internal/waf`: `waf`'s server. It runs the checks in `check`, and holds
`waf`'s question, answer and report. It answers `waf`'s own endpoints in `waf`'s question, answer and report. It answers `waf`'s own endpoints in
@@ -96,8 +96,8 @@ never as a time.
- `internal/alerts`: holds back repeats and alerts past the hourly limit, - `internal/alerts`: holds back repeats and alerts past the hourly limit,
and sends the rest to the webhook, Slack and ntfy, each from a queue of and sends the rest to the webhook, Slack and ntfy, each from a queue of
its own. its own.
- `internal/state`: reads the state files at start, takes in edits while - `internal/state`: reads the state and cache files at start, takes in edits
running, and writes the files when due and at the stop. while running, and writes the files when due and at the stop.
- `internal/requestlog` and `internal/remotelog`: the lines on stdout, and - `internal/requestlog` and `internal/remotelog`: the lines on stdout, and
sending them to the syslog server. sending them to the syslog server.
- `internal/metrics`: the metrics, in the Prometheus text format. - `internal/metrics`: the metrics, in the Prometheus text format.
@@ -106,7 +106,7 @@ never as a time.
container, which `script/example-app` checks with `waf` in another. container, which `script/example-app` checks with `waf` in another.
- Libraries, checked against the owner's Go dependency defaults: - Libraries, checked against the owner's Go dependency defaults:
- the standard library for the reverse proxy, HTTP clients, DNS, logging - the standard library for the reverse proxy, HTTP clients, DNS, logging
(`log/slog`) and the state files (`encoding/json`, `os.Rename`); (`log/slog`) and the state and cache files (`encoding/json`, `os.Rename`);
- `github.com/corazawaf/coraza/v3` and - `github.com/corazawaf/coraza/v3` and
`github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0`, which carries `github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0`, which carries
the Core Rule Set 4.25.0. The changes `smallwebwaf` makes to it, and the the Core Rule Set 4.25.0. The changes `smallwebwaf` makes to it, and the
@@ -367,7 +367,7 @@ below are `waf`'s settings; "`rproxy`" gives the use of `rproxy`'s. Conventions:
one, since `rproxy` never touches disk. one, since `rproxy` never touches disk.
- Settings, files included, are read once at start. Changing one means - Settings, files included, are read once at start. Changing one means
restarting the container. The files `waf` watches while it runs are its state restarting the container. The files `waf` watches while it runs are its state
files, its rule files and the lookup database. and cache files, its rule files and the lookup database.
- An invalid setting stops the process at start with one log line, marked - An invalid setting stops the process at start with one log line, marked
`"type":"process"` with `"msg":"invalid setting"`. The line names the variable `"type":"process"` with `"msg":"invalid setting"`. The line names the variable
and what is wrong with the value; "Each container's processes" says what and what is wrong with the value; "Each container's processes" says what
@@ -397,10 +397,10 @@ below are `waf`'s settings; "`rproxy`" gives the use of `rproxy`'s. Conventions:
start. The file form keeps a token out of the container's environment (see start. The file form keeps a token out of the container's environment (see
"Tokens" under "Deployment"). "Tokens" under "Deployment").
### State files ### State and cache files
- `SWW_STATE_DIR` names the directory of the state files (see "State" under - `SWW_STATE_DIR` names the directory of the state files, and `SWW_CACHE_DIR`
"Deployment"). that of the cache files (see "State" under "Deployment").
- `bans.json` is written `SWW_STATE_WRITE_DELAY` after a ban is made, lifted or - `bans.json` is written `SWW_STATE_WRITE_DELAY` after a ban is made, lifted or
made permanent, so a burst of changes becomes one write. Every file is written made permanent, so a burst of changes becomes one write. Every file is written
every `SWW_STATE_COUNTER_INTERVAL`, and at an orderly stop. every `SWW_STATE_COUNTER_INTERVAL`, and at an orderly stop.
@@ -583,7 +583,7 @@ Each tag and where it comes from:
- They are added before `deny_nets`, so the log line of a request it refused - They are added before `deny_nets`, so the log line of a request it refused
has them. has them.
- `waf` asks routewatch in the background and never waits for it. An answer - `waf` asks routewatch in the background and never waits for it. An answer
is kept for 1 hour, in memory only. is kept for 1 hour, in memory and in `routewatch.json`.
- After a query fails, routewatch is not asked for 1 minute. - After a query fails, routewatch is not asked for 1 minute.
- A request without these tags gives the reason in the log field - A request without these tags gives the reason in the log field
`routewatch_tags_missing` (see `routewatch_tags_missing` (see
@@ -1601,8 +1601,8 @@ together.
is kept whole, comment lines included, on failure and across restarts. is kept whole, comment lines included, on failure and across restarts.
- Each list is fetched again `SWW_BLOCKLIST_REFRESH` after its last try, one - Each list is fetched again `SWW_BLOCKLIST_REFRESH` after its last try, one
list at a time. `reputation.json` keeps when each was last tried, so a restart list at a time. `lists.json` keeps when each was last tried, so a restart does
does not fetch a list again before its time. A fetch fails when: not fetch a list again before its time. A fetch fails when:
- the server answers other than `200`; - the server answers other than `200`;
- the fetch does not finish within a minute; - the fetch does not finish within a minute;
- the list is longer than 16 MiB; - the list is longer than 16 MiB;
@@ -1646,9 +1646,9 @@ together.
DNS blocklists made for mail servers list residential ranges, and would DNS blocklists made for mail servers list residential ranges, and would
block ordinary visitors, so they should not be used. Spamhaus zones need block ordinary visitors, so they should not be used. Spamhaus zones need
their keyed query service, given as a zone name that contains the key. `waf` their keyed query service, given as a zone name that contains the key. `waf`
shows the key as `********` wherever it names the zone. Only shows the key as `********` wherever it names the zone. Only `dnsbl.json`
`reputation.json` keeps the key. `SWW_DNSBL_RESOLVER` names a resolver, keeps the key. `SWW_DNSBL_RESOLVER` names a resolver, since public resolvers
since public resolvers are refused by several list operators. are refused by several list operators.
- AbuseIPDB (`SWW_ABUSEIPDB_KEY`) is asked, at its check endpoint, for the abuse - AbuseIPDB (`SWW_ABUSEIPDB_KEY`) is asked, at its check endpoint, for the abuse
confidence score of a client, from 0 to 100. Only a client that has already confidence score of a client, from 0 to 100. Only a client that has already
@@ -1659,11 +1659,11 @@ together.
client's later requests. For an IPv6 client, AbuseIPDB is asked about the client's later requests. For an IPv6 client, AbuseIPDB is asked about the
address of the request that set off the check, and the score is used for the address of the request that set off the check, and the score is used for the
whole group. A score at or over `SWW_ABUSEIPDB_MIN_SCORE` is a hit. The budget whole group. A score at or over `SWW_ABUSEIPDB_MIN_SCORE` is a hit. The budget
is counted in UTC days and kept in `reputation.json`, so a restart does not is counted in UTC days and kept in `abuseipdb-checks-spent.json`, so a restart
make it whole again. The check that spends the last of it is logged and does not make it whole again. The check that spends the last of it is logged
alerted, and no client is checked until the day ends. The key is sent in the and alerted, and no client is checked until the day ends. The key is sent in
`Key` header and nowhere else, and is never logged. A check fails, after which the `Key` header and nowhere else, and is never logged. A check fails, after
no client is checked for a minute, when: which no client is checked for a minute, when:
- AbuseIPDB answers other than `200`; - AbuseIPDB answers other than `200`;
- the answer gives no score; - the answer gives no score;
- no answer comes within `SWW_REPUTATION_TIMEOUT`. - no answer comes within `SWW_REPUTATION_TIMEOUT`.
@@ -1671,9 +1671,9 @@ together.
by default. Such verdicts are less certain than a blocklist, so a listed by default. Such verdicts are less certain than a blocklist, so a listed
client gets a quarter of every request and byte limit rather than a refusal. client gets a quarter of every request and byte limit rather than a refusal.
- Verdicts and scores are kept for `SWW_REPUTATION_CACHE_TTL`, in memory and in - Verdicts and scores are kept for `SWW_REPUTATION_CACHE_TTL`, in memory and in
`reputation.json`. A source that is down or over quota gives no verdict. The `dnsbl.json` and `abuseipdb-scores.json`. A source that is down or over quota
service continues. Each failure raises a `source_failure` alert (see gives no verdict. The service continues. Each failure raises a
"Alerting"). `source_failure` alert (see "Alerting").
### Alerting ### Alerting
@@ -1776,21 +1776,43 @@ default file.
## Persistent state ## Persistent state
All state lives in memory, and the files in `SWW_STATE_DIR` hold a copy of it, All state lives in memory, and JSON files hold a copy of it, so an orderly stop
so an orderly stop loses nothing. The files are meant for people as well: an loses nothing. The files are meant for people as well: an admin can read or edit
admin can read or edit them at any time, and the running `waf` takes the edit them at any time, and the running `waf` takes the edit in. `docs/state-files.md`
in. `docs/state-files.md` lists the files, what each holds, and their sizes. lists the files, what each holds, and their sizes.
- The state files, in `SWW_STATE_DIR`, hold what cannot be fetched again:
- `bans.json`;
- `clients.json`;
- `abuseipdb-checks-spent.json`;
- `alerts.json`.
- The cache files, in `SWW_CACHE_DIR`, hold the answers of outside services:
- `lookups.json`;
- `lists.json`;
- `dnsbl.json`;
- `abuseipdb-scores.json`;
- `routewatch.json`.
- A lost cache file costs only questions asked again, so the cache directory can
be left out of backups.
- Each file holds one kind of state, as JSON with: - Each file holds one kind of state, as JSON with:
- a top-level `version` number; - a top-level `version` number;
- entries sorted by client address; - entries sorted by client address;
- times in RFC 3339 UTC; - times in RFC 3339 UTC;
- days as `YYYY-MM-DD`; - days as `YYYY-MM-DD`;
- durations and sizes as plain numbers, with the unit in the field name. - durations and sizes as plain numbers, with the unit in the field name.
- `bans.json`, `reputation.json` and `alerts.json` are indented, so that a - These files are indented, so that a person can open `bans.json` in an editor,
person can open `bans.json` in an editor, find an address, and remove or add find an address, and remove or add an entry:
an entry. `clients.json` and `lookups.json` put each entry on one line, which - `bans.json`;
halves their size and lets `grep` show everything about one client. - `abuseipdb-checks-spent.json`;
- `alerts.json`;
- `lists.json`.
- The others put each entry on one line, which halves their size and lets `grep`
show everything about one client:
- `clients.json`;
- `lookups.json`;
- `dnsbl.json`;
- `abuseipdb-scores.json`;
- `routewatch.json`.
- Clients that were never banned are kept too, so every client's history - Clients that were never banned are kept too, so every client's history
survives a restart. survives a restart.
- Each file has a bound, so it can be rewritten whole. Writing one takes well - Each file has a bound, so it can be rewritten whole. Writing one takes well
@@ -1804,8 +1826,8 @@ in. `docs/state-files.md` lists the files, what each holds, and their sizes.
disk, so an admin's edit is never overwritten. disk, so an admin's edit is never overwritten.
- At start, the files are read back. Active bans go into an in-memory lookup by - At start, the files are read back. Active bans go into an in-memory lookup by
netblock, everything else into maps. Expired cache entries are discarded. A netblock, everything else into maps. Expired cache entries are discarded. A
missing file means empty state, and is normal on first run. These stop the missing file means empty state, and is normal on first run. In a state file,
process with a message naming the file and position: these stop the process with a message naming the file and position:
- a file that does not parse, or has an unknown `version`; - a file that does not parse, or has an unknown `version`;
- an entry without a field it needs; - an entry without a field it needs;
- a ban whose cause is not one of the four; - a ban whose cause is not one of the four;
@@ -1815,6 +1837,11 @@ in. `docs/state-files.md` lists the files, what each holds, and their sizes.
Since writes are atomic, a broken file can only come from a hand edit, which Since writes are atomic, a broken file can only come from a hand edit, which
the editor should hear about. the editor should hear about.
- A cache file never stops the start. One that does not parse is set aside as
`<name>.bad`, as an edit is below. One that cannot be read is logged and
written over at its next write. Either way its cache starts empty and fills
again.
- While running, `waf` notices a file that is edited, replaced or added. It - While running, `waf` notices a file that is edited, replaced or added. It
tells its own writes from an admin's by comparing the file with what it last tells its own writes from an admin's by comparing the file with what it last
read or wrote. An edit that parses is taken in at once: what the file says read or wrote. An edit that parses is taken in at once: what the file says
@@ -1905,10 +1932,10 @@ each app. `rproxy` asks `waf` about each visitor's request. `waf` answers
This section is the agreement between the two. The header names, the fields and This section is the agreement between the two. The header names, the fields and
the statuses below are exact. the statuses below are exact.
`waf` reads the settings, the rule files, the lookup database and the state `waf` reads the settings, the rule files, the lookup database and the state and
files, and listens at `SWW_LISTEN_ADDR`. On `SIGTERM` or `SIGINT` it stops cache files, and listens at `SWW_LISTEN_ADDR`. On `SIGTERM` or `SIGINT` it stops
taking requests and gives those in progress five seconds to finish. It then taking requests and gives those in progress five seconds to finish. It then
writes the state files. writes the state and cache files.
### The question ### The question
@@ -2481,8 +2508,10 @@ It sets no `ENTRYPOINT` or `USER` of its own. The container must start
`SWW_COMMAND` is `waf`, and `rproxy` when it is `rproxy`, unset or empty. It `SWW_COMMAND` is `waf`, and `rproxy` when it is `rproxy`, unset or empty. It
runs either as the `sww` user (uid and gid 65532), with runit's `chpst`. For runs either as the `sww` user (uid and gid 65532), with runit's `chpst`. For
`waf`, it first makes `SWW_STATE_DIR` and every file in it belong to that `waf`, it first makes `SWW_STATE_DIR` and every file in it belong to that
user. For `rproxy` it changes no owner. With any other value, it starts user. It also makes `SWW_CACHE_DIR` if it is missing, and gives it and every
nothing, and stops with a message naming `SWW_COMMAND` and its value. file in it to that user. For `rproxy` it changes no owner. With any other
value, it starts nothing, and stops with a message naming `SWW_COMMAND` and
its value.
Every `run` script, the app's included, is a bash script. It starts with Every `run` script, the app's included, is a bash script. It starts with
`#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main` `#!/usr/bin/env bash` and `set -euo pipefail`, and puts its code in a `main`
@@ -2505,7 +2534,7 @@ process that stops at start is started again every second, not in a tight loop.
- invalid configuration; - invalid configuration;
- a lookup database that is missing or unreadable; - a lookup database that is missing or unreadable;
- a rule file or state file that does not parse; - a rule file or state file that does not parse;
- an unwritable `SWW_STATE_DIR`; - an unwritable `SWW_STATE_DIR` or `SWW_CACHE_DIR`;
- a `SWW_RULES_DIR` that does not exist or cannot be read; - a `SWW_RULES_DIR` that does not exist or cannot be read;
- a `SWW_LISTEN_ADDR` it cannot listen on. - a `SWW_LISTEN_ADDR` it cannot listen on.
- Only an invalid setting, or a `SWRP_LISTEN_ADDR` it cannot listen on, stops - Only an invalid setting, or a `SWRP_LISTEN_ADDR` it cannot listen on, stops
@@ -2574,6 +2603,10 @@ needs no backup beyond the host's own. The image declares no volume, since every
app image built on it would inherit it. `rproxy` keeps nothing, and needs no app image built on it would inherit it. `rproxy` keeps nothing, and needs no
volume. volume.
`waf` keeps its cache files in `SWW_CACHE_DIR`, `/var/cache/sww` by default,
which the `run` script makes. The cache directory needs no volume and can be
left out of backups: a new container starts with empty caches, which fill again.
### Tokens ### Tokens
A token given as a file (`SWW_ADMIN_TOKEN_FILE`, `SWW_METRICS_TOKEN_FILE`) stays A token given as a file (`SWW_ADMIN_TOKEN_FILE`, `SWW_METRICS_TOKEN_FILE`) stays
+2 -1
View File
@@ -23,7 +23,8 @@ back or dropped.
- the day's AbuseIPDB checks used up. - the day's AbuseIPDB checks used up.
- `file_error`: one of: - `file_error`: one of:
- a rule file or state file edit with an error; - a rule file or state file edit with an error;
- a state file that cannot be written; - a cache file that does not parse, at start or after an edit;
- a state file or cache file that cannot be written;
- a replacement lookup database that cannot be read. - a replacement lookup database that cannot be read.
- `summary`: the hourly summary of alerts held back. - `summary`: the hourly summary of alerts held back.
+2 -2
View File
@@ -58,9 +58,9 @@ By `source`: each list's URL, each zone, and `abuseipdb`:
- `sww_reputation_last_fetch_timestamp_seconds`, for a list; - `sww_reputation_last_fetch_timestamp_seconds`, for a list;
- `sww_reputation_daily_budget_remaining`, for AbuseIPDB. - `sww_reputation_daily_budget_remaining`, for AbuseIPDB.
## State files ## State and cache files
By `file`: By `file`, the name of a state or cache file:
- `sww_state_file_writes_total`, `sww_state_file_write_failures_total`, - `sww_state_file_writes_total`, `sww_state_file_write_failures_total`,
`sww_state_file_last_write_timestamp_seconds` and `sww_state_file_size_bytes`; `sww_state_file_last_write_timestamp_seconds` and `sww_state_file_size_bytes`;
+5 -3
View File
@@ -50,14 +50,16 @@ the two (see "The image's command" below).
- `SWW_METRICS_TOKEN` (default unset): the token for `/_sww/metrics`. At least - `SWW_METRICS_TOKEN` (default unset): the token for `/_sww/metrics`. At least
32 characters. 32 characters.
## State files ## State and cache files
- `SWW_STATE_DIR` (default `/var/lib/sww`): the directory of the state files, an - `SWW_STATE_DIR` (default `/var/lib/sww`): the directory of the state files, an
absolute path that exists and can be written. absolute path that exists and can be written.
- `SWW_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be - `SWW_STATE_WRITE_DELAY` (default `10s`): how long `bans.json` waits to be
written after a ban is made, lifted or made permanent. A duration. written after a ban is made, lifted or made permanent. A duration.
- `SWW_STATE_COUNTER_INTERVAL` (default `15m`): how often every state file is - `SWW_STATE_COUNTER_INTERVAL` (default `15m`): how often every state and cache
written. A duration. file is written. A duration.
- `SWW_CACHE_DIR` (default `/var/cache/sww`): the directory of the cache files,
an absolute path that can be written. The image's `run` script makes it.
## Logging ## Logging
+73 -18
View File
@@ -1,8 +1,23 @@
# State files # State and cache files
The five JSON files in `SWW_STATE_DIR`, and what each holds. "Persistent state" The JSON files `waf` keeps, and what each holds. "Persistent state" in
in [`SPEC.md`](../SPEC.md) says how they are written, read and edited. Each has [`SPEC.md`](../SPEC.md) says how they are written, read and edited. Each has a
a top-level `version` of 1. top-level `version` of 1.
- The state files are in `SWW_STATE_DIR`, `/var/lib/sww` by default. They hold
what cannot be fetched again, so back them up:
- `bans.json`;
- `clients.json`;
- `abuseipdb-checks-spent.json`;
- `alerts.json`.
- The cache files are in `SWW_CACHE_DIR`, `/var/cache/sww` by default. They hold
answers from outside services, which are asked again when a file is lost, so
the directory can be left out of backups:
- `lookups.json`;
- `lists.json`;
- `dnsbl.json`;
- `abuseipdb-scores.json`;
- `routewatch.json`.
## `bans.json` ## `bans.json`
@@ -67,21 +82,13 @@ since it was first seen:
`blocked_by_core_rule_set`. A file may also hold `token_refused`, which is `blocked_by_core_rule_set`. A file may also hold `token_refused`, which is
kept, and which nothing adds to. kept, and which nothing adds to.
## `lookups.json` ## `abuseipdb-checks-spent.json`
GeoJS answers, one per line: The AbuseIPDB checks spent today, so that a restart does not spend the daily
budget again:
- `client`; - `day`: the day, in UTC;
- `asn`, `as_name` and `country`; - `spent`: the checks spent on it.
- `answered`: when GeoJS answered;
- `used`: when the answer was last used.
## `reputation.json`
- the last good copy of each list fetched from a URL, with when it was last
tried;
- each DNS blocklist verdict still in use;
- AbuseIPDB's scores, and the checks spent today.
## `alerts.json` ## `alerts.json`
@@ -91,6 +98,51 @@ GeoJS answers, one per line:
- the alerts sent and held back in the current hour; - the alerts sent and held back in the current hour;
- the alerts waiting for each destination. - the alerts waiting for each destination.
## `lookups.json`
A cache file. GeoJS answers, one per line:
- `client`;
- `asn`, `as_name` and `country`;
- `answered`: when GeoJS answered;
- `used`: when the answer was last used.
## `lists.json`
A cache file. The last good copy of each list fetched from a URL, with when it
was last tried:
- `url`;
- `tried`: when it was last fetched or tried;
- `fetched` and `lines`: when the last good copy was fetched, and its lines,
comment lines included.
## `dnsbl.json`
A cache file. Each DNS blocklist verdict still in use, one per line:
- `zone` and `client`;
- `listed`: whether the zone lists the client;
- `fetched`: when the zone answered.
## `abuseipdb-scores.json`
A cache file. AbuseIPDB's scores still in use, one per line:
- `client`;
- `score`: from 0 to 100;
- `fetched`: when AbuseIPDB answered.
## `routewatch.json`
A cache file. routewatch's answers from the last hour, one per line:
- `client`;
- `netblock`: the netblock the client's address is announced in, empty for an
address without a live route;
- `asn`: the number of the AS that originates the netblock, 0 for none;
- `answered`: when routewatch answered.
## Sizes ## Sizes
- `clients.json`: about 1 KiB per client, about 20 MiB at the default - `clients.json`: about 1 KiB per client, about 20 MiB at the default
@@ -102,4 +154,7 @@ GeoJS answers, one per line:
more. more.
- `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when - `lookups.json`: about 150 bytes per answer, under 1.5 GiB of writes a day when
full. full.
- `reputation.json` and `alerts.json`: usually a few MiB or less. - `routewatch.json`: about 120 bytes per answer, at most about 12 MiB.
- `abuseipdb-checks-spent.json`: a few bytes.
- `lists.json`, `dnsbl.json`, `abuseipdb-scores.json` and `alerts.json`: usually
a few MiB or less.
+5 -3
View File
@@ -56,9 +56,11 @@ const (
// of a list failing, a query to a DNSBL zone or a check with AbuseIPDB // of a list failing, a query to a DNSBL zone or a check with AbuseIPDB
// failing or refused, or the day's AbuseIPDB checks used up. // failing or refused, or the day's AbuseIPDB checks used up.
EventSourceFailure = "source_failure" EventSourceFailure = "source_failure"
// EventFileError is a rule file or state file edited while sww // EventFileError is one of:
// runs that does not parse, a replacement of the lookup database that // - a rule file or state file edited while sww runs that does not parse;
// cannot be read, or a state file that cannot be written. // - a cache file that does not parse, at start or after an edit;
// - a state file or cache file that cannot be written;
// - a replacement of the lookup database that cannot be read.
EventFileError = "file_error" EventFileError = "file_error"
// EventSummary is the summary sent as an hour ends: of the alerts held // EventSummary is the summary sent as an hour ends: of the alerts held
// back in it past SWW_ALERT_MAX_PER_HOUR, and of the repeats held // back in it past SWW_ALERT_MAX_PER_HOUR, and of the repeats held
+7 -3
View File
@@ -191,12 +191,15 @@ type Config struct {
BanScopeV4Prefix int BanScopeV4Prefix int
// StateDir is the directory of the state files, an absolute path // StateDir is the directory of the state files, an absolute path
// (SWW_STATE_DIR). bans.json is written StateWriteDelay after a ban // (SWW_STATE_DIR). bans.json is written StateWriteDelay after a ban
// is made (SWW_STATE_WRITE_DELAY), and every state file every // is made (SWW_STATE_WRITE_DELAY), and every state and cache file
// StateCounterInterval (SWW_STATE_COUNTER_INTERVAL). Neither can be // every StateCounterInterval (SWW_STATE_COUNTER_INTERVAL). Neither
// off. // can be off.
StateDir string StateDir string
StateWriteDelay time.Duration StateWriteDelay time.Duration
StateCounterInterval time.Duration StateCounterInterval time.Duration
// CacheDir is the directory of the cache files, which keep answers
// from outside services, an absolute path (SWW_CACHE_DIR).
CacheDir string
// LogRequestHeaders are the request headers whose values the request // LogRequestHeaders are the request headers whose values the request
// log gives, in lower case (SWW_LOG_REQUEST_HEADERS). // log gives, in lower case (SWW_LOG_REQUEST_HEADERS).
LogRequestHeaders []string LogRequestHeaders []string
@@ -560,6 +563,7 @@ func FromEnvironment(
StateDir: env.absolutePath("SWW_STATE_DIR", "/var/lib/sww"), StateDir: env.absolutePath("SWW_STATE_DIR", "/var/lib/sww"),
StateWriteDelay: env.durationNotOff("SWW_STATE_WRITE_DELAY", "10s"), StateWriteDelay: env.durationNotOff("SWW_STATE_WRITE_DELAY", "10s"),
StateCounterInterval: env.durationNotOff("SWW_STATE_COUNTER_INTERVAL", "15m"), StateCounterInterval: env.durationNotOff("SWW_STATE_COUNTER_INTERVAL", "15m"),
CacheDir: env.absolutePath("SWW_CACHE_DIR", "/var/cache/sww"),
LogRequestHeaders: env.headerNames("SWW_LOG_REQUEST_HEADERS", LogRequestHeaders: env.headerNames("SWW_LOG_REQUEST_HEADERS",
"accept,accept-language,accept-encoding,content-type,origin,range"), "accept,accept-language,accept-encoding,content-type,origin,range"),
LogLevel: env.logLevel("SWW_LOG_LEVEL", "info"), LogLevel: env.logLevel("SWW_LOG_LEVEL", "info"),
+21 -7
View File
@@ -74,6 +74,7 @@ const (
stateDir = "SWW_STATE_DIR" stateDir = "SWW_STATE_DIR"
stateWriteDelay = "SWW_STATE_WRITE_DELAY" stateWriteDelay = "SWW_STATE_WRITE_DELAY"
stateCounterInterval = "SWW_STATE_COUNTER_INTERVAL" stateCounterInterval = "SWW_STATE_COUNTER_INTERVAL"
cacheDir = "SWW_CACHE_DIR"
adminToken = "SWW_ADMIN_TOKEN" //nolint:gosec // the setting's name adminToken = "SWW_ADMIN_TOKEN" //nolint:gosec // the setting's name
metricsToken = "SWW_METRICS_TOKEN" //nolint:gosec // the setting's name metricsToken = "SWW_METRICS_TOKEN" //nolint:gosec // the setting's name
metricsTopN = "SWW_METRICS_TOP_N" metricsTopN = "SWW_METRICS_TOP_N"
@@ -262,6 +263,7 @@ func TestDefaults(t *testing.T) {
StateDir: "/var/lib/sww", StateDir: "/var/lib/sww",
StateWriteDelay: 10 * time.Second, StateWriteDelay: 10 * time.Second,
StateCounterInterval: 15 * time.Minute, StateCounterInterval: 15 * time.Minute,
CacheDir: "/var/cache/sww",
MetricsToken: "", MetricsToken: "",
MetricsTopN: 50, MetricsTopN: 50,
RulesDir: "/etc/sww/rules.d", RulesDir: "/etc/sww/rules.d",
@@ -346,6 +348,7 @@ func TestValuesAsSet(t *testing.T) {
stateDir: "/srv/waf-state", stateDir: "/srv/waf-state",
stateWriteDelay: "500ms", stateWriteDelay: "500ms",
stateCounterInterval: "1h", stateCounterInterval: "1h",
cacheDir: "/srv/waf-cache",
metricsToken: token, metricsToken: token,
metricsTopN: "10", metricsTopN: "10",
rulesDir: "/srv/waf-rules", rulesDir: "/srv/waf-rules",
@@ -368,6 +371,7 @@ func TestValuesAsSet(t *testing.T) {
StateDir: "/srv/waf-state", StateDir: "/srv/waf-state",
StateWriteDelay: 500 * time.Millisecond, StateWriteDelay: 500 * time.Millisecond,
StateCounterInterval: time.Hour, StateCounterInterval: time.Hour,
CacheDir: "/srv/waf-cache",
MetricsToken: token, MetricsToken: token,
MetricsTopN: 10, MetricsTopN: 10,
RulesDir: "/srv/waf-rules", RulesDir: "/srv/waf-rules",
@@ -2302,6 +2306,7 @@ func TestInvalidBanOrStateValueStopsTheStart(t *testing.T) {
{stateDir, ""}, {stateDir, "state"}, {stateDir, "./var/lib/sww"}, {stateDir, ""}, {stateDir, "state"}, {stateDir, "./var/lib/sww"},
{stateWriteDelay, off}, {stateWriteDelay, "0s"}, {stateWriteDelay, off}, {stateWriteDelay, "0s"},
{stateCounterInterval, off}, {stateCounterInterval, "15"}, {stateCounterInterval, off}, {stateCounterInterval, "15"},
{cacheDir, ""}, {cacheDir, "cache"}, {cacheDir, "./var/cache/sww"},
}) })
} }
@@ -2574,6 +2579,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
stateDir: "/var/lib/sww", stateDir: "/var/lib/sww",
stateWriteDelay: "10s", stateWriteDelay: "10s",
stateCounterInterval: "15m", stateCounterInterval: "15m",
cacheDir: "/var/cache/sww",
adminToken: "", adminToken: "",
metricsToken: "", metricsToken: "",
metricsTopN: "50", metricsTopN: "50",
@@ -2674,8 +2680,8 @@ func wantLookupSettings(t *testing.T, got *config.Config, want config.Config) {
} }
} }
// wantBanSettings checks the settings for bans, the state files, the // wantBanSettings checks the settings for bans, the state and cache
// metrics and the rule files. // files, the metrics and the rule files.
func wantBanSettings(t *testing.T, got *config.Config, want config.Config) { func wantBanSettings(t *testing.T, got *config.Config, want config.Config) {
t.Helper() t.Helper()
@@ -2694,11 +2700,7 @@ func wantBanSettings(t *testing.T, got *config.Config, want config.Config) {
got.RulesDir, got.RulesEnabled, want.RulesDir, want.RulesEnabled) got.RulesDir, got.RulesEnabled, want.RulesDir, want.RulesEnabled)
} }
if got.StateDir != want.StateDir || wantStateSettings(t, got, want)
got.StateWriteDelay != want.StateWriteDelay ||
got.StateCounterInterval != want.StateCounterInterval {
t.Errorf("state settings\n%+v\nwant\n%+v", got, want)
}
if got.MetricsToken != want.MetricsToken || got.MetricsTopN != want.MetricsTopN { if got.MetricsToken != want.MetricsToken || got.MetricsTopN != want.MetricsTopN {
t.Errorf("metrics token %q and top %d, want %q and %d", t.Errorf("metrics token %q and top %d, want %q and %d",
@@ -2706,6 +2708,18 @@ func wantBanSettings(t *testing.T, got *config.Config, want config.Config) {
} }
} }
// wantStateSettings checks the settings for the state and cache files.
func wantStateSettings(t *testing.T, got *config.Config, want config.Config) {
t.Helper()
if got.StateDir != want.StateDir ||
got.StateWriteDelay != want.StateWriteDelay ||
got.StateCounterInterval != want.StateCounterInterval ||
got.CacheDir != want.CacheDir {
t.Errorf("state settings\n%+v\nwant\n%+v", got, want)
}
}
// wantNetblocks checks a list of netblocks. // wantNetblocks checks a list of netblocks.
func wantNetblocks(t *testing.T, got []netip.Prefix, want ...string) { func wantNetblocks(t *testing.T, got []netip.Prefix, want ...string) {
t.Helper() t.Helper()
+12 -10
View File
@@ -120,17 +120,19 @@ func New(topN int, instanceName string) *Metrics {
"went on without it, because GeoJS had not given it in time.", "went on without it, because GeoJS had not given it in time.",
}), }),
stateFileWrites: counterVec("sww_state_file_writes_total", stateFileWrites: counterVec("sww_state_file_writes_total",
"Writes of each state file.", byFile), "Writes of each state or cache file.", byFile),
stateFileWriteFailures: counterVec("sww_state_file_write_failures_total", stateFileWriteFailures: counterVec("sww_state_file_write_failures_total",
"Writes of each state file that failed.", byFile), "Writes of each state or cache file that failed.", byFile),
stateFileLastWrite: gaugeVec("sww_state_file_last_write_timestamp_seconds", stateFileLastWrite: gaugeVec("sww_state_file_last_write_timestamp_seconds",
"When each state file was last written, in seconds since 1970.", byFile), "When each state or cache file was last written, in seconds since 1970.",
byFile),
stateFileSize: gaugeVec("sww_state_file_size_bytes", stateFileSize: gaugeVec("sww_state_file_size_bytes",
"The size of each state file, as it was last written.", byFile), "The size of each state or cache file, as it was last written.", byFile),
stateFileEditsTakenIn: counterVec("sww_state_file_edits_taken_in_total", stateFileEditsTakenIn: counterVec("sww_state_file_edits_taken_in_total",
"Edits of each state file taken in while running.", byFile), "Edits of each state or cache file taken in while running.", byFile),
stateFileEditsSetAside: counterVec("sww_state_file_edits_set_aside_total", stateFileEditsSetAside: counterVec("sww_state_file_edits_set_aside_total",
"Edits of each state file renamed to <name>.bad because they did not parse.", "Edits of each state or cache file renamed to <name>.bad because they "+
"did not parse.",
byFile), byFile),
} }
@@ -447,8 +449,8 @@ func (m *Metrics) CoreRuleSetMatched(mode string, id int) {
m.coreRuleSetMatches.WithLabelValues(mode, strconv.Itoa(id)).Inc() m.coreRuleSetMatches.WithLabelValues(mode, strconv.Itoa(id)).Inc()
} }
// StateFileWritten counts a write of the state file name, of size bytes, // StateFileWritten counts a write of the state or cache file name, of size
// that ended with err. // bytes, that ended with err.
func (m *Metrics) StateFileWritten(name string, size int, err error) { func (m *Metrics) StateFileWritten(name string, size int, err error) {
m.stateFileWrites.WithLabelValues(name).Inc() m.stateFileWrites.WithLabelValues(name).Inc()
@@ -466,13 +468,13 @@ func (m *Metrics) StateFileWritten(name string, size int, err error) {
m.stateFileSize.WithLabelValues(name).Set(float64(size)) m.stateFileSize.WithLabelValues(name).Set(float64(size))
} }
// StateFileEditTakenIn counts an admin's edit of the state file name // StateFileEditTakenIn counts an admin's edit of the state or cache file name
// taken in while sww runs. // taken in while sww runs.
func (m *Metrics) StateFileEditTakenIn(name string) { func (m *Metrics) StateFileEditTakenIn(name string) {
m.stateFileEditsTakenIn.WithLabelValues(name).Inc() m.stateFileEditsTakenIn.WithLabelValues(name).Inc()
} }
// StateFileEditSetAside counts an admin's edit of the state file name // StateFileEditSetAside counts an admin's edit of the state or cache file name
// renamed to name.bad because it did not parse. // renamed to name.bad because it did not parse.
func (m *Metrics) StateFileEditSetAside(name string) { func (m *Metrics) StateFileEditSetAside(name string) {
m.stateFileEditsSetAside.WithLabelValues(name).Inc() m.stateFileEditsSetAside.WithLabelValues(name).Inc()
+40 -24
View File
@@ -36,22 +36,21 @@ var (
"checks spent; none is made until the day ends at 00:00 UTC") "checks spent; none is made until the day ends at 00:00 UTC")
) )
// Score is what AbuseIPDB said about a client, as reputation.json holds // Score is what AbuseIPDB said about a client, as abuseipdb-scores.json
// it: the client, an IPv4 address or an IPv6 group, its abuse confidence // holds it: the client, an IPv4 address or an IPv6 group, its abuse
// score, from 0 to 100, and when AbuseIPDB answered. // confidence score, from 0 to 100, and when AbuseIPDB answered.
type Score struct { type Score struct {
Client netip.Prefix `json:"client"` Client netip.Prefix `json:"client"`
Score int64 `json:"score"` Score int64 `json:"score"`
Fetched time.Time `json:"fetched"` Fetched time.Time `json:"fetched"`
} }
// Checks are what reputation.json keeps of the checks of clients with // ChecksSpent is what abuseipdb-checks-spent.json keeps of the checks of
// AbuseIPDB: the day, in UTC, of the checks Spent counts, zero before the // clients with AbuseIPDB: the day, in UTC, of the checks Spent counts,
// first, and the scores still in use. // zero before the first.
type Checks struct { type ChecksSpent struct {
Day civil.Date `json:"day,omitzero"` Day civil.Date
Spent int `json:"spent"` Spent int
Scores []Score `json:"scores"`
} }
// AbuseIPDBParams are what NewAbuseIPDB needs. // AbuseIPDBParams are what NewAbuseIPDB needs.
@@ -196,35 +195,35 @@ func (a *AbuseIPDB) BudgetLeft() int {
return max(a.params.DailyBudget-a.spent, 0) return max(a.params.DailyBudget-a.spent, 0)
} }
// Snapshot returns the checks spent and every score still in use, sorted // Snapshot returns every score still in use, sorted by client, as
// by client, as reputation.json keeps them. // abuseipdb-scores.json keeps them.
func (a *AbuseIPDB) Snapshot() Checks { func (a *AbuseIPDB) Snapshot() []Score {
a.mu.Lock() a.mu.Lock()
now := a.params.Now() now := a.params.Now()
checks := Checks{Day: a.day, Spent: a.spent, Scores: make([]Score, 0, a.scores.Len())} scores := make([]Score, 0, a.scores.Len())
for _, kept := range a.scores.Values() { for _, kept := range a.scores.Values() {
if now.Sub(kept.Fetched) < a.params.CacheTTL { if now.Sub(kept.Fetched) < a.params.CacheTTL {
checks.Scores = append(checks.Scores, kept) scores = append(scores, kept)
} }
} }
a.mu.Unlock() a.mu.Unlock()
slices.SortFunc(checks.Scores, func(x, y Score) int { slices.SortFunc(scores, func(x, y Score) int {
return x.Client.Compare(y.Client) return x.Client.Compare(y.Client)
}) })
return checks return scores
} }
// Load keeps checks, read from reputation.json, in place of those it // Load keeps scores, read from abuseipdb-scores.json, in place of those
// keeps, but for the scores past maxVerdicts, those fetched longest ago. // it keeps, but for the scores past maxVerdicts, those fetched longest
// One fetched CacheTTL ago or more is neither used nor written, as for any // ago. One fetched CacheTTL ago or more is neither used nor written, as
// score. // for any score.
func (a *AbuseIPDB) Load(checks Checks) { func (a *AbuseIPDB) Load(scores []Score) {
scores := slices.Clone(checks.Scores) scores = slices.Clone(scores)
slices.SortStableFunc(scores, func(x, y Score) int { slices.SortStableFunc(scores, func(x, y Score) int {
return x.Fetched.Compare(y.Fetched) return x.Fetched.Compare(y.Fetched)
}) })
@@ -232,7 +231,6 @@ func (a *AbuseIPDB) Load(checks Checks) {
a.mu.Lock() a.mu.Lock()
defer a.mu.Unlock() defer a.mu.Unlock()
a.day, a.spent = checks.Day, checks.Spent
a.scores.Purge() a.scores.Purge()
for _, kept := range scores { for _, kept := range scores {
@@ -240,6 +238,24 @@ func (a *AbuseIPDB) Load(checks Checks) {
} }
} }
// ChecksSpent returns the checks spent, as abuseipdb-checks-spent.json
// keeps them.
func (a *AbuseIPDB) ChecksSpent() ChecksSpent {
a.mu.Lock()
defer a.mu.Unlock()
return ChecksSpent{Day: a.day, Spent: a.spent}
}
// LoadChecksSpent keeps checks, read from abuseipdb-checks-spent.json, in
// place of the checks spent.
func (a *AbuseIPDB) LoadChecksSpent(checks ChecksSpent) {
a.mu.Lock()
defer a.mu.Unlock()
a.day, a.spent = checks.Day, checks.Spent
}
// check checks client with AbuseIPDB by addr, one of its addresses, keeps // check checks client with AbuseIPDB by addr, one of its addresses, keeps
// the score as client's, and notes the check as no longer under way. A // the score as client's, and notes the check as no longer under way. A
// check that fails gives no score: it is counted, logged and raised as a // check that fails gives no score: it is counted, logged and raised as a
+15 -16
View File
@@ -17,6 +17,7 @@ import (
"testing/synctest" "testing/synctest"
"time" "time"
"cloud.google.com/go/civil"
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/metrics" "sneak.berlin/go/smallwebwaf/internal/metrics"
"sneak.berlin/go/smallwebwaf/internal/reputation" "sneak.berlin/go/smallwebwaf/internal/reputation"
@@ -179,11 +180,13 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
t.Errorf("logged\n%s\nwant the budget used up", log.String()) t.Errorf("logged\n%s\nwant the budget used up", log.String())
} }
// Restarted with what reputation.json keeps, it uses the scores, and // Restarted with what abuseipdb-scores.json and
// checks no client until the day ends. // abuseipdb-checks-spent.json keep, it uses the scores, and checks
// no client until the day ends.
restarted := &abuseIPDBStandIn{} restarted := &abuseIPDBStandIn{}
again := newAbuseIPDB(restarted, p) again := newAbuseIPDB(restarted, p)
again.Load(checker.Snapshot()) again.Load(checker.Snapshot())
again.LoadChecksSpent(checker.ChecksSpent())
wantScore(t, again, suspect, true, 100, true) wantScore(t, again, suspect, true, 100, true)
wantBudgetLeft(t, again, 0) wantBudgetLeft(t, again, 0)
@@ -224,13 +227,9 @@ func TestChecksAreCountedInUTCDaysWhateverTheClocksZone(t *testing.T) {
time.Sleep(2 * time.Hour) time.Sleep(2 * time.Hour)
wantBudgetLeft(t, checker, 0) wantBudgetLeft(t, checker, 0)
encoded, err := json.Marshal(checker.Snapshot()) want := civil.Date{Year: 2000, Month: 1, Day: 1}
if err != nil { if day := checker.ChecksSpent().Day; day != want {
t.Fatalf("encode: %v", err) t.Errorf("checks spent on %s, want on %s", day, want)
}
if !strings.Contains(string(encoded), `"day":"2000-01-01"`) {
t.Errorf("reputation.json would hold %s, want the day 2000-01-01", encoded)
} }
// At 00:00 UTC the budget is whole again. // At 00:00 UTC the budget is whole again.
@@ -302,7 +301,7 @@ func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
wantChecked(t, abuseIPDB, suspect, other) wantChecked(t, abuseIPDB, suspect, other)
wantFailures(t, checker, 2) wantFailures(t, checker, 2)
if scores := checker.Snapshot().Scores; len(scores) != 0 { if scores := checker.Snapshot(); len(scores) != 0 {
t.Errorf("scores %+v, want none", scores) t.Errorf("scores %+v, want none", scores)
} }
@@ -399,7 +398,7 @@ func TestKeyIsSentInTheKeyHeaderAndNeverShown(t *testing.T) {
for name, shown := range map[string]string{ for name, shown := range map[string]string{
"the log": log.String(), "the alerts": string(alerted), "the log": log.String(), "the alerts": string(alerted),
"the metrics": scrapeMetrics(t, m), "reputation.json": string(kept), "the metrics": scrapeMetrics(t, m), "abuseipdb-scores.json": string(kept),
} { } {
if strings.Contains(shown, key) { if strings.Contains(shown, key) {
t.Errorf("%s shows the key:\n%s", name, shown) t.Errorf("%s shows the key:\n%s", name, shown)
@@ -462,12 +461,12 @@ func TestScoreFetchedATTLAgoIsNeitherUsedNorKept(t *testing.T) {
Fetched: now.Add(-cacheTTL), Fetched: now.Add(-cacheTTL),
} }
checker.Load(reputation.Checks{Scores: []reputation.Score{stale, inUse}}) checker.Load([]reputation.Score{stale, inUse})
wantScore(t, checker, suspect, false, 100, true) wantScore(t, checker, suspect, false, 100, true)
wantScore(t, checker, other, false, 0, false) wantScore(t, checker, other, false, 0, false)
got := checker.Snapshot().Scores got := checker.Snapshot()
if !reflect.DeepEqual(got, []reputation.Score{inUse}) { if !reflect.DeepEqual(got, []reputation.Score{inUse}) {
t.Errorf("scores %+v, want only %+v", got, inUse) t.Errorf("scores %+v, want only %+v", got, inUse)
} }
@@ -481,7 +480,7 @@ func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T)
p.Now = func() time.Time { return now } p.Now = func() time.Time { return now }
checker := reputation.NewAbuseIPDB(p) checker := reputation.NewAbuseIPDB(p)
// 100,001 scores, listed by client, as reputation.json lists them, each // 100,001 scores, listed by client, as abuseipdb-scores.json lists them, each
// fetched a millisecond before the one before it: the last is one too // fetched a millisecond before the one before it: the last is one too
// many. // many.
const count = 100001 const count = 100001
@@ -497,9 +496,9 @@ func TestAtMost100000ScoresKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.T)
addr = addr.Next() addr = addr.Next()
} }
checker.Load(reputation.Checks{Scores: scores}) checker.Load(scores)
got := checker.Snapshot().Scores got := checker.Snapshot()
if len(got) != count-1 || !slices.Contains(got, scores[0]) || if len(got) != count-1 || !slices.Contains(got, scores[0]) ||
slices.Contains(got, scores[count-1]) { slices.Contains(got, scores[count-1]) {
t.Errorf("%d scores kept, want all but the one fetched longest ago", len(got)) t.Errorf("%d scores kept, want all but the one fetched longest ago", len(got))
+2 -2
View File
@@ -291,7 +291,7 @@ func crowdSecFailures() []crowdSecFailure {
// other than its decision list, such as the one a redirect names, and that // other than its decision list, such as the one a redirect names, and that
// the key is in none of what the fetches leave behind: log, the process // the key is in none of what the fetches leave behind: log, the process
// log, the alerts waiting in queue, and the copies of lists, which // log, the alerts waiting in queue, and the copies of lists, which
// reputation.json keeps. // lists.json keeps.
func wantKeyNotShown( func wantKeyNotShown(
t *testing.T, e *engine, log string, lists *reputation.Lists, queue *alerts.Queue, t *testing.T, e *engine, log string, lists *reputation.Lists, queue *alerts.Queue,
) { ) {
@@ -326,7 +326,7 @@ func TestCrowdSecDecisionListKeptAcrossARestartEndsWhenItsDecisionsDo(t *testing
lists := start(t, e, crowdSecParams()) lists := start(t, e, crowdSecParams())
kept := lists.Snapshot() kept := lists.Snapshot()
// Restarted half an hour later with what reputation.json keeps, and // Restarted half an hour later with what lists.json keeps, and
// the engine down, the decision still bans, until the end it had at // the engine down, the decision still bans, until the end it had at
// the fetch, half an hour on. // the fetch, half an hour on.
time.Sleep(30 * time.Minute) time.Sleep(30 * time.Minute)
+3 -3
View File
@@ -40,7 +40,7 @@ var (
errNotListing = errors.New("the answer is outside 127.0.0.0/8") errNotListing = errors.New("the answer is outside 127.0.0.0/8")
) )
// Verdict is what a zone said about a client, as reputation.json holds // Verdict is what a zone said about a client, as dnsbl.json holds
// it: the zone, the client's address, whether the zone lists it, and when // it: the zone, the client's address, whether the zone lists it, and when
// the zone answered. // the zone answered.
type Verdict struct { type Verdict struct {
@@ -187,7 +187,7 @@ func (d *DNSBL) Failures(zone string) int {
} }
// Snapshot returns every verdict still in use, sorted by client, then by // Snapshot returns every verdict still in use, sorted by client, then by
// zone, as reputation.json lists them. // zone, as dnsbl.json lists them.
func (d *DNSBL) Snapshot() []Verdict { func (d *DNSBL) Snapshot() []Verdict {
d.mu.Lock() d.mu.Lock()
@@ -209,7 +209,7 @@ func (d *DNSBL) Snapshot() []Verdict {
return verdicts return verdicts
} }
// Load keeps verdicts, read from reputation.json, in place of those it // Load keeps verdicts, read from dnsbl.json, in place of those it
// keeps, but for those of a zone SWW_DNSBL_ZONES does not name, and, // keeps, but for those of a zone SWW_DNSBL_ZONES does not name, and,
// past maxVerdicts, those fetched longest ago. One fetched CacheTTL ago or // past maxVerdicts, those fetched longest ago. One fetched CacheTTL ago or
// more is neither used nor written, as for any verdict. // more is neither used nor written, as for any verdict.
+2 -2
View File
@@ -389,7 +389,7 @@ func TestVerdictsKeptAcrossARestart(t *testing.T) {
t.Errorf("verdicts %+v, want %+v", kept, want) t.Errorf("verdicts %+v, want %+v", kept, want)
} }
// Restarted an hour later with what reputation.json keeps, it uses // Restarted an hour later with what dnsbl.json keeps, it uses
// the verdicts, and asks the zone nothing, until the TTL has passed // the verdicts, and asks the zone nothing, until the TTL has passed
// since they were fetched. // since they were fetched.
time.Sleep(time.Hour) time.Sleep(time.Hour)
@@ -446,7 +446,7 @@ func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(t *testing.
p.Now = func() time.Time { return now } p.Now = func() time.Time { return now }
dnsbl := reputation.NewDNSBL(p) dnsbl := reputation.NewDNSBL(p)
// 100,001 verdicts, listed by client, as reputation.json lists them, // 100,001 verdicts, listed by client, as dnsbl.json lists them,
// each fetched a millisecond before the one before it: the last is one // each fetched a millisecond before the one before it: the last is one
// too many. // too many.
const count = 100001 const count = 100001
+7 -7
View File
@@ -7,8 +7,8 @@
// and when each was last tried. It also asks the DNSBL zones of // and when each was last tried. It also asks the DNSBL zones of
// SWW_DNSBL_ZONES about clients, and keeps their verdicts, and checks // SWW_DNSBL_ZONES about clients, and keeps their verdicts, and checks
// clients with AbuseIPDB, and keeps their scores and the checks spent // clients with AbuseIPDB, and keeps their scores and the checks spent
// today. The state package writes all of these to reputation.json and // today. The state package writes all of these to files and reads them
// reads them from it, so that a restart keeps them too. // from there, so that a restart keeps them too.
package reputation package reputation
import ( import (
@@ -54,7 +54,7 @@ var (
"does not give an address or a netblock and a duration, such as 4h0m0s") "does not give an address or a netblock and a duration, such as 4h0m0s")
) )
// List is a list as reputation.json holds it: the URL it is fetched from, // List is a list as lists.json holds it: the URL it is fetched from,
// when it was last tried, the fetch failed or not, and its last good copy: // when it was last tried, the fetch failed or not, and its last good copy:
// when that was fetched, and its lines, as fetched, comment lines // when that was fetched, and its lines, as fetched, comment lines
// included, both left out while no fetch of it has succeeded. // included, both left out while no fetch of it has succeeded.
@@ -106,7 +106,7 @@ type Lists struct {
lists map[string]*list lists map[string]*list
} }
// list is one list: what reputation.json keeps of it, its last try, zero // list is one list: what lists.json keeps of it, its last try, zero
// before the first, and its last good copy, what that copy says, and how // before the first, and its last good copy, what that copy says, and how
// many fetches of it failed. // many fetches of it failed.
type list struct { type list struct {
@@ -272,7 +272,7 @@ func (l *Lists) Failures(listURL string) int {
// Run fetches each list once it is due, as due tells, until ctx is done. A // Run fetches each list once it is due, as due tells, until ctx is done. A
// list never tried is fetched at once, and so is one that is due by its // list never tried is fetched at once, and so is one that is due by its
// last try or copy read from reputation.json. // last try or copy read from lists.json.
func (l *Lists) Run(ctx context.Context) { func (l *Lists) Run(ctx context.Context) {
if len(l.lists) == 0 { if len(l.lists) == 0 {
return return
@@ -292,7 +292,7 @@ func (l *Lists) Run(ctx context.Context) {
} }
// Snapshot returns each list that has been tried, with its copy, if it // Snapshot returns each list that has been tried, with its copy, if it
// has one, sorted by URL, as reputation.json lists them. // has one, sorted by URL, as lists.json lists them.
func (l *Lists) Snapshot() []List { func (l *Lists) Snapshot() []List {
l.mu.Lock() l.mu.Lock()
@@ -313,7 +313,7 @@ func (l *Lists) Snapshot() []List {
return tried return tried
} }
// Load puts lists, read from reputation.json, in place of the last tries // Load puts lists, read from lists.json, in place of the last tries
// and copies held. A list Params does not name is dropped. A copy with a // and copies held. A list Params does not name is dropped. A copy with a
// line that parse refuses is an error, and then nothing changes. // line that parse refuses is an error, and then nothing changes.
func (l *Lists) Load(lists []List) error { func (l *Lists) Load(lists []List) error {
+3 -3
View File
@@ -260,7 +260,7 @@ func TestKeptCopyIsFetchedAgainOnceRefreshHasPassedSinceItWasFetched(t *testing.
lists.SetTransport(servers) lists.SetTransport(servers)
// drop.txt was fetched an hour ago, and tor.txt a refresh ago, as // drop.txt was fetched an hour ago, and tor.txt a refresh ago, as
// reputation.json says at start. // lists.json says at start.
err := lists.Load([]reputation.List{ err := lists.Load([]reputation.List{
{URL: dropURL, Fetched: time.Now().Add(-time.Hour), Lines: []string{"203.0.113.9"}}, {URL: dropURL, Fetched: time.Now().Add(-time.Hour), Lines: []string{"203.0.113.9"}},
{URL: torURL, Fetched: time.Now().Add(-refresh), Lines: []string{"198.51.100.9"}}, {URL: torURL, Fetched: time.Now().Add(-refresh), Lines: []string{"198.51.100.9"}},
@@ -297,7 +297,7 @@ func TestRestartWaitsRefreshAfterTheLastTryEvenOneThatFailed(t *testing.T) {
time.Sleep(refresh) time.Sleep(refresh)
wantFetches(t, servers, 2) wantFetches(t, servers, 2)
// Restarted with what reputation.json keeps, it waits a refresh // Restarted with what lists.json keeps, it waits a refresh
// after the failed try, as it does while it runs. // after the failed try, as it does while it runs.
restarted := &standIn{lists: map[string]string{dropURL: "198.51.100.2\n"}} restarted := &standIn{lists: map[string]string{dropURL: "198.51.100.2\n"}}
again := reputation.New(params(dropURL)) again := reputation.New(params(dropURL))
@@ -353,7 +353,7 @@ func TestFetchCutOffAsItStopsIsNoFailureButARestartWaitsForIt(t *testing.T) {
waiting(queue)) waiting(queue))
} }
// Restarted an hour later with what reputation.json keeps, it fetches // Restarted an hour later with what lists.json keeps, it fetches
// tor.txt, never tried, at once, and drop.txt a refresh after its // tor.txt, never tried, at once, and drop.txt a refresh after its
// cut-off try. // cut-off try.
time.Sleep(time.Hour) time.Sleep(time.Hour)
+86 -27
View File
@@ -1,6 +1,8 @@
// Package routewatch asks a routewatch instance about client addresses, in the // Package routewatch asks a routewatch instance about client addresses, in the
// background, and keeps its answers in memory. A routewatch instance knows, for // background, and keeps its answers. A routewatch instance knows, for an
// an address, the netblock it is announced in and the AS that originates it. // address, the netblock it is announced in and the AS that originates it. The
// answers are kept in memory, for at most 100,000 clients and for an hour each,
// and are written to routewatch.json and read from it by the state package.
package routewatch package routewatch
import ( import (
@@ -13,6 +15,7 @@ import (
"net/http" "net/http"
"net/netip" "net/netip"
"net/url" "net/url"
"slices"
"sync" "sync"
"time" "time"
@@ -65,6 +68,17 @@ type Route struct {
ASN uint32 ASN uint32
} }
// Answer is what routewatch answered about a client, as routewatch.json holds
// it: the netblock the client's address is announced in, none for an address
// without a live route, the AS that originates it, and when routewatch
// answered.
type Answer struct {
Client netip.Prefix `json:"client"`
Netblock netip.Prefix `json:"netblock"`
ASN uint32 `json:"asn"`
Answered time.Time `json:"answered"`
}
// Params are what New needs. // Params are what New needs.
type Params struct { type Params struct {
// URL is where routewatch is (SWW_ROUTEWATCH_URL). // URL is where routewatch is (SWW_ROUTEWATCH_URL).
@@ -84,23 +98,16 @@ type Client struct {
mu sync.Mutex mu sync.Mutex
// answers are by client. Each is added as it comes and never moved up, // answers are by client. Each is added as it comes and never moved up,
// so that the one that came longest ago is the first dropped. // so that the one that came longest ago is the first dropped.
answers *simplelru.LRU[netip.Prefix, answer] answers *simplelru.LRU[netip.Prefix, Answer]
// asking are the clients whose query is under way. // asking are the clients whose query is under way.
asking map[netip.Prefix]bool asking map[netip.Prefix]bool
// retryAt is when routewatch may be asked again after a query failed. // retryAt is when routewatch may be asked again after a query failed.
retryAt time.Time retryAt time.Time
} }
// answer is one kept answer: the route, or why there is none, and when it came.
type answer struct {
route Route
missing Missing
came time.Time
}
// New returns a Client with no answer yet. // New returns a Client with no answer yet.
func New(params Params) *Client { func New(params Params) *Client {
answers, err := simplelru.NewLRU[netip.Prefix, answer](maxAnswers, nil) answers, err := simplelru.NewLRU[netip.Prefix, Answer](maxAnswers, nil)
if err != nil { if err != nil {
panic(err) // NewLRU fails only for a size below one panic(err) // NewLRU fails only for a size below one
} }
@@ -128,8 +135,12 @@ func (c *Client) Route(
now := c.params.Now() now := c.params.Now()
kept, found := c.answers.Peek(client) kept, found := c.answers.Peek(client)
if found && now.Sub(kept.came) < answerKept { if found && now.Sub(kept.Answered) < answerKept {
return kept.route, kept.missing if !kept.Netblock.IsValid() {
return Route{}, NoRoute
}
return Route{Netblock: kept.Netblock, ASN: kept.ASN}, ""
} }
if now.Before(c.retryAt) { if now.Before(c.retryAt) {
@@ -145,12 +156,58 @@ func (c *Client) Route(
return Route{}, NoAnswerYet return Route{}, NoAnswerYet
} }
// Snapshot returns every answer still in use, sorted by client, as
// routewatch.json lists them.
func (c *Client) Snapshot() []Answer {
c.mu.Lock()
now := c.params.Now()
answers := make([]Answer, 0, c.answers.Len())
for _, kept := range c.answers.Values() {
if now.Sub(kept.Answered) < answerKept {
answers = append(answers, kept)
}
}
c.mu.Unlock()
slices.SortFunc(answers, func(a, b Answer) int {
return a.Client.Compare(b.Client)
})
return answers
}
// Load keeps answers, read from routewatch.json, in place of those it keeps,
// but for those that came answerKept ago or more and, past maxAnswers, those
// that came longest ago.
func (c *Client) Load(answers []Answer) {
answers = slices.Clone(answers)
slices.SortStableFunc(answers, func(a, b Answer) int {
return a.Answered.Compare(b.Answered)
})
c.mu.Lock()
defer c.mu.Unlock()
c.answers.Purge()
now := c.params.Now()
for _, kept := range answers {
if now.Sub(kept.Answered) < answerKept {
c.answers.Add(kept.Client, kept)
}
}
}
// ask asks routewatch about addr, keeps the answer as client's, and notes the // ask asks routewatch about addr, keeps the answer as client's, and notes the
// query as no longer under way. A query that fails gives no answer, and // query as no longer under way. A query that fails gives no answer, and
// routewatch is not asked again for failureKept. Only the first failure of // routewatch is not asked again for failureKept. Only the first failure of
// those under way is logged. // those under way is logged.
func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr) { func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr) {
route, missing, err := c.query(ctx, addr) route, err := c.query(ctx, addr)
now := c.params.Now() now := c.params.Now()
c.mu.Lock() c.mu.Lock()
@@ -160,7 +217,9 @@ func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr)
firstFailure := err != nil && !now.Before(c.retryAt) firstFailure := err != nil && !now.Before(c.retryAt)
if err == nil { if err == nil {
c.answers.Add(client, answer{route: route, missing: missing, came: now}) c.answers.Add(client, Answer{
Client: client, Netblock: route.Netblock, ASN: route.ASN, Answered: now,
})
} else { } else {
c.retryAt = now.Add(failureKept) c.retryAt = now.Add(failureKept)
} }
@@ -174,19 +233,19 @@ func (c *Client) ask(ctx context.Context, client netip.Prefix, addr netip.Addr)
// query asks routewatch's IP info API about addr. An answer other than 200, one // query asks routewatch's IP info API about addr. An answer other than 200, one
// that cannot be read, and none within queryTimeout, fail. A result with no // that cannot be read, and none within queryTimeout, fail. A result with no
// netblock is an address without a live route. // netblock is an address without a live route, and gives a Route without one.
func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, Missing, error) { func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, error) {
ctx, cancel := context.WithTimeout(ctx, queryTimeout) ctx, cancel := context.WithTimeout(ctx, queryTimeout)
defer cancel() defer cancel()
endpoint, err := url.JoinPath(c.params.URL, "api/v1/ip", addr.String()) endpoint, err := url.JoinPath(c.params.URL, "api/v1/ip", addr.String())
if err != nil { if err != nil {
return Route{}, "", fmt.Errorf("make the URL: %w", err) return Route{}, fmt.Errorf("make the URL: %w", err)
} }
req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, http.NoBody) req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, http.NoBody)
if err != nil { if err != nil {
return Route{}, "", fmt.Errorf("make the request: %w", err) return Route{}, fmt.Errorf("make the request: %w", err)
} }
req.Header.Set("Accept", "application/json") req.Header.Set("Accept", "application/json")
@@ -195,7 +254,7 @@ func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, Missing, er
if err != nil { if err != nil {
// Do's error names the URL, which holds the client's address, which // Do's error names the URL, which holds the client's address, which
// is not to be logged: only what went wrong is kept. // is not to be logged: only what went wrong is kept.
return Route{}, "", fmt.Errorf("ask routewatch: %w", errors.Unwrap(err)) return Route{}, fmt.Errorf("ask routewatch: %w", errors.Unwrap(err))
} }
defer func() { defer func() {
@@ -203,14 +262,14 @@ func (c *Client) query(ctx context.Context, addr netip.Addr) (Route, Missing, er
}() }()
if res.StatusCode != http.StatusOK { if res.StatusCode != http.StatusOK {
return Route{}, "", fmt.Errorf("%w %s", errStatus, res.Status) return Route{}, fmt.Errorf("%w %s", errStatus, res.Status)
} }
return readAnswer(io.LimitReader(res.Body, maxAnswerBytes)) return readAnswer(io.LimitReader(res.Body, maxAnswerBytes))
} }
// readAnswer reads routewatch's answer about one address. // readAnswer reads routewatch's answer about one address.
func readAnswer(body io.Reader) (Route, Missing, error) { func readAnswer(body io.Reader) (Route, error) {
var decoded struct { var decoded struct {
Data struct { Data struct {
Results []struct { Results []struct {
@@ -222,22 +281,22 @@ func readAnswer(body io.Reader) (Route, Missing, error) {
err := json.NewDecoder(body).Decode(&decoded) err := json.NewDecoder(body).Decode(&decoded)
if err != nil { if err != nil {
return Route{}, "", fmt.Errorf("read the answer: %w", err) return Route{}, fmt.Errorf("read the answer: %w", err)
} }
if len(decoded.Data.Results) == 0 { if len(decoded.Data.Results) == 0 {
return Route{}, "", errNoResult return Route{}, errNoResult
} }
result := decoded.Data.Results[0] result := decoded.Data.Results[0]
if result.Netblock == "" { if result.Netblock == "" {
return Route{}, NoRoute, nil return Route{}, nil
} }
netblock, err := netip.ParsePrefix(result.Netblock) netblock, err := netip.ParsePrefix(result.Netblock)
if err != nil { if err != nil {
return Route{}, "", fmt.Errorf("%w: %q", errNotNetblock, result.Netblock) return Route{}, fmt.Errorf("%w: %q", errNotNetblock, result.Netblock)
} }
return Route{Netblock: netblock.Masked(), ASN: result.ASN}, "", nil return Route{Netblock: netblock.Masked(), ASN: result.ASN}, nil
} }
+94
View File
@@ -7,6 +7,7 @@ import (
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"net/netip" "net/netip"
"slices"
"strings" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
@@ -336,3 +337,96 @@ func TestRepeatedRequestsMakeOneQueryAndTheAnswerExpires(t *testing.T) {
t.Errorf("%d queries after the answer expired, want 2", got) t.Errorf("%d queries after the answer expired, want 2", got)
} }
} }
func TestSnapshotHoldsTheAnswerUntilItExpires(t *testing.T) {
t.Parallel()
f := newFixture(t, answering(routedAnswer))
f.route()
f.waitFor("")
want := []routewatch.Answer{{
Client: netip.MustParsePrefix(clientAddr + "/32"),
Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496,
Answered: f.clk.Now(),
}}
if got := f.client.Snapshot(); !slices.Equal(got, want) {
t.Errorf("snapshot %+v, want %+v", got, want)
}
f.clk.advance(time.Hour)
if got := f.client.Snapshot(); len(got) != 0 {
t.Errorf("snapshot %+v an hour after the answer, want none", got)
}
}
func TestLoadedAnswersAreUsedForAnHourFromWhenTheyCame(t *testing.T) {
t.Parallel()
f := newFixture(t, answering(routedAnswer))
now := f.clk.Now()
kept := routewatch.Answer{
Client: netip.MustParsePrefix(clientAddr + "/32"),
Netblock: netip.MustParsePrefix("198.51.100.0/24"), ASN: 64497,
Answered: now.Add(-time.Hour + time.Second),
}
stale := routewatch.Answer{
Client: netip.MustParsePrefix("192.0.2.2/32"),
Netblock: netip.MustParsePrefix("192.0.2.0/24"), ASN: 64496,
Answered: now.Add(-time.Hour),
}
f.client.Load([]routewatch.Answer{kept, stale})
if got := f.client.Snapshot(); !slices.Equal(got, []routewatch.Answer{kept}) {
t.Errorf("kept %+v, want only the answer that came less than an hour ago", got)
}
route, missing := f.route()
want := routewatch.Route{Netblock: kept.Netblock, ASN: kept.ASN}
if route != want || missing != "" {
t.Errorf("route %+v (%q), want %+v", route, missing, want)
}
if got := f.queries.Load(); got != 0 {
t.Errorf("%d queries for a client with an answer loaded, want none", got)
}
}
func TestLoadDropsTheAnswerThatCameLongestAgoFirst(t *testing.T) {
t.Parallel()
const maxAnswers = 100000
f := newFixture(t, answering(routedAnswer))
now := f.clk.Now()
// routewatch.json lists the answers by client. Here each came a
// millisecond before the one listed before it, so the last listed is
// the one that came longest ago, and the one dropped.
answers := make([]routewatch.Answer, maxAnswers+1)
addr := netip.MustParseAddr("10.0.0.0")
for i := range answers {
answers[i] = routewatch.Answer{
Client: netip.PrefixFrom(addr, addr.BitLen()),
Answered: now.Add(-time.Duration(i) * time.Millisecond),
}
addr = addr.Next()
}
f.client.Load(answers)
got := f.client.Snapshot()
if len(got) != maxAnswers {
t.Fatalf("%d answers kept, want %d", len(got), maxAnswers)
}
if got[0] != answers[0] || got[maxAnswers-1] != answers[maxAnswers-1] {
t.Errorf("answers kept from %s to %s, want from %s to %s", got[0].Client,
got[maxAnswers-1].Client, answers[0].Client, answers[maxAnswers-1].Client)
}
}
+463 -157
View File
@@ -1,13 +1,23 @@
// Package state keeps sww's state in JSON files in // Package state keeps sww's state in JSON files, as the "Persistent
// SWW_STATE_DIR, as the "Persistent state" section of SPEC.md requires // state" section of SPEC.md requires and docs/state-files.md lists them.
// and docs/state-files.md lists them: //
// bans.json holds the bans, clients.json each client's counters and // The state files are in SWW_STATE_DIR:
// history, lookups.json GeoJS's answers, reputation.json the last try and // - bans.json holds the bans;
// last good copy of each list fetched from a URL, the DNSBL zones' // - clients.json holds each client's counters and history;
// verdicts, and AbuseIPDB's scores and checks spent, and alerts.json the // - abuseipdb-checks-spent.json holds the AbuseIPDB checks spent today;
// cooldowns, the hour under way, the alerts waiting for each destination // - alerts.json holds the cooldowns, the hour under way, the alerts
// and the anomaly counters. Load // waiting for each destination and the anomaly counters.
// reads them at start, Watch takes in an admin's edit of one while //
// The cache files are in SWW_CACHE_DIR. Each keeps answers from an
// outside service, which are fetched again when the file is lost:
// - lookups.json holds GeoJS's answers;
// - lists.json holds the last try and last good copy of each list
// fetched from a URL;
// - dnsbl.json holds the DNSBL zones' verdicts;
// - abuseipdb-scores.json holds AbuseIPDB's scores;
// - routewatch.json holds routewatch's answers.
//
// Load reads them at start, Watch takes in an admin's edit of one while
// sww runs, and Run and WriteAll write them. The disk is read and // sww runs, and Run and WriteAll write them. The disk is read and
// written outside the parts' locks, which are held only to take a // written outside the parts' locks, which are held only to take a
// snapshot or to put in what a file holds, so that no request waits on // snapshot or to put in what a file holds, so that no request waits on
@@ -31,6 +41,7 @@ import (
"sync" "sync"
"time" "time"
"cloud.google.com/go/civil"
"github.com/fsnotify/fsnotify" "github.com/fsnotify/fsnotify"
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly" "sneak.berlin/go/smallwebwaf/internal/anomaly"
@@ -39,6 +50,7 @@ import (
"sneak.berlin/go/smallwebwaf/internal/metrics" "sneak.berlin/go/smallwebwaf/internal/metrics"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation" "sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/routewatch"
) )
// version is the version of the files' format, the only one read. // version is the version of the files' format, the only one read.
@@ -48,15 +60,42 @@ const version = 1
// hold visitors' addresses. // hold visitors' addresses.
const fileMode = 0o600 const fileMode = 0o600
// The state files' names. // The state files' names, in Dir.
const ( const (
bansJSON = "bans.json" bansJSON = "bans.json"
clientsJSON = "clients.json" clientsJSON = "clients.json"
lookupsJSON = "lookups.json" abuseIPDBChecksSpentJSON = "abuseipdb-checks-spent.json"
reputationJSON = "reputation.json" alertsJSON = "alerts.json"
alertsJSON = "alerts.json"
) )
// The cache files' names, in CacheDir.
const (
lookupsJSON = "lookups.json"
listsJSON = "lists.json"
dnsblJSON = "dnsbl.json"
abuseIPDBScoresJSON = "abuseipdb-scores.json"
routeWatchJSON = "routewatch.json"
)
// fileNames returns the names of the state files, then those of the
// cache files, in the order they are written.
func fileNames() []string {
return []string{
bansJSON, clientsJSON, abuseIPDBChecksSpentJSON, alertsJSON,
lookupsJSON, listsJSON, dnsblJSON, abuseIPDBScoresJSON, routeWatchJSON,
}
}
// isCacheFile reports whether name is the name of a cache file.
func isCacheFile(name string) bool {
switch name {
case lookupsJSON, listsJSON, dnsblJSON, abuseIPDBScoresJSON, routeWatchJSON:
return true
default:
return false
}
}
var ( var (
errVersion = errors.New("unknown version") errVersion = errors.New("unknown version")
// errMissing is for an entry without a field it needs. // errMissing is for an entry without a field it needs.
@@ -72,24 +111,29 @@ var (
// Params are what Load needs. // Params are what Load needs.
type Params struct { type Params struct {
// Dir is the directory of the state files (SWW_STATE_DIR). // Dir is the directory of the state files (SWW_STATE_DIR), and
Dir string // CacheDir that of the cache files (SWW_CACHE_DIR).
Dir string
CacheDir string
// WriteDelay is how long after a ban is made bans.json is written // WriteDelay is how long after a ban is made bans.json is written
// (SWW_STATE_WRITE_DELAY), and CounterInterval how often every file // (SWW_STATE_WRITE_DELAY), and CounterInterval how often every file
// is (SWW_STATE_COUNTER_INTERVAL). // is (SWW_STATE_COUNTER_INTERVAL).
WriteDelay time.Duration WriteDelay time.Duration
CounterInterval time.Duration CounterInterval time.Duration
// Ledger, Limiter, GeoJS, Lists, DNSBL, AbuseIPDB, Alerts and Anomalies // Ledger, Limiter, GeoJS, Lists, DNSBL, AbuseIPDB, RouteWatch, Alerts and
// hold the state. Alerts also receive a file_error alert for an edit set // Anomalies hold the state. RouteWatch is nil while SWW_ROUTEWATCH_URL is
// aside, and for a write that fails while sww runs. // unset, and routewatch.json then holds no answers. Alerts also receive a
Ledger *bans.Ledger // file_error alert for an edit set aside, and for a write that fails while
Limiter *ratelimit.Limiter // sww runs.
GeoJS *lookup.GeoJS Ledger *bans.Ledger
Lists *reputation.Lists Limiter *ratelimit.Limiter
DNSBL *reputation.DNSBL GeoJS *lookup.GeoJS
AbuseIPDB *reputation.AbuseIPDB Lists *reputation.Lists
Alerts *alerts.Queue DNSBL *reputation.DNSBL
Anomalies *anomaly.Counters AbuseIPDB *reputation.AbuseIPDB
RouteWatch *routewatch.Client
Alerts *alerts.Queue
Anomalies *anomaly.Counters
// Now tells the time by which the counters' buckets run out, normally // Now tells the time by which the counters' buckets run out, normally
// time.Now in UTC. // time.Now in UTC.
Now func() time.Time Now func() time.Time
@@ -101,7 +145,7 @@ type Params struct {
Metrics *metrics.Metrics Metrics *metrics.Metrics
} }
// Files are the state files of a running sww. // Files are the state and cache files of a running sww.
type Files struct { type Files struct {
params Params params Params
@@ -147,13 +191,38 @@ type lookupsFile struct {
Lookups []lookup.Answer `json:"lookups"` Lookups []lookup.Answer `json:"lookups"`
} }
// reputationFile is reputation.json, indented for an admin to read and // abuseIPDBChecksSpentFile is abuseipdb-checks-spent.json, indented for an
// edit, so that each line of a list's copy is on a line of its own. // admin to read and edit.
type reputationFile struct { type abuseIPDBChecksSpentFile struct {
Version int `json:"version"` Version int `json:"version"`
Lists []reputation.List `json:"lists"` Day civil.Date `json:"day,omitzero"`
Verdicts []reputation.Verdict `json:"verdicts"` Spent int `json:"spent"`
AbuseIPDB reputation.Checks `json:"abuseipdb"` }
// listsFile is lists.json, indented for an admin to read and edit, so
// that each line of a list's copy is on a line of its own.
type listsFile struct {
Version int `json:"version"`
Lists []reputation.List `json:"lists"`
}
// dnsblFile is dnsbl.json, with each verdict on a line of its own.
type dnsblFile struct {
Version int `json:"version"`
Verdicts []reputation.Verdict `json:"verdicts"`
}
// abuseIPDBScoresFile is abuseipdb-scores.json, with each score on a line
// of its own.
type abuseIPDBScoresFile struct {
Version int `json:"version"`
Scores []reputation.Score `json:"scores"`
}
// routeWatchFile is routewatch.json, with each answer on a line of its own.
type routeWatchFile struct {
Version int `json:"version"`
Answers []routewatch.Answer `json:"answers"`
} }
// alertsFile is alerts.json, indented for an admin to read and edit. // alertsFile is alerts.json, indented for an admin to read and edit.
@@ -178,34 +247,45 @@ type stateFile interface {
toUTC() toUTC()
} }
// Load checks that files can be written in Dir, and reads the state files // Load checks that files can be written in Dir and CacheDir, and reads the
// in it into the parts of Params that hold the state. A missing file is // files in them into the parts of Params that hold the state. A missing
// empty state, as on a first start. A file that does not parse, has an // file is empty state, as on a first start. A state file that does not
// unknown version, or has an entry without a field it needs, is an error // parse, has an unknown version, or has an entry without a field it
// that names the file and, where the JSON decoder tells it, the line and // needs, is an error that names the file and, where the JSON decoder
// column, or else the entry. // tells it, the line and column, or else the entry. A cache file is never
// an error, as readCache describes.
func Load(params Params) (*Files, error) { func Load(params Params) (*Files, error) {
err := checkWritable(params.Dir) err := checkWritable(params.Dir)
if err != nil { if err != nil {
return nil, fmt.Errorf("SWW_STATE_DIR cannot be written: %w", err) return nil, fmt.Errorf("SWW_STATE_DIR cannot be written: %w", err)
} }
err = checkWritable(params.CacheDir)
if err != nil {
return nil, fmt.Errorf("SWW_CACHE_DIR cannot be written: %w", err)
}
f := &Files{params: params, sums: map[string][sha256.Size]byte{}} f := &Files{params: params, sums: map[string][sha256.Size]byte{}}
bansRead, bansErr := f.read(bansJSON) bansRead, bansErr := f.read(bansJSON)
clientsRead, clientsErr := f.read(clientsJSON) clientsRead, clientsErr := f.read(clientsJSON)
lookupsRead, lookupsErr := f.read(lookupsJSON) _, checksErr := f.read(abuseIPDBChecksSpentJSON)
reputationRead, reputationErr := f.read(reputationJSON)
alertsRead, alertsErr := f.read(alertsJSON) alertsRead, alertsErr := f.read(alertsJSON)
err = errors.Join(bansErr, clientsErr, lookupsErr, reputationErr, alertsErr) err = errors.Join(bansErr, clientsErr, checksErr, alertsErr)
if err != nil { if err != nil {
return nil, err return nil, err
} }
params.ProcessLog.Info("read the state files", "directory", params.Dir, lookupsRead := f.readCache(lookupsJSON)
"bans", bansRead, "clients", clientsRead, "lookups", lookupsRead, listsRead := f.readCache(listsJSON)
"lists", reputationRead, "alerts_waiting", alertsRead) f.readCache(dnsblJSON)
f.readCache(abuseIPDBScoresJSON)
f.readCache(routeWatchJSON)
params.ProcessLog.Info("read the state and cache files", "directory", params.Dir,
"cache_directory", params.CacheDir, "bans", bansRead, "clients", clientsRead,
"lookups", lookupsRead, "lists", listsRead, "alerts_waiting", alertsRead)
return f, nil return f, nil
} }
@@ -234,69 +314,101 @@ func (f *Files) Run(ctx context.Context) {
f.logFailure(bansJSON, f.writeFile(bansJSON)) f.logFailure(bansJSON, f.writeFile(bansJSON))
case <-interval.C: case <-interval.C:
for _, name := range []string{ for _, name := range fileNames() {
bansJSON, clientsJSON, lookupsJSON, reputationJSON, alertsJSON,
} {
f.logFailure(name, f.writeFile(name)) f.logFailure(name, f.writeFile(name))
} }
} }
} }
} }
// WriteAll writes every state file, as sww stops. A file that // WriteAll writes every state and cache file, as sww stops. A file that
// fails does not keep the others from being written. // fails does not keep the others from being written.
func (f *Files) WriteAll() error { func (f *Files) WriteAll() error {
return errors.Join(f.writeFile(bansJSON), f.writeFile(clientsJSON), names := fileNames()
f.writeFile(lookupsJSON), f.writeFile(reputationJSON), f.writeFile(alertsJSON)) errs := make([]error, 0, len(names))
}
// Watch watches Dir until ctx is done, and takes in an admin's edit of a for _, name := range names {
// state file as soon as it is saved: what the file holds replaces what errs = append(errs, f.writeFile(name))
// sww held for it. An edit that does not parse is left for the
// file's next write, which sets it aside, since a file can be read while
// an editor is still writing it. If Dir cannot be watched, that is
// logged, and an edit is taken in only before its file is written.
func (f *Files) Watch(ctx context.Context) {
watcher, err := fsnotify.NewWatcher()
if err == nil {
defer func() {
_ = watcher.Close()
}()
err = watcher.Add(f.params.Dir)
} }
return errors.Join(errs...)
}
// Watch watches Dir and CacheDir until ctx is done, and takes in an
// admin's edit of a file as soon as it is saved: what the file holds
// replaces what sww held for it. An edit that does not parse is left for
// the file's next write, which sets it aside, since a file can be read
// while an editor is still writing it. A directory that cannot be watched
// is logged, and the other is still watched. An edit in a directory not
// watched is taken in only before its file is written.
func (f *Files) Watch(ctx context.Context) {
watcher, err := fsnotify.NewWatcher()
if err != nil { if err != nil {
f.params.ProcessLog.Error("cannot watch the state files for edits", f.params.ProcessLog.Error("cannot watch the state and cache files for edits",
"error", err.Error()) "error", err.Error())
return return
} }
f.params.ProcessLog.Info("watching the state files for edits", defer func() {
"directory", f.params.Dir) _ = watcher.Close()
}()
var watched []string
for _, dir := range []struct{ path, failed string }{
{f.params.Dir, "cannot watch the state files for edits"},
{f.params.CacheDir, "cannot watch the cache files for edits"},
} {
err = watcher.Add(dir.path)
if err != nil {
f.params.ProcessLog.Error(dir.failed, "directory", dir.path,
"error", err.Error())
continue
}
watched = append(watched, dir.path)
}
watching := "watching the state and cache files for edits"
switch {
case len(watched) == 0:
return
case len(watched) == 1 && watched[0] == f.params.Dir:
watching = "watching the state files for edits"
case len(watched) == 1:
watching = "watching the cache files for edits"
}
f.params.ProcessLog.Info(watching, "directories", watched)
for { for {
select { select {
case <-ctx.Done(): case <-ctx.Done():
return return
case event := <-watcher.Events: case event := <-watcher.Events:
switch name := filepath.Base(event.Name); name { name := filepath.Base(event.Name)
case bansJSON, clientsJSON, lookupsJSON, reputationJSON, alertsJSON: if slices.Contains(fileNames(), name) &&
filepath.Join(f.dir(name), name) == event.Name {
f.fileChanged(name) f.fileChanged(name)
} }
case err = <-watcher.Errors: case err = <-watcher.Errors:
f.params.ProcessLog.Warn("watching the state files failed", f.params.ProcessLog.Warn("watching the state and cache files failed",
"error", err.Error()) "error", err.Error())
} }
} }
} }
// logFailure logs a write of the state file name that failed, and raises // logFailure logs a write of the file name that failed, and raises
// a file_error alert for it. // a file_error alert for it.
func (f *Files) logFailure(name string, err error) { func (f *Files) logFailure(name string, err error) {
if err != nil { if err != nil {
const failed = "writing the state files failed" failed := "writing the state files failed"
if isCacheFile(name) {
failed = "writing the cache files failed"
}
// Raised before it is logged, so that the alert is there once the // Raised before it is logged, so that the alert is there once the
// log line is. // log line is.
@@ -304,14 +416,14 @@ func (f *Files) logFailure(name string, err error) {
Event: alerts.EventFileError, Event: alerts.EventFileError,
Reason: failed, Reason: failed,
Detail: map[string]any{ Detail: map[string]any{
"file": filepath.Join(f.params.Dir, name), "error": err.Error(), "file": filepath.Join(f.dir(name), name), "error": err.Error(),
}, },
}) })
f.params.ProcessLog.Error(failed, "error", err.Error()) f.params.ProcessLog.Error(failed, "error", err.Error())
} }
} }
// fileChanged takes in what the state file name holds, as Watch sees it // fileChanged takes in what the file name holds, as Watch sees it
// change, if that is an edit made since sww last read or wrote // change, if that is an edit made since sww last read or wrote
// the file. A file that cannot be read or does not parse is left for its // the file. A file that cannot be read or does not parse is left for its
// next write. // next write.
@@ -327,7 +439,7 @@ func (f *Files) fileChanged(name string) {
_ = f.takeInEdit(name, data) _ = f.takeInEdit(name, data)
} }
// takeInEdit takes in data, an edit of the state file name, as takeIn // takeInEdit takes in data, an edit of the file name, as takeIn
// does, and counts and logs it. Every edit taken in while sww // does, and counts and logs it. Every edit taken in while sww
// runs, by Watch or by a write, is taken in here. An edit that does not // runs, by Watch or by a write, is taken in here. An edit that does not
// parse is neither counted nor logged, and takeIn's error returned. // parse is neither counted nor logged, and takeIn's error returned.
@@ -337,17 +449,31 @@ func (f *Files) takeInEdit(name string, data []byte) error {
return err return err
} }
tookIn := "took in an edit of a state file"
if isCacheFile(name) {
tookIn = "took in an edit of a cache file"
}
// Counted before it is logged, so that the count is there once the // Counted before it is logged, so that the count is there once the
// log line is. // log line is.
f.params.Metrics.StateFileEditTakenIn(name) f.params.Metrics.StateFileEditTakenIn(name)
f.params.ProcessLog.Info("took in an edit of a state file", f.params.ProcessLog.Info(tookIn, "file", filepath.Join(f.dir(name), name))
"file", filepath.Join(f.params.Dir, name))
return nil return nil
} }
// read takes in the state file name at start, and returns how many // dir returns the directory of the file name: CacheDir for a cache file,
// entries it holds. A missing file holds none. // and Dir for a state file.
func (f *Files) dir(name string) string {
if isCacheFile(name) {
return f.params.CacheDir
}
return f.params.Dir
}
// read takes in the file name at start, and returns how many entries it
// holds. A missing file holds none.
func (f *Files) read(name string) (int, error) { func (f *Files) read(name string) (int, error) {
data, changed, err := f.readChanged(name) data, changed, err := f.readChanged(name)
if err != nil || !changed { if err != nil || !changed {
@@ -357,14 +483,44 @@ func (f *Files) read(name string) (int, error) {
return f.takeIn(name, data, false) return f.takeIn(name, data, false)
} }
// readChanged returns what the state file name holds, and whether that // readCache takes in the cache file name at start, as read does. A cache
// has changed since sww last read or wrote the file, as it has // file that cannot be read, or does not parse, does not stop the start:
// for a file sww never read or wrote. A missing file has not // it is logged, and its cache starts empty and fills again. One that does
// changed: it is written again at its next write. // not parse is set aside as an edit is, and one that cannot be read is
func (f *Files) readChanged(name string) ([]byte, bool, error) { // written over at its next write.
path := filepath.Join(f.params.Dir, name) func (f *Files) readCache(name string) int {
data, changed, err := f.readChanged(name)
if err != nil || !changed {
f.logUnread(name, err)
data, err := os.ReadFile(path) //nolint:gosec // a state file, in SWW_STATE_DIR return 0
}
entries, err := f.takeIn(name, data, false)
if err != nil {
f.logUnread(name, f.setAside(name, err))
}
return entries
}
// logUnread logs err, the reason the cache file name was not read at
// start, unless it is nil.
func (f *Files) logUnread(name string, err error) {
if err != nil {
f.params.ProcessLog.Warn("cannot read a cache file, so its cache starts empty",
"file", filepath.Join(f.dir(name), name), "error", err.Error())
}
}
// readChanged returns what the file name holds, and whether that has
// changed since sww last read or wrote the file, as it has for a file
// sww never read or wrote. A missing file has not changed: it is written
// again at its next write.
func (f *Files) readChanged(name string) ([]byte, bool, error) {
path := filepath.Join(f.dir(name), name)
data, err := os.ReadFile(path) //nolint:gosec // a file in one of the two directories
if errors.Is(err, fs.ErrNotExist) { if errors.Is(err, fs.ErrNotExist) {
return nil, false, nil return nil, false, nil
} }
@@ -376,15 +532,39 @@ func (f *Files) readChanged(name string) ([]byte, bool, error) {
return data, sha256.Sum256(data) != f.sums[name], nil return data, sha256.Sum256(data) != f.sums[name], nil
} }
// takeIn parses data, what the state file name holds, puts it into the // takeIn parses data, what the file name holds, puts it into the
// part that keeps that state, in place of what the part held, and returns // part that keeps that state, in place of what the part held, and returns
// how many entries the file holds. edit is whether data is an admin's // how many entries the file holds. edit is whether data is an admin's
// edit taken in while sww runs, rather than the file read at the // edit taken in while sww runs, rather than the file read at the
// start. An error names the file and, where the JSON decoder tells it, // start. An error names the file and, where the JSON decoder tells it,
// the line and column, or else the entry. // the line and column, or else the entry. abuseipdb-checks-spent.json
// holds no entries.
func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) { func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
path := filepath.Join(f.params.Dir, name) path := filepath.Join(f.dir(name), name)
var (
entries int
err error
)
if isCacheFile(name) {
entries, err = f.takeInCache(name, path, data)
} else {
entries, err = f.takeInState(name, path, data, edit)
}
if err != nil {
return 0, err
}
f.sums[name] = sha256.Sum256(data)
return entries, nil
}
// takeInState parses data, what the state file name, at path, holds, and
// puts it in, as takeIn describes.
func (f *Files) takeInState(name, path string, data []byte, edit bool) (int, error) {
var entries int var entries int
switch name { switch name {
@@ -418,6 +598,33 @@ func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
f.params.Limiter.Load(file.Clients, f.params.Now()) f.params.Limiter.Load(file.Clients, f.params.Now())
entries = len(file.Clients) entries = len(file.Clients)
case abuseIPDBChecksSpentJSON:
var file abuseIPDBChecksSpentFile
err := parse(path, data, &file)
if err != nil {
return 0, err
}
f.params.AbuseIPDB.LoadChecksSpent(reputation.ChecksSpent{
Day: file.Day, Spent: file.Spent,
})
case alertsJSON:
waiting, err := f.takeInAlerts(path, data)
if err != nil {
return 0, err
}
entries = waiting
}
return entries, nil
}
// takeInCache parses data, what the cache file name, at path, holds, and
// puts it in, as takeIn describes.
func (f *Files) takeInCache(name, path string, data []byte) (int, error) {
switch name {
case lookupsJSON: case lookupsJSON:
var file lookupsFile var file lookupsFile
@@ -427,9 +634,10 @@ func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
} }
f.params.GeoJS.Load(file.Lookups) f.params.GeoJS.Load(file.Lookups)
entries = len(file.Lookups)
case reputationJSON: return len(file.Lookups), nil
var file reputationFile case listsJSON:
var file listsFile
err := parse(path, data, &file) err := parse(path, data, &file)
if err != nil { if err != nil {
@@ -441,21 +649,43 @@ func (f *Files) takeIn(name string, data []byte, edit bool) (int, error) {
return 0, fmt.Errorf("%s: %w", path, err) return 0, fmt.Errorf("%s: %w", path, err)
} }
f.params.DNSBL.Load(file.Verdicts) return len(file.Lists), nil
f.params.AbuseIPDB.Load(file.AbuseIPDB) case dnsblJSON:
entries = len(file.Lists) var file dnsblFile
case alertsJSON:
waiting, err := f.takeInAlerts(path, data) err := parse(path, data, &file)
if err != nil { if err != nil {
return 0, err return 0, err
} }
entries = waiting f.params.DNSBL.Load(file.Verdicts)
return len(file.Verdicts), nil
case routeWatchJSON:
var file routeWatchFile
err := parse(path, data, &file)
if err != nil {
return 0, err
}
if f.params.RouteWatch != nil {
f.params.RouteWatch.Load(file.Answers)
}
return len(file.Answers), nil
default: // abuseipdb-scores.json
var file abuseIPDBScoresFile
err := parse(path, data, &file)
if err != nil {
return 0, err
}
f.params.AbuseIPDB.Load(file.Scores)
return len(file.Scores), nil
} }
f.sums[name] = sha256.Sum256(data)
return entries, nil
} }
// takeInAlerts parses data, what alerts.json, at path, holds, puts it // takeInAlerts parses data, what alerts.json, at path, holds, puts it
@@ -493,17 +723,22 @@ func (f *Files) takeInAlerts(path string, data []byte) (int, error) {
return entries, nil return entries, nil
} }
// writeFile writes the state file name from what sww holds. An // writeFile writes the file name from what sww holds. An edit made since
// edit made since sww last read or wrote the file is taken in // sww last read or wrote the file is taken in first, so that it is not
// first, so that it is not overwritten, or set aside if it does not // overwritten, or set aside if it does not parse. A state file that
// parse. A file that cannot be read, or an edit that cannot be set // cannot be read, or an edit that cannot be set aside, is left as it is,
// aside, is left as it is, and the write given up. Every write is counted // and the write given up. A cache file that cannot be read is written
// in the metrics, and one that fails or is given up as a failure. // over, since what it held is fetched again. Every write is counted in
// the metrics, and one that fails or is given up as a failure.
func (f *Files) writeFile(name string) error { func (f *Files) writeFile(name string) error {
f.mu.Lock() f.mu.Lock()
defer f.mu.Unlock() defer f.mu.Unlock()
data, changed, err := f.readChanged(name) data, changed, err := f.readChanged(name)
if err != nil && isCacheFile(name) {
err = nil
}
if err == nil && changed { if err == nil && changed {
err = f.takeInEdit(name, data) err = f.takeInEdit(name, data)
if err != nil { if err != nil {
@@ -519,7 +754,7 @@ func (f *Files) writeFile(name string) error {
} }
if err == nil { if err == nil {
err = write(f.params.Dir, name, data) err = write(f.dir(name), name, data)
} }
if err == nil { if err == nil {
@@ -527,7 +762,7 @@ func (f *Files) writeFile(name string) error {
// fails, so that its next read does not take it for an admin's // fails, so that its next read does not take it for an admin's
// edit. // edit.
f.sums[name] = sha256.Sum256(data) f.sums[name] = sha256.Sum256(data)
err = syncDirectory(f.params.Dir) err = syncDirectory(f.dir(name))
} }
f.params.Metrics.StateFileWritten(name, len(data), err) f.params.Metrics.StateFileWritten(name, len(data), err)
@@ -535,20 +770,23 @@ func (f *Files) writeFile(name string) error {
return err return err
} }
// setAside renames the state file name, an edit that does not parse with // setAside renames the file name, an edit that does not parse with
// parseErr, to name.bad, for the admin to mend, and logs it with where in // parseErr, to name.bad, for the admin to mend, and logs it with where in
// the file the error is, and raises a file_error alert for it. If the // the file the error is, and raises a file_error alert for it. If the
// rename fails, the edit is left as it is, and the error returned is // rename fails, the edit is left as it is, and the error returned is
// parseErr joined with the rename's. // parseErr joined with the rename's.
func (f *Files) setAside(name string, parseErr error) error { func (f *Files) setAside(name string, parseErr error) error {
path := filepath.Join(f.params.Dir, name) path := filepath.Join(f.dir(name), name)
err := os.Rename(path, path+".bad") err := os.Rename(path, path+".bad")
if err != nil { if err != nil {
return errors.Join(parseErr, err) return errors.Join(parseErr, err)
} }
const setAside = "set aside an edit of a state file that does not parse" setAside := "set aside an edit of a state file that does not parse"
if isCacheFile(name) {
setAside = "set aside an edit of a cache file that does not parse"
}
// Raised before it is logged, so that the alert is there once the log // Raised before it is logged, so that the alert is there once the log
// line is. // line is.
@@ -563,7 +801,7 @@ func (f *Files) setAside(name string, parseErr error) error {
return nil return nil
} }
// encode returns the state file name as sww writes it, from a // encode returns the file name as sww writes it, from a
// snapshot of the part that keeps that state. // snapshot of the part that keeps that state.
func (f *Files) encode(name string) ([]byte, error) { func (f *Files) encode(name string) ([]byte, error) {
switch name { switch name {
@@ -575,10 +813,24 @@ func (f *Files) encode(name string) ([]byte, error) {
return encodeOnePerLine("clients", f.params.Limiter.Snapshot()) return encodeOnePerLine("clients", f.params.Limiter.Snapshot())
case lookupsJSON: case lookupsJSON:
return encodeOnePerLine("lookups", f.params.GeoJS.Snapshot()) return encodeOnePerLine("lookups", f.params.GeoJS.Snapshot())
case reputationJSON: case listsJSON:
return encodeIndented(reputationFile{ return encodeIndented(listsFile{Version: version, Lists: f.params.Lists.Snapshot()})
Version: version, Lists: f.params.Lists.Snapshot(), case dnsblJSON:
Verdicts: f.params.DNSBL.Snapshot(), AbuseIPDB: f.params.AbuseIPDB.Snapshot(), return encodeOnePerLine("verdicts", f.params.DNSBL.Snapshot())
case abuseIPDBScoresJSON:
return encodeOnePerLine("scores", f.params.AbuseIPDB.Snapshot())
case routeWatchJSON:
var answers []routewatch.Answer
if f.params.RouteWatch != nil {
answers = f.params.RouteWatch.Snapshot()
}
return encodeOnePerLine("answers", answers)
case abuseIPDBChecksSpentJSON:
checks := f.params.AbuseIPDB.ChecksSpent()
return encodeIndented(abuseIPDBChecksSpentFile{
Version: version, Day: checks.Day, Spent: checks.Spent,
}) })
default: // alerts.json default: // alerts.json
held := f.params.Alerts.Snapshot() held := f.params.Alerts.Snapshot()
@@ -748,14 +1000,8 @@ func (f *lookupsFile) check(data []byte) error {
// check refuses a list without its URL, which would name no list, or the // check refuses a list without its URL, which would name no list, or the
// time it was last tried, which would have it fetched at once, and a copy // time it was last tried, which would have it fetched at once, and a copy
// of it without the time it was fetched, or without its lines, which hold // of it without the time it was fetched, or without its lines, which hold
// the list. It refuses a verdict without its zone or its client, which // the list.
// would be about no one, whether the zone lists the client, or the time func (f *listsFile) check([]byte) error {
// it was fetched, which would drop it, and so an AbuseIPDB score without
// its client, the score, or the time it was fetched. A verdict's listed is
// false for a client the zone does not list, and a score can be 0, which
// the structs cannot tell from a missing one, so each is read again as
// written.
func (f *reputationFile) check(data []byte) error {
for i, kept := range f.Lists { for i, kept := range f.Lists {
switch { switch {
case kept.URL == "": case kept.URL == "":
@@ -769,6 +1015,15 @@ func (f *reputationFile) check(data []byte) error {
} }
} }
return nil
}
// check refuses a verdict without its zone or its client, which would be
// about no one, whether the zone lists the client, or the time it was
// fetched, which would drop it. A verdict's listed is false for a client
// the zone does not list, which Verdict cannot tell from a missing one,
// so each listed is read again as written.
func (f *dnsblFile) check(data []byte) error {
var written struct { var written struct {
Verdicts []struct { Verdicts []struct {
Listed *bool `json:"listed"` Listed *bool `json:"listed"`
@@ -783,28 +1038,27 @@ func (f *reputationFile) check(data []byte) error {
for i, verdict := range f.Verdicts { for i, verdict := range f.Verdicts {
switch { switch {
case verdict.Zone == "": case verdict.Zone == "":
return fmt.Errorf("verdicts %w", missing(i, "zone")) return missing(i, "zone")
case !verdict.Client.IsValid(): case !verdict.Client.IsValid():
return fmt.Errorf("verdicts %w", missing(i, "client")) return missing(i, "client")
case written.Verdicts[i].Listed == nil: case written.Verdicts[i].Listed == nil:
return fmt.Errorf("verdicts %w", missing(i, "listed")) return missing(i, "listed")
case verdict.Fetched.IsZero(): case verdict.Fetched.IsZero():
return fmt.Errorf("verdicts %w", missing(i, "fetched")) return missing(i, "fetched")
} }
} }
return checkScores(f.AbuseIPDB.Scores, data) return nil
} }
// checkScores refuses an AbuseIPDB score, of scores, read from data, as // check refuses an AbuseIPDB score without its client, the score, or the
// reputationFile's check describes. // time it was fetched. A score can be 0, which Score cannot tell from a
func checkScores(scores []reputation.Score, data []byte) error { // missing one, so each score is read again as written.
func (f *abuseIPDBScoresFile) check(data []byte) error {
var written struct { var written struct {
AbuseIPDB struct { Scores []struct {
Scores []struct { Score *int64 `json:"score"`
Score *int64 `json:"score"` } `json:"scores"`
} `json:"scores"`
} `json:"abuseipdb"`
} }
err := json.Unmarshal(data, &written) err := json.Unmarshal(data, &written)
@@ -812,20 +1066,57 @@ func checkScores(scores []reputation.Score, data []byte) error {
return err return err
} }
for i, kept := range scores { for i, kept := range f.Scores {
switch { switch {
case !kept.Client.IsValid(): case !kept.Client.IsValid():
return fmt.Errorf("abuseipdb scores %w", missing(i, "client")) return missing(i, "client")
case written.AbuseIPDB.Scores[i].Score == nil: case written.Scores[i].Score == nil:
return fmt.Errorf("abuseipdb scores %w", missing(i, "score")) return missing(i, "score")
case kept.Fetched.IsZero(): case kept.Fetched.IsZero():
return fmt.Errorf("abuseipdb scores %w", missing(i, "fetched")) return missing(i, "fetched")
} }
} }
return nil return nil
} }
// check refuses an answer without its client, which would answer for
// nobody, its netblock, or the time routewatch gave it, which would drop
// it. "" is the netblock of an address without a live route, which Answer
// cannot tell from a missing one, so each netblock is read again as
// written.
func (f *routeWatchFile) check(data []byte) error {
var written struct {
Answers []struct {
Netblock *string `json:"netblock"`
} `json:"answers"`
}
err := json.Unmarshal(data, &written)
if err != nil {
return err
}
for i, answer := range f.Answers {
switch {
case !answer.Client.IsValid():
return missing(i, "client")
case written.Answers[i].Netblock == nil:
return missing(i, "netblock")
case answer.Answered.IsZero():
return missing(i, "answered")
}
}
return nil
}
// check refuses nothing, since a missing day or count means no check
// spent.
func (f *abuseIPDBChecksSpentFile) check([]byte) error {
return nil
}
// check refuses a cooldown without its event or when its alert was sent, // check refuses a cooldown without its event or when its alert was sent,
// which would hold back no repeat, alerts waiting for a destination with // which would hold back no repeat, alerts waiting for a destination with
// another name than webhook, slack or ntfy, most likely misspelt, an // another name than webhook, slack or ntfy, most likely misspelt, an
@@ -947,22 +1238,37 @@ func (f *lookupsFile) toUTC() {
} }
} }
// toUTC converts every time of each list, verdict and AbuseIPDB score to // toUTC converts both times of each list to UTC.
// UTC. func (f *listsFile) toUTC() {
func (f *reputationFile) toUTC() {
for i := range f.Lists { for i := range f.Lists {
convertToUTC(&f.Lists[i].Tried, &f.Lists[i].Fetched) convertToUTC(&f.Lists[i].Tried, &f.Lists[i].Fetched)
} }
}
// toUTC converts the time of each verdict to UTC.
func (f *dnsblFile) toUTC() {
for i := range f.Verdicts { for i := range f.Verdicts {
convertToUTC(&f.Verdicts[i].Fetched) convertToUTC(&f.Verdicts[i].Fetched)
} }
}
for i := range f.AbuseIPDB.Scores { // toUTC converts the time of each AbuseIPDB score to UTC.
convertToUTC(&f.AbuseIPDB.Scores[i].Fetched) func (f *abuseIPDBScoresFile) toUTC() {
for i := range f.Scores {
convertToUTC(&f.Scores[i].Fetched)
} }
} }
// toUTC converts the time of each answer to UTC.
func (f *routeWatchFile) toUTC() {
for i := range f.Answers {
convertToUTC(&f.Answers[i].Answered)
}
}
// toUTC does nothing: abuseipdb-checks-spent.json holds a day, not a time.
func (f *abuseIPDBChecksSpentFile) toUTC() {}
// toUTC converts every time of each cooldown, of the hour, of each alert // toUTC converts every time of each cooldown, of the hour, of each alert
// waiting and of each anomaly counter to UTC. // waiting and of each anomaly counter to UTC.
func (f *alertsFile) toUTC() { func (f *alertsFile) toUTC() {
@@ -1114,7 +1420,7 @@ func write(dir, name string, data []byte) error {
// syncDirectory syncs dir to the disk, so that a rename in it lasts. // syncDirectory syncs dir to the disk, so that a rename in it lasts.
func syncDirectory(dir string) error { func syncDirectory(dir string) error {
directory, err := os.Open(dir) //nolint:gosec // SWW_STATE_DIR itself directory, err := os.Open(dir) //nolint:gosec // SWW_STATE_DIR or SWW_CACHE_DIR itself
if err != nil { if err != nil {
return err return err
} }
@@ -1124,7 +1430,7 @@ func syncDirectory(dir string) error {
// writeSynced writes data to the file at path, and syncs it to the disk. // writeSynced writes data to the file at path, and syncs it to the disk.
func writeSynced(path string, data []byte) error { func writeSynced(path string, data []byte) error {
//nolint:gosec // a state file's temporary file, in SWW_STATE_DIR //nolint:gosec // a file's temporary file, in SWW_STATE_DIR or SWW_CACHE_DIR
file, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, fileMode) file, err := os.OpenFile(path, os.O_WRONLY|os.O_CREATE|os.O_TRUNC, fileMode)
if err != nil { if err != nil {
return err return err
File diff suppressed because it is too large. Load diff
+16 -14
View File
@@ -1,9 +1,9 @@
// Package sww runs the sww process, as the command it is given. // Package sww runs the sww process, as the command it is given.
// `sww waf` reads the settings, the rule files, the lookup database // `sww waf` reads the settings, the rule files, the lookup database
// and the state files, answers rproxy's questions until it is told to stop, and // and the state and cache files, answers rproxy's questions until it is told to
// then stops in an orderly way, writing the state files. `sww rproxy` // stop, and then stops in an orderly way, writing the state and cache files.
// reads rproxy's settings alone, and runs the reverse proxy in front of one app // `sww rproxy` reads rproxy's settings alone, and runs the reverse proxy in
// that asks waf about each request. `sww healthcheck` is the // front of one app that asks waf about each request. `sww healthcheck` is the
// container's health check. // container's health check.
package sww package sww
@@ -101,8 +101,8 @@ func printUsage(w io.Writer) int {
} }
// RunWAF is `sww waf`: it reads the settings, the rule files, // RunWAF is `sww waf`: it reads the settings, the rule files,
// the lookup database and the state files, then serves requests until ctx is // the lookup database and the state and cache files, then serves requests until
// done. It returns the process's exit status, 1 when waf cannot start. // ctx is done. It returns the process's exit status, 1 when waf cannot start.
func RunWAF(ctx context.Context, params Params) int { func RunWAF(ctx context.Context, params Params) int {
// Until the settings are read, the one message is an invalid setting's // Until the settings are read, the one message is an invalid setting's
// error, which every SWW_LOG_LEVEL lets through. // error, which every SWW_LOG_LEVEL lets through.
@@ -171,7 +171,7 @@ func RunWAF(ctx context.Context, params Params) int {
files, err := loadStateFiles(cfg, server, alertQueue, now, processLog) files, err := loadStateFiles(cfg, server, alertQueue, now, processLog)
if err != nil { if err != nil {
processLog.Error("cannot use the state files", "error", err.Error()) processLog.Error("cannot use the state and cache files", "error", err.Error())
return 1 return 1
} }
@@ -236,14 +236,15 @@ func newServer(
return server, nil return server, nil
} }
// loadStateFiles reads the state files into the parts of server and into // loadStateFiles reads the state and cache files into the parts of server and
// alertQueue, as state.Load does. // into alertQueue, as state.Load does.
func loadStateFiles( func loadStateFiles(
cfg *config.Config, server *waf.Server, alertQueue *alerts.Queue, cfg *config.Config, server *waf.Server, alertQueue *alerts.Queue,
now func() time.Time, processLog *slog.Logger, now func() time.Time, processLog *slog.Logger,
) (*state.Files, error) { ) (*state.Files, error) {
return state.Load(state.Params{ return state.Load(state.Params{
Dir: cfg.StateDir, Dir: cfg.StateDir,
CacheDir: cfg.CacheDir,
WriteDelay: cfg.StateWriteDelay, WriteDelay: cfg.StateWriteDelay,
CounterInterval: cfg.StateCounterInterval, CounterInterval: cfg.StateCounterInterval,
Ledger: server.Ledger, Ledger: server.Ledger,
@@ -252,6 +253,7 @@ func loadStateFiles(
Lists: server.Lists, Lists: server.Lists,
DNSBL: server.DNSBL, DNSBL: server.DNSBL,
AbuseIPDB: server.AbuseIPDB, AbuseIPDB: server.AbuseIPDB,
RouteWatch: server.RouteWatch,
Alerts: alertQueue, Alerts: alertQueue,
Anomalies: server.Anomalies, Anomalies: server.Anomalies,
Now: now, Now: now,
@@ -317,13 +319,13 @@ func startSending(
} }
} }
// serve serves requests on listener, writes the state files as they are // serve serves requests on listener, writes the state and cache files as they
// due, takes in an admin's edits of them, reads the rule files again as // are due, takes in an admin's edits of them, reads the rule files again as
// they change, and the lookup database when it is replaced, fetches the // they change, and the lookup database when it is replaced, fetches the
// lists the settings name by URL as they are due, and sends the alerts, // lists the settings name by URL as they are due, and sends the alerts,
// until ctx is done. Then it gives the requests in progress // until ctx is done. Then it gives the requests in progress
// shutdownTimeout to finish, and writes every state file, alerts.json with // shutdownTimeout to finish, and writes every state and cache file,
// the alerts still waiting. // alerts.json with the alerts still waiting.
func serve( func serve(
ctx context.Context, server *waf.Server, listener net.Listener, ctx context.Context, server *waf.Server, listener net.Listener,
files *state.Files, ruleFiles *rules.Files, alertQueue *alerts.Queue, files *state.Files, ruleFiles *rules.Files, alertQueue *alerts.Queue,
@@ -393,7 +395,7 @@ func serve(
err = files.WriteAll() err = files.WriteAll()
if err != nil { if err != nil {
processLog.Error("writing the state files failed", "error", err.Error()) processLog.Error("writing the state and cache files failed", "error", err.Error())
return 1 return 1
} }
+53 -18
View File
@@ -38,6 +38,7 @@ const (
stateDir = "SWW_STATE_DIR" stateDir = "SWW_STATE_DIR"
stateWriteDelay = "SWW_STATE_WRITE_DELAY" stateWriteDelay = "SWW_STATE_WRITE_DELAY"
stateCounterInterval = "SWW_STATE_COUNTER_INTERVAL" stateCounterInterval = "SWW_STATE_COUNTER_INTERVAL"
cacheDir = "SWW_CACHE_DIR"
rateLimitPerDay = "SWW_RATE_LIMIT_PER_DAY" rateLimitPerDay = "SWW_RATE_LIMIT_PER_DAY"
rateLimitExemptNets = "SWW_RATE_LIMIT_EXEMPT_NETS" rateLimitExemptNets = "SWW_RATE_LIMIT_EXEMPT_NETS"
rulesDir = "SWW_RULES_DIR" rulesDir = "SWW_RULES_DIR"
@@ -160,6 +161,12 @@ func run(ctx context.Context, env map[string]string, out *output) int {
return "off", true return "off", true
} }
// The cache files go beside the state files unless the test
// says otherwise.
if !ok && name == cacheDir {
return env[stateDir], env[stateDir] != ""
}
return value, ok return value, ok
}, },
Stdout: out, Stdout: out,
@@ -699,7 +706,7 @@ func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
} }
} }
func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T) { func TestBlocklistTriesAndCopiesKeptInListsJSONAcrossRestarts(t *testing.T) {
t.Parallel() t.Parallel()
const ( const (
@@ -732,10 +739,11 @@ func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T)
t.Cleanup(server.Close) t.Cleanup(server.Close)
torURL, dropURL := server.URL+torPath, server.URL+dropPath torURL, dropURL := server.URL+torPath, server.URL+dropPath
dir := t.TempDir() cache := t.TempDir()
env := map[string]string{ env := map[string]string{
listenAddr: localhost + ":0", listenAddr: localhost + ":0",
stateDir: dir, stateDir: t.TempDir(),
cacheDir: cache,
rulesDir: t.TempDir(), rulesDir: t.TempDir(),
metricsToken: token, metricsToken: token,
instanceName: instance, instanceName: instance,
@@ -755,7 +763,7 @@ func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T)
}) })
// Restarted with drop.txt named after it and the server answering, tor.txt // Restarted with drop.txt named after it and the server answering, tor.txt
// waits SWW_BLOCKLIST_REFRESH after its failed try, kept in reputation.json, // waits SWW_BLOCKLIST_REFRESH after its failed try, kept in lists.json,
// while drop.txt, never tried, is fetched at once. Lists are fetched in the // while drop.txt, never tried, is fetched at once. Lists are fetched in the
// order named, so once drop.txt refuses the client, tor.txt has had its turn. // order named, so once drop.txt refuses the client, tor.txt has had its turn.
failing.Store(false) failing.Store(false)
@@ -773,8 +781,8 @@ func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T)
} }
// After another restart, with the server failing, the copy of drop.txt // After another restart, with the server failing, the copy of drop.txt
// kept in reputation.json, its copyright line included, refuses the // kept in lists.json, its copyright line included, refuses the client
// client from the first request. // from the first request.
failing.Store(true) failing.Store(true)
out = runUntilStopped(t, env, func(url string) { out = runUntilStopped(t, env, func(url string) {
@@ -782,16 +790,16 @@ func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T)
}) })
wantDeniedByList(t, out.line(t, "type", "request"), dropURL) wantDeniedByList(t, out.line(t, "type", "request"), dropURL)
path := filepath.Join(dir, "reputation.json") path := filepath.Join(cache, "lists.json")
kept, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory kept, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
if err != nil || !strings.Contains(string(kept), `"`+copyright+`"`) { if err != nil || !strings.Contains(string(kept), `"`+copyright+`"`) {
t.Errorf("reputation.json holds\n%s\nwant the copy with %q (%v)", kept, copyright, t.Errorf("lists.json holds\n%s\nwant the copy with %q (%v)", kept, copyright,
err) err)
} }
} }
func TestCrowdSecDecisionListKeptInReputationJSONAcrossARestart(t *testing.T) { func TestCrowdSecDecisionListKeptInListsJSONAcrossARestart(t *testing.T) {
t.Parallel() t.Parallel()
const bouncerKey = "crowdsec-key-0123456789abcdef" const bouncerKey = "crowdsec-key-0123456789abcdef"
@@ -814,10 +822,11 @@ func TestCrowdSecDecisionListKeptInReputationJSONAcrossARestart(t *testing.T) {
})) }))
t.Cleanup(engine.Close) t.Cleanup(engine.Close)
dir := t.TempDir() dir, cache := t.TempDir(), t.TempDir()
env := map[string]string{ env := map[string]string{
listenAddr: localhost + ":0", listenAddr: localhost + ":0",
stateDir: dir, stateDir: dir,
cacheDir: cache,
rulesDir: t.TempDir(), rulesDir: t.TempDir(),
"SWW_CROWDSEC_LAPI_URL": engine.URL, "SWW_CROWDSEC_LAPI_URL": engine.URL,
"SWW_CROWDSEC_LAPI_KEY": bouncerKey, "SWW_CROWDSEC_LAPI_KEY": bouncerKey,
@@ -836,7 +845,7 @@ func TestCrowdSecDecisionListKeptInReputationJSONAcrossARestart(t *testing.T) {
t.Errorf("bans.json holds %v, want the ban for crowdsec on %s", ban, placed) t.Errorf("bans.json holds %v, want the ban for crowdsec on %s", ban, placed)
} }
// Restarted with the engine down, the copy kept in reputation.json bans // Restarted with the engine down, the copy kept in lists.json bans
// another client in the netblock from its first request. // another client in the netblock from its first request.
down.Store(true) down.Store(true)
@@ -844,22 +853,25 @@ func TestCrowdSecDecisionListKeptInReputationJSONAcrossARestart(t *testing.T) {
wantStatus(t, url, "203.0.113.10", http.StatusForbidden) wantStatus(t, url, "203.0.113.10", http.StatusForbidden)
}) })
// The key is in neither run's output, nor in a state file. // The key is in neither run's output, nor in a state or cache file.
files := []string{"bans.json", "reputation.json", "clients.json"} files := []string{
filepath.Join(dir, "bans.json"), filepath.Join(dir, "clients.json"),
filepath.Join(cache, "lists.json"),
}
shown := make([]string, 0, len(files)+2) shown := make([]string, 0, len(files)+2)
shown = append(shown, first.text(), second.text()) shown = append(shown, first.text(), second.text())
for _, name := range files { for _, path := range files {
data, err := os.ReadFile(filepath.Join(dir, name)) //nolint:gosec // the test's data, err := os.ReadFile(path) //nolint:gosec // the test's
if err != nil { if err != nil {
t.Fatalf("read %s: %v", name, err) t.Fatalf("read %s: %v", path, err)
} }
shown = append(shown, string(data)) shown = append(shown, string(data))
} }
if all := strings.Join(shown, "\n"); strings.Contains(all, bouncerKey) { if all := strings.Join(shown, "\n"); strings.Contains(all, bouncerKey) {
t.Errorf("the key is shown in the output or the state files:\n%s", all) t.Errorf("the key is shown in the output or the state and cache files:\n%s", all)
} }
} }
@@ -1207,6 +1219,29 @@ func TestUnwritableStateDirStopsTheStart(t *testing.T) {
"SWW_STATE_DIR cannot be written: ") "SWW_STATE_DIR cannot be written: ")
} }
func TestUnwritableCacheDirStopsTheStart(t *testing.T) {
t.Parallel()
ctx, stop := context.WithTimeout(t.Context(), waitLimit)
defer stop()
out := &output{}
status := run(ctx, map[string]string{
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
cacheDir: filepath.Join(t.TempDir(), "missing"),
}, out)
if status != 1 {
t.Fatalf("exit status %d, want 1", status)
}
line := out.line(t, "msg", "cannot use the state and cache files")
if message, _ := line["error"].(string); !strings.HasPrefix(message,
"SWW_CACHE_DIR cannot be written: ") {
t.Errorf("start refused with %q, want SWW_CACHE_DIR named", message)
}
}
// wantStartRefused runs sww with its state files in dir, and // wantStartRefused runs sww with its state files in dir, and
// checks that it stops at start, with an error that starts with want. If // checks that it stops at start, with an error that starts with want. If
// it starts instead, it is stopped after waitLimit. // it starts instead, it is stopped after waitLimit.
@@ -1225,7 +1260,7 @@ func wantStartRefused(t *testing.T, dir, want string) {
t.Fatalf("exit status %d, want 1", status) t.Fatalf("exit status %d, want 1", status)
} }
line := out.line(t, "msg", "cannot use the state files") line := out.line(t, "msg", "cannot use the state and cache files")
message, _ := line["error"].(string) message, _ := line["error"].(string)
if !strings.HasPrefix(message, want) { if !strings.HasPrefix(message, want) {
+1
View File
@@ -67,6 +67,7 @@ func TestMainRunsWAF(t *testing.T) {
commandForMain+"="+wafCommand, commandForMain+"="+wafCommand,
listenAddr+"="+localhost+":0", listenAddr+"="+localhost+":0",
stateDir+"="+dir, stateDir+"="+dir,
cacheDir+"="+dir,
rulesDir+"="+t.TempDir(), rulesDir+"="+t.TempDir(),
lookupSource+"=off") lookupSource+"=off")
+2 -2
View File
@@ -16,7 +16,7 @@ import (
// The CrowdSec settings, and the tests' engine, which is never asked: each // The CrowdSec settings, and the tests' engine, which is never asked: each
// test puts in the copy of its decision list, at decisionsURL, that it // test puts in the copy of its decision list, at decisionsURL, that it
// needs, as reputation.json would at start. // needs, as lists.json would at start.
const ( const (
crowdSecURL = "SWW_CROWDSEC_LAPI_URL" crowdSecURL = "SWW_CROWDSEC_LAPI_URL"
crowdSecKey = "SWW_CROWDSEC_LAPI_KEY" crowdSecKey = "SWW_CROWDSEC_LAPI_KEY"
@@ -182,7 +182,7 @@ func TestObserveModeForwardsAClientTheCrowdSecDecisionListListsAndAlertsTheBan(
// loadDecisions puts into server's lists the copy of the decision list at // loadDecisions puts into server's lists the copy of the decision list at
// decisionsURL, answer, the engine's answer, fetched at fetched, as // decisionsURL, answer, the engine's answer, fetched at fetched, as
// reputation.json would at start. // lists.json would at start.
func loadDecisions( func loadDecisions(
t *testing.T, server *waf.Server, fetched time.Time, answer string, t *testing.T, server *waf.Server, fetched time.Time, answer string,
) { ) {
+2 -2
View File
@@ -86,8 +86,8 @@ type Params struct {
// Server is the server waf runs, with the parts whose state the state // Server is the server waf runs, with the parts whose state the state
// files keep, the lookup database, nil unless SWW_LOOKUP_SOURCE is file, the // files keep, the lookup database, nil unless SWW_LOOKUP_SOURCE is file, the
// lists fetched from URLs, which its Run fetches, the DNSBL zones' verdicts, // lists fetched from URLs, which its Run fetches, the DNSBL zones' verdicts,
// AbuseIPDB's scores and checks spent, and the metrics. RouteWatch, nil while // AbuseIPDB's scores and checks spent, routewatch's answers, and the metrics.
// SWW_ROUTEWATCH_URL is unset, keeps its answers in memory only. // RouteWatch is nil while SWW_ROUTEWATCH_URL is unset.
type Server struct { type Server struct {
*http.Server *http.Server
+8 -8
View File
@@ -39,7 +39,7 @@ const (
) )
// The lists these tests name, which are never fetched: each test puts in // The lists these tests name, which are never fetched: each test puts in
// the copies it needs, as reputation.json would at start. // the copies it needs, as lists.json would at start.
const ( const (
dropURL = "https://lists.example/drop.txt" dropURL = "https://lists.example/drop.txt"
torURL = "https://lists.example/tor.txt" torURL = "https://lists.example/tor.txt"
@@ -374,7 +374,7 @@ const (
// The DNSBL zones these tests name, which are never asked about the // The DNSBL zones these tests name, which are never asked about the
// clients the tests send requests from: each test puts in the verdicts it // clients the tests send requests from: each test puts in the verdicts it
// needs, as reputation.json would at start. A query a test does start is // needs, as dnsbl.json would at start. A query a test does start is
// sent to noResolver, where nothing listens, so that none leaves the host. // sent to noResolver, where nothing listens, so that none leaves the host.
const ( const (
dnsblZone = "dnsbl.example" dnsblZone = "dnsbl.example"
@@ -759,7 +759,7 @@ func TestOnlyAClientThatHasCommittedAnOffenceIsCheckedWithAbuseIPDB(t *testing.T
clk.advance(time.Hour) clk.advance(time.Hour)
wantReputation(t, s.get(fromDE, http.StatusOK, forward)) wantReputation(t, s.get(fromDE, http.StatusOK, forward))
wantAbuseIPDBChecks(t, server, 1) wantAbuseIPDBChecks(t, server, 1)
waitUntil(func() bool { return len(server.AbuseIPDB.Snapshot().Scores) == 1 }) waitUntil(func() bool { return len(server.AbuseIPDB.Snapshot()) == 1 })
wantReputation(t, s.get(fromDE, http.StatusOK, forward), abuseipdb) wantReputation(t, s.get(fromDE, http.StatusOK, forward), abuseipdb)
s.get(fromKP, http.StatusOK, forward) s.get(fromKP, http.StatusOK, forward)
@@ -1072,7 +1072,7 @@ func verdictsFetched() time.Time {
// loadVerdicts puts into server's DNSBL, for each client listedBy names, // loadVerdicts puts into server's DNSBL, for each client listedBy names,
// a verdict of each zone SWW_DNSBL_ZONES names, fetched at // a verdict of each zone SWW_DNSBL_ZONES names, fetched at
// verdictsFetched, as reputation.json would at start: one that lists the // verdictsFetched, as dnsbl.json would at start: one that lists the
// client from each zone listedBy gives for it, and one that does not from // client from each zone listedBy gives for it, and one that does not from
// each other zone. // each other zone.
func loadVerdicts(server *waf.Server, listedBy map[string][]string) { func loadVerdicts(server *waf.Server, listedBy map[string][]string) {
@@ -1091,8 +1091,8 @@ func loadVerdicts(server *waf.Server, listedBy map[string][]string) {
} }
// loadScores puts into server's AbuseIPDB the score scores gives each // loadScores puts into server's AbuseIPDB the score scores gives each
// client, an IPv4 address, fetched at verdictsFetched, as reputation.json // client, an IPv4 address, fetched at verdictsFetched, as
// would at start. // abuseipdb-scores.json would at start.
func loadScores(server *waf.Server, scores map[string]int64) { func loadScores(server *waf.Server, scores map[string]int64) {
kept := make([]reputation.Score, 0, len(scores)) kept := make([]reputation.Score, 0, len(scores))
for client, score := range scores { for client, score := range scores {
@@ -1102,7 +1102,7 @@ func loadScores(server *waf.Server, scores map[string]int64) {
}) })
} }
server.AbuseIPDB.Load(reputation.Checks{Scores: kept}) server.AbuseIPDB.Load(kept)
} }
// wantAbuseIPDBChecks checks how many clients server has checked with // wantAbuseIPDBChecks checks how many clients server has checked with
@@ -1116,7 +1116,7 @@ func wantAbuseIPDBChecks(t *testing.T, server *waf.Server, want int) {
} }
// loadLists puts copies of lists into server's lists, by URL, each with // loadLists puts copies of lists into server's lists, by URL, each with
// its lines, fetched at listsFetched, as reputation.json would at start. // its lines, fetched at listsFetched, as lists.json would at start.
func loadLists(t *testing.T, server *waf.Server, copies map[string][]string) { func loadLists(t *testing.T, server *waf.Server, copies map[string][]string) {
t.Helper() t.Helper()
+9 -5
View File
@@ -9,6 +9,7 @@
# - that a probe for /.env bans another client, which its next request bans # - that a probe for /.env bans another client, which its next request bans
# for good; # for good;
# - that only waf's container changes the state directory's owner; # - that only waf's container changes the state directory's owner;
# - that waf's container makes the cache directory for the sww user;
# - that the app still answers through rproxy with waf stopped, and # - that the app still answers through rproxy with waf stopped, and
# that once waf has started again, bans apply again; # that once waf has started again, bans apply again;
# - that `sv stop` stops rproxy in order, and `docker stop` stops each # - that `sv stop` stops rproxy in order, and `docker stop` stops each
@@ -153,9 +154,9 @@ refused_from() {
[ "$(status_from "$1" "$2")" = 403 ] [ "$(status_from "$1" "$2")" = 403 ]
} }
# owner_of_state_dir <container>: print who owns /var/lib/sww there. # owner_of <container> <dir>: print who owns dir there.
owner_of_state_dir() { owner_of() {
docker exec "$1" stat --format '%U' /var/lib/sww docker exec "$1" stat --format '%U' "$2"
} }
main() { main() {
@@ -173,11 +174,14 @@ main() {
start_app start_app
echo "example-app: the health checks pass" echo "example-app: the health checks pass"
[ "$(owner_of_state_dir "$WAF")" = sww ] || [ "$(owner_of "$WAF" /var/lib/sww)" = sww ] ||
fail "waf's container did not give the state directory to sww" fail "waf's container did not give the state directory to sww"
[ "$(owner_of_state_dir "$APP")" = root ] || [ "$(owner_of "$APP" /var/lib/sww)" = root ] ||
fail "the app's container changed the state directory's owner" fail "the app's container changed the state directory's owner"
echo "example-app: only waf's container changes the state directory's owner" echo "example-app: only waf's container changes the state directory's owner"
[ "$(owner_of "$WAF" /var/cache/sww)" = sww ] ||
fail "waf's container did not make the cache directory for sww"
echo "example-app: waf's container makes the cache directory for sww"
page="$(curl --fail --silent --show-error --max-time 10 "http://$address/")" || page="$(curl --fail --silent --show-error --max-time 10 "http://$address/")" ||
fail "no answer on port 8080" fail "no answer on port 8080"
+8 -2
View File
@@ -6,8 +6,9 @@
# waf listens on port 8090 unless SWW_LISTEN_ADDR says otherwise, and # waf listens on port 8090 unless SWW_LISTEN_ADDR says otherwise, and
# rproxy asks it there unless SWRP_WAF_BACKEND says otherwise. Unless # rproxy asks it there unless SWRP_WAF_BACKEND says otherwise. Unless
# SWW_STATE_DIR is set, the state files go in bin/state, beside the binary, # SWW_STATE_DIR is set, the state files go in bin/state, beside the binary,
# and unless SWW_RULES_DIR is set, the rule files are those of share/rules.d, # and unless SWW_CACHE_DIR is set, the cache files go in bin/cache. Unless
# which the image ships. # SWW_RULES_DIR is set, the rule files are those of share/rules.d, which the
# image ships.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -31,6 +32,11 @@ main() {
export SWW_STATE_DIR export SWW_STATE_DIR
mkdir -p "$SWW_STATE_DIR" mkdir -p "$SWW_STATE_DIR"
fi fi
if [ -z "${SWW_CACHE_DIR+set}" ]; then
SWW_CACHE_DIR="$ROOT/bin/cache"
export SWW_CACHE_DIR
mkdir -p "$SWW_CACHE_DIR"
fi
if [ -z "${SWW_RULES_DIR+set}" ]; then if [ -z "${SWW_RULES_DIR+set}" ]; then
SWW_RULES_DIR="$ROOT/share/rules.d" SWW_RULES_DIR="$ROOT/share/rules.d"
export SWW_RULES_DIR export SWW_RULES_DIR
+28 -10
View File
@@ -11,7 +11,10 @@ main() {
local command="${SWW_COMMAND:-rproxy}" local command="${SWW_COMMAND:-rproxy}"
case "$command" in case "$command" in
rproxy) ;; rproxy) ;;
waf) give_state_dir_to_sww ;; waf)
give_state_dir_to_sww
make_cache_dir_for_sww
;;
*) *)
printf 'SWW_COMMAND: "%s" is not rproxy or waf\n' \ printf 'SWW_COMMAND: "%s" is not rproxy or waf\n' \
"$command" >&2 "$command" >&2
@@ -31,19 +34,34 @@ main() {
# reports a directory it cannot write. # reports a directory it cannot write.
give_state_dir_to_sww() { give_state_dir_to_sww() {
local dir local dir
if dir="$(state_dir)" && [[ "$dir" == /* && -d "$dir" ]]; then if dir="$(setting SWW_STATE_DIR /var/lib/sww)" &&
[[ "$dir" == /* && -d "$dir" ]]; then
chown -R sww:sww "$dir" || true chown -R sww:sww "$dir" || true
fi fi
} }
# state_dir prints SWW_STATE_DIR, /var/lib/sww when it is unset, # make_cache_dir_for_sww makes the cache directory, SWW_CACHE_DIR in
# or what is in the file SWW_STATE_DIR_FILE names, less every newline at # either form, and gives it and every file in it to the sww user, as
# its end. It fails when both are set or the file cannot be read. # give_state_dir_to_sww does the state directory. Unlike that one, it is
state_dir() { # made when it is missing, since the cache files need no volume. As there,
if [[ ! -v SWW_STATE_DIR_FILE ]]; then # a setting that cannot be read or is not an absolute path changes
printf '%s\n' "${SWW_STATE_DIR-/var/lib/sww}" # nothing, and a failure does not end the script.
elif [[ ! -v SWW_STATE_DIR ]]; then make_cache_dir_for_sww() {
cat -- "$SWW_STATE_DIR_FILE" 2>/dev/null local dir
if dir="$(setting SWW_CACHE_DIR /var/cache/sww)" && [[ "$dir" == /* ]]; then
{ mkdir -p -- "$dir" && chown -R sww:sww "$dir"; } || true
fi
}
# setting prints the setting named $1, $2 when it is unset, or what is in
# the file the setting $1_FILE names, less every newline at its end. It
# fails when both are set or the file cannot be read.
setting() {
local name="$1" file_name="$1_FILE"
if [[ ! -v "$file_name" ]]; then
printf '%s\n' "${!name-$2}"
elif [[ ! -v "$name" ]]; then
cat -- "${!file_name}" 2>/dev/null
else else
return 1 return 1
fi fi