Take in an admin's edits of the state files while running (closes #68)
check / check (push) Successful in 2m59s

smallwebwaf watches SWWAF_STATE_DIR with fsnotify and takes in a saved
edit of a state file in place of what it held. It tells its own writes
from an admin's by the SHA-256 of what it last read or wrote; each write
first takes in an edit made since. An edit that does not parse is
renamed to <name>.bad at the file's next write. Every ban on a netblock
is checked, and the next ban is worked out from the one that ended
last. Two metrics count the edits taken in and set aside. README.md
says how to add and lift a ban.

Judgement call: a broken edit is set aside at the next write, since an
editor's file can be read half written.

Model: opus-5-5
This commit is contained in:
2026-10-06 09:48:06 +00:00
parent 234c5eac60
commit 34ebf1abb9
13 changed files with 995 additions and 176 deletions
+63 -26
View File
@@ -15,14 +15,15 @@ Status: the first two milestones are built
(https://git.eeqj.de/sneak/smallwebwaf/issues/13 and
https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are five parts of
milestone 3: the static lists, the bans that broken rate limits lead to and the
JSON state files, which come next in the build order, and the metrics endpoint
and the header size and the idle time as settings, which come last in it.
`smallwebwaf` passes each request to the app and the app's answer back,
unchanged, within its timeouts and size limits, works out each client's address,
bans a client that sends too many requests, refuses a client that comes from a
country you refuse or from a network you refuse, lets the networks you choose
through, keeps its bans, each client's counters and history, and GeoJS's answers
in JSON files across restarts, writes a JSON log line for every request, and
JSON state files with your edits taken in while it runs, which come next in the
build order, and the metrics endpoint and the header size and the idle time as
settings, which come last in it. `smallwebwaf` passes each request to the app
and the app's answer back, unchanged, within its timeouts and size limits, works
out each client's address, bans a client that sends too many requests, refuses a
client that comes from a country you refuse or from a network you refuse, lets
the networks you choose through, keeps its bans, each client's counters and
history, and GeoJS's answers in JSON files across restarts, takes in your edits
of those files while it runs, writes a JSON log line for every request, and
serves Prometheus metrics to a scraper that holds the metrics token. It comes as
the image the app's own image is built on. The rest of the design comes after
that, in the order of the build order in [`SPEC.md`](SPEC.md). The survey of
@@ -98,9 +99,9 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set.
seen, how many of them the ban has refused, and how many bans the netblock had
before. At most `SWWAF_MAX_BANS` bans are kept, past, active and permanent;
past that, the earliest ban of the netblock that has gone longest without a
request is dropped first. `bans.json` shows the bans and their notes, and a
restart lifts none (see "State files" below); lifting a ban by editing it
comes with https://git.eeqj.de/sneak/smallwebwaf/issues/68.
request is dropped first. `bans.json` shows the bans and their notes, a
restart lifts none, and you add or lift a ban by editing it (see "State files"
below).
- Refuses a request from a country you refuse with `SWWAF_BAN_RESPONSE`, as soon
as the client's country is known and before its body is read; such a request
is not counted for the rate limits. While one of the country lists below is
@@ -311,9 +312,42 @@ without a field it needs, named with the entry's place in the file: a ban's
`netblock`, `start` or `expires`, which is `null` for a permanent ban; a
client's `client`, or the `start` of a window in which it has requests; an
answer's `client`, `country`, which is `""` for a client GeoJS cannot place, or
`answered`. An edit made while `smallwebwaf` runs is overwritten by its next
write: taking it in comes with https://git.eeqj.de/sneak/smallwebwaf/issues/68.
The AS number and AS name come with their lookup.
`answered`. The AS number and AS name come with their lookup.
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
a state file as soon as you save it: what the file then holds replaces what
`smallwebwaf` held for it, as if read at start. It tells its own writes from
yours by comparing the file with what it last read or wrote, and before it
writes a file it takes in any edit made since, so your edit is not overwritten;
a change `smallwebwaf` made after you opened the file, such as a new ban, is
lost when you save over it. An edit that would stop the start, because it does
not parse, has another `version` or leaves out a field an entry needs, does not
stop the running `smallwebwaf`: it keeps what it holds, and at the file's next
write renames your file to `<name>.bad`, such as `bans.json.bad`, writes the
file again from memory, and logs the file and where the error is. It waits for
that write because an editor's file can be read before the editor has finished
writing it. Mend the `.bad` file and move it back. A file you remove is written
again at its next write.
To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start`
and its `expires`, `null` for a ban that never ends; its `notes` may be left
out. This `bans.json` bans `203.0.113.0/24` for good:
```json
{
"version": 1,
"bans": [
{
"netblock": "203.0.113.0/24",
"start": "2026-10-06T12:00:00Z",
"expires": null
}
]
}
```
To lift a ban, delete its entry. `smallwebwaf` then forgets the ban, so it does
not make the netblock's next ban longer.
## Metrics
@@ -352,7 +386,10 @@ other request. No metric carries a client's address.
- `smallwebwaf_state_file_writes_total`,
`smallwebwaf_state_file_write_failures_total`,
`smallwebwaf_state_file_last_write_timestamp_seconds` and
`smallwebwaf_state_file_size_bytes`, by `file`.
`smallwebwaf_state_file_size_bytes`, by `file`; and, by `file` too,
`smallwebwaf_state_file_edits_taken_in_total`: your edits taken in, and
`smallwebwaf_state_file_edits_set_aside_total`: those renamed to `<name>.bad`
because they would stop the start.
- Go's own `go_` metrics and the process's `process_` metrics.
The requests Go's HTTP server ends before `smallwebwaf` sees them (see "Request
@@ -460,9 +497,9 @@ goes through the candidates one by one.
readable JSON files, written regularly and at every stop, so a restart loses
nothing. Edit a file, or add a rule file, and the running `smallwebwaf` picks
up the change. Nothing is read from disk while serving a request. The files
for the bans, the clients and the GeoJS answers are built (see "State files"
above); the others come with their features, and taking in an edit while
running comes with https://git.eeqj.de/sneak/smallwebwaf/issues/68.
for the bans, the clients and the GeoJS answers are built, with an edit taken
in while running (see "State files" above); the others come with their
features.
- Health checks, the metrics, and listing, adding and lifting bans or asking why
a given address was refused, all on the one port every request uses: under
`/_smallwebwaf/` on the app's own address, through traefik like any other
@@ -653,8 +690,8 @@ addresses are never sent to GeoJS.
answers.
- `internal/ratelimit`: the table of clients: counts each client's requests,
tells when one takes it over a rate limit, and keeps each client's history.
- `internal/state`: reads the state files at start, and writes them when they
are due and at the stop.
- `internal/state`: reads the state files at start, takes in an admin's edit of
one while running, and writes them when they are due and at the stop.
- `internal/requestlog`: the lines on stdout: the request log line and the
process's own messages.
- `Dockerfile`: the lint and test phases, then the image, whose last stage
@@ -666,8 +703,9 @@ addresses are never sent to GeoJS.
Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the
table of clients to 20,000, the GeoJS answers to 100,000 and the banned
netblocks to `SWWAF_MAX_BANS`, dropping the least recently seen, and
`github.com/prometheus/client_golang` keeps the metrics and serves them. The
country codes are the list in `internal/config/config.go`.
`github.com/prometheus/client_golang` keeps the metrics and serves them, and
`github.com/fsnotify/fsnotify` tells `smallwebwaf` when a state file is saved.
The country codes are the list in `internal/config/config.go`.
## Entrypoints
@@ -710,10 +748,9 @@ so that they run in minimal containers.
## TODO
- The rest of milestone 3, from taking in an admin's edits to the state files
(https://git.eeqj.de/sneak/smallwebwaf/issues/68) up to the rest of the
request log's fields, and the rest of the design, in the order of the build
order in [`SPEC.md`](SPEC.md).
- The rest of milestone 3, from exemptions up to the rest of the request log's
fields, and the rest of the design, in the order of the build order in
[`SPEC.md`](SPEC.md).
## Documents