Take in an admin's edits of the state files while running (closes #68)
check / check (push) Successful in 2m56s

smallwebwaf watches SWWAF_STATE_DIR with fsnotify and takes in an edit of
a state file as soon as it is saved, in place of what it held. It tells
its own writes from an admin's by the SHA-256 of what it last read or
wrote, and each write takes in an edit made since first. An edit that
does not parse is renamed to <name>.bad at the file's next write. Every
ban on a netblock is checked, so an added ban that starts before the
others still refuses. README.md says how to add and lift a ban.

Judgement call: a broken edit is set aside at the next write, since an
editor's file can be read half written.

Model: opus-5-5
This commit is contained in:
2026-10-06 08:46:39 +00:00
parent 68f687cb0c
commit 09eac675dc
12 changed files with 883 additions and 167 deletions
+61 -28
View File
@@ -15,17 +15,18 @@ Status: the first two milestones are built
(https://git.eeqj.de/sneak/smallwebwaf/issues/13 and
https://git.eeqj.de/sneak/smallwebwaf/issues/14), and so are four parts of
milestone 3: the static lists, the bans that broken rate limits lead to and the
JSON state files, which come next in the build order, and the header size and
the idle time as settings, which come last in it. `smallwebwaf` passes each
request to the app and the app's answer back, unchanged, within its timeouts and
size limits, works out each client's address, bans a client that sends too many
requests, refuses a client that comes from a country you refuse or from a
network you refuse, lets the networks you choose through, keeps its bans, each
client's counters and history, and GeoJS's answers in JSON files across
restarts, and writes a JSON log line for every request. It comes as the image
the app's own image is built on. The rest of the design comes after that, in the
order of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools
that led to the design is in [`EVALUATION.md`](EVALUATION.md).
JSON state files with your edits taken in while it runs, which come next in the
build order, and the header size and the idle time as settings, which come last
in it. `smallwebwaf` passes each request to the app and the app's answer back,
unchanged, within its timeouts and size limits, works out each client's address,
bans a client that sends too many requests, refuses a client that comes from a
country you refuse or from a network you refuse, lets the networks you choose
through, keeps its bans, each client's counters and history, and GeoJS's answers
in JSON files across restarts, takes in your edits of those files while it runs,
and writes a JSON log line for every request. It comes as the image the app's
own image is built on. The rest of the design comes after that, in the order of
the build order in [`SPEC.md`](SPEC.md). The survey of existing tools that led
to the design is in [`EVALUATION.md`](EVALUATION.md).
## Getting started
@@ -97,9 +98,9 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set.
seen, how many of them the ban has refused, and how many bans the netblock had
before. At most `SWWAF_MAX_BANS` bans are kept, past, active and permanent;
past that, the earliest ban of the netblock that has gone longest without a
request is dropped first. `bans.json` shows the bans and their notes, and a
restart lifts none (see "State files" below); lifting a ban by editing it
comes with https://git.eeqj.de/sneak/smallwebwaf/issues/68.
request is dropped first. `bans.json` shows the bans and their notes, a
restart lifts none, and you add or lift a ban by editing it (see "State files"
below).
- Refuses a request from a country you refuse with `SWWAF_BAN_RESPONSE`, as soon
as the client's country is known and before its body is read; such a request
is not counted for the rate limits. While one of the country lists below is
@@ -295,9 +296,42 @@ without a field it needs, named with the entry's place in the file: a ban's
`netblock`, `start` or `expires`, which is `null` for a permanent ban; a
client's `client`, or the `start` of a window in which it has requests; an
answer's `client`, `country`, which is `""` for a client GeoJS cannot place, or
`answered`. An edit made while `smallwebwaf` runs is overwritten by its next
write: taking it in comes with https://git.eeqj.de/sneak/smallwebwaf/issues/68.
The AS number and AS name come with their lookup.
`answered`. The AS number and AS name come with their lookup.
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
a state file as soon as you save it: what the file then holds replaces what
`smallwebwaf` held for it, as if read at start. It tells its own writes from
yours by comparing the file with what it last read or wrote, and before it
writes a file it takes in any edit made since, so your edit is not overwritten;
a change `smallwebwaf` made after you opened the file, such as a new ban, is
lost when you save over it. An edit that would stop the start, because it does
not parse, has another `version` or leaves out a field an entry needs, does not
stop the running `smallwebwaf`: it keeps what it holds, and at the file's next
write renames your file to `<name>.bad`, such as `bans.json.bad`, writes the
file again from memory, and logs the file and where the error is. It waits for
that write because an editor's file can be read before the editor has finished
writing it. Mend the `.bad` file and move it back. A file you remove is written
again at its next write.
To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start`
and its `expires`, `null` for a ban that never ends; its `notes` may be left
out. This `bans.json` bans `203.0.113.0/24` for good:
```json
{
"version": 1,
"bans": [
{
"netblock": "203.0.113.0/24",
"start": "2026-10-06T12:00:00Z",
"expires": null
}
]
}
```
To lift a ban, delete its entry. `smallwebwaf` then forgets the ban, so it does
not make the netblock's next ban longer.
## Why
@@ -400,9 +434,9 @@ goes through the candidates one by one.
readable JSON files, written regularly and at every stop, so a restart loses
nothing. Edit a file, or add a rule file, and the running `smallwebwaf` picks
up the change. Nothing is read from disk while serving a request. The files
for the bans, the clients and the GeoJS answers are built (see "State files"
above); the others come with their features, and taking in an edit while
running comes with https://git.eeqj.de/sneak/smallwebwaf/issues/68.
for the bans, the clients and the GeoJS answers are built, with an edit taken
in while running (see "State files" above); the others come with their
features.
- Health checks, the metrics, and listing, adding and lifting bans or asking why
a given address was refused, all on the one port every request uses: under
`/_smallwebwaf/` on the app's own address, through traefik like any other
@@ -590,8 +624,8 @@ addresses are never sent to GeoJS.
answers.
- `internal/ratelimit`: the table of clients: counts each client's requests,
tells when one takes it over a rate limit, and keeps each client's history.
- `internal/state`: reads the state files at start, and writes them when they
are due and at the stop.
- `internal/state`: reads the state files at start, takes in an admin's edit of
one while running, and writes them when they are due and at the stop.
- `internal/requestlog`: the lines on stdout: the request log line and the
process's own messages.
- `Dockerfile`: the lint and test phases, then the image, whose last stage
@@ -602,8 +636,9 @@ addresses are never sent to GeoJS.
Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the
table of clients to 20,000, the GeoJS answers to 100,000 and the banned
netblocks to `SWWAF_MAX_BANS`, dropping the least recently seen. The country
codes are the list in `internal/config/config.go`.
netblocks to `SWWAF_MAX_BANS`, dropping the least recently seen, and
`github.com/fsnotify/fsnotify` tells `smallwebwaf` when a state file is saved.
The country codes are the list in `internal/config/config.go`.
## Entrypoints
@@ -646,10 +681,8 @@ so that they run in minimal containers.
## TODO
- The rest of milestone 3, from taking in an admin's edits to the state files
(https://git.eeqj.de/sneak/smallwebwaf/issues/68) up to the metrics endpoint,
and the rest of the design, in the order of the build order in
[`SPEC.md`](SPEC.md).
- The rest of milestone 3, from exemptions up to the metrics endpoint, and the
rest of the design, in the order of the build order in [`SPEC.md`](SPEC.md).
## Documents