Admin endpoints for bans and clients on the single listener (closes #27)

SWWAF_ADMIN_TOKEN, or its _FILE form, opens GET and POST
/_smallwebwaf/bans, DELETE /_smallwebwaf/bans/<client> and GET
/_smallwebwaf/clients/<ip>. Unset, they answer 404; a missing or wrong
token gets 401, in observe mode too. They go through every check, as
the metrics do. POST takes a netblock or a client's address, a duration
or permanent, and a reason, and makes an admin ban even while another
lasts. DELETE lifts every active ban covering the address, kept and
marked lifted. Bans come back as bans.json entries; a client as
clients.json holds it, with its bans.

Judgement call: answers leave out bans.json's version field.
Judgement call: DELETE takes an address, not a netblock.
Rule suppressed: gosec G304 on a test reading bans.json.

Model: opus-5-5

README.md

@@ -19,24 +19,26 @@ ledger with the bans you make, keep and lift, the JSON state files with your
 edits taken in while it runs and the paths the rate limits do not count, which
 come next in the build order, `observe` mode and the rest of the request log's
 fields, which come a little later, and the metrics endpoint and the header size
-and the idle time as settings, which come last in it. So are two parts of the
+and the idle time as settings, which come last in it. So are three parts of the
 stage after it: the rule files, the first part, with the bans for a clear sign
-of attack, and remote log sending. `smallwebwaf` passes each request to the app
-and the app's answer back, unchanged, within its timeouts and size limits, works
-out each client's address, bans a client that sends too many requests, not
-counting those for the paths you choose, refuses a client that comes from a
-country you refuse or from a network you refuse, lets the networks you choose
-through, checks each request against the rule files and bans a client whose
-request is a clear sign of attack, keeps its bans, each client's counters and
-history, and GeoJS's answers in JSON files across restarts, takes in your edits
-of those files, such as a ban you make, keep or lift, and of the rule files
-while it runs, writes a JSON log line for every request, sends its log lines to
-a syslog server too if you name one, serves Prometheus metrics to a scraper that
-holds the metrics token, and in `observe` mode passes on the requests it would
-refuse, logging what it would have done with them. It comes as the image the
-app's own image is built on. The rest of the design comes after that, in the
-order of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools
-that led to the design is in [`EVALUATION.md`](EVALUATION.md).
+of attack, the other admin endpoints, the second, and remote log sending.
+`smallwebwaf` passes each request to the app and the app's answer back,
+unchanged, within its timeouts and size limits, works out each client's address,
+bans a client that sends too many requests, not counting those for the paths you
+choose, refuses a client that comes from a country you refuse or from a network
+you refuse, lets the networks you choose through, checks each request against
+the rule files and bans a client whose request is a clear sign of attack, keeps
+its bans, each client's counters and history, and GeoJS's answers in JSON files
+across restarts, takes in your edits of those files, such as a ban you make,
+keep or lift, and of the rule files while it runs, writes a JSON log line for
+every request, sends its log lines to a syslog server too if you name one,
+serves Prometheus metrics to a scraper that holds the metrics token, lets an
+admin who holds the admin token list, add and lift bans and ask what it knows of
+a client, and in `observe` mode passes on the requests it would refuse, logging
+what it would have done with them. It comes as the image the app's own image is
+built on. The rest of the design comes after that, in the order of the build
+order in [`SPEC.md`](SPEC.md). The survey of existing tools that led to the
+design is in [`EVALUATION.md`](EVALUATION.md).
 
 ## Getting started
 
@@ -162,18 +164,23 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
   not make it permanent. The bans in `bans.json` are kept, and refuse requests
   again when `smallwebwaf` next runs in `enforce` mode, as long as they last.
   The timeouts and size limits still apply, since they protect `smallwebwaf` and
-  the app themselves, and a request for the metrics without the token is still
-  answered `401`. It is for trying a configuration before enforcing it.
+  the app themselves, and a request for one of `smallwebwaf`'s own endpoints
+  without its token is still answered `401`. It is for trying a configuration
+  before enforcing it.
 - Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any
   check and without asking the app, for the image's health check.
 - Answers `GET /_smallwebwaf/metrics` with its metrics (see "Metrics" below) for
   a request that carries `SWWAF_METRICS_TOKEN` as
   `Authorization: Bearer <token>`, and with `401` for one that does not. While
-  the token is unset the metrics answer `404`, as does any other request under
-  `/_smallwebwaf/`. Unlike the health check, such a request goes through every
-  check any other request goes through, and is answered where another would be
-  passed to the app: a banned client stays refused, and each counts toward the
-  client's rate limits. None of them reaches the app.
+  the token is unset the metrics answer `404`, as does any request under
+  `/_smallwebwaf/` that is not for one of its endpoints. Unlike the health
+  check, such a request goes through every check any other request goes through,
+  and is answered where another would be passed to the app: a banned client
+  stays refused, and each counts toward the client's rate limits. None of them
+  reaches the app.
+- Lets an admin list, add and lift bans, and ask what it knows of a client,
+  through the endpoints `SWWAF_ADMIN_TOKEN` opens, which go through the checks
+  as the metrics do (see "Admin endpoints" below).
 - Writes a line in the request log for each request (see "Request log" below).
 - Sends every line it writes on stdout to a syslog server as well, while
   `SWWAF_LOG_REMOTE_URL` names one (see "Sending the log to a syslog server"
@@ -286,6 +293,12 @@ effective settings are logged at start.
   (see "Request log" below). An entry naming `Host` or `Transfer-Encoding` stops
   the start, since Go's HTTP server takes both out of the request; the request's
   host is the field `host`.
+- `SWWAF_ADMIN_TOKEN` (default unset): the token an admin sends for the ban
+  endpoints and `/_smallwebwaf/clients/<ip>` (see "Admin endpoints" below), a
+  long random value. While it is unset they are off; one shorter than 32
+  characters stops the start. The settings logged at start show `********` in
+  its place. Given as a file, with `SWWAF_ADMIN_TOKEN_FILE`, it can be kept out
+  of the app's reach (see "Settings given as files" below).
 - `SWWAF_METRICS_TOKEN` (default unset): the token a scraper sends for the
   metrics, a long random value. While it is unset the metrics are off; one
   shorter than 32 characters stops the start. The settings logged at start show
@@ -513,13 +526,13 @@ entries by client address, with times in UTC.
 - `lookups.json`: GeoJS's answers, one to a line, with when GeoJS gave each and
   when it was last used.
 
-`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made or made
-permanent, with every such change in between, and every file every
-`SWWAF_STATE_COUNTER_INTERVAL` and when `smallwebwaf` stops. Each write goes to
-a temporary file in the same directory, which then replaces the file, so a crash
-leaves the old file or the new one, whole. A write that fails is logged, and
-tried again at the next write. A hard kill loses what changed since the last
-write.
+`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted
+through `DELETE /_smallwebwaf/bans/<client>`, or made permanent, with every such
+change in between, and every file every `SWWAF_STATE_COUNTER_INTERVAL` and when
+`smallwebwaf` stops. Each write goes to a temporary file in the same directory,
+which then replaces the file, so a crash leaves the old file or the new one,
+whole. A write that fails is logged, and tried again at the next write. A hard
+kill loses what changed since the last write.
 
 At start the files are read back: each client keeps its counts, so a restart
 gives it no fresh allowance, and each ban keeps refusing every client in its
@@ -584,6 +597,9 @@ next ban longer; it is kept in `bans.json` with its notes, as any other ban is.
 To forget a ban altogether, delete its entry: it then refuses nothing either,
 and does not make the netblock's next ban longer.
 
+The ban endpoints add and lift bans without an edit of the file (see "Admin
+endpoints" below).
+
 ## Rule files
 
 `smallwebwaf` reads every `*.rules` file in `SWWAF_RULES_DIR`,
@@ -677,9 +693,10 @@ other request. No metric carries a client's address.
   `smallwebwaf_size_and_time_limit_hits_total` by `limit`, the setting whose
   limit was passed, `smallwebwaf_offences_total` by `kind`, and
   `smallwebwaf_bans_made_total` by `cause`, `limit`, `attack` or `admin`, the
-  last for the bans whose `cause` is `admin` that you add to `bans.json` while
-  `smallwebwaf` runs; `smallwebwaf_active_bans` and
-  `smallwebwaf_permanent_bans`, neither of which counts a lifted ban.
+  last for the bans you add through `POST /_smallwebwaf/bans`, and those whose
+  `cause` is `admin` that you add to `bans.json` while `smallwebwaf` runs;
+  `smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`, neither of which
+  counts a lifted ban.
 - `smallwebwaf_rule_matches_total`: the requests that matched each rule, by
   `rule_id` and `action`, the rule's own; and `smallwebwaf_rules_loaded`: the
   rules read from the rule files.
@@ -716,6 +733,67 @@ The requests Go's HTTP server ends before `smallwebwaf` sees them (see "Request
 log") are not counted. The metrics of the features still to come, such as the
 Core Rule Set, come with them.
 
+## Admin endpoints
+
+While `SWWAF_ADMIN_TOKEN` is set, `smallwebwaf` answers these requests itself,
+on the app's own address and through traefik like any other request, for a
+request that carries the token as `Authorization: Bearer <token>`:
+
+- `GET /_smallwebwaf/bans`: every ban held, past, active and permanent.
+- `POST /_smallwebwaf/bans`: bans a netblock, as adding an entry to `bans.json`
+  does. The body is a JSON object of `netblock`, `duration` and, if you like,
+  `reason`. `netblock` is a netblock such as `203.0.113.0/24`, or a client's
+  address, which bans the netblock a ban on that client covers: its IPv4
+  address, or the netblock around it that `SWWAF_BAN_SCOPE_V4_PREFIX` sets, or
+  its IPv6 /64. `duration` is a duration such as `1h` or `7d`, or `permanent`.
+  The ban starts at once, its `cause` is `admin`, and it is made even while
+  another ban on the netblock lasts. A body that is not such an object, has
+  another field, or is longer than 4 KiB is answered `400`, saying what is
+  wrong.
+- `DELETE /_smallwebwaf/bans/<client>`: lifts every active ban on a netblock
+  that `<client>`, an address, is in, as adding `lifted` to its entry in
+  `bans.json` does, and answers `404` when no ban on it is active.
+- `GET /_smallwebwaf/clients/<ip>`: what `smallwebwaf` knows of the client at
+  the address `<ip>`: under `client`, the client as `clients.json` holds it,
+  with its counters and its history, which holds its country as last looked up
+  and its offences, or `null` when the table of clients does not hold it; and
+  under `bans`, every ban on a netblock the address is in, with its notes.
+
+The ban endpoints answer with the bans listed, made or lifted, under `bans`,
+each as an entry of `bans.json` (see "State files" above), and a ban they make
+or lift is written to `bans.json` `SWWAF_STATE_WRITE_DELAY` later. Refusals, and
+the answers to requests that cannot be read, are plain text.
+
+A request without the token, or with another, such as the metrics token, is
+answered `401`, in `observe` mode too. While the token is unset, each of these
+answers `404`, as does any request under `/_smallwebwaf/` that is not for one of
+its endpoints. Like the metrics, these requests go through every check any other
+request goes through, and are answered where another would be passed to the app:
+a banned client stays refused, so an admin whose own address is banned lifts
+that ban by editing `bans.json`, and each request counts toward the client's
+rate limits. A client in `SWWAF_ALLOW_NETS` skips the checks, and still needs
+the token.
+
+With the token in `$TOKEN`, for an app at `https://app.example`:
+
+```sh
+# Every ban.
+curl -H "Authorization: Bearer $TOKEN" https://app.example/_smallwebwaf/bans
+
+# Ban 203.0.113.0/24 for seven days.
+curl -H "Authorization: Bearer $TOKEN" \
+    --json '{"netblock": "203.0.113.0/24", "duration": "7d", "reason": "probes for logins"}' \
+    https://app.example/_smallwebwaf/bans
+
+# Lift the bans on 203.0.113.9.
+curl -H "Authorization: Bearer $TOKEN" -X DELETE \
+    https://app.example/_smallwebwaf/bans/203.0.113.9
+
+# What smallwebwaf knows of 203.0.113.9.
+curl -H "Authorization: Bearer $TOKEN" \
+    https://app.example/_smallwebwaf/clients/203.0.113.9
+```
+
 ## Why
 
 Small self-hosted sites now receive a great deal of traffic nobody asked for: