SPEC: requirements and reasons only, reference material in docs/

docs/settings.md is now the one settings reference; the spec keeps the
conventions, the off list and each group's requirements and reasons, and
links to it for the per-setting entries. The endpoint list, the ban
notes, the state file contents, the rule file format and the alert
object likewise live in docs/ and are linked from the spec. The package
list from the README's Design section joins "Architecture". The nine
docs files are byte-identical to those on the README branch. Package
comments that named "Alert webhook schema", "Request log" and "Metrics
endpoint" for their field lists now name the docs files.

Model: fable-5-1

docs/state-files.md

@@ -0,0 +1,102 @@
+# State files
+
+`smallwebwaf` keeps its state in memory, and a copy of it in five JSON files in
+`SWWAF_STATE_DIR`. The files are meant for people as well: you can read or edit
+them at any time, and the running `smallwebwaf` takes the edit in. The
+requirements they meet are under "Persistent state" in [`SPEC.md`](../SPEC.md).
+
+## The files
+
+Each has a top-level `version` of 1.
+
+- `bans.json`: active and past bans, up to `SWWAF_MAX_BANS` that `smallwebwaf`
+  made and every ban you made, indented to be read. Per entry:
+    - the netblock, start and expiry (`null` for permanent);
+    - the cause: `attack`, `limit`, `admin` or `crowdsec`;
+    - a short reason;
+    - when it was lifted, if you lifted it;
+    - the notes (see [`bans.md`](bans.md)).
+- `clients.json`: per client, its minute, hour and day counters and its history
+  since it was first seen, one client per line, so `grep` shows everything about
+  one. The history holds:
+    - the first and last time seen;
+    - the AS number, AS name and country last looked up, and when;
+    - total requests, and how many were forwarded and how many refused;
+    - bytes in each direction;
+    - responses by status class;
+    - offences by kind: `limit`, `attack`, `rule_blocked`, `waf_blocked` and
+      `token_refused`.
+
+    Clients that were never banned are kept too, so every client's history
+    survives a restart.
+
+- `lookups.json`: GeoJS answers, one per line: the AS number, AS name and
+  country, when GeoJS was asked and when the answer was last used.
+- `reputation.json`:
+    - the last good copy of each list fetched from a URL, with when it was last
+      tried;
+    - each DNS blocklist verdict still in use;
+    - AbuseIPDB's scores, and the checks spent today.
+- `alerts.json`:
+    - the anomaly counters;
+    - the alert cooldowns: for each event and netblock, source, file or thing
+      counted, when it was last sent and the count of repeats held back since;
+    - the alerts sent and held back in the current hour;
+    - the alerts waiting for each destination.
+
+Times are RFC 3339 UTC. Durations and sizes are plain numbers, with the unit in
+the field name. `bans.json`, `reputation.json` and `alerts.json` are indented;
+`clients.json` and `lookups.json` put each entry on one line.
+
+## Sizes
+
+`clients.json` takes about 1 KiB per client, about 20 MiB at the default
+`SWWAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of
+disk writes a day. `bans.json` takes about 1.2 KiB per ban, at most about 2.5
+KiB, so about 6 MiB at the default `SWWAF_MAX_BANS`. With a full file, a hundred
+new bans a day come to about 600 MiB of writes. `lookups.json` takes about 150
+bytes per answer, under 1.5 GiB of writes a day when full. The other two files
+are usually a few MiB or less.
+
+## Writing
+
+`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted or
+made permanent. Every file is written every `SWWAF_STATE_COUNTER_INTERVAL`, and
+when `smallwebwaf` stops. Each write goes to a temporary file that then replaces
+the file, so a crash leaves the old file or the new one, whole. Before writing a
+file, `smallwebwaf` takes in any edit you made to it, so your edit is not
+overwritten. A write that fails is logged, raised as a `file_error` alert, and
+tried again at the next write. A hard kill loses what changed since the last
+write:
+
+- ban changes from the last `SWWAF_STATE_WRITE_DELAY` (10 seconds);
+- everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` (15 minutes):
+    - counts and history, and the counts in ban notes;
+    - GeoJS answers;
+    - reputation verdicts and the AbuseIPDB count;
+    - anomaly counters and alert cooldowns.
+
+## Reading at start
+
+The files are read back. Each client keeps its counts, so a restart gives no one
+a fresh allowance. Each ban keeps refusing until it ends. Counter buckets and
+cache entries whose time has passed are discarded. A missing file is empty
+state. These stop the start, with a message naming the file and the place:
+
+- a file that does not parse, or has another `version`;
+- an entry without a field it needs;
+- a ban whose `cause` is not `limit`, `attack`, `admin` or `crowdsec`;
+- a ban whose `expires` is not after its `start`.
+
+## Editing while it runs
+
+`smallwebwaf` watches the directory and takes in your edit of a file as soon as
+you save it. What the file then holds replaces what `smallwebwaf` held for it.
+It tells its own writes from yours by comparing the file with what it last read
+or wrote. A change it made after you opened the file, such as a new ban, is lost
+when you save over it; most editors warn when a file changed on disk while it
+was open. An edit with an error that would stop the start does not stop the
+running `smallwebwaf`. It keeps what it holds. At the file's next write it
+renames your file to `<name>.bad`, writes the file again from memory, and logs
+and alerts the file and the place of the error. Mend the `.bad` file and move it
+back. A file you remove is written again at its next write.