README rewritten from scratch so a newcomer can follow it (closes #154, closes #155)

The old README opened with project history and one list of a hundred
settings. The new one leads with what smallwebwaf is, what it protects
against, how to build and deploy it, how it sits in front of the app,
what happens to a request, the settings most deployments touch, how to
check it works and how to develop it. The reference material, every
setting, bans, rule files, the Core Rule Set, the admin endpoints, the
request log, alerts, metrics, state files, lookups, reputation sources,
the image and gitea, moves to files under docs/, each linked from the
README. Every statement was checked against the code on next.

Model: fable-5-1

docs/state-files.md

@@ -0,0 +1,102 @@
+# State files
+
+`smallwebwaf` keeps its state in memory, and a copy of it in five JSON files in
+`SWWAF_STATE_DIR`. The files are meant for people as well: you can read or edit
+them at any time, and the running `smallwebwaf` takes the edit in. The
+requirements they meet are under "Persistent state" in [`SPEC.md`](../SPEC.md).
+
+## The files
+
+Each has a top-level `version` of 1.
+
+- `bans.json`: active and past bans, up to `SWWAF_MAX_BANS` that `smallwebwaf`
+  made and every ban you made, indented to be read. Per entry:
+    - the netblock, start and expiry (`null` for permanent);
+    - the cause: `attack`, `limit`, `admin` or `crowdsec`;
+    - a short reason;
+    - when it was lifted, if you lifted it;
+    - the notes (see [`bans.md`](bans.md)).
+- `clients.json`: per client, its minute, hour and day counters and its history
+  since it was first seen, one client per line, so `grep` shows everything about
+  one. The history holds:
+    - the first and last time seen;
+    - the AS number, AS name and country last looked up, and when;
+    - total requests, and how many were forwarded and how many refused;
+    - bytes in each direction;
+    - responses by status class;
+    - offences by kind: `limit`, `attack`, `rule_blocked`, `waf_blocked` and
+      `token_refused`.
+
+    Clients that were never banned are kept too, so every client's history
+    survives a restart.
+
+- `lookups.json`: GeoJS answers, one per line: the AS number, AS name and
+  country, when GeoJS was asked and when the answer was last used.
+- `reputation.json`:
+    - the last good copy of each list fetched from a URL, with when it was last
+      tried;
+    - each DNS blocklist verdict still in use;
+    - AbuseIPDB's scores, and the checks spent today.
+- `alerts.json`:
+    - the anomaly counters;
+    - the alert cooldowns: for each event and netblock, source, file or thing
+      counted, when it was last sent and the count of repeats held back since;
+    - the alerts sent and held back in the current hour;
+    - the alerts waiting for each destination.
+
+Times are RFC 3339 UTC. Durations and sizes are plain numbers, with the unit in
+the field name. `bans.json`, `reputation.json` and `alerts.json` are indented;
+`clients.json` and `lookups.json` put each entry on one line.
+
+## Sizes
+
+`clients.json` takes about 1 KiB per client, about 20 MiB at the default
+`SWWAF_MAX_TRACKED_CLIENTS`. Written every 15 minutes, that is under 2 GiB of
+disk writes a day. `bans.json` takes about 1.2 KiB per ban, at most about 2.5
+KiB, so about 6 MiB at the default `SWWAF_MAX_BANS`. With a full file, a hundred
+new bans a day come to about 600 MiB of writes. `lookups.json` takes about 150
+bytes per answer, under 1.5 GiB of writes a day when full. The other two files
+are usually a few MiB or less.
+
+## Writing
+
+`bans.json` is written `SWWAF_STATE_WRITE_DELAY` after a ban is made, lifted or
+made permanent. Every file is written every `SWWAF_STATE_COUNTER_INTERVAL`, and
+when `smallwebwaf` stops. Each write goes to a temporary file that then replaces
+the file, so a crash leaves the old file or the new one, whole. Before writing a
+file, `smallwebwaf` takes in any edit you made to it, so your edit is not
+overwritten. A write that fails is logged, raised as a `file_error` alert, and
+tried again at the next write. A hard kill loses what changed since the last
+write:
+
+- ban changes from the last `SWWAF_STATE_WRITE_DELAY` (10 seconds);
+- everything else from the last `SWWAF_STATE_COUNTER_INTERVAL` (15 minutes):
+    - counts and history, and the counts in ban notes;
+    - GeoJS answers;
+    - reputation verdicts and the AbuseIPDB count;
+    - anomaly counters and alert cooldowns.
+
+## Reading at start
+
+The files are read back. Each client keeps its counts, so a restart gives no one
+a fresh allowance. Each ban keeps refusing until it ends. Counter buckets and
+cache entries whose time has passed are discarded. A missing file is empty
+state. These stop the start, with a message naming the file and the place:
+
+- a file that does not parse, or has another `version`;
+- an entry without a field it needs;
+- a ban whose `cause` is not `limit`, `attack`, `admin` or `crowdsec`;
+- a ban whose `expires` is not after its `start`.
+
+## Editing while it runs
+
+`smallwebwaf` watches the directory and takes in your edit of a file as soon as
+you save it. What the file then holds replaces what `smallwebwaf` held for it.
+It tells its own writes from yours by comparing the file with what it last read
+or wrote. A change it made after you opened the file, such as a new ban, is lost
+when you save over it; most editors warn when a file changed on disk while it
+was open. An edit with an error that would stop the start does not stop the
+running `smallwebwaf`. It keeps what it holds. At the file's next write it
+renames your file to `<name>.bad`, writes the file again from memory, and logs
+and alerts the file and the place of the error. Mend the `.bad` file and move it
+back. A file you remove is written again at its next write.