Adds mediawiki.conf, the eleventh change: on MediaWiki's paths, the fields
that carry a search, a title, a name, a file name or an address skip the
four rules that read them as commands or system files, and the wikitext
fields skip the rules that refused saving a page. The reasons on the admin
pages skip only 930120 and 932120. /index.php counts as MediaWiki's only
with a page title in its query or after its path, since WordPress serves it
too. The requests recorded from MediaWiki 1.46.2, with and without short
URLs and on the admin pages, pass at the defaults and while bodies are
read. The app tests share the check of recorded requests and the list of
hard values.
Model: opus-5-5
Coraza v3.8.1 shares a rule's ctl:ruleRemoveTargetById exclusions
between concurrent requests, so one request can be checked with
another's. Coraza's main fixed it in its pull request 1745; no release
carries it yet, so go.mod names commit d028be399c68 by its
pseudo-version.
Model: opus-5-5
A Drupal and a WordPress request are inspected at once, many times
over, on one Core Rule Set. Coraza v3.8.1 shares each rule's
exclusions between requests, so the test fails under -race.
Model: opus-5-5
Each field skips only the rules its recorded requests set off:
- page on all of wp-admin, and the JSON fields title, excerpt and name
on /wp/v2/ routes: 930120, 932120, 932250 and 932260;
- p, postId and slug on all of wp-admin, and the JSON slug on /wp/v2/
routes: 932120 and 932260, since WordPress writes these as slugs;
- url on the block editor's link previews and embeds, and a commenter's
url: 930120 and 932120.
Every field keeps 932235. Add 61 requests recorded from WordPress 7.1.4
with these fields.
Model: opus-5-5
AGENTS.md holds the branching model, the entrypoints, the editing rule
and two iron rules: query strings are logged as received, and the
trusted-proxy default is the RFC 1918 set. README links it; SPEC.md
step 2 now states that default and that a given list replaces it.
Model: opus-5-5
Adds drupal.conf, the tenth change: on Drupal's paths, the fields that
carry a search, a name, a file name or an address skip the rules that
refused ordinary requests there. The recorded requests from Drupal 11.4
pass at the defaults and while bodies are read; SQL injection, path
traversal and script injection stay refused on those fields. Drupal's
tests share the reader of recorded requests and the attack constants
with the other apps' tests.
Model: opus-5-5
Add the ninth change: wordpress.conf switches off, per path and field, each
rule that refused ordinary WordPress 7.1.4 requests. Three groups of fields
skip four rules on system files and commands, and only the addresses of the
first group also skip 932235. Add the recorded requests, a test at the
defaults and reading bodies, and list WordPress in the README.
Model: opus-5-5 (rework); sonnet-5-5 (copied the recorded lines)
The first half of the naming audit: names now say what they are. `internal/waf` is `internal/coreruleset`. Log fields, the action `blocked_by_core_rule_set`, the metric and the alert event say Core Rule Set. Timings end `_ms`. Percentages say of what, as `rate_limit_percent` and `bytes_limit_percent`. Setting names follow in #234.
Judgement call: `blocked_by_core_rule_set`, not the audit's `core_rule_set_blocked`, to match `blocked_by_block_rule`.
Trips a reader: an existing state file holding `waf_blocked` or the anomaly scope `watch` stops the start and must be removed. `SWW_ALERT_EVENTS` listing `waf_block` must list `core_rule_set_match`.
Model: opus-5-5
The request log's `action` values now say what they mean, as sneak named them: `request_header_too_large`, `client_is_rate_limited`, `blocked_by_block_rule` and `client_denied`, with no aliases. `banned` splits into `temporarily_banned` and `permanently_banned`, by whether the ban runs out. When a permanent ban also covers a client, that ban is the one reported.
Trips a reader: a request over a rate limit stays `client_is_rate_limited` even when the ban it makes is permanent.
Disclosure: the offence key `rule_blocked` is renamed in `clients.json` too, so an existing file holding it stops the start and must be removed. Its `version` stays 1.
Model: opus-5-5
With `SWW_ROUTEWATCH_URL` set, a new heuristic, `routewatch_tags`, adds `asn=<number>` and `prefix=<netblock>` to a request from what a routewatch instance answers about the client. New package `internal/routewatch` asks in the background, never on a request's path, and keeps answers 1 hour in memory. The request log gains `routewatch_tags_missing`, saying why a request has none. `prefix=` works in `SWW_TAG_DENY` and `SWW_TAG_LIMIT_PERCENT`.
Trips a reader: a client's first request has no routewatch tags, so `prefix=` refuses from its next request after the answer comes.
Judgement call: with `SWW_LOOKUP_SOURCE=off` and `SWW_ROUTEWATCH_URL` set, an `asn=` tag no longer stops the start.
Model: opus-5-5
sneak asked for every document to be audited for sentences that do not say what they mean. The byte limits now say which bytes count (response bodies, request bodies or both, as `SWW_BYTES_COUNT` says, WebSocket traffic included) and what passing one does. Other unclear or false sentences in `README.md`, `SPEC.md`, `EVALUATION.md` and `docs/` are rewritten.
Documents only. `REPO_POLICIES.md` is shared policy text and stayed out.
Model: fable-5-1 (first audit); opus-5-5 (rework)
Each request gets tags, logged as `tags`: `asn=`, `country=`, one per `SWW_TAG_LISTS` list holding the client, `crowdsec`, `dnsbl=<zone>`, `abuseipdb` and `exploitsignature`. `SWW_TAG_DENY` refuses by tag and `SWW_TAG_LIMIT_PERCENT` lowers limits by tag. They replace `SWW_ASN_LIMIT_PERCENT`, `SWW_COUNTRY_LIMIT_PERCENT` and `SWW_DENIED_COUNTRIES`, with no aliases. A tag in those settings that nothing configured can give stops the start.
Judgement call: `SWW_ASN_LIMIT_PERCENT_URL` stays, its entries counting like `asn=` tags.
Judgement call: `SWW_TAG_LIMIT_PERCENT` also lowers byte limits, as the settings it replaces did.
Trips a reader: `sww_country_list_refusals_total` now counts only `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES` refusals.
Model: opus-5-5
`script/publish` runs `script/docker` against the Docker host and pushes the image to the registry under four tags: the UTC date, the short commit hash, the branch name and `latest`. It refuses a working tree with uncommitted changes, since the hash tag names a commit.
Also carries sneak's `golines -m 77` line wrapping in three files, with no code change.
Unverified: a real build on the Docker host and a real push from this commit.
Deviation: sneak's FIXME comments are out, because the linter refuses them; they are #227, #228 and #229.
Model: opus-5-5
Co-authored-by: sneak <sneak@sneak.berlin>
Each discriminator check is now a type in its own file behind one small interface, listed in `heuristics.go`. Adding a heuristic is one file and one line. Behaviour does not change.
Refusal and ban log lines gain `heuristic` and `heuristic_figures`. A ban after the answer, by `refusal_limit` or `byte_limits`, logs `after_answer_heuristic` and its figures, so one line can name both.
Deviation: four lists, not the planned three, so `refusal_limit` still bans at the decision.
Rule suppressed: `//nolint:lll` on one struct tag.
Model: opus-5-5
xz-utils, which nix-bin depends on, links its man pages with update-alternatives when it installs, and the image's dpkg config drops man pages, so each link printed a warning. A config file re-includes only the man1 pages for this install; the same step deletes them, their dangling links in /etc/alternatives and the config file. No man page or package is added or removed.
Model: deepseek-v4-pro
The image's `ca-certificates`, `nix-bin` and `runit` now come from Ubuntu's archive instead of a dated snapshot, whose service had failed image builds. apt fetches them over plain HTTP and checks each against the archive's signed package lists, so no CA certificates are borrowed from the Go image. The README, `SPEC.md` and `docs/image.md` say so; the amd64-only limit, which came from the removed hash check, is gone.
Rule cut against: `REPO_POLICIES.md` pins every external reference by hash; sneak's ruling at sneak/prompts#72 (comment) allows Ubuntu's own package index.
Model: deepseek-v4-pro (implementation), opus-5-5 (rework)
New defaults, as sneak approved: 120 requests a minute, 1,500 an hour and 6,000 a day; bytes off a minute, 50G an hour and 100G a day. The README, `SPEC.md`, `docs/settings.md` and `docs/gitea.md` give the new figures and reasoning, and point Actions runners and shared addresses at `SWWAF_RATE_LIMIT_EXEMPT_NETS`. The README examples now hold against the new defaults.
An Actions runner that is not exempted makes more requests an hour than the new limit and will be banned.
Model: opus-5-5
The image's `ca-certificates`, `nix-bin` and `runit` now come from Ubuntu's archive instead of a dated snapshot, whose service had failed image builds. apt fetches them over plain HTTP and checks each against the archive's signed package lists, so no CA certificates are borrowed from the Go image. The README, `SPEC.md` and `docs/image.md` say so; the amd64-only limit, which came from the removed hash check, is gone.
Rule cut against: `REPO_POLICIES.md` pins every external reference by hash; sneak's ruling at sneak/prompts#72 (comment) allows Ubuntu's own package index.
Model: deepseek-v4-pro (implementation), opus-5-5 (rework)
The package passes nothing on since the one-process mode went: it answers rproxy's questions, takes its reports and answers servewaf's own endpoints. It is now named after the command it serves, with `git mv`, and its constructor is `servewaf.New`. `SPEC.md` and the other documents follow. No change of behaviour.
Model: opus-5-5
A compose setup that runs gitea 1.27.3 behind traefik, rproxy and servewaf on one shared network with IPv4 and IPv6, and `script/gitea-setup` (`make gitea-setup`), which brings it up and checks that a page load, a push and a clone pass through rproxy and are checked by servewaf. Containers on the shared network skip servewaf's checks; the gateways do not. gitea gets a secret key made per install and has sign-up off. `docs/gitea.md` explains each setting.
Judgement call: the Core Rule Set runs in `detect` mode here until #198 lands.
Model: deepseek-v4-pro (implementation), opus-5-5 (rework)
Merged by sneak's order of 6 November ("take the current state of smallwebwaf and get it on to main. i am going to deploy it soon in its alpha state").
Model: opus-5-5
`smallwebwaf` with no command now prints the commands and exits with status 2.
The image runs rproxy, or servewaf with SMALLWEBWAF_COMMAND=servewaf. The
passing-on code, its settings and its log fields and metric go. internal/proxy
keeps the checks, answers, reports and own endpoints; its tests run through the
real rproxy. The documents describe the pair.
Judgement call: clients.json keeps token_refused, since state files refuse unknown fields.
Deviation: script/gitea-requests sets SWWAF_LOG_REQUEST_HEADERS, to label requests.
Partially verified: make example-app and make gitea-requests never built the image, as Ubuntu's snapshot service answered 5xx.
Model: opus-5-5
`smallwebwaf rproxy` reads the first 64 KiB of each body, asks servewaf over
HTTP/1.1, and passes what servewaf allows to the app within its limits. Any
other answer goes to the visitor as servewaf gave it. SWRP_DEFAULT_DISPOSITION
decides only when servewaf cannot be connected to, does not answer at all, or
does not answer within SWRP_TIMEOUT_MS. A method SPEC.md does not list gets 405.
Duplication: internal/rproxy copies passing-on code from internal/proxy until
#172 removes the originals.
Rule suppressed: gosec G704 and G705; gosec at
internal/smallwebwaf/rproxy_test.go:431; tagliatelle at
internal/requestlog/requestlog.go:203; paralleltest on one test.
Judgement call: an answer over 1 MiB, or with a status below 200, gets 502.
Judgement call: every target not starting with one / gets 400.
Model: opus-5-5
A question waits on GeoJS at most 800ms, the new default of
SWWAF_LOOKUP_TIMEOUT. In enforce mode it does not wait at all when it is
over the header limits, on a fetched blocklist with deny, on the CrowdSec
decision list, or over a whole rate limit. Not waiting never lowers a
client's limit. Just before the Core Rule Set, servewaf refuses more than
50 cookies or 100 header fields with 431.
Shared default changed: SWWAF_LOOKUP_TIMEOUT also applies to smallwebwaf
with no command.
Judgement call: SWWAF_REPUTATION_TIMEOUT stays 2s, since no request waits
on the DNS blocklists or AbuseIPDB.
Model: opus-5-5
`smallwebwaf servewaf` runs today's checks on the visitor's request each
question carries, answers rproxy, and passes nothing to an app. rproxy reports
every request servewaf answered. A report counts an allowed request's bytes and
writes servewaf's second log line. The table holds a copy of what a report
needs, never the request, so a report cannot reach a request being handled.
Judgement call: a question's body is read before any check; over 64 KiB gets 400.
Judgement call: a report's bytes and durations have ceilings, so they cannot overflow.
Judgement call: a `close` disposition for SWWAF_BAN_RESPONSE=close.
Model: opus-5-5
The default request limits now suit programmatic use of an API, such
as an automation fleet on another host, and still catch abusive bots:
- per minute: 500, room for a fleet's busiest minute and a few first
visits with an empty cache, and passed within seconds by a client
firing as fast as it can;
- per hour: 20000, passed by a client that keeps up 6 a second;
- per day: 300000, passed by a scraper that keeps up 4 a second.
The byte limits and ban lengths are unchanged. SPEC.md, README.md and
docs/settings.md give the reasons; docs/gitea.md redoes the Actions
runner sums.
Judgement call: the docs/alerts.md ban example shows the minute limit.
Model: opus-5-5
TestHealthCheck closed the app's server, and later smallwebwaf, and
expected connections to the freed ports to fail. Another test could take
such a port in between, so the connection succeeded and the test failed.
The failing app check now uses the multicast address 224.0.0.1, to which
TCP cannot connect. The failing smallwebwaf check uses port 0, on which
nothing listens. Neither result depends on other processes. The test now
stops smallwebwaf after its last check, so that it exits before the
test's temporary directories are removed. The code under test is
unchanged.
Model: opus-5-5
The request limits and the hour and day byte limits now sit about twice
above what 40 people busy with heavy web apps make together on one
address:
- requests: 6000 a minute, 100,000 an hour, 1,000,000 a day;
- bytes: 20 GiB an hour, 200 GiB a day.
The largest response drops from 5 GiB to 1 GiB. The minute byte limit
of 4 GiB is sized to hold two of the largest responses with their
requests. SPEC.md gives the new reasoning; README.md and the reference
documents give the new figures.
Model: opus-5-5
SPEC.md is the only place that says how smallwebwaf behaves and why; docs/ holds lists linking to it; the README is short and links to both. Passed independent review in round 8.
Model: fable-5-1 (writing); opus-5-5 (review, merge)
The AbuseIPDB section of README.md listed only a broken rate limit or
byte limit, a ban rule match, and a refusal by a block rule or the Core
Rule Set. A client is checked once its history counts any offence, so it
now also names the error burst, a request for a trap path and a refusal
for a missing or wrong token, as "What it does so far" does. The
definition of an offence under "Bans" in SPEC.md now names a refusal for
a missing or wrong token too, as the code counts one.
Model: opus-5-5
A netblock's next ban for a broken limit within
SWWAF_LIMIT_BAN_REPEAT_WINDOW lasts three times its last limit or admin
ban. A hand-written entry with an expires at or before its start made
that length zero or less, so the next limit ban refused nothing. The
bans.json checks now refuse such an entry: it stops the start, and an
edit holding one is set aside as bans.json.bad while running. README's
"State files" lists it with the other refused entries.
Model: opus-5-5
The run script gave ${SWWAF_STATE_DIR:-/var/lib/smallwebwaf} to the
smallwebwaf user and never read SWWAF_STATE_DIR_FILE, so a new volume
at the directory that file names stayed root's. The run script now
takes the setting from the variable or the file, and changes the owner
only when it is the absolute path of a directory that exists;
otherwise it starts smallwebwaf, which stops with its message naming
the setting. A chown that fails, as on a read-only volume, no longer
ends the run script. script/example-app, and its entry in README.md,
gain two steps: a container on such a volume becomes healthy, and a
missing state directory reaches smallwebwaf's message.
Model: opus-5-5
Go's server adds a Content-Type it guesses from the body when the
response's headers have no Content-Type key, which happened to some
of the app's answers and not others, as the first part of the body or
the headers went out first. When the app sends none, smallwebwaf now
sets the key with no value on the client's response, which sends none.
Model: opus-5-5
A client too slow to send its request line and headers within
SWWAF_CLIENT_REQUEST_TIMEOUT gets either no answer or an error answer
from Go's server, mostly 400 Bad Request; either way its connection is
closed, nothing reaches the app, and no request log line is written.
The README's "Request log" and SPEC's size and time limits said such a
client always gets no answer. Both documents' summaries of
smallwebwaf's own answers now cover only limits passed after the
request line and headers have been read, and give 408 only for a slow
body; SPEC names Go's server as the one answering 431.
Model: opus-5-5
ReverseProxy sends the path URL.EscapedPath gives, which decodes and
escapes again a path holding a character such as |, turning %2F into /.
The path as sent now goes in URL.Opaque, which Go sends as it is; the
rate limit and WAF exempt paths and the log line's path use the same
path. A path starting with // is left as before: Go would send it in
Opaque as a scheme and host.
Model: opus-5-5
ReverseProxy refuses a 101 whose Upgrade is not the one the client asked
for, but modifyResponse had already counted the 101 as sent, so
answerError sent nothing and the client got an empty 200. The 101 now
counts as sent only once ReverseProxy has taken over the client's
connection, and answerError closes the app's switched connection, which
ReverseProxy leaves open when it refuses the switch.
Model: opus-5-5
runit stops smallwebwaf and the app at the same time, so the five seconds
smallwebwaf gives requests in progress help only while the app still
answers them. Both documents now say so, say what the client gets when the
app exits at once on SIGTERM, and say what an app does to let its requests
in progress finish.
Model: opus-5-5
When less than 256 KiB of a client's body is left after the app has
answered, the rest is read by closing the body, which the transport,
ReverseProxy and Go's server each do, not through requestBody.Read.
waiting was then false, so a request timeout that ran out took the 504
branch, which set no read deadline, and the request held its connection
for as long as the client kept it open. Once the app has answered, that
branch now ends reading the client's body at once, whichever of
SWWAF_CLIENT_REQUEST_TIMEOUT and SWWAF_UPSTREAM_REQUEST_TIMEOUT ran out.
Model: opus-5-5
script/gitea-requests runs gitea 1.27.3 behind the image at its
defaults, sends through it the requests that reading gitea's source
found the Core Rule Set refuses, with a short sample of ordinary use,
and prints each request with the rule ids that matched it. make
gitea-requests runs it; make check does not.
Every case still holds, and each fix would let through what SPEC.md
says stays refused, so the notes specific to gitea now name them with
how each refusal plays out: file names in query parameters that read as
Java, script, PowerShell or SQL, such paths, and an image push that
reuses a layer of an image named like a command.
Model: opus-5-5
earlierBans took the counts in the notes of the netblock's first ban
held, so a ban dropped after an admin's ban, which came first, went
uncounted. Each ban held now gives a count, its notes' counts plus one
for it and each ban held after it, and the count that names the most
bans is taken whole. Counting from the newest ban alone would lose the
bans before an admin's ban added to bans.json without notes; taking
each cause from a different count would count twice a ban an admin
keeps, which later bans' notes count under its old cause.
Model: opus-5-5
When the app answered before the client had sent its whole body, and its
answer ended with its last bytes, every timeout was stopped,
SWWAF_CLIENT_REQUEST_TIMEOUT included. The transport went on waiting to
read the body, and Go's server cannot send the answer until that read
returns, so a client that stopped sending held the request until it
closed the connection. The timeouts now run on until the client has sent
its body, as they do before the app answers: once
SWWAF_CLIENT_REQUEST_TIMEOUT runs out the client is sent the app's
answer, the connection is closed, and the log line says timed_out.
Model: opus-5-5
sneak
clawbot