Adds nextcloud.conf, the thirteenth change. Nextcloud's WebDAV paths allow
PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, REPORT, SEARCH, LOCK and UNLOCK, and
the body of a PUT there, an uploaded file, is not read. The cookie
nc_username, and the fields that carry a search, a name, a file name or a
folder, skip the four rules that read them as commands or system files.
Comments, a share's note, a status and a document's text also skip the rules
that refused ordinary sentences, and the document the two that refused a
shell code block. Requests recorded from Nextcloud 35.0.2, its web pages, sync
client 3.16.7 and the OCS API pass at the defaults and while bodies are read.
Model: opus-5-5
user[email] on /auth and the paths under it skips 930120, 932120 and
932260. They refused signing up, signing in and resetting a password
with addresses such as python3@example.com, powershell@example.com and
jane.profile@example.com. The field keeps 932250 and every other rule.
Recorded from Mastodon 4.7.5: sign-up, sign-in, password reset and an
email change.
Model: opus-5-5
Adds mastodon.conf, the twelfth change. On Mastodon's paths, the fields
that carry a search, a name, a title, an address, a password or free text
skip the four rules that read them as commands or system files. They cover
the API and the web app's JSON, the admin and settings pages, sign-up and
sign-in, and the objects other servers deliver to an inbox. A post's text
skips 932125, 932230 and 932235 too. Hashtags and sign-up usernames skip
only 932120 and 932260. Uploaded file names skip 932180, a report's
forward keys skip 930120 by name, and the Accept header of a link lookup
skips 920600 on profile and post links. The requests recorded from
Mastodon 4.7.5 pass at the defaults and while bodies are read.
Model: opus-5-5
Rule 900601 let Referer skip five rules when it named a repository's
search, issues or pull requests with the owner api, such as
/api/v2/search, Mastodon's search. A page opened from there skipped
script and SQL injection checks on Referer. gitea has no owner named
api, so a chained condition now leaves out those paths, as rule 900600
does.
Model: opus-5-5
Rule 900600 took paths of an owner named api for gitea's search, issues,
pull requests and commit search inside a repository. /api/v2/search,
Mastodon's search, /api/v2/issues and /api/v2/pulls were among them, so
attacks in q scored 0 there. gitea has no owner named api, so a chained
condition now leaves out each of those paths under /api/<x>/. Go's regular
expressions have no lookahead, so the exclusion cannot be written inside
the pattern.
Model: opus-5-5
On /api.php, letitle and the generator fields gapto, gapprefix, gaifrom,
gaiprefix and geuquery skip 930120, 932120, 932250 and 932260, as the
fields they stand in for do.
On MediaWiki's pages, wpTitles from Special:EditWatchlist/raw, one title
per line, always skips 930120, 932120 and 932250. It skips 932125,
932230, 932235, 932260 and 932380 only when the body sends it once and it
holds no ";", "|", "&", backtick or "{", checked by a phase 2 rule. So
"; wget http://example.com/x.sh" is still refused.
Recorded from MediaWiki 1.46.2 at the digest the first MediaWiki change
used.
Model: opus-5-5
POST /user/password?_format=json with a mail of python3@example.com or
powershell@example.com was refused while bodies are read. On that path,
the JSON field mail now skips 932120 and 932260, the two rules the
recorded addresses set off, unless the query has a json.mail parameter.
Recorded from Drupal 11.4.11.
Model: opus-5-5
The Drupal and MediaWiki tests repeated the SQL and script attack strings,
which goconst reports. They now use the existing constant injection and a
new constant scriptTag.
Model: opus-5-5
Adds mediawiki.conf, the eleventh change: on MediaWiki's paths, the fields
that carry a search, a title, a name, a file name or an address skip the
four rules that read them as commands or system files, and the wikitext
fields skip the rules that refused saving a page. The reasons on the admin
pages skip only 930120 and 932120. /index.php counts as MediaWiki's only
with a page title in its query or after its path, since WordPress serves it
too. The requests recorded from MediaWiki 1.46.2, with and without short
URLs and on the admin pages, pass at the defaults and while bodies are
read. The app tests share the check of recorded requests and the list of
hard values.
Model: opus-5-5
POST /user/password?_format=json with a name of python3, powershell,
ls -la or .htaccess was refused while bodies are read. On that path,
the JSON field name now skips 930120, 932120, 932250 and 932260, the
four rules those recorded names set off, unless the query has a
json.name parameter. Recorded from Drupal 11.4.11.
Model: opus-5-5
Four tests closed a test server or stopped discriminator, and then
connected to its port. Another test process could take that port in
between and receive the connection.
- TestUnreachableRouteWatchLetsTheRequestThroughAndSaysWhy and
TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient, formerly
TestRefusedConnection..., point routewatch at 224.0.0.1, a multicast
address to which TCP cannot connect.
- TestStalledRemoteLogEndpointHoldsUpNoRequest keeps its endpoint open
and closes each connection it takes, instead of closing the endpoint.
- TestTheSiteAnswersWhileDiscriminatorIsStoppedAndStartedAgain gives
rproxy an address the test holds throughout. The test forwards each
connection there to the running discriminator, and closes it while
discriminator is stopped.
No change to non-test code.
Model: opus-5-5
Tests now share one Core Rule Set for each set of settings, loaded once
through sync.OnceValues in shared(). Exclusions-file tests and
TestCorazaWritesNoFile still load their own: loading is what they check.
Loading was not most of the time: the WordPress and Drupal recorded
requests, inspected one after another, were. Each recorded request now
runs as its own parallel subtest, with the same request and outcome.
Model: opus-5-5
Names, searches and the page to return to on more Drupal paths skip
the rules that misfire on them: 930120, 932120, 932250 and 932260,
plus 932230 and 932235 on destination only. An email address skips
930120, 932120 and 932260; the oEmbed url only 930120 and 932120.
Every Drupal path may carry a language prefix: a first segment of two
or three lowercase letters, such as /fr. Requests recorded from Drupal
11.4.11 with plain words, python3, powershell, ls -la and .htaccess are
added to the test data; tests show attacks and "; wget ..." still
refused on each new field.
Model: opus-5-5
Coraza v3.8.1 shares a rule's ctl:ruleRemoveTargetById exclusions
between concurrent requests, so one request can be checked with
another's. Coraza's main fixed it in its pull request 1745; no release
carries it yet, so go.mod names commit d028be399c68 by its
pseudo-version.
Model: opus-5-5
A Drupal and a WordPress request are inspected at once, many times
over, on one Core Rule Set. Coraza v3.8.1 shares each rule's
exclusions between requests, so the test fails under -race.
Model: opus-5-5
Each field skips only the rules its recorded requests set off:
- page on all of wp-admin, and the JSON fields title, excerpt and name
on /wp/v2/ routes: 930120, 932120, 932250 and 932260;
- p, postId and slug on all of wp-admin, and the JSON slug on /wp/v2/
routes: 932120 and 932260, since WordPress writes these as slugs;
- url on the block editor's link previews and embeds, and a commenter's
url: 930120 and 932120.
Every field keeps 932235. Add 61 requests recorded from WordPress 7.1.4
with these fields.
Model: opus-5-5
AGENTS.md holds the branching model, the entrypoints, the editing rule
and two iron rules: query strings are logged as received, and the
trusted-proxy default is the RFC 1918 set. README links it; SPEC.md
step 2 now states that default and that a given list replaces it.
Model: opus-5-5
Adds drupal.conf, the tenth change: on Drupal's paths, the fields that
carry a search, a name, a file name or an address skip the rules that
refused ordinary requests there. The recorded requests from Drupal 11.4
pass at the defaults and while bodies are read; SQL injection, path
traversal and script injection stay refused on those fields. Drupal's
tests share the reader of recorded requests and the attack constants
with the other apps' tests.
Model: opus-5-5
Add the ninth change: wordpress.conf switches off, per path and field, each
rule that refused ordinary WordPress 7.1.4 requests. Three groups of fields
skip four rules on system files and commands, and only the addresses of the
first group also skip 932235. Add the recorded requests, a test at the
defaults and reading bodies, and list WordPress in the README.
Model: opus-5-5 (rework); sonnet-5-5 (copied the recorded lines)
The first half of the naming audit: names now say what they are. `internal/waf` is `internal/coreruleset`. Log fields, the action `blocked_by_core_rule_set`, the metric and the alert event say Core Rule Set. Timings end `_ms`. Percentages say of what, as `rate_limit_percent` and `bytes_limit_percent`. Setting names follow in #234.
Judgement call: `blocked_by_core_rule_set`, not the audit's `core_rule_set_blocked`, to match `blocked_by_block_rule`.
Trips a reader: an existing state file holding `waf_blocked` or the anomaly scope `watch` stops the start and must be removed. `SWW_ALERT_EVENTS` listing `waf_block` must list `core_rule_set_match`.
Model: opus-5-5
The request log's `action` values now say what they mean, as sneak named them: `request_header_too_large`, `client_is_rate_limited`, `blocked_by_block_rule` and `client_denied`, with no aliases. `banned` splits into `temporarily_banned` and `permanently_banned`, by whether the ban runs out. When a permanent ban also covers a client, that ban is the one reported.
Trips a reader: a request over a rate limit stays `client_is_rate_limited` even when the ban it makes is permanent.
Disclosure: the offence key `rule_blocked` is renamed in `clients.json` too, so an existing file holding it stops the start and must be removed. Its `version` stays 1.
Model: opus-5-5
With `SWW_ROUTEWATCH_URL` set, a new heuristic, `routewatch_tags`, adds `asn=<number>` and `prefix=<netblock>` to a request from what a routewatch instance answers about the client. New package `internal/routewatch` asks in the background, never on a request's path, and keeps answers 1 hour in memory. The request log gains `routewatch_tags_missing`, saying why a request has none. `prefix=` works in `SWW_TAG_DENY` and `SWW_TAG_LIMIT_PERCENT`.
Trips a reader: a client's first request has no routewatch tags, so `prefix=` refuses from its next request after the answer comes.
Judgement call: with `SWW_LOOKUP_SOURCE=off` and `SWW_ROUTEWATCH_URL` set, an `asn=` tag no longer stops the start.
Model: opus-5-5
sneak asked for every document to be audited for sentences that do not say what they mean. The byte limits now say which bytes count (response bodies, request bodies or both, as `SWW_BYTES_COUNT` says, WebSocket traffic included) and what passing one does. Other unclear or false sentences in `README.md`, `SPEC.md`, `EVALUATION.md` and `docs/` are rewritten.
Documents only. `REPO_POLICIES.md` is shared policy text and stayed out.
Model: fable-5-1 (first audit); opus-5-5 (rework)
Each request gets tags, logged as `tags`: `asn=`, `country=`, one per `SWW_TAG_LISTS` list holding the client, `crowdsec`, `dnsbl=<zone>`, `abuseipdb` and `exploitsignature`. `SWW_TAG_DENY` refuses by tag and `SWW_TAG_LIMIT_PERCENT` lowers limits by tag. They replace `SWW_ASN_LIMIT_PERCENT`, `SWW_COUNTRY_LIMIT_PERCENT` and `SWW_DENIED_COUNTRIES`, with no aliases. A tag in those settings that nothing configured can give stops the start.
Judgement call: `SWW_ASN_LIMIT_PERCENT_URL` stays, its entries counting like `asn=` tags.
Judgement call: `SWW_TAG_LIMIT_PERCENT` also lowers byte limits, as the settings it replaces did.
Trips a reader: `sww_country_list_refusals_total` now counts only `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES` refusals.
Model: opus-5-5
`script/publish` runs `script/docker` against the Docker host and pushes the image to the registry under four tags: the UTC date, the short commit hash, the branch name and `latest`. It refuses a working tree with uncommitted changes, since the hash tag names a commit.
Also carries sneak's `golines -m 77` line wrapping in three files, with no code change.
Unverified: a real build on the Docker host and a real push from this commit.
Deviation: sneak's FIXME comments are out, because the linter refuses them; they are #227, #228 and #229.
Model: opus-5-5
Co-authored-by: sneak <sneak@sneak.berlin>
Each discriminator check is now a type in its own file behind one small interface, listed in `heuristics.go`. Adding a heuristic is one file and one line. Behaviour does not change.
Refusal and ban log lines gain `heuristic` and `heuristic_figures`. A ban after the answer, by `refusal_limit` or `byte_limits`, logs `after_answer_heuristic` and its figures, so one line can name both.
Deviation: four lists, not the planned three, so `refusal_limit` still bans at the decision.
Rule suppressed: `//nolint:lll` on one struct tag.
Model: opus-5-5
xz-utils, which nix-bin depends on, links its man pages with update-alternatives when it installs, and the image's dpkg config drops man pages, so each link printed a warning. A config file re-includes only the man1 pages for this install; the same step deletes them, their dangling links in /etc/alternatives and the config file. No man page or package is added or removed.
Model: deepseek-v4-pro
The image's `ca-certificates`, `nix-bin` and `runit` now come from Ubuntu's archive instead of a dated snapshot, whose service had failed image builds. apt fetches them over plain HTTP and checks each against the archive's signed package lists, so no CA certificates are borrowed from the Go image. The README, `SPEC.md` and `docs/image.md` say so; the amd64-only limit, which came from the removed hash check, is gone.
Rule cut against: `REPO_POLICIES.md` pins every external reference by hash; sneak's ruling at sneak/prompts#72 (comment) allows Ubuntu's own package index.
Model: deepseek-v4-pro (implementation), opus-5-5 (rework)
New defaults, as sneak approved: 120 requests a minute, 1,500 an hour and 6,000 a day; bytes off a minute, 50G an hour and 100G a day. The README, `SPEC.md`, `docs/settings.md` and `docs/gitea.md` give the new figures and reasoning, and point Actions runners and shared addresses at `SWWAF_RATE_LIMIT_EXEMPT_NETS`. The README examples now hold against the new defaults.
An Actions runner that is not exempted makes more requests an hour than the new limit and will be banned.
Model: opus-5-5
The image's `ca-certificates`, `nix-bin` and `runit` now come from Ubuntu's archive instead of a dated snapshot, whose service had failed image builds. apt fetches them over plain HTTP and checks each against the archive's signed package lists, so no CA certificates are borrowed from the Go image. The README, `SPEC.md` and `docs/image.md` say so; the amd64-only limit, which came from the removed hash check, is gone.
Rule cut against: `REPO_POLICIES.md` pins every external reference by hash; sneak's ruling at sneak/prompts#72 (comment) allows Ubuntu's own package index.
Model: deepseek-v4-pro (implementation), opus-5-5 (rework)
The package passes nothing on since the one-process mode went: it answers rproxy's questions, takes its reports and answers servewaf's own endpoints. It is now named after the command it serves, with `git mv`, and its constructor is `servewaf.New`. `SPEC.md` and the other documents follow. No change of behaviour.
Model: opus-5-5
A compose setup that runs gitea 1.27.3 behind traefik, rproxy and servewaf on one shared network with IPv4 and IPv6, and `script/gitea-setup` (`make gitea-setup`), which brings it up and checks that a page load, a push and a clone pass through rproxy and are checked by servewaf. Containers on the shared network skip servewaf's checks; the gateways do not. gitea gets a secret key made per install and has sign-up off. `docs/gitea.md` explains each setting.
Judgement call: the Core Rule Set runs in `detect` mode here until #198 lands.
Model: deepseek-v4-pro (implementation), opus-5-5 (rework)
Merged by sneak's order of 6 November ("take the current state of smallwebwaf and get it on to main. i am going to deploy it soon in its alpha state").
Model: opus-5-5
`smallwebwaf` with no command now prints the commands and exits with status 2.
The image runs rproxy, or servewaf with SMALLWEBWAF_COMMAND=servewaf. The
passing-on code, its settings and its log fields and metric go. internal/proxy
keeps the checks, answers, reports and own endpoints; its tests run through the
real rproxy. The documents describe the pair.
Judgement call: clients.json keeps token_refused, since state files refuse unknown fields.
Deviation: script/gitea-requests sets SWWAF_LOG_REQUEST_HEADERS, to label requests.
Partially verified: make example-app and make gitea-requests never built the image, as Ubuntu's snapshot service answered 5xx.
Model: opus-5-5
`smallwebwaf rproxy` reads the first 64 KiB of each body, asks servewaf over
HTTP/1.1, and passes what servewaf allows to the app within its limits. Any
other answer goes to the visitor as servewaf gave it. SWRP_DEFAULT_DISPOSITION
decides only when servewaf cannot be connected to, does not answer at all, or
does not answer within SWRP_TIMEOUT_MS. A method SPEC.md does not list gets 405.
Duplication: internal/rproxy copies passing-on code from internal/proxy until
#172 removes the originals.
Rule suppressed: gosec G704 and G705; gosec at
internal/smallwebwaf/rproxy_test.go:431; tagliatelle at
internal/requestlog/requestlog.go:203; paralleltest on one test.
Judgement call: an answer over 1 MiB, or with a status below 200, gets 502.
Judgement call: every target not starting with one / gets 400.
Model: opus-5-5
A question waits on GeoJS at most 800ms, the new default of
SWWAF_LOOKUP_TIMEOUT. In enforce mode it does not wait at all when it is
over the header limits, on a fetched blocklist with deny, on the CrowdSec
decision list, or over a whole rate limit. Not waiting never lowers a
client's limit. Just before the Core Rule Set, servewaf refuses more than
50 cookies or 100 header fields with 431.
Shared default changed: SWWAF_LOOKUP_TIMEOUT also applies to smallwebwaf
with no command.
Judgement call: SWWAF_REPUTATION_TIMEOUT stays 2s, since no request waits
on the DNS blocklists or AbuseIPDB.
Model: opus-5-5
`smallwebwaf servewaf` runs today's checks on the visitor's request each
question carries, answers rproxy, and passes nothing to an app. rproxy reports
every request servewaf answered. A report counts an allowed request's bytes and
writes servewaf's second log line. The table holds a copy of what a report
needs, never the request, so a report cannot reach a request being handled.
Judgement call: a question's body is read before any check; over 64 KiB gets 400.
Judgement call: a report's bytes and durations have ceilings, so they cannot overflow.
Judgement call: a `close` disposition for SWWAF_BAN_RESPONSE=close.
Model: opus-5-5
The default request limits now suit programmatic use of an API, such
as an automation fleet on another host, and still catch abusive bots:
- per minute: 500, room for a fleet's busiest minute and a few first
visits with an empty cache, and passed within seconds by a client
firing as fast as it can;
- per hour: 20000, passed by a client that keeps up 6 a second;
- per day: 300000, passed by a scraper that keeps up 4 a second.
The byte limits and ban lengths are unchanged. SPEC.md, README.md and
docs/settings.md give the reasons; docs/gitea.md redoes the Actions
runner sums.
Judgement call: the docs/alerts.md ban example shows the minute limit.
Model: opus-5-5
TestHealthCheck closed the app's server, and later smallwebwaf, and
expected connections to the freed ports to fail. Another test could take
such a port in between, so the connection succeeded and the test failed.
The failing app check now uses the multicast address 224.0.0.1, to which
TCP cannot connect. The failing smallwebwaf check uses port 0, on which
nothing listens. Neither result depends on other processes. The test now
stops smallwebwaf after its last check, so that it exits before the
test's temporary directories are removed. The code under test is
unchanged.
Model: opus-5-5
The request limits and the hour and day byte limits now sit about twice
above what 40 people busy with heavy web apps make together on one
address:
- requests: 6000 a minute, 100,000 an hour, 1,000,000 a day;
- bytes: 20 GiB an hour, 200 GiB a day.
The largest response drops from 5 GiB to 1 GiB. The minute byte limit
of 4 GiB is sized to hold two of the largest responses with their
requests. SPEC.md gives the new reasoning; README.md and the reference
documents give the new figures.
Model: opus-5-5
SPEC.md is the only place that says how smallwebwaf behaves and why; docs/ holds lists linking to it; the README is short and links to both. Passed independent review in round 8.
Model: fable-5-1 (writing); opus-5-5 (review, merge)
sneak
clawbot