next into main #223

Open
clawbot wants to merge 23 commits from next into main
pull from: next
23 Commits
Author SHA1 Message Date
clawbot c1bb813a5e Core Rule Set tests: name repeated attack strings (closes #263)
check / check (push) Failing after 4m4s
The Drupal and MediaWiki tests repeated the SQL and script attack strings,
which goconst reports. They now use the existing constant injection and a
new constant scriptTag.

Model: opus-5-5
2026-11-10 01:26:07 +01:00
clawbot 9aa5b303cf Core Rule Set: MediaWiki's ordinary requests pass by default (closes #237)
check / check (push) Failing after 1m48s
Adds mediawiki.conf, the eleventh change: on MediaWiki's paths, the fields
that carry a search, a title, a name, a file name or an address skip the
four rules that read them as commands or system files, and the wikitext
fields skip the rules that refused saving a page. The reasons on the admin
pages skip only 930120 and 932120. /index.php counts as MediaWiki's only
with a page title in its query or after its path, since WordPress serves it
too. The requests recorded from MediaWiki 1.46.2, with and without short
URLs and on the admin pages, pass at the defaults and while bodies are
read. The app tests share the check of recorded requests and the list of
hard values.

Model: opus-5-5
2026-11-10 00:58:15 +01:00
clawbot e29dacad63 Core Rule Set: Drupal's JSON password reset passes (closes #259)
check / check (push) Successful in 4m2s
POST /user/password?_format=json with a name of python3, powershell,
ls -la or .htaccess was refused while bodies are read. On that path,
the JSON field name now skips 930120, 932120, 932250 and 932260, the
four rules those recorded names set off, unless the query has a
json.name parameter. Recorded from Drupal 11.4.11.

Model: opus-5-5
2026-11-10 00:58:08 +01:00
clawbot 9e7a7e60c7 Tests no longer connect to test servers they closed (closes #257)
check / check (push) Failing after 2m5s
Four tests closed a test server or stopped discriminator, and then
connected to its port. Another test process could take that port in
between and receive the connection.

- TestUnreachableRouteWatchLetsTheRequestThroughAndSaysWhy and
  TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient, formerly
  TestRefusedConnection..., point routewatch at 224.0.0.1, a multicast
  address to which TCP cannot connect.
- TestStalledRemoteLogEndpointHoldsUpNoRequest keeps its endpoint open
  and closes each connection it takes, instead of closing the endpoint.
- TestTheSiteAnswersWhileDiscriminatorIsStoppedAndStartedAgain gives
  rproxy an address the test holds throughout. The test forwards each
  connection there to the running discriminator, and closes it while
  discriminator is stopped.

No change to non-test code.

Model: opus-5-5
2026-11-10 00:58:01 +01:00
clawbot c86df7610d Core Rule Set tests: load each set of settings once (closes #252)
check / check (push) Successful in 4m12s
Tests now share one Core Rule Set for each set of settings, loaded once
through sync.OnceValues in shared(). Exclusions-file tests and
TestCorazaWritesNoFile still load their own: loading is what they check.

Loading was not most of the time: the WordPress and Drupal recorded
requests, inspected one after another, were. Each recorded request now
runs as its own parallel subtest, with the same request and outcome.

Model: opus-5-5
2026-11-09 23:57:54 +01:00
clawbot 3a37f82e34 Core Rule Set: more of Drupal's ordinary fields pass (closes #244)
check / check (push) Failing after 17m27s
Names, searches and the page to return to on more Drupal paths skip
the rules that misfire on them: 930120, 932120, 932250 and 932260,
plus 932230 and 932235 on destination only. An email address skips
930120, 932120 and 932260; the oEmbed url only 930120 and 932120.
Every Drupal path may carry a language prefix: a first segment of two
or three lowercase letters, such as /fr. Requests recorded from Drupal
11.4.11 with plain words, python3, powershell, ls -la and .htaccess are
added to the test data; tests show attacks and "; wget ..." still
refused on each new field.

Model: opus-5-5
2026-11-09 23:09:15 +01:00
sneak c0313f7b27 Coraza: use the commit that keeps exclusions per request (closes #254)
check / check (push) Successful in 5m27s
Coraza v3.8.1 shares a rule's ctl:ruleRemoveTargetById exclusions
between concurrent requests, so one request can be checked with
another's. Coraza's main fixed it in its pull request 1745; no release
carries it yet, so go.mod names commit d028be399c68 by its
pseudo-version.

Model: opus-5-5
2026-11-09 21:00:08 +00:00
sneak 35dfd1f5e0 Core Rule Set: test concurrent requests keep their own exclusions
check / check (push) Failing after 2m41s
A Drupal and a WordPress request are inspected at once, many times
over, on one Core Rule Set. Coraza v3.8.1 shares each rule's
exclusions between requests, so the test fails under -race.

Model: opus-5-5
2026-11-09 20:59:45 +00:00
clawbot 9f98982446 Core Rule Set: WordPress's names and addresses pass on more fields (closes #245)
check / check (push) Successful in 4m55s
Each field skips only the rules its recorded requests set off:

- page on all of wp-admin, and the JSON fields title, excerpt and name
  on /wp/v2/ routes: 930120, 932120, 932250 and 932260;
- p, postId and slug on all of wp-admin, and the JSON slug on /wp/v2/
  routes: 932120 and 932260, since WordPress writes these as slugs;
- url on the block editor's link previews and embeds, and a commenter's
  url: 930120 and 932120.

Every field keeps 932235. Add 61 requests recorded from WordPress 7.1.4
with these fields.

Model: opus-5-5
2026-11-09 21:58:18 +01:00
clawbot 3093ed8f0d Add AGENTS.md with the repository's rules for agents (closes #246)
check / check (push) Successful in 4m51s
AGENTS.md holds the branching model, the entrypoints, the editing rule
and two iron rules: query strings are logged as received, and the
trusted-proxy default is the RFC 1918 set. README links it; SPEC.md
step 2 now states that default and that a given list replaces it.

Model: opus-5-5
2026-11-09 20:43:18 +01:00
clawbot decd9b4658 Core Rule Set: Drupal's ordinary requests pass by default (closes #236)
check / check (push) Successful in 5m4s
Adds drupal.conf, the tenth change: on Drupal's paths, the fields that
carry a search, a name, a file name or an address skip the rules that
refused ordinary requests there. The recorded requests from Drupal 11.4
pass at the defaults and while bodies are read; SQL injection, path
traversal and script injection stay refused on those fields. Drupal's
tests share the reader of recorded requests and the attack constants
with the other apps' tests.

Model: opus-5-5
2026-11-09 19:57:53 +01:00
clawbot fd734fcc22 Core Rule Set: WordPress's ordinary requests pass by default (closes #235)
check / check (push) Successful in 4m43s
Add the ninth change: wordpress.conf switches off, per path and field, each
rule that refused ordinary WordPress 7.1.4 requests. Three groups of fields
skip four rules on system files and commands, and only the addresses of the
first group also skip 932235. Add the recorded requests, a test at the
defaults and reading bodies, and list WordPress in the README.

Model: opus-5-5 (rework); sonnet-5-5 (copied the recorded lines)
2026-11-09 19:26:17 +01:00
clawbot 719691d1e8 Name log fields, metrics and the Core Rule Set package for what they are (closes #192)
check / check (push) Failing after 3m22s
The first half of the naming audit: names now say what they are. `internal/waf` is `internal/coreruleset`. Log fields, the action `blocked_by_core_rule_set`, the metric and the alert event say Core Rule Set. Timings end `_ms`. Percentages say of what, as `rate_limit_percent` and `bytes_limit_percent`. Setting names follow in #234.

Judgement call: `blocked_by_core_rule_set`, not the audit's `core_rule_set_blocked`, to match `blocked_by_block_rule`.
Trips a reader: an existing state file holding `waf_blocked` or the anomaly scope `watch` stops the start and must be removed. `SWW_ALERT_EVENTS` listing `waf_block` must list `core_rule_set_match`.

Model: opus-5-5
2026-11-09 08:47:26 +01:00
clawbot aede434f6a Name request log actions so they say what they mean (closes #229)
check / check (push) Successful in 6m30s
The request log's `action` values now say what they mean, as sneak named them: `request_header_too_large`, `client_is_rate_limited`, `blocked_by_block_rule` and `client_denied`, with no aliases. `banned` splits into `temporarily_banned` and `permanently_banned`, by whether the ban runs out. When a permanent ban also covers a client, that ban is the one reported.

Trips a reader: a request over a rate limit stays `client_is_rate_limited` even when the ban it makes is permanent.
Disclosure: the offence key `rule_blocked` is renamed in `clients.json` too, so an existing file holding it stops the start and must be removed. Its `version` stays 1.

Model: opus-5-5
2026-11-09 06:48:26 +01:00
clawbot 6850cb0e9f Add asn= and prefix= tags from a routewatch instance (closes #204)
check / check (push) Successful in 5m0s
With `SWW_ROUTEWATCH_URL` set, a new heuristic, `routewatch_tags`, adds `asn=<number>` and `prefix=<netblock>` to a request from what a routewatch instance answers about the client. New package `internal/routewatch` asks in the background, never on a request's path, and keeps answers 1 hour in memory. The request log gains `routewatch_tags_missing`, saying why a request has none. `prefix=` works in `SWW_TAG_DENY` and `SWW_TAG_LIMIT_PERCENT`.

Trips a reader: a client's first request has no routewatch tags, so `prefix=` refuses from its next request after the answer comes.
Judgement call: with `SWW_LOOKUP_SOURCE=off` and `SWW_ROUTEWATCH_URL` set, an `asn=` tag no longer stops the start.

Model: opus-5-5
2026-11-09 05:47:34 +01:00
clawbot d7c62f2854 Say which bytes the byte limits count, and what passing one does (closes #203)
check / check (push) Failing after 3m48s
sneak asked for every document to be audited for sentences that do not say what they mean. The byte limits now say which bytes count (response bodies, request bodies or both, as `SWW_BYTES_COUNT` says, WebSocket traffic included) and what passing one does. Other unclear or false sentences in `README.md`, `SPEC.md`, `EVALUATION.md` and `docs/` are rewritten.

Documents only. `REPO_POLICIES.md` is shared policy text and stayed out.

Model: fable-5-1 (first audit); opus-5-5 (rework)
2026-11-09 04:03:17 +01:00
clawbot c66277b111 Request tags, with refusing and limits per tag (closes #175)
check / check (push) Failing after 5m8s
Each request gets tags, logged as `tags`: `asn=`, `country=`, one per `SWW_TAG_LISTS` list holding the client, `crowdsec`, `dnsbl=<zone>`, `abuseipdb` and `exploitsignature`. `SWW_TAG_DENY` refuses by tag and `SWW_TAG_LIMIT_PERCENT` lowers limits by tag. They replace `SWW_ASN_LIMIT_PERCENT`, `SWW_COUNTRY_LIMIT_PERCENT` and `SWW_DENIED_COUNTRIES`, with no aliases. A tag in those settings that nothing configured can give stops the start.

Judgement call: `SWW_ASN_LIMIT_PERCENT_URL` stays, its entries counting like `asn=` tags.
Judgement call: `SWW_TAG_LIMIT_PERCENT` also lowers byte limits, as the settings it replaces did.
Trips a reader: `sww_country_list_refusals_total` now counts only `SWW_EXCLUSIVELY_ALLOWED_COUNTRIES` refusals.

Model: opus-5-5
2026-11-09 03:30:49 +01:00
sneak f6435b1a73 Add make publish: build the image on the Docker host and push it to the registry
check / check (push) Successful in 3m23s
`script/publish` runs `script/docker` against the Docker host and pushes the image to the registry under four tags: the UTC date, the short commit hash, the branch name and `latest`. It refuses a working tree with uncommitted changes, since the hash tag names a commit.

Also carries sneak's `golines -m 77` line wrapping in three files, with no code change.

Unverified: a real build on the Docker host and a real push from this commit.
Deviation: sneak's FIXME comments are out, because the linter refuses them; they are #227, #228 and #229.

Model: opus-5-5
Co-authored-by: sneak <sneak@sneak.berlin>
2026-11-09 02:03:39 +01:00
clawbot 2349b2a081 discriminator: each decision heuristic is its own module (closes #195)
check / check (push) Failing after 1m53s
Each discriminator check is now a type in its own file behind one small interface, listed in `heuristics.go`. Adding a heuristic is one file and one line. Behaviour does not change.

Refusal and ban log lines gain `heuristic` and `heuristic_figures`. A ban after the answer, by `refusal_limit` or `byte_limits`, logs `after_answer_heuristic` and its figures, so one line can name both.

Deviation: four lists, not the planned three, so `refusal_limit` still bans at the decision.
Rule suppressed: `//nolint:lll` on one struct tag.

Model: opus-5-5
2026-11-09 01:14:24 +01:00
clawbot 42e06bf298 Quiet the update-alternatives man page warnings in the image build (closes #222) (#226)
check / check (push) Successful in 5m24s
xz-utils, which nix-bin depends on, links its man pages with update-alternatives when it installs, and the image's dpkg config drops man pages, so each link printed a warning. A config file re-includes only the man1 pages for this install; the same step deletes them, their dangling links in /etc/alternatives and the config file. No man page or package is added or removed.

Model: deepseek-v4-pro
2026-11-07 15:21:18 +01:00
clawbot beb503b233 Rename the program to sww and its deciding daemon to discriminator (closes #218) (#221)
check / check (push) Failing after 1m34s
2026-11-07 06:51:06 +01:00
clawbot 2e2f6f7175 Core Rule Set: gitea's ordinary requests pass by default (#208)
check / check (push) Successful in 3m29s
2026-11-07 05:21:22 +01:00
clawbot 7fed6b6d36 Remove what is about deploying behind Gitea (#210)
check / check (push) Successful in 3m30s
2026-11-07 03:52:02 +01:00