Commit Graph
159 Commits
Author SHA1 Message Date
clawbot e83b225a81 Core Rule Set: the searches of the pages gitea and Forgejo share pass (closes #352)
check / check (push) Waiting to run
Recorded with python3, powershell, ls -la, .htaccess, file names and plain
words from gitea 1.27.3 and Forgejo 17.0.1. q on these searches now skips
930120, 932120, 932250 and 932260 (900603), on gitea's /-/admin/ and
Forgejo's /admin/ alike, and in the body of the admin forms that adopt a
repository or activate an email. A repository's packages page and the admin
packages page search package names, so they join 900602, which also skips
944130. 900605, which lets /bin/sh through in q on branches and owner
packages, now also covers the tags page and those two pages, since tag and
package names can hold a slash.

Model: opus-5-5
2026-11-11 15:42:18 +00:00
clawbot 63e58e326b Settings code: one file per daemon (closes #227)
check / check (push) Waiting to run
internal/config now keeps waf's settings in waf.go and rproxy's in
rproxy.go. config.go keeps what both read: the package doc,
SWW_COMMAND, the environment reader, listen addresses, sizes and
netblock lists. The app URL reader and the default trusted proxies,
which only rproxy uses, move to rproxy.go. config_test.go becomes
waf_test.go, beside rproxy_test.go. Code is moved, not changed.

Model: opus-5-5
2026-11-11 15:59:37 +01:00
clawbot f9bec5146b Core Rule Set: Forgejo's searches at a branch or tag, of releases and of job logs pass (closes #350)
check / check (push) Waiting to run
`testdata/forgejo-requests.txt` records these Forgejo 17.0.1 searches with the hard values, file names and plain words, and the code search with code and a page opened from each search. `forgejo.conf` lets them through:

- the code search at a branch or tag, where the home page's search box sends its search without a code indexer: `q` skips what it skips on gitea's code search, and `Referer` what it skips after gitea's searches;
- the release search, page and API: `q` skips 930120, 932120, 932160, 932250 and 932260, since it matches tag names;
- the job log search in the API: the same, without 932160.

In `gitea.conf`, `q` on a repository's branches page and an owner's packages page also skips 932160, so `/bin/sh` passes there. `SPEC.md` lists both.

Model: opus-5-5
2026-11-11 15:44:31 +01:00
clawbot 883969a9cd Core Rule Set: ls -la passes in q on a gitea user's or organisation's page (closes #349)
check / check (push) Waiting to run
On a path of one segment made of the characters a user's or an
organisation's name may hold, q now skips 932250 too, unless the path
ends in a server-side script's extension, such as .php or .cgi. The
searches on a user's and an organisation's page are recorded from gitea
1.27.3 and Forgejo 17.0.1.

Model: opus-5-5
2026-11-11 14:13:33 +01:00
clawbot b6aa2cbc4b Core Rule Set: Forgejo's ordinary requests pass by default (closes #342)
check / check (push) Waiting to run
Forgejo 17.0.1's ordinary traffic, recorded in `testdata/forgejo-requests.txt` (pages signed in and out, admin pages, the API with a token, git clone and push, the generic and container registries, uploads), passes at the default settings. What Forgejo sends and gitea does not: its flash cookie `flash` joins the cookies left out, and in the new `forgejo.conf` `q` on its wiki search skips five rules. In `gitea.conf`, the API's package search joins the name searches, and `q` there also skips 932250, except on user and organisation pages. `README.md` lists Forgejo; `SPEC.md` lists the changes.

- Still refused, each tracked: `ls -la` on user and organisation pages; file and branch names in bodies while bodies are read; Forgejo's searches at a branch or tag, of releases and of job logs; other search fields not yet recorded.

Model: opus-5-5
2026-11-11 12:12:50 +01:00
clawbot 697c2225d9 The image creates SWW_STATE_DIR when it is missing (closes #337)
check / check (push) Waiting to run
Before `waf` starts, the image's `run` script now makes `SWW_STATE_DIR` if it is missing (`mkdir -p`), gives it to `sww:sww` and sets its mode to `0700`, as it already does for the cache directory's owner. A value that is not an absolute path, or an unreadable `SWW_STATE_DIR_FILE`, still changes nothing, and `waf` stops naming the setting. The `Dockerfile` comment, `docs/image.md`, `docs/settings.md` and `SPEC.md` say the image makes the directory. `script/example-app` checks that a missing `SWW_STATE_DIR` is made, owned by `sww` with mode `0700`.

- `chmod 0700` also applies to an existing directory, a mounted volume included, at every start of `waf`.

Model: opus-5-5
2026-11-11 11:33:08 +01:00
clawbot b93bad5842 Answer-time tests check the answer, not wall-clock milliseconds (closes #316)
check / check (push) Waiting to run
Tests only. `TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs` and `TestHangingRoutewatchNeverDelaysAndFailsTheQuery` run in a `synctest` bubble, whose clock load cannot move, instead of comparing wall-clock durations with fixed bounds. The first checks the answer comes after exactly `SWW_LOOKUP_TIMEOUT`, and that the DNS blocklist's query, held by a stand-in resolver, had not failed when it came. The second requires `no_answer_yet` after exactly zero time. The other wall-clock checks in the tests were already in bubbles or are bounds load cannot break.

- The held DNS query still times out on the real clock after two seconds; a stall that long between question and check fails the test.

Model: opus-5-5
2026-11-11 10:56:51 +01:00
clawbot 1748a670f9 sww tests: wait for every starting line as long as the start takes (closes #331)
check / check (push) Waiting to run
Tests only, in `internal/sww`. Every wait for the `starting` line now uses `output.lineBeforeExit`, which waits until the line is written or sww exits, so a slow Core Rule Set compile on a loaded host no longer fails these tests. `TestMainRunsWAF` and `startRProxyProcess` start sww through `startProcess`, whose cleanup kills sww as the test ends and waits for it to exit; `startRProxyProcess` returns a stop function that sends SIGTERM and fails the test unless sww exits with status 0 within `waitLimit`.

Model: opus-5-5
2026-11-11 10:56:47 +01:00
clawbot 83525b5e73 Caches of outside APIs live in /var/cache/sww (closes #314)
check / check (push) Waiting to run
Every cache of answers from an outside service has its own file in `SWW_CACHE_DIR` (default `/var/cache/sww`): `lookups.json` (GeoJS), `lists.json` (lists fetched by URL, CrowdSec's decisions included), `dnsbl.json`, `abuseipdb-scores.json` and `routewatch.json` (at most 100,000 answers, each kept an hour). `reputation.json` is gone; the AbuseIPDB checks spent today stay in `SWW_STATE_DIR` as `abuseipdb-checks-spent.json`, since losing them would spend the daily budget twice.

A cache file never stops the start: one that does not parse is set aside as `<name>.bad` with a `file_error` alert. An unwritable `SWW_CACHE_DIR` stops the start. The `run` script creates the cache directory and gives it to `sww`. The cache directory needs no volume and can be left out of backups.

Model: opus-5-5
2026-11-11 10:56:43 +01:00
sneak 6a59011dee rproxy: cut off an answer that stalls after arriving before the whole request (closes #345)
check / check (push) Waiting to run
When the app's answer arrived before the app had been sent the whole
request, SWRP_MAX_RESPONSE_TIME_MS had no timer: only the end of the
request started one. If the app then stalled, only a write could fail at
the deadline. With nothing written, the connection stayed open until the
request time limit.

modifyResponse now starts the response timer when it ends the request.
Once the timer cuts off the app's answer, nothing is left to flush, so the
last flush in finish succeeded and Go's server read the rest of the body.
finish now also ends that reading at once after a refusal of an answer
already passed on.

Model: opus-5-5
2026-11-11 08:45:51 +00:00
sneak e3c5b4ecf8 rproxy tests: a cut-off answer closes the connection, also after its first part reached the visitor
check / check (push) Waiting to run
The app now sends its late parts only once rproxy has ended the request to
it, not after a sleep timed against the limit. With the body in one part it
stalls after its headers; with two parts, after the first part, which
reaches the visitor. Both cases fail on the current code: nothing cuts the
answer off, and the connection stays open. The 2-second limit lets rproxy
pass on a part sent at once, even on a busy host.

A new test, which passes on the current code, has every flush fail with no
refusal, so that only the failed-flush check in finish ends the reading of
the visitor's body.

Model: opus-5-5
2026-11-11 08:45:51 +00:00
clawbot 6e370b346b rproxy: remove forwarded headers spelt with underscores from an untrusted peer (closes #334)
check / check (push) Waiting to run
For a peer outside `SWRP_TRUSTED_PROXIES`, `rproxy` now reads each header name with `_` as `-` and in any letter case, and removes the header when that name starts with `X-Forwarded-` or is `Forwarded` or `X-Real-IP`. Before, `X_Forwarded_Ssl: on` or `X_Real_IP` from a client connecting directly reached the app unchanged, and an app behind CGI or FastCGI reads them as the dash-spelt headers. A trusted peer's headers pass on as before. `SPEC.md` says so.

Model: opus-5-5
2026-11-11 08:29:59 +01:00
sneak eba697af48 rproxy: close the connection when a short answer is cut off (closes #318)
check / check (push) Waiting to run
When SWRP_MAX_RESPONSE_TIME_MS ran out before the last of a short answer
was written, the flush failed. If the visitor had stopped short of its
announced body, Go's server then read the rest until
SWRP_MAX_REQUEST_TIME_MS ran out, keeping the connection open with the
answer cut off.

finish now flushes once more, and if that fails at its deadline, it ends
the reading of the body at once. ReverseProxy ignores the failed flush when
the app's answer came in one read, and panics when it was still waiting for
more; finish runs after both. Go's server never reuses a connection after a
failed write, so it closes it.

Model: opus-5-5
2026-11-11 07:54:10 +01:00
sneak b9cfbc7fcb rproxy tests: a cut-off answer must close the connection, and the stop-short tests no longer race the clock
A new test fails on the current code. The visitor stops a little short of
its announced body, and the response time limit runs out before rproxy can
write the app's short body. The connection then stays open until the
request time limit instead of closing. The app sends that body in one part,
and in two; with two, ReverseProxy panics instead of returning.

The test that failed under load used a 300 ms response time limit, which a
busy host can use up between the app's headers and its body. It and the
test that answers a visitor that stops sending now use limits far longer
than the test's 10-second read deadline.

Model: opus-5-5
2026-11-11 07:54:10 +01:00
clawbot f077dd7fd0 Say that rproxy passes a refused question's 400 to the visitor (closes #326)
check / check (push) Waiting to run
The comment above `readQuestion` in `internal/waf/waf.go` said `rproxy` treats a refused question (`400`, no disposition) as no answer. It passes the `400` to the visitor, and `SWRP_DEFAULT_DISPOSITION` does not apply, as `SPEC.md` and `internal/rproxy/waf.go` say. Comment only.

Model: opus-5-5
2026-11-11 07:50:59 +01:00
clawbot 81ec6f9bf5 Request log: write & < > as sent (closes #327)
check / check (push) Waiting to run
Request lines of `waf` and `rproxy` now hold `&`, `<` and `>` as sent, where `json.Marshal` wrote `\u0026`, `\u003c` and `\u003e`, and are still valid JSON. `writeJSONLine` encodes with a `json.Encoder` that has HTML escaping off, and still writes each line in one write. This follows the `AGENTS.md` rule that the request log records query strings exactly as received, and lets `grep` find a logged query string.

Alert webhooks and the report `rproxy` posts to `waf` are unchanged: neither is a request line, and both are read only by a JSON parser.

Model: opus-5-5
2026-11-11 07:50:54 +01:00
clawbot 7b52574e5b rproxy: remove every X-Forwarded- header from an untrusted peer (closes #325)
check / check (push) Waiting to run
For a peer outside `SWRP_TRUSTED_PROXIES`, `rproxy` now removes every header whose name starts with `X-Forwarded-`, in any letter case, before it sets `X-Forwarded-For`, `X-Forwarded-Host` and `X-Forwarded-Proto`; `Forwarded` and `X-Real-IP` are removed as before. Until now only traefik's names were removed, so a client connecting directly could send `X-Forwarded-Ssl: on` or `X-Forwarded-Scheme: https` and make an app take a plain request for a secure one. A trusted peer's headers pass on unchanged. `SPEC.md` says so.

Model: opus-5-5
2026-11-11 07:50:50 +01:00
clawbot f76308add3 rproxy tests: give the 1 MiB answer a 60 s time limit (closes #335)
check / check (push) Waiting to run
Test only. `TestAnAnswerOf1MiBGoesToTheVisitor` read an answer of 1 MiB from `waf` within the default `SWRP_TIMEOUT_MS` of 2 s. On a busy host that time ran out first, `rproxy` applied `SWRP_DEFAULT_DISPOSITION` as it should, and the test failed. It now sets `SWRP_TIMEOUT_MS` to `60000`, as the file's other tests do where that limit is not what they test.

Model: opus-5-5
2026-11-11 07:31:27 +01:00
clawbot 937d956d3a waf: the header limits are the first check (closes #311)
check / check (push) Waiting to run
The header limits (more than 50 cookies or 100 header fields, answered `431`) move from just before the Core Rule Set to the first check after `SWW_ALLOW_NETS`. A request over them causes no lookup, no query to RouteWatch, a DNS blocklist or AbuseIPDB, no rule file match and no Core Rule Set. Requests within the limits go through the same checks in the same order.

For a request over the limits: the rate limits no longer count it; a banned client or one in `SWW_DENY_NETS` gets `431` instead of its refusal; it makes no ban, makes no ban permanent and is not counted in a ban's notes; in `observe` mode it is answered `allow` with no lookup. `SPEC.md` and `README.md` say so.

Model: opus-5-5
2026-11-11 07:13:35 +01:00
clawbot b89c53c640 sww tests: wait for the starting line as long as the start takes (closes #324)
check / check (push) Waiting to run
Tests only, in `internal/sww/sww_test.go`. A start compiles the Core Rule Set, so on a loaded host it can take longer than `waitLimit` (10 s) to write `starting`. The new `output.lineBeforeExit` waits until the line is written or sww exits, bounded only by the test binary's `-timeout`. It replaces the waits that span a start, in `runUntilStopped`, `TestServesUntilToldToStop` and `TestLogLevelHoldsBackTheLessSevereProcessLines`. The other waits on `waitLimit` in the file come after the start or after sww has exited.

Model: opus-5-5
2026-11-11 06:31:01 +01:00
clawbot 803a9e9dd8 rproxy tests: give the over-1 MiB answers a 60 s time limit (closes #323)
check / check (push) Waiting to run
Test only. The over-1 MiB cases of `TestAnAnswerThatCannotBePassedOnIsAnswered502`, and `TestAnAnswerThatCannotBePassedOnGetsNoReport`, which sends the same answer, left `SWRP_TIMEOUT_MS` at 2 s. On a loaded host `rproxy` applied `SWRP_DEFAULT_DISPOSITION` before it had read the whole answer, and the tests, which want `502`, failed. They now set `SWRP_TIMEOUT_MS` to `60000`, as the file's other tests do where that limit is not what they test.

Model: opus-5-5
2026-11-11 06:30:55 +01:00
clawbot 9bcbe5a3c8 Every time in UTC, and every day as YYYY-MM-DD (closes #313)
check / check (push) Waiting to run
Every time the program stores, logs or compares is in UTC, whatever TZ says, and AbuseIPDB's day in `reputation.json` is a `YYYY-MM-DD` date. Times read from a state file are converted to UTC as they are read, so a file is written back with `Z` times only. A test runs `sww waf` with `TZ=Pacific/Kiritimati` and checks the process line, the request line and `clients.json`. `SPEC.md` states the rule and `AGENTS.md` makes it an iron rule.

- The rate limit's `day` and `day_bytes` buckets keep a start time: each is a 24-hour bucket begun at an instant, not a calendar day.
- New dependency `cloud.google.com/go`, of which only `civil` is built.
- No migration: a `reputation.json` holding the old day form fails to load, naming the file.

Model: opus-5-5
2026-11-11 06:00:53 +01:00
clawbot f7e3190654 Rename discriminator to waf everywhere (closes #312)
check / check (push) Waiting to run
The command is now `sww waf`, and the package is `internal/waf`. Types,
functions, files, tests, comments, docs, the Dockerfile, the runit
script, `script/`, compose examples, the log field `waf_line` and the
`X-From-Waf` test header follow. No behaviour changes, and no setting
changes name: rproxy already finds the waf through `SWRP_WAF_BACKEND`.

Done with one `sed -i` (Discriminator and DISCRIMINATOR to WAF,
discriminator to waf) over the files `git grep -i discriminator`
listed, as sneak allowed for this rename, then `git mv` for the
package and files. By hand afterwards: the header's spelling, which
the linter requires, and the padding after `waf` in `sww`'s usage
text.

Model: opus-5-5
2026-11-11 02:43:16 +01:00
clawbot ad36dd1775 Discriminator tests: stop timing the sixteen-thousand-cookie requests (closes #308)
check / check (push) Successful in 3m55s
The two tests that send sixteen thousand cookies checked the answer time
on the wall clock, which a busy host can push past the bound. They no
longer check how long the answer took. The first is renamed to say what
it checks: a 431 before the Core Rule Set reads the request.

TestARefusedRequestDoesNotWaitForASlowGeoJS still checks that a refusal
over the header limits waits on nothing.

Model: opus-5-5
2026-11-11 01:09:22 +01:00
clawbot cb6b73a0be Core Rule Set: MediaWiki's password fields pass on a wiki in any language (closes #299)
check / check (push) Successful in 3m44s
A wiki in another language posts its sign-up, password change and
new-password forms to its own names for the special pages, such as
Spezial:Benutzerkonto_anlegen. So password and retype no longer match on
the English page name. They skip the rules when authAction names the
form that sends them: change, create or login-continue. MediaWiki sends
the same value in every language. The page must still be MediaWiki's,
with a namespace in its title, so another app's paths under /index.php/
keep every rule. The two rules move to phase 2 to read the body.

Adds requests recorded from a German MediaWiki 1.46.2.

Model: opus-5-5
2026-11-11 00:58:00 +01:00
clawbot 83f50dd11f Core Rule Set tests: only the concurrency test runs with the race detector (closes #306)
check / check (push) Successful in 3m36s
Of internal/coreruleset's tests, the race pass runs only those written to
check concurrent inspection, selected by the name prefix TestConcurrent.
Every other test of the package moves to the pass that runs the recorded
requests, without the race detector. Other packages run under -race as
before. The 90-second limit stays.

Model: opus-5-5
2026-11-11 00:43:18 +01:00
clawbot e2b27d1c7c rproxy: send its own answers at once to a visitor that stops sending its body (closes #303)
check / check (push) Successful in 6m26s
An answer rproxy gives itself, such as a deny or a refusal, waited while
Go's server read up to 256 KiB of a body that had not arrived whole. A
visitor that stopped sending got nothing until SWRP_MAX_REQUEST_TIME_MS
ran out, and no bytes at all once that was past the 10 seconds the
answer has to be sent.

send now sets Connection: close in that case, as modifyResponse does for
the app's answer; both call closeAfterAnswer. requestTimedOut no longer
ends that read of Go's server, which no longer happens, so the time the
app's answer started is no longer noted.

Model: opus-5-5
2026-11-10 23:57:52 +01:00
clawbot f64752e3b7 rproxy: answer a visitor that stops sending its body after the app has answered (closes #298)
check / check (push) Failing after 3m40s
When the app answered before the visitor's body had arrived whole, Go's
server read up to 256 KiB of the rest before it sent the answer. A
visitor that stopped sending got nothing until SWRP_MAX_REQUEST_TIME_MS
ran out, and no bytes at all once that was past the 10 seconds the
status has to be sent.

Such an answer now carries Connection: close, which Go sends at once.
finish first ends the transport's read of the body: once the handler
returns, Go's server reads the rest with no deadline while another read
of it waits. requestBody.Close does nothing, so that ReverseProxy's
close does not wait on the visitor.

Model: opus-5-5
2026-11-10 22:43:13 +01:00
clawbot abb8a26a48 Core Rule Set: random passwords pass on the multisite network profile page (closes #300)
check / check (push) Failing after 6m7s
/wp-admin/network/profile.php posts pass1 and pass2 to itself. The
exclusion that lets them skip the random-password rules on WordPress's
other profile pages now names it too.

Model: opus-5-5
2026-11-10 20:43:17 +01:00
clawbot aeaa9d1c77 rproxy: leave no visitor without an answer as the response time runs out (closes #295)
check / check (push) Failing after 4m37s
A visitor could get a closed connection and no answer when
SWRP_MAX_RESPONSE_TIME_MS ran out about when the answer started: the
write deadline passed before the first byte was written.

- An answer rproxy gives itself has at least 10 seconds to be sent.
- An app's answer that arrives once the time has run out is answered
  504.
- Otherwise the app's status and headers are sent at once, before the
  body is read, with at least 10 seconds. The exact limit applies to
  the rest.

Model: opus-5-5
2026-11-10 20:09:19 +01:00
clawbot db0912b335 Core Rule Set: random passwords pass on the remaining password fields (closes #288)
check / check (push) Failing after 2m44s
Each app's sign-in, sign-up, password change and reset fields that had no
exclusion now skip the 35 rules random passwords set off, on that app's
paths and only when the query has no field of that name. Added gitea,
Vaultwarden's admin token, WordPress pass1/pass2/post_password and its REST
password, Drupal's forms and JSON, MediaWiki retype and password on the
pages that take them, Mastodon's challenge, and Nextcloud's confirm, reset,
change, share, LDAP renewal, encryption and OCS fields. Vaultwarden's
clients send a hash, which needs no exclusion.

Model: opus-5-5
2026-11-10 19:57:58 +01:00
clawbot 8ab4280792 Test phase: run the two test passes at the same time (closes #292)
check / check (push) Successful in 4m40s
The race pass and the recorded-request pass are now separate stages, which
BuildKit runs at once. The test stage is built from scratch, so writing its
image takes no time.

Model: opus-5-5
2026-11-10 17:43:10 +01:00
clawbot 3268ec95c1 Core Rule Set: random passwords pass at sign-in (closes #276)
check / check (push) Successful in 4m37s
A password manager's random password set off 35 Core Rule Set rules,
about 1 in 40 refused. The password fields the app exclusions name now
skip those rules on their paths: WordPress pwd, Drupal pass, MediaWiki
wpPassword, lgpassword and password, Mastodon's sign-in, sign-up,
password change, move and delete passwords, and Nextcloud's sign-in
password. The tests draw 2,000 random passwords from a fixed seed, plus
one rare password for each rule the 2,000 do not set off. Those that set
off a rule in a field no exclusion names are sent to every password
field. `' OR '1'='1`, `../../etc/passwd`, `; cat /etc/passwd` and
`<img src=x onerror=x()>` sent as a password now pass;
`<script>alert(1)</script>` and `javascript:alert(1)` are still refused.

Model: opus-5-5
2026-11-10 15:57:59 +01:00
clawbot fb25f5ebd2 Core Rule Set tests: check WordPress's recorded requests as the other apps' (closes #284)
check / check (push) Successful in 5m4s
TestOrdinaryWordPressRequestsPass skipped the inspection that reads
bodies for a request with no body, including one with a Content-Type.
It now calls wantRecordedRequestsPass.

Model: opus-5-5
2026-11-10 15:57:53 +01:00
clawbot f8d769fc17 Core Rule Set tests: check the recorded requests without the race detector (closes #290)
check / check (push) Failing after 4m37s
The tests that check the apps' recorded requests move to
internal/coreruleset/recorded_test.go, behind the recorded build tag. The
test phase runs only those tests in a second go test, without -race. The
lint phase reads the same file, with the same tag.

The Gitea and WordPress tests are split in two. Their recorded-request loops
become TestOrdinaryGiteaRequestsPass and TestOrdinaryWordPressRequestsPass;
their attack checks stay in the race pass as TestGiteaAttacksAreRefused and
TestWordPressAttacksAreRefused.

Model: opus-5-5
2026-11-10 15:26:11 +01:00
clawbot 2ac7f1ccbf Core Rule Set: Nextcloud's calendar and contacts sync pass (closes #280)
check / check (push) Failing after 4m1s
rproxy passes MKCALENDAR on. nextcloud.conf allows it on
/remote.php/dav/calendars/ and PROPFIND on /.well-known/caldav and
carddav. The XML of an MKCOL there skips 930120, 932120, 932250 and
932260. That of an MKCALENDAR, and of an MKCOL on the address books'
paths, also skips 932125 and 932230, for the description. A PUT of
text/calendar or text/vcard there skips 920530: the Calendar and
Contacts apps name the charset twice when they change an event or a
contact.

Recorded from Nextcloud 35.0.2 with Calendar 6.6.5, Contacts 8.9.5 and
requests imitating DAVx5.

Model: opus-5-5
2026-11-10 11:57:58 +01:00
clawbot 463a19ef2a Core Rule Set: Nextcloud's ordinary requests pass by default (closes #240)
check / check (push) Successful in 5m6s
Adds nextcloud.conf, the fourteenth change. Nextcloud's WebDAV paths allow
PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, REPORT, SEARCH, LOCK and UNLOCK, and
the body of a PUT there, an uploaded file, is not read. The cookie
nc_username, and the fields that carry a search, a name, a file name or a
folder, skip the four rules that read them as commands or system files.
Comments, a share's note, a status and a document's text also skip the rules
that refused ordinary sentences, and the document the two that refused a
shell code block. Requests recorded from Nextcloud 35.0.2, its web pages, sync
client 3.16.7 and the OCS API pass at the defaults and while bodies are read.

Model: opus-5-5
2026-11-10 09:43:18 +01:00
clawbot c557b175ad Core Rule Set tests: inspect requests with no body and no Content-Type once (closes #281)
check / check (push) Failing after 4m25s
wantRecordedRequestsPass inspects a recorded request with neither a body
nor a Content-Type header only at the defaults, as the WordPress test does
for a request with no body. With neither, the set that reads bodies has
nothing more to check: what it adds picks how to read a body from the
Content-Type, then reads the body. Every other request is still inspected
both ways. This cuts the package's CPU time under -race by about a fifth.

Model: opus-5-5
2026-11-10 09:43:11 +01:00
clawbot 3b987b9074 rproxy: pass WebDAV methods on (closes #278)
check / check (push) Failing after 2m8s
rproxy now passes PROPFIND, PROPPATCH, MKCOL, COPY, MOVE, REPORT,
SEARCH, LOCK and UNLOCK to discriminator instead of answering 405, so
Nextcloud's Files app and sync client work. The Core Rule Set's 911100
still refuses them wherever no app's exclusions allow them.

Model: opus-5-5
2026-11-10 07:43:18 +01:00
clawbot 2d83961900 Core Rule Set: Vaultwarden's ordinary requests pass by default (closes #239)
check / check (push) Failing after 3m1s
Adds vaultwarden.conf, the thirteenth change, with rule ids 900150 to
900153. On Vaultwarden's account, organization and admin paths, names and
password hints skip 930120, 932120, 932250 and 932260, and so does a
security key's name. Email addresses skip 930120, 932120 and 932260
there, on the two-factor email paths and when signing in with another
device. So does the address of the admin page's test email. The data
breach report's address does too. The encrypted names of uploaded
attachments and file sends skip 920120. Requests recorded from
Vaultwarden 1.37.5 through the web vault, the Bitwarden command line
client and the admin page pass at the defaults and while bodies are read.

Model: opus-5-5
2026-11-10 07:26:13 +01:00
clawbot 183a6072d5 Core Rule Set: Mastodon's email field passes ordinary addresses (closes #273)
check / check (push) Failing after 16m29s
user[email] on /auth and the paths under it skips 930120, 932120 and
932260. They refused signing up, signing in and resetting a password
with addresses such as python3@example.com, powershell@example.com and
jane.profile@example.com. The field keeps 932250 and every other rule.

Recorded from Mastodon 4.7.5: sign-up, sign-in, password reset and an
email change.

Model: opus-5-5
2026-11-10 06:43:23 +01:00
clawbot 20e4b3b080 Core Rule Set: Mastodon's ordinary requests pass by default (closes #238)
check / check (push) Failing after 5m17s
Adds mastodon.conf, the twelfth change. On Mastodon's paths, the fields
that carry a search, a name, a title, an address, a password or free text
skip the four rules that read them as commands or system files. They cover
the API and the web app's JSON, the admin and settings pages, sign-up and
sign-in, and the objects other servers deliver to an inbox. A post's text
skips 932125, 932230 and 932235 too. Hashtags and sign-up usernames skip
only 932120 and 932260. Uploaded file names skip 932180, a report's
forward keys skip 930120 by name, and the Accept header of a link lookup
skips 920600 on profile and post links. The requests recorded from
Mastodon 4.7.5 pass at the defaults and while bodies are read.

Model: opus-5-5
2026-11-10 05:43:18 +01:00
clawbot 3d9e3976b0 Core Rule Set: gitea's search Referer exclusion skips /api/v2/search (closes #270)
check / check (push) Successful in 4m32s
Rule 900601 let Referer skip five rules when it named a repository's
search, issues or pull requests with the owner api, such as
/api/v2/search, Mastodon's search. A page opened from there skipped
script and SQL injection checks on Referer. gitea has no owner named
api, so a chained condition now leaves out those paths, as rule 900600
does.

Model: opus-5-5
2026-11-10 04:26:08 +01:00
clawbot 56d6a12cc2 Core Rule Set: gitea's repository search exclusion skips /api/v2/search (closes #266)
check / check (push) Successful in 7m49s
Rule 900600 took paths of an owner named api for gitea's search, issues,
pull requests and commit search inside a repository. /api/v2/search,
Mastodon's search, /api/v2/issues and /api/v2/pulls were among them, so
attacks in q scored 0 there. gitea has no owner named api, so a chained
condition now leaves out each of those paths under /api/<x>/. Go's regular
expressions have no lookahead, so the exclusion cannot be written inside
the pattern.

Model: opus-5-5
2026-11-10 03:26:14 +01:00
clawbot 922fe0cf71 Core Rule Set: MediaWiki's log, generator and watchlist fields pass (closes #258)
check / check (push) Failing after 16m32s
On /api.php, letitle and the generator fields gapto, gapprefix, gaifrom,
gaiprefix and geuquery skip 930120, 932120, 932250 and 932260, as the
fields they stand in for do.

On MediaWiki's pages, wpTitles from Special:EditWatchlist/raw, one title
per line, always skips 930120, 932120 and 932250. It skips 932125,
932230, 932235, 932260 and 932380 only when the body sends it once and it
holds no ";", "|", "&", backtick or "{", checked by a phase 2 rule. So
"; wget http://example.com/x.sh" is still refused.

Recorded from MediaWiki 1.46.2 at the digest the first MediaWiki change
used.

Model: opus-5-5
2026-11-10 03:09:24 +01:00
clawbot d70fb34f40 Core Rule Set: Drupal's JSON password reset takes an address (closes #262)
check / check (push) Failing after 5m8s
POST /user/password?_format=json with a mail of python3@example.com or
powershell@example.com was refused while bodies are read. On that path,
the JSON field mail now skips 932120 and 932260, the two rules the
recorded addresses set off, unless the query has a json.mail parameter.
Recorded from Drupal 11.4.11.

Model: opus-5-5
2026-11-10 03:09:17 +01:00
clawbot c1bb813a5e Core Rule Set tests: name repeated attack strings (closes #263)
check / check (push) Failing after 4m4s
The Drupal and MediaWiki tests repeated the SQL and script attack strings,
which goconst reports. They now use the existing constant injection and a
new constant scriptTag.

Model: opus-5-5
2026-11-10 01:26:07 +01:00
clawbot 9aa5b303cf Core Rule Set: MediaWiki's ordinary requests pass by default (closes #237)
check / check (push) Failing after 1m48s
Adds mediawiki.conf, the eleventh change: on MediaWiki's paths, the fields
that carry a search, a title, a name, a file name or an address skip the
four rules that read them as commands or system files, and the wikitext
fields skip the rules that refused saving a page. The reasons on the admin
pages skip only 930120 and 932120. /index.php counts as MediaWiki's only
with a page title in its query or after its path, since WordPress serves it
too. The requests recorded from MediaWiki 1.46.2, with and without short
URLs and on the admin pages, pass at the defaults and while bodies are
read. The app tests share the check of recorded requests and the list of
hard values.

Model: opus-5-5
2026-11-10 00:58:15 +01:00
clawbot e29dacad63 Core Rule Set: Drupal's JSON password reset passes (closes #259)
check / check (push) Successful in 4m2s
POST /user/password?_format=json with a name of python3, powershell,
ls -la or .htaccess was refused while bodies are read. On that path,
the JSON field name now skips 930120, 932120, 932250 and 932260, the
four rules those recorded names set off, unless the query has a
json.name parameter. Recorded from Drupal 11.4.11.

Model: opus-5-5
2026-11-10 00:58:08 +01:00
clawbot 9e7a7e60c7 Tests no longer connect to test servers they closed (closes #257)
check / check (push) Failing after 2m5s
Four tests closed a test server or stopped discriminator, and then
connected to its port. Another test process could take that port in
between and receive the connection.

- TestUnreachableRouteWatchLetsTheRequestThroughAndSaysWhy and
  TestUnreachableRoutewatchFailsTheQueryAndTheLogOmitsTheClient, formerly
  TestRefusedConnection..., point routewatch at 224.0.0.1, a multicast
  address to which TCP cannot connect.
- TestStalledRemoteLogEndpointHoldsUpNoRequest keeps its endpoint open
  and closes each connection it takes, instead of closing the endpoint.
- TestTheSiteAnswersWhileDiscriminatorIsStoppedAndStartedAgain gives
  rproxy an address the test holds throughout. The test forwards each
  connection there to the running discriminator, and closes it while
  discriminator is stopped.

No change to non-test code.

Model: opus-5-5
2026-11-10 00:58:01 +01:00