Commit Graph
38 Commits
Author SHA1 Message Date
sneak de78a12801 Give ban causes, limit kinds, alert events, actions and dispositions named types (closes #428)
check / check (push) Waiting to run
Each fixed set of string constants becomes a named string type in the
package that owns it, and the fields and parameters that carry it use
the type: bans.Cause, ratelimit.LimitKind, alerts.Event,
alerts.DestinationName, rules.Action, requestlog.Action,
requestlog.Offence, requestlog.WAFLineKind and wafprotocol.Disposition.
Every value keeps its string, so the logs, state files, alerts and
metrics do not change.

The switches exhaustive now checks list their missing cases.

Model: opus-5-5
2026-11-19 22:13:39 +00:00
clawbot 86bb33bf07 waf: list its own endpoints on a chi router in routes.go (closes #426)
check / check (push) Waiting to run
routes.go lists every own endpoint. One middleware checks a token and
another writes the log line. answerOwnEndpoint, answerAdmin and
endpoint() are gone; the handlers return http.HandlerFunc. The router
matches the decoded path, as the old code did. A test checks the
request_bytes of an own endpoint's log line.

Model: opus-5-5
2026-11-19 23:13:24 +01:00
clawbot 91444a37b3 metrics: build from one Params in place of seven Add calls (closes #438)
check / check (push) Waiting to run
metrics.New takes a Params holding every part at once, each behind a
small interface declared in metrics beside the metrics it reads. A part
that is off is left nil and adds no metric. The rule-match and
reputation-hit counters are made in New, so no metric field is nil and
no call order is needed. metrics no longer imports remotelog or rules.

Model: opus-5-5
2026-11-19 22:59:44 +01:00
clawbot e15edef583 Move what rproxy and waf send each other into wafprotocol (closes #425)
check / check (push) Waiting to run
internal/wafprotocol writes and reads the question, the answer and the
report, and declares once each header, path and limit both daemons use.
rproxy no longer imports internal/waf. Nothing on the wire changes.

Model: opus-5-5
2026-11-19 22:27:16 +01:00
clawbot 56a57125f1 waf: each heuristic holds its own settings and parts (closes #441)
check / check (push) Waiting to run
Every heuristic is a struct built once in newHandler with the settings
and collaborators it reads; no check reads the handler. The six lists
are handler fields built there. Code several heuristics share takes what
it needs as arguments or lives on a struct they hold: banMaker for bans
and their alerts, reputationHitNoter for reputation hits,
biasedThresholds for limit percentages. The handler keeps blocklists,
rateLimits, trapPaths, ruleFiles and coreRuleSet as fields too, since
refusedWhateverTheAnswer and findBesideLookup use them.

Model: opus-5-5
2026-11-19 21:43:55 +01:00
clawbot 9a8ed745bd Constructors take Params: ratelimit.New and processlog.New (closes #450)
check / check (push) Waiting to run
ratelimit.New takes ratelimit.Params (Limits, MaxClients) in place of
two arguments by position. The process logger moves out of requestlog
into internal/processlog, whose New takes Params (Output, InstanceName,
Level); its two tests move with it unchanged. geojs.GeoJS keeps its
params whole, as DNSBL, AbuseIPDB and routewatch.Client do. SPEC and
README list the new package.

Model: opus-5-5
2026-11-19 21:28:01 +01:00
clawbot 40386dce6e waf: wait as long as it takes for a test request's answer (closes #454)
check / check (push) Waiting to run
requestWithBody gave the answer 10 seconds, and endedLines gave the log
line 10 seconds. On a busy host the Core Rule Set's inspection of a
request with 1000 query parameters, under the race detector, took
longer. Both now wait until the answer or line comes; go test's
-timeout ends a test where it never does.

Model: opus-5-5
2026-11-19 20:46:01 +01:00
clawbot 433d5cc0a9 bans: one ledger method makes a request's ban, one place decides observe mode (closes #444)
check / check (push) Waiting to run
The ledger's BanForLimit, WouldBanForLimit, BanForAttack and
WouldBanForAttack become Ban and WouldBan, which take the cause and the
notes in one bans.Grounds. The waf's banForLimit and banForAttack fill in
what differs and call banClient, the one function that tests observe
mode on the way to a ban. Behaviour is unchanged; tests change only
where they called the four methods.

Model: opus-5-5
2026-11-19 20:29:53 +01:00
clawbot 98d7432077 waf: the steps after the answer as plain statements (closes #445)
check / check (push) Waiting to run
banAndCount runs the heuristics that only ban, then countAnomalies,
then addToHistory. A refused request's decision and an allowed
request's report both call it, each with its own list. answerQuestion
keeps one defer, which writes the log line and holds the decision for
the report.

A check that panics now leaves the request out of its client's history
and the anomaly counts, as the issue states.

Model: opus-5-5
2026-11-19 19:45:13 +01:00
clawbot 11ac8aee50 config: export the setting names waf logs (closes #448)
check / check (push) Waiting to run
waf wrote eight setting names as string literals into log lines and
ban notes, and config wrote each again where it reads the setting. A
rename that missed one would log a setting that does not exist. config
now exports a constant for each name, used by its reader and by waf.

Model: opus-5-5
2026-11-19 19:29:49 +01:00
clawbot 805c64a256 config: give the fixed-value settings named types (closes #427)
check / check (push) Successful in 4m46s
BytesCounted, LookupSource, BlocklistAction, DNSBLAbuseIPDBAction and
CoreRuleSetMode become named string types declared in config, with
constants for their values. Every comparison in waf and sww uses the
constants, and the private duplicates fileSource and deny go. The two
action settings share ReputationAction. The values and the strings
written to logs, metrics and alerts are unchanged.

Model: opus-5-5
2026-11-19 18:32:29 +01:00
clawbot 967ccdf75e sww: build the Core Rule Set from the settings (closes #434)
check / check (push) Failing after 4m0s
The body of waf.NewCoreRuleSet moves into newCoreRuleSet in
internal/sww, beside the providers of the other parts; waf no longer
exports it. The waf test helper builds the Core Rule Set with
coreruleset.New while the mode is not off.

Model: opus-5-5
2026-11-19 18:15:33 +01:00
clawbot 809d2664d7 waf: ban the clients on AbuseIPDB's blacklist (closes #411)
check / check (push) Successful in 8m16s
While SWW_WAF_ABUSEIPDB_KEY is set, waf fetches AbuseIPDB's blacklist at
the start and every SWW_WAF_ABUSEIPDB_BLACKLIST_REFRESH_INTERVAL_MINS
(60), cut at SWW_WAF_ABUSEIPDB_THRESHOLD (90, from 25 to 100). Each
listed client's netblock gets a ban of SWW_WAF_ABUSEIPDB_BAN_DAYS (30)
with the new cause abuseipdb. The blacklist itself is not kept.

These bans do not count toward SWW_WAF_MAX_BANS, so a 10,000-address
blacklist cannot push out the bans for limits and attacks. An ended
abuseipdb ban is forgotten at each fetch and each read of bans.json. A
netblock with an active ban is left alone, and so is one whose abuseipdb
ban an admin lifted, until that ban would have ended. Observe mode makes
no ban.

Model: opus-5-5
2026-11-19 12:42:41 +01:00
clawbot 01e6b5d47e sww: wire the daemons with fx (closes #414)
check / check (push) Successful in 3m54s
go.uber.org/fx builds the parts of waf and rproxy in internal/sww, one
provider per part. Each background loop is a lifecycle hook added where
its part is built. waf.Params takes every part built, and waf.New builds
none. GeoJS calls back into waf, so waf exports AddLookup and
GeoJSFailed, and clientGroup and banNetblock take the settings.

newServer, loadStateFiles, the inBackground list and serveRProxy are
gone. serve is the one hook that serves both daemons, and reports a
failure to run on a channel of its own; fx's signal handling is not
used. Once fx has stopped, run takes each daemon's last step: waf writes
the state files, and rproxy waits for its reports.

Model: opus-5-5
2026-11-19 11:58:20 +01:00
clawbot 9cb8f362bc sww: lift waf's check time limit and rproxy's wait in tests (closes #418)
check / check (push) Failing after 1m52s
Under heavy host load the sww tests got 503 where they expected 200 or
403: waf refused requests whose trap paths, rule files and Core Rule Set
took longer than 500 ms, and rproxy gave up on waf after 2 s.

waf.Params and sww.Params gain BesideLookupTimeLimit; zero keeps the
500 ms limit, so the binary is unchanged. The sww tests' run gives waf
an hour, and startRProxy sets SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS to
60000, as the rproxy package's tests do. waf's own tests set the limit
through the new field instead of SetBesideLookupTimeLimit, which is gone.

Model: opus-5-5
2026-11-19 10:44:53 +01:00
clawbot 8c9f20d346 waf: run the trap paths, rule files and Core Rule Set during the GeoJS wait (closes #405)
check / check (push) Failing after 1m51s
While the lookup waits for GeoJS, the trap paths, the rule files and the
Core Rule Set run at the same time as the checks after the lookup. The
first side to refuse answers the request; what the other side finds leaves
no trace. Those three checks have 500 ms together: past it, the request is
refused with 503 as checks_timed_out, with no ban and no offence.

Tests hold GeoJS and the Core Rule Set with stand-ins under synctest. The
test helper lifts the 500 ms limit for tests that run the real Core Rule
Set, which the race detector slows past it.

Model: opus-5-5
2026-11-17 06:11:52 +01:00
clawbot caa3079847 geojs: a visitor's wait no longer cuts off the request to GeoJS (closes #404)
check / check (push) Failing after 2m16s
The GeoJS lookup and its cache move from internal/lookup to internal/geojs,
which imports no other package of this repository. waf hands it functions
to count requests, report failures and count lookups that went without an
answer; waf keeps the log line, the source_failure alert and the metrics.

The caller now passes its wait to each lookup. waf passes
SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS. A request to GeoJS has its own
fixed limit of 10 seconds, and only passing it or an error from GeoJS is a
failure. An answer that comes after the caller stopped waiting is kept.

internal/lookup keeps the lookup database and returns geojs.Answer.

Model: opus-5-5
2026-11-17 00:42:58 +01:00
clawbot f33d0a4fce Remove every reference to our own servers (closes #396)
check / check (push) Successful in 4m48s
script/publish no longer defaults DOCKER_HOST to an SSH target on one of
our servers or PUBLISH_IMAGE to the registry. Both are required, and an
unset one stops it with a message naming it. The README says so.

The instance name in docs and tests was our server's name; it is now
host1/waf in the docs and host1/blog in the tests.

Model: opus-5-5
2026-11-16 07:29:16 +01:00
clawbot 0a2b2d809f waf: decide every visitor request within rproxy's timeout (closes #394)
check / check (push) Failing after 2m17s
The Core Rule Set is the one check whose time a visitor can stretch.
At paranoia level 4 it took over 2 s on a request with an 8 KiB
request line, three 8 KiB Cookie lines and a 64 KiB form body.
SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE now takes at most 8K. The
Core Rule Set also reads no JSON body over 1 KiB, since Coraza turns
JSON into up to 8 times its size in names and values. That brings
such a request under 1 s.

A test per kind of input times the Core Rule Set on the worst request
within the limits, at the slowest settings. SPEC.md lists the limits
that bound it.

Model: opus-5-5
2026-11-15 20:11:42 +01:00
clawbot 53d9bf6af6 waf: refuse request lines and header lines over 8 KiB first (closes #339)
check / check (push) Failing after 4m24s
The header limits now also refuse a request line over 8 KiB with 414
(action request_line_too_long) and a header line over 8 KiB, Host
included, with 431. Like the other header limits they are fixed, run
first and make no ban. One long query string or cookie no longer keeps
the Core Rule Set busy for a third of a second.

Model: opus-5-5
2026-11-13 21:56:09 +01:00
clawbot 3550956cbb rproxy passes the app the client's AS number and country by default (closes #379)
check / check (push) Successful in 4m19s
SWW_WAF_ADD_LOOKUP_HEADERS now defaults to true, so the app gets
X-Client-ASN and X-Client-Country with no setting. While
SWW_WAF_LOOKUP_SOURCE is off it defaults to false, so turning lookups
off alone still starts; true set with the lookup off still stops the
start. With GeoJS, a new client's first request now waits for its
answer, up to SWW_WAF_ASN_AND_COUNTRY_LOOKUP_TIMEOUT_MS.

Model: opus-5-5
2026-11-13 21:27:07 +01:00
clawbot 2bea6516a8 Remove the CrowdSec decision feed (closes #315)
check / check (push) Successful in 4m50s
sneak ruled to remove it. Gone: the heuristic crowdsec, the ban cause
crowdsec and its earlier_bans count, the tag crowdsec, the settings
SWW_WAF_CROWDSEC_LAPI_URL and SWW_WAF_CROWDSEC_LAPI_KEY, the decision
list fetch and parse in internal/reputation, their tests, and every
mention in the README, SPEC.md and docs/. EVALUATION.md keeps its
comparison of tools.

The ban ledger's internal ban function loses its expires parameter,
which only CrowdSec bans used.

Model: opus-5-5
Co-authored-by: clawbot <sneak+clawbot@sneak.cloud>
2026-11-13 20:43:05 +01:00
clawbot 4dad4fff43 Rename the settings whose names do not say what they control (closes #234)
check / check (push) Failing after 2m3s
Applies the setting renames of the naming audit under the SWW_WAF_ and
SWW_RPROXY_ prefixes, with sneak's names for the IP reputation cache
time and the two time limits, which now take milliseconds. The rproxy
log field default_disposition becomes waf_backend_no_answer, and
config.Config and config.FromEnvironment become config.WAF and
config.WAFFromEnvironment. Go fields, test names and prose follow. No
aliases; nothing reads the old names.

Model: opus-5-5
2026-11-12 11:13:27 +01:00
clawbot 0cb6dded98 Read every setting through viper and rename the settings (closes #211)
check / check (push) Failing after 1m55s
Every setting of waf and rproxy is read through viper, in place of the
os.LookupEnv function Main passed down. NewViper turns AutomaticEnv and
AllowEmptyEnv on, so a setting that is set but empty reads as set. A
setting may also be given as a file, the _FILE form, which rproxy's
settings now have too; rproxy still writes nothing to disk.

The settings are renamed: waf's take the prefix SWW_WAF_, rproxy's take
SWW_RPROXY_, and the shared SWW_COMMAND keeps SWW_. The Core Rule Set's
settings take SWW_WAF_CORE_RULE_SET_ rather than SWW_WAF_WAF_.

Model: deepseek-v4-pro (first version); opus-5-5 (rework)
2026-11-12 05:43:47 +01:00
clawbot 4b62bc84ce SWW_APPS: an app's Core Rule Set fixes apply only to the hostnames that serve it (closes #220)
check / check (push) Failing after 2m3s
Each app's fixes are one .conf file and one entry in apps. They apply
only while Inspect has set the app's transaction variable, such as
tx.app_gitea. SWW_APPS names each hostname's apps. A hostname gets only
its apps' fixes; while SWW_APPS is unset, every hostname gets every
app's fixes. Listing forgejo, a fork of gitea, brings gitea's fixes too.

A hostname in SWW_APPS that is not a DNS name, such as *.example.com,
stops the start. Unset, SWW_APPS is logged at start as null; set
empty, as "".

Model: opus-5-5
2026-11-12 03:15:16 +01:00
clawbot 0b23352767 Format Go with gofmt, then golines -m 77 (closes #215)
check / check (push) Failing after 3m31s
script/fmt now runs golines -m 77 after gofmt, repeating golines until
it changes nothing, since one pass can leave a line the next splits.
script/fmt-check fails when either would change a file.
script/bootstrap installs golines v0.13.0 with go install, pinned to
its commit, and checks the installed version with go version -m.

The tree is reformatted. The longer lines this makes are fixed by
hand: nolint directives moved onto their own line above the statement,
one string split, and 20 functions over funlen's 80 lines shortened or
split, RunWAF and limitPercentages among them.

Model: opus-5-5
2026-11-11 17:29:12 +01:00
clawbot b93bad5842 Answer-time tests check the answer, not wall-clock milliseconds (closes #316)
check / check (push) Canceled after 0s
Tests only. `TestWAFAnswersWithinTheTimeLimitWhileEveryOutsideSourceHangs` and `TestHangingRoutewatchNeverDelaysAndFailsTheQuery` run in a `synctest` bubble, whose clock load cannot move, instead of comparing wall-clock durations with fixed bounds. The first checks the answer comes after exactly `SWW_LOOKUP_TIMEOUT`, and that the DNS blocklist's query, held by a stand-in resolver, had not failed when it came. The second requires `no_answer_yet` after exactly zero time. The other wall-clock checks in the tests were already in bubbles or are bounds load cannot break.

- The held DNS query still times out on the real clock after two seconds; a stall that long between question and check fails the test.

Model: opus-5-5
2026-11-11 10:56:51 +01:00
clawbot 83525b5e73 Caches of outside APIs live in /var/cache/sww (closes #314)
check / check (push) Successful in 5m32s
Every cache of answers from an outside service has its own file in `SWW_CACHE_DIR` (default `/var/cache/sww`): `lookups.json` (GeoJS), `lists.json` (lists fetched by URL, CrowdSec's decisions included), `dnsbl.json`, `abuseipdb-scores.json` and `routewatch.json` (at most 100,000 answers, each kept an hour). `reputation.json` is gone; the AbuseIPDB checks spent today stay in `SWW_STATE_DIR` as `abuseipdb-checks-spent.json`, since losing them would spend the daily budget twice.

A cache file never stops the start: one that does not parse is set aside as `<name>.bad` with a `file_error` alert. An unwritable `SWW_CACHE_DIR` stops the start. The `run` script creates the cache directory and gives it to `sww`. The cache directory needs no volume and can be left out of backups.

Model: opus-5-5
2026-11-11 10:56:43 +01:00
clawbot f077dd7fd0 Say that rproxy passes a refused question's 400 to the visitor (closes #326)
check / check (push) Successful in 4m25s
The comment above `readQuestion` in `internal/waf/waf.go` said `rproxy` treats a refused question (`400`, no disposition) as no answer. It passes the `400` to the visitor, and `SWRP_DEFAULT_DISPOSITION` does not apply, as `SPEC.md` and `internal/rproxy/waf.go` say. Comment only.

Model: opus-5-5
2026-11-11 07:50:59 +01:00
clawbot 937d956d3a waf: the header limits are the first check (closes #311)
check / check (push) Failing after 2m8s
The header limits (more than 50 cookies or 100 header fields, answered `431`) move from just before the Core Rule Set to the first check after `SWW_ALLOW_NETS`. A request over them causes no lookup, no query to RouteWatch, a DNS blocklist or AbuseIPDB, no rule file match and no Core Rule Set. Requests within the limits go through the same checks in the same order.

For a request over the limits: the rate limits no longer count it; a banned client or one in `SWW_DENY_NETS` gets `431` instead of its refusal; it makes no ban, makes no ban permanent and is not counted in a ban's notes; in `observe` mode it is answered `allow` with no lookup. `SPEC.md` and `README.md` say so.

Model: opus-5-5
2026-11-11 07:13:35 +01:00
clawbot f7e3190654 Rename discriminator to waf everywhere (closes #312)
check / check (push) Successful in 5m58s
The command is now `sww waf`, and the package is `internal/waf`. Types,
functions, files, tests, comments, docs, the Dockerfile, the runit
script, `script/`, compose examples, the log field `waf_line` and the
`X-From-Waf` test header follow. No behaviour changes, and no setting
changes name: rproxy already finds the waf through `SWRP_WAF_BACKEND`.

Done with one `sed -i` (Discriminator and DISCRIMINATOR to WAF,
discriminator to waf) over the files `git grep -i discriminator`
listed, as sneak allowed for this rename, then `git mv` for the
package and files. By hand afterwards: the header's spelling, which
the linter requires, and the padding after `waf` in `sww`'s usage
text.

Model: opus-5-5
2026-11-11 02:43:16 +01:00
clawbot 719691d1e8 Name log fields, metrics and the Core Rule Set package for what they are (closes #192)
check / check (push) Failing after 3m22s
The first half of the naming audit: names now say what they are. `internal/waf` is `internal/coreruleset`. Log fields, the action `blocked_by_core_rule_set`, the metric and the alert event say Core Rule Set. Timings end `_ms`. Percentages say of what, as `rate_limit_percent` and `bytes_limit_percent`. Setting names follow in #234.

Judgement call: `blocked_by_core_rule_set`, not the audit's `core_rule_set_blocked`, to match `blocked_by_block_rule`.
Trips a reader: an existing state file holding `waf_blocked` or the anomaly scope `watch` stops the start and must be removed. `SWW_ALERT_EVENTS` listing `waf_block` must list `core_rule_set_match`.

Model: opus-5-5
2026-11-09 08:47:26 +01:00
clawbot beb503b233 Rename the program to sww and its deciding daemon to discriminator (closes #218) (#221)
check / check (push) Failing after 1m34s
2026-11-07 06:51:06 +01:00
clawbot 2e2f6f7175 Core Rule Set: gitea's ordinary requests pass by default (#208)
check / check (push) Successful in 3m29s
2026-11-07 05:21:22 +01:00
clawbot 344ce13c96 rproxy and servewaf replace the one-process smallwebwaf (closes #172, closes #165)
check / check (push) Failing after 14m44s
`smallwebwaf` with no command now prints the commands and exits with status 2.
The image runs rproxy, or servewaf with SMALLWEBWAF_COMMAND=servewaf. The
passing-on code, its settings and its log fields and metric go. internal/proxy
keeps the checks, answers, reports and own endpoints; its tests run through the
real rproxy. The documents describe the pair.

Judgement call: clients.json keeps token_refused, since state files refuse unknown fields.
Deviation: script/gitea-requests sets SWWAF_LOG_REQUEST_HEADERS, to label requests.
Partially verified: make example-app and make gitea-requests never built the image, as Ubuntu's snapshot service answered 5xx.

Model: opus-5-5
2026-11-05 13:15:05 +01:00
clawbot f408f42b28 SPEC, README and docs rewritten from scratch (closes #154, closes #155, closes #156)
check / check (push) Canceled after 0s
SPEC.md is the only place that says how smallwebwaf behaves and why; docs/ holds lists linking to it; the README is short and links to both. Passed independent review in round 8.

Model: fable-5-1 (writing); opus-5-5 (review, merge)
2026-10-26 03:06:28 +01:00
clawbot 6fcbda6ece Core Rule Set reads request bodies up to SWWAF_WAF_BODY_LIMIT (closes #116)
check / check (push) Canceled after 0s
SWWAF_WAF_BODY_LIMIT (default off, at most 1G) has the Core Rule Set read
form data and multipart up to the limit, the rest streaming on, and JSON
and XML no larger than it, with text/json and the application and text
types ending in +json or +xml. The part read is held for the app. A size
or time limit met while reading ends the request. Content-Encoding is
refused again on these kinds. A body Coraza cannot parse, or a multipart
body failing its strict checks, adds 5, as does a multipart body the limit
cuts in a part's headers before a colon or a line feed. Coraza is built
with no_fs_access, so writes no file. Rule 900300 moves to phase 2.

Judgement call: Content-Encoding is refused on a JSON or XML body too
large to read, as SPEC.md allows.

Model: opus-5-5
2026-10-08 12:59:13 +02:00
clawbot 80f4c2cc61 The Core Rule Set, run by Coraza, on each request's method, URL and headers (closes #25)
check / check (push) Canceled after 0s
Coraza v3.8.1 runs the Core Rule Set 4.25.0 (coraza-coreruleset v4.25.0)
after the rule files, with the six changes and the default
SWWAF_WAF_DISABLED_RULES that SPEC.md gives; no body, no response. The
parameter names in the third and fourth changes are matched in any case,
as Coraza does. SWWAF_WAF_DISABLED_RULES refuses 900000 to 900999,
smallwebwaf's own rules. A request with more than 1000 query parameters
adds 5 (rule 900300). In block mode a match is refused with 403, an
offence counted toward the error burst; in detect mode it is let
through. Both log waf_rule_ids, waf_score and duration_waf, raise
waf_block, and count smallwebwaf_waf_matches_total.

Judgement call: waf_block is raised in block mode too.
Deviation: no engine-error path; with no body read, Coraza cannot fail.

Model: opus-5-5
2026-10-08 09:37:13 +02:00