Compare commits

..
Author SHA1 Message Date
clawbot eca8f4d8cc waf: the server's types and constructor in waf.go, active_ban in its own file (closes #435)
check / check (push) Failing after 3m45s
Moves only. waf.go now holds the package comment, Params, Server, New,
newHandler, handler and ServeHTTP. The code that answers questions and
takes reports moves to questionsandreports.go, and activeBan to
activeban.go. refusedWhateverTheAnswer moves to lookup.go, beside its
only caller. proxy.go is gone. The only changed lines are imports and
the file comment of bans.go, split between it and activeban.go.

Model: opus-5-5
2026-11-20 01:54:31 +00:00
clawbot 4774518cb3 waf: a verdict carries the status the visitor gets (closes #443)
check / check (push) Failing after 1m42s
Each heuristic that refuses a request sets the status on its verdict.
check copies it into the refusal, or in observe mode notes it, and no
longer works the status out from the action. refusalLimit takes the
status from that verdict instead of from a switch over the action.

Model: opus-5-5
2026-11-20 02:42:08 +01:00
clawbot 5d5e9542f8 coreruleset: embed the apps' fixes as one directory (closes #430)
check / check (push) Failing after 3m38s
The nine app files move to internal/coreruleset/apps/ and are embedded
once as appFixes. An app's name is its file's name; the per-app
variables go, and apps keeps only gitea's and Forgejo's cookies and
forkOf. Comments and SPEC name each of sww's changes to the Core Rule
Set by what it does instead of by number. The directives Coraza gets
are unchanged.

Model: opus-5-5
2026-11-20 02:13:14 +01:00
clawbot 1b9ca94774 Hold Go's server privately in waf.Server and rproxy.Server (closes #437)
check / check (push) Failing after 1m46s
Neither Server embeds *http.Server any more. Each holds it as a private
field, is its handler, and has Serve, Shutdown, Close and ServeHTTP.
sww's serve hook takes the new daemonServer interface of those first
three, and fx provides each Server as it.

Model: opus-5-5
2026-11-20 01:58:25 +01:00
clawbot 9afbd1b683 config: each waf setting's code in one place (closes #449)
check / check (push) Failing after 3m59s
internal/config/waf.go is reordered by setting, in the order of the
settings sections of SPEC and docs/settings.md. The struct WAF and
WAFFromEnvironment stay first. Each setting's reader, parser, errors
and check sit together. A helper several settings share sits before
the first of them.

Moves only. The one error block and the one block of constants are
split into a block beside each user, so gofmt re-aligns the = of some
of those lines.

Model: opus-5-5
2026-11-20 01:44:08 +01:00
34 changed files with 2432 additions and 2260 deletions

No files matched your search

+31 -30
View File
@@ -828,8 +828,8 @@ The supported apps are:
- `vaultwarden`; - `vaultwarden`;
- `wordpress`. - `wordpress`.
Each app's fixes are in one file, such as `internal/coreruleset/gitea.conf`, and Each app's fixes are in one file named for the app, such as
one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are: `internal/coreruleset/apps/gitea.conf`. gitea's fixes are:
1. Ten parameters are not checked against the Core Rule Set's lists of system 1. Ten parameters are not checked against the Core Rule Set's lists of system
files (930120), shell paths (932160) and command names (932260), on any path: files (930120), shell paths (932160) and command names (932260), on any path:
@@ -862,8 +862,8 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
3. On gitea's paths, some fields skip the rules that refused ordinary gitea 3. On gitea's paths, some fields skip the rules that refused ordinary gitea
requests there. They are the fields where gitea sends a search, a file's name requests there. They are the fields where gitea sends a search, a file's name
or text, a commit message, a branch name or a repository's name. or text, a commit message, a branch name or a repository's name.
`internal/coreruleset/gitea.conf` lists each rule and the request it refused. `internal/coreruleset/apps/gitea.conf` lists each rule and the request it
The fields are: refused. The fields are:
- `q` on gitea's searches of code, of issues and pull requests, and of - `q` on gitea's searches of code, of issues and pull requests, and of
commits. A code search holds code, such as `<script>`, commits. A code search holds code, such as `<script>`,
`SELECT * FROM users WHERE` or `../`. The searches of issues and pull `SELECT * FROM users WHERE` or `../`. The searches of issues and pull
@@ -1048,7 +1048,7 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
- The `Content-Range` header of an artifact upload from - The `Content-Range` header of an artifact upload from
`actions/upload-artifact@v3` (920450). `actions/upload-artifact@v3` (920450).
- The passwords in a body, for each rule that random passwords set off. The - The passwords in a body, for each rule that random passwords set off. The
numbers are in `internal/coreruleset/gitea.conf`. numbers are in `internal/coreruleset/apps/gitea.conf`.
- `password` on these paths: - `password` on these paths:
- `/user/login`, `/user/sign_up`, `/user/activate`; - `/user/login`, `/user/sign_up`, `/user/activate`;
- `/user/link_account_signin` and `/user/link_account_signup`; - `/user/link_account_signin` and `/user/link_account_signup`;
@@ -1080,8 +1080,8 @@ one entry of `apps` in `internal/coreruleset/coreruleset.go`. gitea's fixes are:
such as `owner/python3-app` would have that push refused. such as `owner/python3-app` would have that push refused.
WordPress's fixes: on WordPress's paths, some fields skip the rules that refused WordPress's fixes: on WordPress's paths, some fields skip the rules that refused
ordinary WordPress requests there. `internal/coreruleset/wordpress.conf` lists ordinary WordPress requests there. `internal/coreruleset/apps/wordpress.conf`
each rule and the request it refused. The REST API has four addresses: lists each rule and the request it refused. The REST API has four addresses:
- `/wp-json/...` - `/wp-json/...`
- `/index.php/wp-json/...` - `/index.php/wp-json/...`
@@ -1173,9 +1173,9 @@ Elsewhere WordPress's fixes do not apply. The paths are those of WordPress at
the site root. the site root.
Drupal's fixes: on Drupal's paths, some fields skip the rules that refused Drupal's fixes: on Drupal's paths, some fields skip the rules that refused
ordinary Drupal requests there. `internal/coreruleset/drupal.conf` lists each ordinary Drupal requests there. `internal/coreruleset/apps/drupal.conf` lists
rule and the request it refused. These query fields skip the lists of system each rule and the request it refused. These query fields skip the lists of
files and command names (930120, 932120, 932250, 932260): system files and command names (930120, 932120, 932250, 932260):
- On every page under `/admin`: `title`, `name`, `user`, `filename`, `filemime`, - On every page under `/admin`: `title`, `name`, `user`, `filename`, `filemime`,
`info`, `subject`, `author_name`, `search`, `q` and `destination`. `info`, `subject`, `author_name`, `search`, `q` and `destination`.
@@ -1265,10 +1265,10 @@ first path segment of two or three lowercase letters, optionally followed by a
hyphen and more letters, so `/api` counts too. hyphen and more letters, so `/api` counts too.
MediaWiki's fixes: on MediaWiki's paths, some fields skip the rules that refused MediaWiki's fixes: on MediaWiki's paths, some fields skip the rules that refused
ordinary MediaWiki requests there. `internal/coreruleset/mediawiki.conf` lists ordinary MediaWiki requests there. `internal/coreruleset/apps/mediawiki.conf`
each rule and the request it refused. These fields skip the lists of system lists each rule and the request it refused. These fields skip the lists of
files and command names (930120, 932120, 932250, 932260), in the query or the system files and command names (930120, 932120, 932250, 932260), in the query or
body: the body:
- On MediaWiki's pages, which are `/index.php` and the paths under `/index.php/` - On MediaWiki's pages, which are `/index.php` and the paths under `/index.php/`
and `/wiki/`: and `/wiki/`:
@@ -1374,9 +1374,9 @@ Elsewhere these fields keep every rule. The paths are those of MediaWiki at the
site root or under `/w/`. site root or under `/w/`.
Mastodon's fixes: on Mastodon's paths, some fields skip the rules that refused Mastodon's fixes: on Mastodon's paths, some fields skip the rules that refused
ordinary Mastodon requests there. `internal/coreruleset/mastodon.conf` lists ordinary Mastodon requests there. `internal/coreruleset/apps/mastodon.conf`
each rule and the request it refused. These fields skip the lists of system lists each rule and the request it refused. These fields skip the lists of
files and command names (930120, 932120, 932250, 932260), in the query: system files and command names (930120, 932120, 932250, 932260), in the query:
- On the API's search, accounts and admin accounts, under `/api/v1/` and - On the API's search, accounts and admin accounts, under `/api/v1/` and
`/api/v2/`, the searches and names `q`, `acct`, `username`, `display_name`, `/api/v2/`, the searches and names `q`, `acct`, `username`, `display_name`,
@@ -1537,7 +1537,7 @@ every rule.
Vaultwarden's fixes: on Vaultwarden's paths, some fields skip the rules that Vaultwarden's fixes: on Vaultwarden's paths, some fields skip the rules that
refused ordinary Vaultwarden requests there. refused ordinary Vaultwarden requests there.
`internal/coreruleset/vaultwarden.conf` lists each rule and the request it `internal/coreruleset/apps/vaultwarden.conf` lists each rule and the request it
refused. The apps encrypt vault items, folders, collections and sends before refused. The apps encrypt vault items, folders, collections and sends before
sending them, so those fields skip no rules. The fields below hold plain text. sending them, so those fields skip no rules. The fields below hold plain text.
@@ -1588,8 +1588,8 @@ has a parameter of the same name.
`token` in the body of `/admin`, the admin token typed on the admin page, skips `token` in the body of `/admin`, the admin token typed on the admin page, skips
each rule that random passwords set off. The numbers are in each rule that random passwords set off. The numbers are in
`internal/coreruleset/vaultwarden.conf`. It applies only when the query has no `internal/coreruleset/apps/vaultwarden.conf`. It applies only when the query has
`token` parameter. `' OR '1'='1`, `../../etc/passwd`, `; cat /etc/passwd` and no `token` parameter. `' OR '1'='1`, `../../etc/passwd`, `; cat /etc/passwd` and
`<img src=x onerror=x()>` sent as the token reach Vaultwarden; `<img src=x onerror=x()>` sent as the token reach Vaultwarden;
`<script>alert(1)</script>` and `javascript:alert(1)` are still refused. `<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
@@ -1611,7 +1611,7 @@ Elsewhere these fields keep every rule.
Nextcloud's fixes: on Nextcloud's paths, some fields skip the rules that refused Nextcloud's fixes: on Nextcloud's paths, some fields skip the rules that refused
ordinary Nextcloud requests there, and its WebDAV methods are allowed. ordinary Nextcloud requests there, and its WebDAV methods are allowed.
`internal/coreruleset/nextcloud.conf` lists each rule and the request it `internal/coreruleset/apps/nextcloud.conf` lists each rule and the request it
refused. refused.
The sync client and the Files app use WebDAV on these paths: The sync client and the Files app use WebDAV on these paths:
@@ -1621,7 +1621,8 @@ The sync client and the Files app use WebDAV on these paths:
- `/remote.php/webdav/`; - `/remote.php/webdav/`;
- `/public.php/webdav/`. - `/public.php/webdav/`.
These methods are allowed there, besides those of the first change: These methods are allowed there, besides `PUT`, `PATCH` and `DELETE`, which
every app gets:
- `PROPFIND` and `PROPPATCH`; - `PROPFIND` and `PROPPATCH`;
- `MKCOL`; - `MKCOL`;
@@ -1765,7 +1766,7 @@ gets gitea's fixes too. Forgejo's own fixes hold only what Forgejo sends and
gitea does not. gitea does not.
On Forgejo's paths, some fields skip the rules that refused ordinary Forgejo On Forgejo's paths, some fields skip the rules that refused ordinary Forgejo
requests there. `internal/coreruleset/forgejo.conf` lists each rule and the requests there. `internal/coreruleset/apps/forgejo.conf` lists each rule and the
request it refused. request it refused.
`q` skips rules on some of Forgejo's searches. The searches are: `q` skips rules on some of Forgejo's searches. The searches are:
@@ -1811,10 +1812,10 @@ The Core Rule Set reads the request without the `flash` cookie, in which Forgejo
puts what gitea puts in `gitea_flash` (gitea's second fix). puts what gitea puts in `gitea_flash` (gitea's second fix).
Ghost's fixes: on Ghost's paths, some fields skip the rules that refused Ghost's fixes: on Ghost's paths, some fields skip the rules that refused
ordinary Ghost requests there. `internal/coreruleset/ghost.conf` lists each rule ordinary Ghost requests there. `internal/coreruleset/apps/ghost.conf` lists each
and the request it refused. Ghost's admin and integrations call the Admin API rule and the request it refused. Ghost's admin and integrations call the Admin
under `/ghost/api/admin/`. Themes and Ghost's scripts call the Content API under API under `/ghost/api/admin/`. Themes and Ghost's scripts call the Content API
`/ghost/api/content/`, and members call `/members/api/`. The admin calls under `/ghost/api/content/`, and members call `/members/api/`. The admin calls
ActivityPub, a service of its own, under `/.ghost/activitypub/`. ActivityPub, a service of its own, under `/.ghost/activitypub/`.
Eight rules refused words that name commands and files, such as `python3`, Eight rules refused words that name commands and files, such as `python3`,
@@ -1952,9 +1953,9 @@ That name holds `.profile`, a file on the list of system files, so the name
skips 930120 when a profile is saved. skips 930120 when a profile is saved.
The passwords skip each rule that random passwords set off. The numbers are in The passwords skip each rule that random passwords set off. The numbers are in
`internal/coreruleset/ghost.conf`. They are `password` when signing in, and the `internal/coreruleset/apps/ghost.conf`. They are `password` when signing in, and
passwords sent when creating the first staff user, accepting an invitation, and the passwords sent when creating the first staff user, accepting an invitation,
resetting or changing a password. `' OR '1'='1`, `../../etc/passwd`, and resetting or changing a password. `' OR '1'='1`, `../../etc/passwd`,
`; cat /etc/passwd` and `<img src=x onerror=x()>` sent as a password reach `; cat /etc/passwd` and `<img src=x onerror=x()>` sent as a password reach
Ghost; `<script>alert(1)</script>` and `javascript:alert(1)` are still refused. Ghost; `<script>alert(1)</script>` and `javascript:alert(1)` are still refused.
+1621 -1444
View File
File diff suppressed because it is too large. Load diff
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -37,9 +37,10 @@
SecRule &TX:app_nextcloud "@eq 0" \ SecRule &TX:app_nextcloud "@eq 0" \
"id:900090,phase:1,pass,nolog,skipAfter:END_NEXTCLOUD" "id:900090,phase:1,pass,nolog,skipAfter:END_NEXTCLOUD"
# The sync client and the Files app use these WebDAV methods besides those # The sync client and the Files app use these WebDAV methods besides PUT,
# the first change allows, on /remote.php/dav/ and /public.php/dav/ and on the # PATCH and DELETE, which sww allows for every app, on /remote.php/dav/ and
# older /remote.php/webdav/ and /public.php/webdav/. 911100 refused each: # /public.php/dav/ and on the older /remote.php/webdav/ and /public.php/webdav/.
# 911100 refused each:
# PROPFIND, PROPPATCH: listing a folder, marking a file a favourite # PROPFIND, PROPPATCH: listing a folder, marking a file a favourite
# MKCOL: making a folder, and starting a chunked upload # MKCOL: making a folder, and starting a chunked upload
# COPY, MOVE: copying, moving and renaming, and ending a chunked upload # COPY, MOVE: copying, moving and renaming, and ending a chunked upload
File renamed without changes.
+40 -62
View File
@@ -12,11 +12,10 @@
package coreruleset package coreruleset
import ( import (
_ "embed" "embed"
"errors" "errors"
"fmt" "fmt"
"io" "io"
"maps"
"net" "net"
"net/http" "net/http"
"net/netip" "net/netip"
@@ -35,7 +34,7 @@ import (
// directives are the Core Rule Set as sww runs it, with the // directives are the Core Rule Set as sww runs it, with the
// paranoia level for %d, bodyDirectives for the first %s while // paranoia level for %d, bodyDirectives for the first %s while
// SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set, the exclusions file for // SWW_WAF_CORE_RULE_SET_BODY_INSPECTION_SIZE is set, the exclusions file for
// the second, and the directives of every app in apps for the third. // the second, and the fixes of every app in appFixes for the third.
// Each rule sww adds has an id from 900000 to 900999, the ids the // Each rule sww adds has an id from 900000 to 900999, the ids the
// Core Rule Set keeps for the rules that set it up, which // Core Rule Set keeps for the rules that set it up, which
// SWW_WAF_CORE_RULE_SET_DISABLED_RULES and the exclusions file refuse, so that no // SWW_WAF_CORE_RULE_SET_DISABLED_RULES and the exclusions file refuse, so that no
@@ -54,13 +53,12 @@ Include @crs-setup.conf.example
SecAction "id:900000,phase:1,pass,nolog,\ SecAction "id:900000,phase:1,pass,nolog,\
setvar:tx.blocking_paranoia_level=%d" setvar:tx.blocking_paranoia_level=%d"
# The first change: PUT, PATCH and DELETE are allowed besides GET, HEAD, # PUT, PATCH and DELETE are allowed besides GET, HEAD, POST and OPTIONS.
# POST and OPTIONS.
SecAction "id:900200,phase:1,pass,nolog,\ SecAction "id:900200,phase:1,pass,nolog,\
setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE'" setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE'"
# The second: Expect and Content-Encoding are taken off the Core Rule Set's # Expect and Content-Encoding are taken off the Core Rule Set's list of the
# list of the headers it refuses. Content-Encoding goes back on it for a # headers it refuses. Content-Encoding goes back on it for a
# body the Core Rule Set reads (900260 in bodyDirectives). # body the Core Rule Set reads (900260 in bodyDirectives).
SecAction "id:900250,phase:1,pass,nolog,\ SecAction "id:900250,phase:1,pass,nolog,\
setvar:'tx.restricted_headers_basic=/proxy/ /lock-token/ /content-range/ \ setvar:'tx.restricted_headers_basic=/proxy/ /lock-token/ /content-range/ \
@@ -86,18 +84,17 @@ SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
# The apps' fixes come before the rules for the same reason. # The apps' fixes come before the rules for the same reason.
%s %s
# The fifth: only the rules for requests are loaded, and no response is # Only the rules for requests are loaded, and no response is inspected.
# inspected.
Include @owasp_crs/REQUEST-*.conf Include @owasp_crs/REQUEST-*.conf
# The third: redirect_uri is not checked for a URL naming an IP address or # redirect_uri is not checked for a URL naming an IP address or
# localhost. Coraza matches a parameter name here without regard to case. # localhost. Coraza matches a parameter name here without regard to case.
# ARGS holds the fields of a form data or multipart body Coraza reads as # ARGS holds the fields of a form data or multipart body Coraza reads as
# well as the query parameters, so a field of this name is left out too. # well as the query parameters, so a field of this name is left out too.
SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri" SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri"
SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri" SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri"
# The fourth: Referer is not checked for a Unix command without arguments # Referer is not checked for a Unix command without arguments
# or for Java starting a process. # or for Java starting a process.
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer" SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer" SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
@@ -122,9 +119,9 @@ SecRule REQUEST_HEADERS:Content-Type \
"@rx ^(?:application|text)/(?:[a-z0-9.-]+[+])?json" \ "@rx ^(?:application|text)/(?:[a-z0-9.-]+[+])?json" \
"id:900420,phase:1,pass,nolog,t:none,t:lowercase,ctl:requestBodyProcessor=JSON" "id:900420,phase:1,pass,nolog,t:none,t:lowercase,ctl:requestBodyProcessor=JSON"
# The rest of the second change: Content-Encoding is refused again on a # Content-Encoding goes back on the list of the headers the Core Rule Set
# body of a kind the Core Rule Set reads, since a compressed body cannot be # refuses (900250 in directives) for a body of a kind it reads, since a
# inspected. # compressed body cannot be inspected.
SecRule REQBODY_PROCESSOR "@rx ^(?:URLENCODED|MULTIPART|JSON|XML)$" \ SecRule REQBODY_PROCESSOR "@rx ^(?:URLENCODED|MULTIPART|JSON|XML)$" \
"id:900260,phase:1,pass,nolog,\ "id:900260,phase:1,pass,nolog,\
setvar:'tx.restricted_headers_basic=%%{tx.restricted_headers_basic} \ setvar:'tx.restricted_headers_basic=%%{tx.restricted_headers_basic} \
@@ -145,15 +142,19 @@ SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'" severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
` `
// app is an app whose ordinary requests the Core Rule Set would refuse, // appFixes holds the fixes of each app whose ordinary requests the Core
// with the fixes that let them pass. // Rule Set would refuse, in a file named for the app by the name
// SWW_WAF_APPS gives it, such as apps/gitea.conf. The fixes are Coraza
// exclusions, loaded before the Core Rule Set's rules. Their first rule
// skips to the marker they end with, unless Inspect sets the transaction
// variable named for the app, such as app_gitea. The skip reaches only
// phase 1, so a rule of a later phase must check that variable itself.
//
//go:embed apps/*.conf
var appFixes embed.FS
// app is what an app's file in appFixes cannot say about it.
type app struct { type app struct {
// directives are Coraza exclusions, loaded before the Core Rule Set's
// rules. Their first rule skips to the marker they end with, unless
// Inspect sets the transaction variable named for the app, such as
// app_gitea. The skip reaches only phase 1, so a rule of a later phase
// must check that variable itself.
directives string
// cookies are the cookies the Core Rule Set reads a request without. // cookies are the cookies the Core Rule Set reads a request without.
cookies []string cookies []string
// forkOf names the app this one is a fork of, "" for none. A hostname // forkOf names the app this one is a fork of, "" for none. A hostname
@@ -161,60 +162,34 @@ type app struct {
forkOf string forkOf string
} }
//go:embed gitea.conf // apps are the apps that need more than their file in appFixes, by the
var giteaDirectives string // name SWW_WAF_APPS gives each. flash is Forgejo's name for gitea_flash.
//go:embed wordpress.conf
var wordpressDirectives string
//go:embed drupal.conf
var drupalDirectives string
//go:embed mediawiki.conf
var mediawikiDirectives string
//go:embed mastodon.conf
var mastodonDirectives string
//go:embed vaultwarden.conf
var vaultwardenDirectives string
//go:embed nextcloud.conf
var nextcloudDirectives string
//go:embed forgejo.conf
var forgejoDirectives string
//go:embed ghost.conf
var ghostDirectives string
// apps are the apps whose fixes sww has, by the name SWW_WAF_APPS gives each.
// flash is Forgejo's name for gitea_flash.
// //
//nolint:gochecknoglobals // a constant cannot be a map //nolint:gochecknoglobals // a constant cannot be a map
var apps = map[string]app{ var apps = map[string]app{
"drupal": {directives: drupalDirectives},
"forgejo": { "forgejo": {
directives: forgejoDirectives,
cookies: []string{"flash"}, cookies: []string{"flash"},
forkOf: "gitea", forkOf: "gitea",
}, },
"ghost": {directives: ghostDirectives},
"gitea": { "gitea": {
directives: giteaDirectives,
cookies: []string{"gitea_flash", "redirect_to"}, cookies: []string{"gitea_flash", "redirect_to"},
}, },
"mastodon": {directives: mastodonDirectives},
"mediawiki": {directives: mediawikiDirectives},
"nextcloud": {directives: nextcloudDirectives},
"vaultwarden": {directives: vaultwardenDirectives},
"wordpress": {directives: wordpressDirectives},
} }
// AppNames returns the names of the apps whose fixes sww has, in // AppNames returns the names of the apps whose fixes sww has, in
// alphabetical order. // alphabetical order.
func AppNames() []string { func AppNames() []string {
return slices.Sorted(maps.Keys(apps)) // An embedded directory always reads.
files, _ := appFixes.ReadDir("apps")
names := make([]string, len(files))
for i, file := range files {
names[i] = strings.TrimSuffix(file.Name(), ".conf")
}
slices.Sort(names)
return names
} }
// Params are what New needs. // Params are what New needs.
@@ -373,8 +348,11 @@ func New(params Params) (*CoreRuleSet, error) {
} }
var fixes strings.Builder var fixes strings.Builder
for _, name := range AppNames() { for _, name := range AppNames() {
fixes.WriteString(apps[name].directives + "\n") // An embedded file always reads.
file, _ := appFixes.ReadFile("apps/" + name + ".conf")
fixes.WriteString(string(file) + "\n")
} }
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions, text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions,
+3 -3
View File
@@ -17,7 +17,7 @@ const (
) )
// giteaSearches are searches of the pages gitea and Forgejo share whose q // giteaSearches are searches of the pages gitea and Forgejo share whose q
// skips the lists of system files and command names, in the eighth change. // skips the lists of system files and command names, in gitea's fixes.
// gitea serves its admin pages under /-/admin/ and Forgejo under /admin/. // gitea serves its admin pages under /-/admin/ and Forgejo under /admin/.
// //
//nolint:gochecknoglobals // a constant cannot be a list //nolint:gochecknoglobals // a constant cannot be a list
@@ -222,8 +222,8 @@ func TestGiteaSearchExclusionsAreNarrow(t *testing.T) {
} }
// Under /admin/, where Forgejo serves its admin pages, only the hard // Under /admin/, where Forgejo serves its admin pages, only the hard
// values are left out: the tenth change lets them through in q on every // values are left out: Drupal's fixes let them through in q on every path
// path there, for Drupal. // there.
for _, target := range []string{ for _, target := range []string{
"/admin/packages/delete?q=", "/admin/packages/delete?q=",
"/admin/packages?other=", "/admin/packages?other=",
+9 -5
View File
@@ -1,7 +1,8 @@
package waf package waf
// The heuristic abuseipdb. It reads SWW_WAF_ABUSEIPDB_KEY, // The heuristic abuseipdb. It reads SWW_WAF_ABUSEIPDB_KEY,
// SWW_WAF_ABUSEIPDB_MIN_SCORE and SWW_WAF_DNSBL_ABUSEIPDB_ACTION. // SWW_WAF_ABUSEIPDB_MIN_SCORE, SWW_WAF_DNSBL_ABUSEIPDB_ACTION and
// SWW_WAF_BAN_RESPONSE.
import ( import (
"context" "context"
@@ -16,10 +17,11 @@ import (
// abuseIPDB adds the tag abuseipdb and notes AbuseIPDB, as noteHit does, with // abuseIPDB adds the tag abuseipdb and notes AbuseIPDB, as noteHit does, with
// the score, when its score of the client is a hit, and refuses the request // the score, when its score of the client is a hit, and refuses the request
// when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is deny, as dnsbl does for a zone. While // with SWW_WAF_BAN_RESPONSE when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is deny, as
// SWW_WAF_ABUSEIPDB_KEY is unset it does nothing. A client without a score is // dnsbl does for a zone. While SWW_WAF_ABUSEIPDB_KEY is unset it does nothing.
// checked in the background, by the request's address, if its history counts // A client without a score is checked in the background, by the request's
// an offence, and the request does not wait for the answer. The score is then // address, if its history counts an offence, and the request does not wait for
// the answer. The score is then
// used for each address of the client. Its figures are the score and // used for each address of the client. Its figures are the score and
// SWW_WAF_ABUSEIPDB_MIN_SCORE. // SWW_WAF_ABUSEIPDB_MIN_SCORE.
type abuseIPDB struct { type abuseIPDB struct {
@@ -30,6 +32,7 @@ type abuseIPDB struct {
keySet bool // SWW_WAF_ABUSEIPDB_KEY is set keySet bool // SWW_WAF_ABUSEIPDB_KEY is set
minScore int64 // SWW_WAF_ABUSEIPDB_MIN_SCORE minScore int64 // SWW_WAF_ABUSEIPDB_MIN_SCORE
action config.ReputationAction // SWW_WAF_DNSBL_ABUSEIPDB_ACTION action config.ReputationAction // SWW_WAF_DNSBL_ABUSEIPDB_ACTION
banResponse int // SWW_WAF_BAN_RESPONSE
ipv6GroupPrefix int // SWW_WAF_IPV6_GROUP_PREFIX ipv6GroupPrefix int // SWW_WAF_IPV6_GROUP_PREFIX
hitNoter reputationHitNoter hitNoter reputationHitNoter
} }
@@ -71,6 +74,7 @@ func (a abuseIPDB) check(
return &verdict{ return &verdict{
action: requestlog.ActionClientDenied, action: requestlog.ActionClientDenied,
status: a.banResponse,
figures: map[string]any{ figures: map[string]any{
figureScore: score, "min_score": a.minScore, figureScore: score, "min_score": a.minScore,
}, },
+58
View File
@@ -0,0 +1,58 @@
package waf
// The heuristic active_ban. It reads SWW_WAF_MODE and SWW_WAF_BAN_RESPONSE; the
// bans are the ban ledger's.
import (
"context"
"time"
)
// activeBan refuses a client a ban covers, with SWW_WAF_BAN_RESPONSE: a ban on
// a netblock the client is in, in force at now. It notes for the log line when
// that ban ends. A request that makes the ban permanent, or in observe mode
// would have, raises the alert for it. Its figures are the ban's netblock and
// when it ends.
type activeBan struct {
banMaker banMaker
}
func (activeBan) name() string { return "active_ban" }
func (a activeBan) check(
_ context.Context,
rq *request,
now time.Time,
) *verdict {
ledger := a.banMaker.ledger
check := ledger.Check
if a.banMaker.observe {
check = ledger.Find // the ban refuses nothing, and stays as it is
}
ban, banned, madePermanent := check(rq.client, now)
var expires string
if banned {
expires = banExpires(ban)
rq.line.BanExpires = expires
}
if madePermanent {
ban.Expires = time.Time{} // the ban made permanent, which Find leaves as it is
a.banMaker.alertBan(rq, ban)
}
if !banned {
return nil
}
return &verdict{
action: banAction(ban.Permanent()),
status: a.banMaker.banResponse,
figures: map[string]any{
figureNetblock: ban.Netblock.String(), "ban_expires": expires,
},
}
}
+5 -58
View File
@@ -1,10 +1,9 @@
package waf package waf
// The heuristic active_ban, and banMaker, which makes the bans of several // banMaker, which makes the bans of several heuristics. The bans are the ban
// heuristics. active_ban reads SWW_WAF_MODE; the bans are the ban ledger's. // ledger's.
import ( import (
"context"
"net/netip" "net/netip"
"time" "time"
@@ -16,59 +15,6 @@ import (
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
) )
// banResponse is a refusal answered with SWW_WAF_BAN_RESPONSE, and logged
// with action.
func (rq *request) banResponse(action requestlog.Action) *refusal {
return &refusal{status: rq.h.config.BanResponse, action: action}
}
// activeBan refuses a client a ban covers: a ban on a netblock the client is
// in, in force at now. It notes for the log line when that ban ends. A request
// that makes the ban permanent, or in observe mode would have, raises the alert
// for it. Its figures are the ban's netblock and when it ends.
type activeBan struct {
banMaker banMaker
}
func (activeBan) name() string { return "active_ban" }
func (a activeBan) check(
_ context.Context,
rq *request,
now time.Time,
) *verdict {
ledger := a.banMaker.ledger
check := ledger.Check
if a.banMaker.observe {
check = ledger.Find // the ban refuses nothing, and stays as it is
}
ban, banned, madePermanent := check(rq.client, now)
var expires string
if banned {
expires = banExpires(ban)
rq.line.BanExpires = expires
}
if madePermanent {
ban.Expires = time.Time{} // the ban made permanent, which Find leaves as it is
a.banMaker.alertBan(rq, ban)
}
if !banned {
return nil
}
return &verdict{
action: banAction(ban.Permanent()),
figures: map[string]any{
figureNetblock: ban.Netblock.String(), "ban_expires": expires,
},
}
}
// banAction is the action of a request refused under a ban or for making one: // banAction is the action of a request refused under a ban or for making one:
// permanent says the ban never runs out. // permanent says the ban never runs out.
func banAction(permanent bool) requestlog.Action { func banAction(permanent bool) requestlog.Action {
@@ -118,8 +64,9 @@ type banMaker struct {
alerts *alerts.Queue alerts *alerts.Queue
// observe is true in observe mode (SWW_WAF_MODE). // observe is true in observe mode (SWW_WAF_MODE).
observe bool observe bool
// banResponse is SWW_WAF_BAN_RESPONSE, the status the notes of a ban for a // banResponse is SWW_WAF_BAN_RESPONSE: the status of a request refused
// clear sign of attack give. // under a ban or for a clear sign of attack, which the notes of a ban for
// a clear sign of attack give.
banResponse int banResponse int
// banScopeV4Prefix is SWW_WAF_BAN_SCOPE_V4_PREFIX and ipv6GroupPrefix // banScopeV4Prefix is SWW_WAF_BAN_SCOPE_V4_PREFIX and ipv6GroupPrefix
// SWW_WAF_IPV6_GROUP_PREFIX, which set the netblock a ban covers. // SWW_WAF_IPV6_GROUP_PREFIX, which set the netblock a ban covers.
+2
View File
@@ -4,6 +4,7 @@ package waf
import ( import (
"context" "context"
"net/http"
"sync/atomic" "sync/atomic"
"time" "time"
@@ -161,6 +162,7 @@ func (rq *request) applyFindings(
func (rq *request) timedOut(running string) *verdict { func (rq *request) timedOut(running string) *verdict {
v := &verdict{ v := &verdict{
action: requestlog.ActionChecksTimedOut, action: requestlog.ActionChecksTimedOut,
status: http.StatusServiceUnavailable,
figures: map[string]any{ figures: map[string]any{
"time_limit_ms": rq.h.besideLookupLimit.Milliseconds(), "time_limit_ms": rq.h.besideLookupLimit.Milliseconds(),
}, },
+8 -5
View File
@@ -1,7 +1,7 @@
package waf package waf
// The heuristic blocklists. It reads SWW_WAF_BLOCKLIST_ACTION, and the blocklists // The heuristic blocklists. It reads SWW_WAF_BLOCKLIST_ACTION and
// through the reputation lists. // SWW_WAF_BAN_RESPONSE, and the blocklists through the reputation lists.
import ( import (
"context" "context"
@@ -13,12 +13,14 @@ import (
) )
// blocklists notes the blocklists that list the client, as noteListed does, // blocklists notes the blocklists that list the client, as noteListed does,
// and refuses the request when SWW_WAF_BLOCKLIST_ACTION is deny. Being limit, it // and refuses the request with SWW_WAF_BAN_RESPONSE when
// lowers the client's limits instead (see limitPercentages), and being log, it // SWW_WAF_BLOCKLIST_ACTION is deny. Being limit, it lowers the client's limits
// does nothing more. Its figure is the blocklists that list the client. // instead (see limitPercentages), and being log, it does nothing more. Its
// figure is the blocklists that list the client.
type blocklists struct { type blocklists struct {
lists *reputation.Lists lists *reputation.Lists
action config.ReputationAction // SWW_WAF_BLOCKLIST_ACTION action config.ReputationAction // SWW_WAF_BLOCKLIST_ACTION
banResponse int // SWW_WAF_BAN_RESPONSE
hitNoter reputationHitNoter hitNoter reputationHitNoter
} }
@@ -39,6 +41,7 @@ func (b blocklists) check(
return &verdict{ return &verdict{
action: requestlog.ActionClientDenied, action: requestlog.ActionClientDenied,
status: b.banResponse,
figures: map[string]any{"lists": listedBy}, figures: map[string]any{"lists": listedBy},
} }
} }
+2
View File
@@ -5,6 +5,7 @@ package waf
import ( import (
"context" "context"
"net/http"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
@@ -77,6 +78,7 @@ func (c coreRuleSet) check(
return &verdict{ return &verdict{
action: requestlog.ActionBlockedByCoreRuleSet, action: requestlog.ActionBlockedByCoreRuleSet,
status: http.StatusForbidden,
figures: map[string]any{ figures: map[string]any{
figureScore: result.Score, figureScore: result.Score,
"threshold": threshold, "threshold": threshold,
+7 -3
View File
@@ -1,6 +1,7 @@
package waf package waf
// The heuristic countries. It reads SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES. // The heuristic countries. It reads SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES and
// SWW_WAF_BAN_RESPONSE.
import ( import (
"context" "context"
@@ -13,10 +14,12 @@ import (
// countries refuses a request when SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES is set and // countries refuses a request when SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES is set and
// does not hold the client's country as it was looked up. A client without a // does not hold the client's country as it was looked up. A client without a
// country, or whose country cannot be found, is refused. Its figures are the // country, or whose country cannot be found, is refused. It refuses with
// country and the setting that refused it. // SWW_WAF_BAN_RESPONSE. Its figures are the country and the setting that
// refused it.
type countries struct { type countries struct {
allowed []string // SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES allowed []string // SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES
banResponse int // SWW_WAF_BAN_RESPONSE
} }
func (countries) name() string { return "countries" } func (countries) name() string { return "countries" }
@@ -35,6 +38,7 @@ func (c countries) check(
return &verdict{ return &verdict{
action: requestlog.ActionCountryDenied, action: requestlog.ActionCountryDenied,
status: c.banResponse,
figures: map[string]any{ figures: map[string]any{
"country": country, "country": country,
"setting": config.ExclusivelyAllowedCountriesSetting, "setting": config.ExclusivelyAllowedCountriesSetting,
+5 -3
View File
@@ -1,6 +1,6 @@
package waf package waf
// The heuristic deny_nets. It reads SWW_WAF_DENY_NETS. // The heuristic deny_nets. It reads SWW_WAF_DENY_NETS and SWW_WAF_BAN_RESPONSE.
import ( import (
"context" "context"
@@ -10,10 +10,11 @@ import (
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
) )
// denyNets refuses a client in SWW_WAF_DENY_NETS. Its figure is the netblock of // denyNets refuses a client in SWW_WAF_DENY_NETS, with SWW_WAF_BAN_RESPONSE.
// the setting the client is in. // Its figure is the netblock of the setting the client is in.
type denyNets struct { type denyNets struct {
netblocks []netip.Prefix // SWW_WAF_DENY_NETS netblocks []netip.Prefix // SWW_WAF_DENY_NETS
banResponse int // SWW_WAF_BAN_RESPONSE
} }
func (denyNets) name() string { return "deny_nets" } func (denyNets) name() string { return "deny_nets" }
@@ -27,6 +28,7 @@ func (d denyNets) check(
if netblock.Contains(rq.client) { if netblock.Contains(rq.client) {
return &verdict{ return &verdict{
action: requestlog.ActionClientDenied, action: requestlog.ActionClientDenied,
status: d.banResponse,
figures: map[string]any{figureNetblock: netblock.String()}, figures: map[string]any{figureNetblock: netblock.String()},
} }
} }
+11 -7
View File
@@ -1,7 +1,8 @@
package waf package waf
// The heuristic dnsbl. It reads SWW_WAF_DNSBL_ABUSEIPDB_ACTION, and the DNSBL zones' // The heuristic dnsbl. It reads SWW_WAF_DNSBL_ABUSEIPDB_ACTION and
// verdicts through the DNSBL checker. // SWW_WAF_BAN_RESPONSE, and the DNSBL zones' verdicts through the DNSBL
// checker.
import ( import (
"context" "context"
@@ -15,14 +16,16 @@ import (
// dnsbl notes the DNSBL zones whose verdict lists the client, as noteListed // dnsbl notes the DNSBL zones whose verdict lists the client, as noteListed
// does, adds the tag dnsbl=<zone> for each, the zone's key masked, and refuses // does, adds the tag dnsbl=<zone> for each, the zone's key masked, and refuses
// the request when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is deny. Being // the request with SWW_WAF_BAN_RESPONSE when SWW_WAF_DNSBL_ABUSEIPDB_ACTION is
// limit, it lowers the client's limits instead (see limitPercentages), and // deny. Being limit, it lowers the client's limits instead (see
// being log, it does nothing more. A zone without a verdict on the client is // limitPercentages), and being log, it does nothing more. A zone without a
// asked about it in the background, and the request does not wait for the // verdict on the client is asked about it in the background, and the request
// answer. Its figure is the zones that list the client, their keys masked. // does not wait for the answer. Its figure is the zones that list the client,
// their keys masked.
type dnsbl struct { type dnsbl struct {
dnsbl *reputation.DNSBL dnsbl *reputation.DNSBL
action config.ReputationAction // SWW_WAF_DNSBL_ABUSEIPDB_ACTION action config.ReputationAction // SWW_WAF_DNSBL_ABUSEIPDB_ACTION
banResponse int // SWW_WAF_BAN_RESPONSE
hitNoter reputationHitNoter hitNoter reputationHitNoter
} }
@@ -47,6 +50,7 @@ func (d dnsbl) check(
return &verdict{ return &verdict{
action: requestlog.ActionClientDenied, action: requestlog.ActionClientDenied,
status: d.banResponse,
figures: map[string]any{"zones": listedBy}, figures: map[string]any{"zones": listedBy},
} }
} }
+3
View File
@@ -4,6 +4,7 @@ package waf
import ( import (
"context" "context"
"net/http"
"strings" "strings"
"time" "time"
@@ -39,6 +40,7 @@ func (headerLimits) check(
if requestLine > maxLineBytes { if requestLine > maxLineBytes {
return &verdict{ return &verdict{
action: requestlog.ActionRequestLineTooLong, action: requestlog.ActionRequestLineTooLong,
status: http.StatusRequestURITooLong,
figures: map[string]any{ figures: map[string]any{
"request_line_bytes": requestLine, "request_line_bytes": requestLine,
"request_line_bytes_limit": maxLineBytes, "request_line_bytes_limit": maxLineBytes,
@@ -56,6 +58,7 @@ func (headerLimits) check(
return &verdict{ return &verdict{
action: requestlog.ActionRequestHeaderTooLarge, action: requestlog.ActionRequestHeaderTooLarge,
status: http.StatusRequestHeaderFieldsTooLarge,
figures: map[string]any{ figures: map[string]any{
"longest_header_line_bytes": longest, "longest_header_line_bytes": longest,
"header_line_bytes_limit": maxLineBytes, "header_line_bytes_limit": maxLineBytes,
+5 -2
View File
@@ -42,10 +42,13 @@ type refusesWithoutLookup interface {
} }
// verdict is a heuristic refusing a request, or banning its client: the action // verdict is a heuristic refusing a request, or banning its client: the action
// the log line names, and the figures the heuristic acted on, by name, such as // the log line names, the status the visitor is answered with, and the figures
// the count and the limit. // the heuristic acted on, by name, such as the count and the limit. A status of
// 0 closes the visitor's connection without an answer. A verdict of afterAnswer
// or onReport refuses nothing, and has neither action nor status.
type verdict struct { type verdict struct {
action requestlog.Action action requestlog.Action
status int
figures map[string]any figures map[string]any
} }
+20
View File
@@ -80,6 +80,26 @@ func (rq *request) startLookUp(
return result return result
} }
// refusedWhateverTheAnswer reports whether the request is one that does not wait
// for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
// its client: a heuristic of afterLookup that implements refusesWithoutLookup
// says it will refuse it. In observe mode, which refuses nothing, none is.
// rproxy lets a request through when waf answers too late.
func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
if rq.h.config.Observe {
return false
}
for _, each := range rq.h.afterLookup {
knows, ok := each.(refusesWithoutLookup)
if ok && knows.refusesWhateverTheLookupAnswers(rq, now) {
return true
}
}
return false
}
// noteLookup notes result for the log line and as the request's asn= and // noteLookup notes result for the log line and as the request's asn= and
// country= tags. // country= tags.
func (rq *request) noteLookup(result lookupResult) { func (rq *request) noteLookup(result lookupResult) {
-322
View File
@@ -1,322 +0,0 @@
// Package waf is the server the waf command runs. It answers
// rproxy's questions, takes rproxy's reports and answers its own endpoints, and
// writes a log line for each. It also bans the clients on AbuseIPDB's
// blacklist each time the reputation package fetches it.
package waf
import (
"cmp"
"io"
"log/slog"
"net/http"
"net/netip"
"time"
lru "github.com/hashicorp/golang-lru/v2"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/metrics"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/routewatch"
"sneak.berlin/go/smallwebwaf/internal/rules"
)
// MetricsPath is where the metrics are, for a request that carries
// SWW_WAF_METRICS_TOKEN.
const MetricsPath = "/_sww/metrics"
// BansPath is where an admin lists and adds bans, and, followed by / and
// a client's address, lifts them, with SWW_WAF_ADMIN_TOKEN.
const BansPath = "/_sww/bans"
// ClientsPath is where an admin asks what waf knows of a client, by
// the client's address after it, with SWW_WAF_ADMIN_TOKEN.
const ClientsPath = "/_sww/clients/"
// Params are what New needs.
type Params struct {
Config *config.WAF
// RequestLog receives two JSON lines per question, one at its decision and
// one when its report comes, and one per request to waf's own
// endpoints.
RequestLog io.Writer
// ProcessLog receives the process's own messages.
ProcessLog *slog.Logger
// Metrics count what waf does. Whoever builds them adds the metrics
// read from the other parts.
Metrics *metrics.Metrics
// Limiter is the table of clients the rate limits count, with each
// client's history.
Limiter *ratelimit.Limiter
// Ledger holds the bans.
Ledger *bans.Ledger
// Anomalies are the counters of the anomaly thresholds.
Anomalies *anomaly.Counters
// Lists are the lists fetched from URLs, DNSBL the DNSBL zones'
// verdicts, and AbuseIPDB AbuseIPDB's scores and the checks spent.
Lists *reputation.Lists
DNSBL *reputation.DNSBL
AbuseIPDB *reputation.AbuseIPDB
// RouteWatch asks routewatch about clients, and is nil while
// SWW_WAF_ROUTEWATCH_URL is unset.
RouteWatch *routewatch.Client
// GeoJS looks clients' AS numbers and countries up while
// SWW_WAF_LOOKUP_SOURCE is geojs. It is given AddLookup as its Answered
// and GeoJSFailed as its Failed.
GeoJS *geojs.GeoJS
// LookupFile is the lookup database they are looked up in while
// SWW_WAF_LOOKUP_SOURCE is file, and nil otherwise.
LookupFile *lookup.File
// CoreRuleSet is what inspects requests while
// SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
CoreRuleSet CoreRuleSetInspector
// Now tells the time by which requests are counted for the rate
// limits, bans are made and run out, and GeoJS's answers are kept,
// normally time.Now in UTC, the time the state files give.
Now func() time.Time
// Rules are the rule files' rules, which each request is checked
// against.
Rules *rules.Files
// Alerts receive the alert for each ban waf makes or makes
// permanent, for each count over an anomaly threshold, for each request
// whose client a blocklist, a DNSBL zone or AbuseIPDB lists, for each
// request the Core Rule Set scores at or over
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD, and for GeoJS failing, a fetch of a list
// failing, a query to a DNSBL zone or a check with AbuseIPDB failing,
// or the day's AbuseIPDB checks used up.
Alerts *alerts.Queue
// BesideLookupTimeLimit is how long the heuristics of besideLookup have
// for a request. Zero means besideLookupTimeLimit. Tests that run the
// Core Rule Set lengthen it, since the race detector and a busy host can
// slow an inspection past besideLookupTimeLimit.
BesideLookupTimeLimit time.Duration
}
// CoreRuleSetInspector inspects a request with the Core Rule Set, as
// coreruleset.CoreRuleSet's Inspect says. Tests give waf a stand-in for it.
type CoreRuleSetInspector interface {
Inspect(
r *http.Request, client netip.Addr, body io.Reader,
) (coreruleset.Result, []byte, error)
}
// Server is the server waf runs.
type Server struct {
httpServer *http.Server
h *handler
}
// newHandler returns the handler with what every request shares, the parts
// in params among it. It builds the heuristics, each with the settings and
// parts it reads, and the lists they run in.
//
//nolint:funlen // a few lines for each heuristic, a list that grows with them
func newHandler(params Params) *handler {
awaitingReport, err := lru.New[string, *decision](maxAwaitingReport)
if err != nil {
panic(err) // New fails only for a size below one
}
cfg := params.Config
h := &handler{
config: cfg,
requestLog: params.RequestLog,
processLog: params.ProcessLog,
now: params.Now,
metrics: params.Metrics,
awaitingReport: awaitingReport,
limiter: params.Limiter,
ledger: params.Ledger,
geojs: params.GeoJS,
anomalies: params.Anomalies,
lookupFile: params.LookupFile,
// The country list, the headers, the tag settings that name an asn=
// or country= tag and the biased thresholds act on the answer before
// the request goes on.
waitForGeoJS: len(cfg.ExclusivelyAllowedCountries) > 0 ||
cfg.AddLookupHeaders ||
cfg.TagsNeedLookup() ||
biasedThresholdsSet(cfg),
besideLookupLimit: cmp.Or(
params.BesideLookupTimeLimit,
besideLookupTimeLimit,
),
}
sharedBanMaker := banMaker{
ledger: params.Ledger,
limiter: params.Limiter,
metrics: params.Metrics,
alerts: params.Alerts,
observe: cfg.Observe,
banResponse: cfg.BanResponse,
banScopeV4Prefix: cfg.BanScopeV4Prefix,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
}
hitNoter := reputationHitNoter{
metrics: params.Metrics,
alerts: params.Alerts,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
}
h.trapPaths = trapPaths{paths: cfg.TrapPaths, banMaker: sharedBanMaker}
h.ruleFiles = ruleFiles{
rules: params.Rules,
metrics: params.Metrics,
banMaker: sharedBanMaker,
}
h.coreRuleSet = coreRuleSet{
inspector: params.CoreRuleSet,
metrics: params.Metrics,
alerts: params.Alerts,
mode: cfg.CoreRuleSetMode,
exemptPaths: cfg.CoreRuleSetExemptPaths,
scoreThreshold: cfg.CoreRuleSetScoreThreshold,
observe: cfg.Observe,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
}
h.beforeLookup = []heuristic{
headerLimits{},
tagLists{lists: params.Lists},
routeWatchTags{
routeWatch: params.RouteWatch,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
},
denyNets{netblocks: cfg.DenyNets},
activeBan{banMaker: sharedBanMaker},
}
h.afterLookup = []heuristic{
countries{allowed: cfg.ExclusivelyAllowedCountries},
blocklists{
lists: params.Lists,
action: cfg.BlocklistAction,
hitNoter: hitNoter,
},
dnsbl{
dnsbl: params.DNSBL,
action: cfg.DNSBLAbuseIPDBAction,
hitNoter: hitNoter,
},
abuseIPDB{
abuseIPDB: params.AbuseIPDB,
limiter: params.Limiter,
keySet: cfg.AbuseIPDBKey != "",
minScore: cfg.AbuseIPDBMinScore,
action: cfg.DNSBLAbuseIPDBAction,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
hitNoter: hitNoter,
},
tagDeny{tags: cfg.TagDeny},
rateLimits{
limiter: params.Limiter,
exemptNets: cfg.RateLimitExemptNets,
exemptPaths: cfg.RateLimitExemptPaths,
banResponse: cfg.BanResponse,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
biasedThresholds: biasedThresholds{
lists: params.Lists,
tagLimitPercent: cfg.TagLimitPercent,
asnBytesLimitPercent: cfg.ASNBytesLimitPercent,
countryBytesLimitPercent: cfg.CountryBytesLimitPercent,
unknownCountryLimitPercent: cfg.UnknownCountryLimitPercent,
blocklistAction: cfg.BlocklistAction,
blocklistLimitPercent: cfg.BlocklistLimitPercent,
dnsblAbuseIPDBAction: cfg.DNSBLAbuseIPDBAction,
dnsblAbuseIPDBLimitPercent: cfg.DNSBLAbuseIPDBLimitPercent,
},
banMaker: sharedBanMaker,
},
}
h.besideLookup = []heuristic{h.trapPaths, h.ruleFiles, h.coreRuleSet}
h.afterBesideLookup = []heuristic{tagDeny{tags: cfg.TagDeny}}
h.afterAnswer = []heuristic{
refusalLimit{
limiter: params.Limiter,
perMinute: cfg.RefusalsLimitPerMinute,
banResponse: cfg.BanResponse,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
banMaker: sharedBanMaker,
},
}
h.onReport = []heuristic{
byteLimits{
limiter: params.Limiter,
bytesCounted: cfg.BytesCounted,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
banMaker: sharedBanMaker,
},
}
h.ownEndpoints = h.routes()
return h
}
// handler is waf's handler. It holds what every request shares; what
// belongs to one request is in a request.
type handler struct {
config *config.WAF
requestLog io.Writer
processLog *slog.Logger
now func() time.Time
metrics *metrics.Metrics
limiter *ratelimit.Limiter
ledger *bans.Ledger
geojs *geojs.GeoJS
anomalies *anomaly.Counters
lookupFile *lookup.File
// waitForGeoJS is true when a setting needs each request's answer from
// GeoJS before the request goes on.
waitForGeoJS bool
// besideLookupLimit is how long the heuristics of besideLookup have for a
// request, as Params.BesideLookupTimeLimit says.
besideLookupLimit time.Duration
// awaitingReport holds the decision on each answered question by its id
// until its report comes, and nil for a question still being handled.
awaitingReport *lru.Cache[string, *decision]
// ownEndpoints answers every request that is not a question, as routes
// lists the endpoints.
ownEndpoints http.Handler
// The lists of heuristics, each in the order its heuristics run. A client
// in SWW_WAF_ALLOW_NETS skips them all.
// beforeLookup are the heuristics that run before the client's AS number
// and country are looked up, so that a client they refuse is not looked
// up. header_limits comes first: it only measures and counts, and a
// request it refuses causes no other work.
beforeLookup []heuristic
// afterLookup are the heuristics that run once the lookup has been made.
afterLookup []heuristic
// besideLookup are the heuristics that only read the request. Their work
// runs beside the lookup and afterLookup, in findBesideLookup, which
// changes nothing of the request. Their check then applies what that work
// found, and runs only once waf takes it.
besideLookup []heuristic
// afterBesideLookup are the heuristics that run once afterLookup and
// besideLookup have both let the request through. tag_deny runs a second
// time, on the tag rule_files and core_rule_set can add.
afterBesideLookup []heuristic
// afterAnswer are the heuristics that run on a request once waf has
// answered it, before its decision line is written. They ban the client,
// and do not change what the request got: their verdict's action is not
// used.
afterAnswer []heuristic
// onReport are the heuristics that run on an allowed request when
// rproxy's report of it comes, before its report line is written. Like
// afterAnswer, they only ban.
onReport []heuristic
// The heuristics of besideLookup, whose work findBesideLookup does.
trapPaths trapPaths
ruleFiles ruleFiles
coreRuleSet coreRuleSet
}
+235
View File
@@ -0,0 +1,235 @@
package waf
import (
"context"
"errors"
"net/http"
"strings"
"time"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/wafprotocol"
)
// answerQuestion runs the checks on the visitor's request a question
// carries, and answers rproxy what to do with it. A denied request's counting
// is finished here, and an allowed one's by its report.
func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
question, ok := h.readQuestion(w, r)
if !ok {
return
}
rq := h.startRequest(
w,
question.Visitor,
question.Client,
question.Peer,
true,
)
// These run even when a check panics. finish runs first, so that what is
// held has the finished log line. Go's server sends the answer only once
// this function has returned, so the report cannot come before it is held.
defer func() {
rq.finish()
rq.holdForReport()
}()
rq.site = r.Host
rq.line.WAFLine = requestlog.WAFLineDecision
refused := rq.check(r.Context())
rq.checked = time.Now()
underOwnPrefix := strings.HasPrefix(
r.URL.Path,
wafprotocol.OwnEndpointPrefix,
)
if refused == nil && !underOwnPrefix {
rq.allow()
// In observe mode, what enforce mode would have refused counts here.
rq.runBanningHeuristics(r.Context(), rq.h.afterAnswer)
return
}
if refused != nil {
rq.deny(*refused)
} else {
// The app never sees this prefix. waf's own endpoints are at its
// own address, so this answers as a path with no endpoint does.
rq.line.Action = requestlog.ActionOwnEndpoint
wafprotocol.SetAnswerHeaders(
rq.out.Header(),
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
)
http.Error(
rq.out,
http.StatusText(http.StatusNotFound),
http.StatusNotFound,
)
}
rq.banAndCount(r.Context(), rq.h.afterAnswer)
}
// readQuestion reads a question, and reports whether waf takes it. It answers
// a question it does not take with 400 and no disposition. rproxy passes that
// 400 to the visitor, and SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION does
// not apply. The id of a question waf takes is held from here on, without a
// decision yet, so that another question with that id is refused while this
// one is being handled.
//
// A question whose body stops arriving gets no answer, and is not looked at.
func (h *handler) readQuestion(
w http.ResponseWriter, r *http.Request,
) (wafprotocol.Question, bool) {
question, err := wafprotocol.ReadQuestion(w, r)
if errors.Is(err, wafprotocol.ErrBodyStoppedArriving) {
h.processLog.Warn(
"a question's body stopped arriving",
"error",
err.Error(),
)
// This panic has Go's server close the connection without an answer,
// and log nothing.
panic(http.ErrAbortHandler)
}
if err != nil {
h.refuseQuestion(w, err.Error())
return question, false
}
held, _ := h.awaitingReport.ContainsOrAdd(question.RequestID, nil)
if held {
h.refuseQuestion(
w,
wafprotocol.RequestIDHeader+" is the id of a request waf holds",
)
return question, false
}
return question, true
}
// refuseQuestion answers a question waf does not take, for reason.
func (h *handler) refuseQuestion(w http.ResponseWriter, reason string) {
h.processLog.Warn("a question is refused", "reason", reason)
http.Error(w, reason, http.StatusBadRequest)
}
// allow answers a question with wafprotocol.DispositionAllow and, while
// SWW_WAF_ADD_LOOKUP_HEADERS is true, the client's AS number and country.
func (rq *request) allow() {
rq.allowed = true
answer := wafprotocol.Answer{Disposition: wafprotocol.DispositionAllow}
if rq.h.config.AddLookupHeaders {
answer.ASN = rq.lookupAnswer.ASN
answer.Country = rq.lookupAnswer.Country
}
wafprotocol.SetAnswerHeaders(rq.out.Header(), answer)
rq.out.WriteHeader(http.StatusOK)
}
// holdForReport holds a copy of what the report's steps need, by the request's
// id. A report only ever reaches that copy, never the request being handled.
func (rq *request) holdForReport() {
held := rq.decision
rq.h.awaitingReport.Add(rq.line.RequestID, &held)
}
// deny answers a question with what the visitor is to get for r.
func (rq *request) deny(r refusal) {
rq.refused = &r
if r.status == 0 {
// SWW_WAF_BAN_RESPONSE is close. Written past rq.out, since the log line's
// status stays 0: the visitor is sent nothing.
wafprotocol.SetAnswerHeaders(
rq.out.Header(),
wafprotocol.Answer{Disposition: wafprotocol.DispositionClose},
)
rq.out.ResponseWriter.WriteHeader(http.StatusOK)
return
}
wafprotocol.SetAnswerHeaders(
rq.out.Header(),
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
)
http.Error(rq.out, http.StatusText(r.status), r.status)
}
// handleReport answers POST wafprotocol.ReportPath: it finishes the request
// the report names, if it holds that request's decision, and ignores the
// report otherwise.
func (h *handler) handleReport() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
report, err := wafprotocol.ReadReport(w, r)
if err != nil {
h.processLog.Warn("a report is refused", "error", err.Error())
http.Error(w, "the body is not a report", http.StatusBadRequest)
return
}
held, found := h.awaitingReport.Peek(report.RequestID)
// Remove reports true to one report only, of several that name the
// id. A question still being handled is held as nil.
if found && held != nil &&
h.awaitingReport.Remove(report.RequestID) {
h.finishWith(r.Context(), held, report)
}
w.WriteHeader(http.StatusNoContent)
}
}
// finishWith finishes a request with what its report says, and writes the
// request's last log line, which holds its decision's fields and the report's.
// For an allowed request it first runs the heuristics of onReport, and adds the
// request to its client's history, the anomaly counters and the metrics, with
// the status and bytes the report gives. A denied request's counting was
// finished at its decision. ctx is the report's own context.
func (h *handler) finishWith(
ctx context.Context,
held *decision,
report wafprotocol.Report,
) {
rq := &request{
decision: *held,
h: h,
out: &responseWriter{
status: report.Status,
bytes: report.ResponseBytes,
},
requestBytes: report.RequestBytes,
}
rq.line.WAFLine = requestlog.WAFLineReport
rq.line.Status = report.Status
rq.line.RequestBytes = report.RequestBytes
rq.line.ResponseBytes = report.ResponseBytes
rq.line.DurationTotalMS = report.DurationTotalMS
if rq.allowed {
rq.line.DurationUpstreamTotalMS = &report.DurationUpstreamTotalMS
rq.banAndCount(ctx, rq.h.onReport)
rq.h.metrics.RequestReported(
&rq.line,
time.Duration(report.DurationTotalMS*float64(time.Millisecond)),
time.Duration(
report.DurationUpstreamTotalMS*float64(time.Millisecond),
),
)
}
rq.writeLine()
}
+6 -4
View File
@@ -17,9 +17,9 @@ import (
// countedByRateLimits exempts it, by the client's limit percentages, which it // countedByRateLimits exempts it, by the client's limit percentages, which it
// notes for the log line with the client's counts. A request that takes the // notes for the log line with the client's counts. A request that takes the
// client over a rate limit, as its limit percentage lowers it, breaks it: it is // client over a rate limit, as its limit percentage lowers it, breaks it: it is
// refused, and bans the client. A request exempt from the rate limits is exempt // refused with SWW_WAF_BAN_RESPONSE, and bans the client. A request exempt
// from the byte limits too. Its figures are the window, the count and the // from the rate limits is exempt from the byte limits too. Its figures are the
// limit. // window, the count and the limit.
type rateLimits struct { type rateLimits struct {
limiter *ratelimit.Limiter limiter *ratelimit.Limiter
exemptNets []netip.Prefix // SWW_WAF_RATE_LIMIT_EXEMPT_NETS exemptNets []netip.Prefix // SWW_WAF_RATE_LIMIT_EXEMPT_NETS
@@ -63,7 +63,9 @@ func (r rateLimits) check(
r.banMaker.banForLimit(rq, now, hit, r.banResponse) r.banMaker.banForLimit(rq, now, hit, r.banResponse)
return &verdict{ return &verdict{
action: requestlog.ActionClientIsRateLimited, figures: limitFigures(hit), action: requestlog.ActionClientIsRateLimited,
status: r.banResponse,
figures: limitFigures(hit),
} }
} }
+4 -27
View File
@@ -1,15 +1,12 @@
package waf package waf
// The heuristic refusal_limit. It reads SWW_WAF_REFUSALS_LIMIT_PER_MINUTE and // The heuristic refusal_limit. It reads SWW_WAF_REFUSALS_LIMIT_PER_MINUTE.
// SWW_WAF_BAN_RESPONSE.
import ( import (
"context" "context"
"net/http"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
) )
// refusalLimit counts the request once it has been answered, if waf // refusalLimit counts the request once it has been answered, if waf
@@ -18,11 +15,11 @@ import (
// SWW_WAF_REFUSALS_LIMIT_PER_MINUTE such refusals of the client within a minute break // SWW_WAF_REFUSALS_LIMIT_PER_MINUTE such refusals of the client within a minute break
// a limit, which bans the client. A client in SWW_WAF_ALLOW_NETS, which the // a limit, which bans the client. A client in SWW_WAF_ALLOW_NETS, which the
// heuristics skip, is not counted, and nothing is while the limit is off. // heuristics skip, is not counted, and nothing is while the limit is off.
// Its figures are the window, the count and the limit. // The ban's notes give the status the client was sent, or in observe mode
// would have been. Its figures are the window, the count and the limit.
type refusalLimit struct { type refusalLimit struct {
limiter *ratelimit.Limiter limiter *ratelimit.Limiter
perMinute int64 // SWW_WAF_REFUSALS_LIMIT_PER_MINUTE perMinute int64 // SWW_WAF_REFUSALS_LIMIT_PER_MINUTE
banResponse int // SWW_WAF_BAN_RESPONSE
ipv6GroupPrefix int // SWW_WAF_IPV6_GROUP_PREFIX ipv6GroupPrefix int // SWW_WAF_IPV6_GROUP_PREFIX
banMaker banMaker banMaker banMaker
} }
@@ -51,27 +48,7 @@ func (r refusalLimit) check(
return nil return nil
} }
// What the client was sent, or in observe mode would have been. r.banMaker.banForLimit(rq, now, hit, rq.refusalStatus)
status := rq.out.status
switch rq.wouldAction {
case requestlog.ActionBlockedByBlockRule,
requestlog.ActionBlockedByCoreRuleSet:
status = http.StatusForbidden
case requestlog.ActionTemporarilyBanned,
requestlog.ActionPermanentlyBanned:
status = r.banResponse
case requestlog.ActionForward,
requestlog.ActionRequestLineTooLong,
requestlog.ActionRequestHeaderTooLarge,
requestlog.ActionClientIsRateLimited,
requestlog.ActionClientDenied,
requestlog.ActionCountryDenied,
requestlog.ActionChecksTimedOut,
requestlog.ActionOwnEndpoint:
}
r.banMaker.banForLimit(rq, now, hit, status)
return &verdict{figures: limitFigures(hit)} return &verdict{figures: limitFigures(hit)}
} }
+12 -25
View File
@@ -69,6 +69,10 @@ type decision struct {
// its client's history counts. trapPaths and ruleFiles set attack, // its client's history counts. trapPaths and ruleFiles set attack,
// ruleFiles blockedByBlockRule, and coreRuleSet blockedByCoreRuleSet. // ruleFiles blockedByBlockRule, and coreRuleSet blockedByCoreRuleSet.
attack, blockedByBlockRule, blockedByCoreRuleSet bool attack, blockedByBlockRule, blockedByCoreRuleSet bool
// refusalStatus is the status of the verdict that refused the request, or
// in observe mode would have: what the visitor was answered with, or would
// have been. check sets it.
refusalStatus int
// wouldAction is, in observe mode, the action enforce mode would have // wouldAction is, in observe mode, the action enforce mode would have
// refused the request with, one of requestlog's Action constants, and "" // refused the request with, one of requestlog's Action constants, and ""
// otherwise. check sets it. // otherwise. check sets it.
@@ -203,9 +207,8 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
// check is the one place where a request can be refused. It returns nil to let // check is the one place where a request can be refused. It returns nil to let
// the request through. The heuristics of checkClient decide: a request they // the request through. The heuristics of checkClient decide: a request they
// refuse is answered with SWW_WAF_BAN_RESPONSE, 403 for a block rule or the Core // refuse is answered with the status and logged with the action of the verdict
// Rule Set, 503 for heuristics that ran out of time, or 414 or 431 for the // that refused it. In observe mode nothing is refused, and the log line names
// header limits. In observe mode nothing is refused, and the log line names
// what enforce mode would have done. ctx is the request's own context. // what enforce mode would have done. ctx is the request's own context.
func (rq *request) check(ctx context.Context) *refusal { func (rq *request) check(ctx context.Context) *refusal {
v := rq.checkClient(ctx) v := rq.checkClient(ctx)
@@ -213,32 +216,16 @@ func (rq *request) check(ctx context.Context) *refusal {
return nil return nil
} }
action := v.action rq.refusalStatus = v.status
switch { if rq.h.config.Observe {
case rq.h.config.Observe: rq.wouldAction = v.action
rq.wouldAction = action rq.line.WouldAction = v.action
rq.line.WouldAction = action
return nil return nil
case action == requestlog.ActionBlockedByBlockRule ||
action == requestlog.ActionBlockedByCoreRuleSet:
return &refusal{status: http.StatusForbidden, action: action}
case action == requestlog.ActionChecksTimedOut:
return &refusal{
status: http.StatusServiceUnavailable,
action: action,
}
case action == requestlog.ActionRequestLineTooLong:
return &refusal{status: http.StatusRequestURITooLong, action: action}
case action == requestlog.ActionRequestHeaderTooLarge:
return &refusal{
status: http.StatusRequestHeaderFieldsTooLarge,
action: action,
}
default:
return rq.banResponse(action)
} }
return &refusal{status: v.status, action: v.action}
} }
// checkClient runs the heuristics on the request, and returns the first // checkClient runs the heuristics on the request, and returns the first
+8 -3
View File
@@ -1,10 +1,12 @@
package waf package waf
// The heuristic rule_files. It reads the rules of the rule files, which // The heuristic rule_files. It reads the rules of the rule files, which
// SWW_WAF_RULE_FILES_DIR and SWW_WAF_RULE_FILES_ENABLED name. // SWW_WAF_RULE_FILES_DIR and SWW_WAF_RULE_FILES_ENABLED name, and
// SWW_WAF_BAN_RESPONSE.
import ( import (
"context" "context"
"net/http"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
@@ -17,8 +19,9 @@ import (
// ruleFiles takes the rules of the rule files the request matched, which // ruleFiles takes the rules of the rule files the request matched, which
// findBesideLookup found, notes their ids in the log line, adds the tag // findBesideLookup found, notes their ids in the log line, adds the tag
// exploitsignature when it matches any, and refuses the request // exploitsignature when it matches any, and refuses the request
// when the last rule it matches refuses it: with ActionBlockedByBlockRule for a // when the last rule it matches refuses it: with 403 and
// block rule, and for a ban rule with the action for the ban it makes. A ban // ActionBlockedByBlockRule for a block rule, and for a ban rule with
// SWW_WAF_BAN_RESPONSE and the action for the ban it makes. A ban
// rule bans the client's netblock for a clear sign of attack, or in observe // rule bans the client's netblock for a clear sign of attack, or in observe
// mode raises the alert for the ban // mode raises the alert for the ban
// it would have made. Either rule's match is noted as an offence, for the // it would have made. Either rule's match is noted as an offence, for the
@@ -59,6 +62,7 @@ func (r ruleFiles) check(
return &verdict{ return &verdict{
action: requestlog.ActionBlockedByBlockRule, action: requestlog.ActionBlockedByBlockRule,
status: http.StatusForbidden,
figures: figures, figures: figures,
} }
case rules.ActionBan: case rules.ActionBan:
@@ -70,6 +74,7 @@ func (r ruleFiles) check(
now, now,
bans.Notes{RuleID: last.ID, Target: last.Target}, bans.Notes{RuleID: last.ID, Target: last.Target},
), ),
status: r.banMaker.banResponse,
figures: figures, figures: figures,
} }
case rules.ActionLog: case rules.ActionLog:
+5 -3
View File
@@ -1,6 +1,6 @@
package waf package waf
// The heuristic tag_deny. It reads SWW_WAF_TAG_DENY. // The heuristic tag_deny. It reads SWW_WAF_TAG_DENY and SWW_WAF_BAN_RESPONSE.
import ( import (
"context" "context"
@@ -10,10 +10,11 @@ import (
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
) )
// tagDeny refuses a request that carries a tag in SWW_WAF_TAG_DENY. Its figure is // tagDeny refuses a request that carries a tag in SWW_WAF_TAG_DENY, with
// the first such tag, in sorted order. // SWW_WAF_BAN_RESPONSE. Its figure is the first such tag, in sorted order.
type tagDeny struct { type tagDeny struct {
tags []string // SWW_WAF_TAG_DENY tags []string // SWW_WAF_TAG_DENY
banResponse int // SWW_WAF_BAN_RESPONSE
} }
func (tagDeny) name() string { return "tag_deny" } func (tagDeny) name() string { return "tag_deny" }
@@ -27,6 +28,7 @@ func (t tagDeny) check(
if slices.Contains(t.tags, tag) { if slices.Contains(t.tags, tag) {
return &verdict{ return &verdict{
action: requestlog.ActionClientDenied, action: requestlog.ActionClientDenied,
status: t.banResponse,
figures: map[string]any{"tag": tag}, figures: map[string]any{"tag": tag},
} }
} }
+7 -5
View File
@@ -1,6 +1,7 @@
package waf package waf
// The heuristic trap_paths. It reads SWW_WAF_TRAP_PATHS. // The heuristic trap_paths. It reads SWW_WAF_TRAP_PATHS and
// SWW_WAF_BAN_RESPONSE.
import ( import (
"context" "context"
@@ -12,10 +13,10 @@ import (
) )
// trapPaths refuses a request for a path in SWW_WAF_TRAP_PATHS, as trapPath // trapPaths refuses a request for a path in SWW_WAF_TRAP_PATHS, as trapPath
// finds it. Such a request is a clear sign of attack, as a ban rule's match is: // finds it, with SWW_WAF_BAN_RESPONSE. Such a request is a clear sign of
// it bans the client's netblock, or in observe mode raises the alert for the // attack, as a ban rule's match is: it bans the client's netblock, or in
// ban it would have made. The ban's notes give the path as sent, and so does // observe mode raises the alert for the ban it would have made. The ban's
// its figure. // notes give the path as sent, and so does its figure.
type trapPaths struct { type trapPaths struct {
paths []string // SWW_WAF_TRAP_PATHS paths []string // SWW_WAF_TRAP_PATHS
banMaker banMaker banMaker banMaker
@@ -41,6 +42,7 @@ func (t trapPaths) check(
now, now,
bans.Notes{TrapPath: path}, bans.Notes{TrapPath: path},
), ),
status: t.banMaker.banResponse,
figures: map[string]any{"path": path}, figures: map[string]any{"path": path},
} }
} }
+321 -246
View File
@@ -1,18 +1,48 @@
// Package waf is the server the waf command runs. It answers
// rproxy's questions, takes rproxy's reports and answers its own endpoints, and
// writes a log line for each. It also bans the clients on AbuseIPDB's
// blacklist each time the reputation package fetches it.
package waf package waf
import ( import (
"cmp"
"context" "context"
"errors" "io"
"log/slog" "log/slog"
"net" "net"
"net/http" "net/http"
"strings" "net/netip"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/requestlog" lru "github.com/hashicorp/golang-lru/v2"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
"sneak.berlin/go/smallwebwaf/internal/geojs"
"sneak.berlin/go/smallwebwaf/internal/lookup"
"sneak.berlin/go/smallwebwaf/internal/metrics"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/reputation"
"sneak.berlin/go/smallwebwaf/internal/routewatch"
"sneak.berlin/go/smallwebwaf/internal/rules"
"sneak.berlin/go/smallwebwaf/internal/wafprotocol" "sneak.berlin/go/smallwebwaf/internal/wafprotocol"
) )
// MetricsPath is where the metrics are, for a request that carries
// SWW_WAF_METRICS_TOKEN.
const MetricsPath = "/_sww/metrics"
// BansPath is where an admin lists and adds bans, and, followed by / and
// a client's address, lifts them, with SWW_WAF_ADMIN_TOKEN.
const BansPath = "/_sww/bans"
// ClientsPath is where an admin asks what waf knows of a client, by
// the client's address after it, with SWW_WAF_ADMIN_TOKEN.
const ClientsPath = "/_sww/clients/"
const ( const (
// maxAwaitingReport is the most answered requests held until their // maxAwaitingReport is the most answered requests held until their
// reports come. Past it, the request answered longest ago is dropped. // reports come. Past it, the request answered longest ago is dropped.
@@ -24,6 +54,79 @@ const (
wafIdleTimeout = 120 * time.Second wafIdleTimeout = 120 * time.Second
) )
// Params are what New needs.
type Params struct {
Config *config.WAF
// RequestLog receives two JSON lines per question, one at its decision and
// one when its report comes, and one per request to waf's own
// endpoints.
RequestLog io.Writer
// ProcessLog receives the process's own messages.
ProcessLog *slog.Logger
// Metrics count what waf does. Whoever builds them adds the metrics
// read from the other parts.
Metrics *metrics.Metrics
// Limiter is the table of clients the rate limits count, with each
// client's history.
Limiter *ratelimit.Limiter
// Ledger holds the bans.
Ledger *bans.Ledger
// Anomalies are the counters of the anomaly thresholds.
Anomalies *anomaly.Counters
// Lists are the lists fetched from URLs, DNSBL the DNSBL zones'
// verdicts, and AbuseIPDB AbuseIPDB's scores and the checks spent.
Lists *reputation.Lists
DNSBL *reputation.DNSBL
AbuseIPDB *reputation.AbuseIPDB
// RouteWatch asks routewatch about clients, and is nil while
// SWW_WAF_ROUTEWATCH_URL is unset.
RouteWatch *routewatch.Client
// GeoJS looks clients' AS numbers and countries up while
// SWW_WAF_LOOKUP_SOURCE is geojs. It is given AddLookup as its Answered
// and GeoJSFailed as its Failed.
GeoJS *geojs.GeoJS
// LookupFile is the lookup database they are looked up in while
// SWW_WAF_LOOKUP_SOURCE is file, and nil otherwise.
LookupFile *lookup.File
// CoreRuleSet is what inspects requests while
// SWW_WAF_CORE_RULE_SET_MODE is not off, and nil while it is.
CoreRuleSet CoreRuleSetInspector
// Now tells the time by which requests are counted for the rate
// limits, bans are made and run out, and GeoJS's answers are kept,
// normally time.Now in UTC, the time the state files give.
Now func() time.Time
// Rules are the rule files' rules, which each request is checked
// against.
Rules *rules.Files
// Alerts receive the alert for each ban waf makes or makes
// permanent, for each count over an anomaly threshold, for each request
// whose client a blocklist, a DNSBL zone or AbuseIPDB lists, for each
// request the Core Rule Set scores at or over
// SWW_WAF_CORE_RULE_SET_SCORE_THRESHOLD, and for GeoJS failing, a fetch of a list
// failing, a query to a DNSBL zone or a check with AbuseIPDB failing,
// or the day's AbuseIPDB checks used up.
Alerts *alerts.Queue
// BesideLookupTimeLimit is how long the heuristics of besideLookup have
// for a request. Zero means besideLookupTimeLimit. Tests that run the
// Core Rule Set lengthen it, since the race detector and a busy host can
// slow an inspection past besideLookupTimeLimit.
BesideLookupTimeLimit time.Duration
}
// CoreRuleSetInspector inspects a request with the Core Rule Set, as
// coreruleset.CoreRuleSet's Inspect says. Tests give waf a stand-in for it.
type CoreRuleSetInspector interface {
Inspect(
r *http.Request, client netip.Addr, body io.Reader,
) (coreruleset.Result, []byte, error)
}
// Server is the server waf runs.
type Server struct {
httpServer *http.Server
h *handler
}
// New returns the server `sww waf` runs. It answers // New returns the server `sww waf` runs. It answers
// rproxy's questions, takes rproxy's reports and answers its own endpoints. It // rproxy's questions, takes rproxy's reports and answers its own endpoints. It
// passes nothing to an app. rproxy applies the size and time limits to the // passes nothing to an app. rproxy applies the size and time limits to the
@@ -70,6 +173,221 @@ func (s *Server) ServeHTTP(w http.ResponseWriter, r *http.Request) {
s.h.ServeHTTP(w, r) s.h.ServeHTTP(w, r)
} }
// newHandler returns the handler with what every request shares, the parts
// in params among it. It builds the heuristics, each with the settings and
// parts it reads, and the lists they run in.
//
//nolint:funlen // a few lines for each heuristic, a list that grows with them
func newHandler(params Params) *handler {
awaitingReport, err := lru.New[string, *decision](maxAwaitingReport)
if err != nil {
panic(err) // New fails only for a size below one
}
cfg := params.Config
h := &handler{
config: cfg,
requestLog: params.RequestLog,
processLog: params.ProcessLog,
now: params.Now,
metrics: params.Metrics,
awaitingReport: awaitingReport,
limiter: params.Limiter,
ledger: params.Ledger,
geojs: params.GeoJS,
anomalies: params.Anomalies,
lookupFile: params.LookupFile,
// The country list, the headers, the tag settings that name an asn=
// or country= tag and the biased thresholds act on the answer before
// the request goes on.
waitForGeoJS: len(cfg.ExclusivelyAllowedCountries) > 0 ||
cfg.AddLookupHeaders ||
cfg.TagsNeedLookup() ||
biasedThresholdsSet(cfg),
besideLookupLimit: cmp.Or(
params.BesideLookupTimeLimit,
besideLookupTimeLimit,
),
}
sharedBanMaker := banMaker{
ledger: params.Ledger,
limiter: params.Limiter,
metrics: params.Metrics,
alerts: params.Alerts,
observe: cfg.Observe,
banResponse: cfg.BanResponse,
banScopeV4Prefix: cfg.BanScopeV4Prefix,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
}
hitNoter := reputationHitNoter{
metrics: params.Metrics,
alerts: params.Alerts,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
}
h.trapPaths = trapPaths{paths: cfg.TrapPaths, banMaker: sharedBanMaker}
h.ruleFiles = ruleFiles{
rules: params.Rules,
metrics: params.Metrics,
banMaker: sharedBanMaker,
}
h.coreRuleSet = coreRuleSet{
inspector: params.CoreRuleSet,
metrics: params.Metrics,
alerts: params.Alerts,
mode: cfg.CoreRuleSetMode,
exemptPaths: cfg.CoreRuleSetExemptPaths,
scoreThreshold: cfg.CoreRuleSetScoreThreshold,
observe: cfg.Observe,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
}
h.beforeLookup = []heuristic{
headerLimits{},
tagLists{lists: params.Lists},
routeWatchTags{
routeWatch: params.RouteWatch,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
},
denyNets{netblocks: cfg.DenyNets, banResponse: cfg.BanResponse},
activeBan{banMaker: sharedBanMaker},
}
h.afterLookup = []heuristic{
countries{
allowed: cfg.ExclusivelyAllowedCountries,
banResponse: cfg.BanResponse,
},
blocklists{
lists: params.Lists,
action: cfg.BlocklistAction,
banResponse: cfg.BanResponse,
hitNoter: hitNoter,
},
dnsbl{
dnsbl: params.DNSBL,
action: cfg.DNSBLAbuseIPDBAction,
banResponse: cfg.BanResponse,
hitNoter: hitNoter,
},
abuseIPDB{
abuseIPDB: params.AbuseIPDB,
limiter: params.Limiter,
keySet: cfg.AbuseIPDBKey != "",
minScore: cfg.AbuseIPDBMinScore,
action: cfg.DNSBLAbuseIPDBAction,
banResponse: cfg.BanResponse,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
hitNoter: hitNoter,
},
tagDeny{tags: cfg.TagDeny, banResponse: cfg.BanResponse},
rateLimits{
limiter: params.Limiter,
exemptNets: cfg.RateLimitExemptNets,
exemptPaths: cfg.RateLimitExemptPaths,
banResponse: cfg.BanResponse,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
biasedThresholds: biasedThresholds{
lists: params.Lists,
tagLimitPercent: cfg.TagLimitPercent,
asnBytesLimitPercent: cfg.ASNBytesLimitPercent,
countryBytesLimitPercent: cfg.CountryBytesLimitPercent,
unknownCountryLimitPercent: cfg.UnknownCountryLimitPercent,
blocklistAction: cfg.BlocklistAction,
blocklistLimitPercent: cfg.BlocklistLimitPercent,
dnsblAbuseIPDBAction: cfg.DNSBLAbuseIPDBAction,
dnsblAbuseIPDBLimitPercent: cfg.DNSBLAbuseIPDBLimitPercent,
},
banMaker: sharedBanMaker,
},
}
h.besideLookup = []heuristic{h.trapPaths, h.ruleFiles, h.coreRuleSet}
h.afterBesideLookup = []heuristic{
tagDeny{tags: cfg.TagDeny, banResponse: cfg.BanResponse},
}
h.afterAnswer = []heuristic{
refusalLimit{
limiter: params.Limiter,
perMinute: cfg.RefusalsLimitPerMinute,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
banMaker: sharedBanMaker,
},
}
h.onReport = []heuristic{
byteLimits{
limiter: params.Limiter,
bytesCounted: cfg.BytesCounted,
ipv6GroupPrefix: cfg.IPv6GroupPrefix,
banMaker: sharedBanMaker,
},
}
h.ownEndpoints = h.routes()
return h
}
// handler is waf's handler. It holds what every request shares; what
// belongs to one request is in a request.
type handler struct {
config *config.WAF
requestLog io.Writer
processLog *slog.Logger
now func() time.Time
metrics *metrics.Metrics
limiter *ratelimit.Limiter
ledger *bans.Ledger
geojs *geojs.GeoJS
anomalies *anomaly.Counters
lookupFile *lookup.File
// waitForGeoJS is true when a setting needs each request's answer from
// GeoJS before the request goes on.
waitForGeoJS bool
// besideLookupLimit is how long the heuristics of besideLookup have for a
// request, as Params.BesideLookupTimeLimit says.
besideLookupLimit time.Duration
// awaitingReport holds the decision on each answered question by its id
// until its report comes, and nil for a question still being handled.
awaitingReport *lru.Cache[string, *decision]
// ownEndpoints answers every request that is not a question, as routes
// lists the endpoints.
ownEndpoints http.Handler
// The lists of heuristics, each in the order its heuristics run. A client
// in SWW_WAF_ALLOW_NETS skips them all.
// beforeLookup are the heuristics that run before the client's AS number
// and country are looked up, so that a client they refuse is not looked
// up. header_limits comes first: it only measures and counts, and a
// request it refuses causes no other work.
beforeLookup []heuristic
// afterLookup are the heuristics that run once the lookup has been made.
afterLookup []heuristic
// besideLookup are the heuristics that only read the request. Their work
// runs beside the lookup and afterLookup, in findBesideLookup, which
// changes nothing of the request. Their check then applies what that work
// found, and runs only once waf takes it.
besideLookup []heuristic
// afterBesideLookup are the heuristics that run once afterLookup and
// besideLookup have both let the request through. tag_deny runs a second
// time, on the tag rule_files and core_rule_set can add.
afterBesideLookup []heuristic
// afterAnswer are the heuristics that run on a request once waf has
// answered it, before its decision line is written. They ban the client,
// and do not change what the request got: their verdict's action is not
// used.
afterAnswer []heuristic
// onReport are the heuristics that run on an allowed request when
// rproxy's report of it comes, before its report line is written. Like
// afterAnswer, they only ban.
onReport []heuristic
// The heuristics of besideLookup, whose work findBesideLookup does.
trapPaths trapPaths
ruleFiles ruleFiles
coreRuleSet coreRuleSet
}
// ServeHTTP handles one request to waf: a request carrying // ServeHTTP handles one request to waf: a request carrying
// wafprotocol.ClientHeader is rproxy's question, and any other is for one of // wafprotocol.ClientHeader is rproxy's question, and any other is for one of
// waf's own endpoints. // waf's own endpoints.
@@ -82,246 +400,3 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
h.ownEndpoints.ServeHTTP(w, r) h.ownEndpoints.ServeHTTP(w, r)
} }
// refusedWhateverTheAnswer reports whether the request is one that does not wait
// for GeoJS, since the checks will refuse it at now whatever GeoJS answers about
// its client: a heuristic of afterLookup that implements refusesWithoutLookup
// says it will refuse it. In observe mode, which refuses nothing, none is.
// rproxy lets a request through when waf answers too late.
func (rq *request) refusedWhateverTheAnswer(now time.Time) bool {
if rq.h.config.Observe {
return false
}
for _, each := range rq.h.afterLookup {
knows, ok := each.(refusesWithoutLookup)
if ok && knows.refusesWhateverTheLookupAnswers(rq, now) {
return true
}
}
return false
}
// answerQuestion runs the checks on the visitor's request a question
// carries, and answers rproxy what to do with it. A denied request's counting
// is finished here, and an allowed one's by its report.
func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
question, ok := h.readQuestion(w, r)
if !ok {
return
}
rq := h.startRequest(
w,
question.Visitor,
question.Client,
question.Peer,
true,
)
// These run even when a check panics. finish runs first, so that what is
// held has the finished log line. Go's server sends the answer only once
// this function has returned, so the report cannot come before it is held.
defer func() {
rq.finish()
rq.holdForReport()
}()
rq.site = r.Host
rq.line.WAFLine = requestlog.WAFLineDecision
refused := rq.check(r.Context())
rq.checked = time.Now()
underOwnPrefix := strings.HasPrefix(
r.URL.Path,
wafprotocol.OwnEndpointPrefix,
)
if refused == nil && !underOwnPrefix {
rq.allow()
// In observe mode, what enforce mode would have refused counts here.
rq.runBanningHeuristics(r.Context(), rq.h.afterAnswer)
return
}
if refused != nil {
rq.deny(*refused)
} else {
// The app never sees this prefix. waf's own endpoints are at its
// own address, so this answers as a path with no endpoint does.
rq.line.Action = requestlog.ActionOwnEndpoint
wafprotocol.SetAnswerHeaders(
rq.out.Header(),
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
)
http.Error(
rq.out,
http.StatusText(http.StatusNotFound),
http.StatusNotFound,
)
}
rq.banAndCount(r.Context(), rq.h.afterAnswer)
}
// readQuestion reads a question, and reports whether waf takes it. It answers
// a question it does not take with 400 and no disposition. rproxy passes that
// 400 to the visitor, and SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION does
// not apply. The id of a question waf takes is held from here on, without a
// decision yet, so that another question with that id is refused while this
// one is being handled.
//
// A question whose body stops arriving gets no answer, and is not looked at.
func (h *handler) readQuestion(
w http.ResponseWriter, r *http.Request,
) (wafprotocol.Question, bool) {
question, err := wafprotocol.ReadQuestion(w, r)
if errors.Is(err, wafprotocol.ErrBodyStoppedArriving) {
h.processLog.Warn(
"a question's body stopped arriving",
"error",
err.Error(),
)
// This panic has Go's server close the connection without an answer,
// and log nothing.
panic(http.ErrAbortHandler)
}
if err != nil {
h.refuseQuestion(w, err.Error())
return question, false
}
held, _ := h.awaitingReport.ContainsOrAdd(question.RequestID, nil)
if held {
h.refuseQuestion(
w,
wafprotocol.RequestIDHeader+" is the id of a request waf holds",
)
return question, false
}
return question, true
}
// refuseQuestion answers a question waf does not take, for reason.
func (h *handler) refuseQuestion(w http.ResponseWriter, reason string) {
h.processLog.Warn("a question is refused", "reason", reason)
http.Error(w, reason, http.StatusBadRequest)
}
// allow answers a question with wafprotocol.DispositionAllow and, while
// SWW_WAF_ADD_LOOKUP_HEADERS is true, the client's AS number and country.
func (rq *request) allow() {
rq.allowed = true
answer := wafprotocol.Answer{Disposition: wafprotocol.DispositionAllow}
if rq.h.config.AddLookupHeaders {
answer.ASN = rq.lookupAnswer.ASN
answer.Country = rq.lookupAnswer.Country
}
wafprotocol.SetAnswerHeaders(rq.out.Header(), answer)
rq.out.WriteHeader(http.StatusOK)
}
// holdForReport holds a copy of what the report's steps need, by the request's
// id. A report only ever reaches that copy, never the request being handled.
func (rq *request) holdForReport() {
held := rq.decision
rq.h.awaitingReport.Add(rq.line.RequestID, &held)
}
// deny answers a question with what the visitor is to get for r.
func (rq *request) deny(r refusal) {
rq.refused = &r
if r.status == 0 {
// SWW_WAF_BAN_RESPONSE is close. Written past rq.out, since the log line's
// status stays 0: the visitor is sent nothing.
wafprotocol.SetAnswerHeaders(
rq.out.Header(),
wafprotocol.Answer{Disposition: wafprotocol.DispositionClose},
)
rq.out.ResponseWriter.WriteHeader(http.StatusOK)
return
}
wafprotocol.SetAnswerHeaders(
rq.out.Header(),
wafprotocol.Answer{Disposition: wafprotocol.DispositionDeny},
)
http.Error(rq.out, http.StatusText(r.status), r.status)
}
// handleReport answers POST wafprotocol.ReportPath: it finishes the request
// the report names, if it holds that request's decision, and ignores the
// report otherwise.
func (h *handler) handleReport() http.HandlerFunc {
return func(w http.ResponseWriter, r *http.Request) {
report, err := wafprotocol.ReadReport(w, r)
if err != nil {
h.processLog.Warn("a report is refused", "error", err.Error())
http.Error(w, "the body is not a report", http.StatusBadRequest)
return
}
held, found := h.awaitingReport.Peek(report.RequestID)
// Remove reports true to one report only, of several that name the
// id. A question still being handled is held as nil.
if found && held != nil &&
h.awaitingReport.Remove(report.RequestID) {
h.finishWith(r.Context(), held, report)
}
w.WriteHeader(http.StatusNoContent)
}
}
// finishWith finishes a request with what its report says, and writes the
// request's last log line, which holds its decision's fields and the report's.
// For an allowed request it first runs the heuristics of onReport, and adds the
// request to its client's history, the anomaly counters and the metrics, with
// the status and bytes the report gives. A denied request's counting was
// finished at its decision. ctx is the report's own context.
func (h *handler) finishWith(
ctx context.Context,
held *decision,
report wafprotocol.Report,
) {
rq := &request{
decision: *held,
h: h,
out: &responseWriter{
status: report.Status,
bytes: report.ResponseBytes,
},
requestBytes: report.RequestBytes,
}
rq.line.WAFLine = requestlog.WAFLineReport
rq.line.Status = report.Status
rq.line.RequestBytes = report.RequestBytes
rq.line.ResponseBytes = report.ResponseBytes
rq.line.DurationTotalMS = report.DurationTotalMS
if rq.allowed {
rq.line.DurationUpstreamTotalMS = &report.DurationUpstreamTotalMS
rq.banAndCount(ctx, rq.h.onReport)
rq.h.metrics.RequestReported(
&rq.line,
time.Duration(report.DurationTotalMS*float64(time.Millisecond)),
time.Duration(
report.DurationUpstreamTotalMS*float64(time.Millisecond),
),
)
}
rq.writeLine()
}