Author SHA1 Message Date
sneak 828af271a6 SWW_APPS: an app's Core Rule Set fixes apply only to the hostnames that serve it (closes #220)
check / check (push) Failing after 14m14s
Each app's fixes are one .conf file and one entry in apps. They apply
only while Inspect has set the app's transaction variable, such as
tx.app_gitea. SWW_APPS names each hostname's apps. A hostname gets only
its apps' fixes; while SWW_APPS is unset, every hostname gets every
app's fixes.

A hostname in SWW_APPS that is not a DNS name, such as *.example.com,
stops the start. Unset, SWW_APPS is logged at start as null; set
empty, as "".

Model: opus-5-5
2026-11-10 11:02:15 +00:00
23 changed files with 1369 additions and 755 deletions

No files matched your search

-6
View File
@@ -43,12 +43,6 @@ RUN --mount=type=tmpfs,target=/root/.cache/go-build \
{ echo "--- Rerunning with -v for details ---"; \ { echo "--- Rerunning with -v for details ---"; \
go test -tags no_fs_access -timeout 90s -race -v ./...; exit 1; } go test -tags no_fs_access -timeout 90s -race -v ./...; exit 1; }
# Second pass: the recorded requests, too slow for make test under -race.
RUN --mount=type=tmpfs,target=/root/.cache/go-build \
go test -tags no_fs_access,recorded -timeout 90s -cover ./internal/coreruleset/... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -tags no_fs_access,recorded -timeout 90s -v ./internal/coreruleset/...; exit 1; }
# Tidy stage: `go mod tidy` in the test phase's Go, so that the files it # Tidy stage: `go mod tidy` in the test phase's Go, so that the files it
# writes pass the test phase's check. Nothing else depends on it, so only # writes pass the test phase's check. Nothing else depends on it, so only
# script/tidy, which names the stage after it, builds it. # script/tidy, which names the stage after it, builds it.
+28 -24
View File
@@ -221,7 +221,7 @@ account, a key, a token or an alert destination stays off until you give one.
Every setting the code reads, with its default: Every setting the code reads, with its default:
| Variable | Default | What it does | | Variable | Default | What it does |
| ---------------------------------------- | ---------------------------------------------------------------------------------------- | --------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- | | ---------------------------------------- | ---------------------------------------------------------------------------------------- | ----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| `SWW_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `discriminator`. | | `SWW_COMMAND` | `rproxy` | Which command the image runs, and its health check checks: `rproxy` or `discriminator`. |
| `SWRP_WAF_BACKEND` | required | `discriminator`'s URL, such as `http://discriminator:8080`. | | `SWRP_WAF_BACKEND` | required | `discriminator`'s URL, such as `http://discriminator:8080`. |
| `SWRP_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. | | `SWRP_UPSTREAM_URL` | `http://127.0.0.1:8081` | The app's URL: a scheme, a host and an optional port, nothing more. |
@@ -280,6 +280,7 @@ Every setting the code reads, with its default:
| `SWW_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. | | `SWW_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
| `SWW_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. | | `SWW_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
| `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. | | `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
| `SWW_APPS` | unset | The apps each hostname serves, `drupal`, `gitea`, `mastodon`, `mediawiki`, `nextcloud`, `vaultwarden` or `wordpress`, such as `git.example.com:gitea;blog.example.com:wordpress`; only those apps' fixes apply there. Unset, every app's fixes apply to every hostname. |
| `SWW_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. | | `SWW_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
| `SWW_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. | | `SWW_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
| `SWW_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. | | `SWW_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
@@ -581,27 +582,31 @@ client's limits by tag. See "Request tags" in [`SPEC.md`](SPEC.md).
headers of every request, and no body unless `SWW_WAF_BODY_LIMIT` is set. In headers of every request, and no body unless `SWW_WAF_BODY_LIMIT` is set. In
front of a code forge, bodies carry what people write and publish, which the front of a code forge, bodies carry what people write and publish, which the
rules take for attacks. rules take for attacks.
- `smallwebwaf` changes the Core Rule Set in fourteen ways so that ordinary - `smallwebwaf` changes the Core Rule Set in six ways so that it lets ordinary
requests get through to the apps listed below. No setting undoes the changes; requests to a code forge through, such as git over HTTP and sign-in through
"Attack detection" in [`SPEC.md`](SPEC.md) lists them. Git Credential Manager. No setting undoes the changes; "Attack detection" in
- These apps work out of the box at the default settings, and their recorded [`SPEC.md`](SPEC.md) lists them.
ordinary requests pass the Core Rule Set: - Each supported app also has its own fixes, which let its ordinary requests
- gitea: git over HTTP and sign-in through Git Credential Manager get through. Its recorded ordinary requests pass the Core Rule Set at the default
through. On gitea's own paths, its searches, file names and branch names settings. The supported apps, by their names in `SWW_APPS`, are:
do too. - `gitea`, whose fixes let its searches, file names and branch names
- WordPress: saving a post from the block editor gets through. On through;
WordPress's own paths, its searches, names and file names do too. - `wordpress`, whose fixes let through its searches, its names, its file
- Drupal: on Drupal's own paths, the page to return to, its searches and its names and a post saved from the block editor;
names get through. - `drupal`, whose fixes let through the page to return to, its searches and
- MediaWiki: on MediaWiki's own paths, its searches, page titles, names and its names;
wikitext get through. - `mediawiki`, whose fixes let through its searches, its page titles, its
- Mastodon: on Mastodon's own paths, its searches, names, email addresses, names and its wikitext;
posts and deliveries from other servers get through. - `mastodon`, whose fixes let through its searches, its names, its email
- Vaultwarden: on Vaultwarden's own paths, its names, password hints, email addresses, its posts and deliveries from other servers;
addresses and encrypted file names get through. - `vaultwarden`, whose fixes let through its names, its password hints, its
- Nextcloud: its sync client gets through. On Nextcloud's own paths, its email addresses and its encrypted file names;
WebDAV methods, searches, file names, account names, comments and - `nextcloud`, whose fixes let through its sync client, its WebDAV methods,
documents do too. its searches, its file names, its account names, its comments and its
documents.
- `SWW_APPS` names the apps each hostname serves. A hostname gets only its own
apps' fixes, and a hostname it does not list gets none. Unset, every app's
fixes apply to every hostname.
- `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's own - `SWW_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's own
fields or paths. fields or paths.
- Rule files are plain text, one rule to a line, edited by hand and read again - Rule files are plain text, one rule to a line, edited by hand and read again
@@ -695,8 +700,7 @@ on the host.
- `script/projectname`: prints `smallwebwaf`, which the other scripts tag their - `script/projectname`: prints `smallwebwaf`, which the other scripts tag their
images with. images with.
- `script/test`: checks that `go.mod` and `go.sum` are tidy, then runs the - `script/test`: checks that `go.mod` and `go.sum` are tidy, then runs the
tests, as the `test` phase of the `Dockerfile`. It runs them twice: once with tests, as the `test` phase of the `Dockerfile`. `make test`.
the race detector, and once without it for the recorded requests. `make test`.
- `script/tidy`: writes `go.mod` and `go.sum` as `go mod tidy` writes them, in - `script/tidy`: writes `go.mod` and `go.sum` as `go mod tidy` writes them, in
the `tidy` stage of the `Dockerfile`. `make tidy`. the `tidy` stage of the `Dockerfile`. `make tidy`.
- `script/lint`: runs golangci-lint, as the `lint` phase of the `Dockerfile`. - `script/lint`: runs golangci-lint, as the `lint` phase of the `Dockerfile`.
+412 -387
View File
File diff suppressed because it is too large. Load diff
+25
View File
@@ -255,6 +255,31 @@ the request first.
a range. So does a directive that removes or changes rules as the file loads, a range. So does a directive that removes or changes rules as the file loads,
such as `SecRuleRemoveById`, since none of the Core Rule Set's rules is loaded such as `SecRuleRemoveById`, since none of the Core Rule Set's rules is loaded
yet. It has no `_FILE` form. yet. It has no `_FILE` form.
- `SWW_APPS` (default unset): the apps each hostname serves. A request gets the
fixes of its hostname's apps only (see "Attack detection" in
[`SPEC.md`](../SPEC.md)).
- Entries are separated by semicolons. Each is a hostname, a colon, then one
or more apps separated by commas, for example
`git.example.com:gitea;blog.example.com:wordpress`.
- The supported apps are:
- `drupal`;
- `gitea`;
- `mastodon`;
- `mediawiki`;
- `nextcloud`;
- `vaultwarden`;
- `wordpress`.
- The request's `Host` is matched by exact name, without its port and
without regard to case. There are no wildcards.
- A hostname not listed gets no app's fixes, so every rule checks it in
full. Set empty, `SWW_APPS` lists no hostname.
- Unset, every app's fixes apply to every hostname.
- Any of these mistakes stops the start:
- an unknown app;
- a hostname that is not a DNS name, such as `*.example.com`;
- a hostname listed twice;
- an app listed twice for one hostname;
- an entry without a colon.
- `SWW_ERROR_BURST_THRESHOLD` (default `30`): the most refusals a client may get - `SWW_ERROR_BURST_THRESHOLD` (default `30`): the most refusals a client may get
in a minute from a rule file, a trap path or the Core Rule Set, a count or in a minute from a rule file, a trap path or the Core Rule Set, a count or
`off`. The next one bans the client (see "Attack detection" in `SPEC.md`). `off`. The next one bans the client (see "Attack detection" in `SPEC.md`).
+92 -1
View File
@@ -29,6 +29,7 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly" "sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
"sneak.berlin/go/smallwebwaf/internal/remotelog" "sneak.berlin/go/smallwebwaf/internal/remotelog"
) )
@@ -235,6 +236,11 @@ type Config struct {
WAFExemptPaths []string WAFExemptPaths []string
WAFBodyLimit int64 WAFBodyLimit int64
CoreRuleSetExclusionsFile string CoreRuleSetExclusionsFile string
// Apps are the names of the apps each hostname serves, by hostname in
// lowercase (SWW_APPS): a request gets only its hostname's apps' fixes
// to the Core Rule Set. Apps is nil while SWW_APPS is unset, and every
// request gets every app's fixes.
Apps map[string][]string
// TrapPaths are the paths a request for which is a clear sign of // TrapPaths are the paths a request for which is a clear sign of
// attack (SWW_TRAP_PATHS), each starting with / and without a ?. // attack (SWW_TRAP_PATHS), each starting with / and without a ?.
TrapPaths []string TrapPaths []string
@@ -404,6 +410,11 @@ var (
errNotTrapPath = errors.New( errNotTrapPath = errors.New(
"is not a path starting with / and without a ?, such as /wp-login.php", "is not a path starting with / and without a ?, such as /wp-login.php",
) )
errNotAppsEntry = errors.New(
"is not a hostname, : and a list of apps, such as git.example.com:gitea",
)
errNotHostname = errors.New("is not a DNS name such as git.example.com")
errNotApp = errors.New("is not an app sww has fixes for; the apps are")
errNotWAFMode = errors.New("is not off, detect or block") errNotWAFMode = errors.New("is not off, detect or block")
errNotParanoiaLevel = errors.New("is not a paranoia level, from 1 to 4") errNotParanoiaLevel = errors.New("is not a paranoia level, from 1 to 4")
errNotRuleID = errors.New( errNotRuleID = errors.New(
@@ -577,6 +588,7 @@ func FromEnvironment(
WAFBodyLimit: env.wafBodyLimit("SWW_WAF_BODY_LIMIT", off), WAFBodyLimit: env.wafBodyLimit("SWW_WAF_BODY_LIMIT", off),
CoreRuleSetExclusionsFile: env.filePath( CoreRuleSetExclusionsFile: env.filePath(
"SWW_CORE_RULE_SET_EXCLUSIONS_FILE"), "SWW_CORE_RULE_SET_EXCLUSIONS_FILE"),
Apps: env.apps("SWW_APPS"),
TrapPaths: env.trapPaths("SWW_TRAP_PATHS"), TrapPaths: env.trapPaths("SWW_TRAP_PATHS"),
ErrorBurstThreshold: env.count("SWW_ERROR_BURST_THRESHOLD", "30"), ErrorBurstThreshold: env.count("SWW_ERROR_BURST_THRESHOLD", "30"),
LogRemoteURL: env.logRemoteURL("SWW_LOG_REMOTE_URL"), LogRemoteURL: env.logRemoteURL("SWW_LOG_REMOTE_URL"),
@@ -832,6 +844,25 @@ func (e *environment) pathPrefixes(name, defaultValue string) []string {
return prefixes return prefixes
} }
// apps reads the setting that is the apps each hostname serves. It is nil
// while the setting is unset, and logged as null, unlike a setting set
// empty.
func (e *environment) apps(name string) map[string][]string {
value, set := e.lookup(name)
if !set {
e.settings = append(e.settings, slog.Any(name, nil))
return nil
}
e.settings = append(e.settings, slog.String(name, value))
apps, err := parseApps(value)
e.check(name, err)
return apps
}
// trapPaths reads the setting that is the list of trap paths. It is empty // trapPaths reads the setting that is the list of trap paths. It is empty
// by default. // by default.
func (e *environment) trapPaths(name string) []string { func (e *environment) trapPaths(name string) []string {
@@ -1849,6 +1880,65 @@ func parseTrapPaths(value string) ([]string, error) {
return paths, nil return paths, nil
} }
// parseApps reads the apps each hostname serves: entries separated by
// semicolons, each a hostname, a colon, and a comma-separated list of the
// names coreruleset.AppNames returns, such as
// git.example.com:gitea;cloud.example.com:nextcloud,wordpress. Each
// hostname is a DNS name, without a wildcard. It returns them by hostname
// in lowercase, since a hostname is matched without regard to case. An
// empty value lists no hostname.
func parseApps(value string) (map[string][]string, error) {
apps := map[string][]string{}
if strings.TrimSpace(value) == "" {
return apps, nil
}
for entry := range strings.SplitSeq(value, ";") {
hostname, list, found := strings.Cut(entry, ":")
if !found {
return nil, fmt.Errorf("%q %w", entry, errNotAppsEntry)
}
hostname = strings.ToLower(strings.TrimSpace(hostname))
if hostname == "" {
return nil, fmt.Errorf("%q %w", entry, errNotAppsEntry)
}
if !isZone(hostname) {
return nil, fmt.Errorf("%q %w", hostname, errNotHostname)
}
if _, listed := apps[hostname]; listed {
return nil, fmt.Errorf("%q %w", hostname, errListedTwice)
}
names, err := parseList(list)
if err != nil {
return nil, err
}
if len(names) == 0 {
return nil, fmt.Errorf("%q %w", entry, errNotAppsEntry)
}
for i, name := range names {
if !slices.Contains(coreruleset.AppNames(), name) {
return nil, fmt.Errorf("%q %w %s", name, errNotApp,
strings.Join(coreruleset.AppNames(), ", "))
}
if slices.Contains(names[:i], name) {
return nil, fmt.Errorf("%q %w", name, errListedTwice)
}
}
apps[hostname] = names
}
return apps, nil
}
// parseRuleIDs reads a comma-separated list of the ids of Core Rule Set // parseRuleIDs reads a comma-separated list of the ids of Core Rule Set
// rules, each a whole number above zero and outside firstSetupRuleID to // rules, each a whole number above zero and outside firstSetupRuleID to
// lastSetupRuleID. // lastSetupRuleID.
@@ -2455,7 +2545,8 @@ func MaskZoneKey(zone string) string {
return masked + "." + rest return masked + "." + rest
} }
// isZone reports whether each label of zone is as parseZones takes it. // isZone reports whether each label of zone is as parseZones takes it. It
// also checks the hostnames of SWW_APPS, which are DNS names too.
func isZone(zone string) bool { func isZone(zone string) bool {
for label := range strings.SplitSeq(zone, ".") { for label := range strings.SplitSeq(zone, ".") {
badChar := strings.ContainsFunc(label, func(char rune) bool { badChar := strings.ContainsFunc(label, func(char rune) bool {
+97
View File
@@ -89,6 +89,7 @@ const (
wafExemptPaths = "SWW_WAF_EXEMPT_PATHS" wafExemptPaths = "SWW_WAF_EXEMPT_PATHS"
wafBodyLimit = "SWW_WAF_BODY_LIMIT" wafBodyLimit = "SWW_WAF_BODY_LIMIT"
exclusionsFile = "SWW_CORE_RULE_SET_EXCLUSIONS_FILE" exclusionsFile = "SWW_CORE_RULE_SET_EXCLUSIONS_FILE"
apps = "SWW_APPS"
trapPaths = "SWW_TRAP_PATHS" trapPaths = "SWW_TRAP_PATHS"
errorBurstThreshold = "SWW_ERROR_BURST_THRESHOLD" errorBurstThreshold = "SWW_ERROR_BURST_THRESHOLD"
logRemoteURL = "SWW_LOG_REMOTE_URL" logRemoteURL = "SWW_LOG_REMOTE_URL"
@@ -582,6 +583,101 @@ func TestCoreRuleSetSettings(t *testing.T) {
} }
} }
// gitea is gitea's name in SWW_APPS.
const gitea = "gitea"
func TestApps(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
env environment
want map[string][]string
}{
{environment{}, nil},
{environment{apps: ""}, map[string][]string{}},
{
environment{apps: "Git.Example.com:gitea; cloud.example.com : gitea "},
map[string][]string{
"git.example.com": {gitea}, "cloud.example.com": {gitea},
},
},
{
environment{apps: "cloud.example.com:gitea,wordpress"},
map[string][]string{"cloud.example.com": {gitea, "wordpress"}},
},
} {
cfg := fromEnvironment(t, tc.env)
if !reflect.DeepEqual(cfg.Apps, tc.want) {
t.Errorf("%v gave %#v, want %#v", tc.env, cfg.Apps, tc.want)
}
}
}
// Unset and set empty, SWW_APPS has opposite effects, so it logs differently.
func TestUnsetAndEmptyAppsAreLoggedDifferently(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
env environment
want string
}{
{environment{}, `"` + apps + `":null`},
{environment{apps: ""}, `"` + apps + `":""`},
} {
var out bytes.Buffer
slog.New(slog.NewJSONHandler(&out, nil)).
Info("starting", "settings", fromEnvironment(t, tc.env))
if !strings.Contains(out.String(), tc.want) {
t.Errorf("%v logged %s, want %s in it", tc.env, out.String(), tc.want)
}
}
}
func TestInvalidAppsStopsTheStart(t *testing.T) {
t.Parallel()
const (
notEntry = " is not a hostname, : and a list of apps, " +
"such as git.example.com:gitea"
notHostname = " is not a DNS name such as git.example.com"
)
for _, tc := range []struct{ value, want string }{
{"git.example.com:gitea;blog.example.com", `"blog.example.com"` + notEntry},
{"git.example.com:gitea;", `""` + notEntry},
{":gitea", `":gitea"` + notEntry},
{"git.example.com:", `"git.example.com:"` + notEntry},
{"*.example.com:gitea", `"*.example.com"` + notHostname},
{"git.example.com/:gitea", `"git.example.com/"` + notHostname},
{"git example.com:gitea", `"git example.com"` + notHostname},
{
"git.example.com:gitea,jellyfin",
`"jellyfin" is not an app sww has fixes for; ` +
`the apps are drupal, gitea, mastodon, mediawiki, nextcloud, ` +
`vaultwarden, wordpress`,
},
{
"git.example.com:gitea;GIT.example.com:gitea",
`"git.example.com" is listed twice`,
},
{"git.example.com:gitea,gitea", `"gitea" is listed twice`},
{"git.example.com:gitea,", `"gitea," has an empty item in its list`},
} {
t.Run(tc.value, func(t *testing.T) {
t.Parallel()
_, err := config.FromEnvironment(environment{apps: tc.value}.lookupEnv)
want := apps + ": " + tc.want
if err == nil || err.Error() != want {
t.Errorf("error %v, want %s", err, want)
}
})
}
}
func TestWAFBodyLimitOf64K(t *testing.T) { func TestWAFBodyLimitOf64K(t *testing.T) {
t.Parallel() t.Parallel()
@@ -2589,6 +2685,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
wafExemptPaths: "", wafExemptPaths: "",
wafBodyLimit: off, wafBodyLimit: off,
exclusionsFile: "", exclusionsFile: "",
apps: "", // logged as null, which decodes as ""
trapPaths: "", trapPaths: "",
errorBurstThreshold: "30", errorBurstThreshold: "30",
logRemoteURL: "", logRemoteURL: "",
+115 -110
View File
@@ -1,9 +1,9 @@
// Package coreruleset runs the OWASP Core Rule Set 4.25.0, through Coraza, on the // Package coreruleset runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
// method, the URL with its query and the headers of a request, and on its // method, the URL with its query and the headers of a request, and on its
// body while SWW_WAF_BODY_LIMIT is set, with the fourteen changes sww // body while SWW_WAF_BODY_LIMIT is set, with the six changes sww
// makes to it and the exclusions in SWW_CORE_RULE_SET_EXCLUSIONS_FILE, as // makes to it, the fixes for the apps SWW_APPS names, and the exclusions in
// "Attack detection" under "Settings" in SPEC.md describes them. It reads no // SWW_CORE_RULE_SET_EXCLUSIONS_FILE, as "Attack detection" under "Settings"
// response. // in SPEC.md describes them. It reads no response.
// //
// sww writes only to its state directory, so Coraza is built with // sww writes only to its state directory, so Coraza is built with
// its no_fs_access tag, as the Dockerfile and script/build build it: of a // its no_fs_access tag, as the Dockerfile and script/build build it: of a
@@ -16,6 +16,8 @@ import (
"errors" "errors"
"fmt" "fmt"
"io" "io"
"maps"
"net"
"net/http" "net/http"
"net/netip" "net/netip"
"os" "os"
@@ -32,11 +34,8 @@ import (
// directives are the Core Rule Set as sww runs it, with the // directives are the Core Rule Set as sww runs it, with the
// paranoia level for %d, bodyDirectives for the first %s while // paranoia level for %d, bodyDirectives for the first %s while
// SWW_WAF_BODY_LIMIT is set, the exclusions file for the second, // SWW_WAF_BODY_LIMIT is set, the exclusions file for the second, and the
// giteaExclusions for the third, wordpressExclusions for the fourth, // directives of every app in apps for the third.
// drupalExclusions for the fifth, mediawikiExclusions for the sixth,
// mastodonExclusions for the seventh, vaultwardenExclusions for the eighth,
// and nextcloudExclusions for the ninth.
// Each rule sww adds has an id from 900000 to 900999, the ids the // Each rule sww adds has an id from 900000 to 900999, the ids the
// Core Rule Set keeps for the rules that set it up, which // Core Rule Set keeps for the rules that set it up, which
// SWW_WAF_DISABLED_RULES and the exclusions file refuse, so that no // SWW_WAF_DISABLED_RULES and the exclusions file refuse, so that no
@@ -84,62 +83,28 @@ SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
# before the rules of phase 1 too. # before the rules of phase 1 too.
%s %s
# The eighth change comes before the rules for the same reason. # The apps' fixes come before the rules for the same reason.
%s %s
# The ninth change comes before the rules for the same reason. # The fifth: only the rules for requests are loaded, and no response is
%s
# So does the tenth.
%s
# So does the eleventh.
%s
# So does the twelfth.
%s
# So does the thirteenth.
%s
# And the fourteenth.
%s
# The sixth: only the rules for requests are loaded, and no response is
# inspected. # inspected.
Include @owasp_crs/REQUEST-*.conf Include @owasp_crs/REQUEST-*.conf
# The third: redirect_uri is not checked for a URL naming an IP address or # The third: redirect_uri is not checked for a URL naming an IP address or
# localhost. Coraza matches a parameter name here, and in the fourth, # localhost. Coraza matches a parameter name here without regard to case.
# without regard to case. ARGS holds the fields of a form data or multipart # ARGS holds the fields of a form data or multipart body Coraza reads as
# body Coraza reads as well as the query parameters, so a field of one of # well as the query parameters, so a field of this name is left out too.
# these names is left out too.
SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri" SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri"
SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri" SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri"
# The fourth: the query parameters in which gitea sends names within a # The fourth: Referer is not checked for a Unix command without arguments,
# repository or its own records, or a page of its own site, are not # for Java starting a process, or for a Java class name. 944130 refused
# checked against the lists of system files, shell paths and command # every page opened from the page of java.lang.String.md.
# names. Coraza takes one rule id per directive.
SecRuleUpdateTargetById 930120 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
!ARGS:artifactName|!ARGS:redirect_to"
SecRuleUpdateTargetById 932160 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
!ARGS:artifactName|!ARGS:redirect_to"
SecRuleUpdateTargetById 932260 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
!ARGS:sub_path|!ARGS:ref|!ARGS:sha|!ARGS:branch|!ARGS:workflow|\
!ARGS:artifactName|!ARGS:redirect_to"
# The fifth, for Referer: it is not checked for a Unix command without
# arguments, for Java starting a process, or for a Java class name. 944130
# refused every page opened from the page of java.lang.String.md. The
# cookies are left out in Inspect.
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer" SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer" SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944130 "!REQUEST_HEADERS:Referer" SecRuleUpdateTargetById 944130 "!REQUEST_HEADERS:Referer"
# The seventh: a container image push that reuses a layer names the image # The sixth: a container image push that reuses a layer names the image
# it takes the layer from in the query parameter from, which is not # it takes the layer from in the query parameter from, which is not
# checked for command names. 932260 refused the push of a layer from # checked for command names. 932260 refused the push of a layer from
# owner/python3-app. # owner/python3-app.
@@ -189,61 +154,61 @@ SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'" severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
` `
// giteaExclusions are the eighth change: the fields of gitea's own // app is an app whose ordinary requests the Core Rule Set would refuse,
// requests that skip the rules that refused them there, as Coraza // with the fixes that let them pass.
// directives. type app struct {
// // directives are Coraza exclusions, loaded before the Core Rule Set's
// rules. Their first rule skips to the marker they end with, unless
// Inspect sets the transaction variable named for the app, such as
// app_gitea. The skip reaches only phase 1, so a rule of a later phase
// must check that variable itself.
directives string
// cookies are the cookies the Core Rule Set reads a request without.
cookies []string
}
//go:embed gitea.conf //go:embed gitea.conf
var giteaExclusions string var giteaDirectives string
// wordpressExclusions are the ninth change: the fields of WordPress's own
// requests that skip the rules that refused them there, as Coraza
// directives.
//
//go:embed wordpress.conf //go:embed wordpress.conf
var wordpressExclusions string var wordpressDirectives string
// drupalExclusions are the tenth change: the fields of Drupal's own
// requests that skip the rules that refused them there, as Coraza
// directives.
//
//go:embed drupal.conf //go:embed drupal.conf
var drupalExclusions string var drupalDirectives string
// mediawikiExclusions are the eleventh change: the fields of MediaWiki's
// own requests that skip the rules that refused them there, as Coraza
// directives.
//
//go:embed mediawiki.conf //go:embed mediawiki.conf
var mediawikiExclusions string var mediawikiDirectives string
// mastodonExclusions are the twelfth change: the fields of Mastodon's own
// requests that skip the rules that refused them there, as Coraza
// directives.
//
//go:embed mastodon.conf //go:embed mastodon.conf
var mastodonExclusions string var mastodonDirectives string
// vaultwardenExclusions are the thirteenth change: the fields of
// Vaultwarden's own requests that skip the rules that refused them there, as
// Coraza directives.
//
//go:embed vaultwarden.conf //go:embed vaultwarden.conf
var vaultwardenExclusions string var vaultwardenDirectives string
// nextcloudExclusions are the fourteenth change: the fields of Nextcloud's
// own requests that skip the rules that refused them there, the WebDAV
// methods it uses, its cookie of the account's name, and its uploads,
// whose bodies are not read, as Coraza directives.
//
//go:embed nextcloud.conf //go:embed nextcloud.conf
var nextcloudExclusions string var nextcloudDirectives string
// cookiesNotRead are the cookies the Core Rule Set reads a request // apps are the apps whose fixes sww has, by the name SWW_APPS gives each.
// without, the rest of the fifth change.
// //
//nolint:gochecknoglobals // a constant cannot be a list //nolint:gochecknoglobals // a constant cannot be a map
var cookiesNotRead = []string{"gitea_flash", "redirect_to"} var apps = map[string]app{
"drupal": {directives: drupalDirectives},
"gitea": {
directives: giteaDirectives,
cookies: []string{"gitea_flash", "redirect_to"},
},
"mastodon": {directives: mastodonDirectives},
"mediawiki": {directives: mediawikiDirectives},
"nextcloud": {directives: nextcloudDirectives},
"vaultwarden": {directives: vaultwardenDirectives},
"wordpress": {directives: wordpressDirectives},
}
// AppNames returns the names of the apps whose fixes sww has, in
// alphabetical order.
func AppNames() []string {
return slices.Sorted(maps.Keys(apps))
}
// Params are what New needs. // Params are what New needs.
type Params struct { type Params struct {
@@ -259,6 +224,10 @@ type Params struct {
// Core Rule Set's rules (SWW_CORE_RULE_SET_EXCLUSIONS_FILE), "" while // Core Rule Set's rules (SWW_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
// there is none. // there is none.
ExclusionsFile string ExclusionsFile string
// Apps are the names of the apps each hostname serves, by hostname in
// lowercase (SWW_APPS). A request gets the fixes of its hostname's
// apps only, or of every app while Apps is nil.
Apps map[string][]string
} }
// The ids of sww's own rules, which an exclusions file may not // The ids of sww's own rules, which an exclusions file may not
@@ -350,13 +319,15 @@ func checkExclusions(exclusions string) error {
type CoreRuleSet struct { type CoreRuleSet struct {
engine coraza.WAF engine coraza.WAF
bodyLimit int64 bodyLimit int64
apps map[string][]string
} }
// New returns the Core Rule Set with the fourteen changes, at params' // New returns the Core Rule Set with the six changes and the apps' fixes,
// paranoia level, without the rules it switches off, reading request // at params' paranoia level, without the rules it switches off, reading
// bodies up to params' limit, and with params' exclusions file loaded // request bodies up to params' limit, and with params' exclusions file
// before the Core Rule Set's rules. An exclusions file is an error if it // loaded before the Core Rule Set's rules. An exclusions file is an error
// cannot be read or does not parse, or if checkExclusions refuses it. // if it cannot be read or does not parse, or if checkExclusions refuses
// it.
func New(params Params) (*CoreRuleSet, error) { func New(params Params) (*CoreRuleSet, error) {
body := "" body := ""
if params.BodyLimit > 0 { if params.BodyLimit > 0 {
@@ -379,10 +350,13 @@ func New(params Params) (*CoreRuleSet, error) {
} }
} }
var fixes strings.Builder
for _, name := range AppNames() {
fixes.WriteString(apps[name].directives + "\n")
}
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions, text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions,
giteaExclusions, wordpressExclusions, drupalExclusions, fixes.String())
mediawikiExclusions, mastodonExclusions, vaultwardenExclusions,
nextcloudExclusions)
if len(params.DisabledRules) > 0 { if len(params.DisabledRules) > 0 {
ids := make([]string, len(params.DisabledRules)) ids := make([]string, len(params.DisabledRules))
@@ -400,7 +374,9 @@ func New(params Params) (*CoreRuleSet, error) {
return nil, fmt.Errorf("load the Core Rule Set: %w", err) return nil, fmt.Errorf("load the Core Rule Set: %w", err)
} }
return &CoreRuleSet{engine: engine, bodyLimit: params.BodyLimit}, nil return &CoreRuleSet{
engine: engine, bodyLimit: params.BodyLimit, apps: params.Apps,
}, nil
} }
// Result is what the Core Rule Set found in a request. // Result is what the Core Rule Set found in a request.
@@ -412,11 +388,12 @@ type Result struct {
Score int Score int
} }
// Inspect runs the Core Rule Set on r, a request from client: on its // Inspect runs the Core Rule Set on r, a request from client, with the
// method, its URL with the query, and its headers, the Cookie header // fixes of the apps appsFor names for r's hostname: on its method, its URL
// without the cookies in cookiesNotRead, and on body, r's body as the // with the query, and its headers, the Cookie header without those apps'
// caller has it, as readBody reads it. It returns what it found, what it // cookies, and on body, r's body as the caller has it, as readBody reads
// read of body, and the error that ended the reading early, if one did. // it. It returns what it found, what it read of body, and the error that
// ended the reading early, if one did.
func (c *CoreRuleSet) Inspect( func (c *CoreRuleSet) Inspect(
r *http.Request, client netip.Addr, body io.Reader, r *http.Request, client netip.Addr, body io.Reader,
) (Result, []byte, error) { ) (Result, []byte, error) {
@@ -424,13 +401,24 @@ func (c *CoreRuleSet) Inspect(
// Closing would remove the files Coraza wrote, and it writes none. // Closing would remove the files Coraza wrote, and it writes none.
defer func() { _ = tx.Close() }() defer func() { _ = tx.Close() }()
// The transaction's variables are set only through Coraza's interface
// for plugins.
state := tx.(plugintypes.TransactionState) //nolint:forcetypeassert // every one is
var cookiesNotRead []string
for _, name := range c.appsFor(r.Host) {
state.Variables().TX().Set("app_"+name, []string{"1"})
cookiesNotRead = append(cookiesNotRead, apps[name].cookies...)
}
tx.ProcessConnection(client.String(), 0, "", 0) tx.ProcessConnection(client.String(), 0, "", 0)
tx.ProcessURI(r.URL.String(), r.Method, r.Proto) tx.ProcessURI(r.URL.String(), r.Method, r.Proto)
for name, values := range r.Header { for name, values := range r.Header {
for _, value := range values { for _, value := range values {
if name == "Cookie" { if name == "Cookie" {
value = withoutCookiesNotRead(value) value = withoutCookies(value, cookiesNotRead)
if value == "" { if value == "" {
continue // it held those cookies alone continue // it held those cookies alone
} }
@@ -523,9 +511,26 @@ func score(tx types.Transaction) int {
return n return n
} }
// withoutCookiesNotRead returns value, a Cookie header's, without the // appsFor returns the names of the apps whose fixes a request for host,
// cookies in cookiesNotRead. // its Host header, gets: every app while SWW_APPS is unset, and otherwise
func withoutCookiesNotRead(value string) string { // those SWW_APPS lists for host's name, without the port and in
// lowercase.
func (c *CoreRuleSet) appsFor(host string) []string {
if c.apps == nil {
return AppNames()
}
hostname, _, err := net.SplitHostPort(host)
if err != nil {
hostname = host // it has no port
}
return c.apps[strings.ToLower(hostname)]
}
// withoutCookies returns value, a Cookie header's, without the cookies
// named in cookiesNotRead.
func withoutCookies(value string, cookiesNotRead []string) string {
var kept []string var kept []string
for cookie := range strings.SplitSeq(value, ";") { for cookie := range strings.SplitSeq(value, ";") {
+299 -8
View File
@@ -1,6 +1,7 @@
package coreruleset_test package coreruleset_test
import ( import (
"encoding/json"
"fmt" "fmt"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
@@ -35,12 +36,14 @@ var loaded = struct {
}{loaders: map[string]func() (*coreruleset.CoreRuleSet, error){}} }{loaders: map[string]func() (*coreruleset.CoreRuleSet, error){}}
// loaderFor returns the function that loads the Core Rule Set with params // loaderFor returns the function that loads the Core Rule Set with params
// once, making it the first time params are asked for. // once, making it the first time params are asked for. %#v writes a nil
// Apps, which gives every app's fixes, apart from an empty one, which
// gives none.
func loaderFor(params coreruleset.Params) func() (*coreruleset.CoreRuleSet, error) { func loaderFor(params coreruleset.Params) func() (*coreruleset.CoreRuleSet, error) {
loaded.Lock() loaded.Lock()
defer loaded.Unlock() defer loaded.Unlock()
settings := fmt.Sprintf("%+v", params) settings := fmt.Sprintf("%#v", params)
load, found := loaded.loaders[settings] load, found := loaded.loaders[settings]
if !found { if !found {
@@ -154,9 +157,10 @@ func inspectBody(
} }
for _, header := range r.headers { for _, header := range r.headers {
// Go's server keeps Host and Transfer-Encoding out of the headers. // Go's server keeps Host and Transfer-Encoding out of the headers,
// whatever their case.
name, value, _ := strings.Cut(header, ": ") name, value, _ := strings.Cut(header, ": ")
switch name { switch http.CanonicalHeaderKey(name) {
case "Host": case "Host":
req.Host = value req.Host = value
case "Transfer-Encoding": case "Transfer-Encoding":
@@ -201,8 +205,8 @@ func atDefaults(t *testing.T) *coreruleset.CoreRuleSet {
} }
// wantChange checks that crs lets through passes, a request one of the // wantChange checks that crs lets through passes, a request one of the
// fourteen changes is for, and still finds result in refused, a request like // six changes or an app's fixes are for, and still finds result in
// it that the change is not for. // refused, a request like it that they are not for.
func wantChange( func wantChange(
t *testing.T, crs *coreruleset.CoreRuleSet, passes, refused request, t *testing.T, crs *coreruleset.CoreRuleSet, passes, refused request,
result coreruleset.Result, result coreruleset.Result,
@@ -463,6 +467,123 @@ func TestImagePushThatReusesALayerIsNotCheckedForCommandNames(t *testing.T) {
matched(930120, 932160)) matched(930120, 932160))
} }
// giteaFixed are gitea requests that only gitea's fixes let through, one
// for each kind of fix, each with what the Core Rule Set finds in it
// without them.
//
//nolint:gochecknoglobals // a constant cannot be a list
var giteaFixed = []struct {
request request
result coreruleset.Result
}{
// A search of code, on the path of gitea's search.
{get("/explore/code?q=SELECT%20*%20FROM%20users%20WHERE"), matched(942100)},
// A file name, in a parameter gitea sends names in, on any path.
{get("/?path=.gitignore"), matched(930120)},
// A file name, in a cookie of gitea's.
{
get("/owner/repo", "Cookie: gitea_flash=success%3DFile%2Bpackage.json%2Bdeleted"),
matched(930120),
},
}
// The hostnames of the tests of SWW_APPS: the one that serves gitea, and
// one that serves another app.
const (
giteaHost = "git.example"
otherHost = "blog.example"
)
// gitea is gitea's name in SWW_APPS.
const gitea = "gitea"
// wordpress is WordPress's name in SWW_APPS.
const wordpress = "wordpress"
// withHost returns r sent to host.
func withHost(r request, host string) request {
return request{r.method, r.target, append([]string{"Host: " + host}, r.headers...)}
}
// servingApps returns the Core Rule Set at the defaults, with SWW_APPS
// giving apps.
func servingApps(t *testing.T, apps map[string][]string) *coreruleset.CoreRuleSet {
t.Helper()
return shared(t, coreruleset.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, Apps: apps,
})
}
func TestAppsUnsetGivesEveryHostnameEveryAppsFixes(t *testing.T) {
t.Parallel()
crs := servingApps(t, nil)
for _, fixed := range giteaFixed {
for _, host := range []string{giteaHost, otherHost} {
wantResult(t, crs, withHost(fixed.request, host), coreruleset.Result{})
}
}
}
func TestListedHostnameGetsItsAppsFixes(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{giteaHost: {gitea}})
// The hostname is matched without the port and without regard to case.
for _, host := range []string{giteaHost, "GIT.Example:8443"} {
for _, fixed := range giteaFixed {
wantResult(t, crs, withHost(fixed.request, host), coreruleset.Result{})
}
}
}
func TestUnlistedHostnameGetsTheFullCheck(t *testing.T) {
t.Parallel()
for _, apps := range []map[string][]string{
{giteaHost: {gitea}},
// SWW_APPS set empty lists no hostname.
{},
} {
crs := servingApps(t, apps)
for _, host := range []string{otherHost, otherHost + ":8443", "git.example.org"} {
for _, fixed := range giteaFixed {
wantResult(t, crs, withHost(fixed.request, host), fixed.result)
}
}
}
}
func TestWordPressFixesApplyOnlyToAHostnameThatServesWordPress(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {wordpress},
})
// A search, which only WordPress's fixes let through.
search := get("/?s=python3")
wantResult(t, crs, withHost(search, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(search, giteaHost), matched(932260))
}
func TestHostnameListingTwoAppsGetsTheFixesOfBoth(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{giteaHost: {gitea, wordpress}})
// A file name that only gitea's fixes let through, and a search that only
// WordPress's fixes let through.
for _, target := range []string{"/?path=.gitignore", "/?s=python3"} {
wantResult(t, crs, withHost(get(target), giteaHost), coreruleset.Result{})
}
}
// withExclusions returns the Core Rule Set at the defaults, with the // withExclusions returns the Core Rule Set at the defaults, with the
// exclusions file at path. // exclusions file at path.
func withExclusions(path string) (*coreruleset.CoreRuleSet, error) { func withExclusions(path string) (*coreruleset.CoreRuleSet, error) {
@@ -572,6 +693,38 @@ func TestExclusionsFileThatCannotBeUsedIsAnError(t *testing.T) {
} }
} }
// giteaRequests returns the recorded ordinary gitea requests in testdata.
// Each has the Referer of a page that names the same, as a page opened
// from it has.
func giteaRequests(t *testing.T) []request {
t.Helper()
text, err := os.ReadFile(filepath.Join("testdata", "gitea-requests.txt"))
if err != nil {
t.Fatal(err)
}
var requests []request
for line := range strings.Lines(string(text)) {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") {
continue
}
target, header, _ := strings.Cut(line, " ")
headers := []string{"Referer: http://git.example" + target}
if header != "" {
headers = append(headers, header)
}
requests = append(requests, get(target, headers...))
}
return requests
}
// An SQL injection, a script injection and a shell command after a ";", as // An SQL injection, a script injection and a shell command after a ";", as
// query values. Every app's tests check that its fields still refuse the // query values. Every app's tests check that its fields still refuse the
// first two. // first two.
@@ -591,13 +744,71 @@ const passwordField = "password="
//nolint:gochecknoglobals // a constant cannot be a list //nolint:gochecknoglobals // a constant cannot be a list
var hardValues = []string{"python3", "powershell", "ls%20-la", ".htaccess"} var hardValues = []string{"python3", "powershell", "ls%20-la", ".htaccess"}
func TestGiteaAttacksAreRefused(t *testing.T) { // wantRecordedRequestsPass checks that none of the recorded ordinary
// requests in the file name in testdata is refused on example.com, where
// they were recorded, while SWW_APPS lists only app for it. They are
// checked at the defaults and while bodies are read. With SWW_APPS unset,
// every app's fixes apply. These only switch checks off, so the requests
// pass then too. A request with neither a body nor a Content-Type header is
// checked only at the defaults: with neither, the set that reads bodies has
// nothing more to check. Each request runs as its own parallel subtest.
func wantRecordedRequestsPass(t *testing.T, name, app string) {
t.Helper()
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
apps := map[string][]string{"example.com": {app}}
defaults := servingApps(t, apps)
bodies := shared(t, coreruleset.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
Apps: apps,
})
for _, recorded := range recordedRequests(t, name) {
t.Run(recorded.Method+" "+recorded.Target, func(t *testing.T) {
t.Parallel()
r := request{recorded.Method, recorded.Target, recorded.Headers}
got, _ := inspectBody(t, defaults, r, recorded.Body)
if got.Score >= threshold {
t.Errorf("%s %s refused: %+v", r.method, r.target, got)
}
typed := slices.ContainsFunc(recorded.Headers, func(header string) bool {
name, _, _ := strings.Cut(header, ":")
return strings.EqualFold(name, "Content-Type")
})
if recorded.Body == "" && !typed {
return
}
got, _ = inspectBody(t, bodies, r, recorded.Body)
if got.Score >= threshold {
t.Errorf("%s %s %.200q refused while bodies are read: %+v",
r.method, r.target, recorded.Body, got)
}
})
}
}
func TestOrdinaryGiteaRequestsPassAndAttacksAreRefused(t *testing.T) {
t.Parallel() t.Parallel()
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
// The recorded requests pass on giteaHost with gitea's fixes alone. The
// attacks are refused at the defaults, where every app's fixes apply.
giteaOnly := servingApps(t, map[string][]string{giteaHost: {gitea}})
crs := atDefaults(t) crs := atDefaults(t)
for _, r := range giteaRequests(t) {
if got := inspect(t, giteaOnly, withHost(r, giteaHost)); got.Score >= threshold {
t.Errorf("%s %q refused: %+v", r.target, r.headers[1:], got)
}
}
// In q too, off gitea's search pages. // In q too, off gitea's search pages.
for _, parameter := range []string{"id", "q"} { for _, parameter := range []string{"id", "q"} {
for _, attack := range []string{ for _, attack := range []string{
@@ -650,6 +861,46 @@ const (
wpTags = "/wp-json/wp/v2/tags" wpTags = "/wp-json/wp/v2/tags"
) )
// recordedRequest is a line of a file of recorded requests in testdata.
type recordedRequest struct {
Method string `json:"method"`
Target string `json:"target"`
Headers []string `json:"headers"`
Body string `json:"body"`
}
// recordedRequests returns the recorded ordinary requests in the file name
// in testdata, each with its body.
func recordedRequests(t *testing.T, name string) []recordedRequest {
t.Helper()
path := filepath.Join("testdata", name)
text, err := os.ReadFile(path) //nolint:gosec // a file in testdata
if err != nil {
t.Fatal(err)
}
var requests []recordedRequest
for line := range strings.Lines(string(text)) {
if strings.HasPrefix(line, "#") {
continue
}
var r recordedRequest
err := json.Unmarshal([]byte(line), &r)
if err != nil {
t.Fatalf("%q: %v", line, err)
}
requests = append(requests, r)
}
return requests
}
// readingBodiesUpTo64K returns the Core Rule Set as sww runs it by default, // readingBodiesUpTo64K returns the Core Rule Set as sww runs it by default,
// but reading bodies up to the largest SWW_WAF_BODY_LIMIT. // but reading bodies up to the largest SWW_WAF_BODY_LIMIT.
func readingBodiesUpTo64K(t *testing.T) *coreruleset.CoreRuleSet { func readingBodiesUpTo64K(t *testing.T) *coreruleset.CoreRuleSet {
@@ -715,11 +966,51 @@ func wordpressAttackTargets(attack string) []string {
} }
} }
func TestWordPressAttacksAreRefused(t *testing.T) { // wantWordPressRequestsPass checks that the recorded ordinary WordPress
// requests pass on otherHost, where they were recorded, while SWW_APPS lists
// only wordpress for it. They pass at the defaults, and those with a body
// pass while bodies up to 64 KiB are read too. Every app's fixes only switch
// checks off, so the requests pass with SWW_APPS unset too.
func wantWordPressRequestsPass(t *testing.T) {
t.Helper()
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
apps := map[string][]string{otherHost: {wordpress}}
defaults := servingApps(t, apps)
bodies := shared(t, coreruleset.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: 64<<10 - 1,
Apps: apps,
})
for _, recorded := range recordedRequests(t, "wordpress-requests.txt") {
t.Run(recorded.Method+" "+recorded.Target, func(t *testing.T) {
t.Parallel()
r := request{recorded.Method, recorded.Target, recorded.Headers}
crsSets := []*coreruleset.CoreRuleSet{defaults}
if recorded.Body != "" {
crsSets = append(crsSets, bodies)
}
for _, crs := range crsSets {
got, _ := inspectBody(t, crs, r, recorded.Body)
if got.Score >= threshold {
t.Errorf("%s %s %.200q refused: %+v", r.method, r.target, recorded.Body, got)
}
}
})
}
}
func TestOrdinaryWordPressRequestsPassAndAttacksAreRefused(t *testing.T) {
t.Parallel() t.Parallel()
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
wantWordPressRequestsPass(t)
defaults, bodies := atDefaults(t), readingBodiesUpTo64K(t) defaults, bodies := atDefaults(t), readingBodiesUpTo64K(t)
for _, attack := range []string{ for _, attack := range []string{
+10 -1
View File
@@ -1,4 +1,4 @@
# The tenth change: on Drupal's paths, some fields skip the rules that refused # Drupal's fixes: on Drupal's paths, some fields skip the rules that refused
# ordinary Drupal requests there. They are the fields where Drupal sends a # ordinary Drupal requests there. They are the fields where Drupal sends a
# search, a name, a file name or an address. Each exclusion runs with the # search, a name, a file name or an address. Each exclusion runs with the
# request, before the rules it switches off, so this file is loaded before the # request, before the rules it switches off, so this file is loaded before the
@@ -25,6 +25,13 @@
# 932230: ls -la # 932230: ls -la
# 932235: python3 # 932235: python3
# Unless SWW_APPS lists drupal for the request's hostname, every rule below is
# skipped. Inspect sets tx.app_drupal when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_drupal itself.
SecRule &TX:app_drupal "@eq 0" \
"id:900790,phase:1,pass,nolog,skipAfter:END_DRUPAL"
# Every admin page's filters and autocompletes, and the page to return to: # Every admin page's filters and autocompletes, and the page to return to:
# title: /admin/content?title=python3 # title: /admin/content?title=python3
# name: /admin/content/media?name=python3 # name: /admin/content/media?name=python3
@@ -237,3 +244,5 @@ SecRule REQUEST_FILENAME "@rx ^(?:/[a-z]{2,3}(?:-[a-z]+)?)?/user/password$" \
SecRule &ARGS_GET:json.mail "@eq 0" \ SecRule &ARGS_GET:json.mail "@eq 0" \
"ctl:ruleRemoveTargetById=932120;ARGS:json.mail,\ "ctl:ruleRemoveTargetById=932120;ARGS:json.mail,\
ctl:ruleRemoveTargetById=932260;ARGS:json.mail" ctl:ruleRemoveTargetById=932260;ARGS:json.mail"
SecMarker END_DRUPAL
+22
View File
@@ -5,6 +5,8 @@ import (
"net/url" "net/url"
"slices" "slices"
"testing" "testing"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
) )
// drupalThreshold is SWW_WAF_ANOMALY_THRESHOLD's default. // drupalThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
@@ -31,6 +33,26 @@ const drupalEditAccount = "/user/2/edit"
// drupalName is the field for a person's name, as the start of a form body. // drupalName is the field for a person's name, as the start of a form body.
const drupalName = "name=" const drupalName = "name="
func TestOrdinaryDrupalRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "drupal-requests.txt", "drupal")
}
func TestDrupalFixesApplyOnlyToAHostnameThatServesDrupal(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {"drupal"},
})
// A search, which only Drupal's fixes let through.
search := get("/search?keys=python3")
wantResult(t, crs, withHost(search, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(search, giteaHost), matched(932260))
}
// drupalAttacks are an SQL injection, a path traversal and a script // drupalAttacks are an SQL injection, a path traversal and a script
// injection, as a query value. // injection, as a query value.
// //
+56 -7
View File
@@ -1,8 +1,55 @@
# The eighth change: on gitea's paths, some fields skip the rules that refused # gitea's fixes: some fields skip the rules that refused ordinary gitea requests.
# ordinary gitea requests there. They are the fields where gitea sends a search, # They are the fields where gitea sends a search, a file name or a branch name.
# a file name or a branch name. Each exclusion runs with the request, before the # Each exclusion runs with the request, before the rules it switches off, those
# rules it switches off, those of phase 1 included, so this file is loaded before # of phase 1 included, so this file is loaded before the Core Rule Set's rules.
# the Core Rule Set's rules. Each comment names a request the rule refused. # Each comment names a request the rule refused.
# Unless SWW_APPS lists gitea for the request's hostname, every rule below is
# skipped. Inspect sets tx.app_gitea when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_gitea itself.
SecRule &TX:app_gitea "@eq 0" \
"id:900590,phase:1,pass,nolog,skipAfter:END_GITEA"
# The query parameters in which gitea sends names within a repository or its
# own records, or a page of its own site, are not checked against the lists of
# system files, shell paths and command names, on any path:
# 930120: .gitignore
# 932160: bin/docker-entrypoint
# 932260: docker-build
# ARGS holds the fields of a form data or multipart body Coraza reads as well
# as the query parameters, so a field of these names is left out too.
SecAction "id:900591,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:path,\
ctl:ruleRemoveTargetById=930120;ARGS:files,\
ctl:ruleRemoveTargetById=930120;ARGS:skip-to,\
ctl:ruleRemoveTargetById=930120;ARGS:sub_path,\
ctl:ruleRemoveTargetById=930120;ARGS:ref,\
ctl:ruleRemoveTargetById=930120;ARGS:sha,\
ctl:ruleRemoveTargetById=930120;ARGS:branch,\
ctl:ruleRemoveTargetById=930120;ARGS:workflow,\
ctl:ruleRemoveTargetById=930120;ARGS:artifactName,\
ctl:ruleRemoveTargetById=930120;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=932160;ARGS:path,\
ctl:ruleRemoveTargetById=932160;ARGS:files,\
ctl:ruleRemoveTargetById=932160;ARGS:skip-to,\
ctl:ruleRemoveTargetById=932160;ARGS:sub_path,\
ctl:ruleRemoveTargetById=932160;ARGS:ref,\
ctl:ruleRemoveTargetById=932160;ARGS:sha,\
ctl:ruleRemoveTargetById=932160;ARGS:branch,\
ctl:ruleRemoveTargetById=932160;ARGS:workflow,\
ctl:ruleRemoveTargetById=932160;ARGS:artifactName,\
ctl:ruleRemoveTargetById=932160;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=932260;ARGS:path,\
ctl:ruleRemoveTargetById=932260;ARGS:files,\
ctl:ruleRemoveTargetById=932260;ARGS:skip-to,\
ctl:ruleRemoveTargetById=932260;ARGS:sub_path,\
ctl:ruleRemoveTargetById=932260;ARGS:ref,\
ctl:ruleRemoveTargetById=932260;ARGS:sha,\
ctl:ruleRemoveTargetById=932260;ARGS:branch,\
ctl:ruleRemoveTargetById=932260;ARGS:workflow,\
ctl:ruleRemoveTargetById=932260;ARGS:artifactName,\
ctl:ruleRemoveTargetById=932260;ARGS:redirect_to"
# gitea's searches of code, of issues and pull requests, and of commits send the # gitea's searches of code, of issues and pull requests, and of commits send the
# text searched for in q, which skips these rules there: # text searched for in q, which skips these rules there:
@@ -163,8 +210,8 @@ SecRule REQUEST_FILENAME \
# 942140: information_schema.sql # 942140: information_schema.sql
# 944110: java/lang/Runtime.java # 944110: java/lang/Runtime.java
# 944130: java.lang.String.md # 944130: java.lang.String.md
# Each request skips the rules that refused it. The fourth change already has # Each request skips the rules that refused it. Rule 900591 already has path and
# path and redirect_to skip 930120, 932160 and 932260 on every path. # redirect_to skip 930120, 932160 and 932260 on every path.
# A file's commit graph, in file. # A file's commit graph, in file.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/graph$" \ SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/graph$" \
@@ -289,3 +336,5 @@ SecRule REQUEST_FILENAME \
"@rx ^/api/actions_pipeline/_apis/pipelines/workflows/[0-9]+/artifacts/[^/]+/upload$" \ "@rx ^/api/actions_pipeline/_apis/pipelines/workflows/[0-9]+/artifacts/[^/]+/upload$" \
"id:900640,phase:1,pass,nolog,\ "id:900640,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=920450;REQUEST_HEADERS_NAMES:Content-Range" ctl:ruleRemoveTargetById=920450;REQUEST_HEADERS_NAMES:Content-Range"
SecMarker END_GITEA
+10 -1
View File
@@ -1,4 +1,4 @@
# The twelfth change: on Mastodon's paths, some fields skip the rules that # Mastodon's fixes: on Mastodon's paths, some fields skip the rules that
# refused ordinary Mastodon requests there. They are the fields where Mastodon # refused ordinary Mastodon requests there. They are the fields where Mastodon
# sends a search, a name, a title, a file name, an address or free text. Each # sends a search, a name, a title, a file name, an address or free text. Each
# exclusion runs with the request, before the rules it switches off, so this # exclusion runs with the request, before the rules it switches off, so this
@@ -17,6 +17,13 @@
# 932250: ls -la # 932250: ls -la
# 932260: python3 # 932260: python3
# Unless SWW_APPS lists mastodon for the request's hostname, every rule below is
# skipped. Inspect sets tx.app_mastodon when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_mastodon itself.
SecRule &TX:app_mastodon "@eq 0" \
"id:900940,phase:1,pass,nolog,skipAfter:END_MASTODON"
# The API's searches and names, in the query: # The API's searches and names, in the query:
# q: /api/v2/search?q=python3, /api/v1/accounts/search?q=ls+-la # q: /api/v2/search?q=python3, /api/v1/accounts/search?q=ls+-la
# acct: /api/v1/accounts/lookup?acct=python3 # acct: /api/v1/accounts/lookup?acct=python3
@@ -216,3 +223,5 @@ SecRule REQUEST_FILENAME "@rx ^/auth(?:/|$)" \
"ctl:ruleRemoveTargetById=930120;ARGS:/^user\[email\]$/,\ "ctl:ruleRemoveTargetById=930120;ARGS:/^user\[email\]$/,\
ctl:ruleRemoveTargetById=932120;ARGS:/^user\[email\]$/,\ ctl:ruleRemoveTargetById=932120;ARGS:/^user\[email\]$/,\
ctl:ruleRemoveTargetById=932260;ARGS:/^user\[email\]$/" ctl:ruleRemoveTargetById=932260;ARGS:/^user\[email\]$/"
SecMarker END_MASTODON
+20
View File
@@ -54,6 +54,26 @@ const (
mastodonStatus = "status=" mastodonStatus = "status="
) )
func TestOrdinaryMastodonRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "mastodon-requests.txt", "mastodon")
}
func TestMastodonFixesApplyOnlyToAHostnameThatServesMastodon(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {"mastodon"},
})
// A search, which only Mastodon's fixes let through.
search := get("/api/v2/search?q=python3")
wantResult(t, crs, withHost(search, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(search, giteaHost), matched(932260))
}
// mastodonQueryFields sends each field that skips 930120, 932120, 932250 // mastodonQueryFields sends each field that skips 930120, 932120, 932250
// and 932260 in the query, on a path where it skips them, ending in "=". // and 932260 in the query, on a path where it skips them, ending in "=".
// //
+13 -2
View File
@@ -1,4 +1,4 @@
# The eleventh change: on MediaWiki's paths, some fields skip the rules that # MediaWiki's fixes: on MediaWiki's paths, some fields skip the rules that
# refused ordinary MediaWiki requests there. They are the fields where MediaWiki # refused ordinary MediaWiki requests there. They are the fields where MediaWiki
# sends a search, a title, a name, a file name, an address, a summary or a # sends a search, a title, a name, a file name, an address, a summary or a
# reason, and the fields that hold wikitext. Each exclusion runs with the # reason, and the fields that hold wikitext. Each exclusion runs with the
@@ -21,6 +21,13 @@
# 932250: ls -la # 932250: ls -la
# 932260: python3 # 932260: python3
# Unless SWW_APPS lists mediawiki for the request's hostname, every rule below
# is skipped. Inspect sets tx.app_mediawiki when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_mediawiki itself.
SecRule &TX:app_mediawiki "@eq 0" \
"id:900890,phase:1,pass,nolog,skipAfter:END_MEDIAWIKI"
# The pages and their titles, the searches, and the special pages' fields # The pages and their titles, the searches, and the special pages' fields
# for a page, a user, a file or an address: # for a page, a user, a file or an address:
# title: /index.php?title=Python3_notes&action=history # title: /index.php?title=Python3_notes&action=history
@@ -198,10 +205,12 @@ SecRule REQUEST_FILENAME "@rx ^/(?:w/)?index\.php(?:$|/.)|^/wiki/" \
# This rule is in phase 2, because a phase 1 rule runs before the body is read # This rule is in phase 2, because a phase 1 rule runs before the body is read
# and cannot see what wpTitles holds. It checks that the body sends wpTitles # and cannot see what wpTitles holds. It checks that the body sends wpTitles
# once, since it would otherwise skip these rules for a second wpTitles # once, since it would otherwise skip these rules for a second wpTitles
# holding a command. # holding a command. It checks tx.app_mediawiki, since the skip above does not
# reach phase 2.
SecRule REQUEST_FILENAME "@rx ^/(?:w/)?index\.php(?:$|/.)|^/wiki/" \ SecRule REQUEST_FILENAME "@rx ^/(?:w/)?index\.php(?:$|/.)|^/wiki/" \
"id:900916,phase:2,pass,nolog,chain" "id:900916,phase:2,pass,nolog,chain"
SecRule REQUEST_FILENAME|ARGS_GET_NAMES "@rx ^/(?:w/)?index\.php/.|^/wiki/|^title$" "chain" SecRule REQUEST_FILENAME|ARGS_GET_NAMES "@rx ^/(?:w/)?index\.php/.|^/wiki/|^title$" "chain"
SecRule &TX:app_mediawiki "@eq 1" "chain"
SecRule &ARGS_GET:wpTitles "@eq 0" "chain" SecRule &ARGS_GET:wpTitles "@eq 0" "chain"
SecRule &ARGS_POST:wpTitles "@eq 1" "chain" SecRule &ARGS_POST:wpTitles "@eq 1" "chain"
SecRule ARGS_POST:wpTitles "!@rx [;|&`{]" \ SecRule ARGS_POST:wpTitles "!@rx [;|&`{]" \
@@ -210,3 +219,5 @@ SecRule REQUEST_FILENAME "@rx ^/(?:w/)?index\.php(?:$|/.)|^/wiki/" \
ctl:ruleRemoveTargetById=932235;ARGS:wpTitles,\ ctl:ruleRemoveTargetById=932235;ARGS:wpTitles,\
ctl:ruleRemoveTargetById=932260;ARGS:wpTitles,\ ctl:ruleRemoveTargetById=932260;ARGS:wpTitles,\
ctl:ruleRemoveTargetById=932380;ARGS:wpTitles" ctl:ruleRemoveTargetById=932380;ARGS:wpTitles"
SecMarker END_MEDIAWIKI
+48
View File
@@ -4,11 +4,16 @@ import (
"net/http" "net/http"
"slices" "slices"
"testing" "testing"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
) )
// mediawikiThreshold is SWW_WAF_ANOMALY_THRESHOLD's default. // mediawikiThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
const mediawikiThreshold = 5 const mediawikiThreshold = 5
// mediawiki is MediaWiki's name in SWW_APPS.
const mediawiki = "mediawiki"
// mediawikiEdit is the address MediaWiki's edit form posts to. // mediawikiEdit is the address MediaWiki's edit form posts to.
const mediawikiEdit = "/index.php?title=Example&action=submit" const mediawikiEdit = "/index.php?title=Example&action=submit"
@@ -42,6 +47,49 @@ const (
mediawikiUploadPage = "/index.php/Special:Upload" mediawikiUploadPage = "/index.php/Special:Upload"
) )
func TestOrdinaryMediaWikiRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "mediawiki-requests.txt", mediawiki)
}
func TestMediaWikiFixesApplyOnlyToAHostnameThatServesMediaWiki(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {mediawiki},
})
// A search, which only MediaWiki's fixes let through.
search := get("/index.php?title=Special:Search&search=python3")
wantResult(t, crs, withHost(search, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(search, giteaHost), matched(932260))
}
func TestMediaWikiWatchlistFixAppliesOnlyToAHostnameThatServesMediaWiki(t *testing.T) {
t.Parallel()
crs := shared(t, coreruleset.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
Apps: map[string][]string{giteaHost: {gitea}, otherHost: {mediawiki}},
})
// A watchlist that only the phase 2 rule 900916 lets through.
r := request{http.MethodPost, mediawikiWatchlist.target, []string{formData}}
body := mediawikiWatchlist.field + "Lighthouse%20notes%0D%0APython3%20notes"
got, _ := inspectBody(t, crs, withHost(r, otherHost), body)
if got.Score >= mediawikiThreshold {
t.Errorf("POST %s %s on %s refused: %+v", r.target, body, otherHost, got)
}
got, _ = inspectBody(t, crs, withHost(r, giteaHost), body)
if got.Score < mediawikiThreshold {
t.Errorf("POST %s %s on %s let through: %+v", r.target, body, giteaHost, got)
}
}
// mediawikiQueryFields and mediawikiBodyFields together hold every field that // mediawikiQueryFields and mediawikiBodyFields together hold every field that
// skips 930120, 932120, 932250 and 932260, other than wikitext, wpTitles and // skips 930120, 932120, 932250 and 932260, other than wikitext, wpTitles and
// the JSON fields of /rest.php/. mediawikiQueryFields sends each in the query, on a // the JSON fields of /rest.php/. mediawikiQueryFields sends each in the query, on a
+10 -1
View File
@@ -1,4 +1,4 @@
# The fourteenth change: on Nextcloud's paths, some fields skip the rules that # Nextcloud's fixes: on Nextcloud's paths, some fields skip the rules that
# refused ordinary Nextcloud requests there. They are the fields where Nextcloud # refused ordinary Nextcloud requests there. They are the fields where Nextcloud
# sends a search, a name, a file name, a folder, a comment, a note, a status or # sends a search, a name, a file name, a folder, a comment, a note, a status or
# a document's text. Each exclusion runs with the request, before the rules it # a document's text. Each exclusion runs with the request, before the rules it
@@ -29,6 +29,13 @@
# "; wget http://example.com/x.sh" reaches Nextcloud in these fields. SQL # "; wget http://example.com/x.sh" reaches Nextcloud in these fields. SQL
# injection, path traversal and script injection are still refused there. # injection, path traversal and script injection are still refused there.
# Unless SWW_APPS lists nextcloud for the request's hostname, every rule below
# is skipped. Inspect sets tx.app_nextcloud when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_nextcloud itself.
SecRule &TX:app_nextcloud "@eq 0" \
"id:900090,phase:1,pass,nolog,skipAfter:END_NEXTCLOUD"
# The sync client and the Files app use these WebDAV methods besides those # The sync client and the Files app use these WebDAV methods besides those
# the first change allows, on /remote.php/dav/ and /public.php/dav/ and on the # the first change allows, on /remote.php/dav/ and /public.php/dav/ and on the
# older /remote.php/webdav/ and /public.php/webdav/. 911100 refused each: # older /remote.php/webdav/ and /public.php/webdav/. 911100 refused each:
@@ -306,3 +313,5 @@ SecRule REQUEST_FILENAME "@rx ^/remote\.php/dav/addressbooks/" \
SecRule REQUEST_METHOD "@streq MKCOL" \ SecRule REQUEST_METHOD "@streq MKCOL" \
"ctl:ruleRemoveTargetById=932125;XML:/*,\ "ctl:ruleRemoveTargetById=932125;XML:/*,\
ctl:ruleRemoveTargetById=932230;XML:/*" ctl:ruleRemoveTargetById=932230;XML:/*"
SecMarker END_NEXTCLOUD
+43
View File
@@ -48,6 +48,49 @@ const (
// nextcloudOther is a JSON field no exclusion names. // nextcloudOther is a JSON field no exclusion names.
const nextcloudOther = `{"other":"python3"}` const nextcloudOther = `{"other":"python3"}`
// nextcloud is Nextcloud's name in SWW_APPS.
const nextcloud = "nextcloud"
func TestOrdinaryNextcloudRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "nextcloud-requests.txt", nextcloud)
}
func TestNextcloudFixesApplyOnlyToAHostnameThatServesNextcloud(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {nextcloud},
})
// A search, which only Nextcloud's fixes let through.
search := get(nextcloudUsers + "?search=python3")
wantResult(t, crs, withHost(search, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(search, giteaHost), matched(932260))
}
func TestNextcloudWebDAVMethodsPassOnlyOnHostnamesThatServeNextcloud(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {nextcloud},
})
// Listing a folder, as the sync client does.
list := request{nextcloudPropFind, nextcloudFiles, nil}
wantResult(t, crs, withHost(list, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(list, giteaHost), matched(911100))
// Making a calendar, as a CalDAV client does.
calendar := request{nextcloudMkcalendar, nextcloudCalendars + "work/", nil}
wantResult(t, crs, withHost(calendar, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(calendar, giteaHost), matched(911100))
}
// nextcloudQueryFields are the query fields that skip 930120, 932120, 932250 // nextcloudQueryFields are the query fields that skip 930120, 932120, 932250
// and 932260, each on a path where it skips them, ending in "=". // and 932260, each on a path where it skips them, ending in "=".
// //
-204
View File
@@ -1,204 +0,0 @@
//go:build recorded
// The tests that check the apps' recorded requests in testdata. The recorded
// tag keeps them out of the test phase's pass with the race detector, and
// its second pass, without the race detector, runs them.
package coreruleset_test
import (
"encoding/json"
"os"
"path/filepath"
"slices"
"strings"
"testing"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
)
// giteaRequests returns the recorded ordinary gitea requests in testdata.
// Each has the Referer of a page that names the same, as a page opened
// from it has.
func giteaRequests(t *testing.T) []request {
t.Helper()
text, err := os.ReadFile(filepath.Join("testdata", "gitea-requests.txt"))
if err != nil {
t.Fatal(err)
}
var requests []request
for line := range strings.Lines(string(text)) {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") {
continue
}
target, header, _ := strings.Cut(line, " ")
headers := []string{"Referer: http://git.example" + target}
if header != "" {
headers = append(headers, header)
}
requests = append(requests, get(target, headers...))
}
return requests
}
func TestOrdinaryGiteaRequestsPass(t *testing.T) {
t.Parallel()
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
crs := atDefaults(t)
for _, r := range giteaRequests(t) {
if got := inspect(t, crs, r); got.Score >= threshold {
t.Errorf("%s %q refused: %+v", r.target, r.headers[1:], got)
}
}
}
// recordedRequest is a line of a file of recorded requests in testdata.
type recordedRequest struct {
Method string `json:"method"`
Target string `json:"target"`
Headers []string `json:"headers"`
Body string `json:"body"`
}
// recordedRequests returns the recorded ordinary requests in the file name
// in testdata, each with its body.
func recordedRequests(t *testing.T, name string) []recordedRequest {
t.Helper()
path := filepath.Join("testdata", name)
text, err := os.ReadFile(path) //nolint:gosec // a file in testdata
if err != nil {
t.Fatal(err)
}
var requests []recordedRequest
for line := range strings.Lines(string(text)) {
if strings.HasPrefix(line, "#") {
continue
}
var r recordedRequest
err := json.Unmarshal([]byte(line), &r)
if err != nil {
t.Fatalf("%q: %v", line, err)
}
requests = append(requests, r)
}
return requests
}
// wantRecordedRequestsPass checks that none of the recorded ordinary
// requests in the file name in testdata is refused, at the defaults and
// while bodies are read. A request with neither a body nor a Content-Type
// header is checked only at the defaults: with neither, the set that reads
// bodies has nothing more to check. Each request runs as its own parallel
// subtest.
func wantRecordedRequestsPass(t *testing.T, name string) {
t.Helper()
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
defaults, bodies := atDefaults(t), readingBodies(t)
for _, recorded := range recordedRequests(t, name) {
t.Run(recorded.Method+" "+recorded.Target, func(t *testing.T) {
t.Parallel()
r := request{recorded.Method, recorded.Target, recorded.Headers}
got, _ := inspectBody(t, defaults, r, recorded.Body)
if got.Score >= threshold {
t.Errorf("%s %s refused: %+v", r.method, r.target, got)
}
typed := slices.ContainsFunc(recorded.Headers, func(header string) bool {
name, _, _ := strings.Cut(header, ":")
return strings.EqualFold(name, "Content-Type")
})
if recorded.Body == "" && !typed {
return
}
got, _ = inspectBody(t, bodies, r, recorded.Body)
if got.Score >= threshold {
t.Errorf("%s %s %.200q refused while bodies are read: %+v",
r.method, r.target, recorded.Body, got)
}
})
}
}
func TestOrdinaryDrupalRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "drupal-requests.txt")
}
func TestOrdinaryMastodonRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "mastodon-requests.txt")
}
func TestOrdinaryMediaWikiRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "mediawiki-requests.txt")
}
func TestOrdinaryNextcloudRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "nextcloud-requests.txt")
}
func TestOrdinaryVaultwardenRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "vaultwarden-requests.txt")
}
func TestOrdinaryWordPressRequestsPass(t *testing.T) {
t.Parallel()
const threshold = 5 // SWW_WAF_ANOMALY_THRESHOLD's default
defaults, bodies := atDefaults(t), readingBodiesUpTo64K(t)
for _, recorded := range recordedRequests(t, "wordpress-requests.txt") {
t.Run(recorded.Method+" "+recorded.Target, func(t *testing.T) {
t.Parallel()
r := request{recorded.Method, recorded.Target, recorded.Headers}
crsSets := []*coreruleset.CoreRuleSet{defaults}
if recorded.Body != "" {
crsSets = append(crsSets, bodies)
}
for _, crs := range crsSets {
got, _ := inspectBody(t, crs, r, recorded.Body)
if got.Score >= threshold {
t.Errorf("%s %s %.200q refused: %+v", r.method, r.target, recorded.Body, got)
}
}
})
}
}
+10 -1
View File
@@ -1,4 +1,4 @@
# The thirteenth change: on Vaultwarden's paths, some fields skip the rules that # Vaultwarden's fixes: on Vaultwarden's paths, some fields skip the rules that
# refused ordinary Vaultwarden requests there. They are the fields where # refused ordinary Vaultwarden requests there. They are the fields where
# Vaultwarden receives a name, a password hint or an email address in plain # Vaultwarden receives a name, a password hint or an email address in plain
# text, and the names of uploaded files. Each exclusion runs with the request, # text, and the names of uploaded files. Each exclusion runs with the request,
@@ -17,6 +17,13 @@
# 932250: ls -la # 932250: ls -la
# 932260: python3 # 932260: python3
# Unless SWW_APPS lists vaultwarden for the request's hostname, every rule below
# is skipped. Inspect sets tx.app_vaultwarden when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_vaultwarden itself.
SecRule &TX:app_vaultwarden "@eq 0" \
"id:900140,phase:1,pass,nolog,skipAfter:END_VAULTWARDEN"
# Names and password hints, in JSON bodies, skip the four rules: # Names and password hints, in JSON bodies, skip the four rules:
# name: signing up, editing the account's profile, and creating or renaming # name: signing up, editing the account's profile, and creating or renaming
# an organization or a group, "python3 .htaccess team" # an organization or a group, "python3 .htaccess team"
@@ -82,3 +89,5 @@ SecRule REQUEST_FILENAME "@rx ^/api/hibp/breach$" \
SecRule REQUEST_FILENAME "@rx ^/api/ciphers/[0-9a-f-]{36}/attachment(?:-admin|/[0-9a-f]+(?:/share)?)?$|^/api/sends/[0-9a-f-]{36}/file/[0-9a-f]+$" \ SecRule REQUEST_FILENAME "@rx ^/api/ciphers/[0-9a-f-]{36}/attachment(?:-admin|/[0-9a-f]+(?:/share)?)?$|^/api/sends/[0-9a-f-]{36}/file/[0-9a-f]+$" \
"id:900153,phase:1,pass,nolog,\ "id:900153,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=920120;FILES" ctl:ruleRemoveTargetById=920120;FILES"
SecMarker END_VAULTWARDEN
+23
View File
@@ -7,6 +7,8 @@ import (
"slices" "slices"
"strings" "strings"
"testing" "testing"
"sneak.berlin/go/smallwebwaf/internal/coreruleset"
) )
// vaultwardenThreshold is SWW_WAF_ANOMALY_THRESHOLD's default. // vaultwardenThreshold is SWW_WAF_ANOMALY_THRESHOLD's default.
@@ -27,6 +29,27 @@ const (
// vaultwardenEmail is a JSON body that sends an address in email. // vaultwardenEmail is a JSON body that sends an address in email.
const vaultwardenEmail = `{"email":"VALUE"}` const vaultwardenEmail = `{"email":"VALUE"}`
func TestOrdinaryVaultwardenRequestsPass(t *testing.T) {
t.Parallel()
wantRecordedRequestsPass(t, "vaultwarden-requests.txt", "vaultwarden")
}
func TestVaultwardenFixesApplyOnlyToAHostnameThatServesVaultwarden(t *testing.T) {
t.Parallel()
crs := servingApps(t, map[string][]string{
giteaHost: {gitea}, otherHost: {"vaultwarden"},
})
// An address for the data breach report, which only Vaultwarden's fixes
// let through.
report := get(vaultwardenBreachReport + "python3%40example.com")
wantResult(t, crs, withHost(report, otherHost), coreruleset.Result{})
wantResult(t, crs, withHost(report, giteaHost), matched(932260))
}
// vaultwardenField is a field Vaultwarden receives in plain text: the // vaultwardenField is a field Vaultwarden receives in plain text: the
// request that sends it, and its body, with VALUE where the field's value // request that sends it, and its body, with VALUE where the field's value
// goes. // goes.
+10 -1
View File
@@ -1,10 +1,17 @@
# The ninth change: on WordPress's paths, some fields skip the rules that refused # WordPress's fixes: on WordPress's paths, some fields skip the rules that refused
# ordinary WordPress requests there. Each exclusion runs with the request, before # ordinary WordPress requests there. Each exclusion runs with the request, before
# the rules it switches off, so this file is loaded before the Core Rule Set's # the rules it switches off, so this file is loaded before the Core Rule Set's
# rules. Each comment names a request the rule refused. An exclusion on a field # rules. Each comment names a request the rule refused. An exclusion on a field
# of a body applies only when the query has no parameter of that name, so it # of a body applies only when the query has no parameter of that name, so it
# matters only while SWW_WAF_BODY_LIMIT is set. # matters only while SWW_WAF_BODY_LIMIT is set.
# Unless SWW_APPS lists wordpress for the request's hostname, every rule below is
# skipped. Inspect sets tx.app_wordpress when it does.
# The skip reaches only phase 1, so a rule of a later phase must check
# tx.app_wordpress itself.
SecRule &TX:app_wordpress "@eq 0" \
"id:900690,phase:1,pass,nolog,skipAfter:END_WORDPRESS"
# WordPress takes a call to its REST API at any of four addresses: # WordPress takes a call to its REST API at any of four addresses:
# /wp-json/wp/v2/posts/1 # /wp-json/wp/v2/posts/1
# /index.php/wp-json/wp/v2/posts/1 # /index.php/wp-json/wp/v2/posts/1
@@ -195,3 +202,5 @@ SecRule REQUEST_FILENAME "@rx ^/(?:(?:index\.php/)?page/[0-9]+/?|index\.php)?$"
# so skips 932340 there. # so skips 932340 there.
SecRule REQUEST_FILENAME "@streq /wp-admin/upload.php" \ SecRule REQUEST_FILENAME "@streq /wp-admin/upload.php" \
"id:900711,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932340;ARGS:_wp_http_referer" "id:900711,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932340;ARGS:_wp_http_referer"
SecMarker END_WORDPRESS
@@ -10,6 +10,7 @@ import (
"testing" "testing"
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/discriminator"
"sneak.berlin/go/smallwebwaf/internal/ratelimit" "sneak.berlin/go/smallwebwaf/internal/ratelimit"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
) )
@@ -21,6 +22,7 @@ const (
wafDisabledRules = "SWW_WAF_DISABLED_RULES" wafDisabledRules = "SWW_WAF_DISABLED_RULES"
wafExemptPaths = "SWW_WAF_EXEMPT_PATHS" wafExemptPaths = "SWW_WAF_EXEMPT_PATHS"
wafBodyLimit = "SWW_WAF_BODY_LIMIT" wafBodyLimit = "SWW_WAF_BODY_LIMIT"
apps = "SWW_APPS"
block = "block" block = "block"
detect = "detect" detect = "detect"
) )
@@ -290,6 +292,27 @@ func TestDisabledRulesSwitchOffWhatGiteaWouldBeRefused(t *testing.T) {
} }
} }
func TestGiteasFixesApplyOnlyToAHostnameThatServesGitea(t *testing.T) {
t.Parallel()
// Only gitea's fixes let a file name through in path, a parameter
// gitea sends file names in.
const fileName = "/?path=.gitignore"
s := startDiscriminator(t, "", map[string]string{
wafMode: block, apps: siteA + ":gitea",
})
wantDisposition(t, do(t, s.question(siteA, client, fileName)),
http.StatusOK, discriminator.DispositionAllow)
wantDisposition(t, do(t, s.question(siteB, client, fileName)),
http.StatusForbidden, discriminator.DispositionDeny)
lines := s.out.requestLines(t, 2)
wantLine(t, lines[0], http.StatusOK, requestlog.ActionForward)
wantLine(t, lines[1], http.StatusForbidden, requestlog.ActionBlockedByCoreRuleSet)
}
func TestAttackInAFormBodyIsRefusedOnlyWhileBodiesAreRead(t *testing.T) { func TestAttackInAFormBodyIsRefusedOnlyWhileBodiesAreRead(t *testing.T) {
t.Parallel() t.Parallel()
+3 -1
View File
@@ -221,7 +221,8 @@ func newReputation(
// NewCoreRuleSet returns the Core Rule Set at SWW_WAF_PARANOIA_LEVEL, // NewCoreRuleSet returns the Core Rule Set at SWW_WAF_PARANOIA_LEVEL,
// without the rules SWW_WAF_DISABLED_RULES switches off, reading bodies // without the rules SWW_WAF_DISABLED_RULES switches off, reading bodies
// up to SWW_WAF_BODY_LIMIT, with the exclusions in // up to SWW_WAF_BODY_LIMIT, with the exclusions in
// SWW_CORE_RULE_SET_EXCLUSIONS_FILE, or nil while SWW_WAF_MODE is off. It // SWW_CORE_RULE_SET_EXCLUSIONS_FILE and the fixes of the apps in SWW_APPS,
// or nil while SWW_WAF_MODE is off. It
// reads at most QuestionBodyMaxBytes less one byte: a question's body of // reads at most QuestionBodyMaxBytes less one byte: a question's body of
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a // QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
// JSON or XML body over its limit. An exclusions file that coreruleset.New refuses is // JSON or XML body over its limit. An exclusions file that coreruleset.New refuses is
@@ -236,6 +237,7 @@ func NewCoreRuleSet(cfg *config.Config) (*coreruleset.CoreRuleSet, error) {
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules, ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1), BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
ExclusionsFile: cfg.CoreRuleSetExclusionsFile, ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
Apps: cfg.Apps,
}) })
if err != nil { if err != nil {
return nil, fmt.Errorf("SWW_CORE_RULE_SET_EXCLUSIONS_FILE: %w", err) return nil, fmt.Errorf("SWW_CORE_RULE_SET_EXCLUSIONS_FILE: %w", err)