sneak/smallwebwaf
1
0
Fork 0
Code Issues 19 Pull Requests 4 Actions Packages Projects Releases Wiki Activity

Compare commits

base: sneak/smallwebwaf:issue-211-viper-config
Branches Tags
sneak/smallwebwaf:main sneak/smallwebwaf:next sneak/smallwebwaf:issue-218-renames sneak/smallwebwaf:issue-211-viper-config sneak/smallwebwaf:publish-image sneak/smallwebwaf:issue-195-heuristic-modules sneak/smallwebwaf:readme-rewrite-155
sneak/smallwebwaf:v1.0.0-rc2 sneak/smallwebwaf:v1.0.0-rc1
..
compare: sneak/smallwebwaf:1d75ad2d48ed072a842ead5152ef7048da46fcf3
Branches Tags
sneak/smallwebwaf:next sneak/smallwebwaf:issue-218-renames sneak/smallwebwaf:issue-211-viper-config sneak/smallwebwaf:publish-image sneak/smallwebwaf:main sneak/smallwebwaf:issue-195-heuristic-modules sneak/smallwebwaf:readme-rewrite-155
sneak/smallwebwaf:v1.0.0-rc2 sneak/smallwebwaf:v1.0.0-rc1
1 Commits
issue-211-viper-config .. 1d75ad2d48
Author SHA1 Message Date
clawbot 1d75ad2d48 Core Rule Set: gitea's ordinary requests pass by default (closes #198)
check / check (push) Failing after 13m57s
Details 
The eighth change, internal/waf/gitea.conf, has the fields where gitea
sends a search, a file name or a branch name skip the rules that refused
ordinary gitea requests, on gitea's paths only. Referer skips 944130 too.
A registry's layer reuse skips 932260 on from, the seventh change.

The new setting SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE loads an operator's
own exclusions before the Core Rule Set's rules. A set file that cannot
be read, does not parse or names smallwebwaf's own ids stops the start
in every mode.

A test sends the recorded gitea requests at the defaults, none refused,
and three attacks, all refused.

Model: opus-5-5
 2026-11-07 02:15:39 +00:00
24 changed files with 1234 additions and 292 deletions
Expand all files Collapse all files
README.md
+10 -6
View File
@@ -192,9 +192,9 @@ account, a key, a token or an alert destination stays off until you give one.
- Any `SWWAF_` setting can be given as a file instead, for secrets and long - Any `SWWAF_` setting can be given as a file instead, for secrets and long
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
The file's contents are the value, less one newline at the end. Setting both The file's contents are the value, less one newline at the end. Setting both
forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` already names a file, so forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` and
it has no `_FILE` form, and no `SWRP_` setting has one, since `rproxy` never `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` already name a file, so they have no
touches disk. `_FILE` form. No `SWRP_` setting has one, since `rproxy` never touches disk.
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. - A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of - A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
1024: `512K`, `100M`, `5G`. 1024: `512K`, `100M`, `5G`.
@@ -272,6 +272,7 @@ Every setting the code reads, with its default:
| `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. | | `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
| `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. | | `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
| `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. | | `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
| `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
| `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. | | `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
| `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. | | `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
| `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. | | `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
@@ -636,10 +637,13 @@ and the status for a size or time limit, such as `413` for a body too large.
headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In
front of a code forge, bodies carry what people write and publish, which the front of a code forge, bodies carry what people write and publish, which the
rules take for attacks. rules take for attacks.
- `smallwebwaf` changes the Core Rule Set in six ways so that it lets ordinary - `smallwebwaf` changes the Core Rule Set in eight ways so that it lets ordinary
requests to a code forge through, such as git over HTTP and sign-in through requests to a code forge through, such as git over HTTP and sign-in through
Git Credential Manager. No setting undoes them; "Attack detection" in Git Credential Manager. On gitea's own paths, its searches, file names and
[`SPEC.md`](SPEC.md) lists them. branch names get through too. No setting undoes the changes; "Attack
detection" in [`SPEC.md`](SPEC.md) lists them.
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's
own fields or paths.
- Rule files are plain text, one rule to a line, edited by hand and read again - Rule files are plain text, one rule to a line, edited by hand and read again
while `servewaf` runs. The image ships one file of probes no real visitor while `servewaf` runs. The image ships one file of probes no real visitor
sends (see [`docs/rules.md`](docs/rules.md)). sends (see [`docs/rules.md`](docs/rules.md)).
SPEC.md
+69 -16
View File
@@ -36,7 +36,8 @@ to them.
- the state files; - the state files;
- a lookup database; - a lookup database;
- a `SWWAF_` setting given as a file; - a `SWWAF_` setting given as a file;
- the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names. - the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names;
- the file `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names.
## Non-goals ## Non-goals
@@ -81,7 +82,7 @@ to them.
of bans is full. of bans is full.
- `internal/rules`: reads the rule files at start and again when they - `internal/rules`: reads the rule files at start and again when they
change, and tells which rules a request matches. change, and tells which rules a request matches.
- `internal/waf`: the Core Rule Set, with the six changes listed under - `internal/waf`: the Core Rule Set, with the eight changes listed under
"Attack detection", as Coraza directives. "Attack detection", as Coraza directives.
- `internal/lookup`: looks clients up through GeoJS or in the lookup - `internal/lookup`: looks clients up through GeoJS or in the lookup
database. database.
@@ -325,9 +326,10 @@ Conventions:
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
The file's contents are the value, less one newline at the end, and are The file's contents are the value, less one newline at the end, and are
checked as the value itself would be. Setting both forms stops the start, and checked as the value itself would be. Setting both forms stops the start, and
so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` and
already holds a file name, so it has no `_FILE` form. No `SWRP_` setting has `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` already hold a file name, so they have
one, since `rproxy` never touches disk. no `_FILE` form. No `SWRP_` setting has one, since `rproxy` never touches
disk.
- Settings, files included, are read once at start. Changing one means - Settings, files included, are read once at start. Changing one means
restarting the container. The files `servewaf` watches while it runs are its restarting the container. The files `servewaf` watches while it runs are its
state files, its rule files and the lookup database. state files, its rule files and the lookup database.
@@ -583,6 +585,24 @@ sends it.
probes these rules caught (see "Rule files"). A list you give replaces the probes these rules caught (see "Rule files"). A list you give replaces the
default, so name these six ids in it to keep them off. default, so name these six ids in it to keep them off.
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names a file of exclusions, written as
Coraza directives. An exclusion switches one rule off for some fields or paths
only, where `SWWAF_WAF_DISABLED_RULES` switches it off everywhere. The file
holds an app's own exclusions; the eighth change below holds gitea's.
- `servewaf` loads the file after `smallwebwaf`'s own setup rules and before
the Core Rule Set's rules. The Core Rule Set puts its own file of
exclusions that run before the rules in the same place.
- So the file holds only `SecAction` and `SecRule` lines that exclude with
`ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. These run with each
request, before the rules they exclude, those of phase 1 included.
- A rule that `SWWAF_WAF_DISABLED_RULES` switches off and the file names is
simply off.
- The start stops on a file that cannot be read or does not parse, whatever
`SWWAF_WAF_MODE` is.
- It stops too on a file that names a number from 900000 to 900999, the ids
of `smallwebwaf`'s own rules, alone or in a range. Comment lines are not
checked.
- `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. It takes - `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. It takes
at most `64K`, since `servewaf` gets no more of a body than that (see "The at most `64K`, since `servewaf` gets no more of a body than that (see "The
answer"). Form data and multipart bodies are read up to the size. JSON and XML answer"). Form data and multipart bodies are read up to the size. JSON and XML
@@ -612,7 +632,7 @@ sends it.
scanner from an ordinary client. In front of gitea, registry and git clients scanner from an ordinary client. In front of gitea, registry and git clients
get `404` and `401` by design, hundreds of times a minute. get `404` and `401` by design, hundreds of times a minute.
`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of `smallwebwaf` changes the Core Rule Set 4.25.0 in eight ways, since in front of
gitea it would otherwise refuse ordinary requests. The changes hold in front of gitea it would otherwise refuse ordinary requests. The changes hold in front of
every app, and no setting undoes them: every app, and no setting undoes them:
@@ -644,9 +664,10 @@ every app, and no setting undoes them:
or `docker-build`. In these parameters, what only these three rules would or `docker-build`. In these parameters, what only these three rules would
catch reaches the app, such as the command `|cat /etc/passwd`. Path catch reaches the app, such as the command `|cat /etc/passwd`. Path
traversal, SQL and script injection, and PHP, Java and Node.js code are traversal, SQL and script injection, and PHP, Java and Node.js code are
still refused there. Every other parameter keeps all three rules. An app still refused there. Other parameters keep all three rules, except on the
that uses one of these parameters as a file on the server, or passes it to a gitea paths the eighth change names. An app that uses one of these
shell, must check the value itself. A rule file can refuse it instead. parameters as a file on the server, or passes it to a shell, must check the
value itself. A rule file can refuse it instead.
5. Two cookies are left out, and `Referer` keeps fewer rules: 5. Two cookies are left out, and `Referer` keeps fewer rules:
- The Core Rule Set reads the request without the `gitea_flash` and - The Core Rule Set reads the request without the `gitea_flash` and
@@ -660,15 +681,47 @@ every app, and no setting undoes them:
only to its own pages. Every other cookie is read in full. An app that only to its own pages. Every other cookie is read in full. An app that
reads a cookie of either name must check it itself. reads a cookie of either name must check it itself.
- `Referer` is not checked for a Unix command given without arguments - `Referer` is not checked for a Unix command given without arguments
(932340) or for Java starting a process (944110). `Referer` names the page (932340), for Java starting a process (944110) or for a Java class name
a request comes from, on everything that page loads and on every link (944130). `Referer` names the page a request comes from, on everything
followed from it. The Core Rule Set takes some page addresses for attacks, that page loads and on every link followed from it. The Core Rule Set
such as a search for `env`, or OpenJDK's `Runtime.java`, whose path holds takes some page addresses for attacks, such as a search for `env`,
both `runtime` and `java.`. All of those requests would be refused. OpenJDK's `Runtime.java`, whose path holds both `runtime` and `java.`, or
`Referer` keeps the other rules. An app that passes `Referer` to a shell a file named `java.lang.String.md`. All of those requests would be
must check it itself. refused. `Referer` keeps the other rules, except after a gitea search (the
eighth change). An app that passes `Referer` to a shell must check it
itself.
6. Responses are not inspected. A raw file from a repository, such as a shell 6. Responses are not inspected. A raw file from a repository, such as a shell
script, looks to the response rules like source code leaking from the server. script, looks to the response rules like source code leaking from the server.
7. The `from` parameter of a container registry's upload path,
`/v2/<image>/blobs/uploads/`, is not checked for command names (932260). When
a push reuses a layer, `from` names the image the layer comes from. An image
such as `owner/python3-app` would have that push refused.
8. On gitea's own paths, the fields where gitea sends a search, a file name or a
branch name skip the rules that refused ordinary gitea requests there.
`internal/waf/gitea.conf` lists each rule and the request it refused. The
fields are:
- `q` on gitea's search pages, for code, issues, pull requests, commits,
repositories, branches, tags, packages and users. A code search holds
code, such as `<script>`, `SELECT * FROM users WHERE` or `../`.
- `Referer` of a page opened from such a search, for script and SQL
injection and Java code (941110, 941120, 941160, 942100, 944100).
- The path of a file's page, raw file, download, blame, history, edit and
delete pages, and of the API's call for its contents. Names such as
`notes.txt~`, `process.env.d.ts` and `java.lang.String.md` were refused.
- A file name in a query parameter: `file` of a commit graph, `path` of the
API's list of commits, `body` of the API's call for several files,
`template` of a new issue, `redirect_to` of the sign-in page, and `name`
of an attachment uploaded through the API.
- A branch or tag name in a query parameter, for command names (932260).
gitea sends one to delete or restore a branch, to start a release from a
tag, to edit a branch protection rule, and to list a directory's last
commits.
- The `Content-Range` header of an artifact upload from
`actions/upload-artifact@v3` (920450).
Elsewhere these fields keep every rule. The paths are those of gitea at the
site root. An app other than gitea that serves one of them must check these
fields itself.
The parameter and cookie names in these changes are matched without regard to The parameter and cookie names in these changes are matched without regard to
case, as Coraza matches them. case, as Coraza matches them.
docs/settings.md
+7
View File
@@ -192,6 +192,13 @@ chooses between the two (see "The image's command" below).
the Core Rule Set reads, a size of at most `64K`. A question carries no more the Core Rule Set reads, a size of at most `64K`. A question carries no more
of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one
byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)). byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule Set
exclusions, written as Coraza directives. It is loaded before the Core Rule
Set's rules, so it holds only `SecAction` and `SecRule` lines that exclude
with `ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. A file that cannot be
read or does not parse stops the start, whatever `SWWAF_WAF_MODE` is. So does
a file that names a number from 900000 to 900999 outside a comment, alone or
in a range. It has no `_FILE` form.
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client - `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
in a minute (see "Attack detection" in `SPEC.md`). A count or `off`. in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
go.mod
-10
View File
@@ -10,14 +10,12 @@ require (
github.com/maxmind/mmdbwriter v1.2.0 github.com/maxmind/mmdbwriter v1.2.0
github.com/oschwald/maxminddb-golang/v2 v2.7.0 github.com/oschwald/maxminddb-golang/v2 v2.7.0
github.com/prometheus/client_golang v1.24.1 github.com/prometheus/client_golang v1.24.1
github.com/spf13/viper v1.21.0
) )
require ( require (
github.com/beorn7/perks v1.0.1 // indirect github.com/beorn7/perks v1.0.1 // indirect
github.com/cespare/xxhash/v2 v2.3.0 // indirect github.com/cespare/xxhash/v2 v2.3.0 // indirect
github.com/corazawaf/libinjection-go v0.3.3 // indirect github.com/corazawaf/libinjection-go v0.3.3 // indirect
github.com/go-viper/mapstructure/v2 v2.4.0 // indirect
github.com/goccy/go-json v0.10.5 // indirect github.com/goccy/go-json v0.10.5 // indirect
github.com/goccy/go-yaml v1.19.2 // indirect github.com/goccy/go-yaml v1.19.2 // indirect
github.com/gotnospirit/makeplural v0.0.0-20180622080156-a5f48d94d976 // indirect github.com/gotnospirit/makeplural v0.0.0-20180622080156-a5f48d94d976 // indirect
@@ -27,22 +25,14 @@ require (
github.com/kylelemons/godebug v1.1.0 // indirect github.com/kylelemons/godebug v1.1.0 // indirect
github.com/magefile/mage v1.17.0 // indirect github.com/magefile/mage v1.17.0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/petar-dambovaliev/aho-corasick v0.0.0-20250424160509-463d218d4745 // indirect github.com/petar-dambovaliev/aho-corasick v0.0.0-20250424160509-463d218d4745 // indirect
github.com/prometheus/client_model v0.6.2 // indirect github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.70.1 // indirect github.com/prometheus/common v0.70.1 // indirect
github.com/prometheus/procfs v0.21.1 // indirect github.com/prometheus/procfs v0.21.1 // indirect
github.com/sagikazarmark/locafero v0.11.0 // indirect
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 // indirect
github.com/spf13/afero v1.15.0 // indirect
github.com/spf13/cast v1.10.0 // indirect
github.com/spf13/pflag v1.0.10 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/tidwall/gjson v1.18.0 // indirect github.com/tidwall/gjson v1.18.0 // indirect
github.com/tidwall/match v1.1.1 // indirect github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.1 // indirect github.com/tidwall/pretty v1.2.1 // indirect
github.com/valllabh/ocsf-schema-golang v1.0.3 // indirect github.com/valllabh/ocsf-schema-golang v1.0.3 // indirect
go.yaml.in/yaml/v3 v3.0.5 // indirect
go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect go4.org/netipx v0.0.0-20231129151722-fdeea329fbba // indirect
golang.org/x/net v0.58.0 // indirect golang.org/x/net v0.58.0 // indirect
golang.org/x/sync v0.23.0 // indirect golang.org/x/sync v0.23.0 // indirect
go.sum
-24
View File
@@ -12,12 +12,8 @@ github.com/corazawaf/libinjection-go v0.3.3 h1:NhbXKRfRpqKzBMzv8zpCcnjyEw7BCVhBO
github.com/corazawaf/libinjection-go v0.3.3/go.mod h1:Ik/+w3UmTWH9yn366RgS9D95K3y7Atb5m/H/gXzzPCk= github.com/corazawaf/libinjection-go v0.3.3/go.mod h1:Ik/+w3UmTWH9yn366RgS9D95K3y7Atb5m/H/gXzzPCk=
github.com/foxcpp/go-mockdns v1.2.0 h1:omK3OrHRD1IWJz1FuFBCFquhXslXoF17OvBS6JPzZF0= github.com/foxcpp/go-mockdns v1.2.0 h1:omK3OrHRD1IWJz1FuFBCFquhXslXoF17OvBS6JPzZF0=
github.com/foxcpp/go-mockdns v1.2.0/go.mod h1:IhLeSFGed3mJIAXPH2aiRQB+kqz7oqu8ld2qVbOu7Wk= github.com/foxcpp/go-mockdns v1.2.0/go.mod h1:IhLeSFGed3mJIAXPH2aiRQB+kqz7oqu8ld2qVbOu7Wk=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
github.com/frankban/quicktest v1.14.6/go.mod h1:4ptaffx2x8+WTWXmUCuVU6aPUX1/Mz7zb5vbUoiM6w0=
github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho= github.com/fsnotify/fsnotify v1.10.1 h1:b0/UzAf9yR5rhf3RPm9gf3ehBPpf0oZKIjtpKrx59Ho=
github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo= github.com/fsnotify/fsnotify v1.10.1/go.mod h1:TLheqan6HD6GBK6PrDWyDPBaEV8LspOxvPSjC+bVfgo=
github.com/go-viper/mapstructure/v2 v2.4.0 h1:EBsztssimR/CONLSZZ04E8qAkxNYq4Qp9LvH92wZUgs=
github.com/go-viper/mapstructure/v2 v2.4.0/go.mod h1:oJDH3BJKyqBA2TXFhDsKDGDTlndYOZ6rGS0BRZIxGhM=
github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4= github.com/goccy/go-json v0.10.5 h1:Fq85nIqj+gXn/S5ahsiTlK3TmC85qgirsdTP/+DeaC4=
github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M= github.com/goccy/go-json v0.10.5/go.mod h1:oq7eo15ShAhp70Anwd5lgX2pLfOS3QCiwU/PULtXL6M=
github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM= github.com/goccy/go-yaml v1.19.2 h1:PmFC1S6h8ljIz6gMRBopkjP1TVT7xuwrButHID66PoM=
@@ -38,10 +34,6 @@ github.com/kaptinlin/jsonschema v0.4.6 h1:vOSFg5tjmfkOdKg+D6Oo4fVOM/pActWu/ntkPs
github.com/kaptinlin/jsonschema v0.4.6/go.mod h1:1DUd7r5SdyB2ZnMtyB7uLv64dE3zTFTiYytDCd+AEL0= github.com/kaptinlin/jsonschema v0.4.6/go.mod h1:1DUd7r5SdyB2ZnMtyB7uLv64dE3zTFTiYytDCd+AEL0=
github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk= github.com/klauspost/compress v1.19.1 h1:VsB4HPswih7mmZ8WleSFQ75c/Ui1M4trX5oAsJnhSlk=
github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ= github.com/klauspost/compress v1.19.1/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc= github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw= github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/magefile/mage v1.17.0 h1:dS4tkq997Ism03akafC8509iqDjeE7TNTexI25Y7sXM= github.com/magefile/mage v1.17.0 h1:dS4tkq997Ism03akafC8509iqDjeE7TNTexI25Y7sXM=
@@ -66,24 +58,8 @@ github.com/prometheus/common v0.70.1 h1:1HvjP4D5oL3t8RsPlwxA9onvvStjtIHYE5XuuwOi
github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc= github.com/prometheus/common v0.70.1/go.mod h1:VdFUQDMZK3VLkurFUVhia6uys/0suUp86TJz5qbJRhc=
github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI= github.com/prometheus/procfs v0.21.1 h1:GljZCt+zSTS+NZq88cyQ1LjZ+RCHp3uVuabBWA5+OJI=
github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY= github.com/prometheus/procfs v0.21.1/go.mod h1:aB55Cww9pdSJVHk0hUf0inxWyyjPogFIjmHKYgMKmtY=
github.com/rogpeppe/go-internal v1.9.0 h1:73kH8U+JUqXU8lRuOHeVHaa/SZPifC7BkcraZVejAe8=
github.com/rogpeppe/go-internal v1.9.0/go.mod h1:WtVeX8xhTBvf0smdhujwtBcq4Qrzq/fJaraNFVN+nFs=
github.com/sagikazarmark/locafero v0.11.0 h1:1iurJgmM9G3PA/I+wWYIOw/5SyBtxapeHDcg+AAIFXc=
github.com/sagikazarmark/locafero v0.11.0/go.mod h1:nVIGvgyzw595SUSUE6tvCp3YYTeHs15MvlmU87WwIik=
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8 h1:+jumHNA0Wrelhe64i8F6HNlS8pkoyMv5sreGx2Ry5Rw=
github.com/sourcegraph/conc v0.3.1-0.20240121214520-5f936abd7ae8/go.mod h1:3n1Cwaq1E1/1lhQhtRK2ts/ZwZEhjcQeJQ1RuC6Q/8U=
github.com/spf13/afero v1.15.0 h1:b/YBCLWAJdFWJTN9cLhiXXcD7mzKn9Dm86dNnfyQw1I=
github.com/spf13/afero v1.15.0/go.mod h1:NC2ByUVxtQs4b3sIUphxK0NioZnmxgyCrfzeuq8lxMg=
github.com/spf13/cast v1.10.0 h1:h2x0u2shc1QuLHfxi+cTJvs30+ZAHOGRic8uyGTDWxY=
github.com/spf13/cast v1.10.0/go.mod h1:jNfB8QC9IA6ZuY2ZjDp0KtFO2LZZlg4S/7bzP6qqeHo=
github.com/spf13/pflag v1.0.10 h1:4EBh2KAYBwaONj6b2Ye1GiHfwjqyROoF4RwYO+vPwFk=
github.com/spf13/pflag v1.0.10/go.mod h1:McXfInJRrz4CZXVZOBLb0bTZqETkiAhM9Iw0y3An2Bg=
github.com/spf13/viper v1.21.0 h1:x5S+0EU27Lbphp4UKm1C+1oQO+rKx36vfCoaVebLFSU=
github.com/spf13/viper v1.21.0/go.mod h1:P0lhsswPGWD/1lZJ9ny3fYnVqxiegrlNrEmgLjbTCAY=
github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE= github.com/stretchr/testify v1.12.1 h1:EuwCh5fleGS7H32xRwO3wRGT7DxrDhLAT6FF8MpWDWE=
github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg= github.com/stretchr/testify v1.12.1/go.mod h1:MDEgiDPPsNp5cuIrHPPCyornHKgEVbtFUmoNlxoYthg=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY= github.com/tidwall/gjson v1.18.0 h1:FIDeeyB800efLX89e5a8Y0BNH+LOngJyGrIWxG2FKQY=
github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk= github.com/tidwall/gjson v1.18.0/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA= github.com/tidwall/match v1.1.1 h1:+Ho715JplO36QYgwN9PGYNhgZvoUSc9X2c80KVTi+GA=
internal/config/config.go
+41 -46
View File
@@ -26,8 +26,6 @@ import (
"unicode" "unicode"
"unicode/utf8" "unicode/utf8"
"github.com/spf13/viper"
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly" "sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/remotelog" "sneak.berlin/go/smallwebwaf/internal/remotelog"
@@ -223,12 +221,15 @@ type Config struct {
// WAFExemptPaths the path prefixes it does not inspect // WAFExemptPaths the path prefixes it does not inspect
// (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body // (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body
// it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none. // it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
WAFMode string // CoreRuleSetExclusionsFile is the path of the file of its exclusions
WAFParanoiaLevel int // (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while there is none.
WAFAnomalyThreshold int WAFMode string
WAFDisabledRules []int WAFParanoiaLevel int
WAFExemptPaths []string WAFAnomalyThreshold int
WAFBodyLimit int64 WAFDisabledRules []int
WAFExemptPaths []string
WAFBodyLimit int64
CoreRuleSetExclusionsFile string
// TrapPaths are the paths a request for which is a clear sign of // TrapPaths are the paths a request for which is a clear sign of
// attack (SWWAF_TRAP_PATHS), each starting with / and without a ?. // attack (SWWAF_TRAP_PATHS), each starting with / and without a ?.
TrapPaths []string TrapPaths []string
@@ -452,16 +453,15 @@ var (
"is in SWWAF_BLOCKLIST_URLS or is SWWAF_ASN_LIMIT_PERCENT_URL too") "is in SWWAF_BLOCKLIST_URLS or is SWWAF_ASN_LIMIT_PERCENT_URL too")
) )
// FromEnvironment reads the settings through v, normally the viper // FromEnvironment reads the settings with lookupEnv, normally
// NewViper returns, bound to the process's environment. A setting may // os.LookupEnv. A setting may instead be given as a file: the variable
// instead be given as a file: the variable named by the setting's name // named by the setting's name with _FILE added names the file, which is
// with _FILE added names the file, which is read now (see lookup). A // read now (see lookup). A setting that is not set takes its default. A
// setting that is not set takes its default. A setting that is set but // setting that is set but invalid is an error that names it.
// invalid is an error that names it.
// //
//nolint:funlen // one line for each setting, a list that grows with them //nolint:funlen // one line for each setting, a list that grows with them
func FromEnvironment(v *viper.Viper) (*Config, error) { func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
env := &environment{v: v, readFiles: true} env := &environment{lookupEnv: lookupEnv}
cfg := &Config{ cfg := &Config{
ListenAddr: env.address("SWWAF_LISTEN_ADDR"), ListenAddr: env.address("SWWAF_LISTEN_ADDR"),
InstanceName: env.instanceName(), InstanceName: env.instanceName(),
@@ -526,8 +526,10 @@ func FromEnvironment(v *viper.Viper) (*Config, error) {
WAFAnomalyThreshold: env.numberOrOff("SWWAF_WAF_ANOMALY_THRESHOLD", "5"), WAFAnomalyThreshold: env.numberOrOff("SWWAF_WAF_ANOMALY_THRESHOLD", "5"),
WAFDisabledRules: env.ruleIDs("SWWAF_WAF_DISABLED_RULES", WAFDisabledRules: env.ruleIDs("SWWAF_WAF_DISABLED_RULES",
"920340,920420,920440,920640,930130,930140"), "920340,920420,920440,920640,930130,930140"),
WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""), WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off), WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
CoreRuleSetExclusionsFile: env.filePath(
"SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"),
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"), TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"), ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"), LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
@@ -578,8 +580,8 @@ func FromEnvironment(v *viper.Viper) (*Config, error) {
// Command reads only SMALLWEBWAF_COMMAND, the command the image runs and its // Command reads only SMALLWEBWAF_COMMAND, the command the image runs and its
// health check checks: rproxy when it is unset or empty, or servewaf. It cannot // health check checks: rproxy when it is unset or empty, or servewaf. It cannot
// be given as a file. Any other value is an error that names the setting. // be given as a file. Any other value is an error that names the setting.
func Command(v *viper.Viper) (string, error) { func Command(lookupEnv func(string) (string, bool)) (string, error) {
command, _ := readSetting(v, "SMALLWEBWAF_COMMAND") command, _ := lookupEnv("SMALLWEBWAF_COMMAND")
switch command { switch command {
case "", "rproxy": case "", "rproxy":
@@ -594,8 +596,8 @@ func Command(v *viper.Viper) (string, error) {
// ServeWAFListenAddr reads only SWWAF_LISTEN_ADDR, which may be given as a // ServeWAFListenAddr reads only SWWAF_LISTEN_ADDR, which may be given as a
// file, as FromEnvironment does. The health check in servewaf's container needs // file, as FromEnvironment does. The health check in servewaf's container needs
// no other setting, so it reads no other, nor a file that another names. // no other setting, so it reads no other, nor a file that another names.
func ServeWAFListenAddr(v *viper.Viper) (string, error) { func ServeWAFListenAddr(lookupEnv func(string) (string, bool)) (string, error) {
env := &environment{v: v, readFiles: true} env := &environment{lookupEnv: lookupEnv}
listenAddr := env.address("SWWAF_LISTEN_ADDR") listenAddr := env.address("SWWAF_LISTEN_ADDR")
return listenAddr, env.err return listenAddr, env.err
@@ -605,8 +607,8 @@ func ServeWAFListenAddr(v *viper.Viper) (string, error) {
// file, as FromEnvironment does, so that the line saying a setting is // file, as FromEnvironment does, so that the line saying a setting is
// invalid carries it too. A file that cannot be read gives the default // invalid carries it too. A file that cannot be read gives the default
// here, and FromEnvironment then stops the start over it. // here, and FromEnvironment then stops the start over it.
func InstanceName(v *viper.Viper) string { func InstanceName(lookupEnv func(string) (string, bool)) string {
env := &environment{v: v, readFiles: true} env := &environment{lookupEnv: lookupEnv}
return env.instanceName() return env.instanceName()
} }
@@ -622,26 +624,13 @@ func (c *Config) LogValue() slog.Value {
} }
// environment is where FromEnvironment reads the settings: it notes each // environment is where FromEnvironment reads the settings: it notes each
// value for the log, and keeps the first error. It reads every setting // value for the log, and keeps the first error.
// through v, the viper NewViper returns, and, unless readFiles is false,
// honours each setting's _FILE form too.
type environment struct { type environment struct {
v *viper.Viper lookupEnv func(string) (string, bool)
readFiles bool
settings []slog.Attr settings []slog.Attr
err error err error
} }
// readSetting returns a setting's value and whether it is set, reading it
// through v.
func readSetting(v *viper.Viper, name string) (string, bool) {
if !v.IsSet(name) {
return "", false
}
return v.GetString(name), true
}
// value returns a setting's value, or its default when it is not set, // value returns a setting's value, or its default when it is not set,
// and notes it for the log. // and notes it for the log.
func (e *environment) value(name, defaultValue string) string { func (e *environment) value(name, defaultValue string) string {
@@ -660,14 +649,10 @@ func (e *environment) value(name, defaultValue string) string {
// names, less one newline at their end. It notes that file's path for the // names, less one newline at their end. It notes that file's path for the
// log. Both variables set, or a file that cannot be read, is an error. // log. Both variables set, or a file that cannot be read, is an error.
func (e *environment) lookup(name string) (string, bool) { func (e *environment) lookup(name string) (string, bool) {
value, set := readSetting(e.v, name) value, set := e.lookupEnv(name)
if !e.readFiles {
return value, set
}
fileName := name + "_FILE" fileName := name + "_FILE"
path, inFile := readSetting(e.v, fileName) path, inFile := e.lookupEnv(fileName)
if !inFile { if !inFile {
return value, set return value, set
} }
@@ -1209,11 +1194,21 @@ func (e *environment) logRemoteURL(name string) *url.URL {
return remote return remote
} }
// filePath reads a setting that is the path of a file, "" while it is
// unset. Its value names a file already, so, unlike the other settings,
// it has no _FILE form. Whoever uses the file reads it.
func (e *environment) filePath(name string) string {
path, _ := e.lookupEnv(name)
e.settings = append(e.settings, slog.String(name, path))
return path
}
// certificates reads a setting that is the path of a file of PEM // certificates reads a setting that is the path of a file of PEM
// certificates. Unset or empty, it is nil. Its value names a file // certificates. Unset or empty, it is nil. Its value names a file
// already, so, unlike the other settings, it has no _FILE form. // already, so, unlike the other settings, it has no _FILE form.
func (e *environment) certificates(name string) *x509.CertPool { func (e *environment) certificates(name string) *x509.CertPool {
path, _ := readSetting(e.v, name) path, _ := e.lookupEnv(name)
e.settings = append(e.settings, slog.String(name, path)) e.settings = append(e.settings, slog.String(name, path))
if path == "" { if path == "" {
internal/config/config_test.go
+44 -46
View File
@@ -17,8 +17,6 @@ import (
"testing" "testing"
"time" "time"
"github.com/spf13/viper"
"sneak.berlin/go/smallwebwaf/internal/anomaly" "sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
) )
@@ -89,6 +87,7 @@ const (
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES" wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS" wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT" wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
exclusionsFile = "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"
trapPaths = "SWWAF_TRAP_PATHS" trapPaths = "SWWAF_TRAP_PATHS"
errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD" errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
logRemoteURL = "SWWAF_LOG_REMOTE_URL" logRemoteURL = "SWWAF_LOG_REMOTE_URL"
@@ -219,22 +218,18 @@ const (
// environment is a set of environment variables, for FromEnvironment. // environment is a set of environment variables, for FromEnvironment.
type environment map[string]string type environment map[string]string
// viper returns a viper Set with each of the variables, as the viper // lookupEnv reads one of the variables, as os.LookupEnv does.
// NewViper returns one bound to a process's environment. func (e environment) lookupEnv(name string) (string, bool) {
func (e environment) viper() *viper.Viper { value, ok := e[name]
v := viper.New()
for name, value := range e {
v.Set(name, value)
}
return v return value, ok
} }
// fromEnvironment reads the settings from env, which must be valid. // fromEnvironment reads the settings from env, which must be valid.
func fromEnvironment(t *testing.T, env environment) *config.Config { func fromEnvironment(t *testing.T, env environment) *config.Config {
t.Helper() t.Helper()
cfg, err := config.FromEnvironment(env.viper()) cfg, err := config.FromEnvironment(env.lookupEnv)
if err != nil { if err != nil {
t.Fatalf("settings %v: %v", env, err) t.Fatalf("settings %v: %v", env, err)
} }
@@ -461,7 +456,7 @@ func TestPathPrefixNotStartingWithSlashStopsTheStart(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment( _, err := config.FromEnvironment(
environment{rateLimitExemptPaths: "/favicon.ico,assets/"}.viper()) environment{rateLimitExemptPaths: "/favicon.ico,assets/"}.lookupEnv)
want := rateLimitExemptPaths + `: "assets/" is not a path prefix ` + want := rateLimitExemptPaths + `: "assets/" is not a path prefix ` +
`starting with /, such as /assets/` `starting with /, such as /assets/`
@@ -516,7 +511,7 @@ func TestInvalidTrapPathOrErrorBurstThresholdStopsTheStart(t *testing.T) {
t.Run(tc.name+"="+tc.value, func(t *testing.T) { t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper()) _, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
want := tc.name + ": " + tc.want want := tc.name + ": " + tc.want
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -545,13 +540,14 @@ func TestCoreRuleSetSettings(t *testing.T) {
environment{ environment{
wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10", wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10",
wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/", wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/",
wafBodyLimit: "32K", wafBodyLimit: "32K", exclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
}, },
config.Config{ config.Config{
WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10, WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10,
WAFDisabledRules: []int{942100, 920350}, WAFDisabledRules: []int{942100, 920350},
WAFExemptPaths: []string{"/api/", "/static/"}, WAFExemptPaths: []string{"/api/", "/static/"},
WAFBodyLimit: 32 << 10, WAFBodyLimit: 32 << 10,
CoreRuleSetExclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
}, },
}, },
{ {
@@ -571,7 +567,8 @@ func TestCoreRuleSetSettings(t *testing.T) {
WAFMode: cfg.WAFMode, WAFParanoiaLevel: cfg.WAFParanoiaLevel, WAFMode: cfg.WAFMode, WAFParanoiaLevel: cfg.WAFParanoiaLevel,
WAFAnomalyThreshold: cfg.WAFAnomalyThreshold, WAFAnomalyThreshold: cfg.WAFAnomalyThreshold,
WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths, WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths,
WAFBodyLimit: cfg.WAFBodyLimit, WAFBodyLimit: cfg.WAFBodyLimit,
CoreRuleSetExclusionsFile: cfg.CoreRuleSetExclusionsFile,
} }
if !reflect.DeepEqual(got, tc.want) { if !reflect.DeepEqual(got, tc.want) {
t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want) t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want)
@@ -642,7 +639,7 @@ func TestInvalidCoreRuleSetSettingStopsTheStart(t *testing.T) {
t.Run(tc.name+"="+tc.value, func(t *testing.T) { t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper()) _, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
want := tc.name + ": " + tc.want want := tc.name + ": " + tc.want
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -767,7 +764,7 @@ func TestInvalidRemoteLogSettingStopsTheStart(t *testing.T) {
{logRemoteAppName, ""}, {logRemoteAppName, "my app"}, {logRemoteAppName, ""}, {logRemoteAppName, "my app"},
{logRemoteAppName, "gitéa"}, {logRemoteAppName, strings.Repeat("a", 49)}, {logRemoteAppName, "gitéa"}, {logRemoteAppName, strings.Repeat("a", 49)},
} { } {
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper()) _, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
if err == nil || !strings.HasPrefix(err.Error(), tc.name+": ") { if err == nil || !strings.HasPrefix(err.Error(), tc.name+": ") {
t.Errorf("%s=%q: error %v, want one naming it", tc.name, tc.value, err) t.Errorf("%s=%q: error %v, want one naming it", tc.name, tc.value, err)
} }
@@ -784,7 +781,7 @@ func TestRemoteLogCAFileWithoutCertificateStopsTheStart(t *testing.T) {
t.Fatalf("write %s: %v", caFile, err) t.Fatalf("write %s: %v", caFile, err)
} }
_, err = config.FromEnvironment(environment{logRemoteTLSCAFile: caFile}.viper()) _, err = config.FromEnvironment(environment{logRemoteTLSCAFile: caFile}.lookupEnv)
want := logRemoteTLSCAFile + `: "` + caFile + `" holds no PEM certificate` want := logRemoteTLSCAFile + `: "` + caFile + `" holds no PEM certificate`
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -799,7 +796,7 @@ func TestInstanceNameNotAnAppNameStopsTheStartOnlyWhileSending(t *testing.T) {
sending := environment{logRemoteURL: remoteURL, instanceName: spaced} sending := environment{logRemoteURL: remoteURL, instanceName: spaced}
_, err := config.FromEnvironment(sending.viper()) _, err := config.FromEnvironment(sending.lookupEnv)
want := logRemoteAppName + `: is unset, and ` + instanceName + want := logRemoteAppName + `: is unset, and ` + instanceName +
` "fsn1 app1", its default, is not 1 to 48 printable ASCII characters ` + ` "fsn1 app1", its default, is not 1 to 48 printable ASCII characters ` +
@@ -829,7 +826,7 @@ func TestAppNameSetStopsTheStartWhileSending(t *testing.T) {
logRemoteURL: remoteURL, logRemoteURL: remoteURL,
instanceName: instance, instanceName: instance,
logRemoteAppName: "my app", logRemoteAppName: "my app",
}.viper()) }.lookupEnv)
want := logRemoteAppName + `: "my app" is not 1 to 48 printable ASCII ` + want := logRemoteAppName + `: "my app" is not 1 to 48 printable ASCII ` +
`characters without a space, such as gitea` `characters without a space, such as gitea`
@@ -960,7 +957,7 @@ func TestWebhookHeadersAreLoggedMaskedAndNeverShown(t *testing.T) {
// An item that is not a header is named by its place, not shown. // An item that is not a header is named by its place, not shown.
_, err := config.FromEnvironment(environment{ _, err := config.FromEnvironment(environment{
alertWebhookHeaders: "X-Team:ops," + secret, alertWebhookHeaders: "X-Team:ops," + secret,
}.viper()) }.lookupEnv)
want := alertWebhookHeaders + ": item 2 is not a header name followed by : " + want := alertWebhookHeaders + ": item 2 is not a header name followed by : " +
"and the header's value, such as Authorization:Bearer <token>" "and the header's value, such as Authorization:Bearer <token>"
@@ -997,7 +994,7 @@ func TestWebhookURLIsLoggedWithoutItsPathOrQueryAndNeverShown(t *testing.T) {
"ftp://hooks.example/services/" + secret, "ftp://hooks.example/services/" + secret,
"https://hooks.example/services/%zz" + secret, "https://hooks.example/services/%zz" + secret,
} { } {
_, err := config.FromEnvironment(environment{alertWebhookURL: value}.viper()) _, err := config.FromEnvironment(environment{alertWebhookURL: value}.lookupEnv)
want := alertWebhookURL + ": is not an http or https URL without a user or " + want := alertWebhookURL + ": is not an http or https URL without a user or " +
"a fragment, such as https://alerts.example/smallwebwaf" "a fragment, such as https://alerts.example/smallwebwaf"
@@ -1048,7 +1045,7 @@ func TestNtfyTokenWithAControlCharacterStopsTheStartWithoutShowingIt(t *testing.
"set": {alertNtfyToken: token + "\r"}, "set": {alertNtfyToken: token + "\r"},
"in a file": {alertNtfyToken + "_FILE": writeFile(t, token+"\r\n")}, "in a file": {alertNtfyToken + "_FILE": writeFile(t, token+"\r\n")},
} { } {
_, err := config.FromEnvironment(env.viper()) _, err := config.FromEnvironment(env.lookupEnv)
want := alertNtfyToken + ": holds a control character, such as the " + want := alertNtfyToken + ": holds a control character, such as the " +
"carriage return of a Windows line end" "carriage return of a Windows line end"
@@ -1065,7 +1062,7 @@ func TestInstanceNameWithAControlCharacterStopsTheStartOnlyWithNtfySet(t *testin
_, err := config.FromEnvironment(environment{ _, err := config.FromEnvironment(environment{
instanceName: name, alertNtfyURL: "https://ntfy.example/smallwebwaf-alerts", instanceName: name, alertNtfyURL: "https://ntfy.example/smallwebwaf-alerts",
}.viper()) }.lookupEnv)
want := instanceName + `: "fsn1app1\r" holds a control character, such as the ` + want := instanceName + `: "fsn1app1\r" holds a control character, such as the ` +
`carriage return of a Windows line end, and is sent to ntfy in a header ` + `carriage return of a Windows line end, and is sent to ntfy in a header ` +
@@ -1090,7 +1087,7 @@ func TestInstanceNameNotUTF8StopsTheStart(t *testing.T) {
"set": {instanceName: latin1}, "set": {instanceName: latin1},
"in a file": {instanceName + "_FILE": writeFile(t, latin1+"\n")}, "in a file": {instanceName + "_FILE": writeFile(t, latin1+"\n")},
} { } {
_, err := config.FromEnvironment(env.viper()) _, err := config.FromEnvironment(env.lookupEnv)
want := instanceName + `: "caf\xe9" is not valid UTF-8` want := instanceName + `: "caf\xe9" is not valid UTF-8`
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -1105,7 +1102,7 @@ func TestCodeOnBothCountryListsStopsTheStart(t *testing.T) {
_, err := config.FromEnvironment(environment{ _, err := config.FromEnvironment(environment{
deniedCountries: "cn,ru", deniedCountries: "cn,ru",
allowedCountries: "de,RU", allowedCountries: "de,RU",
}.viper()) }.lookupEnv)
if err == nil { if err == nil {
t.Fatal("ru on both country lists was accepted") t.Fatal("ru on both country lists was accepted")
} }
@@ -1167,7 +1164,7 @@ func TestLookupDBPathGoesWithTheFileSourceAlone(t *testing.T) {
lookupDBPath + ": is set while " + lookupSource + " is off; only file reads it", lookupDBPath + ": is set while " + lookupSource + " is off; only file reads it",
}, },
} { } {
_, err := config.FromEnvironment(tc.env.viper()) _, err := config.FromEnvironment(tc.env.lookupEnv)
if err == nil || err.Error() != tc.want { if err == nil || err.Error() != tc.want {
t.Errorf("settings %v: error %v, want %s", tc.env, err, tc.want) t.Errorf("settings %v: error %v, want %s", tc.env, err, tc.want)
} }
@@ -1197,7 +1194,7 @@ func TestSettingNeedingLookupsStopsTheStartWhileTheyAreOff(t *testing.T) {
_, err := config.FromEnvironment(environment{ _, err := config.FromEnvironment(environment{
lookupSource: off, name: value, lookupSource: off, name: value,
}.viper()) }.lookupEnv)
if err == nil || !strings.HasPrefix(err.Error(), name+": ") || if err == nil || !strings.HasPrefix(err.Error(), name+": ") ||
!strings.Contains(err.Error(), lookupSource+" is off") { !strings.Contains(err.Error(), lookupSource+" is off") {
t.Errorf("error %v, want one naming %s and %s", err, name, lookupSource) t.Errorf("error %v, want one naming %s and %s", err, name, lookupSource)
@@ -1347,7 +1344,7 @@ func TestInvalidAnomalySettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
t.Run(tc.name+"="+tc.value, func(t *testing.T) { t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper()) _, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
want := tc.name + ": " + tc.want want := tc.name + ": " + tc.want
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -1426,7 +1423,7 @@ func TestInvalidBiasedThresholdStopsTheStartSayingWhatIsWrong(t *testing.T) {
t.Run(tc.name+"="+tc.value, func(t *testing.T) { t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper()) _, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
want := tc.name + ": " + tc.want want := tc.name + ": " + tc.want
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -1523,7 +1520,7 @@ func TestInvalidReputationSettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
t.Run(tc.name+"="+tc.value, func(t *testing.T) { t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper()) _, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
want := tc.name + ": " + tc.want want := tc.name + ": " + tc.want
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -1538,7 +1535,7 @@ func TestASNLimitPercentURLThatIsABlocklistStopsTheStart(t *testing.T) {
_, err := config.FromEnvironment(environment{ _, err := config.FromEnvironment(environment{
blocklistURLs: dropURL + "," + asnURL, asnLimitPercentURL: asnURL, blocklistURLs: dropURL + "," + asnURL, asnLimitPercentURL: asnURL,
}.viper()) }.lookupEnv)
want := asnLimitPercentURL + `: "` + asnURL + `" is in SWWAF_BLOCKLIST_URLS too` want := asnLimitPercentURL + `: "` + asnURL + `" is in SWWAF_BLOCKLIST_URLS too`
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -1665,7 +1662,7 @@ func TestInvalidDNSBLSettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
t.Run(tc.name+"="+tc.value, func(t *testing.T) { t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper()) _, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
want := tc.name + ": " + tc.want want := tc.name + ": " + tc.want
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -1733,7 +1730,7 @@ func TestDNSBLZoneKeyIsLoggedMaskedAndNeverShown(t *testing.T) {
{spamhaus + "," + otherZone, `"` + spamhausMasked + `" is listed twice`}, {spamhaus + "," + otherZone, `"` + spamhausMasked + `" is listed twice`},
{spamhaus + ",,", "has an empty item in its list"}, {spamhaus + ",,", "has an empty item in its list"},
} { } {
_, err := config.FromEnvironment(environment{dnsblZones: tc.value}.viper()) _, err := config.FromEnvironment(environment{dnsblZones: tc.value}.lookupEnv)
want := dnsblZones + ": " + tc.want want := dnsblZones + ": " + tc.want
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -1786,7 +1783,7 @@ func TestInvalidAbuseIPDBSettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
t.Run(tc.name+"="+tc.value, func(t *testing.T) { t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper()) _, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
want := tc.name + ": " + tc.want want := tc.name + ": " + tc.want
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -1900,7 +1897,7 @@ func TestInvalidCrowdSecSettingStopsTheStartSayingWhatIsWrong(t *testing.T) {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment(tc.env.viper()) _, err := config.FromEnvironment(tc.env.lookupEnv)
if err == nil || err.Error() != tc.want { if err == nil || err.Error() != tc.want {
t.Errorf("error %v, want %s", err, tc.want) t.Errorf("error %v, want %s", err, tc.want)
} }
@@ -2060,7 +2057,7 @@ func wantStartStopped(t *testing.T, invalid []struct{ name, value string }) {
t.Run(tc.name+"="+tc.value, func(t *testing.T) { t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment(environment{tc.name: tc.value}.viper()) _, err := config.FromEnvironment(environment{tc.name: tc.value}.lookupEnv)
if err == nil { if err == nil {
t.Fatalf("%s=%q was accepted", tc.name, tc.value) t.Fatalf("%s=%q was accepted", tc.name, tc.value)
} }
@@ -2085,7 +2082,7 @@ func TestHostOrTransferEncodingStopsTheStart(t *testing.T) {
t.Run(value, func(t *testing.T) { t.Run(value, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment(environment{logRequestHeaders: value}.viper()) _, err := config.FromEnvironment(environment{logRequestHeaders: value}.lookupEnv)
if err == nil || err.Error() != logRequestHeaders+": "+want { if err == nil || err.Error() != logRequestHeaders+": "+want {
t.Errorf("error %v, want %s: %s", err, logRequestHeaders, want) t.Errorf("error %v, want %s: %s", err, logRequestHeaders, want)
} }
@@ -2102,7 +2099,7 @@ func TestShortTokenStopsTheStartWithoutShowingIt(t *testing.T) {
t.Run(name+"="+value, func(t *testing.T) { t.Run(name+"="+value, func(t *testing.T) {
t.Parallel() t.Parallel()
_, err := config.FromEnvironment(environment{name: value}.viper()) _, err := config.FromEnvironment(environment{name: value}.lookupEnv)
want := name + ": is shorter than 32 characters" want := name + ": is shorter than 32 characters"
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -2158,7 +2155,7 @@ func TestSettingFromFileIsCheckedAsTheSettingItself(t *testing.T) {
_, err := config.FromEnvironment(environment{ _, err := config.FromEnvironment(environment{
bytesLimitPerMinute + "_FILE": writeFile(t, "lots\n"), bytesLimitPerMinute + "_FILE": writeFile(t, "lots\n"),
}.viper()) }.lookupEnv)
want := bytesLimitPerMinute + `: "lots" is not a size such as 512K, 100M or 5G, or off` want := bytesLimitPerMinute + `: "lots" is not a size such as 512K, 100M or 5G, or off`
if err == nil || err.Error() != want { if err == nil || err.Error() != want {
@@ -2169,7 +2166,7 @@ func TestSettingFromFileIsCheckedAsTheSettingItself(t *testing.T) {
logRemoteURL: remoteURL, logRemoteURL: remoteURL,
instanceName: instance, instanceName: instance,
logRemoteAppName + "_FILE": writeFile(t, "my app\n"), logRemoteAppName + "_FILE": writeFile(t, "my app\n"),
}.viper()) }.lookupEnv)
want = logRemoteAppName + `: "my app" is not 1 to 48 printable ASCII ` + want = logRemoteAppName + `: "my app" is not 1 to 48 printable ASCII ` +
`characters without a space, such as gitea` `characters without a space, such as gitea`
@@ -2184,7 +2181,7 @@ func TestSettingAndItsFileBothSetStopsTheStart(t *testing.T) {
_, err := config.FromEnvironment(environment{ _, err := config.FromEnvironment(environment{
metricsToken: token, metricsToken: token,
metricsToken + "_FILE": writeFile(t, token), metricsToken + "_FILE": writeFile(t, token),
}.viper()) }.lookupEnv)
want := metricsToken + ": is set, and so is " + metricsToken + want := metricsToken + ": is set, and so is " + metricsToken +
"_FILE; set only one of them" "_FILE; set only one of them"
@@ -2199,7 +2196,7 @@ func TestUnreadableSettingFileStopsTheStart(t *testing.T) {
dir := t.TempDir() dir := t.TempDir()
for _, path := range []string{filepath.Join(dir, "missing"), dir} { for _, path := range []string{filepath.Join(dir, "missing"), dir} {
_, err := config.FromEnvironment(environment{metricsToken + "_FILE": path}.viper()) _, err := config.FromEnvironment(environment{metricsToken + "_FILE": path}.lookupEnv)
want := metricsToken + "_FILE: cannot be read: " want := metricsToken + "_FILE: cannot be read: "
if err == nil || !strings.HasPrefix(err.Error(), want) { if err == nil || !strings.HasPrefix(err.Error(), want) {
@@ -2333,6 +2330,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
wafDisabledRules: defaultWAFDisabledRules, wafDisabledRules: defaultWAFDisabledRules,
wafExemptPaths: "", wafExemptPaths: "",
wafBodyLimit: off, wafBodyLimit: off,
exclusionsFile: "",
trapPaths: "", trapPaths: "",
errorBurstThreshold: "30", errorBurstThreshold: "30",
logRemoteURL: "", logRemoteURL: "",
internal/config/rproxy.go
+24 -11
View File
@@ -8,9 +8,8 @@ import (
"net/netip" "net/netip"
"net/url" "net/url"
"strconv" "strconv"
"strings"
"time" "time"
"github.com/spf13/viper"
) )
// RProxy is the settings of `smallwebwaf rproxy`. A size or time limit of zero // RProxy is the settings of `smallwebwaf rproxy`. A size or time limit of zero
@@ -59,13 +58,13 @@ var (
"is not a whole number of milliseconds above zero, such as 60000, or off") "is not a whole number of milliseconds above zero, such as 60000, or off")
) )
// RProxyFromEnvironment reads rproxy's settings through v, normally the viper // RProxyFromEnvironment reads rproxy's settings with lookupEnv, normally
// NewViper returns, bound to the process's environment. A setting that is not // os.LookupEnv. A setting that is not set takes its default, and
// set takes its default, and SWRP_WAF_BACKEND, which has none, must be set. A // SWRP_WAF_BACKEND, which has none, must be set. A setting that is set but
// setting that is set but invalid is an error that names it. Since rproxy // invalid is an error that names it. Since rproxy never touches disk, its
// never touches disk, its settings have no _FILE form. // settings have no _FILE form.
func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) { func RProxyFromEnvironment(lookupEnv func(string) (string, bool)) (*RProxy, error) {
env := &environment{v: v, readFiles: false} env := &environment{lookupEnv: withoutFileForms(lookupEnv)}
cfg := &RProxy{ cfg := &RProxy{
ListenAddr: env.address("SWRP_LISTEN_ADDR"), ListenAddr: env.address("SWRP_LISTEN_ADDR"),
UpstreamURL: env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL), UpstreamURL: env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL),
@@ -92,9 +91,9 @@ func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
// SWRP_UPSTREAM_URL. The health check in rproxy's container needs no other // SWRP_UPSTREAM_URL. The health check in rproxy's container needs no other
// setting, so it reads no other. // setting, so it reads no other.
func RProxyListenAddrAndUpstreamURL( func RProxyListenAddrAndUpstreamURL(
v *viper.Viper, lookupEnv func(string) (string, bool),
) (string, *url.URL, error) { ) (string, *url.URL, error) {
env := &environment{v: v, readFiles: false} env := &environment{lookupEnv: withoutFileForms(lookupEnv)}
listenAddr := env.address("SWRP_LISTEN_ADDR") listenAddr := env.address("SWRP_LISTEN_ADDR")
upstreamURL := env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL) upstreamURL := env.appURL("SWRP_UPSTREAM_URL", defaultUpstreamURL)
@@ -105,6 +104,20 @@ func RProxyListenAddrAndUpstreamURL(
return listenAddr, upstreamURL, nil return listenAddr, upstreamURL, nil
} }
// withoutFileForms returns lookupEnv finding no setting named with _FILE, since
// rproxy never touches disk.
func withoutFileForms(
lookupEnv func(string) (string, bool),
) func(string) (string, bool) {
return func(name string) (string, bool) {
if strings.HasSuffix(name, "_FILE") {
return "", false
}
return lookupEnv(name)
}
}
// LogValue makes an RProxy log as each setting's name with the value it was // LogValue makes an RProxy log as each setting's name with the value it was
// given, or its default. // given, or its default.
func (c *RProxy) LogValue() slog.Value { func (c *RProxy) LogValue() slog.Value {
internal/config/rproxy_test.go
+4 -7
View File
@@ -9,8 +9,6 @@ import (
"testing" "testing"
"time" "time"
"github.com/spf13/viper"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
) )
@@ -34,12 +32,11 @@ const (
// rproxyConfig reads rproxy's settings from env. // rproxyConfig reads rproxy's settings from env.
func rproxyConfig(env map[string]string) (*config.RProxy, error) { func rproxyConfig(env map[string]string) (*config.RProxy, error) {
v := viper.New() return config.RProxyFromEnvironment(func(name string) (string, bool) {
for name, value := range env { value, ok := env[name]
v.Set(name, value)
}
return config.RProxyFromEnvironment(v) return value, ok
})
} }
func TestRProxyDefaults(t *testing.T) { func TestRProxyDefaults(t *testing.T) {
internal/config/viper.go
-16
View File
@@ -1,16 +0,0 @@
package config
import "github.com/spf13/viper"
// NewViper returns the viper that reads the settings: each setting's
// environment variable, bound to its own name. Every setting of servewaf and
// rproxy is read through it, and none is read with os.Getenv or os.LookupEnv
// outside it. An environment variable that is set but empty is set, as one
// that holds a value is, which viper does not take for granted.
func NewViper() *viper.Viper {
v := viper.New()
v.AutomaticEnv()
v.AllowEmptyEnv(true)
return v
}
internal/rproxy/limits_test.go
+7 -3
View File
@@ -33,9 +33,13 @@ func announcedLength(size int) string {
func TestServerHasTheFixedLimits(t *testing.T) { func TestServerHasTheFixedLimits(t *testing.T) {
t.Parallel() t.Parallel()
cfg, err := config.RProxyFromEnvironment(viperFrom(map[string]string{ cfg, err := config.RProxyFromEnvironment(func(name string) (string, bool) {
"SWRP_WAF_BACKEND": nowhere, if name == "SWRP_WAF_BACKEND" {
})) return nowhere, true
}
return "", false
})
if err != nil { if err != nil {
t.Fatalf("settings: %v", err) t.Fatalf("settings: %v", err)
} }
internal/rproxy/rproxy_test.go
+5 -14
View File
@@ -18,8 +18,6 @@ import (
"testing" "testing"
"time" "time"
"github.com/spf13/viper"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rproxy" "sneak.berlin/go/smallwebwaf/internal/rproxy"
@@ -343,17 +341,6 @@ func startServer(
return addr, out, server return addr, out, server
} }
// viperFrom returns a viper Set with each of settings, as the viper
// config.NewViper returns one bound to a process's environment.
func viperFrom(settings map[string]string) *viper.Viper {
v := viper.New()
for name, value := range settings {
v.Set(name, value)
}
return v
}
// startServerLogging is startServer with rproxy's request lines going to // startServerLogging is startServer with rproxy's request lines going to
// requestLog and its process lines to processLog. // requestLog and its process lines to processLog.
func startServerLogging( func startServerLogging(
@@ -367,7 +354,11 @@ func startServerLogging(
} }
maps.Copy(settings, env) maps.Copy(settings, env)
cfg, err := config.RProxyFromEnvironment(viperFrom(settings)) cfg, err := config.RProxyFromEnvironment(func(name string) (string, bool) {
value, ok := settings[name]
return value, ok
})
if err != nil { if err != nil {
t.Fatalf("settings %v: %v", settings, err) t.Fatalf("settings %v: %v", settings, err)
} }
internal/servewaf/proxy.go
+24 -14
View File
@@ -4,6 +4,7 @@
package servewaf package servewaf
import ( import (
"fmt"
"io" "io"
"log/slog" "log/slog"
"net/http" "net/http"
@@ -60,6 +61,9 @@ type Params struct {
// LookupFile is the lookup database they are looked up in while // LookupFile is the lookup database they are looked up in while
// SWWAF_LOOKUP_SOURCE is file, and nil otherwise. // SWWAF_LOOKUP_SOURCE is file, and nil otherwise.
LookupFile *lookup.File LookupFile *lookup.File
// CoreRuleSet is what NewCoreRuleSet returns: what inspects requests
// while SWWAF_WAF_MODE is not off, and nil while it is.
CoreRuleSet *waf.CoreRuleSet
// Now tells the time by which requests are counted for the rate // Now tells the time by which requests are counted for the rate
// limits, bans are made and run out, and GeoJS's answers are kept, // limits, bans are made and run out, and GeoJS's answers are kept,
// normally time.Now in UTC, the time the state files give. // normally time.Now in UTC, the time the state files give.
@@ -144,7 +148,7 @@ func newHandler(params Params) *handler {
dnsbl: dnsbl, dnsbl: dnsbl,
abuseIPDB: abuseIPDB, abuseIPDB: abuseIPDB,
rules: params.Rules, rules: params.Rules,
coreRuleSet: newCoreRuleSet(params.Config), coreRuleSet: params.CoreRuleSet,
alerts: params.Alerts, alerts: params.Alerts,
} }
h.geojs = lookup.New(lookup.Params{ h.geojs = lookup.New(lookup.Params{
@@ -202,29 +206,35 @@ func newReputation(
return lists, dnsbl, abuseIPDB return lists, dnsbl, abuseIPDB
} }
// newCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL, // NewCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL,
// without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies // without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies
// up to SWWAF_WAF_BODY_LIMIT, or nil while SWWAF_WAF_MODE is off. It reads at // up to SWWAF_WAF_BODY_LIMIT, with the exclusions in
// most QuestionBodyMaxBytes less one byte: a question's body of // SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, or nil while SWWAF_WAF_MODE is off. It
// reads at most QuestionBodyMaxBytes less one byte: a question's body of
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a // QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
// JSON or XML body over its limit. // JSON or XML body over its limit. An exclusions file that cannot be read, does
func newCoreRuleSet(cfg *config.Config) *waf.CoreRuleSet { // not parse or names an id of smallwebwaf's own rules is an error in every
if cfg.WAFMode == config.WAFModeOff { // mode. While the mode is off, a set file is loaded with the Core Rule Set only
return nil // to check it.
func NewCoreRuleSet(cfg *config.Config) (*waf.CoreRuleSet, error) {
if cfg.WAFMode == config.WAFModeOff && cfg.CoreRuleSetExclusionsFile == "" {
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
} }
coreRuleSet, err := waf.New(waf.Params{ coreRuleSet, err := waf.New(waf.Params{
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules, ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1), BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
}) })
if err != nil { if err != nil {
// The Core Rule Set is built in, and the settings cannot break it: return nil, fmt.Errorf("SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: %w", err)
// the paranoia level is from 1 to 4, and the id of no rule switches
// nothing off.
panic(err)
} }
return coreRuleSet if cfg.WAFMode == config.WAFModeOff {
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
}
return coreRuleSet, nil
} }
// handler is servewaf's handler. It holds what every request shares; what // handler is servewaf's handler. It holds what every request shares; what
internal/servewaf/proxy_test.go
+16 -15
View File
@@ -15,8 +15,6 @@ import (
"testing" "testing"
"time" "time"
"github.com/spf13/viper"
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup" "sneak.berlin/go/smallwebwaf/internal/lookup"
@@ -349,17 +347,6 @@ func serve(t *testing.T, server *servewaf.Server) {
// at serveWAFAddr, with the settings in env on top of the defaults, and returns // at serveWAFAddr, with the settings in env on top of the defaults, and returns
// where it listens. What it writes is dropped. SWRP_TIMEOUT_MS is a minute unless // where it listens. What it writes is dropped. SWRP_TIMEOUT_MS is a minute unless
// env sets it, so that on a busy host SWRP_DEFAULT_DISPOSITION does not decide. // env sets it, so that on a busy host SWRP_DEFAULT_DISPOSITION does not decide.
// viperFrom returns a viper Set with each of settings, as the viper
// config.NewViper returns one bound to a process's environment.
func viperFrom(settings map[string]string) *viper.Viper {
v := viper.New()
for name, value := range settings {
v.Set(name, value)
}
return v
}
func startRProxy( func startRProxy(
t *testing.T, serveWAFAddr, appURL string, env map[string]string, t *testing.T, serveWAFAddr, appURL string, env map[string]string,
) string { ) string {
@@ -372,7 +359,11 @@ func startRProxy(
} }
maps.Copy(settings, env) maps.Copy(settings, env)
cfg, err := config.RProxyFromEnvironment(viperFrom(settings)) cfg, err := config.RProxyFromEnvironment(func(name string) (string, bool) {
value, ok := settings[name]
return value, ok
})
if err != nil { if err != nil {
t.Fatalf("settings %v: %v", settings, err) t.Fatalf("settings %v: %v", settings, err)
} }
@@ -432,7 +423,11 @@ func newServeWAFWith(
maps.Copy(settings, env) maps.Copy(settings, env)
cfg, err := config.FromEnvironment(viperFrom(settings)) cfg, err := config.FromEnvironment(func(name string) (string, bool) {
value, ok := settings[name]
return value, ok
})
if err != nil { if err != nil {
t.Fatalf("settings %v: %v", settings, err) t.Fatalf("settings %v: %v", settings, err)
} }
@@ -468,6 +463,11 @@ func newServeWAFWith(
} }
} }
coreRuleSet, err := servewaf.NewCoreRuleSet(cfg)
if err != nil {
t.Fatalf("Core Rule Set: %v", err)
}
server := build(servewaf.Params{ server := build(servewaf.Params{
Config: cfg, Config: cfg,
RequestLog: out, RequestLog: out,
@@ -475,6 +475,7 @@ func newServeWAFWith(
GeoJSURL: geojsURL, GeoJSURL: geojsURL,
AbuseIPDBURL: abuseIPDBURL, AbuseIPDBURL: abuseIPDBURL,
LookupFile: lookupFile, LookupFile: lookupFile,
CoreRuleSet: coreRuleSet,
Now: now, Now: now,
Rules: ruleFiles, Rules: ruleFiles,
Alerts: alertQueue, Alerts: alertQueue,
internal/smallwebwaf/healthcheck.go
+8 -10
View File
@@ -10,8 +10,6 @@ import (
"net/url" "net/url"
"time" "time"
"github.com/spf13/viper"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/servewaf" "sneak.berlin/go/smallwebwaf/internal/servewaf"
) )
@@ -33,7 +31,7 @@ var errHealthEndpoint = errors.New("the health endpoint answered")
// args are the arguments after `healthcheck`; it takes none, and given // args are the arguments after `healthcheck`; it takes none, and given
// one it names it on stderr and returns 1 without checking anything. // one it names it on stderr and returns 1 without checking anything.
func HealthCheck( func HealthCheck(
ctx context.Context, args []string, v *viper.Viper, ctx context.Context, args []string, lookupEnv func(string) (string, bool),
stderr io.Writer, stderr io.Writer,
) int { ) int {
if len(args) > 0 { if len(args) > 0 {
@@ -46,15 +44,15 @@ func HealthCheck(
ctx, cancel := context.WithTimeout(ctx, healthCheckTimeout) ctx, cancel := context.WithTimeout(ctx, healthCheckTimeout)
defer cancel() defer cancel()
command, err := config.Command(v) command, err := config.Command(lookupEnv)
switch { switch {
case err != nil: case err != nil:
err = fmt.Errorf("invalid setting: %w", err) err = fmt.Errorf("invalid setting: %w", err)
case command == "servewaf": case command == "servewaf":
err = checkServeWAF(ctx, v) err = checkServeWAF(ctx, lookupEnv)
default: default:
err = checkRProxyAndApp(ctx, v) err = checkRProxyAndApp(ctx, lookupEnv)
} }
if err != nil { if err != nil {
@@ -67,8 +65,8 @@ func HealthCheck(
} }
// checkServeWAF asks servewaf's health endpoint. // checkServeWAF asks servewaf's health endpoint.
func checkServeWAF(ctx context.Context, v *viper.Viper) error { func checkServeWAF(ctx context.Context, lookupEnv func(string) (string, bool)) error {
listenAddr, err := config.ServeWAFListenAddr(v) listenAddr, err := config.ServeWAFListenAddr(lookupEnv)
if err != nil { if err != nil {
return fmt.Errorf("invalid setting: %w", err) return fmt.Errorf("invalid setting: %w", err)
} }
@@ -83,9 +81,9 @@ func checkServeWAF(ctx context.Context, v *viper.Viper) error {
// checkRProxyAndApp asks rproxy's health endpoint, and connects to the app. // checkRProxyAndApp asks rproxy's health endpoint, and connects to the app.
func checkRProxyAndApp( func checkRProxyAndApp(
ctx context.Context, v *viper.Viper, ctx context.Context, lookupEnv func(string) (string, bool),
) error { ) error {
listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL(v) listenAddr, upstreamURL, err := config.RProxyListenAddrAndUpstreamURL(lookupEnv)
if err != nil { if err != nil {
return fmt.Errorf("invalid setting: %w", err) return fmt.Errorf("invalid setting: %w", err)
} }
internal/smallwebwaf/healthcheck_test.go
+10 -2
View File
@@ -145,7 +145,11 @@ func TestHealthCheckRefusesAnArgument(t *testing.T) {
var stderr bytes.Buffer var stderr bytes.Buffer
got := smallwebwaf.HealthCheck(t.Context(), []string{"now"}, viperFrom(nil), &stderr) noSettings := func(string) (string, bool) {
return "", false
}
got := smallwebwaf.HealthCheck(t.Context(), []string{"now"}, noSettings, &stderr)
want := "smallwebwaf healthcheck: unexpected argument \"now\"\n" want := "smallwebwaf healthcheck: unexpected argument \"now\"\n"
if got != 1 || stderr.String() != want { if got != 1 || stderr.String() != want {
@@ -162,7 +166,11 @@ func wantHealthCheck(t *testing.T, env map[string]string, status int, message st
var stderr bytes.Buffer var stderr bytes.Buffer
got := smallwebwaf.HealthCheck(t.Context(), nil, viperFrom(env), &stderr) got := smallwebwaf.HealthCheck(t.Context(), nil, func(name string) (string, bool) {
value, ok := env[name]
return value, ok
}, &stderr)
wrote := stderr.String() wrote := stderr.String()
if got != status || !strings.HasPrefix(wrote, message) || if got != status || !strings.HasPrefix(wrote, message) ||
internal/smallwebwaf/rproxy.go
+1 -1
View File
@@ -23,7 +23,7 @@ func RunRProxy(ctx context.Context, params Params) int {
hostname, _ := os.Hostname() // "" when the host has no name to give hostname, _ := os.Hostname() // "" when the host has no name to give
processLog := requestlog.NewProcessLogger(params.Stdout, hostname, slog.LevelInfo) processLog := requestlog.NewProcessLogger(params.Stdout, hostname, slog.LevelInfo)
cfg, err := config.RProxyFromEnvironment(params.Viper) cfg, err := config.RProxyFromEnvironment(params.LookupEnv)
if err != nil { if err != nil {
processLog.Error("invalid setting", "error", err.Error()) processLog.Error("invalid setting", "error", err.Error())
internal/smallwebwaf/rproxy_test.go
+6 -2
View File
@@ -43,8 +43,12 @@ const (
func runRProxy(ctx context.Context, env map[string]string, out *output) int { func runRProxy(ctx context.Context, env map[string]string, out *output) int {
return smallwebwaf.RunRProxy(ctx, smallwebwaf.Params{ return smallwebwaf.RunRProxy(ctx, smallwebwaf.Params{
Version: testVersion, Version: testVersion,
Viper: viperFrom(env), LookupEnv: func(name string) (string, bool) {
Stdout: out, value, ok := env[name]
return value, ok
},
Stdout: out,
}) })
} }
internal/smallwebwaf/smallwebwaf.go
+18 -12
View File
@@ -20,8 +20,6 @@ import (
"syscall" "syscall"
"time" "time"
"github.com/spf13/viper"
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup" "sneak.berlin/go/smallwebwaf/internal/lookup"
@@ -31,6 +29,7 @@ import (
"sneak.berlin/go/smallwebwaf/internal/rules" "sneak.berlin/go/smallwebwaf/internal/rules"
"sneak.berlin/go/smallwebwaf/internal/servewaf" "sneak.berlin/go/smallwebwaf/internal/servewaf"
"sneak.berlin/go/smallwebwaf/internal/state" "sneak.berlin/go/smallwebwaf/internal/state"
"sneak.berlin/go/smallwebwaf/internal/waf"
) )
// shutdownTimeout is how long requests in progress may take to finish // shutdownTimeout is how long requests in progress may take to finish
@@ -60,9 +59,8 @@ commands:
type Params struct { type Params struct {
// Version is the version of the binary, set when it is built. // Version is the version of the binary, set when it is built.
Version string Version string
// Viper reads the settings, normally the viper config.NewViper returns // LookupEnv reads an environment variable, normally os.LookupEnv.
// for the process's environment. LookupEnv func(string) (string, bool)
Viper *viper.Viper
// Stdout receives the request log and the process's own messages. // Stdout receives the request log and the process's own messages.
Stdout io.Writer Stdout io.Writer
} }
@@ -81,7 +79,7 @@ func Main(version string) int {
syscall.SIGTERM, os.Interrupt) syscall.SIGTERM, os.Interrupt)
defer stop() defer stop()
params := Params{Version: version, Viper: config.NewViper(), Stdout: os.Stdout} params := Params{Version: version, LookupEnv: os.LookupEnv, Stdout: os.Stdout}
switch args[0] { switch args[0] {
case "rproxy": case "rproxy":
@@ -89,7 +87,7 @@ func Main(version string) int {
case "servewaf": case "servewaf":
return RunServeWAF(ctx, params) return RunServeWAF(ctx, params)
case "healthcheck": case "healthcheck":
return HealthCheck(ctx, args[1:], params.Viper, os.Stderr) return HealthCheck(ctx, args[1:], os.LookupEnv, os.Stderr)
default: default:
return printUsage(os.Stderr) return printUsage(os.Stderr)
} }
@@ -109,9 +107,9 @@ func RunServeWAF(ctx context.Context, params Params) int {
// Until the settings are read, the one message is an invalid setting's // Until the settings are read, the one message is an invalid setting's
// error, which every SWWAF_LOG_LEVEL lets through. // error, which every SWWAF_LOG_LEVEL lets through.
processLog := requestlog.NewProcessLogger(params.Stdout, processLog := requestlog.NewProcessLogger(params.Stdout,
config.InstanceName(params.Viper), slog.LevelError) config.InstanceName(params.LookupEnv), slog.LevelError)
cfg, err := config.FromEnvironment(params.Viper) cfg, err := config.FromEnvironment(params.LookupEnv)
if err != nil { if err != nil {
processLog.Error("invalid setting", "error", err.Error()) processLog.Error("invalid setting", "error", err.Error())
@@ -153,7 +151,14 @@ func RunServeWAF(ctx context.Context, params Params) int {
return 1 return 1
} }
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue) crs, err := servewaf.NewCoreRuleSet(cfg)
if err != nil {
processLog.Error("cannot use the Core Rule Set", "error", err.Error())
return 1
}
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue, crs)
if err != nil { if err != nil {
processLog.Error("cannot use the lookup database", "error", err.Error()) processLog.Error("cannot use the lookup database", "error", err.Error())
@@ -173,8 +178,7 @@ func RunServeWAF(ctx context.Context, params Params) int {
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr) listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
if err != nil { if err != nil {
processLog.Error("cannot listen on SWWAF_LISTEN_ADDR", processLog.Error("cannot listen on SWWAF_LISTEN_ADDR", "error", err.Error())
"error", err.Error())
return 1 return 1
} }
@@ -193,6 +197,7 @@ func RunServeWAF(ctx context.Context, params Params) int {
func newServer( func newServer(
cfg *config.Config, stdout io.Writer, processLog *slog.Logger, cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue, now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
crs *waf.CoreRuleSet,
) (*servewaf.Server, error) { ) (*servewaf.Server, error) {
var lookupFile *lookup.File var lookupFile *lookup.File
@@ -217,6 +222,7 @@ func newServer(
GeoJSURL: lookup.URL, GeoJSURL: lookup.URL,
AbuseIPDBURL: reputation.AbuseIPDBURL, AbuseIPDBURL: reputation.AbuseIPDBURL,
LookupFile: lookupFile, LookupFile: lookupFile,
CoreRuleSet: crs,
Now: now, Now: now,
Rules: ruleFiles, Rules: ruleFiles,
Alerts: alertQueue, Alerts: alertQueue,
internal/smallwebwaf/smallwebwaf_test.go
+60 -20
View File
@@ -19,8 +19,7 @@ import (
"testing" "testing"
"time" "time"
"github.com/spf13/viper" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest" "sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
"sneak.berlin/go/smallwebwaf/internal/servewaf" "sneak.berlin/go/smallwebwaf/internal/servewaf"
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf" "sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
@@ -124,30 +123,21 @@ func (o *output) text() string {
return o.buf.String() return o.buf.String()
} }
// viperFrom returns a viper Set with each of settings, as the viper
// config.NewViper returns one bound to a process's environment.
func viperFrom(settings map[string]string) *viper.Viper {
v := viper.New()
for name, value := range settings {
v.Set(name, value)
}
return v
}
// run runs `smallwebwaf servewaf` with the settings in env until ctx is done, // run runs `smallwebwaf servewaf` with the settings in env until ctx is done,
// and returns its exit status. SWWAF_LOOKUP_SOURCE is off unless env sets it, // and returns its exit status. SWWAF_LOOKUP_SOURCE is off unless env sets it,
// so that no test sends GeoJS its clients' addresses. // so that no test sends GeoJS its clients' addresses.
func run(ctx context.Context, env map[string]string, out *output) int { func run(ctx context.Context, env map[string]string, out *output) int {
v := viperFrom(env)
if _, set := env["SWWAF_LOOKUP_SOURCE"]; !set {
v.Set("SWWAF_LOOKUP_SOURCE", "off")
}
return smallwebwaf.RunServeWAF(ctx, smallwebwaf.Params{ return smallwebwaf.RunServeWAF(ctx, smallwebwaf.Params{
Version: testVersion, Version: testVersion,
Viper: v, LookupEnv: func(name string) (string, bool) {
Stdout: out, value, ok := env[name]
if !ok && name == "SWWAF_LOOKUP_SOURCE" {
return "off", true
}
return value, ok
},
Stdout: out,
}) })
} }
@@ -562,6 +552,56 @@ func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
": no such file or directory") ": no such file or directory")
} }
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "crs-exclusions.conf")
out := &output{}
status := run(t.Context(), map[string]string{
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
"SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
}, out)
if status != 1 {
t.Fatalf("exit status %d, want 1", status)
}
line := out.line(t, "msg", "cannot use the Core Rule Set")
want := "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: read the exclusions file: open " +
path + ": no such file or directory"
if line["error"] != want {
t.Errorf("start refused with %v, want the error %q", line, want)
}
}
func TestCoreRuleSetExclusionsFileThatCannotBeUsedStopsTheStartWhileTheCoreRuleSetIsOff(
t *testing.T,
) {
t.Parallel()
broken := filepath.Join(t.TempDir(), "broken.conf")
err := os.WriteFile(broken, []byte("SecNoSuchDirective On\n"), 0o600)
if err != nil {
t.Fatal(err)
}
for _, path := range []string{filepath.Join(t.TempDir(), "missing.conf"), broken} {
out := &output{}
status := run(t.Context(), map[string]string{
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
"SWWAF_WAF_MODE": config.WAFModeOff, "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
}, out)
if status != 1 {
t.Fatalf("%s: exit status %d, want 1", path, status)
}
out.line(t, "msg", "cannot use the Core Rule Set")
}
}
func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) { func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
t.Parallel() t.Parallel()
internal/waf/gitea.conf
+236
View File
@@ -0,0 +1,236 @@
# The eighth change: on gitea's own paths, the fields in which gitea sends a
# search, a file name or a branch name skip the rules that refused ordinary
# gitea requests there. Each exclusion runs with the request, before the rules
# it switches off, those of phase 1 included, so this file is loaded before the
# Core Rule Set's rules. Each comment names a request the rule refused.
# gitea's searches send the text searched for in q: searches of code, of issues
# and pull requests, of commits, and of repositories, branches, tags, packages
# and users. q skips these rules there:
# 930110: ../
# 930120: .gitignore
# 931100: http://127.0.0.1:8080
# 932120: Invoke-WebRequest
# 932130: $(pwd)
# 932140: if (a == b) {
# 932160: /bin/sh
# 932235: `make check`
# 932250: ls -la
# 932260: python3
# 932380: `whoami`
# 933100: <?php echo
# 933130: $_GET[
# 933135: ${VAR}
# 934100: process.env
# 934110: http://127.0.0.1:8080
# 934130: __proto__
# 941100: <script>
# 941110: <script>
# 941160: <img src=
# 941180: document.cookie
# 941390: fetch(
# 942100: SELECT * FROM users WHERE
# 942140: information_schema
# 942190: union select
# 942230: if (a == b) {
# 944100: java.lang.Runtime
# 944110: java.lang.Runtime
# 944130: java.lang.Runtime
SecRule REQUEST_FILENAME "@rx ^/(?:explore/(?:repos|code)|issues(?:/search)?|pulls|\
user/search_candidates|[^/]+/-/(?:code|packages)|\
[^/]+/[^/]+/(?:search|issues(?:/search)?|pulls|branches|tags|\
commits/(?:branch|tag|commit)/.+/search)|\
api/v1/repos/(?:search|issues/search|[^/]+/[^/]+/issues))$" \
"id:900600,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930110;ARGS:q,\
ctl:ruleRemoveTargetById=930120;ARGS:q,\
ctl:ruleRemoveTargetById=931100;ARGS:q,\
ctl:ruleRemoveTargetById=932120;ARGS:q,\
ctl:ruleRemoveTargetById=932130;ARGS:q,\
ctl:ruleRemoveTargetById=932140;ARGS:q,\
ctl:ruleRemoveTargetById=932160;ARGS:q,\
ctl:ruleRemoveTargetById=932235;ARGS:q,\
ctl:ruleRemoveTargetById=932250;ARGS:q,\
ctl:ruleRemoveTargetById=932260;ARGS:q,\
ctl:ruleRemoveTargetById=932380;ARGS:q,\
ctl:ruleRemoveTargetById=933100;ARGS:q,\
ctl:ruleRemoveTargetById=933130;ARGS:q,\
ctl:ruleRemoveTargetById=933135;ARGS:q,\
ctl:ruleRemoveTargetById=934100;ARGS:q,\
ctl:ruleRemoveTargetById=934110;ARGS:q,\
ctl:ruleRemoveTargetById=934130;ARGS:q,\
ctl:ruleRemoveTargetById=941100;ARGS:q,\
ctl:ruleRemoveTargetById=941110;ARGS:q,\
ctl:ruleRemoveTargetById=941160;ARGS:q,\
ctl:ruleRemoveTargetById=941180;ARGS:q,\
ctl:ruleRemoveTargetById=941390;ARGS:q,\
ctl:ruleRemoveTargetById=942100;ARGS:q,\
ctl:ruleRemoveTargetById=942140;ARGS:q,\
ctl:ruleRemoveTargetById=942190;ARGS:q,\
ctl:ruleRemoveTargetById=942230;ARGS:q,\
ctl:ruleRemoveTargetById=944100;ARGS:q,\
ctl:ruleRemoveTargetById=944110;ARGS:q,\
ctl:ruleRemoveTargetById=944130;ARGS:q"
# A page opened from the results of one of those searches has the search's
# address in Referer, which skips these rules then:
# 941110: <script>
# 941120: onClick={
# 941160: <img src=
# 942100: LIKE '%foo%'
# 944100: java.lang.Runtime
SecRule REQUEST_HEADERS:Referer "@rx ^https?://[^/]+/(?:explore/(?:repos|code)|\
issues(?:/search)?|pulls|user/search_candidates|[^/]+/-/(?:code|packages)|\
[^/]+/[^/]+/(?:search|issues(?:/search)?|pulls|branches|tags|\
commits/(?:branch|tag|commit)/.+/search)|\
api/v1/repos/(?:search|issues/search|[^/]+/[^/]+/issues))\?" \
"id:900601,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=941110;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=941120;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=941160;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=942100;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=944100;REQUEST_HEADERS:Referer"
# A file's page, raw file and download, its blame, history, edit and delete
# pages, and the API's call for its contents name the file in the path, which
# skips these rules there:
# 920500: notes.txt~
# 934100: process.env.d.ts
# 941130: docs/a b.xhtml
# 944130: java.lang.String.md
SecRule REQUEST_FILENAME \
"@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
"id:900610,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=920500;REQUEST_FILENAME,\
ctl:ruleRemoveTargetById=934100;REQUEST_FILENAME,\
ctl:ruleRemoveTargetById=941130;REQUEST_FILENAME,\
ctl:ruleRemoveTargetById=944130;REQUEST_FILENAME"
# The requests below name a file in a query parameter. These rules refused
# them for the file named:
# 930120: .gitignore
# 932120: install-script.sh
# 932130: [draft].md
# 932160: bin/docker-entrypoint
# 932260: docker-compose.yml
# 934100: process.env.d.ts
# 934130: __proto__.json
# 941130: index.xhtml
# 941180: document.write.js
# 942140: information_schema.sql
# 944110: java/lang/Runtime.java
# 944130: java.lang.String.md
# Each request skips the rules that refused it. The fourth change already has
# path and redirect_to skip 930120, 932160 and 932260 on every path.
# A file's commit graph, in file.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/graph$" \
"id:900620,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:file,\
ctl:ruleRemoveTargetById=932120;ARGS:file,\
ctl:ruleRemoveTargetById=932160;ARGS:file,\
ctl:ruleRemoveTargetById=932260;ARGS:file,\
ctl:ruleRemoveTargetById=934100;ARGS:file,\
ctl:ruleRemoveTargetById=934130;ARGS:file,\
ctl:ruleRemoveTargetById=941130;ARGS:file,\
ctl:ruleRemoveTargetById=941180;ARGS:file,\
ctl:ruleRemoveTargetById=942140;ARGS:file,\
ctl:ruleRemoveTargetById=944110;ARGS:file,\
ctl:ruleRemoveTargetById=944130;ARGS:file"
# The API's list of a file's commits, in path.
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/commits$" \
"id:900621,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=932120;ARGS:path,\
ctl:ruleRemoveTargetById=934100;ARGS:path,\
ctl:ruleRemoveTargetById=934130;ARGS:path,\
ctl:ruleRemoveTargetById=941130;ARGS:path,\
ctl:ruleRemoveTargetById=941180;ARGS:path,\
ctl:ruleRemoveTargetById=942140;ARGS:path,\
ctl:ruleRemoveTargetById=944110;ARGS:path,\
ctl:ruleRemoveTargetById=944130;ARGS:path"
# The API's call for the contents of several files, in body, as JSON.
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/file-contents$" \
"id:900622,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:body,\
ctl:ruleRemoveTargetById=932120;ARGS:body,\
ctl:ruleRemoveTargetById=932160;ARGS:body,\
ctl:ruleRemoveTargetById=934100;ARGS:body,\
ctl:ruleRemoveTargetById=934130;ARGS:body,\
ctl:ruleRemoveTargetById=941130;ARGS:body,\
ctl:ruleRemoveTargetById=941180;ARGS:body,\
ctl:ruleRemoveTargetById=942140;ARGS:body,\
ctl:ruleRemoveTargetById=944110;ARGS:body,\
ctl:ruleRemoveTargetById=944130;ARGS:body"
# An issue opened from a template, in template.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/issues/new$" \
"id:900623,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:template,\
ctl:ruleRemoveTargetById=932120;ARGS:template,\
ctl:ruleRemoveTargetById=932130;ARGS:template,\
ctl:ruleRemoveTargetById=932160;ARGS:template,\
ctl:ruleRemoveTargetById=932260;ARGS:template,\
ctl:ruleRemoveTargetById=934100;ARGS:template,\
ctl:ruleRemoveTargetById=934130;ARGS:template,\
ctl:ruleRemoveTargetById=941130;ARGS:template,\
ctl:ruleRemoveTargetById=941180;ARGS:template,\
ctl:ruleRemoveTargetById=942140;ARGS:template,\
ctl:ruleRemoveTargetById=944110;ARGS:template,\
ctl:ruleRemoveTargetById=944130;ARGS:template"
# The sign-in page on the way to a file's edit page, in redirect_to.
SecRule REQUEST_FILENAME "@rx ^/user/login$" \
"id:900624,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=932120;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=932130;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=934100;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=934130;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=941130;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=941180;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=942140;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=944110;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=944130;ARGS:redirect_to"
# An attachment uploaded through the API to a release, an issue or a comment,
# in name.
SecRule REQUEST_FILENAME \
"@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
"id:900625,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:name,\
ctl:ruleRemoveTargetById=932120;ARGS:name,\
ctl:ruleRemoveTargetById=932260;ARGS:name,\
ctl:ruleRemoveTargetById=934100;ARGS:name,\
ctl:ruleRemoveTargetById=934130;ARGS:name,\
ctl:ruleRemoveTargetById=941130;ARGS:name,\
ctl:ruleRemoveTargetById=941180;ARGS:name,\
ctl:ruleRemoveTargetById=942140;ARGS:name,\
ctl:ruleRemoveTargetById=944130;ARGS:name"
# The requests below name a branch or tag in a query parameter, which skips
# the list of command names (932260) there. It refused names such as
# docker-build, python3-migration and feat/docker-support.
# Deleting or restoring a branch, in name.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/branches/(?:delete|restore)$" \
"id:900630,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:name"
# A release started from a tag, in tag.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/releases/new$" \
"id:900631,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:tag"
# A branch protection rule opened for editing, in rule_name.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/settings/branches/edit$" \
"id:900632,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:rule_name"
# A directory listing's last commits, in refSubUrl.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/lastcommit/" \
"id:900633,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:refSubUrl"
# An artifact upload from actions/upload-artifact@v3 sends the header
# Content-Range, which the Core Rule Set refused (920450).
SecRule REQUEST_FILENAME \
"@rx ^/api/actions_pipeline/_apis/pipelines/workflows/[0-9]+/artifacts/[^/]+/upload$" \
"id:900640,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=920450;REQUEST_HEADERS_NAMES:Content-Range"
internal/waf/testdata/gitea-requests.txt
Vendored
+367
View File
@@ -0,0 +1,367 @@
# Ordinary gitea 1.27 requests that the Core Rule Set must let through at the
# defaults. Each line is a path and query, and may end with one header after a
# space. The tests of internal/waf inspect each as a GET. The Core Rule Set reads
# the query and the headers the same way for any method.
# Searches with ordinary words and symbols.
/owner/repo/search?q=package.json
/owner/repo/search?q=.gitignore
/owner/repo/search?q=docker-compose.yml
/owner/repo/search?q=python3
/owner/repo/search?q=ssh%20key
/owner/repo/search?q=feat%2Fdocker-support
/owner/repo/search?q=ls%20-la
/owner/repo/search?q=sed%20-i
/owner/repo/search?q=%2Fbin%2Fsh
/owner/repo/search?q=%60make%20check%60
/owner/repo/search?q=%60whoami%60
/owner/repo/search?q=fetch%28
/owner/repo/search?q=%24%7BVAR%7D
/owner/repo/search?q=%24%28pwd%29
/owner/repo/search?q=process.env
/owner/repo/search?q=%3Cimg%20src%3D
/owner/repo/search?q=%3Cscript%3E
/owner/repo/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/search?q=union%20select
/owner/repo/search?q=http%3A%2F%2F127.0.0.1%3A8080
/owner/repo/search?q=..%2F
/owner/repo/search?q=%2Fetc%2Fpasswd
/owner/repo/search?q=eval%28
/owner/repo/search?q=chmod%20%2Bx
/owner/repo/search?q=echo%20%24PATH
/owner/repo/search?q=java.lang.Runtime
/owner/repo/search?q=document.cookie
/owner/repo/search?q=alert%281%29
/owner/repo/search?q=__proto__
/owner/repo/search?q=C%3A%5CWindows%5CSystem32
/owner/repo/search?q=Invoke-WebRequest
/owner/repo/search?q=if%20%28a%20%3D%3D%20b%29%20%7B
/owner/repo/search?q=sudo%20apt%20install
/owner/repo/search?q=~%2F.ssh%2Fconfig
/owner/repo/search?q=.htaccess
/owner/repo/search?q=%24_GET%5B
/owner/repo/search?q=%24_POST
/owner/repo/search?q=%24_SERVER%5B%27REQUEST_URI%27%5D
/owner/repo/search?q=%3C%3Fphp%20echo
/owner/repo/search?q=information_schema
/owner/repo/search?q=LIKE%20%27%25foo%25%27
/owner/repo/search?q=onClick%3D%7B
/explore/repos?q=SELECT%20%2A%20FROM%20users%20WHERE
/explore/repos?q=%3Cimg%20src%3D
/explore/code?q=SELECT%20%2A%20FROM%20users%20WHERE
/explore/code?q=%3Cimg%20src%3D
/owner/-/code?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/-/code?q=%3Cimg%20src%3D
/owner/repo/issues?type=all&state=open&q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/issues?type=all&state=open&q=%3Cimg%20src%3D
/owner/repo/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/issues/search?q=%3Cimg%20src%3D
/owner/repo/pulls?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/pulls?q=%3Cimg%20src%3D
/issues?type=your_repositories&q=SELECT%20%2A%20FROM%20users%20WHERE
/issues?type=your_repositories&q=%3Cimg%20src%3D
/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/issues/search?q=%3Cimg%20src%3D
/pulls?type=your_repositories&q=SELECT%20%2A%20FROM%20users%20WHERE
/pulls?type=your_repositories&q=%3Cimg%20src%3D
/owner/repo/commits/branch/main/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/commits/branch/main/search?q=%3Cimg%20src%3D
/owner/repo/branches?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/branches?q=%3Cimg%20src%3D
/owner/repo/tags?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/tags?q=%3Cimg%20src%3D
/owner/-/packages?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/-/packages?q=%3Cimg%20src%3D
/user/search_candidates?q=SELECT%20%2A%20FROM%20users%20WHERE
/user/search_candidates?q=%3Cimg%20src%3D
/api/v1/repos/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/api/v1/repos/search?q=%3Cimg%20src%3D
/api/v1/repos/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/api/v1/repos/issues/search?q=%3Cimg%20src%3D
/api/v1/repos/owner/repo/issues?q=SELECT%20%2A%20FROM%20users%20WHERE
/api/v1/repos/owner/repo/issues?q=%3Cimg%20src%3D
# Pages, raw files and edit pages of files, and the API's call for their
# contents.
/owner/repo/src/branch/main/package.json
/owner/repo/raw/branch/main/package.json
/owner/repo/_edit/main/package.json
/api/v1/repos/owner/repo/contents/package.json
/owner/repo/src/branch/main/.gitignore
/owner/repo/raw/branch/main/.gitignore
/owner/repo/_edit/main/.gitignore
/api/v1/repos/owner/repo/contents/.gitignore
/owner/repo/src/branch/main/docker-compose.yml
/owner/repo/raw/branch/main/docker-compose.yml
/owner/repo/_edit/main/docker-compose.yml
/api/v1/repos/owner/repo/contents/docker-compose.yml
/owner/repo/src/branch/main/Dockerfile
/owner/repo/raw/branch/main/Dockerfile
/owner/repo/_edit/main/Dockerfile
/api/v1/repos/owner/repo/contents/Dockerfile
/owner/repo/src/branch/main/.htaccess
/owner/repo/raw/branch/main/.htaccess
/owner/repo/_edit/main/.htaccess
/api/v1/repos/owner/repo/contents/.htaccess
/owner/repo/src/branch/main/rootfs/etc/passwd
/owner/repo/raw/branch/main/rootfs/etc/passwd
/owner/repo/_edit/main/rootfs/etc/passwd
/api/v1/repos/owner/repo/contents/rootfs/etc/passwd
/owner/repo/src/branch/main/bin/docker-entrypoint
/owner/repo/raw/branch/main/bin/docker-entrypoint
/owner/repo/_edit/main/bin/docker-entrypoint
/api/v1/repos/owner/repo/contents/bin/docker-entrypoint
/owner/repo/src/branch/main/scripts/python3-setup.sh
/owner/repo/raw/branch/main/scripts/python3-setup.sh
/owner/repo/_edit/main/scripts/python3-setup.sh
/api/v1/repos/owner/repo/contents/scripts/python3-setup.sh
/owner/repo/src/branch/main/src/java.base/share/classes/java/lang/Runtime.java
/owner/repo/raw/branch/main/src/java.base/share/classes/java/lang/Runtime.java
/owner/repo/_edit/main/src/java.base/share/classes/java/lang/Runtime.java
/api/v1/repos/owner/repo/contents/src/java.base/share/classes/java/lang/Runtime.java
/owner/repo/src/branch/main/java.lang.String.md
/owner/repo/raw/branch/main/java.lang.String.md
/owner/repo/_edit/main/java.lang.String.md
/api/v1/repos/owner/repo/contents/java.lang.String.md
/owner/repo/src/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
/owner/repo/raw/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
/owner/repo/_edit/main/META-INF/services/org.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/contents/META-INF/services/org.apache.commons.logging.LogFactory
/owner/repo/src/branch/main/process.env.d.ts
/owner/repo/raw/branch/main/process.env.d.ts
/owner/repo/_edit/main/process.env.d.ts
/api/v1/repos/owner/repo/contents/process.env.d.ts
/owner/repo/src/branch/main/document.write.js
/owner/repo/raw/branch/main/document.write.js
/owner/repo/_edit/main/document.write.js
/api/v1/repos/owner/repo/contents/document.write.js
/owner/repo/src/branch/main/test/fixtures/__proto__.json
/owner/repo/raw/branch/main/test/fixtures/__proto__.json
/owner/repo/_edit/main/test/fixtures/__proto__.json
/api/v1/repos/owner/repo/contents/test/fixtures/__proto__.json
/owner/repo/src/branch/main/install-script.sh
/owner/repo/raw/branch/main/install-script.sh
/owner/repo/_edit/main/install-script.sh
/api/v1/repos/owner/repo/contents/install-script.sh
/owner/repo/src/branch/main/src/backend/catalog/information_schema.sql
/owner/repo/raw/branch/main/src/backend/catalog/information_schema.sql
/owner/repo/_edit/main/src/backend/catalog/information_schema.sql
/api/v1/repos/owner/repo/contents/src/backend/catalog/information_schema.sql
/owner/repo/src/branch/main/docs/a%20b.xhtml
/owner/repo/raw/branch/main/docs/a%20b.xhtml
/owner/repo/_edit/main/docs/a%20b.xhtml
/api/v1/repos/owner/repo/contents/docs/a%20b.xhtml
/owner/repo/src/branch/main/%5Bdraft%5D.md
/owner/repo/raw/branch/main/%5Bdraft%5D.md
/owner/repo/_edit/main/%5Bdraft%5D.md
/api/v1/repos/owner/repo/contents/%5Bdraft%5D.md
/owner/repo/src/branch/main/notes.txt~
/owner/repo/raw/branch/main/notes.txt~
/owner/repo/_edit/main/notes.txt~
/api/v1/repos/owner/repo/contents/notes.txt~
# The download, blame, history and delete pages of the files above whose names
# the Core Rule Set checks in the path.
/owner/repo/media/branch/main/notes.txt~
/owner/repo/blame/branch/main/notes.txt~
/owner/repo/commits/branch/main/notes.txt~
/owner/repo/_delete/main/notes.txt~
/owner/repo/media/branch/main/process.env.d.ts
/owner/repo/blame/branch/main/process.env.d.ts
/owner/repo/commits/branch/main/process.env.d.ts
/owner/repo/_delete/main/process.env.d.ts
/owner/repo/media/branch/main/docs/a%20b.xhtml
/owner/repo/blame/branch/main/docs/a%20b.xhtml
/owner/repo/commits/branch/main/docs/a%20b.xhtml
/owner/repo/_delete/main/docs/a%20b.xhtml
/owner/repo/media/branch/main/java.lang.String.md
/owner/repo/blame/branch/main/java.lang.String.md
/owner/repo/commits/branch/main/java.lang.String.md
/owner/repo/_delete/main/java.lang.String.md
# File names in the query string: the commit graph of a file, the API's lists
# of its commits and of its contents, an issue opened from a template, and the
# sign-in page on the way to its edit page.
/owner/repo/graph?file=package.json
/api/v1/repos/owner/repo/commits?path=package.json
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22package.json%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fpackage.json
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fpackage.json
/owner/repo/graph?file=.gitignore
/api/v1/repos/owner/repo/commits?path=.gitignore
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.gitignore%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.gitignore
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.gitignore
/owner/repo/graph?file=docker-compose.yml
/api/v1/repos/owner/repo/commits?path=docker-compose.yml
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docker-compose.yml%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocker-compose.yml
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocker-compose.yml
/owner/repo/graph?file=.htaccess
/api/v1/repos/owner/repo/commits?path=.htaccess
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.htaccess%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.htaccess
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.htaccess
/owner/repo/graph?file=rootfs%2Fetc%2Fpasswd
/api/v1/repos/owner/repo/commits?path=rootfs%2Fetc%2Fpasswd
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22rootfs%2Fetc%2Fpasswd%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Frootfs%2Fetc%2Fpasswd
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Frootfs%2Fetc%2Fpasswd
/owner/repo/graph?file=bin%2Fdocker-entrypoint
/api/v1/repos/owner/repo/commits?path=bin%2Fdocker-entrypoint
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22bin%2Fdocker-entrypoint%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fbin%2Fdocker-entrypoint
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fbin%2Fdocker-entrypoint
/owner/repo/graph?file=scripts%2Fpython3-setup.sh
/api/v1/repos/owner/repo/commits?path=scripts%2Fpython3-setup.sh
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22scripts%2Fpython3-setup.sh%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fscripts%2Fpython3-setup.sh
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fscripts%2Fpython3-setup.sh
/owner/repo/graph?file=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/api/v1/repos/owner/repo/commits?path=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/owner/repo/graph?file=java.lang.String.md
/api/v1/repos/owner/repo/commits?path=java.lang.String.md
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22java.lang.String.md%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fjava.lang.String.md
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fjava.lang.String.md
/owner/repo/graph?file=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/commits?path=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/owner/repo/graph?file=process.env.d.ts
/api/v1/repos/owner/repo/commits?path=process.env.d.ts
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22process.env.d.ts%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fprocess.env.d.ts
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fprocess.env.d.ts
/owner/repo/graph?file=document.write.js
/api/v1/repos/owner/repo/commits?path=document.write.js
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22document.write.js%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocument.write.js
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocument.write.js
/owner/repo/graph?file=test%2Ffixtures%2F__proto__.json
/api/v1/repos/owner/repo/commits?path=test%2Ffixtures%2F__proto__.json
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22test%2Ffixtures%2F__proto__.json%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Ftest%2Ffixtures%2F__proto__.json
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Ftest%2Ffixtures%2F__proto__.json
/owner/repo/graph?file=install-script.sh
/api/v1/repos/owner/repo/commits?path=install-script.sh
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22install-script.sh%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Finstall-script.sh
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Finstall-script.sh
/owner/repo/graph?file=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
/api/v1/repos/owner/repo/commits?path=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fbackend%2Fcatalog%2Finformation_schema.sql%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
/owner/repo/graph?file=docs%2Fa%20b.xhtml
/api/v1/repos/owner/repo/commits?path=docs%2Fa%20b.xhtml
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docs%2Fa%20b.xhtml%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocs%2Fa%20b.xhtml
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocs%2Fa%20b.xhtml
/owner/repo/graph?file=%5Bdraft%5D.md
/api/v1/repos/owner/repo/commits?path=%5Bdraft%5D.md
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22%5Bdraft%5D.md%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F%5Bdraft%5D.md
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F%5Bdraft%5D.md
# File names of attachments uploaded through the API, to a release, an issue and
# a comment.
/api/v1/repos/owner/repo/releases/1/assets?name=package.json
/api/v1/repos/owner/repo/releases/1/assets?name=.gitignore
/api/v1/repos/owner/repo/releases/1/assets?name=docker-compose.yml
/api/v1/repos/owner/repo/releases/1/assets?name=.htaccess
/api/v1/repos/owner/repo/releases/1/assets?name=docker-entrypoint
/api/v1/repos/owner/repo/releases/1/assets?name=python3-setup.sh
/api/v1/repos/owner/repo/releases/1/assets?name=Runtime.java
/api/v1/repos/owner/repo/releases/1/assets?name=org.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/releases/1/assets?name=%5Bdraft%5D.md
/api/v1/repos/owner/repo/releases/1/assets?name=notes.txt~
/api/v1/repos/owner/repo/releases/1/assets?name=install-script.sh
/api/v1/repos/owner/repo/issues/1/assets?name=install-script.sh
/api/v1/repos/owner/repo/issues/comments/1/assets?name=install-script.sh
/api/v1/repos/owner/repo/releases/1/assets?name=process.env.d.ts
/api/v1/repos/owner/repo/issues/1/assets?name=process.env.d.ts
/api/v1/repos/owner/repo/issues/comments/1/assets?name=process.env.d.ts
/api/v1/repos/owner/repo/releases/1/assets?name=java.lang.String.md
/api/v1/repos/owner/repo/issues/1/assets?name=java.lang.String.md
/api/v1/repos/owner/repo/issues/comments/1/assets?name=java.lang.String.md
/api/v1/repos/owner/repo/releases/1/assets?name=information_schema.sql
/api/v1/repos/owner/repo/issues/1/assets?name=information_schema.sql
/api/v1/repos/owner/repo/issues/comments/1/assets?name=information_schema.sql
/api/v1/repos/owner/repo/releases/1/assets?name=document.write.js
/api/v1/repos/owner/repo/issues/1/assets?name=document.write.js
/api/v1/repos/owner/repo/issues/comments/1/assets?name=document.write.js
/api/v1/repos/owner/repo/releases/1/assets?name=__proto__.json
/api/v1/repos/owner/repo/issues/1/assets?name=__proto__.json
/api/v1/repos/owner/repo/issues/comments/1/assets?name=__proto__.json
/api/v1/repos/owner/repo/releases/1/assets?name=index.xhtml
/api/v1/repos/owner/repo/issues/1/assets?name=index.xhtml
/api/v1/repos/owner/repo/issues/comments/1/assets?name=index.xhtml
# An artifact upload from actions/upload-artifact@v3, which sends Content-Range.
/api/actions_pipeline/_apis/pipelines/workflows/1/artifacts/9b7c1f3a/upload?itemPath=artifact%2Freport.txt Content-Range: bytes 0-9/10
# Branch and tag names in the query string: deleting and restoring a branch,
# starting a release from a tag, a branch protection rule, a directory listing's
# last commits, and a container image push that reuses a layer.
/owner/repo/branches/delete?name=docker-build&page=1
/owner/repo/branches/restore?branch_id=1&name=docker-build&page=1
/owner/repo/releases/new?tag=docker-build
/owner/repo/settings/branches/edit?rule_name=docker-build
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fdocker-build
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fdocker-build
/owner/repo/branches/delete?name=feat%2Fdocker-support&page=1
/owner/repo/branches/restore?branch_id=1&name=feat%2Fdocker-support&page=1
/owner/repo/releases/new?tag=feat%2Fdocker-support
/owner/repo/settings/branches/edit?rule_name=feat%2Fdocker-support
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Ffeat%2Fdocker-support
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Ffeat%2Fdocker-support
/owner/repo/branches/delete?name=renovate%2Fdocker-build-push-action-6.x&page=1
/owner/repo/branches/restore?branch_id=1&name=renovate%2Fdocker-build-push-action-6.x&page=1
/owner/repo/releases/new?tag=renovate%2Fdocker-build-push-action-6.x
/owner/repo/settings/branches/edit?rule_name=renovate%2Fdocker-build-push-action-6.x
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Frenovate%2Fdocker-build-push-action-6.x
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Frenovate%2Fdocker-build-push-action-6.x
/owner/repo/branches/delete?name=python3-migration&page=1
/owner/repo/branches/restore?branch_id=1&name=python3-migration&page=1
/owner/repo/releases/new?tag=python3-migration
/owner/repo/settings/branches/edit?rule_name=python3-migration
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fpython3-migration
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fpython3-migration
/owner/repo/branches/delete?name=ansible-roles&page=1
/owner/repo/branches/restore?branch_id=1&name=ansible-roles&page=1
/owner/repo/releases/new?tag=ansible-roles
/owner/repo/settings/branches/edit?rule_name=ansible-roles
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fansible-roles
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fansible-roles
/owner/repo/branches/delete?name=whoami&page=1
/owner/repo/branches/restore?branch_id=1&name=whoami&page=1
/owner/repo/releases/new?tag=whoami
/owner/repo/settings/branches/edit?rule_name=whoami
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fwhoami
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fwhoami
# Pages a signed-in user opens.
/
/owner/repo
/owner/repo/issues
/owner/repo/pulls
/owner/repo/releases
/owner/repo/wiki
/owner/repo/activity
/owner/repo/branches
/owner/repo/tags
/owner/repo/settings
/owner/repo/actions
/owner/repo/compare/main...main
/owner/repo/commits/branch/main
/owner/repo/archive/main.zip
/notifications
/user/settings
/user/settings/applications
/explore/repos
/-/admin
internal/waf/waf.go
+110 -15
View File
@@ -1,8 +1,9 @@
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the // Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
// method, the URL with its query and the headers of a request, and on its // method, the URL with its query and the headers of a request, and on its
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf // body while SWWAF_WAF_BODY_LIMIT is set, with the eight changes smallwebwaf
// makes to it, as "Attack detection" under "Settings" in SPEC.md describes // makes to it and the exclusions in SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, as
// them. It reads no response. // "Attack detection" under "Settings" in SPEC.md describes them. It reads no
// response.
// //
// smallwebwaf writes only to its state directory, so Coraza is built with // smallwebwaf writes only to its state directory, so Coraza is built with
// its no_fs_access tag, as the Dockerfile and script/build build it: of a // its no_fs_access tag, as the Dockerfile and script/build build it: of a
@@ -11,10 +12,14 @@
package waf package waf
import ( import (
_ "embed"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
"net/netip" "net/netip"
"os"
"regexp"
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
@@ -26,12 +31,14 @@ import (
) )
// directives are the Core Rule Set as smallwebwaf runs it, with the // directives are the Core Rule Set as smallwebwaf runs it, with the
// paranoia level for %d, and bodyDirectives for %s while // paranoia level for %d, bodyDirectives for the first %s while
// SWWAF_WAF_BODY_LIMIT is set. Each rule smallwebwaf adds has an id from // SWWAF_WAF_BODY_LIMIT is set, the exclusions file for the second, and
// 900000 to 900999, the ids the Core Rule Set keeps for the rules that set // giteaExclusions for the third.
// it up, which SWWAF_WAF_DISABLED_RULES refuses, so that no setting // Each rule smallwebwaf adds has an id from 900000 to 900999, the ids the
// switches one off. Coraza joins a line ending in \ to the next, without // Core Rule Set keeps for the rules that set it up, which
// the spaces at the start of the next. // SWWAF_WAF_DISABLED_RULES and the exclusions file refuse, so that no
// setting switches one off. Coraza joins a line ending in \ to the next,
// without the spaces at the start of the next.
const directives = ` const directives = `
# The engine only detects. smallwebwaf compares the request's anomaly # The engine only detects. smallwebwaf compares the request's anomaly
# score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode # score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
@@ -69,6 +76,14 @@ SecArgumentsLimit 1000
SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\ SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'" severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
# The exclusions file comes before the rules, where the Core Rule Set puts
# its own file of exclusions that run before the rules, so that they run
# before the rules of phase 1 too.
%s
# The eighth change comes before the rules for the same reason.
%s
# The sixth: only the rules for requests are loaded, and no response is # The sixth: only the rules for requests are loaded, and no response is
# inspected. # inspected.
Include @owasp_crs/REQUEST-*.conf Include @owasp_crs/REQUEST-*.conf
@@ -96,10 +111,19 @@ SecRuleUpdateTargetById 932260 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
!ARGS:artifactName|!ARGS:redirect_to" !ARGS:artifactName|!ARGS:redirect_to"
# The fifth, for Referer: it is not checked for a Unix command without # The fifth, for Referer: it is not checked for a Unix command without
# arguments, or for Java starting a process. The cookies are left out in # arguments, for Java starting a process, or for a Java class name. 944130
# Inspect. # refused every page opened from the page of java.lang.String.md. The
# cookies are left out in Inspect.
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer" SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer" SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944130 "!REQUEST_HEADERS:Referer"
# The seventh: a container image push that reuses a layer names the image
# it takes the layer from in the query parameter from, which is not
# checked for command names. 932260 refused the push of a layer from
# owner/python3-app.
SecRule REQUEST_FILENAME "@rx ^/v2/.+/blobs/uploads/$" \
"id:900500,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:from"
` `
// bodyDirectives have the Core Rule Set read the part of a request body // bodyDirectives have the Core Rule Set read the part of a request body
@@ -144,6 +168,13 @@ SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'" severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
` `
// giteaExclusions are the eighth change: the fields of gitea's own
// requests that skip the rules that refused them there, as Coraza
// directives.
//
//go:embed gitea.conf
var giteaExclusions string
// cookiesNotRead are the cookies the Core Rule Set reads a request // cookiesNotRead are the cookies the Core Rule Set reads a request
// without, the rest of the fifth change. // without, the rest of the fifth change.
// //
@@ -160,6 +191,51 @@ type Params struct {
// BodyLimit is the most of a request body the Core Rule Set reads // BodyLimit is the most of a request body the Core Rule Set reads
// (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none. // (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
BodyLimit int64 BodyLimit int64
// ExclusionsFile is the path of a file of exclusions loaded before the
// Core Rule Set's rules (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
// there is none.
ExclusionsFile string
}
// The ids of smallwebwaf's own rules, which an exclusions file may not
// name.
const (
firstOwnRuleID = 900000
lastOwnRuleID = 900999
)
// numberOrRange matches a number, or a range of numbers such as
// 900000-900999.
var numberOrRange = regexp.MustCompile(`(\d+)(?:-(\d+))?`)
// errSmallwebwafRuleID is the error of an exclusions file that names an
// id of smallwebwaf's own rules.
var errSmallwebwafRuleID = errors.New("an id of smallwebwaf's own rules")
// ownRuleNamed returns the first number or range in exclusions, the text
// of an exclusions file, that takes in an id of smallwebwaf's own rules,
// or "" if none does. Comment lines are skipped, as Coraza skips them.
func ownRuleNamed(exclusions string) string {
for line := range strings.Lines(exclusions) {
if strings.HasPrefix(strings.TrimSpace(line), "#") {
continue
}
for _, match := range numberOrRange.FindAllStringSubmatch(line, -1) {
first, _ := strconv.Atoi(match[1])
last := first
if match[2] != "" {
last, _ = strconv.Atoi(match[2])
}
if first <= lastOwnRuleID && last >= firstOwnRuleID {
return match[0]
}
}
}
return ""
} }
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe // CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
@@ -169,16 +245,35 @@ type CoreRuleSet struct {
bodyLimit int64 bodyLimit int64
} }
// New returns the Core Rule Set with the six changes, at params' // New returns the Core Rule Set with the eight changes, at params'
// paranoia level, without the rules it switches off, and reading request // paranoia level, without the rules it switches off, reading request
// bodies up to params' limit. // bodies up to params' limit, and with params' exclusions file loaded
// before the Core Rule Set's rules. An exclusions file that cannot be
// read, does not parse or names an id of smallwebwaf's own rules is an
// error.
func New(params Params) (*CoreRuleSet, error) { func New(params Params) (*CoreRuleSet, error) {
body := "" body := ""
if params.BodyLimit > 0 { if params.BodyLimit > 0 {
body = fmt.Sprintf(bodyDirectives, params.BodyLimit) body = fmt.Sprintf(bodyDirectives, params.BodyLimit)
} }
text := fmt.Sprintf(directives, params.ParanoiaLevel, body) exclusions := ""
if params.ExclusionsFile != "" {
file, err := os.ReadFile(params.ExclusionsFile)
if err != nil {
return nil, fmt.Errorf("read the exclusions file: %w", err)
}
exclusions = string(file)
if named := ownRuleNamed(exclusions); named != "" {
return nil, fmt.Errorf("the exclusions file names %s: %w", named,
errSmallwebwafRuleID)
}
}
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions, giteaExclusions)
if len(params.DisabledRules) > 0 { if len(params.DisabledRules) > 0 {
ids := make([]string, len(params.DisabledRules)) ids := make([]string, len(params.DisabledRules))
internal/waf/waf_test.go
+167 -2
View File
@@ -5,6 +5,7 @@ import (
"net/http/httptest" "net/http/httptest"
"net/netip" "net/netip"
"net/url" "net/url"
"os"
"path/filepath" "path/filepath"
"reflect" "reflect"
"strconv" "strconv"
@@ -119,7 +120,7 @@ func atDefaults(t *testing.T) *waf.CoreRuleSet {
} }
// wantChange checks that crs lets through passes, a gitea request one of // wantChange checks that crs lets through passes, a gitea request one of
// the six changes is for, and still finds result in refused, a request // the eight changes is for, and still finds result in refused, a request
// like it that the change is not for. // like it that the change is not for.
func wantChange( func wantChange(
t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result, t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result,
@@ -308,13 +309,14 @@ func TestCookiesGiteaFlashAndRedirectToAreNotRead(t *testing.T) {
matched(930120)) matched(930120))
} }
func TestRefererIsNotCheckedForACommandOrJavaStartingAProcess(t *testing.T) { func TestRefererIsNotCheckedForACommandOrJava(t *testing.T) {
t.Parallel() t.Parallel()
const ( const (
search = "https://git.example/explore/repos?q=env" search = "https://git.example/explore/repos?q=env"
runtimeJava = "https://git.example/openjdk/jdk/src/branch/master/src/" + runtimeJava = "https://git.example/openjdk/jdk/src/branch/master/src/" +
"java.base/share/classes/java/lang/Runtime.java" "java.base/share/classes/java/lang/Runtime.java"
stringMD = "https://git.example/owner/repo/src/branch/main/java.lang.String.md"
) )
crs := atDefaults(t) crs := atDefaults(t)
@@ -323,6 +325,8 @@ func TestRefererIsNotCheckedForACommandOrJavaStartingAProcess(t *testing.T) {
matched(932340)) matched(932340))
wantChange(t, crs, get("/", "Referer: "+runtimeJava), wantChange(t, crs, get("/", "Referer: "+runtimeJava),
get("/", "X-Page: "+runtimeJava), matched(944110)) get("/", "X-Page: "+runtimeJava), matched(944110))
wantChange(t, crs, get("/", "Referer: "+stringMD), get("/", "X-Page: "+stringMD),
matched(944130))
// It is still checked for script and SQL injection. // It is still checked for script and SQL injection.
wantResult(t, crs, wantResult(t, crs,
@@ -362,6 +366,167 @@ func TestEachDisabledRuleIsSwitchedOff(t *testing.T) {
wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{}) wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{})
} }
func TestImagePushThatReusesALayerIsNotCheckedForCommandNames(t *testing.T) {
t.Parallel()
crs := atDefaults(t)
reuse := "?mount=sha256:0000&from=owner/python3-app"
wantChange(t, crs, get("/v2/owner/app/blobs/uploads/"+reuse), get("/upload/"+reuse),
matched(932260))
// A command after the image's name is still refused.
wantResult(t, crs,
get("/v2/owner/app/blobs/uploads/?from=owner/app;cat%20/etc/passwd"),
matched(930120, 932160))
}
// withExclusions returns the Core Rule Set at the defaults, with the
// exclusions file at path.
func withExclusions(path string) (*waf.CoreRuleSet, error) {
return waf.New(waf.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, ExclusionsFile: path,
})
}
// exclusionsFile writes text to a file of exclusions and returns its path.
func exclusionsFile(t *testing.T, text string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "exclusions.conf")
err := os.WriteFile(path, []byte(text), 0o600)
if err != nil {
t.Fatal(err)
}
return path
}
// backupFileExclusion is an exclusions file that switches 920500, a rule
// of phase 1, off for the path of a backup file under /files/.
const backupFileExclusion = `SecRule REQUEST_FILENAME "@beginsWith /files/" ` +
`"id:1,phase:1,pass,nolog,ctl:ruleRemoveTargetById=920500;REQUEST_FILENAME"` + "\n"
func TestExclusionsFileIsLoadedBeforeTheRules(t *testing.T) {
t.Parallel()
crs, err := withExclusions(exclusionsFile(t, backupFileExclusion))
if err != nil {
t.Fatalf("load the exclusions: %v", err)
}
wantChange(t, crs, get("/files/notes.txt~"), get("/notes.txt~"), matched(920500))
}
func TestRuleSwitchedOffAndNamedInTheExclusionsFileIsOff(t *testing.T) {
t.Parallel()
crs, err := waf.New(waf.Params{
ParanoiaLevel: 1, DisabledRules: []int{920500},
ExclusionsFile: exclusionsFile(t, backupFileExclusion),
})
if err != nil {
t.Fatalf("load the exclusions: %v", err)
}
wantResult(t, crs, get("/notes.txt~"), waf.Result{})
}
func TestExclusionsFileMayNameAnyNumberInAComment(t *testing.T) {
t.Parallel()
_, err := withExclusions(exclusionsFile(t,
"# 942100: GET /issues/900123\n"+backupFileExclusion))
if err != nil {
t.Errorf("the exclusions did not load: %v", err)
}
}
func TestExclusionsFileThatCannotBeUsedIsAnError(t *testing.T) {
t.Parallel()
for _, path := range []string{
filepath.Join(t.TempDir(), "missing.conf"),
exclusionsFile(t, "SecNoSuchDirective On\n"),
// The Core Rule Set's rules are not loaded yet.
exclusionsFile(t, "SecRuleUpdateTargetById 942100 \"!ARGS:id\"\n"),
// smallwebwaf's own rules may not be switched off or changed, by
// id or in a range.
exclusionsFile(t, "SecRuleRemoveById 900300\n"),
exclusionsFile(t, "SecRuleRemoveById 899999-901000\n"),
exclusionsFile(t, "SecAction \"id:1,phase:1,pass,nolog,"+
"ctl:ruleRemoveById=899999-901000\"\n"),
exclusionsFile(t, "SecAction \"id:1,phase:1,pass,nolog,"+
"ctl:ruleRemoveTargetById=900500;ARGS:from\"\n"),
} {
_, err := withExclusions(path)
if err == nil {
t.Errorf("%s loaded", path)
}
}
}
// giteaRequests returns the recorded ordinary gitea requests in testdata.
// Each has the Referer of a page that names the same, as a page opened
// from it has.
func giteaRequests(t *testing.T) []request {
t.Helper()
text, err := os.ReadFile(filepath.Join("testdata", "gitea-requests.txt"))
if err != nil {
t.Fatal(err)
}
var requests []request
for line := range strings.Lines(string(text)) {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") {
continue
}
target, header, _ := strings.Cut(line, " ")
headers := []string{"Referer: http://git.example" + target}
if header != "" {
headers = append(headers, header)
}
requests = append(requests, get(target, headers...))
}
return requests
}
func TestOrdinaryGiteaRequestsPassAndAttacksAreRefused(t *testing.T) {
t.Parallel()
const threshold = 5 // SWWAF_WAF_ANOMALY_THRESHOLD's default
crs := atDefaults(t)
for _, r := range giteaRequests(t) {
if got := inspect(t, crs, r); got.Score >= threshold {
t.Errorf("%s %q refused: %+v", r.target, r.headers[1:], got)
}
}
// In q too, off gitea's search pages.
for _, parameter := range []string{"id", "q"} {
for _, attack := range []string{
"1'%20OR%20'1'='1",
"..%2Fcustom%2Fconf%2Fapp.ini",
"%3Cscript%3Ealert(1)%3C%2Fscript%3E",
} {
target := "/owner/repo?" + parameter + "=" + attack
if got := inspect(t, crs, get(target)); got.Score < threshold {
t.Errorf("%s let through: %+v", target, got)
}
}
}
}
// bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies. // bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies.
const bodyLimit = 8 << 10 const bodyLimit = 8 << 10