Author SHA1 Message Date
clawbot 1d75ad2d48 Core Rule Set: gitea's ordinary requests pass by default (closes #198)
check / check (push) Failing after 13m57s
The eighth change, internal/waf/gitea.conf, has the fields where gitea
sends a search, a file name or a branch name skip the rules that refused
ordinary gitea requests, on gitea's paths only. Referer skips 944130 too.
A registry's layer reuse skips 932260 on from, the seventh change.

The new setting SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE loads an operator's
own exclusions before the Core Rule Set's rules. A set file that cannot
be read, does not parse or names smallwebwaf's own ids stops the start
in every mode.

A test sends the recorded gitea requests at the defaults, none refused,
and three attacks, all refused.

Model: opus-5-5
2026-11-07 02:15:39 +00:00
17 changed files with 1240 additions and 543 deletions
+3 -7
View File
@@ -1,5 +1,5 @@
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker publish \ .PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
hooks build run example-app gitea-requests gitea-setup run example-app gitea-requests gitea-setup
# Makefile targets are thin shims; the implementations live in script/ # Makefile targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern (see the Entrypoints section # per the scripts-to-rule-them-all pattern (see the Entrypoints section
@@ -7,8 +7,7 @@
# which test checks. build and run are for working on the code by hand; # which test checks. build and run are for working on the code by hand;
# example-app checks the image with an app built on it; gitea-requests # example-app checks the image with an app built on it; gitea-requests
# reports the gitea requests the Core Rule Set refuses at the defaults; # reports the gitea requests the Core Rule Set refuses at the defaults;
# gitea-setup checks the setup in deploy/gitea; publish builds the image # gitea-setup checks the setup in deploy/gitea.
# on the Docker host and pushes it to the registry.
bootstrap: bootstrap:
@script/bootstrap @script/bootstrap
@@ -37,9 +36,6 @@ check:
docker: docker:
@script/docker @script/docker
publish:
@script/publish
hooks: hooks:
@script/install-precommit @script/install-precommit
+10 -11
View File
@@ -192,9 +192,9 @@ account, a key, a token or an alert destination stays off until you give one.
- Any `SWWAF_` setting can be given as a file instead, for secrets and long - Any `SWWAF_` setting can be given as a file instead, for secrets and long
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
The file's contents are the value, less one newline at the end. Setting both The file's contents are the value, less one newline at the end. Setting both
forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` already names a file, so forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` and
it has no `_FILE` form, and no `SWRP_` setting has one, since `rproxy` never `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` already name a file, so they have no
touches disk. `_FILE` form. No `SWRP_` setting has one, since `rproxy` never touches disk.
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`. - A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of - A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
1024: `512K`, `100M`, `5G`. 1024: `512K`, `100M`, `5G`.
@@ -272,6 +272,7 @@ Every setting the code reads, with its default:
| `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. | | `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
| `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. | | `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
| `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. | | `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
| `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only. |
| `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. | | `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
| `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. | | `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
| `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. | | `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
@@ -636,10 +637,13 @@ and the status for a size or time limit, such as `413` for a body too large.
headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In
front of a code forge, bodies carry what people write and publish, which the front of a code forge, bodies carry what people write and publish, which the
rules take for attacks. rules take for attacks.
- `smallwebwaf` changes the Core Rule Set in six ways so that it lets ordinary - `smallwebwaf` changes the Core Rule Set in eight ways so that it lets ordinary
requests to a code forge through, such as git over HTTP and sign-in through requests to a code forge through, such as git over HTTP and sign-in through
Git Credential Manager. No setting undoes them; "Attack detection" in Git Credential Manager. On gitea's own paths, its searches, file names and
[`SPEC.md`](SPEC.md) lists them. branch names get through too. No setting undoes the changes; "Attack
detection" in [`SPEC.md`](SPEC.md) lists them.
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's
own fields or paths.
- Rule files are plain text, one rule to a line, edited by hand and read again - Rule files are plain text, one rule to a line, edited by hand and read again
while `servewaf` runs. The image ships one file of probes no real visitor while `servewaf` runs. The image ships one file of probes no real visitor
sends (see [`docs/rules.md`](docs/rules.md)). sends (see [`docs/rules.md`](docs/rules.md)).
@@ -743,11 +747,6 @@ on the host.
`make check`. `make check`.
- `script/docker`: builds the image, whose build runs the tests and the linter - `script/docker`: builds the image, whose build runs the tests and the linter
first. `make docker`. first. `make docker`.
- `script/publish`: runs `script/docker` on the Docker host `DOCKER_HOST`
(`ssh://root@fsn1app1` unless set) and pushes the image to `PUBLISH_IMAGE`
(`git.eeqj.de/sneak/smallwebwaf` unless set) under four tags: the UTC date as
`YYYY-MM-DD`, the short commit hash, the branch name and `latest`. Refuses a
working tree with uncommitted changes. `make publish`.
- `script/cibuild`: what CI runs: `script/bootstrap`, `script/check`, then the - `script/cibuild`: what CI runs: `script/bootstrap`, `script/check`, then the
image build. image build.
- `script/precommit`: run by the git pre-commit hook; runs `script/check`. - `script/precommit`: run by the git pre-commit hook; runs `script/check`.
+69 -16
View File
@@ -36,7 +36,8 @@ to them.
- the state files; - the state files;
- a lookup database; - a lookup database;
- a `SWWAF_` setting given as a file; - a `SWWAF_` setting given as a file;
- the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names. - the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names;
- the file `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names.
## Non-goals ## Non-goals
@@ -81,7 +82,7 @@ to them.
of bans is full. of bans is full.
- `internal/rules`: reads the rule files at start and again when they - `internal/rules`: reads the rule files at start and again when they
change, and tells which rules a request matches. change, and tells which rules a request matches.
- `internal/waf`: the Core Rule Set, with the six changes listed under - `internal/waf`: the Core Rule Set, with the eight changes listed under
"Attack detection", as Coraza directives. "Attack detection", as Coraza directives.
- `internal/lookup`: looks clients up through GeoJS or in the lookup - `internal/lookup`: looks clients up through GeoJS or in the lookup
database. database.
@@ -325,9 +326,10 @@ Conventions:
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`. lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
The file's contents are the value, less one newline at the end, and are The file's contents are the value, less one newline at the end, and are
checked as the value itself would be. Setting both forms stops the start, and checked as the value itself would be. Setting both forms stops the start, and
so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` and
already holds a file name, so it has no `_FILE` form. No `SWRP_` setting has `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` already hold a file name, so they have
one, since `rproxy` never touches disk. no `_FILE` form. No `SWRP_` setting has one, since `rproxy` never touches
disk.
- Settings, files included, are read once at start. Changing one means - Settings, files included, are read once at start. Changing one means
restarting the container. The files `servewaf` watches while it runs are its restarting the container. The files `servewaf` watches while it runs are its
state files, its rule files and the lookup database. state files, its rule files and the lookup database.
@@ -583,6 +585,24 @@ sends it.
probes these rules caught (see "Rule files"). A list you give replaces the probes these rules caught (see "Rule files"). A list you give replaces the
default, so name these six ids in it to keep them off. default, so name these six ids in it to keep them off.
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names a file of exclusions, written as
Coraza directives. An exclusion switches one rule off for some fields or paths
only, where `SWWAF_WAF_DISABLED_RULES` switches it off everywhere. The file
holds an app's own exclusions; the eighth change below holds gitea's.
- `servewaf` loads the file after `smallwebwaf`'s own setup rules and before
the Core Rule Set's rules. The Core Rule Set puts its own file of
exclusions that run before the rules in the same place.
- So the file holds only `SecAction` and `SecRule` lines that exclude with
`ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. These run with each
request, before the rules they exclude, those of phase 1 included.
- A rule that `SWWAF_WAF_DISABLED_RULES` switches off and the file names is
simply off.
- The start stops on a file that cannot be read or does not parse, whatever
`SWWAF_WAF_MODE` is.
- It stops too on a file that names a number from 900000 to 900999, the ids
of `smallwebwaf`'s own rules, alone or in a range. Comment lines are not
checked.
- `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. It takes - `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. It takes
at most `64K`, since `servewaf` gets no more of a body than that (see "The at most `64K`, since `servewaf` gets no more of a body than that (see "The
answer"). Form data and multipart bodies are read up to the size. JSON and XML answer"). Form data and multipart bodies are read up to the size. JSON and XML
@@ -612,7 +632,7 @@ sends it.
scanner from an ordinary client. In front of gitea, registry and git clients scanner from an ordinary client. In front of gitea, registry and git clients
get `404` and `401` by design, hundreds of times a minute. get `404` and `401` by design, hundreds of times a minute.
`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of `smallwebwaf` changes the Core Rule Set 4.25.0 in eight ways, since in front of
gitea it would otherwise refuse ordinary requests. The changes hold in front of gitea it would otherwise refuse ordinary requests. The changes hold in front of
every app, and no setting undoes them: every app, and no setting undoes them:
@@ -644,9 +664,10 @@ every app, and no setting undoes them:
or `docker-build`. In these parameters, what only these three rules would or `docker-build`. In these parameters, what only these three rules would
catch reaches the app, such as the command `|cat /etc/passwd`. Path catch reaches the app, such as the command `|cat /etc/passwd`. Path
traversal, SQL and script injection, and PHP, Java and Node.js code are traversal, SQL and script injection, and PHP, Java and Node.js code are
still refused there. Every other parameter keeps all three rules. An app still refused there. Other parameters keep all three rules, except on the
that uses one of these parameters as a file on the server, or passes it to a gitea paths the eighth change names. An app that uses one of these
shell, must check the value itself. A rule file can refuse it instead. parameters as a file on the server, or passes it to a shell, must check the
value itself. A rule file can refuse it instead.
5. Two cookies are left out, and `Referer` keeps fewer rules: 5. Two cookies are left out, and `Referer` keeps fewer rules:
- The Core Rule Set reads the request without the `gitea_flash` and - The Core Rule Set reads the request without the `gitea_flash` and
@@ -660,15 +681,47 @@ every app, and no setting undoes them:
only to its own pages. Every other cookie is read in full. An app that only to its own pages. Every other cookie is read in full. An app that
reads a cookie of either name must check it itself. reads a cookie of either name must check it itself.
- `Referer` is not checked for a Unix command given without arguments - `Referer` is not checked for a Unix command given without arguments
(932340) or for Java starting a process (944110). `Referer` names the page (932340), for Java starting a process (944110) or for a Java class name
a request comes from, on everything that page loads and on every link (944130). `Referer` names the page a request comes from, on everything
followed from it. The Core Rule Set takes some page addresses for attacks, that page loads and on every link followed from it. The Core Rule Set
such as a search for `env`, or OpenJDK's `Runtime.java`, whose path holds takes some page addresses for attacks, such as a search for `env`,
both `runtime` and `java.`. All of those requests would be refused. OpenJDK's `Runtime.java`, whose path holds both `runtime` and `java.`, or
`Referer` keeps the other rules. An app that passes `Referer` to a shell a file named `java.lang.String.md`. All of those requests would be
must check it itself. refused. `Referer` keeps the other rules, except after a gitea search (the
eighth change). An app that passes `Referer` to a shell must check it
itself.
6. Responses are not inspected. A raw file from a repository, such as a shell 6. Responses are not inspected. A raw file from a repository, such as a shell
script, looks to the response rules like source code leaking from the server. script, looks to the response rules like source code leaking from the server.
7. The `from` parameter of a container registry's upload path,
`/v2/<image>/blobs/uploads/`, is not checked for command names (932260). When
a push reuses a layer, `from` names the image the layer comes from. An image
such as `owner/python3-app` would have that push refused.
8. On gitea's own paths, the fields where gitea sends a search, a file name or a
branch name skip the rules that refused ordinary gitea requests there.
`internal/waf/gitea.conf` lists each rule and the request it refused. The
fields are:
- `q` on gitea's search pages, for code, issues, pull requests, commits,
repositories, branches, tags, packages and users. A code search holds
code, such as `<script>`, `SELECT * FROM users WHERE` or `../`.
- `Referer` of a page opened from such a search, for script and SQL
injection and Java code (941110, 941120, 941160, 942100, 944100).
- The path of a file's page, raw file, download, blame, history, edit and
delete pages, and of the API's call for its contents. Names such as
`notes.txt~`, `process.env.d.ts` and `java.lang.String.md` were refused.
- A file name in a query parameter: `file` of a commit graph, `path` of the
API's list of commits, `body` of the API's call for several files,
`template` of a new issue, `redirect_to` of the sign-in page, and `name`
of an attachment uploaded through the API.
- A branch or tag name in a query parameter, for command names (932260).
gitea sends one to delete or restore a branch, to start a release from a
tag, to edit a branch protection rule, and to list a directory's last
commits.
- The `Content-Range` header of an artifact upload from
`actions/upload-artifact@v3` (920450).
Elsewhere these fields keep every rule. The paths are those of gitea at the
site root. An app other than gitea that serves one of them must check these
fields itself.
The parameter and cookie names in these changes are matched without regard to The parameter and cookie names in these changes are matched without regard to
case, as Coraza matches them. case, as Coraza matches them.
+7
View File
@@ -192,6 +192,13 @@ chooses between the two (see "The image's command" below).
the Core Rule Set reads, a size of at most `64K`. A question carries no more the Core Rule Set reads, a size of at most `64K`. A question carries no more
of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one
byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)). byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule Set
exclusions, written as Coraza directives. It is loaded before the Core Rule
Set's rules, so it holds only `SecAction` and `SecRule` lines that exclude
with `ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. A file that cannot be
read or does not parse stops the start, whatever `SWWAF_WAF_MODE` is. So does
a file that names a number from 900000 to 900999 outside a comment, alone or
in a range. It has no `_FILE` form.
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client - `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
in a minute (see "Attack detection" in `SPEC.md`). A count or `off`. in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
+159 -353
View File
@@ -7,9 +7,6 @@
// the image runs. // the image runs.
package config package config
// FIXME: the rproxy config is in rproxy.go, the servewaf is in config.go -
// wrong and inconsistent.
import ( import (
"crypto/x509" "crypto/x509"
"errors" "errors"
@@ -224,12 +221,15 @@ type Config struct {
// WAFExemptPaths the path prefixes it does not inspect // WAFExemptPaths the path prefixes it does not inspect
// (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body // (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body
// it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none. // it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
WAFMode string // CoreRuleSetExclusionsFile is the path of the file of its exclusions
WAFParanoiaLevel int // (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while there is none.
WAFAnomalyThreshold int WAFMode string
WAFDisabledRules []int WAFParanoiaLevel int
WAFExemptPaths []string WAFAnomalyThreshold int
WAFBodyLimit int64 WAFDisabledRules []int
WAFExemptPaths []string
WAFBodyLimit int64
CoreRuleSetExclusionsFile string
// TrapPaths are the paths a request for which is a clear sign of // TrapPaths are the paths a request for which is a clear sign of
// attack (SWWAF_TRAP_PATHS), each starting with / and without a ?. // attack (SWWAF_TRAP_PATHS), each starting with / and without a ?.
TrapPaths []string TrapPaths []string
@@ -366,16 +366,13 @@ var (
"is not a header name such as accept-language") "is not a header name such as accept-language")
errHeaderTakenOut = errors.New( errHeaderTakenOut = errors.New(
"is taken out of every request by Go's HTTP server, so it can never " + "is taken out of every request by Go's HTTP server, so it can never " +
"be logged", "be logged")
)
errOnBothLists = errors.New("is in SWWAF_DENIED_COUNTRIES too") errOnBothLists = errors.New("is in SWWAF_DENIED_COUNTRIES too")
errNotLookupSource = errors.New("is not geojs, file or off") errNotLookupSource = errors.New("is not geojs, file or off")
errNeedsLookups = errors.New("it needs each client looked up") errNeedsLookups = errors.New("it needs each client looked up")
errNeedsDBPath = errors.New( errNeedsDBPath = errors.New("it names the file to look clients up in")
"it names the file to look clients up in", errDBPathUnused = errors.New("only file reads it")
) errOver64K = errors.New(
errDBPathUnused = errors.New("only file reads it")
errOver64K = errors.New(
"is more than 64K, the most servewaf gets of a body") "is more than 64K, the most servewaf gets of a body")
errNotCommand = errors.New("is not rproxy or servewaf") errNotCommand = errors.New("is not rproxy or servewaf")
errNotDurationAboveZero = errors.New( errNotDurationAboveZero = errors.New(
@@ -400,29 +397,22 @@ var (
errNotPathPrefix = errors.New( errNotPathPrefix = errors.New(
"is not a path prefix starting with /, such as /assets/") "is not a path prefix starting with /, such as /assets/")
errNotTrapPath = errors.New( errNotTrapPath = errors.New(
"is not a path starting with / and without a ?, such as /wp-login.php", "is not a path starting with / and without a ?, such as /wp-login.php")
)
errNotWAFMode = errors.New("is not off, detect or block") errNotWAFMode = errors.New("is not off, detect or block")
errNotParanoiaLevel = errors.New("is not a paranoia level, from 1 to 4") errNotParanoiaLevel = errors.New("is not a paranoia level, from 1 to 4")
errNotRuleID = errors.New( errNotRuleID = errors.New(
"is not the id of a Core Rule Set rule, a whole number such as 942100", "is not the id of a Core Rule Set rule, a whole number such as 942100")
)
errSetupRuleID = errors.New( errSetupRuleID = errors.New(
"is from 900000 to 900999, the ids of the rules that set the Core Rule Set " + "is from 900000 to 900999, the ids of the rules that set the Core Rule Set " +
"up and of smallwebwaf's own, which cannot be switched off", "up and of smallwebwaf's own, which cannot be switched off")
)
errNotBoolean = errors.New("is not true or false") errNotBoolean = errors.New("is not true or false")
errNotLogRemoteURL = errors.New( errNotLogRemoteURL = errors.New(
"is not syslog+udp, syslog+tcp or syslog+tls with a host and a port, " + "is not syslog+udp, syslog+tcp or syslog+tls with a host and a port, " +
"and nothing more, such as syslog+tls://logs.example:6514", "and nothing more, such as syslog+tls://logs.example:6514")
)
errNoCertificate = errors.New("holds no PEM certificate") errNoCertificate = errors.New("holds no PEM certificate")
errNotFacility = errors.New( errNotFacility = errors.New("is not a syslog facility such as local0 or daemon")
"is not a syslog facility such as local0 or daemon", errNotAppName = errors.New(
) "is not 1 to 48 printable ASCII characters without a space, such as gitea")
errNotAppName = errors.New(
"is not 1 to 48 printable ASCII characters without a space, such as gitea",
)
errSetTwice = errors.New("set only one of them") errSetTwice = errors.New("set only one of them")
errNotWebhookURL = errors.New( errNotWebhookURL = errors.New(
"is not an http or https URL without a user or a fragment, " + "is not an http or https URL without a user or a fragment, " +
@@ -431,50 +421,35 @@ var (
"is not a header name followed by : and the header's value, " + "is not a header name followed by : and the header's value, " +
"such as Authorization:Bearer <token>") "such as Authorization:Bearer <token>")
errControlCharacter = errors.New( errControlCharacter = errors.New(
"holds a control character, such as the carriage return of a Windows line end", "holds a control character, such as the carriage return of a Windows line end")
)
errNotAlertEvent = errors.New( errNotAlertEvent = errors.New(
"is not ban, permanent_ban, waf_block, anomaly, reputation_hit, " + "is not ban, permanent_ban, waf_block, anomaly, reputation_hit, " +
"source_failure or file_error") "source_failure or file_error")
errNotNumberOrOff = errors.New( errNotNumberOrOff = errors.New("is not a whole number above zero, such as 60, or off")
"is not a whole number above zero, such as 60, or off",
)
errNotUTF8 = errors.New("is not valid UTF-8") errNotUTF8 = errors.New("is not valid UTF-8")
errNotASN = errors.New("is not an AS number such as AS64496") errNotASN = errors.New("is not an AS number such as AS64496")
errNotPercentItem = errors.New( errNotPercentItem = errors.New(
"is not a code, : and a percentage, such as AS64496:50 or cn:25") "is not a code, : and a percentage, such as AS64496:50 or cn:25")
errNotPercent = errors.New( errNotPercent = errors.New("is not a percentage, a whole number from 0 to 100")
"is not a percentage, a whole number from 0 to 100",
)
errListedTwice = errors.New("is listed twice") errListedTwice = errors.New("is listed twice")
errNotListURL = errors.New( errNotListURL = errors.New(
"is not an http or https URL without a user or a fragment, " + "is not an http or https URL without a user or a fragment, " +
"such as https://www.spamhaus.org/drop/drop.txt") "such as https://www.spamhaus.org/drop/drop.txt")
errInBlocklistURLs = errors.New("is in SWWAF_BLOCKLIST_URLS too") errInBlocklistURLs = errors.New("is in SWWAF_BLOCKLIST_URLS too")
errNotAnHourOrMore = errors.New( errNotAnHourOrMore = errors.New("is not a duration of 1h or more, such as 24h")
"is not a duration of 1h or more, such as 24h", errNotAction = errors.New(
)
errNotAction = errors.New(
"is not deny, limit:<percent> such as limit:25, or log") "is not deny, limit:<percent> such as limit:25, or log")
errNotZone = errors.New( errNotZone = errors.New("is not a DNS zone such as dnsbl.dronebl.org")
"is not a DNS zone such as dnsbl.dronebl.org", errZoneTooLong = errors.New("is longer than 189 characters, too long for the " +
) "names IPv6 clients are asked about by")
errZoneTooLong = errors.New( errNotResolver = errors.New("is not an IP address with an optional port, " +
"is longer than 189 characters, too long for the " + "such as 192.0.2.53 or [2001:db8::53]:5353")
"names IPv6 clients are asked about by",
)
errNotResolver = errors.New(
"is not an IP address with an optional port, " +
"such as 192.0.2.53 or [2001:db8::53]:5353",
)
errNotLAPIURL = errors.New( errNotLAPIURL = errors.New(
"is not an http or https URL without a user or a fragment, " + "is not an http or https URL without a user or a fragment, " +
"such as http://172.17.0.1:8080") "such as http://172.17.0.1:8080")
errNeedsLAPIKey = errors.New("the engine answers no request without it") errNeedsLAPIKey = errors.New("the engine answers no request without it")
errLAPIKeyUnused = errors.New( errLAPIKeyUnused = errors.New("it is sent only to the engine at that URL")
"it is sent only to the engine at that URL", errAnotherList = errors.New(
)
errAnotherList = errors.New(
"is in SWWAF_BLOCKLIST_URLS or is SWWAF_ASN_LIMIT_PERCENT_URL too") "is in SWWAF_BLOCKLIST_URLS or is SWWAF_ASN_LIMIT_PERCENT_URL too")
) )
@@ -485,208 +460,99 @@ var (
// setting that is set but invalid is an error that names it. // setting that is set but invalid is an error that names it.
// //
//nolint:funlen // one line for each setting, a list that grows with them //nolint:funlen // one line for each setting, a list that grows with them
func FromEnvironment( func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
lookupEnv func(string) (string, bool),
) (*Config, error) {
env := &environment{lookupEnv: lookupEnv} env := &environment{lookupEnv: lookupEnv}
cfg := &Config{ cfg := &Config{
ListenAddr: env.address("SWWAF_LISTEN_ADDR"), ListenAddr: env.address("SWWAF_LISTEN_ADDR"),
InstanceName: env.instanceName(), InstanceName: env.instanceName(),
Observe: env.observe("SWWAF_MODE", "enforce"), Observe: env.observe("SWWAF_MODE", "enforce"),
IPv6GroupPrefix: env.ipv6GroupPrefix( IPv6GroupPrefix: env.ipv6GroupPrefix("SWWAF_IPV6_GROUP_PREFIX", "64"),
"SWWAF_IPV6_GROUP_PREFIX", MaxTrackedClients: env.numberNotOff("SWWAF_MAX_TRACKED_CLIENTS", "20000"),
"64", AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""),
), RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
MaxTrackedClients: env.numberNotOff( DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
"SWWAF_MAX_TRACKED_CLIENTS", RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "120"),
"20000", RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "1500"),
), RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "6000"),
AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""), RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
RateLimitExemptNets: env.netblocks( BytesLimitPerMinute: env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "off"),
"SWWAF_RATE_LIMIT_EXEMPT_NETS", BytesLimitPerHour: env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "50G"),
"", BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "100G"),
), BytesCount: env.bytesCount("SWWAF_BYTES_COUNT", "both"),
DenyNets: env.netblocks("SWWAF_DENY_NETS", ""), LookupSource: env.lookupSource("SWWAF_LOOKUP_SOURCE", "geojs"),
RateLimitPerMinute: env.count( LookupDBPath: env.value("SWWAF_LOOKUP_DB_PATH", ""),
"SWWAF_RATE_LIMIT_PER_MINUTE", LookupTimeout: env.durationNotOff("SWWAF_LOOKUP_TIMEOUT", "800ms"),
"120", AddLookupHeaders: env.boolean("SWWAF_ADD_LOOKUP_HEADERS", "false"),
), DeniedCountries: env.countries("SWWAF_DENIED_COUNTRIES", ""),
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "1500"),
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "6000"),
RateLimitExemptPaths: env.pathPrefixes(
"SWWAF_RATE_LIMIT_EXEMPT_PATHS",
"",
),
BytesLimitPerMinute: env.size(
"SWWAF_BYTES_LIMIT_PER_MINUTE",
"off",
),
BytesLimitPerHour: env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "50G"),
BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "100G"),
BytesCount: env.bytesCount("SWWAF_BYTES_COUNT", "both"),
LookupSource: env.lookupSource(
"SWWAF_LOOKUP_SOURCE",
"geojs",
),
LookupDBPath: env.value("SWWAF_LOOKUP_DB_PATH", ""),
LookupTimeout: env.durationNotOff(
"SWWAF_LOOKUP_TIMEOUT",
"800ms",
),
AddLookupHeaders: env.boolean(
"SWWAF_ADD_LOOKUP_HEADERS",
"false",
),
DeniedCountries: env.countries("SWWAF_DENIED_COUNTRIES", ""),
ExclusivelyAllowedCountries: env.countries( ExclusivelyAllowedCountries: env.countries(
"SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES", ""), "SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES", ""),
ASNLimitPercent: env.percents( ASNLimitPercent: env.percents("SWWAF_ASN_LIMIT_PERCENT", ParseASN),
"SWWAF_ASN_LIMIT_PERCENT", CountryLimitPercent: env.percents("SWWAF_COUNTRY_LIMIT_PERCENT", parseCountry),
ParseASN, ASNBytesPercent: env.percents("SWWAF_ASN_BYTES_PERCENT", ParseASN),
), CountryBytesPercent: env.percents("SWWAF_COUNTRY_BYTES_PERCENT", parseCountry),
CountryLimitPercent: env.percents( UnknownLimitPercent: env.percent("SWWAF_UNKNOWN_LIMIT_PERCENT", "100"),
"SWWAF_COUNTRY_LIMIT_PERCENT", ASNLimitPercentURL: env.listURL("SWWAF_ASN_LIMIT_PERCENT_URL"),
parseCountry, BlocklistURLs: env.listURLs("SWWAF_BLOCKLIST_URLS"),
), BlocklistRefresh: env.refresh("SWWAF_BLOCKLIST_REFRESH", "24h"),
ASNBytesPercent: env.percents( DNSBLZones: env.zones("SWWAF_DNSBL_ZONES"),
"SWWAF_ASN_BYTES_PERCENT", DNSBLResolver: env.resolver("SWWAF_DNSBL_RESOLVER"),
ParseASN, AbuseIPDBKey: env.secret("SWWAF_ABUSEIPDB_KEY"),
), AbuseIPDBMinScore: env.percent("SWWAF_ABUSEIPDB_MIN_SCORE", "75"),
CountryBytesPercent: env.percents( AbuseIPDBDailyBudget: env.numberNotOff("SWWAF_ABUSEIPDB_DAILY_BUDGET", "900"),
"SWWAF_COUNTRY_BYTES_PERCENT", ReputationCacheTTL: env.durationNotOff("SWWAF_REPUTATION_CACHE_TTL", "24h"),
parseCountry, ReputationTimeout: env.durationNotOff("SWWAF_REPUTATION_TIMEOUT", "2s"),
), CrowdSecDecisionsURL: env.crowdSecDecisionsURL("SWWAF_CROWDSEC_LAPI_URL"),
UnknownLimitPercent: env.percent( CrowdSecKey: env.secret("SWWAF_CROWDSEC_LAPI_KEY"),
"SWWAF_UNKNOWN_LIMIT_PERCENT", BanResponse: env.banResponse("SWWAF_BAN_RESPONSE", "403"),
"100", LimitBanDuration: env.durationNotOff("SWWAF_LIMIT_BAN_DURATION", "1h"),
), LimitBanRepeatWindow: env.durationNotOff("SWWAF_LIMIT_BAN_REPEAT_WINDOW", "24h"),
ASNLimitPercentURL: env.listURL("SWWAF_ASN_LIMIT_PERCENT_URL"), MaxBanDuration: env.durationNotOff("SWWAF_MAX_BAN_DURATION", "7d"),
BlocklistURLs: env.listURLs("SWWAF_BLOCKLIST_URLS"), AttackBanDuration: env.durationNotOff("SWWAF_ATTACK_BAN_DURATION", "7d"),
BlocklistRefresh: env.refresh("SWWAF_BLOCKLIST_REFRESH", "24h"), MaxBans: env.numberNotOff("SWWAF_MAX_BANS", "5000"),
DNSBLZones: env.zones("SWWAF_DNSBL_ZONES"), BanScopeV4Prefix: env.v4Prefix("SWWAF_BAN_SCOPE_V4_PREFIX", "32"),
DNSBLResolver: env.resolver("SWWAF_DNSBL_RESOLVER"), StateDir: env.absolutePath("SWWAF_STATE_DIR", "/var/lib/smallwebwaf"),
AbuseIPDBKey: env.secret("SWWAF_ABUSEIPDB_KEY"), StateWriteDelay: env.durationNotOff("SWWAF_STATE_WRITE_DELAY", "10s"),
AbuseIPDBMinScore: env.percent("SWWAF_ABUSEIPDB_MIN_SCORE", "75"), StateCounterInterval: env.durationNotOff("SWWAF_STATE_COUNTER_INTERVAL", "15m"),
AbuseIPDBDailyBudget: env.numberNotOff( LogRequestHeaders: env.headerNames("SWWAF_LOG_REQUEST_HEADERS",
"SWWAF_ABUSEIPDB_DAILY_BUDGET", "accept,accept-language,accept-encoding,content-type,origin,range"),
"900", LogLevel: env.logLevel("SWWAF_LOG_LEVEL", "info"),
), AdminToken: env.token("SWWAF_ADMIN_TOKEN"),
ReputationCacheTTL: env.durationNotOff( MetricsToken: env.token("SWWAF_METRICS_TOKEN"),
"SWWAF_REPUTATION_CACHE_TTL", MetricsTopN: env.numberNotOff("SWWAF_METRICS_TOP_N", "50"),
"24h", RulesDir: env.value("SWWAF_RULES_DIR", "/etc/smallwebwaf/rules.d"),
), RulesEnabled: env.boolean("SWWAF_RULES_ENABLED", "true"),
ReputationTimeout: env.durationNotOff( WAFMode: env.wafMode("SWWAF_WAF_MODE", WAFModeBlock),
"SWWAF_REPUTATION_TIMEOUT", WAFParanoiaLevel: env.paranoiaLevel("SWWAF_WAF_PARANOIA_LEVEL", "1"),
"2s", WAFAnomalyThreshold: env.numberOrOff("SWWAF_WAF_ANOMALY_THRESHOLD", "5"),
),
CrowdSecDecisionsURL: env.crowdSecDecisionsURL(
"SWWAF_CROWDSEC_LAPI_URL",
),
CrowdSecKey: env.secret("SWWAF_CROWDSEC_LAPI_KEY"),
BanResponse: env.banResponse("SWWAF_BAN_RESPONSE", "403"),
LimitBanDuration: env.durationNotOff(
"SWWAF_LIMIT_BAN_DURATION",
"1h",
),
LimitBanRepeatWindow: env.durationNotOff(
"SWWAF_LIMIT_BAN_REPEAT_WINDOW",
"24h",
),
MaxBanDuration: env.durationNotOff(
"SWWAF_MAX_BAN_DURATION",
"7d",
),
AttackBanDuration: env.durationNotOff(
"SWWAF_ATTACK_BAN_DURATION",
"7d",
),
MaxBans: env.numberNotOff("SWWAF_MAX_BANS", "5000"),
BanScopeV4Prefix: env.v4Prefix(
"SWWAF_BAN_SCOPE_V4_PREFIX",
"32",
),
StateDir: env.absolutePath(
"SWWAF_STATE_DIR",
"/var/lib/smallwebwaf",
),
StateWriteDelay: env.durationNotOff(
"SWWAF_STATE_WRITE_DELAY",
"10s",
),
StateCounterInterval: env.durationNotOff(
"SWWAF_STATE_COUNTER_INTERVAL",
"15m",
),
LogRequestHeaders: env.headerNames(
"SWWAF_LOG_REQUEST_HEADERS",
"accept,accept-language,accept-encoding,content-type,origin,range",
),
LogLevel: env.logLevel("SWWAF_LOG_LEVEL", "info"),
AdminToken: env.token("SWWAF_ADMIN_TOKEN"),
MetricsToken: env.token("SWWAF_METRICS_TOKEN"),
MetricsTopN: env.numberNotOff("SWWAF_METRICS_TOP_N", "50"),
RulesDir: env.value(
"SWWAF_RULES_DIR",
"/etc/smallwebwaf/rules.d",
),
RulesEnabled: env.boolean("SWWAF_RULES_ENABLED", "true"),
WAFMode: env.wafMode("SWWAF_WAF_MODE", WAFModeBlock),
WAFParanoiaLevel: env.paranoiaLevel(
"SWWAF_WAF_PARANOIA_LEVEL",
"1",
),
WAFAnomalyThreshold: env.numberOrOff(
"SWWAF_WAF_ANOMALY_THRESHOLD",
"5",
),
WAFDisabledRules: env.ruleIDs("SWWAF_WAF_DISABLED_RULES", WAFDisabledRules: env.ruleIDs("SWWAF_WAF_DISABLED_RULES",
"920340,920420,920440,920640,930130,930140"), "920340,920420,920440,920640,930130,930140"),
WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""), WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off), WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"), CoreRuleSetExclusionsFile: env.filePath(
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"), "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"),
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"), TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
LogRemoteTLSCAs: env.certificates( ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
"SWWAF_LOG_REMOTE_TLS_CA_FILE", LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
), LogRemoteTLSCAs: env.certificates("SWWAF_LOG_REMOTE_TLS_CA_FILE"),
LogRemoteBuffer: env.numberNotOff( LogRemoteBuffer: env.numberNotOff("SWWAF_LOG_REMOTE_BUFFER", "10000"),
"SWWAF_LOG_REMOTE_BUFFER", LogRemoteFacility: env.facility("SWWAF_LOG_REMOTE_FACILITY", "local0"),
"10000", AlertWebhookURL: env.webhookURL("SWWAF_ALERT_WEBHOOK_URL"),
), AlertWebhookHeaders: env.webhookHeaders("SWWAF_ALERT_WEBHOOK_HEADERS"),
LogRemoteFacility: env.facility( AlertSlackWebhookURL: env.webhookURL("SWWAF_ALERT_SLACK_WEBHOOK_URL"),
"SWWAF_LOG_REMOTE_FACILITY", AlertNtfyURL: env.webhookURL("SWWAF_ALERT_NTFY_URL"),
"local0", AlertNtfyToken: env.secret("SWWAF_ALERT_NTFY_TOKEN"),
),
AlertWebhookURL: env.webhookURL("SWWAF_ALERT_WEBHOOK_URL"),
AlertWebhookHeaders: env.webhookHeaders(
"SWWAF_ALERT_WEBHOOK_HEADERS",
),
AlertSlackWebhookURL: env.webhookURL(
"SWWAF_ALERT_SLACK_WEBHOOK_URL",
),
AlertNtfyURL: env.webhookURL("SWWAF_ALERT_NTFY_URL"),
AlertNtfyToken: env.secret("SWWAF_ALERT_NTFY_TOKEN"),
AlertEvents: env.alertEvents("SWWAF_ALERT_EVENTS", AlertEvents: env.alertEvents("SWWAF_ALERT_EVENTS",
strings.Join(alerts.Events(), ",")), strings.Join(alerts.Events(), ",")),
AlertCooldown: env.duration("SWWAF_ALERT_COOLDOWN", "15m"), AlertCooldown: env.duration("SWWAF_ALERT_COOLDOWN", "15m"),
AlertMaxPerHour: env.numberOrOff( AlertMaxPerHour: env.numberOrOff("SWWAF_ALERT_MAX_PER_HOUR", "60"),
"SWWAF_ALERT_MAX_PER_HOUR", AnomalyClient: env.thresholds("SWWAF_ANOMALY_CLIENT_"),
"60", AnomalyNet: env.thresholds("SWWAF_ANOMALY_NET_"),
), AnomalyASN: env.thresholds("SWWAF_ANOMALY_ASN_"),
AnomalyClient: env.thresholds("SWWAF_ANOMALY_CLIENT_"), AnomalyTotal: env.thresholds("SWWAF_ANOMALY_TOTAL_"),
AnomalyNet: env.thresholds("SWWAF_ANOMALY_NET_"), AnomalyWatch: env.thresholds("SWWAF_WATCH_"),
AnomalyASN: env.thresholds("SWWAF_ANOMALY_ASN_"), AnomalyNetV4Prefix: env.v4Prefix("SWWAF_ANOMALY_NET_V4_PREFIX", "24"),
AnomalyTotal: env.thresholds("SWWAF_ANOMALY_TOTAL_"), AnomalyNetV6Prefix: env.v6Prefix("SWWAF_ANOMALY_NET_V6_PREFIX", "48"),
AnomalyWatch: env.thresholds("SWWAF_WATCH_"), WatchNets: env.namedNetblocks("SWWAF_WATCH_NETS"),
AnomalyNetV4Prefix: env.v4Prefix(
"SWWAF_ANOMALY_NET_V4_PREFIX",
"24",
),
AnomalyNetV6Prefix: env.v6Prefix(
"SWWAF_ANOMALY_NET_V6_PREFIX",
"48",
),
WatchNets: env.namedNetblocks("SWWAF_WATCH_NETS"),
} }
cfg.LogRemoteAppName = env.appName("SWWAF_LOG_REMOTE_APP_NAME", cfg.LogRemoteAppName = env.appName("SWWAF_LOG_REMOTE_APP_NAME",
@@ -723,20 +589,14 @@ func Command(lookupEnv func(string) (string, bool)) (string, error) {
case "servewaf": case "servewaf":
return command, nil return command, nil
default: default:
return "", fmt.Errorf( return "", fmt.Errorf("SMALLWEBWAF_COMMAND: %q %w", command, errNotCommand)
"SMALLWEBWAF_COMMAND: %q %w",
command,
errNotCommand,
)
} }
} }
// ServeWAFListenAddr reads only SWWAF_LISTEN_ADDR, which may be given as a // ServeWAFListenAddr reads only SWWAF_LISTEN_ADDR, which may be given as a
// file, as FromEnvironment does. The health check in servewaf's container needs // file, as FromEnvironment does. The health check in servewaf's container needs
// no other setting, so it reads no other, nor a file that another names. // no other setting, so it reads no other, nor a file that another names.
func ServeWAFListenAddr( func ServeWAFListenAddr(lookupEnv func(string) (string, bool)) (string, error) {
lookupEnv func(string) (string, bool),
) (string, error) {
env := &environment{lookupEnv: lookupEnv} env := &environment{lookupEnv: lookupEnv}
listenAddr := env.address("SWWAF_LISTEN_ADDR") listenAddr := env.address("SWWAF_LISTEN_ADDR")
@@ -798,19 +658,14 @@ func (e *environment) lookup(name string) (string, bool) {
} }
if set { if set {
e.check( e.check(name, fmt.Errorf("is set, and so is %s; %w", fileName, errSetTwice))
name,
fmt.Errorf("is set, and so is %s; %w", fileName, errSetTwice),
)
return value, set return value, set
} }
e.settings = append(e.settings, slog.String(fileName, path)) e.settings = append(e.settings, slog.String(fileName, path))
contents, err := os.ReadFile( contents, err := os.ReadFile(path) //nolint:gosec // a file the admin names
path,
) //nolint:gosec // a file the admin names
if err != nil { if err != nil {
e.check(fileName, fmt.Errorf("cannot be read: %w", err)) e.check(fileName, fmt.Errorf("cannot be read: %w", err))
@@ -1061,10 +916,7 @@ func (e *environment) zones(name string) []string {
logged[i] = MaskZoneKey(zone) logged[i] = MaskZoneKey(zone)
} }
e.settings = append( e.settings = append(e.settings, slog.String(name, strings.Join(logged, ",")))
e.settings,
slog.String(name, strings.Join(logged, ",")),
)
return zones return zones
} }
@@ -1116,15 +968,10 @@ func (e *environment) checkLookupDBPath(cfg *Config) {
switch { switch {
case cfg.LookupSource == fileSource && cfg.LookupDBPath == "": case cfg.LookupSource == fileSource && cfg.LookupDBPath == "":
e.check("SWWAF_LOOKUP_SOURCE", fmt.Errorf( e.check("SWWAF_LOOKUP_SOURCE", fmt.Errorf(
"is file while SWWAF_LOOKUP_DB_PATH is unset; %w", "is file while SWWAF_LOOKUP_DB_PATH is unset; %w", errNeedsDBPath))
errNeedsDBPath,
))
case cfg.LookupSource != fileSource && cfg.LookupDBPath != "": case cfg.LookupSource != fileSource && cfg.LookupDBPath != "":
e.check("SWWAF_LOOKUP_DB_PATH", fmt.Errorf( e.check("SWWAF_LOOKUP_DB_PATH", fmt.Errorf(
"is set while SWWAF_LOOKUP_SOURCE is %s; %w", "is set while SWWAF_LOOKUP_SOURCE is %s; %w", cfg.LookupSource, errDBPathUnused))
cfg.LookupSource,
errDBPathUnused,
))
} }
} }
@@ -1165,11 +1012,8 @@ func (e *environment) checkCountriesAndLookups(cfg *Config) {
{"SWWAF_ANOMALY_ASN_BYTES_PER_HOUR", cfg.AnomalyASN.BytesPerHour > 0}, {"SWWAF_ANOMALY_ASN_BYTES_PER_HOUR", cfg.AnomalyASN.BytesPerHour > 0},
} { } {
if setting.set { if setting.set {
e.check( e.check(setting.name, fmt.Errorf("is set while SWWAF_LOOKUP_SOURCE is off; %w",
setting.name, errNeedsLookups))
fmt.Errorf("is set while SWWAF_LOOKUP_SOURCE is off; %w",
errNeedsLookups),
)
} }
} }
} }
@@ -1192,21 +1036,14 @@ func (e *environment) checkCrowdSec(cfg *Config) {
switch { switch {
case decisionsURL != "" && cfg.CrowdSecKey == "": case decisionsURL != "" && cfg.CrowdSecKey == "":
e.check("SWWAF_CROWDSEC_LAPI_URL", fmt.Errorf( e.check("SWWAF_CROWDSEC_LAPI_URL", fmt.Errorf(
"is set while SWWAF_CROWDSEC_LAPI_KEY is unset; %w", "is set while SWWAF_CROWDSEC_LAPI_KEY is unset; %w", errNeedsLAPIKey))
errNeedsLAPIKey,
))
case decisionsURL == "" && cfg.CrowdSecKey != "": case decisionsURL == "" && cfg.CrowdSecKey != "":
e.check("SWWAF_CROWDSEC_LAPI_KEY", fmt.Errorf( e.check("SWWAF_CROWDSEC_LAPI_KEY", fmt.Errorf(
"is set while SWWAF_CROWDSEC_LAPI_URL is unset; %w", "is set while SWWAF_CROWDSEC_LAPI_URL is unset; %w", errLAPIKeyUnused))
errLAPIKeyUnused,
))
case decisionsURL != "" && (slices.Contains(cfg.BlocklistURLs, decisionsURL) || case decisionsURL != "" && (slices.Contains(cfg.BlocklistURLs, decisionsURL) ||
decisionsURL == cfg.ASNLimitPercentURL): decisionsURL == cfg.ASNLimitPercentURL):
e.check( e.check("SWWAF_CROWDSEC_LAPI_URL", fmt.Errorf("gives the decision list %q, which %w",
"SWWAF_CROWDSEC_LAPI_URL", decisionsURL, errAnotherList))
fmt.Errorf("gives the decision list %q, which %w",
decisionsURL, errAnotherList),
)
} }
} }
@@ -1221,9 +1058,7 @@ func (e *environment) headerNames(name, defaultValue string) []string {
// durationNotOff reads a setting that is a duration and, unlike a // durationNotOff reads a setting that is a duration and, unlike a
// timeout, cannot be off. // timeout, cannot be off.
func (e *environment) durationNotOff( func (e *environment) durationNotOff(name, defaultValue string) time.Duration {
name, defaultValue string,
) time.Duration {
duration, err := ParseDurationNotOff(e.value(name, defaultValue)) duration, err := ParseDurationNotOff(e.value(name, defaultValue))
e.check(name, err) e.check(name, err)
@@ -1359,6 +1194,16 @@ func (e *environment) logRemoteURL(name string) *url.URL {
return remote return remote
} }
// filePath reads a setting that is the path of a file, "" while it is
// unset. Its value names a file already, so, unlike the other settings,
// it has no _FILE form. Whoever uses the file reads it.
func (e *environment) filePath(name string) string {
path, _ := e.lookupEnv(name)
e.settings = append(e.settings, slog.String(name, path))
return path
}
// certificates reads a setting that is the path of a file of PEM // certificates reads a setting that is the path of a file of PEM
// certificates. Unset or empty, it is nil. Its value names a file // certificates. Unset or empty, it is nil. Its value names a file
// already, so, unlike the other settings, it has no _FILE form. // already, so, unlike the other settings, it has no _FILE form.
@@ -1403,10 +1248,7 @@ func (e *environment) instanceName() string {
value := e.value("SWWAF_INSTANCE_NAME", hostname) value := e.value("SWWAF_INSTANCE_NAME", hostname)
if !utf8.ValidString(value) { if !utf8.ValidString(value) {
e.check( e.check("SWWAF_INSTANCE_NAME", fmt.Errorf("%q %w", value, errNotUTF8))
"SWWAF_INSTANCE_NAME",
fmt.Errorf("%q %w", value, errNotUTF8),
)
} }
return value return value
@@ -1415,10 +1257,7 @@ func (e *environment) instanceName() string {
// appName reads the setting that is the APP-NAME of the records the log // appName reads the setting that is the APP-NAME of the records the log
// lines are sent in, by default the instance name. Its value is checked // lines are sent in, by default the instance name. Its value is checked
// when it is set, and, while lines are sent, when it is the instance name. // when it is set, and, while lines are sent, when it is the instance name.
func (e *environment) appName( func (e *environment) appName(name, instanceName string, sending bool) string {
name, instanceName string,
sending bool,
) string {
value, set := e.lookup(name) value, set := e.lookup(name)
if !set { if !set {
value = instanceName value = instanceName
@@ -1431,14 +1270,8 @@ func (e *environment) appName(
case set: case set:
e.check(name, fmt.Errorf("%q %w", value, errNotAppName)) e.check(name, fmt.Errorf("%q %w", value, errNotAppName))
case sending: case sending:
e.check( e.check(name, fmt.Errorf("is unset, and SWWAF_INSTANCE_NAME %q, its default, %w",
name, value, errNotAppName))
fmt.Errorf(
"is unset, and SWWAF_INSTANCE_NAME %q, its default, %w",
value,
errNotAppName,
),
)
} }
return value return value
@@ -1447,16 +1280,11 @@ func (e *environment) appName(
// checkInstanceNameForNtfy refuses an instance name that holds a control // checkInstanceNameForNtfy refuses an instance name that holds a control
// character while ntfySet, SWWAF_ALERT_NTFY_URL being set: ntfy is sent // character while ntfySet, SWWAF_ALERT_NTFY_URL being set: ntfy is sent
// the instance name in a header, which cannot hold one. // the instance name in a header, which cannot hold one.
func (e *environment) checkInstanceNameForNtfy( func (e *environment) checkInstanceNameForNtfy(instanceName string, ntfySet bool) {
instanceName string,
ntfySet bool,
) {
if ntfySet && strings.ContainsFunc(instanceName, unicode.IsControl) { if ntfySet && strings.ContainsFunc(instanceName, unicode.IsControl) {
e.check("SWWAF_INSTANCE_NAME", fmt.Errorf( e.check("SWWAF_INSTANCE_NAME", fmt.Errorf(
"%q %w, and is sent to ntfy in a header while SWWAF_ALERT_NTFY_URL is set", "%q %w, and is sent to ntfy in a header while SWWAF_ALERT_NTFY_URL is set",
instanceName, instanceName, errControlCharacter))
errControlCharacter,
))
} }
} }
@@ -1776,10 +1604,7 @@ func parseNamedNetblocks(value string) ([]anomaly.NamedNetblock, error) {
return nil, fmt.Errorf("%q %w", name, errListedTwice) return nil, fmt.Errorf("%q %w", name, errListedTwice)
} }
named = append( named = append(named, anomaly.NamedNetblock{Name: name, Netblock: netblock})
named,
anomaly.NamedNetblock{Name: name, Netblock: netblock},
)
} }
return named, nil return named, nil
@@ -2018,11 +1843,8 @@ func parseHeaderNames(value string) ([]string, error) {
header := strings.ToLower(item) header := strings.ToLower(item)
switch header { switch header {
case "host": case "host":
return nil, fmt.Errorf( return nil, fmt.Errorf("%q %w; the request's host is the field host",
"%q %w; the request's host is the field host", item, errHeaderTakenOut)
item,
errHeaderTakenOut,
)
case "transfer-encoding": case "transfer-encoding":
return nil, fmt.Errorf("%q %w", item, errHeaderTakenOut) return nil, fmt.Errorf("%q %w", item, errHeaderTakenOut)
} }
@@ -2059,11 +1881,9 @@ func parseUpstreamURL(value string) (*url.URL, error) {
} }
onlySchemeAndHost := (upstream.Scheme == "http" || upstream.Scheme == "https") && onlySchemeAndHost := (upstream.Scheme == "http" || upstream.Scheme == "https") &&
upstream.Hostname() != "" && upstream.User == nil && upstream.Hostname() != "" && upstream.User == nil && upstream.Opaque == "" &&
upstream.Opaque == "" &&
(upstream.Path == "" || upstream.Path == "/") && (upstream.Path == "" || upstream.Path == "/") &&
upstream.RawQuery == "" && upstream.RawQuery == "" && upstream.Fragment == ""
upstream.Fragment == ""
if !onlySchemeAndHost { if !onlySchemeAndHost {
return nil, fmt.Errorf("%q %w", value, errNotUpstreamURL) return nil, fmt.Errorf("%q %w", value, errNotUpstreamURL)
} }
@@ -2087,11 +1907,7 @@ func parseLogRemoteURL(value string) (*url.URL, error) {
return nil, fmt.Errorf("%q %w", value, errNotLogRemoteURL) return nil, fmt.Errorf("%q %w", value, errNotLogRemoteURL)
} }
schemes := []string{ schemes := []string{remotelog.SchemeUDP, remotelog.SchemeTCP, remotelog.SchemeTLS}
remotelog.SchemeUDP,
remotelog.SchemeTCP,
remotelog.SchemeTLS,
}
port, err := strconv.ParseUint(remote.Port(), 10, 16) port, err := strconv.ParseUint(remote.Port(), 10, 16)
onlySchemeHostAndPort := slices.Contains(schemes, remote.Scheme) && onlySchemeHostAndPort := slices.Contains(schemes, remote.Scheme) &&
@@ -2154,9 +1970,7 @@ func isHTTPURL(u *url.URL) bool {
return (u.Scheme == "http" || u.Scheme == "https") && u.Hostname() != "" && return (u.Scheme == "http" || u.Scheme == "https") && u.Hostname() != "" &&
(u.Port() == "" || (err == nil && port != 0)) && (u.Port() == "" || (err == nil && port != 0)) &&
u.User == nil && u.User == nil && u.Opaque == "" && u.Fragment == ""
u.Opaque == "" &&
u.Fragment == ""
} }
// isListURL reports whether value is a URL a list can be fetched from, as // isListURL reports whether value is a URL a list can be fetched from, as
@@ -2223,12 +2037,9 @@ func parseZones(value string) ([]string, error) {
for i, zone := range zones { for i, zone := range zones {
shown := MaskZoneKey(zone) shown := MaskZoneKey(zone)
listedBefore := slices.ContainsFunc( listedBefore := slices.ContainsFunc(zones[:i], func(earlier string) bool {
zones[:i], return strings.EqualFold(MaskZoneKey(earlier), shown)
func(earlier string) bool { })
return strings.EqualFold(MaskZoneKey(earlier), shown)
},
)
switch { switch {
case len(zone) > maxZoneLength: case len(zone) > maxZoneLength:
@@ -2315,13 +2126,8 @@ func parseWebhookHeaders(value string) (http.Header, string, error) {
name, headerValue, found := strings.Cut(item, ":") name, headerValue, found := strings.Cut(item, ":")
name = strings.TrimSpace(name) name = strings.TrimSpace(name)
if !found || !IsHeaderName(name) || if !found || !IsHeaderName(name) || strings.ContainsAny(headerValue, "\r\n\x00") {
strings.ContainsAny(headerValue, "\r\n\x00") { return nil, "", fmt.Errorf("item %d %w", i+1, errNotWebhookHeader)
return nil, "", fmt.Errorf(
"item %d %w",
i+1,
errNotWebhookHeader,
)
} }
headers.Add(name, strings.TrimSpace(headerValue)) headers.Add(name, strings.TrimSpace(headerValue))
+9 -5
View File
@@ -87,6 +87,7 @@ const (
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES" wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS" wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT" wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
exclusionsFile = "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"
trapPaths = "SWWAF_TRAP_PATHS" trapPaths = "SWWAF_TRAP_PATHS"
errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD" errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
logRemoteURL = "SWWAF_LOG_REMOTE_URL" logRemoteURL = "SWWAF_LOG_REMOTE_URL"
@@ -539,13 +540,14 @@ func TestCoreRuleSetSettings(t *testing.T) {
environment{ environment{
wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10", wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10",
wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/", wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/",
wafBodyLimit: "32K", wafBodyLimit: "32K", exclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
}, },
config.Config{ config.Config{
WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10, WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10,
WAFDisabledRules: []int{942100, 920350}, WAFDisabledRules: []int{942100, 920350},
WAFExemptPaths: []string{"/api/", "/static/"}, WAFExemptPaths: []string{"/api/", "/static/"},
WAFBodyLimit: 32 << 10, WAFBodyLimit: 32 << 10,
CoreRuleSetExclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
}, },
}, },
{ {
@@ -565,7 +567,8 @@ func TestCoreRuleSetSettings(t *testing.T) {
WAFMode: cfg.WAFMode, WAFParanoiaLevel: cfg.WAFParanoiaLevel, WAFMode: cfg.WAFMode, WAFParanoiaLevel: cfg.WAFParanoiaLevel,
WAFAnomalyThreshold: cfg.WAFAnomalyThreshold, WAFAnomalyThreshold: cfg.WAFAnomalyThreshold,
WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths, WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths,
WAFBodyLimit: cfg.WAFBodyLimit, WAFBodyLimit: cfg.WAFBodyLimit,
CoreRuleSetExclusionsFile: cfg.CoreRuleSetExclusionsFile,
} }
if !reflect.DeepEqual(got, tc.want) { if !reflect.DeepEqual(got, tc.want) {
t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want) t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want)
@@ -2327,6 +2330,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
wafDisabledRules: defaultWAFDisabledRules, wafDisabledRules: defaultWAFDisabledRules,
wafExemptPaths: "", wafExemptPaths: "",
wafBodyLimit: off, wafBodyLimit: off,
exclusionsFile: "",
trapPaths: "", trapPaths: "",
errorBurstThreshold: "30", errorBurstThreshold: "30",
logRemoteURL: "", logRemoteURL: "",
+6 -28
View File
@@ -16,11 +16,6 @@ import (
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
) )
// FIXME: you can't call things just "File" - that's super vague.
// FIXME: "lookup database" - in memory on disk? you must not be so
// ambiguous when describing things.
// quietTime is how long the lookup database must go without a change // quietTime is how long the lookup database must go without a change
// before it is read again, so that a file still being copied in is read // before it is read again, so that a file still being copied in is read
// only once whole. // only once whole.
@@ -138,11 +133,8 @@ func (f *File) Watch(ctx context.Context) {
} }
if err != nil { if err != nil {
f.params.ProcessLog.Error( f.params.ProcessLog.Error("cannot watch the lookup database for replacements",
"cannot watch the lookup database for replacements", "error", err.Error())
"error",
err.Error(),
)
return return
} }
@@ -202,10 +194,7 @@ func (f *File) readAgain() {
f.params.Alerts.Raise(alerts.Alert{ f.params.Alerts.Raise(alerts.Alert{
Event: alerts.EventFileError, Event: alerts.EventFileError,
Reason: kept, Reason: kept,
Detail: map[string]any{ Detail: map[string]any{"file": f.params.Path, "error": err.Error()},
"file": f.params.Path,
"error": err.Error(),
},
}) })
f.params.ProcessLog.Error(kept, "error", err.Error()) f.params.ProcessLog.Error(kept, "error", err.Error())
@@ -223,11 +212,7 @@ func (f *File) use(reader *maxminddb.Reader) {
f.lastRead = f.params.Now() f.lastRead = f.params.Now()
f.mu.Unlock() f.mu.Unlock()
f.params.ProcessLog.Info( f.params.ProcessLog.Info("read the lookup database", "file", f.params.Path)
"read the lookup database",
"file",
f.params.Path,
)
} }
// read reads the lookup database at path. The whole file is read into // read reads the lookup database at path. The whole file is read into
@@ -236,19 +221,12 @@ func (f *File) use(reader *maxminddb.Reader) {
func read(path string) (*maxminddb.Reader, error) { func read(path string) (*maxminddb.Reader, error) {
data, err := os.ReadFile(path) //nolint:gosec // the file the admin names data, err := os.ReadFile(path) //nolint:gosec // the file the admin names
if err != nil { if err != nil {
return nil, fmt.Errorf( return nil, fmt.Errorf("SWWAF_LOOKUP_DB_PATH cannot be read: %w", err)
"SWWAF_LOOKUP_DB_PATH cannot be read: %w",
err,
)
} }
reader, err := maxminddb.OpenBytes(data) reader, err := maxminddb.OpenBytes(data)
if err != nil { if err != nil {
return nil, fmt.Errorf( return nil, fmt.Errorf("SWWAF_LOOKUP_DB_PATH %s is not a .mmdb file: %w", path, err)
"SWWAF_LOOKUP_DB_PATH %s is not a .mmdb file: %w",
path,
err,
)
} }
return reader, nil return reader, nil
+4 -38
View File
@@ -19,51 +19,29 @@ const (
// ActionForward is a request servewaf allowed, which rproxy passes to the // ActionForward is a request servewaf allowed, which rproxy passes to the
// app. // app.
ActionForward = "forward" ActionForward = "forward"
//FIXME: "too_large" doesn't tell a reader what that means.
//request_header_too_large does. you must not use ambiguous terms.
// ActionTooLarge is a request over the header limits. // ActionTooLarge is a request over the header limits.
ActionTooLarge = "too_large" ActionTooLarge = "too_large"
// ActionRateLimited is a request refused because it took its client // ActionRateLimited is a request refused because it took its client
// over a rate limit, which bans the client. // over a rate limit, which bans the client.
//FIXME: better would be "client_is_rate_limited"
ActionRateLimited = "rate_limited" ActionRateLimited = "rate_limited"
// ActionBanned is a request refused because a ban covers its client, // ActionBanned is a request refused because a ban covers its client,
// or because it matched a ban rule, asked for a trap path or the // or because it matched a ban rule, asked for a trap path or the
// CrowdSec decision list lists its client, each of which bans the // CrowdSec decision list lists its client, each of which bans the
// client. // client.
// FIXME: if I saw this, i would assume they are permanently banned. is that
// true? if not, there should be TemporarilyBanned and
// PermanentlyBanned
ActionBanned = "banned" ActionBanned = "banned"
// ActionRuleBlocked is a request refused because it matched a block // ActionRuleBlocked is a request refused because it matched a block
// rule. // rule.
// FIXME: "blocked_by_block_rule"
ActionRuleBlocked = "rule_blocked" ActionRuleBlocked = "rule_blocked"
// ActionWAFBlocked is a request refused because the Core Rule Set // ActionWAFBlocked is a request refused because the Core Rule Set
// scored it at or over SWWAF_WAF_ANOMALY_THRESHOLD. // scored it at or over SWWAF_WAF_ANOMALY_THRESHOLD.
ActionWAFBlocked = "waf_blocked" ActionWAFBlocked = "waf_blocked"
// ActionDenied is a request refused because its client is in // ActionDenied is a request refused because its client is in
// SWWAF_DENY_NETS, in a blocklist while SWWAF_BLOCKLIST_ACTION is deny, // SWWAF_DENY_NETS, in a blocklist while SWWAF_BLOCKLIST_ACTION is deny,
// or listed by a DNSBL zone, or scored a hit by AbuseIPDB, while // or listed by a DNSBL zone, or scored a hit by AbuseIPDB, while
// SWWAF_REPUTATION_ACTION is deny. // SWWAF_REPUTATION_ACTION is deny.
ActionDenied = "denied" ActionDenied = "denied"
// ActionCountryDenied is a request refused for its client's country. // ActionCountryDenied is a request refused for its client's country.
ActionCountryDenied = "country_denied" ActionCountryDenied = "country_denied"
// FIXME: you have country_denied and denied. why isn't it
// client_denied? ALWAYS BE CONSISTENT AND UNAMBIGUOUS
// ActionAdmin is a request servewaf answered at one of its own endpoints, // ActionAdmin is a request servewaf answered at one of its own endpoints,
// or a question about a path under /_smallwebwaf/. // or a question about a path under /_smallwebwaf/.
ActionAdmin = "admin" ActionAdmin = "admin"
@@ -267,11 +245,7 @@ func FormatTime(t time.Time) string {
// Milliseconds is d in milliseconds, to the microsecond. // Milliseconds is d in milliseconds, to the microsecond.
func Milliseconds(d time.Duration) float64 { func Milliseconds(d time.Duration) float64 {
return float64( return float64(d.Microseconds()) / float64(time.Millisecond/time.Microsecond)
d.Microseconds(),
) / float64(
time.Millisecond/time.Microsecond,
)
} }
// NewProcessLogger returns the logger for the process's own messages: // NewProcessLogger returns the logger for the process's own messages:
@@ -279,25 +253,17 @@ func Milliseconds(d time.Duration) float64 {
// as a request line's, and instanceName, SWWAF_INSTANCE_NAME, as instance. // as a request line's, and instanceName, SWWAF_INSTANCE_NAME, as instance.
// It writes only the messages at level, SWWAF_LOG_LEVEL, or more severe; // It writes only the messages at level, SWWAF_LOG_LEVEL, or more severe;
// the request lines Write writes are never held back. // the request lines Write writes are never held back.
func NewProcessLogger( func NewProcessLogger(w io.Writer, instanceName string, level slog.Level) *slog.Logger {
w io.Writer,
instanceName string,
level slog.Level,
) *slog.Logger {
handler := slog.NewJSONHandler(w, &slog.HandlerOptions{ handler := slog.NewJSONHandler(w, &slog.HandlerOptions{
Level: level, Level: level,
ReplaceAttr: func(groups []string, attr slog.Attr) slog.Attr { ReplaceAttr: func(groups []string, attr slog.Attr) slog.Attr {
if attr.Key == slog.TimeKey && len(groups) == 0 { if attr.Key == slog.TimeKey && len(groups) == 0 {
return slog.String( return slog.String(slog.TimeKey, FormatTime(attr.Value.Time()))
slog.TimeKey,
FormatTime(attr.Value.Time()),
)
} }
return attr return attr
}, },
}) })
return slog.New(handler). return slog.New(handler).With("type", "process", "instance", instanceName)
With("type", "process", "instance", instanceName)
} }
+24 -14
View File
@@ -4,6 +4,7 @@
package servewaf package servewaf
import ( import (
"fmt"
"io" "io"
"log/slog" "log/slog"
"net/http" "net/http"
@@ -60,6 +61,9 @@ type Params struct {
// LookupFile is the lookup database they are looked up in while // LookupFile is the lookup database they are looked up in while
// SWWAF_LOOKUP_SOURCE is file, and nil otherwise. // SWWAF_LOOKUP_SOURCE is file, and nil otherwise.
LookupFile *lookup.File LookupFile *lookup.File
// CoreRuleSet is what NewCoreRuleSet returns: what inspects requests
// while SWWAF_WAF_MODE is not off, and nil while it is.
CoreRuleSet *waf.CoreRuleSet
// Now tells the time by which requests are counted for the rate // Now tells the time by which requests are counted for the rate
// limits, bans are made and run out, and GeoJS's answers are kept, // limits, bans are made and run out, and GeoJS's answers are kept,
// normally time.Now in UTC, the time the state files give. // normally time.Now in UTC, the time the state files give.
@@ -144,7 +148,7 @@ func newHandler(params Params) *handler {
dnsbl: dnsbl, dnsbl: dnsbl,
abuseIPDB: abuseIPDB, abuseIPDB: abuseIPDB,
rules: params.Rules, rules: params.Rules,
coreRuleSet: newCoreRuleSet(params.Config), coreRuleSet: params.CoreRuleSet,
alerts: params.Alerts, alerts: params.Alerts,
} }
h.geojs = lookup.New(lookup.Params{ h.geojs = lookup.New(lookup.Params{
@@ -202,29 +206,35 @@ func newReputation(
return lists, dnsbl, abuseIPDB return lists, dnsbl, abuseIPDB
} }
// newCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL, // NewCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL,
// without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies // without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies
// up to SWWAF_WAF_BODY_LIMIT, or nil while SWWAF_WAF_MODE is off. It reads at // up to SWWAF_WAF_BODY_LIMIT, with the exclusions in
// most QuestionBodyMaxBytes less one byte: a question's body of // SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, or nil while SWWAF_WAF_MODE is off. It
// reads at most QuestionBodyMaxBytes less one byte: a question's body of
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a // QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
// JSON or XML body over its limit. // JSON or XML body over its limit. An exclusions file that cannot be read, does
func newCoreRuleSet(cfg *config.Config) *waf.CoreRuleSet { // not parse or names an id of smallwebwaf's own rules is an error in every
if cfg.WAFMode == config.WAFModeOff { // mode. While the mode is off, a set file is loaded with the Core Rule Set only
return nil // to check it.
func NewCoreRuleSet(cfg *config.Config) (*waf.CoreRuleSet, error) {
if cfg.WAFMode == config.WAFModeOff && cfg.CoreRuleSetExclusionsFile == "" {
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
} }
coreRuleSet, err := waf.New(waf.Params{ coreRuleSet, err := waf.New(waf.Params{
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules, ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1), BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
}) })
if err != nil { if err != nil {
// The Core Rule Set is built in, and the settings cannot break it: return nil, fmt.Errorf("SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: %w", err)
// the paranoia level is from 1 to 4, and the id of no rule switches
// nothing off.
panic(err)
} }
return coreRuleSet if cfg.WAFMode == config.WAFModeOff {
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
}
return coreRuleSet, nil
} }
// handler is servewaf's handler. It holds what every request shares; what // handler is servewaf's handler. It holds what every request shares; what
+6
View File
@@ -463,6 +463,11 @@ func newServeWAFWith(
} }
} }
coreRuleSet, err := servewaf.NewCoreRuleSet(cfg)
if err != nil {
t.Fatalf("Core Rule Set: %v", err)
}
server := build(servewaf.Params{ server := build(servewaf.Params{
Config: cfg, Config: cfg,
RequestLog: out, RequestLog: out,
@@ -470,6 +475,7 @@ func newServeWAFWith(
GeoJSURL: geojsURL, GeoJSURL: geojsURL,
AbuseIPDBURL: abuseIPDBURL, AbuseIPDBURL: abuseIPDBURL,
LookupFile: lookupFile, LookupFile: lookupFile,
CoreRuleSet: coreRuleSet,
Now: now, Now: now,
Rules: ruleFiles, Rules: ruleFiles,
Alerts: alertQueue, Alerts: alertQueue,
+12 -3
View File
@@ -29,6 +29,7 @@ import (
"sneak.berlin/go/smallwebwaf/internal/rules" "sneak.berlin/go/smallwebwaf/internal/rules"
"sneak.berlin/go/smallwebwaf/internal/servewaf" "sneak.berlin/go/smallwebwaf/internal/servewaf"
"sneak.berlin/go/smallwebwaf/internal/state" "sneak.berlin/go/smallwebwaf/internal/state"
"sneak.berlin/go/smallwebwaf/internal/waf"
) )
// shutdownTimeout is how long requests in progress may take to finish // shutdownTimeout is how long requests in progress may take to finish
@@ -150,7 +151,14 @@ func RunServeWAF(ctx context.Context, params Params) int {
return 1 return 1
} }
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue) crs, err := servewaf.NewCoreRuleSet(cfg)
if err != nil {
processLog.Error("cannot use the Core Rule Set", "error", err.Error())
return 1
}
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue, crs)
if err != nil { if err != nil {
processLog.Error("cannot use the lookup database", "error", err.Error()) processLog.Error("cannot use the lookup database", "error", err.Error())
@@ -170,8 +178,7 @@ func RunServeWAF(ctx context.Context, params Params) int {
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr) listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
if err != nil { if err != nil {
processLog.Error("cannot listen on SWWAF_LISTEN_ADDR", processLog.Error("cannot listen on SWWAF_LISTEN_ADDR", "error", err.Error())
"error", err.Error())
return 1 return 1
} }
@@ -190,6 +197,7 @@ func RunServeWAF(ctx context.Context, params Params) int {
func newServer( func newServer(
cfg *config.Config, stdout io.Writer, processLog *slog.Logger, cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue, now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
crs *waf.CoreRuleSet,
) (*servewaf.Server, error) { ) (*servewaf.Server, error) {
var lookupFile *lookup.File var lookupFile *lookup.File
@@ -214,6 +222,7 @@ func newServer(
GeoJSURL: lookup.URL, GeoJSURL: lookup.URL,
AbuseIPDBURL: reputation.AbuseIPDBURL, AbuseIPDBURL: reputation.AbuseIPDBURL,
LookupFile: lookupFile, LookupFile: lookupFile,
CoreRuleSet: crs,
Now: now, Now: now,
Rules: ruleFiles, Rules: ruleFiles,
Alerts: alertQueue, Alerts: alertQueue,
+51
View File
@@ -19,6 +19,7 @@ import (
"testing" "testing"
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest" "sneak.berlin/go/smallwebwaf/internal/lookup/lookuptest"
"sneak.berlin/go/smallwebwaf/internal/servewaf" "sneak.berlin/go/smallwebwaf/internal/servewaf"
"sneak.berlin/go/smallwebwaf/internal/smallwebwaf" "sneak.berlin/go/smallwebwaf/internal/smallwebwaf"
@@ -551,6 +552,56 @@ func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
": no such file or directory") ": no such file or directory")
} }
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(t *testing.T) {
t.Parallel()
path := filepath.Join(t.TempDir(), "crs-exclusions.conf")
out := &output{}
status := run(t.Context(), map[string]string{
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
"SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
}, out)
if status != 1 {
t.Fatalf("exit status %d, want 1", status)
}
line := out.line(t, "msg", "cannot use the Core Rule Set")
want := "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: read the exclusions file: open " +
path + ": no such file or directory"
if line["error"] != want {
t.Errorf("start refused with %v, want the error %q", line, want)
}
}
func TestCoreRuleSetExclusionsFileThatCannotBeUsedStopsTheStartWhileTheCoreRuleSetIsOff(
t *testing.T,
) {
t.Parallel()
broken := filepath.Join(t.TempDir(), "broken.conf")
err := os.WriteFile(broken, []byte("SecNoSuchDirective On\n"), 0o600)
if err != nil {
t.Fatal(err)
}
for _, path := range []string{filepath.Join(t.TempDir(), "missing.conf"), broken} {
out := &output{}
status := run(t.Context(), map[string]string{
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
"SWWAF_WAF_MODE": config.WAFModeOff, "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
}, out)
if status != 1 {
t.Fatalf("%s: exit status %d, want 1", path, status)
}
out.line(t, "msg", "cannot use the Core Rule Set")
}
}
func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) { func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
t.Parallel() t.Parallel()
+236
View File
@@ -0,0 +1,236 @@
# The eighth change: on gitea's own paths, the fields in which gitea sends a
# search, a file name or a branch name skip the rules that refused ordinary
# gitea requests there. Each exclusion runs with the request, before the rules
# it switches off, those of phase 1 included, so this file is loaded before the
# Core Rule Set's rules. Each comment names a request the rule refused.
# gitea's searches send the text searched for in q: searches of code, of issues
# and pull requests, of commits, and of repositories, branches, tags, packages
# and users. q skips these rules there:
# 930110: ../
# 930120: .gitignore
# 931100: http://127.0.0.1:8080
# 932120: Invoke-WebRequest
# 932130: $(pwd)
# 932140: if (a == b) {
# 932160: /bin/sh
# 932235: `make check`
# 932250: ls -la
# 932260: python3
# 932380: `whoami`
# 933100: <?php echo
# 933130: $_GET[
# 933135: ${VAR}
# 934100: process.env
# 934110: http://127.0.0.1:8080
# 934130: __proto__
# 941100: <script>
# 941110: <script>
# 941160: <img src=
# 941180: document.cookie
# 941390: fetch(
# 942100: SELECT * FROM users WHERE
# 942140: information_schema
# 942190: union select
# 942230: if (a == b) {
# 944100: java.lang.Runtime
# 944110: java.lang.Runtime
# 944130: java.lang.Runtime
SecRule REQUEST_FILENAME "@rx ^/(?:explore/(?:repos|code)|issues(?:/search)?|pulls|\
user/search_candidates|[^/]+/-/(?:code|packages)|\
[^/]+/[^/]+/(?:search|issues(?:/search)?|pulls|branches|tags|\
commits/(?:branch|tag|commit)/.+/search)|\
api/v1/repos/(?:search|issues/search|[^/]+/[^/]+/issues))$" \
"id:900600,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930110;ARGS:q,\
ctl:ruleRemoveTargetById=930120;ARGS:q,\
ctl:ruleRemoveTargetById=931100;ARGS:q,\
ctl:ruleRemoveTargetById=932120;ARGS:q,\
ctl:ruleRemoveTargetById=932130;ARGS:q,\
ctl:ruleRemoveTargetById=932140;ARGS:q,\
ctl:ruleRemoveTargetById=932160;ARGS:q,\
ctl:ruleRemoveTargetById=932235;ARGS:q,\
ctl:ruleRemoveTargetById=932250;ARGS:q,\
ctl:ruleRemoveTargetById=932260;ARGS:q,\
ctl:ruleRemoveTargetById=932380;ARGS:q,\
ctl:ruleRemoveTargetById=933100;ARGS:q,\
ctl:ruleRemoveTargetById=933130;ARGS:q,\
ctl:ruleRemoveTargetById=933135;ARGS:q,\
ctl:ruleRemoveTargetById=934100;ARGS:q,\
ctl:ruleRemoveTargetById=934110;ARGS:q,\
ctl:ruleRemoveTargetById=934130;ARGS:q,\
ctl:ruleRemoveTargetById=941100;ARGS:q,\
ctl:ruleRemoveTargetById=941110;ARGS:q,\
ctl:ruleRemoveTargetById=941160;ARGS:q,\
ctl:ruleRemoveTargetById=941180;ARGS:q,\
ctl:ruleRemoveTargetById=941390;ARGS:q,\
ctl:ruleRemoveTargetById=942100;ARGS:q,\
ctl:ruleRemoveTargetById=942140;ARGS:q,\
ctl:ruleRemoveTargetById=942190;ARGS:q,\
ctl:ruleRemoveTargetById=942230;ARGS:q,\
ctl:ruleRemoveTargetById=944100;ARGS:q,\
ctl:ruleRemoveTargetById=944110;ARGS:q,\
ctl:ruleRemoveTargetById=944130;ARGS:q"
# A page opened from the results of one of those searches has the search's
# address in Referer, which skips these rules then:
# 941110: <script>
# 941120: onClick={
# 941160: <img src=
# 942100: LIKE '%foo%'
# 944100: java.lang.Runtime
SecRule REQUEST_HEADERS:Referer "@rx ^https?://[^/]+/(?:explore/(?:repos|code)|\
issues(?:/search)?|pulls|user/search_candidates|[^/]+/-/(?:code|packages)|\
[^/]+/[^/]+/(?:search|issues(?:/search)?|pulls|branches|tags|\
commits/(?:branch|tag|commit)/.+/search)|\
api/v1/repos/(?:search|issues/search|[^/]+/[^/]+/issues))\?" \
"id:900601,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=941110;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=941120;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=941160;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=942100;REQUEST_HEADERS:Referer,\
ctl:ruleRemoveTargetById=944100;REQUEST_HEADERS:Referer"
# A file's page, raw file and download, its blame, history, edit and delete
# pages, and the API's call for its contents name the file in the path, which
# skips these rules there:
# 920500: notes.txt~
# 934100: process.env.d.ts
# 941130: docs/a b.xhtml
# 944130: java.lang.String.md
SecRule REQUEST_FILENAME \
"@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
"id:900610,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=920500;REQUEST_FILENAME,\
ctl:ruleRemoveTargetById=934100;REQUEST_FILENAME,\
ctl:ruleRemoveTargetById=941130;REQUEST_FILENAME,\
ctl:ruleRemoveTargetById=944130;REQUEST_FILENAME"
# The requests below name a file in a query parameter. These rules refused
# them for the file named:
# 930120: .gitignore
# 932120: install-script.sh
# 932130: [draft].md
# 932160: bin/docker-entrypoint
# 932260: docker-compose.yml
# 934100: process.env.d.ts
# 934130: __proto__.json
# 941130: index.xhtml
# 941180: document.write.js
# 942140: information_schema.sql
# 944110: java/lang/Runtime.java
# 944130: java.lang.String.md
# Each request skips the rules that refused it. The fourth change already has
# path and redirect_to skip 930120, 932160 and 932260 on every path.
# A file's commit graph, in file.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/graph$" \
"id:900620,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:file,\
ctl:ruleRemoveTargetById=932120;ARGS:file,\
ctl:ruleRemoveTargetById=932160;ARGS:file,\
ctl:ruleRemoveTargetById=932260;ARGS:file,\
ctl:ruleRemoveTargetById=934100;ARGS:file,\
ctl:ruleRemoveTargetById=934130;ARGS:file,\
ctl:ruleRemoveTargetById=941130;ARGS:file,\
ctl:ruleRemoveTargetById=941180;ARGS:file,\
ctl:ruleRemoveTargetById=942140;ARGS:file,\
ctl:ruleRemoveTargetById=944110;ARGS:file,\
ctl:ruleRemoveTargetById=944130;ARGS:file"
# The API's list of a file's commits, in path.
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/commits$" \
"id:900621,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=932120;ARGS:path,\
ctl:ruleRemoveTargetById=934100;ARGS:path,\
ctl:ruleRemoveTargetById=934130;ARGS:path,\
ctl:ruleRemoveTargetById=941130;ARGS:path,\
ctl:ruleRemoveTargetById=941180;ARGS:path,\
ctl:ruleRemoveTargetById=942140;ARGS:path,\
ctl:ruleRemoveTargetById=944110;ARGS:path,\
ctl:ruleRemoveTargetById=944130;ARGS:path"
# The API's call for the contents of several files, in body, as JSON.
SecRule REQUEST_FILENAME "@rx ^/api/v1/repos/[^/]+/[^/]+/file-contents$" \
"id:900622,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:body,\
ctl:ruleRemoveTargetById=932120;ARGS:body,\
ctl:ruleRemoveTargetById=932160;ARGS:body,\
ctl:ruleRemoveTargetById=934100;ARGS:body,\
ctl:ruleRemoveTargetById=934130;ARGS:body,\
ctl:ruleRemoveTargetById=941130;ARGS:body,\
ctl:ruleRemoveTargetById=941180;ARGS:body,\
ctl:ruleRemoveTargetById=942140;ARGS:body,\
ctl:ruleRemoveTargetById=944110;ARGS:body,\
ctl:ruleRemoveTargetById=944130;ARGS:body"
# An issue opened from a template, in template.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/issues/new$" \
"id:900623,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:template,\
ctl:ruleRemoveTargetById=932120;ARGS:template,\
ctl:ruleRemoveTargetById=932130;ARGS:template,\
ctl:ruleRemoveTargetById=932160;ARGS:template,\
ctl:ruleRemoveTargetById=932260;ARGS:template,\
ctl:ruleRemoveTargetById=934100;ARGS:template,\
ctl:ruleRemoveTargetById=934130;ARGS:template,\
ctl:ruleRemoveTargetById=941130;ARGS:template,\
ctl:ruleRemoveTargetById=941180;ARGS:template,\
ctl:ruleRemoveTargetById=942140;ARGS:template,\
ctl:ruleRemoveTargetById=944110;ARGS:template,\
ctl:ruleRemoveTargetById=944130;ARGS:template"
# The sign-in page on the way to a file's edit page, in redirect_to.
SecRule REQUEST_FILENAME "@rx ^/user/login$" \
"id:900624,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=932120;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=932130;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=934100;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=934130;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=941130;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=941180;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=942140;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=944110;ARGS:redirect_to,\
ctl:ruleRemoveTargetById=944130;ARGS:redirect_to"
# An attachment uploaded through the API to a release, an issue or a comment,
# in name.
SecRule REQUEST_FILENAME \
"@rx ^/api/v1/repos/[^/]+/[^/]+/(?:releases|issues|issues/comments)/[0-9]+/assets$" \
"id:900625,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=930120;ARGS:name,\
ctl:ruleRemoveTargetById=932120;ARGS:name,\
ctl:ruleRemoveTargetById=932260;ARGS:name,\
ctl:ruleRemoveTargetById=934100;ARGS:name,\
ctl:ruleRemoveTargetById=934130;ARGS:name,\
ctl:ruleRemoveTargetById=941130;ARGS:name,\
ctl:ruleRemoveTargetById=941180;ARGS:name,\
ctl:ruleRemoveTargetById=942140;ARGS:name,\
ctl:ruleRemoveTargetById=944130;ARGS:name"
# The requests below name a branch or tag in a query parameter, which skips
# the list of command names (932260) there. It refused names such as
# docker-build, python3-migration and feat/docker-support.
# Deleting or restoring a branch, in name.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/branches/(?:delete|restore)$" \
"id:900630,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:name"
# A release started from a tag, in tag.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/releases/new$" \
"id:900631,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:tag"
# A branch protection rule opened for editing, in rule_name.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/settings/branches/edit$" \
"id:900632,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:rule_name"
# A directory listing's last commits, in refSubUrl.
SecRule REQUEST_FILENAME "@rx ^/[^/]+/[^/]+/lastcommit/" \
"id:900633,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:refSubUrl"
# An artifact upload from actions/upload-artifact@v3 sends the header
# Content-Range, which the Core Rule Set refused (920450).
SecRule REQUEST_FILENAME \
"@rx ^/api/actions_pipeline/_apis/pipelines/workflows/[0-9]+/artifacts/[^/]+/upload$" \
"id:900640,phase:1,pass,nolog,\
ctl:ruleRemoveTargetById=920450;REQUEST_HEADERS_NAMES:Content-Range"
+367
View File
@@ -0,0 +1,367 @@
# Ordinary gitea 1.27 requests that the Core Rule Set must let through at the
# defaults. Each line is a path and query, and may end with one header after a
# space. The tests of internal/waf inspect each as a GET. The Core Rule Set reads
# the query and the headers the same way for any method.
# Searches with ordinary words and symbols.
/owner/repo/search?q=package.json
/owner/repo/search?q=.gitignore
/owner/repo/search?q=docker-compose.yml
/owner/repo/search?q=python3
/owner/repo/search?q=ssh%20key
/owner/repo/search?q=feat%2Fdocker-support
/owner/repo/search?q=ls%20-la
/owner/repo/search?q=sed%20-i
/owner/repo/search?q=%2Fbin%2Fsh
/owner/repo/search?q=%60make%20check%60
/owner/repo/search?q=%60whoami%60
/owner/repo/search?q=fetch%28
/owner/repo/search?q=%24%7BVAR%7D
/owner/repo/search?q=%24%28pwd%29
/owner/repo/search?q=process.env
/owner/repo/search?q=%3Cimg%20src%3D
/owner/repo/search?q=%3Cscript%3E
/owner/repo/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/search?q=union%20select
/owner/repo/search?q=http%3A%2F%2F127.0.0.1%3A8080
/owner/repo/search?q=..%2F
/owner/repo/search?q=%2Fetc%2Fpasswd
/owner/repo/search?q=eval%28
/owner/repo/search?q=chmod%20%2Bx
/owner/repo/search?q=echo%20%24PATH
/owner/repo/search?q=java.lang.Runtime
/owner/repo/search?q=document.cookie
/owner/repo/search?q=alert%281%29
/owner/repo/search?q=__proto__
/owner/repo/search?q=C%3A%5CWindows%5CSystem32
/owner/repo/search?q=Invoke-WebRequest
/owner/repo/search?q=if%20%28a%20%3D%3D%20b%29%20%7B
/owner/repo/search?q=sudo%20apt%20install
/owner/repo/search?q=~%2F.ssh%2Fconfig
/owner/repo/search?q=.htaccess
/owner/repo/search?q=%24_GET%5B
/owner/repo/search?q=%24_POST
/owner/repo/search?q=%24_SERVER%5B%27REQUEST_URI%27%5D
/owner/repo/search?q=%3C%3Fphp%20echo
/owner/repo/search?q=information_schema
/owner/repo/search?q=LIKE%20%27%25foo%25%27
/owner/repo/search?q=onClick%3D%7B
/explore/repos?q=SELECT%20%2A%20FROM%20users%20WHERE
/explore/repos?q=%3Cimg%20src%3D
/explore/code?q=SELECT%20%2A%20FROM%20users%20WHERE
/explore/code?q=%3Cimg%20src%3D
/owner/-/code?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/-/code?q=%3Cimg%20src%3D
/owner/repo/issues?type=all&state=open&q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/issues?type=all&state=open&q=%3Cimg%20src%3D
/owner/repo/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/issues/search?q=%3Cimg%20src%3D
/owner/repo/pulls?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/pulls?q=%3Cimg%20src%3D
/issues?type=your_repositories&q=SELECT%20%2A%20FROM%20users%20WHERE
/issues?type=your_repositories&q=%3Cimg%20src%3D
/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/issues/search?q=%3Cimg%20src%3D
/pulls?type=your_repositories&q=SELECT%20%2A%20FROM%20users%20WHERE
/pulls?type=your_repositories&q=%3Cimg%20src%3D
/owner/repo/commits/branch/main/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/commits/branch/main/search?q=%3Cimg%20src%3D
/owner/repo/branches?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/branches?q=%3Cimg%20src%3D
/owner/repo/tags?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/repo/tags?q=%3Cimg%20src%3D
/owner/-/packages?q=SELECT%20%2A%20FROM%20users%20WHERE
/owner/-/packages?q=%3Cimg%20src%3D
/user/search_candidates?q=SELECT%20%2A%20FROM%20users%20WHERE
/user/search_candidates?q=%3Cimg%20src%3D
/api/v1/repos/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/api/v1/repos/search?q=%3Cimg%20src%3D
/api/v1/repos/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
/api/v1/repos/issues/search?q=%3Cimg%20src%3D
/api/v1/repos/owner/repo/issues?q=SELECT%20%2A%20FROM%20users%20WHERE
/api/v1/repos/owner/repo/issues?q=%3Cimg%20src%3D
# Pages, raw files and edit pages of files, and the API's call for their
# contents.
/owner/repo/src/branch/main/package.json
/owner/repo/raw/branch/main/package.json
/owner/repo/_edit/main/package.json
/api/v1/repos/owner/repo/contents/package.json
/owner/repo/src/branch/main/.gitignore
/owner/repo/raw/branch/main/.gitignore
/owner/repo/_edit/main/.gitignore
/api/v1/repos/owner/repo/contents/.gitignore
/owner/repo/src/branch/main/docker-compose.yml
/owner/repo/raw/branch/main/docker-compose.yml
/owner/repo/_edit/main/docker-compose.yml
/api/v1/repos/owner/repo/contents/docker-compose.yml
/owner/repo/src/branch/main/Dockerfile
/owner/repo/raw/branch/main/Dockerfile
/owner/repo/_edit/main/Dockerfile
/api/v1/repos/owner/repo/contents/Dockerfile
/owner/repo/src/branch/main/.htaccess
/owner/repo/raw/branch/main/.htaccess
/owner/repo/_edit/main/.htaccess
/api/v1/repos/owner/repo/contents/.htaccess
/owner/repo/src/branch/main/rootfs/etc/passwd
/owner/repo/raw/branch/main/rootfs/etc/passwd
/owner/repo/_edit/main/rootfs/etc/passwd
/api/v1/repos/owner/repo/contents/rootfs/etc/passwd
/owner/repo/src/branch/main/bin/docker-entrypoint
/owner/repo/raw/branch/main/bin/docker-entrypoint
/owner/repo/_edit/main/bin/docker-entrypoint
/api/v1/repos/owner/repo/contents/bin/docker-entrypoint
/owner/repo/src/branch/main/scripts/python3-setup.sh
/owner/repo/raw/branch/main/scripts/python3-setup.sh
/owner/repo/_edit/main/scripts/python3-setup.sh
/api/v1/repos/owner/repo/contents/scripts/python3-setup.sh
/owner/repo/src/branch/main/src/java.base/share/classes/java/lang/Runtime.java
/owner/repo/raw/branch/main/src/java.base/share/classes/java/lang/Runtime.java
/owner/repo/_edit/main/src/java.base/share/classes/java/lang/Runtime.java
/api/v1/repos/owner/repo/contents/src/java.base/share/classes/java/lang/Runtime.java
/owner/repo/src/branch/main/java.lang.String.md
/owner/repo/raw/branch/main/java.lang.String.md
/owner/repo/_edit/main/java.lang.String.md
/api/v1/repos/owner/repo/contents/java.lang.String.md
/owner/repo/src/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
/owner/repo/raw/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
/owner/repo/_edit/main/META-INF/services/org.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/contents/META-INF/services/org.apache.commons.logging.LogFactory
/owner/repo/src/branch/main/process.env.d.ts
/owner/repo/raw/branch/main/process.env.d.ts
/owner/repo/_edit/main/process.env.d.ts
/api/v1/repos/owner/repo/contents/process.env.d.ts
/owner/repo/src/branch/main/document.write.js
/owner/repo/raw/branch/main/document.write.js
/owner/repo/_edit/main/document.write.js
/api/v1/repos/owner/repo/contents/document.write.js
/owner/repo/src/branch/main/test/fixtures/__proto__.json
/owner/repo/raw/branch/main/test/fixtures/__proto__.json
/owner/repo/_edit/main/test/fixtures/__proto__.json
/api/v1/repos/owner/repo/contents/test/fixtures/__proto__.json
/owner/repo/src/branch/main/install-script.sh
/owner/repo/raw/branch/main/install-script.sh
/owner/repo/_edit/main/install-script.sh
/api/v1/repos/owner/repo/contents/install-script.sh
/owner/repo/src/branch/main/src/backend/catalog/information_schema.sql
/owner/repo/raw/branch/main/src/backend/catalog/information_schema.sql
/owner/repo/_edit/main/src/backend/catalog/information_schema.sql
/api/v1/repos/owner/repo/contents/src/backend/catalog/information_schema.sql
/owner/repo/src/branch/main/docs/a%20b.xhtml
/owner/repo/raw/branch/main/docs/a%20b.xhtml
/owner/repo/_edit/main/docs/a%20b.xhtml
/api/v1/repos/owner/repo/contents/docs/a%20b.xhtml
/owner/repo/src/branch/main/%5Bdraft%5D.md
/owner/repo/raw/branch/main/%5Bdraft%5D.md
/owner/repo/_edit/main/%5Bdraft%5D.md
/api/v1/repos/owner/repo/contents/%5Bdraft%5D.md
/owner/repo/src/branch/main/notes.txt~
/owner/repo/raw/branch/main/notes.txt~
/owner/repo/_edit/main/notes.txt~
/api/v1/repos/owner/repo/contents/notes.txt~
# The download, blame, history and delete pages of the files above whose names
# the Core Rule Set checks in the path.
/owner/repo/media/branch/main/notes.txt~
/owner/repo/blame/branch/main/notes.txt~
/owner/repo/commits/branch/main/notes.txt~
/owner/repo/_delete/main/notes.txt~
/owner/repo/media/branch/main/process.env.d.ts
/owner/repo/blame/branch/main/process.env.d.ts
/owner/repo/commits/branch/main/process.env.d.ts
/owner/repo/_delete/main/process.env.d.ts
/owner/repo/media/branch/main/docs/a%20b.xhtml
/owner/repo/blame/branch/main/docs/a%20b.xhtml
/owner/repo/commits/branch/main/docs/a%20b.xhtml
/owner/repo/_delete/main/docs/a%20b.xhtml
/owner/repo/media/branch/main/java.lang.String.md
/owner/repo/blame/branch/main/java.lang.String.md
/owner/repo/commits/branch/main/java.lang.String.md
/owner/repo/_delete/main/java.lang.String.md
# File names in the query string: the commit graph of a file, the API's lists
# of its commits and of its contents, an issue opened from a template, and the
# sign-in page on the way to its edit page.
/owner/repo/graph?file=package.json
/api/v1/repos/owner/repo/commits?path=package.json
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22package.json%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fpackage.json
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fpackage.json
/owner/repo/graph?file=.gitignore
/api/v1/repos/owner/repo/commits?path=.gitignore
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.gitignore%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.gitignore
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.gitignore
/owner/repo/graph?file=docker-compose.yml
/api/v1/repos/owner/repo/commits?path=docker-compose.yml
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docker-compose.yml%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocker-compose.yml
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocker-compose.yml
/owner/repo/graph?file=.htaccess
/api/v1/repos/owner/repo/commits?path=.htaccess
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.htaccess%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.htaccess
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.htaccess
/owner/repo/graph?file=rootfs%2Fetc%2Fpasswd
/api/v1/repos/owner/repo/commits?path=rootfs%2Fetc%2Fpasswd
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22rootfs%2Fetc%2Fpasswd%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Frootfs%2Fetc%2Fpasswd
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Frootfs%2Fetc%2Fpasswd
/owner/repo/graph?file=bin%2Fdocker-entrypoint
/api/v1/repos/owner/repo/commits?path=bin%2Fdocker-entrypoint
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22bin%2Fdocker-entrypoint%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fbin%2Fdocker-entrypoint
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fbin%2Fdocker-entrypoint
/owner/repo/graph?file=scripts%2Fpython3-setup.sh
/api/v1/repos/owner/repo/commits?path=scripts%2Fpython3-setup.sh
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22scripts%2Fpython3-setup.sh%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fscripts%2Fpython3-setup.sh
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fscripts%2Fpython3-setup.sh
/owner/repo/graph?file=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/api/v1/repos/owner/repo/commits?path=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
/owner/repo/graph?file=java.lang.String.md
/api/v1/repos/owner/repo/commits?path=java.lang.String.md
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22java.lang.String.md%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fjava.lang.String.md
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fjava.lang.String.md
/owner/repo/graph?file=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/commits?path=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
/owner/repo/graph?file=process.env.d.ts
/api/v1/repos/owner/repo/commits?path=process.env.d.ts
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22process.env.d.ts%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fprocess.env.d.ts
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fprocess.env.d.ts
/owner/repo/graph?file=document.write.js
/api/v1/repos/owner/repo/commits?path=document.write.js
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22document.write.js%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocument.write.js
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocument.write.js
/owner/repo/graph?file=test%2Ffixtures%2F__proto__.json
/api/v1/repos/owner/repo/commits?path=test%2Ffixtures%2F__proto__.json
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22test%2Ffixtures%2F__proto__.json%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Ftest%2Ffixtures%2F__proto__.json
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Ftest%2Ffixtures%2F__proto__.json
/owner/repo/graph?file=install-script.sh
/api/v1/repos/owner/repo/commits?path=install-script.sh
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22install-script.sh%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Finstall-script.sh
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Finstall-script.sh
/owner/repo/graph?file=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
/api/v1/repos/owner/repo/commits?path=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fbackend%2Fcatalog%2Finformation_schema.sql%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
/owner/repo/graph?file=docs%2Fa%20b.xhtml
/api/v1/repos/owner/repo/commits?path=docs%2Fa%20b.xhtml
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docs%2Fa%20b.xhtml%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocs%2Fa%20b.xhtml
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocs%2Fa%20b.xhtml
/owner/repo/graph?file=%5Bdraft%5D.md
/api/v1/repos/owner/repo/commits?path=%5Bdraft%5D.md
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22%5Bdraft%5D.md%22%5D%7D
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F%5Bdraft%5D.md
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F%5Bdraft%5D.md
# File names of attachments uploaded through the API, to a release, an issue and
# a comment.
/api/v1/repos/owner/repo/releases/1/assets?name=package.json
/api/v1/repos/owner/repo/releases/1/assets?name=.gitignore
/api/v1/repos/owner/repo/releases/1/assets?name=docker-compose.yml
/api/v1/repos/owner/repo/releases/1/assets?name=.htaccess
/api/v1/repos/owner/repo/releases/1/assets?name=docker-entrypoint
/api/v1/repos/owner/repo/releases/1/assets?name=python3-setup.sh
/api/v1/repos/owner/repo/releases/1/assets?name=Runtime.java
/api/v1/repos/owner/repo/releases/1/assets?name=org.apache.commons.logging.LogFactory
/api/v1/repos/owner/repo/releases/1/assets?name=%5Bdraft%5D.md
/api/v1/repos/owner/repo/releases/1/assets?name=notes.txt~
/api/v1/repos/owner/repo/releases/1/assets?name=install-script.sh
/api/v1/repos/owner/repo/issues/1/assets?name=install-script.sh
/api/v1/repos/owner/repo/issues/comments/1/assets?name=install-script.sh
/api/v1/repos/owner/repo/releases/1/assets?name=process.env.d.ts
/api/v1/repos/owner/repo/issues/1/assets?name=process.env.d.ts
/api/v1/repos/owner/repo/issues/comments/1/assets?name=process.env.d.ts
/api/v1/repos/owner/repo/releases/1/assets?name=java.lang.String.md
/api/v1/repos/owner/repo/issues/1/assets?name=java.lang.String.md
/api/v1/repos/owner/repo/issues/comments/1/assets?name=java.lang.String.md
/api/v1/repos/owner/repo/releases/1/assets?name=information_schema.sql
/api/v1/repos/owner/repo/issues/1/assets?name=information_schema.sql
/api/v1/repos/owner/repo/issues/comments/1/assets?name=information_schema.sql
/api/v1/repos/owner/repo/releases/1/assets?name=document.write.js
/api/v1/repos/owner/repo/issues/1/assets?name=document.write.js
/api/v1/repos/owner/repo/issues/comments/1/assets?name=document.write.js
/api/v1/repos/owner/repo/releases/1/assets?name=__proto__.json
/api/v1/repos/owner/repo/issues/1/assets?name=__proto__.json
/api/v1/repos/owner/repo/issues/comments/1/assets?name=__proto__.json
/api/v1/repos/owner/repo/releases/1/assets?name=index.xhtml
/api/v1/repos/owner/repo/issues/1/assets?name=index.xhtml
/api/v1/repos/owner/repo/issues/comments/1/assets?name=index.xhtml
# An artifact upload from actions/upload-artifact@v3, which sends Content-Range.
/api/actions_pipeline/_apis/pipelines/workflows/1/artifacts/9b7c1f3a/upload?itemPath=artifact%2Freport.txt Content-Range: bytes 0-9/10
# Branch and tag names in the query string: deleting and restoring a branch,
# starting a release from a tag, a branch protection rule, a directory listing's
# last commits, and a container image push that reuses a layer.
/owner/repo/branches/delete?name=docker-build&page=1
/owner/repo/branches/restore?branch_id=1&name=docker-build&page=1
/owner/repo/releases/new?tag=docker-build
/owner/repo/settings/branches/edit?rule_name=docker-build
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fdocker-build
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fdocker-build
/owner/repo/branches/delete?name=feat%2Fdocker-support&page=1
/owner/repo/branches/restore?branch_id=1&name=feat%2Fdocker-support&page=1
/owner/repo/releases/new?tag=feat%2Fdocker-support
/owner/repo/settings/branches/edit?rule_name=feat%2Fdocker-support
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Ffeat%2Fdocker-support
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Ffeat%2Fdocker-support
/owner/repo/branches/delete?name=renovate%2Fdocker-build-push-action-6.x&page=1
/owner/repo/branches/restore?branch_id=1&name=renovate%2Fdocker-build-push-action-6.x&page=1
/owner/repo/releases/new?tag=renovate%2Fdocker-build-push-action-6.x
/owner/repo/settings/branches/edit?rule_name=renovate%2Fdocker-build-push-action-6.x
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Frenovate%2Fdocker-build-push-action-6.x
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Frenovate%2Fdocker-build-push-action-6.x
/owner/repo/branches/delete?name=python3-migration&page=1
/owner/repo/branches/restore?branch_id=1&name=python3-migration&page=1
/owner/repo/releases/new?tag=python3-migration
/owner/repo/settings/branches/edit?rule_name=python3-migration
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fpython3-migration
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fpython3-migration
/owner/repo/branches/delete?name=ansible-roles&page=1
/owner/repo/branches/restore?branch_id=1&name=ansible-roles&page=1
/owner/repo/releases/new?tag=ansible-roles
/owner/repo/settings/branches/edit?rule_name=ansible-roles
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fansible-roles
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fansible-roles
/owner/repo/branches/delete?name=whoami&page=1
/owner/repo/branches/restore?branch_id=1&name=whoami&page=1
/owner/repo/releases/new?tag=whoami
/owner/repo/settings/branches/edit?rule_name=whoami
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fwhoami
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fwhoami
# Pages a signed-in user opens.
/
/owner/repo
/owner/repo/issues
/owner/repo/pulls
/owner/repo/releases
/owner/repo/wiki
/owner/repo/activity
/owner/repo/branches
/owner/repo/tags
/owner/repo/settings
/owner/repo/actions
/owner/repo/compare/main...main
/owner/repo/commits/branch/main
/owner/repo/archive/main.zip
/notifications
/user/settings
/user/settings/applications
/explore/repos
/-/admin
+110 -15
View File
@@ -1,8 +1,9 @@
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the // Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
// method, the URL with its query and the headers of a request, and on its // method, the URL with its query and the headers of a request, and on its
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf // body while SWWAF_WAF_BODY_LIMIT is set, with the eight changes smallwebwaf
// makes to it, as "Attack detection" under "Settings" in SPEC.md describes // makes to it and the exclusions in SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, as
// them. It reads no response. // "Attack detection" under "Settings" in SPEC.md describes them. It reads no
// response.
// //
// smallwebwaf writes only to its state directory, so Coraza is built with // smallwebwaf writes only to its state directory, so Coraza is built with
// its no_fs_access tag, as the Dockerfile and script/build build it: of a // its no_fs_access tag, as the Dockerfile and script/build build it: of a
@@ -11,10 +12,14 @@
package waf package waf
import ( import (
_ "embed"
"errors"
"fmt" "fmt"
"io" "io"
"net/http" "net/http"
"net/netip" "net/netip"
"os"
"regexp"
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
@@ -26,12 +31,14 @@ import (
) )
// directives are the Core Rule Set as smallwebwaf runs it, with the // directives are the Core Rule Set as smallwebwaf runs it, with the
// paranoia level for %d, and bodyDirectives for %s while // paranoia level for %d, bodyDirectives for the first %s while
// SWWAF_WAF_BODY_LIMIT is set. Each rule smallwebwaf adds has an id from // SWWAF_WAF_BODY_LIMIT is set, the exclusions file for the second, and
// 900000 to 900999, the ids the Core Rule Set keeps for the rules that set // giteaExclusions for the third.
// it up, which SWWAF_WAF_DISABLED_RULES refuses, so that no setting // Each rule smallwebwaf adds has an id from 900000 to 900999, the ids the
// switches one off. Coraza joins a line ending in \ to the next, without // Core Rule Set keeps for the rules that set it up, which
// the spaces at the start of the next. // SWWAF_WAF_DISABLED_RULES and the exclusions file refuse, so that no
// setting switches one off. Coraza joins a line ending in \ to the next,
// without the spaces at the start of the next.
const directives = ` const directives = `
# The engine only detects. smallwebwaf compares the request's anomaly # The engine only detects. smallwebwaf compares the request's anomaly
# score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode # score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
@@ -69,6 +76,14 @@ SecArgumentsLimit 1000
SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\ SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'" severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
# The exclusions file comes before the rules, where the Core Rule Set puts
# its own file of exclusions that run before the rules, so that they run
# before the rules of phase 1 too.
%s
# The eighth change comes before the rules for the same reason.
%s
# The sixth: only the rules for requests are loaded, and no response is # The sixth: only the rules for requests are loaded, and no response is
# inspected. # inspected.
Include @owasp_crs/REQUEST-*.conf Include @owasp_crs/REQUEST-*.conf
@@ -96,10 +111,19 @@ SecRuleUpdateTargetById 932260 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
!ARGS:artifactName|!ARGS:redirect_to" !ARGS:artifactName|!ARGS:redirect_to"
# The fifth, for Referer: it is not checked for a Unix command without # The fifth, for Referer: it is not checked for a Unix command without
# arguments, or for Java starting a process. The cookies are left out in # arguments, for Java starting a process, or for a Java class name. 944130
# Inspect. # refused every page opened from the page of java.lang.String.md. The
# cookies are left out in Inspect.
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer" SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer" SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944130 "!REQUEST_HEADERS:Referer"
# The seventh: a container image push that reuses a layer names the image
# it takes the layer from in the query parameter from, which is not
# checked for command names. 932260 refused the push of a layer from
# owner/python3-app.
SecRule REQUEST_FILENAME "@rx ^/v2/.+/blobs/uploads/$" \
"id:900500,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:from"
` `
// bodyDirectives have the Core Rule Set read the part of a request body // bodyDirectives have the Core Rule Set read the part of a request body
@@ -144,6 +168,13 @@ SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'" severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
` `
// giteaExclusions are the eighth change: the fields of gitea's own
// requests that skip the rules that refused them there, as Coraza
// directives.
//
//go:embed gitea.conf
var giteaExclusions string
// cookiesNotRead are the cookies the Core Rule Set reads a request // cookiesNotRead are the cookies the Core Rule Set reads a request
// without, the rest of the fifth change. // without, the rest of the fifth change.
// //
@@ -160,6 +191,51 @@ type Params struct {
// BodyLimit is the most of a request body the Core Rule Set reads // BodyLimit is the most of a request body the Core Rule Set reads
// (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none. // (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
BodyLimit int64 BodyLimit int64
// ExclusionsFile is the path of a file of exclusions loaded before the
// Core Rule Set's rules (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while
// there is none.
ExclusionsFile string
}
// The ids of smallwebwaf's own rules, which an exclusions file may not
// name.
const (
firstOwnRuleID = 900000
lastOwnRuleID = 900999
)
// numberOrRange matches a number, or a range of numbers such as
// 900000-900999.
var numberOrRange = regexp.MustCompile(`(\d+)(?:-(\d+))?`)
// errSmallwebwafRuleID is the error of an exclusions file that names an
// id of smallwebwaf's own rules.
var errSmallwebwafRuleID = errors.New("an id of smallwebwaf's own rules")
// ownRuleNamed returns the first number or range in exclusions, the text
// of an exclusions file, that takes in an id of smallwebwaf's own rules,
// or "" if none does. Comment lines are skipped, as Coraza skips them.
func ownRuleNamed(exclusions string) string {
for line := range strings.Lines(exclusions) {
if strings.HasPrefix(strings.TrimSpace(line), "#") {
continue
}
for _, match := range numberOrRange.FindAllStringSubmatch(line, -1) {
first, _ := strconv.Atoi(match[1])
last := first
if match[2] != "" {
last, _ = strconv.Atoi(match[2])
}
if first <= lastOwnRuleID && last >= firstOwnRuleID {
return match[0]
}
}
}
return ""
} }
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe // CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
@@ -169,16 +245,35 @@ type CoreRuleSet struct {
bodyLimit int64 bodyLimit int64
} }
// New returns the Core Rule Set with the six changes, at params' // New returns the Core Rule Set with the eight changes, at params'
// paranoia level, without the rules it switches off, and reading request // paranoia level, without the rules it switches off, reading request
// bodies up to params' limit. // bodies up to params' limit, and with params' exclusions file loaded
// before the Core Rule Set's rules. An exclusions file that cannot be
// read, does not parse or names an id of smallwebwaf's own rules is an
// error.
func New(params Params) (*CoreRuleSet, error) { func New(params Params) (*CoreRuleSet, error) {
body := "" body := ""
if params.BodyLimit > 0 { if params.BodyLimit > 0 {
body = fmt.Sprintf(bodyDirectives, params.BodyLimit) body = fmt.Sprintf(bodyDirectives, params.BodyLimit)
} }
text := fmt.Sprintf(directives, params.ParanoiaLevel, body) exclusions := ""
if params.ExclusionsFile != "" {
file, err := os.ReadFile(params.ExclusionsFile)
if err != nil {
return nil, fmt.Errorf("read the exclusions file: %w", err)
}
exclusions = string(file)
if named := ownRuleNamed(exclusions); named != "" {
return nil, fmt.Errorf("the exclusions file names %s: %w", named,
errSmallwebwafRuleID)
}
}
text := fmt.Sprintf(directives, params.ParanoiaLevel, body, exclusions, giteaExclusions)
if len(params.DisabledRules) > 0 { if len(params.DisabledRules) > 0 {
ids := make([]string, len(params.DisabledRules)) ids := make([]string, len(params.DisabledRules))
+167 -2
View File
@@ -5,6 +5,7 @@ import (
"net/http/httptest" "net/http/httptest"
"net/netip" "net/netip"
"net/url" "net/url"
"os"
"path/filepath" "path/filepath"
"reflect" "reflect"
"strconv" "strconv"
@@ -119,7 +120,7 @@ func atDefaults(t *testing.T) *waf.CoreRuleSet {
} }
// wantChange checks that crs lets through passes, a gitea request one of // wantChange checks that crs lets through passes, a gitea request one of
// the six changes is for, and still finds result in refused, a request // the eight changes is for, and still finds result in refused, a request
// like it that the change is not for. // like it that the change is not for.
func wantChange( func wantChange(
t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result, t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result,
@@ -308,13 +309,14 @@ func TestCookiesGiteaFlashAndRedirectToAreNotRead(t *testing.T) {
matched(930120)) matched(930120))
} }
func TestRefererIsNotCheckedForACommandOrJavaStartingAProcess(t *testing.T) { func TestRefererIsNotCheckedForACommandOrJava(t *testing.T) {
t.Parallel() t.Parallel()
const ( const (
search = "https://git.example/explore/repos?q=env" search = "https://git.example/explore/repos?q=env"
runtimeJava = "https://git.example/openjdk/jdk/src/branch/master/src/" + runtimeJava = "https://git.example/openjdk/jdk/src/branch/master/src/" +
"java.base/share/classes/java/lang/Runtime.java" "java.base/share/classes/java/lang/Runtime.java"
stringMD = "https://git.example/owner/repo/src/branch/main/java.lang.String.md"
) )
crs := atDefaults(t) crs := atDefaults(t)
@@ -323,6 +325,8 @@ func TestRefererIsNotCheckedForACommandOrJavaStartingAProcess(t *testing.T) {
matched(932340)) matched(932340))
wantChange(t, crs, get("/", "Referer: "+runtimeJava), wantChange(t, crs, get("/", "Referer: "+runtimeJava),
get("/", "X-Page: "+runtimeJava), matched(944110)) get("/", "X-Page: "+runtimeJava), matched(944110))
wantChange(t, crs, get("/", "Referer: "+stringMD), get("/", "X-Page: "+stringMD),
matched(944130))
// It is still checked for script and SQL injection. // It is still checked for script and SQL injection.
wantResult(t, crs, wantResult(t, crs,
@@ -362,6 +366,167 @@ func TestEachDisabledRuleIsSwitchedOff(t *testing.T) {
wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{}) wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{})
} }
func TestImagePushThatReusesALayerIsNotCheckedForCommandNames(t *testing.T) {
t.Parallel()
crs := atDefaults(t)
reuse := "?mount=sha256:0000&from=owner/python3-app"
wantChange(t, crs, get("/v2/owner/app/blobs/uploads/"+reuse), get("/upload/"+reuse),
matched(932260))
// A command after the image's name is still refused.
wantResult(t, crs,
get("/v2/owner/app/blobs/uploads/?from=owner/app;cat%20/etc/passwd"),
matched(930120, 932160))
}
// withExclusions returns the Core Rule Set at the defaults, with the
// exclusions file at path.
func withExclusions(path string) (*waf.CoreRuleSet, error) {
return waf.New(waf.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, ExclusionsFile: path,
})
}
// exclusionsFile writes text to a file of exclusions and returns its path.
func exclusionsFile(t *testing.T, text string) string {
t.Helper()
path := filepath.Join(t.TempDir(), "exclusions.conf")
err := os.WriteFile(path, []byte(text), 0o600)
if err != nil {
t.Fatal(err)
}
return path
}
// backupFileExclusion is an exclusions file that switches 920500, a rule
// of phase 1, off for the path of a backup file under /files/.
const backupFileExclusion = `SecRule REQUEST_FILENAME "@beginsWith /files/" ` +
`"id:1,phase:1,pass,nolog,ctl:ruleRemoveTargetById=920500;REQUEST_FILENAME"` + "\n"
func TestExclusionsFileIsLoadedBeforeTheRules(t *testing.T) {
t.Parallel()
crs, err := withExclusions(exclusionsFile(t, backupFileExclusion))
if err != nil {
t.Fatalf("load the exclusions: %v", err)
}
wantChange(t, crs, get("/files/notes.txt~"), get("/notes.txt~"), matched(920500))
}
func TestRuleSwitchedOffAndNamedInTheExclusionsFileIsOff(t *testing.T) {
t.Parallel()
crs, err := waf.New(waf.Params{
ParanoiaLevel: 1, DisabledRules: []int{920500},
ExclusionsFile: exclusionsFile(t, backupFileExclusion),
})
if err != nil {
t.Fatalf("load the exclusions: %v", err)
}
wantResult(t, crs, get("/notes.txt~"), waf.Result{})
}
func TestExclusionsFileMayNameAnyNumberInAComment(t *testing.T) {
t.Parallel()
_, err := withExclusions(exclusionsFile(t,
"# 942100: GET /issues/900123\n"+backupFileExclusion))
if err != nil {
t.Errorf("the exclusions did not load: %v", err)
}
}
func TestExclusionsFileThatCannotBeUsedIsAnError(t *testing.T) {
t.Parallel()
for _, path := range []string{
filepath.Join(t.TempDir(), "missing.conf"),
exclusionsFile(t, "SecNoSuchDirective On\n"),
// The Core Rule Set's rules are not loaded yet.
exclusionsFile(t, "SecRuleUpdateTargetById 942100 \"!ARGS:id\"\n"),
// smallwebwaf's own rules may not be switched off or changed, by
// id or in a range.
exclusionsFile(t, "SecRuleRemoveById 900300\n"),
exclusionsFile(t, "SecRuleRemoveById 899999-901000\n"),
exclusionsFile(t, "SecAction \"id:1,phase:1,pass,nolog,"+
"ctl:ruleRemoveById=899999-901000\"\n"),
exclusionsFile(t, "SecAction \"id:1,phase:1,pass,nolog,"+
"ctl:ruleRemoveTargetById=900500;ARGS:from\"\n"),
} {
_, err := withExclusions(path)
if err == nil {
t.Errorf("%s loaded", path)
}
}
}
// giteaRequests returns the recorded ordinary gitea requests in testdata.
// Each has the Referer of a page that names the same, as a page opened
// from it has.
func giteaRequests(t *testing.T) []request {
t.Helper()
text, err := os.ReadFile(filepath.Join("testdata", "gitea-requests.txt"))
if err != nil {
t.Fatal(err)
}
var requests []request
for line := range strings.Lines(string(text)) {
line = strings.TrimSpace(line)
if line == "" || strings.HasPrefix(line, "#") {
continue
}
target, header, _ := strings.Cut(line, " ")
headers := []string{"Referer: http://git.example" + target}
if header != "" {
headers = append(headers, header)
}
requests = append(requests, get(target, headers...))
}
return requests
}
func TestOrdinaryGiteaRequestsPassAndAttacksAreRefused(t *testing.T) {
t.Parallel()
const threshold = 5 // SWWAF_WAF_ANOMALY_THRESHOLD's default
crs := atDefaults(t)
for _, r := range giteaRequests(t) {
if got := inspect(t, crs, r); got.Score >= threshold {
t.Errorf("%s %q refused: %+v", r.target, r.headers[1:], got)
}
}
// In q too, off gitea's search pages.
for _, parameter := range []string{"id", "q"} {
for _, attack := range []string{
"1'%20OR%20'1'='1",
"..%2Fcustom%2Fconf%2Fapp.ini",
"%3Cscript%3Ealert(1)%3C%2Fscript%3E",
} {
target := "/owner/repo?" + parameter + "=" + attack
if got := inspect(t, crs, get(target)); got.Score < threshold {
t.Errorf("%s let through: %+v", target, got)
}
}
}
}
// bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies. // bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies.
const bodyLimit = 8 << 10 const bodyLimit = 8 << 10
-51
View File
@@ -1,51 +0,0 @@
#!/bin/sh
# script/publish: build the image on the Docker host and push it to the
# registry under four tags: the UTC date (YYYY-MM-DD), the short commit
# hash, the branch name and latest. The build is script/docker's, so the
# tests and the linter run first. The host and the registry name are
# environment variables with defaults, for running it against another.
#
# The working tree must be clean: the hash tag names a commit, and an
# image built from uncommitted changes would carry a hash that does not
# describe it.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
: "${DOCKER_HOST:=ssh://root@fsn1app1}"
: "${PUBLISH_IMAGE:=git.eeqj.de/sneak/smallwebwaf}"
export DOCKER_HOST
main() {
cd "$ROOT"
if [ -n "$(git status --porcelain)" ]; then
echo "script/publish: the working tree is not clean; commit or stash first" >&2
exit 1
fi
date="$(date -u +%Y-%m-%d)"
hash="$(git rev-parse --short HEAD)"
branch="$(git rev-parse --abbrev-ref HEAD)"
if [ "$branch" = "HEAD" ]; then
echo "script/publish: detached HEAD; check out a branch first" >&2
exit 1
fi
# A tag is [A-Za-z0-9_.-]; a branch name such as feature/x is not.
branch="$(printf '%s' "$branch" | tr -c 'A-Za-z0-9_.-' '-')"
local_tag="$("$SCRIPT_DIR/projectname")"
"$SCRIPT_DIR/docker"
for tag in "$date" "$hash" "$branch" latest; do
docker tag "$local_tag" "$PUBLISH_IMAGE:$tag"
done
for tag in "$date" "$hash" "$branch" latest; do
docker push "$PUBLISH_IMAGE:$tag"
done
echo "pushed $PUBLISH_IMAGE with tags: $date $hash $branch latest"
}
main "$@"