Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
7e26fe37bc |
@@ -1,5 +1,5 @@
|
||||
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker publish \
|
||||
hooks build run example-app gitea-requests gitea-setup
|
||||
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
|
||||
run example-app gitea-requests gitea-setup
|
||||
|
||||
# Makefile targets are thin shims; the implementations live in script/
|
||||
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
|
||||
@@ -7,8 +7,7 @@
|
||||
# which test checks. build and run are for working on the code by hand;
|
||||
# example-app checks the image with an app built on it; gitea-requests
|
||||
# reports the gitea requests the Core Rule Set refuses at the defaults;
|
||||
# gitea-setup checks the setup in deploy/gitea; publish builds the image
|
||||
# on the Docker host and pushes it to the registry.
|
||||
# gitea-setup checks the setup in deploy/gitea.
|
||||
|
||||
bootstrap:
|
||||
@script/bootstrap
|
||||
@@ -37,9 +36,6 @@ check:
|
||||
docker:
|
||||
@script/docker
|
||||
|
||||
publish:
|
||||
@script/publish
|
||||
|
||||
hooks:
|
||||
@script/install-precommit
|
||||
|
||||
|
||||
@@ -192,9 +192,9 @@ account, a key, a token or an alert destination stays off until you give one.
|
||||
- Any `SWWAF_` setting can be given as a file instead, for secrets and long
|
||||
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
|
||||
The file's contents are the value, less one newline at the end. Setting both
|
||||
forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` already names a file, so
|
||||
it has no `_FILE` form, and no `SWRP_` setting has one, since `rproxy` never
|
||||
touches disk.
|
||||
forms stops the start. `SWWAF_LOG_REMOTE_TLS_CA_FILE` and
|
||||
`SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` already name a file, so they have no
|
||||
`_FILE` form. No `SWRP_` setting has one, since `rproxy` never touches disk.
|
||||
- A duration is Go's syntax, with `d` for days: `90s`, `15m`, `24h`, `7d`.
|
||||
- A size is a number of bytes with an optional `K`, `M` or `G`, in powers of
|
||||
1024: `512K`, `100M`, `5G`.
|
||||
@@ -272,6 +272,7 @@ Every setting the code reads, with its default:
|
||||
| `SWWAF_WAF_DISABLED_RULES` | `920340,920420,920440,920640,930130,930140` | Core Rule Set rule ids switched off; a list you give replaces the default, so repeat these six to keep them off. |
|
||||
| `SWWAF_WAF_EXEMPT_PATHS` | empty | Path prefixes the Core Rule Set does not inspect. |
|
||||
| `SWWAF_WAF_BODY_LIMIT` | `off` | How much of a request body the Core Rule Set reads, a size up to `64K`; `off` reads none. |
|
||||
| `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` | unset | A file of Core Rule Set exclusions, as Coraza directives, that switch rules off for some fields or paths only; `deploy/gitea/crs-exclusions.conf` is gitea's. |
|
||||
| `SWWAF_BLOCKLIST_URLS` | empty | Blocklists to fetch, text files of addresses and netblocks one to a line, such as `https://www.spamhaus.org/drop/drop.txt`. |
|
||||
| `SWWAF_BLOCKLIST_REFRESH` | `24h` | How long after a list was last tried it is fetched again, `1h` or more. |
|
||||
| `SWWAF_BLOCKLIST_ACTION` | `deny` | What a client on a blocklist gets: `deny`, `limit:<percent>` of every limit, or `log`. |
|
||||
@@ -636,10 +637,14 @@ and the status for a size or time limit, such as `413` for a body too large.
|
||||
headers of every request, and no body unless `SWWAF_WAF_BODY_LIMIT` is set. In
|
||||
front of a code forge, bodies carry what people write and publish, which the
|
||||
rules take for attacks.
|
||||
- `smallwebwaf` changes the Core Rule Set in six ways so that it lets ordinary
|
||||
- `smallwebwaf` changes the Core Rule Set in seven ways so that it lets ordinary
|
||||
requests to a code forge through, such as git over HTTP and sign-in through
|
||||
Git Credential Manager. No setting undoes them; "Attack detection" in
|
||||
[`SPEC.md`](SPEC.md) lists them.
|
||||
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` switches single rules off for an app's
|
||||
own fields or paths. gitea's exclusions, in
|
||||
`deploy/gitea/crs-exclusions.conf`, let its searches, file names and branch
|
||||
names through.
|
||||
- Rule files are plain text, one rule to a line, edited by hand and read again
|
||||
while `servewaf` runs. The image ships one file of probes no real visitor
|
||||
sends (see [`docs/rules.md`](docs/rules.md)).
|
||||
@@ -743,11 +748,6 @@ on the host.
|
||||
`make check`.
|
||||
- `script/docker`: builds the image, whose build runs the tests and the linter
|
||||
first. `make docker`.
|
||||
- `script/publish`: runs `script/docker` on the Docker host `DOCKER_HOST`
|
||||
(`ssh://root@fsn1app1` unless set) and pushes the image to `PUBLISH_IMAGE`
|
||||
(`git.eeqj.de/sneak/smallwebwaf` unless set) under four tags: the UTC date as
|
||||
`YYYY-MM-DD`, the short commit hash, the branch name and `latest`. Refuses a
|
||||
working tree with uncommitted changes. `make publish`.
|
||||
- `script/cibuild`: what CI runs: `script/bootstrap`, `script/check`, then the
|
||||
image build.
|
||||
- `script/precommit`: run by the git pre-commit hook; runs `script/check`.
|
||||
|
||||
@@ -36,7 +36,8 @@ to them.
|
||||
- the state files;
|
||||
- a lookup database;
|
||||
- a `SWWAF_` setting given as a file;
|
||||
- the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names.
|
||||
- the file `SWWAF_LOG_REMOTE_TLS_CA_FILE` names;
|
||||
- the file `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names.
|
||||
|
||||
## Non-goals
|
||||
|
||||
@@ -81,7 +82,7 @@ to them.
|
||||
of bans is full.
|
||||
- `internal/rules`: reads the rule files at start and again when they
|
||||
change, and tells which rules a request matches.
|
||||
- `internal/waf`: the Core Rule Set, with the six changes listed under
|
||||
- `internal/waf`: the Core Rule Set, with the seven changes listed under
|
||||
"Attack detection", as Coraza directives.
|
||||
- `internal/lookup`: looks clients up through GeoJS or in the lookup
|
||||
database.
|
||||
@@ -325,9 +326,10 @@ Conventions:
|
||||
lists: `SWWAF_ADMIN_TOKEN_FILE` names the file that holds `SWWAF_ADMIN_TOKEN`.
|
||||
The file's contents are the value, less one newline at the end, and are
|
||||
checked as the value itself would be. Setting both forms stops the start, and
|
||||
so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE`
|
||||
already holds a file name, so it has no `_FILE` form. No `SWRP_` setting has
|
||||
one, since `rproxy` never touches disk.
|
||||
so does naming a file that cannot be read. `SWWAF_LOG_REMOTE_TLS_CA_FILE` and
|
||||
`SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` already hold a file name, so they have
|
||||
no `_FILE` form. No `SWRP_` setting has one, since `rproxy` never touches
|
||||
disk.
|
||||
- Settings, files included, are read once at start. Changing one means
|
||||
restarting the container. The files `servewaf` watches while it runs are its
|
||||
state files, its rule files and the lookup database.
|
||||
@@ -583,6 +585,14 @@ sends it.
|
||||
probes these rules caught (see "Rule files"). A list you give replaces the
|
||||
default, so name these six ids in it to keep them off.
|
||||
|
||||
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` names a file of exclusions, written as
|
||||
Coraza directives, which `servewaf` loads after the Core Rule Set and the
|
||||
changes below. An exclusion switches one rule off for some fields or paths
|
||||
only, where `SWWAF_WAF_DISABLED_RULES` switches it off everywhere.
|
||||
`deploy/gitea/crs-exclusions.conf` holds gitea's. The start stops on a file
|
||||
that cannot be read or does not parse. It stops too on a file that names any
|
||||
number from 900000 to 900999, the ids of `smallwebwaf`'s own rules.
|
||||
|
||||
- `SWWAF_WAF_BODY_LIMIT` switches body inspection on, up to that size. It takes
|
||||
at most `64K`, since `servewaf` gets no more of a body than that (see "The
|
||||
answer"). Form data and multipart bodies are read up to the size. JSON and XML
|
||||
@@ -612,7 +622,7 @@ sends it.
|
||||
scanner from an ordinary client. In front of gitea, registry and git clients
|
||||
get `404` and `401` by design, hundreds of times a minute.
|
||||
|
||||
`smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of
|
||||
`smallwebwaf` changes the Core Rule Set 4.25.0 in seven ways, since in front of
|
||||
gitea it would otherwise refuse ordinary requests. The changes hold in front of
|
||||
every app, and no setting undoes them:
|
||||
|
||||
@@ -669,6 +679,10 @@ every app, and no setting undoes them:
|
||||
must check it itself.
|
||||
6. Responses are not inspected. A raw file from a repository, such as a shell
|
||||
script, looks to the response rules like source code leaking from the server.
|
||||
7. The `from` parameter of a container registry's upload path,
|
||||
`/v2/<image>/blobs/uploads/`, is not checked for command names (932260). A
|
||||
push that reuses a layer names there the image it takes the layer from. An
|
||||
image such as `owner/python3-app` would have that push refused.
|
||||
|
||||
The parameter and cookie names in these changes are matched without regard to
|
||||
case, as Coraza matches them.
|
||||
|
||||
@@ -104,11 +104,12 @@ services:
|
||||
# address of an automation fleet on another host here too; without
|
||||
# that, it is counted like any client.
|
||||
SWWAF_ALLOW_NETS: 10.99.0.128/25,fd10:99:0:0:8000::/65
|
||||
# In block mode the Core Rule Set refuses everyday Gitea searches
|
||||
# and file names, and 31 refusals in a minute ban the address.
|
||||
SWWAF_WAF_MODE: detect
|
||||
# The Core Rule Set's exclusions for Gitea's searches, file names
|
||||
# and branch names.
|
||||
SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: /etc/smallwebwaf/crs-exclusions.conf
|
||||
volumes:
|
||||
- servewaf-state:/var/lib/smallwebwaf
|
||||
- ./crs-exclusions.conf:/etc/smallwebwaf/crs-exclusions.conf:ro
|
||||
networks:
|
||||
- shared
|
||||
logging: *logging
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
# The Core Rule Set's exclusions for Gitea, which servewaf loads from
|
||||
# SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE. Each switches one rule off for the
|
||||
# Gitea fields or paths where it refused ordinary Gitea requests. Gitea
|
||||
# sends these in the query string:
|
||||
# - q: the text of every search;
|
||||
# - file, path, template, body and redirect_to: a file name, alone or in
|
||||
# JSON or a link;
|
||||
# - name, tag, rule_name and refSubUrl: a branch or tag name.
|
||||
# docs/gitea.md says how to add an exclusion.
|
||||
|
||||
# Searches.
|
||||
# 930110: a search for ../
|
||||
SecRuleUpdateTargetById 930110 "!ARGS:q"
|
||||
# 931100: a search for http://127.0.0.1:8080
|
||||
SecRuleUpdateTargetById 931100 "!ARGS:q"
|
||||
# 932140: a search for if (a == b) {
|
||||
SecRuleUpdateTargetById 932140 "!ARGS:q"
|
||||
# 932235: a search for `make check`
|
||||
SecRuleUpdateTargetById 932235 "!ARGS:q"
|
||||
# 932250: a search for ls -la
|
||||
SecRuleUpdateTargetById 932250 "!ARGS:q"
|
||||
# 932380: a search for `whoami`
|
||||
SecRuleUpdateTargetById 932380 "!ARGS:q"
|
||||
# 933135: a search for ${VAR}
|
||||
SecRuleUpdateTargetById 933135 "!ARGS:q"
|
||||
# 934110: a search for http://127.0.0.1:8080
|
||||
SecRuleUpdateTargetById 934110 "!ARGS:q"
|
||||
# 941100: a search for <script>
|
||||
SecRuleUpdateTargetById 941100 "!ARGS:q"
|
||||
# 941110: a search for <script>, or a page opened from its results
|
||||
SecRuleUpdateTargetById 941110 "!ARGS:q|!REQUEST_HEADERS:Referer"
|
||||
# 941160: a search for <img src=, or a page opened from its results
|
||||
SecRuleUpdateTargetById 941160 "!ARGS:q|!REQUEST_HEADERS:Referer"
|
||||
# 941390: a search for fetch(
|
||||
SecRuleUpdateTargetById 941390 "!ARGS:q"
|
||||
# 942100: a search for SELECT * FROM users WHERE
|
||||
SecRuleUpdateTargetById 942100 "!ARGS:q"
|
||||
# 942190: a search for union select
|
||||
SecRuleUpdateTargetById 942190 "!ARGS:q"
|
||||
# 942230: a search for if (a == b) {
|
||||
SecRuleUpdateTargetById 942230 "!ARGS:q"
|
||||
# 944100: a search for java.lang.Runtime, or a page opened from its results
|
||||
SecRuleUpdateTargetById 944100 "!ARGS:q|!REQUEST_HEADERS:Referer"
|
||||
|
||||
# Searches, and file, branch and tag names.
|
||||
# 930120: a search for package.json, or the commit graph of .gitignore
|
||||
SecRuleUpdateTargetById 930120 "!ARGS:q|!ARGS:file|!ARGS:template|!ARGS:body|\
|
||||
!ARGS:refSubUrl|!ARGS:name|!ARGS:tag|!ARGS:rule_name"
|
||||
# 932160: a search for /bin/sh, or the API's contents of bin/docker-entrypoint
|
||||
SecRuleUpdateTargetById 932160 "!ARGS:q|!ARGS:file|!ARGS:template|!ARGS:body|\
|
||||
!ARGS:refSubUrl|!ARGS:name|!ARGS:tag|!ARGS:rule_name"
|
||||
# 932260: a search for python3, or deleting the branch docker-build
|
||||
SecRuleUpdateTargetById 932260 "!ARGS:q|!ARGS:file|!ARGS:template|!ARGS:body|\
|
||||
!ARGS:refSubUrl|!ARGS:name|!ARGS:tag|!ARGS:rule_name"
|
||||
|
||||
# Searches, and file names in the query string.
|
||||
# 932120: a search for Invoke-WebRequest, or the commits of install-script.sh
|
||||
SecRuleUpdateTargetById 932120 "!ARGS:q|!ARGS:file|!ARGS:path|!ARGS:template|\
|
||||
!ARGS:body|!ARGS:redirect_to"
|
||||
# 932130: a search for $(pwd), or an issue opened from the template [draft].md
|
||||
SecRuleUpdateTargetById 932130 "!ARGS:q|!ARGS:file|!ARGS:path|!ARGS:template|\
|
||||
!ARGS:body|!ARGS:redirect_to"
|
||||
# 934100: a search for process.env, or the commits of process.env.d.ts
|
||||
SecRuleUpdateTargetById 934100 "!ARGS:q|!ARGS:file|!ARGS:path|!ARGS:template|\
|
||||
!ARGS:body|!ARGS:redirect_to"
|
||||
# 934130: a search for __proto__, or the commits of __proto__.json
|
||||
SecRuleUpdateTargetById 934130 "!ARGS:q|!ARGS:file|!ARGS:path|!ARGS:template|\
|
||||
!ARGS:body|!ARGS:redirect_to"
|
||||
# 941130: the commits of index.xhtml
|
||||
SecRuleUpdateTargetById 941130 "!ARGS:file|!ARGS:path|!ARGS:template|\
|
||||
!ARGS:body|!ARGS:redirect_to"
|
||||
# 941180: a search for document.cookie, or the commits of document.write.js
|
||||
SecRuleUpdateTargetById 941180 "!ARGS:q|!ARGS:file|!ARGS:path|!ARGS:template|\
|
||||
!ARGS:body|!ARGS:redirect_to"
|
||||
# 942140: the commits of information_schema.sql
|
||||
SecRuleUpdateTargetById 942140 "!ARGS:file|!ARGS:path|!ARGS:template|\
|
||||
!ARGS:body|!ARGS:redirect_to"
|
||||
# 944110: a search for java.lang.Runtime, or the commits of Runtime.java
|
||||
SecRuleUpdateTargetById 944110 "!ARGS:q|!ARGS:file|!ARGS:path|!ARGS:template|\
|
||||
!ARGS:body|!ARGS:redirect_to"
|
||||
# 944130: a search for java.lang.Runtime, the commits of java.lang.String.md,
|
||||
# or any page opened from that file's page, which names it in the Referer
|
||||
SecRuleUpdateTargetById 944130 "!ARGS:q|!ARGS:file|!ARGS:path|!ARGS:template|\
|
||||
!ARGS:body|!ARGS:redirect_to|!REQUEST_HEADERS:Referer"
|
||||
|
||||
# File names in the path of a file's page, raw file, blame, history and edit
|
||||
# page, and of the API's calls for its contents.
|
||||
# 934100: the page of process.env.d.ts
|
||||
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
|
||||
"id:10001,phase:1,pass,nolog,ctl:ruleRemoveTargetById=934100;REQUEST_FILENAME"
|
||||
# 941130: the page of docs/a b.xhtml
|
||||
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
|
||||
"id:10002,phase:1,pass,nolog,ctl:ruleRemoveTargetById=941130;REQUEST_FILENAME"
|
||||
# 944130: the page of java.lang.String.md
|
||||
SecRule REQUEST_FILENAME "@rx ^/(?:api/v1/repos/)?[^/]+/[^/]+/(?:src|raw|media|blame|commits|_edit|_delete|contents)/" \
|
||||
"id:10003,phase:1,pass,nolog,ctl:ruleRemoveTargetById=944130;REQUEST_FILENAME"
|
||||
+50
-183
@@ -13,7 +13,8 @@ Four containers share one Docker network, the shared network:
|
||||
plain HTTP to HTTPS;
|
||||
- `rproxy`, in front of gitea;
|
||||
- gitea 1.27.3, as its image comes;
|
||||
- `servewaf`, with a volume for its state files.
|
||||
- `servewaf`, with a volume for its state files, and with
|
||||
`deploy/gitea/crs-exclusions.conf` mounted read-only.
|
||||
|
||||
To start it:
|
||||
|
||||
@@ -30,7 +31,8 @@ To start it:
|
||||
`docker run --rm <gitea's image> gitea generate secret SECRET_KEY`, with
|
||||
gitea's image as `deploy/gitea/compose.yaml` names it. Do this once per
|
||||
install, and keep the key.
|
||||
5. Run `docker compose up --detach` in `deploy/gitea`.
|
||||
5. Run `docker compose up --detach` in `deploy/gitea`, with
|
||||
`crs-exclusions.conf` beside `compose.yaml`.
|
||||
6. Make gitea's first user with `gitea admin user create`, in gitea's container,
|
||||
as the user `git`.
|
||||
|
||||
@@ -46,6 +48,11 @@ under a name of its own, and checks:
|
||||
`rproxy`, and that `servewaf` checks them;
|
||||
- that gitea logs the visitor's address, not `rproxy`'s;
|
||||
- that a request from a container on the shared network skips the checks;
|
||||
- that the Core Rule Set refuses none of the ordinary gitea requests in
|
||||
`internal/waf/testdata/gitea-requests.txt`, sent by a signed-in user, nor an
|
||||
API call with a JSON body, nor a file saved through the API;
|
||||
- that it refuses SQL injection, path traversal and script injection in a query
|
||||
string;
|
||||
- that with `servewaf` stopped, gitea still answers through `rproxy`.
|
||||
|
||||
## Uploads and downloads
|
||||
@@ -79,11 +86,8 @@ GiB a day.
|
||||
|
||||
## The checks
|
||||
|
||||
`SWWAF_WAF_MODE` is `detect`. The Core Rule Set then logs a match, with its rule
|
||||
ids, and refuses nothing. In `block` mode, the default, it refuses everyday
|
||||
gitea searches and file names. The section "What the Core Rule Set refuses in
|
||||
`block` mode" below lists them. Each such refusal counts toward the error burst,
|
||||
so 31 in one minute ban the address.
|
||||
The Core Rule Set runs in `block` mode, the default, with gitea's exclusions
|
||||
(see "The Core Rule Set's exclusions" below).
|
||||
|
||||
Every other check stays at its default. A client may make 120 requests a minute,
|
||||
1,500 an hour and 6,000 a day. Archive downloads and blame and history pages are
|
||||
@@ -153,188 +157,51 @@ within two minutes. Its address is then banned, its running jobs fail, and each
|
||||
repeat within a day makes the ban longer. Newer runners ask for work less often.
|
||||
Put the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||
|
||||
## What the Core Rule Set refuses in `block` mode
|
||||
## The Core Rule Set's exclusions
|
||||
|
||||
These notes record what the Core Rule Set does in front of gitea in `block`
|
||||
mode, at the other defaults. They are observations, not requirements.
|
||||
`script/gitea-requests` runs gitea behind `rproxy` and `servewaf` at the
|
||||
defaults. It sends through them a sample of gitea's ordinary use and the
|
||||
requests named below. It reports each request the Core Rule Set refuses, so that
|
||||
these notes can be checked again when gitea or the Core Rule Set changes
|
||||
version. In `detect` mode, as the setup runs it, the same requests are logged
|
||||
with their rule ids, and pass.
|
||||
`deploy/gitea/crs-exclusions.conf` switches single Core Rule Set rules off where
|
||||
they refused ordinary gitea requests. `servewaf` loads it from
|
||||
`SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE`. Each exclusion covers one rule, in these
|
||||
places only:
|
||||
|
||||
Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts
|
||||
toward). The request log names the rule in `waf_rule_ids`, which
|
||||
`SWWAF_WAF_DISABLED_RULES` can switch off.
|
||||
- `q`, the text of every search;
|
||||
- the query parameters that hold a file name: `file`, `path`, `template`, `body`
|
||||
and `redirect_to`;
|
||||
- the query parameters that hold a branch or tag name: `name`, `tag`,
|
||||
`rule_name` and `refSubUrl`;
|
||||
- `Referer`, on a page opened from search results or from a file's page;
|
||||
- the path of a file's page, raw file, blame, history and edit page, and of the
|
||||
API's call for its contents.
|
||||
|
||||
### What it lets through
|
||||
gitea reads these as text to search for, or as names in a repository. It never
|
||||
runs them as SQL, a command or a page's script. So an attack that only the
|
||||
excluded rules would catch there reaches gitea and does nothing. Every other
|
||||
field and path keeps every rule.
|
||||
|
||||
The Core Rule Set lets gitea's ordinary use through, apart from the refusals in
|
||||
the sections after this one:
|
||||
To add an exclusion for a request the Core Rule Set refuses:
|
||||
|
||||
- browsing and views of files in a repository, with their history, blame and the
|
||||
file tree;
|
||||
- diffs, including their hidden lines and large files, and pull request review;
|
||||
- git's clone, fetch and push over HTTP;
|
||||
- signing in, including the return to the page a visitor came from, and sign-in
|
||||
with Git Credential Manager, git-credential-oauth or tea;
|
||||
- the API's calls for a file and its commits;
|
||||
- pushing and pulling container images and packages;
|
||||
- Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`;
|
||||
- posting issues, pull requests, comments, wiki pages and files saved in the web
|
||||
editor, code included, since no body is read;
|
||||
- the page gitea shows after a change, with its message naming what was done.
|
||||
1. Find the request in `servewaf`'s log. Its line gives the path and query, and
|
||||
names the rules in `waf_rule_ids`.
|
||||
2. For each rule, add a `SecRuleUpdateTargetById` line naming the field it
|
||||
matched, as the file's other lines do. For a path, copy one of the file's
|
||||
`SecRule REQUEST_FILENAME` rules and give it a new id. An id from 900000 to
|
||||
900999 stops the start.
|
||||
3. Above it, write a one-line comment naming the request it refused.
|
||||
4. Add the request to `internal/waf/testdata/gitea-requests.txt`, and run
|
||||
`make check`.
|
||||
5. Restart `servewaf`.
|
||||
|
||||
### A query string that reads as an attack
|
||||
The Core Rule Set still refuses these gitea requests:
|
||||
|
||||
Most often this is a search. A search term is a query parameter, which the Core
|
||||
Rule Set checks against its lists of system files and commands (see the fourth
|
||||
change under "Attack detection" in `SPEC.md`). A release attachment uploaded
|
||||
through the API as `docker-compose.yml` is refused the same way. Examples:
|
||||
- the page of a file whose name ends in `~`, such as an editor's backup file.
|
||||
Rule 920500 checks the path before any exclusion takes effect.
|
||||
- the page of a file in a directory named `...`, which reads as path traversal;
|
||||
- an artifact upload from `actions/upload-artifact@v3`, which sends the header
|
||||
`Content-Range`. `actions/upload-artifact@v4` does not.
|
||||
|
||||
- a name on those lists, such as `package.json`, `.gitignore` or
|
||||
`docker-compose.yml`;
|
||||
- text that starts with a command name, such as `python3` or `ssh key`, or has
|
||||
one right after a `/` in its first word, such as `feat/docker-support`;
|
||||
- a shell command with its options or a system path (`ls -la`, `sed -i`,
|
||||
`/bin/sh`), or a command in backticks;
|
||||
- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`) or an SQL
|
||||
statement (`SELECT * FROM users WHERE`);
|
||||
- a URL naming an IP address or `localhost`.
|
||||
With `SWWAF_WAF_BODY_LIMIT` set, it also refuses issue text, comments, wiki
|
||||
pages and files saved in the web editor that hold shell commands or code.
|
||||
|
||||
### A branch, tag or file name that starts with a command name
|
||||
|
||||
The Core Rule Set's list of commands (932260) holds names such as `docker-`,
|
||||
`python3`, `ansible`, `base64` and `whoami`. A query parameter holding a branch,
|
||||
tag or file name that starts with one is refused. So is one that has such a name
|
||||
right after a `/`, such as `feat/docker-support`, `fix/docker-build` or
|
||||
`renovate/docker-build-push-action-6.x`. gitea's own pages send the name in a
|
||||
query parameter that 930120, 932160 and 932260 still check:
|
||||
|
||||
- `refSubUrl`, the branch or tag of a directory listing. A listing sends it when
|
||||
it asks for its entries' last commits in a second request. It does so whenever
|
||||
gitea takes more than a second to work them out;
|
||||
- `name`, when a branch is deleted or restored on the branches page;
|
||||
- `tag`, when a release is started from a tag;
|
||||
- `template`, the file of an issue template, when an issue is opened from it;
|
||||
- `rule_name`, when a branch protection rule is opened for editing in the
|
||||
repository's settings or has just been saved.
|
||||
|
||||
What a visitor sees, for a branch named `docker-build`:
|
||||
|
||||
- A directory listing that makes that second request leaves those last commits
|
||||
out and shows an error.
|
||||
- The branches page shows an error instead of deleting or restoring the branch.
|
||||
- A release started from such a tag gets the `403` answer of `smallwebwaf` in
|
||||
place of its form. So does an issue opened from such a template, and such a
|
||||
rule opened for editing.
|
||||
- Saving such a rule stores it. gitea then returns to the branch settings page,
|
||||
which gets the `403` answer, so the save looks as if it failed.
|
||||
|
||||
### A file name in a query parameter that trips the code rules
|
||||
|
||||
The Core Rule Set's rules for Java, script, PowerShell and SQL find what they
|
||||
look for in some file names. This holds too in the ten query parameters that
|
||||
930120, 932160 and 932260 do not check (see the fourth change under "Attack
|
||||
detection" in `SPEC.md`). Examples:
|
||||
|
||||
- `runtime` in OpenJDK's `src/java.base/share/classes/java/lang/Runtime.java`
|
||||
(944110);
|
||||
- a Java class, as in `java.lang.String.md` (944130);
|
||||
- `document.write.js` (941180);
|
||||
- `process.env.d.ts` (934100);
|
||||
- `__proto__` (934130);
|
||||
- the PowerShell command `Install-Script`, as in `install-script.sh` (932120);
|
||||
- a database's name, as in PostgreSQL's `information_schema.sql` (942140).
|
||||
|
||||
For such a file, these requests are refused:
|
||||
|
||||
- the API's list of its commits (`path`);
|
||||
- the API's call for the contents of files that names it in its query (`body`);
|
||||
- the commit graph of that file alone (`file`);
|
||||
- the hidden lines of its diff (`path`), which stay hidden;
|
||||
- the sign-in page, for a visitor who opens a page of the file that needs
|
||||
signing in, such as its edit page (`redirect_to`). The visitor can still sign
|
||||
in by opening the sign-in page itself.
|
||||
|
||||
### A path
|
||||
|
||||
- A file name ending in `~`, or an `.xhtml` file whose path holds a space.
|
||||
- A path that holds what the Core Rule Set takes for a Java class (944130), as
|
||||
`java.lang.String.md` and
|
||||
`META-INF/services/org.apache.commons.logging.LogFactory` do.
|
||||
- A path that holds what it takes for Node.js code (934100), as
|
||||
`process.env.d.ts` does.
|
||||
- A path with a directory named `...` (930100, 930110), such as `.../notes.txt`.
|
||||
|
||||
For such a file, these are refused:
|
||||
|
||||
- the page of the file;
|
||||
- its raw file;
|
||||
- its history and its blame;
|
||||
- its edit page.
|
||||
|
||||
The directory that holds it lists it, and a clone carries it.
|
||||
|
||||
### A container image push that reuses a layer
|
||||
|
||||
A push can reuse a layer of an image docker has pushed to that gitea before.
|
||||
docker then asks gitea to take the layer from that image, naming it in the query
|
||||
parameter `from`, which 932260 checks. The request is refused when the image's
|
||||
owner starts with an entry on the list of commands. It is refused too when the
|
||||
image's name after the owner starts with one. Examples are `owner/python3-app`,
|
||||
`owner/docker-base`, and any image of an owner named `ansible` or `docker-team`.
|
||||
|
||||
What happens next depends on docker's image store. A docker that keeps its
|
||||
images in containerd's image store fails the push, and fails each new try the
|
||||
same way. A new install of Docker Engine 29 does this. A docker that keeps them
|
||||
in its own older image store uploads the layer instead. The push then succeeds
|
||||
after one refusal for each layer it reuses, and each of those refusals counts
|
||||
toward the error burst ("Attack detection" in `SPEC.md`).
|
||||
|
||||
### Artifact uploads from `actions/upload-artifact@v3`
|
||||
|
||||
This action sends the header `Content-Range`, which the Core Rule Set refuses
|
||||
(920450). The action sends two files at a time, does not retry a `403`, and
|
||||
sends no further file once one is refused. So an upload is refused once or
|
||||
twice, however many files it holds, and the step fails. One upload does not ban
|
||||
the runner. The error burst ("Attack detection" in `SPEC.md`) bans it after more
|
||||
than 30 such refusals from its address in a minute, as when many jobs upload at
|
||||
once. `actions/upload-artifact@v4` does not send the header.
|
||||
|
||||
### What body inspection refuses
|
||||
|
||||
With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In
|
||||
front of gitea it then refuses these, until their rule ids, named in the request
|
||||
log, are added to `SWWAF_WAF_DISABLED_RULES`:
|
||||
|
||||
- issue and comment text, wiki pages and files saved in the web editor that hold
|
||||
shell commands or code (932125, 932235, 932250 and others);
|
||||
- package descriptions that show code;
|
||||
- PyPI uploads (922130);
|
||||
- attachments named like `debug.log` or `config.yml` (932180).
|
||||
|
||||
### What `script/gitea-requests` sends
|
||||
|
||||
Of the requests above, the script sends:
|
||||
|
||||
- for each file named under "A file name in a query parameter that trips the
|
||||
code rules", the API's list of its commits;
|
||||
- for `Runtime.java` alone, the commit graph of that file, the hidden lines of
|
||||
its diff, and the API's call for the contents of files that names it;
|
||||
- the sign-in page for a visitor who opens the edit page of `document.write.js`;
|
||||
- the pages of `java.lang.String.md`,
|
||||
`META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
|
||||
and `.../notes.txt`;
|
||||
- one container image push that reuses a layer, which it does not try again.
|
||||
|
||||
It does not send the raw file, history, blame or edit page of those four files,
|
||||
or the pages of the directories `META-INF/services` and `...`. It also sends a
|
||||
short sample of gitea's ordinary use:
|
||||
|
||||
- signing in;
|
||||
- pages of files and commits, and a diff;
|
||||
- the API;
|
||||
- a push, and clone and fetch over HTTP;
|
||||
- two container image pushes.
|
||||
`make gitea-requests` runs gitea behind `rproxy` and `servewaf` at the defaults,
|
||||
without these exclusions. It reports each request the Core Rule Set refuses
|
||||
there.
|
||||
|
||||
@@ -192,6 +192,10 @@ chooses between the two (see "The image's command" below).
|
||||
the Core Rule Set reads, a size of at most `64K`. A question carries no more
|
||||
of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one
|
||||
byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
|
||||
- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule Set
|
||||
exclusions, written as Coraza directives, loaded after the Core Rule Set. A
|
||||
file that cannot be read or does not parse stops the start. So does a file
|
||||
that names any number from 900000 to 900999. It has no `_FILE` form.
|
||||
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
|
||||
in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.
|
||||
|
||||
|
||||
+159
-353
@@ -7,9 +7,6 @@
|
||||
// the image runs.
|
||||
package config
|
||||
|
||||
// FIXME: the rproxy config is in rproxy.go, the servewaf is in config.go -
|
||||
// wrong and inconsistent.
|
||||
|
||||
import (
|
||||
"crypto/x509"
|
||||
"errors"
|
||||
@@ -224,12 +221,15 @@ type Config struct {
|
||||
// WAFExemptPaths the path prefixes it does not inspect
|
||||
// (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body
|
||||
// it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
||||
WAFMode string
|
||||
WAFParanoiaLevel int
|
||||
WAFAnomalyThreshold int
|
||||
WAFDisabledRules []int
|
||||
WAFExemptPaths []string
|
||||
WAFBodyLimit int64
|
||||
// CoreRuleSetExclusionsFile is the path of the file of its exclusions
|
||||
// (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while there is none.
|
||||
WAFMode string
|
||||
WAFParanoiaLevel int
|
||||
WAFAnomalyThreshold int
|
||||
WAFDisabledRules []int
|
||||
WAFExemptPaths []string
|
||||
WAFBodyLimit int64
|
||||
CoreRuleSetExclusionsFile string
|
||||
// TrapPaths are the paths a request for which is a clear sign of
|
||||
// attack (SWWAF_TRAP_PATHS), each starting with / and without a ?.
|
||||
TrapPaths []string
|
||||
@@ -366,16 +366,13 @@ var (
|
||||
"is not a header name such as accept-language")
|
||||
errHeaderTakenOut = errors.New(
|
||||
"is taken out of every request by Go's HTTP server, so it can never " +
|
||||
"be logged",
|
||||
)
|
||||
"be logged")
|
||||
errOnBothLists = errors.New("is in SWWAF_DENIED_COUNTRIES too")
|
||||
errNotLookupSource = errors.New("is not geojs, file or off")
|
||||
errNeedsLookups = errors.New("it needs each client looked up")
|
||||
errNeedsDBPath = errors.New(
|
||||
"it names the file to look clients up in",
|
||||
)
|
||||
errDBPathUnused = errors.New("only file reads it")
|
||||
errOver64K = errors.New(
|
||||
errNeedsDBPath = errors.New("it names the file to look clients up in")
|
||||
errDBPathUnused = errors.New("only file reads it")
|
||||
errOver64K = errors.New(
|
||||
"is more than 64K, the most servewaf gets of a body")
|
||||
errNotCommand = errors.New("is not rproxy or servewaf")
|
||||
errNotDurationAboveZero = errors.New(
|
||||
@@ -400,29 +397,22 @@ var (
|
||||
errNotPathPrefix = errors.New(
|
||||
"is not a path prefix starting with /, such as /assets/")
|
||||
errNotTrapPath = errors.New(
|
||||
"is not a path starting with / and without a ?, such as /wp-login.php",
|
||||
)
|
||||
"is not a path starting with / and without a ?, such as /wp-login.php")
|
||||
errNotWAFMode = errors.New("is not off, detect or block")
|
||||
errNotParanoiaLevel = errors.New("is not a paranoia level, from 1 to 4")
|
||||
errNotRuleID = errors.New(
|
||||
"is not the id of a Core Rule Set rule, a whole number such as 942100",
|
||||
)
|
||||
"is not the id of a Core Rule Set rule, a whole number such as 942100")
|
||||
errSetupRuleID = errors.New(
|
||||
"is from 900000 to 900999, the ids of the rules that set the Core Rule Set " +
|
||||
"up and of smallwebwaf's own, which cannot be switched off",
|
||||
)
|
||||
"up and of smallwebwaf's own, which cannot be switched off")
|
||||
errNotBoolean = errors.New("is not true or false")
|
||||
errNotLogRemoteURL = errors.New(
|
||||
"is not syslog+udp, syslog+tcp or syslog+tls with a host and a port, " +
|
||||
"and nothing more, such as syslog+tls://logs.example:6514",
|
||||
)
|
||||
"and nothing more, such as syslog+tls://logs.example:6514")
|
||||
errNoCertificate = errors.New("holds no PEM certificate")
|
||||
errNotFacility = errors.New(
|
||||
"is not a syslog facility such as local0 or daemon",
|
||||
)
|
||||
errNotAppName = errors.New(
|
||||
"is not 1 to 48 printable ASCII characters without a space, such as gitea",
|
||||
)
|
||||
errNotFacility = errors.New("is not a syslog facility such as local0 or daemon")
|
||||
errNotAppName = errors.New(
|
||||
"is not 1 to 48 printable ASCII characters without a space, such as gitea")
|
||||
errSetTwice = errors.New("set only one of them")
|
||||
errNotWebhookURL = errors.New(
|
||||
"is not an http or https URL without a user or a fragment, " +
|
||||
@@ -431,50 +421,35 @@ var (
|
||||
"is not a header name followed by : and the header's value, " +
|
||||
"such as Authorization:Bearer <token>")
|
||||
errControlCharacter = errors.New(
|
||||
"holds a control character, such as the carriage return of a Windows line end",
|
||||
)
|
||||
"holds a control character, such as the carriage return of a Windows line end")
|
||||
errNotAlertEvent = errors.New(
|
||||
"is not ban, permanent_ban, waf_block, anomaly, reputation_hit, " +
|
||||
"source_failure or file_error")
|
||||
errNotNumberOrOff = errors.New(
|
||||
"is not a whole number above zero, such as 60, or off",
|
||||
)
|
||||
errNotNumberOrOff = errors.New("is not a whole number above zero, such as 60, or off")
|
||||
errNotUTF8 = errors.New("is not valid UTF-8")
|
||||
errNotASN = errors.New("is not an AS number such as AS64496")
|
||||
errNotPercentItem = errors.New(
|
||||
"is not a code, : and a percentage, such as AS64496:50 or cn:25")
|
||||
errNotPercent = errors.New(
|
||||
"is not a percentage, a whole number from 0 to 100",
|
||||
)
|
||||
errNotPercent = errors.New("is not a percentage, a whole number from 0 to 100")
|
||||
errListedTwice = errors.New("is listed twice")
|
||||
errNotListURL = errors.New(
|
||||
"is not an http or https URL without a user or a fragment, " +
|
||||
"such as https://www.spamhaus.org/drop/drop.txt")
|
||||
errInBlocklistURLs = errors.New("is in SWWAF_BLOCKLIST_URLS too")
|
||||
errNotAnHourOrMore = errors.New(
|
||||
"is not a duration of 1h or more, such as 24h",
|
||||
)
|
||||
errNotAction = errors.New(
|
||||
errNotAnHourOrMore = errors.New("is not a duration of 1h or more, such as 24h")
|
||||
errNotAction = errors.New(
|
||||
"is not deny, limit:<percent> such as limit:25, or log")
|
||||
errNotZone = errors.New(
|
||||
"is not a DNS zone such as dnsbl.dronebl.org",
|
||||
)
|
||||
errZoneTooLong = errors.New(
|
||||
"is longer than 189 characters, too long for the " +
|
||||
"names IPv6 clients are asked about by",
|
||||
)
|
||||
errNotResolver = errors.New(
|
||||
"is not an IP address with an optional port, " +
|
||||
"such as 192.0.2.53 or [2001:db8::53]:5353",
|
||||
)
|
||||
errNotZone = errors.New("is not a DNS zone such as dnsbl.dronebl.org")
|
||||
errZoneTooLong = errors.New("is longer than 189 characters, too long for the " +
|
||||
"names IPv6 clients are asked about by")
|
||||
errNotResolver = errors.New("is not an IP address with an optional port, " +
|
||||
"such as 192.0.2.53 or [2001:db8::53]:5353")
|
||||
errNotLAPIURL = errors.New(
|
||||
"is not an http or https URL without a user or a fragment, " +
|
||||
"such as http://172.17.0.1:8080")
|
||||
errNeedsLAPIKey = errors.New("the engine answers no request without it")
|
||||
errLAPIKeyUnused = errors.New(
|
||||
"it is sent only to the engine at that URL",
|
||||
)
|
||||
errAnotherList = errors.New(
|
||||
errLAPIKeyUnused = errors.New("it is sent only to the engine at that URL")
|
||||
errAnotherList = errors.New(
|
||||
"is in SWWAF_BLOCKLIST_URLS or is SWWAF_ASN_LIMIT_PERCENT_URL too")
|
||||
)
|
||||
|
||||
@@ -485,208 +460,99 @@ var (
|
||||
// setting that is set but invalid is an error that names it.
|
||||
//
|
||||
//nolint:funlen // one line for each setting, a list that grows with them
|
||||
func FromEnvironment(
|
||||
lookupEnv func(string) (string, bool),
|
||||
) (*Config, error) {
|
||||
func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
|
||||
env := &environment{lookupEnv: lookupEnv}
|
||||
cfg := &Config{
|
||||
ListenAddr: env.address("SWWAF_LISTEN_ADDR"),
|
||||
InstanceName: env.instanceName(),
|
||||
Observe: env.observe("SWWAF_MODE", "enforce"),
|
||||
IPv6GroupPrefix: env.ipv6GroupPrefix(
|
||||
"SWWAF_IPV6_GROUP_PREFIX",
|
||||
"64",
|
||||
),
|
||||
MaxTrackedClients: env.numberNotOff(
|
||||
"SWWAF_MAX_TRACKED_CLIENTS",
|
||||
"20000",
|
||||
),
|
||||
AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""),
|
||||
RateLimitExemptNets: env.netblocks(
|
||||
"SWWAF_RATE_LIMIT_EXEMPT_NETS",
|
||||
"",
|
||||
),
|
||||
DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
|
||||
RateLimitPerMinute: env.count(
|
||||
"SWWAF_RATE_LIMIT_PER_MINUTE",
|
||||
"120",
|
||||
),
|
||||
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "1500"),
|
||||
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "6000"),
|
||||
RateLimitExemptPaths: env.pathPrefixes(
|
||||
"SWWAF_RATE_LIMIT_EXEMPT_PATHS",
|
||||
"",
|
||||
),
|
||||
BytesLimitPerMinute: env.size(
|
||||
"SWWAF_BYTES_LIMIT_PER_MINUTE",
|
||||
"off",
|
||||
),
|
||||
BytesLimitPerHour: env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "50G"),
|
||||
BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "100G"),
|
||||
BytesCount: env.bytesCount("SWWAF_BYTES_COUNT", "both"),
|
||||
LookupSource: env.lookupSource(
|
||||
"SWWAF_LOOKUP_SOURCE",
|
||||
"geojs",
|
||||
),
|
||||
LookupDBPath: env.value("SWWAF_LOOKUP_DB_PATH", ""),
|
||||
LookupTimeout: env.durationNotOff(
|
||||
"SWWAF_LOOKUP_TIMEOUT",
|
||||
"800ms",
|
||||
),
|
||||
AddLookupHeaders: env.boolean(
|
||||
"SWWAF_ADD_LOOKUP_HEADERS",
|
||||
"false",
|
||||
),
|
||||
DeniedCountries: env.countries("SWWAF_DENIED_COUNTRIES", ""),
|
||||
ListenAddr: env.address("SWWAF_LISTEN_ADDR"),
|
||||
InstanceName: env.instanceName(),
|
||||
Observe: env.observe("SWWAF_MODE", "enforce"),
|
||||
IPv6GroupPrefix: env.ipv6GroupPrefix("SWWAF_IPV6_GROUP_PREFIX", "64"),
|
||||
MaxTrackedClients: env.numberNotOff("SWWAF_MAX_TRACKED_CLIENTS", "20000"),
|
||||
AllowNets: env.netblocks("SWWAF_ALLOW_NETS", ""),
|
||||
RateLimitExemptNets: env.netblocks("SWWAF_RATE_LIMIT_EXEMPT_NETS", ""),
|
||||
DenyNets: env.netblocks("SWWAF_DENY_NETS", ""),
|
||||
RateLimitPerMinute: env.count("SWWAF_RATE_LIMIT_PER_MINUTE", "120"),
|
||||
RateLimitPerHour: env.count("SWWAF_RATE_LIMIT_PER_HOUR", "1500"),
|
||||
RateLimitPerDay: env.count("SWWAF_RATE_LIMIT_PER_DAY", "6000"),
|
||||
RateLimitExemptPaths: env.pathPrefixes("SWWAF_RATE_LIMIT_EXEMPT_PATHS", ""),
|
||||
BytesLimitPerMinute: env.size("SWWAF_BYTES_LIMIT_PER_MINUTE", "off"),
|
||||
BytesLimitPerHour: env.size("SWWAF_BYTES_LIMIT_PER_HOUR", "50G"),
|
||||
BytesLimitPerDay: env.size("SWWAF_BYTES_LIMIT_PER_DAY", "100G"),
|
||||
BytesCount: env.bytesCount("SWWAF_BYTES_COUNT", "both"),
|
||||
LookupSource: env.lookupSource("SWWAF_LOOKUP_SOURCE", "geojs"),
|
||||
LookupDBPath: env.value("SWWAF_LOOKUP_DB_PATH", ""),
|
||||
LookupTimeout: env.durationNotOff("SWWAF_LOOKUP_TIMEOUT", "800ms"),
|
||||
AddLookupHeaders: env.boolean("SWWAF_ADD_LOOKUP_HEADERS", "false"),
|
||||
DeniedCountries: env.countries("SWWAF_DENIED_COUNTRIES", ""),
|
||||
ExclusivelyAllowedCountries: env.countries(
|
||||
"SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES", ""),
|
||||
ASNLimitPercent: env.percents(
|
||||
"SWWAF_ASN_LIMIT_PERCENT",
|
||||
ParseASN,
|
||||
),
|
||||
CountryLimitPercent: env.percents(
|
||||
"SWWAF_COUNTRY_LIMIT_PERCENT",
|
||||
parseCountry,
|
||||
),
|
||||
ASNBytesPercent: env.percents(
|
||||
"SWWAF_ASN_BYTES_PERCENT",
|
||||
ParseASN,
|
||||
),
|
||||
CountryBytesPercent: env.percents(
|
||||
"SWWAF_COUNTRY_BYTES_PERCENT",
|
||||
parseCountry,
|
||||
),
|
||||
UnknownLimitPercent: env.percent(
|
||||
"SWWAF_UNKNOWN_LIMIT_PERCENT",
|
||||
"100",
|
||||
),
|
||||
ASNLimitPercentURL: env.listURL("SWWAF_ASN_LIMIT_PERCENT_URL"),
|
||||
BlocklistURLs: env.listURLs("SWWAF_BLOCKLIST_URLS"),
|
||||
BlocklistRefresh: env.refresh("SWWAF_BLOCKLIST_REFRESH", "24h"),
|
||||
DNSBLZones: env.zones("SWWAF_DNSBL_ZONES"),
|
||||
DNSBLResolver: env.resolver("SWWAF_DNSBL_RESOLVER"),
|
||||
AbuseIPDBKey: env.secret("SWWAF_ABUSEIPDB_KEY"),
|
||||
AbuseIPDBMinScore: env.percent("SWWAF_ABUSEIPDB_MIN_SCORE", "75"),
|
||||
AbuseIPDBDailyBudget: env.numberNotOff(
|
||||
"SWWAF_ABUSEIPDB_DAILY_BUDGET",
|
||||
"900",
|
||||
),
|
||||
ReputationCacheTTL: env.durationNotOff(
|
||||
"SWWAF_REPUTATION_CACHE_TTL",
|
||||
"24h",
|
||||
),
|
||||
ReputationTimeout: env.durationNotOff(
|
||||
"SWWAF_REPUTATION_TIMEOUT",
|
||||
"2s",
|
||||
),
|
||||
CrowdSecDecisionsURL: env.crowdSecDecisionsURL(
|
||||
"SWWAF_CROWDSEC_LAPI_URL",
|
||||
),
|
||||
CrowdSecKey: env.secret("SWWAF_CROWDSEC_LAPI_KEY"),
|
||||
BanResponse: env.banResponse("SWWAF_BAN_RESPONSE", "403"),
|
||||
LimitBanDuration: env.durationNotOff(
|
||||
"SWWAF_LIMIT_BAN_DURATION",
|
||||
"1h",
|
||||
),
|
||||
LimitBanRepeatWindow: env.durationNotOff(
|
||||
"SWWAF_LIMIT_BAN_REPEAT_WINDOW",
|
||||
"24h",
|
||||
),
|
||||
MaxBanDuration: env.durationNotOff(
|
||||
"SWWAF_MAX_BAN_DURATION",
|
||||
"7d",
|
||||
),
|
||||
AttackBanDuration: env.durationNotOff(
|
||||
"SWWAF_ATTACK_BAN_DURATION",
|
||||
"7d",
|
||||
),
|
||||
MaxBans: env.numberNotOff("SWWAF_MAX_BANS", "5000"),
|
||||
BanScopeV4Prefix: env.v4Prefix(
|
||||
"SWWAF_BAN_SCOPE_V4_PREFIX",
|
||||
"32",
|
||||
),
|
||||
StateDir: env.absolutePath(
|
||||
"SWWAF_STATE_DIR",
|
||||
"/var/lib/smallwebwaf",
|
||||
),
|
||||
StateWriteDelay: env.durationNotOff(
|
||||
"SWWAF_STATE_WRITE_DELAY",
|
||||
"10s",
|
||||
),
|
||||
StateCounterInterval: env.durationNotOff(
|
||||
"SWWAF_STATE_COUNTER_INTERVAL",
|
||||
"15m",
|
||||
),
|
||||
LogRequestHeaders: env.headerNames(
|
||||
"SWWAF_LOG_REQUEST_HEADERS",
|
||||
"accept,accept-language,accept-encoding,content-type,origin,range",
|
||||
),
|
||||
LogLevel: env.logLevel("SWWAF_LOG_LEVEL", "info"),
|
||||
AdminToken: env.token("SWWAF_ADMIN_TOKEN"),
|
||||
MetricsToken: env.token("SWWAF_METRICS_TOKEN"),
|
||||
MetricsTopN: env.numberNotOff("SWWAF_METRICS_TOP_N", "50"),
|
||||
RulesDir: env.value(
|
||||
"SWWAF_RULES_DIR",
|
||||
"/etc/smallwebwaf/rules.d",
|
||||
),
|
||||
RulesEnabled: env.boolean("SWWAF_RULES_ENABLED", "true"),
|
||||
WAFMode: env.wafMode("SWWAF_WAF_MODE", WAFModeBlock),
|
||||
WAFParanoiaLevel: env.paranoiaLevel(
|
||||
"SWWAF_WAF_PARANOIA_LEVEL",
|
||||
"1",
|
||||
),
|
||||
WAFAnomalyThreshold: env.numberOrOff(
|
||||
"SWWAF_WAF_ANOMALY_THRESHOLD",
|
||||
"5",
|
||||
),
|
||||
ASNLimitPercent: env.percents("SWWAF_ASN_LIMIT_PERCENT", ParseASN),
|
||||
CountryLimitPercent: env.percents("SWWAF_COUNTRY_LIMIT_PERCENT", parseCountry),
|
||||
ASNBytesPercent: env.percents("SWWAF_ASN_BYTES_PERCENT", ParseASN),
|
||||
CountryBytesPercent: env.percents("SWWAF_COUNTRY_BYTES_PERCENT", parseCountry),
|
||||
UnknownLimitPercent: env.percent("SWWAF_UNKNOWN_LIMIT_PERCENT", "100"),
|
||||
ASNLimitPercentURL: env.listURL("SWWAF_ASN_LIMIT_PERCENT_URL"),
|
||||
BlocklistURLs: env.listURLs("SWWAF_BLOCKLIST_URLS"),
|
||||
BlocklistRefresh: env.refresh("SWWAF_BLOCKLIST_REFRESH", "24h"),
|
||||
DNSBLZones: env.zones("SWWAF_DNSBL_ZONES"),
|
||||
DNSBLResolver: env.resolver("SWWAF_DNSBL_RESOLVER"),
|
||||
AbuseIPDBKey: env.secret("SWWAF_ABUSEIPDB_KEY"),
|
||||
AbuseIPDBMinScore: env.percent("SWWAF_ABUSEIPDB_MIN_SCORE", "75"),
|
||||
AbuseIPDBDailyBudget: env.numberNotOff("SWWAF_ABUSEIPDB_DAILY_BUDGET", "900"),
|
||||
ReputationCacheTTL: env.durationNotOff("SWWAF_REPUTATION_CACHE_TTL", "24h"),
|
||||
ReputationTimeout: env.durationNotOff("SWWAF_REPUTATION_TIMEOUT", "2s"),
|
||||
CrowdSecDecisionsURL: env.crowdSecDecisionsURL("SWWAF_CROWDSEC_LAPI_URL"),
|
||||
CrowdSecKey: env.secret("SWWAF_CROWDSEC_LAPI_KEY"),
|
||||
BanResponse: env.banResponse("SWWAF_BAN_RESPONSE", "403"),
|
||||
LimitBanDuration: env.durationNotOff("SWWAF_LIMIT_BAN_DURATION", "1h"),
|
||||
LimitBanRepeatWindow: env.durationNotOff("SWWAF_LIMIT_BAN_REPEAT_WINDOW", "24h"),
|
||||
MaxBanDuration: env.durationNotOff("SWWAF_MAX_BAN_DURATION", "7d"),
|
||||
AttackBanDuration: env.durationNotOff("SWWAF_ATTACK_BAN_DURATION", "7d"),
|
||||
MaxBans: env.numberNotOff("SWWAF_MAX_BANS", "5000"),
|
||||
BanScopeV4Prefix: env.v4Prefix("SWWAF_BAN_SCOPE_V4_PREFIX", "32"),
|
||||
StateDir: env.absolutePath("SWWAF_STATE_DIR", "/var/lib/smallwebwaf"),
|
||||
StateWriteDelay: env.durationNotOff("SWWAF_STATE_WRITE_DELAY", "10s"),
|
||||
StateCounterInterval: env.durationNotOff("SWWAF_STATE_COUNTER_INTERVAL", "15m"),
|
||||
LogRequestHeaders: env.headerNames("SWWAF_LOG_REQUEST_HEADERS",
|
||||
"accept,accept-language,accept-encoding,content-type,origin,range"),
|
||||
LogLevel: env.logLevel("SWWAF_LOG_LEVEL", "info"),
|
||||
AdminToken: env.token("SWWAF_ADMIN_TOKEN"),
|
||||
MetricsToken: env.token("SWWAF_METRICS_TOKEN"),
|
||||
MetricsTopN: env.numberNotOff("SWWAF_METRICS_TOP_N", "50"),
|
||||
RulesDir: env.value("SWWAF_RULES_DIR", "/etc/smallwebwaf/rules.d"),
|
||||
RulesEnabled: env.boolean("SWWAF_RULES_ENABLED", "true"),
|
||||
WAFMode: env.wafMode("SWWAF_WAF_MODE", WAFModeBlock),
|
||||
WAFParanoiaLevel: env.paranoiaLevel("SWWAF_WAF_PARANOIA_LEVEL", "1"),
|
||||
WAFAnomalyThreshold: env.numberOrOff("SWWAF_WAF_ANOMALY_THRESHOLD", "5"),
|
||||
WAFDisabledRules: env.ruleIDs("SWWAF_WAF_DISABLED_RULES",
|
||||
"920340,920420,920440,920640,930130,930140"),
|
||||
WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
|
||||
WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
|
||||
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
|
||||
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
|
||||
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
|
||||
LogRemoteTLSCAs: env.certificates(
|
||||
"SWWAF_LOG_REMOTE_TLS_CA_FILE",
|
||||
),
|
||||
LogRemoteBuffer: env.numberNotOff(
|
||||
"SWWAF_LOG_REMOTE_BUFFER",
|
||||
"10000",
|
||||
),
|
||||
LogRemoteFacility: env.facility(
|
||||
"SWWAF_LOG_REMOTE_FACILITY",
|
||||
"local0",
|
||||
),
|
||||
AlertWebhookURL: env.webhookURL("SWWAF_ALERT_WEBHOOK_URL"),
|
||||
AlertWebhookHeaders: env.webhookHeaders(
|
||||
"SWWAF_ALERT_WEBHOOK_HEADERS",
|
||||
),
|
||||
AlertSlackWebhookURL: env.webhookURL(
|
||||
"SWWAF_ALERT_SLACK_WEBHOOK_URL",
|
||||
),
|
||||
AlertNtfyURL: env.webhookURL("SWWAF_ALERT_NTFY_URL"),
|
||||
AlertNtfyToken: env.secret("SWWAF_ALERT_NTFY_TOKEN"),
|
||||
WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
|
||||
WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
|
||||
CoreRuleSetExclusionsFile: env.filePath(
|
||||
"SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"),
|
||||
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
|
||||
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
|
||||
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
|
||||
LogRemoteTLSCAs: env.certificates("SWWAF_LOG_REMOTE_TLS_CA_FILE"),
|
||||
LogRemoteBuffer: env.numberNotOff("SWWAF_LOG_REMOTE_BUFFER", "10000"),
|
||||
LogRemoteFacility: env.facility("SWWAF_LOG_REMOTE_FACILITY", "local0"),
|
||||
AlertWebhookURL: env.webhookURL("SWWAF_ALERT_WEBHOOK_URL"),
|
||||
AlertWebhookHeaders: env.webhookHeaders("SWWAF_ALERT_WEBHOOK_HEADERS"),
|
||||
AlertSlackWebhookURL: env.webhookURL("SWWAF_ALERT_SLACK_WEBHOOK_URL"),
|
||||
AlertNtfyURL: env.webhookURL("SWWAF_ALERT_NTFY_URL"),
|
||||
AlertNtfyToken: env.secret("SWWAF_ALERT_NTFY_TOKEN"),
|
||||
AlertEvents: env.alertEvents("SWWAF_ALERT_EVENTS",
|
||||
strings.Join(alerts.Events(), ",")),
|
||||
AlertCooldown: env.duration("SWWAF_ALERT_COOLDOWN", "15m"),
|
||||
AlertMaxPerHour: env.numberOrOff(
|
||||
"SWWAF_ALERT_MAX_PER_HOUR",
|
||||
"60",
|
||||
),
|
||||
AnomalyClient: env.thresholds("SWWAF_ANOMALY_CLIENT_"),
|
||||
AnomalyNet: env.thresholds("SWWAF_ANOMALY_NET_"),
|
||||
AnomalyASN: env.thresholds("SWWAF_ANOMALY_ASN_"),
|
||||
AnomalyTotal: env.thresholds("SWWAF_ANOMALY_TOTAL_"),
|
||||
AnomalyWatch: env.thresholds("SWWAF_WATCH_"),
|
||||
AnomalyNetV4Prefix: env.v4Prefix(
|
||||
"SWWAF_ANOMALY_NET_V4_PREFIX",
|
||||
"24",
|
||||
),
|
||||
AnomalyNetV6Prefix: env.v6Prefix(
|
||||
"SWWAF_ANOMALY_NET_V6_PREFIX",
|
||||
"48",
|
||||
),
|
||||
WatchNets: env.namedNetblocks("SWWAF_WATCH_NETS"),
|
||||
AlertCooldown: env.duration("SWWAF_ALERT_COOLDOWN", "15m"),
|
||||
AlertMaxPerHour: env.numberOrOff("SWWAF_ALERT_MAX_PER_HOUR", "60"),
|
||||
AnomalyClient: env.thresholds("SWWAF_ANOMALY_CLIENT_"),
|
||||
AnomalyNet: env.thresholds("SWWAF_ANOMALY_NET_"),
|
||||
AnomalyASN: env.thresholds("SWWAF_ANOMALY_ASN_"),
|
||||
AnomalyTotal: env.thresholds("SWWAF_ANOMALY_TOTAL_"),
|
||||
AnomalyWatch: env.thresholds("SWWAF_WATCH_"),
|
||||
AnomalyNetV4Prefix: env.v4Prefix("SWWAF_ANOMALY_NET_V4_PREFIX", "24"),
|
||||
AnomalyNetV6Prefix: env.v6Prefix("SWWAF_ANOMALY_NET_V6_PREFIX", "48"),
|
||||
WatchNets: env.namedNetblocks("SWWAF_WATCH_NETS"),
|
||||
}
|
||||
|
||||
cfg.LogRemoteAppName = env.appName("SWWAF_LOG_REMOTE_APP_NAME",
|
||||
@@ -723,20 +589,14 @@ func Command(lookupEnv func(string) (string, bool)) (string, error) {
|
||||
case "servewaf":
|
||||
return command, nil
|
||||
default:
|
||||
return "", fmt.Errorf(
|
||||
"SMALLWEBWAF_COMMAND: %q %w",
|
||||
command,
|
||||
errNotCommand,
|
||||
)
|
||||
return "", fmt.Errorf("SMALLWEBWAF_COMMAND: %q %w", command, errNotCommand)
|
||||
}
|
||||
}
|
||||
|
||||
// ServeWAFListenAddr reads only SWWAF_LISTEN_ADDR, which may be given as a
|
||||
// file, as FromEnvironment does. The health check in servewaf's container needs
|
||||
// no other setting, so it reads no other, nor a file that another names.
|
||||
func ServeWAFListenAddr(
|
||||
lookupEnv func(string) (string, bool),
|
||||
) (string, error) {
|
||||
func ServeWAFListenAddr(lookupEnv func(string) (string, bool)) (string, error) {
|
||||
env := &environment{lookupEnv: lookupEnv}
|
||||
listenAddr := env.address("SWWAF_LISTEN_ADDR")
|
||||
|
||||
@@ -798,19 +658,14 @@ func (e *environment) lookup(name string) (string, bool) {
|
||||
}
|
||||
|
||||
if set {
|
||||
e.check(
|
||||
name,
|
||||
fmt.Errorf("is set, and so is %s; %w", fileName, errSetTwice),
|
||||
)
|
||||
e.check(name, fmt.Errorf("is set, and so is %s; %w", fileName, errSetTwice))
|
||||
|
||||
return value, set
|
||||
}
|
||||
|
||||
e.settings = append(e.settings, slog.String(fileName, path))
|
||||
|
||||
contents, err := os.ReadFile(
|
||||
path,
|
||||
) //nolint:gosec // a file the admin names
|
||||
contents, err := os.ReadFile(path) //nolint:gosec // a file the admin names
|
||||
if err != nil {
|
||||
e.check(fileName, fmt.Errorf("cannot be read: %w", err))
|
||||
|
||||
@@ -1061,10 +916,7 @@ func (e *environment) zones(name string) []string {
|
||||
logged[i] = MaskZoneKey(zone)
|
||||
}
|
||||
|
||||
e.settings = append(
|
||||
e.settings,
|
||||
slog.String(name, strings.Join(logged, ",")),
|
||||
)
|
||||
e.settings = append(e.settings, slog.String(name, strings.Join(logged, ",")))
|
||||
|
||||
return zones
|
||||
}
|
||||
@@ -1116,15 +968,10 @@ func (e *environment) checkLookupDBPath(cfg *Config) {
|
||||
switch {
|
||||
case cfg.LookupSource == fileSource && cfg.LookupDBPath == "":
|
||||
e.check("SWWAF_LOOKUP_SOURCE", fmt.Errorf(
|
||||
"is file while SWWAF_LOOKUP_DB_PATH is unset; %w",
|
||||
errNeedsDBPath,
|
||||
))
|
||||
"is file while SWWAF_LOOKUP_DB_PATH is unset; %w", errNeedsDBPath))
|
||||
case cfg.LookupSource != fileSource && cfg.LookupDBPath != "":
|
||||
e.check("SWWAF_LOOKUP_DB_PATH", fmt.Errorf(
|
||||
"is set while SWWAF_LOOKUP_SOURCE is %s; %w",
|
||||
cfg.LookupSource,
|
||||
errDBPathUnused,
|
||||
))
|
||||
"is set while SWWAF_LOOKUP_SOURCE is %s; %w", cfg.LookupSource, errDBPathUnused))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1165,11 +1012,8 @@ func (e *environment) checkCountriesAndLookups(cfg *Config) {
|
||||
{"SWWAF_ANOMALY_ASN_BYTES_PER_HOUR", cfg.AnomalyASN.BytesPerHour > 0},
|
||||
} {
|
||||
if setting.set {
|
||||
e.check(
|
||||
setting.name,
|
||||
fmt.Errorf("is set while SWWAF_LOOKUP_SOURCE is off; %w",
|
||||
errNeedsLookups),
|
||||
)
|
||||
e.check(setting.name, fmt.Errorf("is set while SWWAF_LOOKUP_SOURCE is off; %w",
|
||||
errNeedsLookups))
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -1192,21 +1036,14 @@ func (e *environment) checkCrowdSec(cfg *Config) {
|
||||
switch {
|
||||
case decisionsURL != "" && cfg.CrowdSecKey == "":
|
||||
e.check("SWWAF_CROWDSEC_LAPI_URL", fmt.Errorf(
|
||||
"is set while SWWAF_CROWDSEC_LAPI_KEY is unset; %w",
|
||||
errNeedsLAPIKey,
|
||||
))
|
||||
"is set while SWWAF_CROWDSEC_LAPI_KEY is unset; %w", errNeedsLAPIKey))
|
||||
case decisionsURL == "" && cfg.CrowdSecKey != "":
|
||||
e.check("SWWAF_CROWDSEC_LAPI_KEY", fmt.Errorf(
|
||||
"is set while SWWAF_CROWDSEC_LAPI_URL is unset; %w",
|
||||
errLAPIKeyUnused,
|
||||
))
|
||||
"is set while SWWAF_CROWDSEC_LAPI_URL is unset; %w", errLAPIKeyUnused))
|
||||
case decisionsURL != "" && (slices.Contains(cfg.BlocklistURLs, decisionsURL) ||
|
||||
decisionsURL == cfg.ASNLimitPercentURL):
|
||||
e.check(
|
||||
"SWWAF_CROWDSEC_LAPI_URL",
|
||||
fmt.Errorf("gives the decision list %q, which %w",
|
||||
decisionsURL, errAnotherList),
|
||||
)
|
||||
e.check("SWWAF_CROWDSEC_LAPI_URL", fmt.Errorf("gives the decision list %q, which %w",
|
||||
decisionsURL, errAnotherList))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1221,9 +1058,7 @@ func (e *environment) headerNames(name, defaultValue string) []string {
|
||||
|
||||
// durationNotOff reads a setting that is a duration and, unlike a
|
||||
// timeout, cannot be off.
|
||||
func (e *environment) durationNotOff(
|
||||
name, defaultValue string,
|
||||
) time.Duration {
|
||||
func (e *environment) durationNotOff(name, defaultValue string) time.Duration {
|
||||
duration, err := ParseDurationNotOff(e.value(name, defaultValue))
|
||||
e.check(name, err)
|
||||
|
||||
@@ -1359,6 +1194,16 @@ func (e *environment) logRemoteURL(name string) *url.URL {
|
||||
return remote
|
||||
}
|
||||
|
||||
// filePath reads a setting that is the path of a file, "" while it is
|
||||
// unset. Its value names a file already, so, unlike the other settings,
|
||||
// it has no _FILE form. Whoever uses the file reads it.
|
||||
func (e *environment) filePath(name string) string {
|
||||
path, _ := e.lookupEnv(name)
|
||||
e.settings = append(e.settings, slog.String(name, path))
|
||||
|
||||
return path
|
||||
}
|
||||
|
||||
// certificates reads a setting that is the path of a file of PEM
|
||||
// certificates. Unset or empty, it is nil. Its value names a file
|
||||
// already, so, unlike the other settings, it has no _FILE form.
|
||||
@@ -1403,10 +1248,7 @@ func (e *environment) instanceName() string {
|
||||
|
||||
value := e.value("SWWAF_INSTANCE_NAME", hostname)
|
||||
if !utf8.ValidString(value) {
|
||||
e.check(
|
||||
"SWWAF_INSTANCE_NAME",
|
||||
fmt.Errorf("%q %w", value, errNotUTF8),
|
||||
)
|
||||
e.check("SWWAF_INSTANCE_NAME", fmt.Errorf("%q %w", value, errNotUTF8))
|
||||
}
|
||||
|
||||
return value
|
||||
@@ -1415,10 +1257,7 @@ func (e *environment) instanceName() string {
|
||||
// appName reads the setting that is the APP-NAME of the records the log
|
||||
// lines are sent in, by default the instance name. Its value is checked
|
||||
// when it is set, and, while lines are sent, when it is the instance name.
|
||||
func (e *environment) appName(
|
||||
name, instanceName string,
|
||||
sending bool,
|
||||
) string {
|
||||
func (e *environment) appName(name, instanceName string, sending bool) string {
|
||||
value, set := e.lookup(name)
|
||||
if !set {
|
||||
value = instanceName
|
||||
@@ -1431,14 +1270,8 @@ func (e *environment) appName(
|
||||
case set:
|
||||
e.check(name, fmt.Errorf("%q %w", value, errNotAppName))
|
||||
case sending:
|
||||
e.check(
|
||||
name,
|
||||
fmt.Errorf(
|
||||
"is unset, and SWWAF_INSTANCE_NAME %q, its default, %w",
|
||||
value,
|
||||
errNotAppName,
|
||||
),
|
||||
)
|
||||
e.check(name, fmt.Errorf("is unset, and SWWAF_INSTANCE_NAME %q, its default, %w",
|
||||
value, errNotAppName))
|
||||
}
|
||||
|
||||
return value
|
||||
@@ -1447,16 +1280,11 @@ func (e *environment) appName(
|
||||
// checkInstanceNameForNtfy refuses an instance name that holds a control
|
||||
// character while ntfySet, SWWAF_ALERT_NTFY_URL being set: ntfy is sent
|
||||
// the instance name in a header, which cannot hold one.
|
||||
func (e *environment) checkInstanceNameForNtfy(
|
||||
instanceName string,
|
||||
ntfySet bool,
|
||||
) {
|
||||
func (e *environment) checkInstanceNameForNtfy(instanceName string, ntfySet bool) {
|
||||
if ntfySet && strings.ContainsFunc(instanceName, unicode.IsControl) {
|
||||
e.check("SWWAF_INSTANCE_NAME", fmt.Errorf(
|
||||
"%q %w, and is sent to ntfy in a header while SWWAF_ALERT_NTFY_URL is set",
|
||||
instanceName,
|
||||
errControlCharacter,
|
||||
))
|
||||
instanceName, errControlCharacter))
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1776,10 +1604,7 @@ func parseNamedNetblocks(value string) ([]anomaly.NamedNetblock, error) {
|
||||
return nil, fmt.Errorf("%q %w", name, errListedTwice)
|
||||
}
|
||||
|
||||
named = append(
|
||||
named,
|
||||
anomaly.NamedNetblock{Name: name, Netblock: netblock},
|
||||
)
|
||||
named = append(named, anomaly.NamedNetblock{Name: name, Netblock: netblock})
|
||||
}
|
||||
|
||||
return named, nil
|
||||
@@ -2018,11 +1843,8 @@ func parseHeaderNames(value string) ([]string, error) {
|
||||
header := strings.ToLower(item)
|
||||
switch header {
|
||||
case "host":
|
||||
return nil, fmt.Errorf(
|
||||
"%q %w; the request's host is the field host",
|
||||
item,
|
||||
errHeaderTakenOut,
|
||||
)
|
||||
return nil, fmt.Errorf("%q %w; the request's host is the field host",
|
||||
item, errHeaderTakenOut)
|
||||
case "transfer-encoding":
|
||||
return nil, fmt.Errorf("%q %w", item, errHeaderTakenOut)
|
||||
}
|
||||
@@ -2059,11 +1881,9 @@ func parseUpstreamURL(value string) (*url.URL, error) {
|
||||
}
|
||||
|
||||
onlySchemeAndHost := (upstream.Scheme == "http" || upstream.Scheme == "https") &&
|
||||
upstream.Hostname() != "" && upstream.User == nil &&
|
||||
upstream.Opaque == "" &&
|
||||
upstream.Hostname() != "" && upstream.User == nil && upstream.Opaque == "" &&
|
||||
(upstream.Path == "" || upstream.Path == "/") &&
|
||||
upstream.RawQuery == "" &&
|
||||
upstream.Fragment == ""
|
||||
upstream.RawQuery == "" && upstream.Fragment == ""
|
||||
if !onlySchemeAndHost {
|
||||
return nil, fmt.Errorf("%q %w", value, errNotUpstreamURL)
|
||||
}
|
||||
@@ -2087,11 +1907,7 @@ func parseLogRemoteURL(value string) (*url.URL, error) {
|
||||
return nil, fmt.Errorf("%q %w", value, errNotLogRemoteURL)
|
||||
}
|
||||
|
||||
schemes := []string{
|
||||
remotelog.SchemeUDP,
|
||||
remotelog.SchemeTCP,
|
||||
remotelog.SchemeTLS,
|
||||
}
|
||||
schemes := []string{remotelog.SchemeUDP, remotelog.SchemeTCP, remotelog.SchemeTLS}
|
||||
port, err := strconv.ParseUint(remote.Port(), 10, 16)
|
||||
|
||||
onlySchemeHostAndPort := slices.Contains(schemes, remote.Scheme) &&
|
||||
@@ -2154,9 +1970,7 @@ func isHTTPURL(u *url.URL) bool {
|
||||
|
||||
return (u.Scheme == "http" || u.Scheme == "https") && u.Hostname() != "" &&
|
||||
(u.Port() == "" || (err == nil && port != 0)) &&
|
||||
u.User == nil &&
|
||||
u.Opaque == "" &&
|
||||
u.Fragment == ""
|
||||
u.User == nil && u.Opaque == "" && u.Fragment == ""
|
||||
}
|
||||
|
||||
// isListURL reports whether value is a URL a list can be fetched from, as
|
||||
@@ -2223,12 +2037,9 @@ func parseZones(value string) ([]string, error) {
|
||||
|
||||
for i, zone := range zones {
|
||||
shown := MaskZoneKey(zone)
|
||||
listedBefore := slices.ContainsFunc(
|
||||
zones[:i],
|
||||
func(earlier string) bool {
|
||||
return strings.EqualFold(MaskZoneKey(earlier), shown)
|
||||
},
|
||||
)
|
||||
listedBefore := slices.ContainsFunc(zones[:i], func(earlier string) bool {
|
||||
return strings.EqualFold(MaskZoneKey(earlier), shown)
|
||||
})
|
||||
|
||||
switch {
|
||||
case len(zone) > maxZoneLength:
|
||||
@@ -2315,13 +2126,8 @@ func parseWebhookHeaders(value string) (http.Header, string, error) {
|
||||
name, headerValue, found := strings.Cut(item, ":")
|
||||
name = strings.TrimSpace(name)
|
||||
|
||||
if !found || !IsHeaderName(name) ||
|
||||
strings.ContainsAny(headerValue, "\r\n\x00") {
|
||||
return nil, "", fmt.Errorf(
|
||||
"item %d %w",
|
||||
i+1,
|
||||
errNotWebhookHeader,
|
||||
)
|
||||
if !found || !IsHeaderName(name) || strings.ContainsAny(headerValue, "\r\n\x00") {
|
||||
return nil, "", fmt.Errorf("item %d %w", i+1, errNotWebhookHeader)
|
||||
}
|
||||
|
||||
headers.Add(name, strings.TrimSpace(headerValue))
|
||||
|
||||
@@ -87,6 +87,7 @@ const (
|
||||
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
|
||||
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
|
||||
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
|
||||
exclusionsFile = "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE"
|
||||
trapPaths = "SWWAF_TRAP_PATHS"
|
||||
errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
|
||||
logRemoteURL = "SWWAF_LOG_REMOTE_URL"
|
||||
@@ -539,13 +540,14 @@ func TestCoreRuleSetSettings(t *testing.T) {
|
||||
environment{
|
||||
wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10",
|
||||
wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/",
|
||||
wafBodyLimit: "32K",
|
||||
wafBodyLimit: "32K", exclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
|
||||
},
|
||||
config.Config{
|
||||
WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10,
|
||||
WAFDisabledRules: []int{942100, 920350},
|
||||
WAFExemptPaths: []string{"/api/", "/static/"},
|
||||
WAFBodyLimit: 32 << 10,
|
||||
WAFDisabledRules: []int{942100, 920350},
|
||||
WAFExemptPaths: []string{"/api/", "/static/"},
|
||||
WAFBodyLimit: 32 << 10,
|
||||
CoreRuleSetExclusionsFile: "/etc/smallwebwaf/crs-exclusions.conf",
|
||||
},
|
||||
},
|
||||
{
|
||||
@@ -565,7 +567,8 @@ func TestCoreRuleSetSettings(t *testing.T) {
|
||||
WAFMode: cfg.WAFMode, WAFParanoiaLevel: cfg.WAFParanoiaLevel,
|
||||
WAFAnomalyThreshold: cfg.WAFAnomalyThreshold,
|
||||
WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths,
|
||||
WAFBodyLimit: cfg.WAFBodyLimit,
|
||||
WAFBodyLimit: cfg.WAFBodyLimit,
|
||||
CoreRuleSetExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
||||
}
|
||||
if !reflect.DeepEqual(got, tc.want) {
|
||||
t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want)
|
||||
@@ -2327,6 +2330,7 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
|
||||
wafDisabledRules: defaultWAFDisabledRules,
|
||||
wafExemptPaths: "",
|
||||
wafBodyLimit: off,
|
||||
exclusionsFile: "",
|
||||
trapPaths: "",
|
||||
errorBurstThreshold: "30",
|
||||
logRemoteURL: "",
|
||||
|
||||
+6
-28
@@ -16,11 +16,6 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/alerts"
|
||||
)
|
||||
|
||||
// FIXME: you can't call things just "File" - that's super vague.
|
||||
|
||||
// FIXME: "lookup database" - in memory on disk? you must not be so
|
||||
// ambiguous when describing things.
|
||||
|
||||
// quietTime is how long the lookup database must go without a change
|
||||
// before it is read again, so that a file still being copied in is read
|
||||
// only once whole.
|
||||
@@ -138,11 +133,8 @@ func (f *File) Watch(ctx context.Context) {
|
||||
}
|
||||
|
||||
if err != nil {
|
||||
f.params.ProcessLog.Error(
|
||||
"cannot watch the lookup database for replacements",
|
||||
"error",
|
||||
err.Error(),
|
||||
)
|
||||
f.params.ProcessLog.Error("cannot watch the lookup database for replacements",
|
||||
"error", err.Error())
|
||||
|
||||
return
|
||||
}
|
||||
@@ -202,10 +194,7 @@ func (f *File) readAgain() {
|
||||
f.params.Alerts.Raise(alerts.Alert{
|
||||
Event: alerts.EventFileError,
|
||||
Reason: kept,
|
||||
Detail: map[string]any{
|
||||
"file": f.params.Path,
|
||||
"error": err.Error(),
|
||||
},
|
||||
Detail: map[string]any{"file": f.params.Path, "error": err.Error()},
|
||||
})
|
||||
f.params.ProcessLog.Error(kept, "error", err.Error())
|
||||
|
||||
@@ -223,11 +212,7 @@ func (f *File) use(reader *maxminddb.Reader) {
|
||||
f.lastRead = f.params.Now()
|
||||
f.mu.Unlock()
|
||||
|
||||
f.params.ProcessLog.Info(
|
||||
"read the lookup database",
|
||||
"file",
|
||||
f.params.Path,
|
||||
)
|
||||
f.params.ProcessLog.Info("read the lookup database", "file", f.params.Path)
|
||||
}
|
||||
|
||||
// read reads the lookup database at path. The whole file is read into
|
||||
@@ -236,19 +221,12 @@ func (f *File) use(reader *maxminddb.Reader) {
|
||||
func read(path string) (*maxminddb.Reader, error) {
|
||||
data, err := os.ReadFile(path) //nolint:gosec // the file the admin names
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf(
|
||||
"SWWAF_LOOKUP_DB_PATH cannot be read: %w",
|
||||
err,
|
||||
)
|
||||
return nil, fmt.Errorf("SWWAF_LOOKUP_DB_PATH cannot be read: %w", err)
|
||||
}
|
||||
|
||||
reader, err := maxminddb.OpenBytes(data)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf(
|
||||
"SWWAF_LOOKUP_DB_PATH %s is not a .mmdb file: %w",
|
||||
path,
|
||||
err,
|
||||
)
|
||||
return nil, fmt.Errorf("SWWAF_LOOKUP_DB_PATH %s is not a .mmdb file: %w", path, err)
|
||||
}
|
||||
|
||||
return reader, nil
|
||||
|
||||
@@ -19,51 +19,29 @@ const (
|
||||
// ActionForward is a request servewaf allowed, which rproxy passes to the
|
||||
// app.
|
||||
ActionForward = "forward"
|
||||
|
||||
//FIXME: "too_large" doesn't tell a reader what that means.
|
||||
//request_header_too_large does. you must not use ambiguous terms.
|
||||
|
||||
// ActionTooLarge is a request over the header limits.
|
||||
ActionTooLarge = "too_large"
|
||||
|
||||
// ActionRateLimited is a request refused because it took its client
|
||||
// over a rate limit, which bans the client.
|
||||
|
||||
//FIXME: better would be "client_is_rate_limited"
|
||||
ActionRateLimited = "rate_limited"
|
||||
|
||||
// ActionBanned is a request refused because a ban covers its client,
|
||||
// or because it matched a ban rule, asked for a trap path or the
|
||||
// CrowdSec decision list lists its client, each of which bans the
|
||||
// client.
|
||||
|
||||
// FIXME: if I saw this, i would assume they are permanently banned. is that
|
||||
// true? if not, there should be TemporarilyBanned and
|
||||
// PermanentlyBanned
|
||||
ActionBanned = "banned"
|
||||
|
||||
// ActionRuleBlocked is a request refused because it matched a block
|
||||
// rule.
|
||||
|
||||
// FIXME: "blocked_by_block_rule"
|
||||
ActionRuleBlocked = "rule_blocked"
|
||||
|
||||
// ActionWAFBlocked is a request refused because the Core Rule Set
|
||||
// scored it at or over SWWAF_WAF_ANOMALY_THRESHOLD.
|
||||
ActionWAFBlocked = "waf_blocked"
|
||||
|
||||
// ActionDenied is a request refused because its client is in
|
||||
// SWWAF_DENY_NETS, in a blocklist while SWWAF_BLOCKLIST_ACTION is deny,
|
||||
// or listed by a DNSBL zone, or scored a hit by AbuseIPDB, while
|
||||
// SWWAF_REPUTATION_ACTION is deny.
|
||||
ActionDenied = "denied"
|
||||
|
||||
// ActionCountryDenied is a request refused for its client's country.
|
||||
ActionCountryDenied = "country_denied"
|
||||
|
||||
// FIXME: you have country_denied and denied. why isn't it
|
||||
// client_denied? ALWAYS BE CONSISTENT AND UNAMBIGUOUS
|
||||
|
||||
// ActionAdmin is a request servewaf answered at one of its own endpoints,
|
||||
// or a question about a path under /_smallwebwaf/.
|
||||
ActionAdmin = "admin"
|
||||
@@ -267,11 +245,7 @@ func FormatTime(t time.Time) string {
|
||||
|
||||
// Milliseconds is d in milliseconds, to the microsecond.
|
||||
func Milliseconds(d time.Duration) float64 {
|
||||
return float64(
|
||||
d.Microseconds(),
|
||||
) / float64(
|
||||
time.Millisecond/time.Microsecond,
|
||||
)
|
||||
return float64(d.Microseconds()) / float64(time.Millisecond/time.Microsecond)
|
||||
}
|
||||
|
||||
// NewProcessLogger returns the logger for the process's own messages:
|
||||
@@ -279,25 +253,17 @@ func Milliseconds(d time.Duration) float64 {
|
||||
// as a request line's, and instanceName, SWWAF_INSTANCE_NAME, as instance.
|
||||
// It writes only the messages at level, SWWAF_LOG_LEVEL, or more severe;
|
||||
// the request lines Write writes are never held back.
|
||||
func NewProcessLogger(
|
||||
w io.Writer,
|
||||
instanceName string,
|
||||
level slog.Level,
|
||||
) *slog.Logger {
|
||||
func NewProcessLogger(w io.Writer, instanceName string, level slog.Level) *slog.Logger {
|
||||
handler := slog.NewJSONHandler(w, &slog.HandlerOptions{
|
||||
Level: level,
|
||||
ReplaceAttr: func(groups []string, attr slog.Attr) slog.Attr {
|
||||
if attr.Key == slog.TimeKey && len(groups) == 0 {
|
||||
return slog.String(
|
||||
slog.TimeKey,
|
||||
FormatTime(attr.Value.Time()),
|
||||
)
|
||||
return slog.String(slog.TimeKey, FormatTime(attr.Value.Time()))
|
||||
}
|
||||
|
||||
return attr
|
||||
},
|
||||
})
|
||||
|
||||
return slog.New(handler).
|
||||
With("type", "process", "instance", instanceName)
|
||||
return slog.New(handler).With("type", "process", "instance", instanceName)
|
||||
}
|
||||
|
||||
+17
-13
@@ -4,6 +4,7 @@
|
||||
package servewaf
|
||||
|
||||
import (
|
||||
"fmt"
|
||||
"io"
|
||||
"log/slog"
|
||||
"net/http"
|
||||
@@ -60,6 +61,9 @@ type Params struct {
|
||||
// LookupFile is the lookup database they are looked up in while
|
||||
// SWWAF_LOOKUP_SOURCE is file, and nil otherwise.
|
||||
LookupFile *lookup.File
|
||||
// CoreRuleSet is what NewCoreRuleSet returns: what inspects requests
|
||||
// while SWWAF_WAF_MODE is not off, and nil while it is.
|
||||
CoreRuleSet *waf.CoreRuleSet
|
||||
// Now tells the time by which requests are counted for the rate
|
||||
// limits, bans are made and run out, and GeoJS's answers are kept,
|
||||
// normally time.Now in UTC, the time the state files give.
|
||||
@@ -144,7 +148,7 @@ func newHandler(params Params) *handler {
|
||||
dnsbl: dnsbl,
|
||||
abuseIPDB: abuseIPDB,
|
||||
rules: params.Rules,
|
||||
coreRuleSet: newCoreRuleSet(params.Config),
|
||||
coreRuleSet: params.CoreRuleSet,
|
||||
alerts: params.Alerts,
|
||||
}
|
||||
h.geojs = lookup.New(lookup.Params{
|
||||
@@ -202,29 +206,29 @@ func newReputation(
|
||||
return lists, dnsbl, abuseIPDB
|
||||
}
|
||||
|
||||
// newCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL,
|
||||
// NewCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL,
|
||||
// without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies
|
||||
// up to SWWAF_WAF_BODY_LIMIT, or nil while SWWAF_WAF_MODE is off. It reads at
|
||||
// most QuestionBodyMaxBytes less one byte: a question's body of
|
||||
// up to SWWAF_WAF_BODY_LIMIT, with the exclusions in
|
||||
// SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, or nil while SWWAF_WAF_MODE is off. It
|
||||
// reads at most QuestionBodyMaxBytes less one byte: a question's body of
|
||||
// QuestionBodyMaxBytes may be cut short, and the Core Rule Set does not read a
|
||||
// JSON or XML body over its limit.
|
||||
func newCoreRuleSet(cfg *config.Config) *waf.CoreRuleSet {
|
||||
// JSON or XML body over its limit. An exclusions file that cannot be read, does
|
||||
// not parse or names an id of smallwebwaf's own rules is an error.
|
||||
func NewCoreRuleSet(cfg *config.Config) (*waf.CoreRuleSet, error) {
|
||||
if cfg.WAFMode == config.WAFModeOff {
|
||||
return nil
|
||||
return nil, nil //nolint:nilnil // off is no Core Rule Set, and no error
|
||||
}
|
||||
|
||||
coreRuleSet, err := waf.New(waf.Params{
|
||||
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
|
||||
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
|
||||
BodyLimit: min(cfg.WAFBodyLimit, QuestionBodyMaxBytes-1),
|
||||
ExclusionsFile: cfg.CoreRuleSetExclusionsFile,
|
||||
})
|
||||
if err != nil {
|
||||
// The Core Rule Set is built in, and the settings cannot break it:
|
||||
// the paranoia level is from 1 to 4, and the id of no rule switches
|
||||
// nothing off.
|
||||
panic(err)
|
||||
return nil, fmt.Errorf("SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: %w", err)
|
||||
}
|
||||
|
||||
return coreRuleSet
|
||||
return coreRuleSet, nil
|
||||
}
|
||||
|
||||
// handler is servewaf's handler. It holds what every request shares; what
|
||||
|
||||
@@ -463,6 +463,11 @@ func newServeWAFWith(
|
||||
}
|
||||
}
|
||||
|
||||
coreRuleSet, err := servewaf.NewCoreRuleSet(cfg)
|
||||
if err != nil {
|
||||
t.Fatalf("Core Rule Set: %v", err)
|
||||
}
|
||||
|
||||
server := build(servewaf.Params{
|
||||
Config: cfg,
|
||||
RequestLog: out,
|
||||
@@ -470,6 +475,7 @@ func newServeWAFWith(
|
||||
GeoJSURL: geojsURL,
|
||||
AbuseIPDBURL: abuseIPDBURL,
|
||||
LookupFile: lookupFile,
|
||||
CoreRuleSet: coreRuleSet,
|
||||
Now: now,
|
||||
Rules: ruleFiles,
|
||||
Alerts: alertQueue,
|
||||
|
||||
@@ -29,6 +29,7 @@ import (
|
||||
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||
"sneak.berlin/go/smallwebwaf/internal/servewaf"
|
||||
"sneak.berlin/go/smallwebwaf/internal/state"
|
||||
"sneak.berlin/go/smallwebwaf/internal/waf"
|
||||
)
|
||||
|
||||
// shutdownTimeout is how long requests in progress may take to finish
|
||||
@@ -150,7 +151,14 @@ func RunServeWAF(ctx context.Context, params Params) int {
|
||||
return 1
|
||||
}
|
||||
|
||||
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue)
|
||||
crs, err := servewaf.NewCoreRuleSet(cfg)
|
||||
if err != nil {
|
||||
processLog.Error("cannot use the Core Rule Set", "error", err.Error())
|
||||
|
||||
return 1
|
||||
}
|
||||
|
||||
server, err := newServer(cfg, stdout, processLog, now, ruleFiles, alertQueue, crs)
|
||||
if err != nil {
|
||||
processLog.Error("cannot use the lookup database", "error", err.Error())
|
||||
|
||||
@@ -170,8 +178,7 @@ func RunServeWAF(ctx context.Context, params Params) int {
|
||||
|
||||
listener, err := (&net.ListenConfig{}).Listen(ctx, "tcp", cfg.ListenAddr)
|
||||
if err != nil {
|
||||
processLog.Error("cannot listen on SWWAF_LISTEN_ADDR",
|
||||
"error", err.Error())
|
||||
processLog.Error("cannot listen on SWWAF_LISTEN_ADDR", "error", err.Error())
|
||||
|
||||
return 1
|
||||
}
|
||||
@@ -190,6 +197,7 @@ func RunServeWAF(ctx context.Context, params Params) int {
|
||||
func newServer(
|
||||
cfg *config.Config, stdout io.Writer, processLog *slog.Logger,
|
||||
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
|
||||
crs *waf.CoreRuleSet,
|
||||
) (*servewaf.Server, error) {
|
||||
var lookupFile *lookup.File
|
||||
|
||||
@@ -214,6 +222,7 @@ func newServer(
|
||||
GeoJSURL: lookup.URL,
|
||||
AbuseIPDBURL: reputation.AbuseIPDBURL,
|
||||
LookupFile: lookupFile,
|
||||
CoreRuleSet: crs,
|
||||
Now: now,
|
||||
Rules: ruleFiles,
|
||||
Alerts: alertQueue,
|
||||
|
||||
@@ -551,6 +551,29 @@ func TestRulesDirThatDoesNotExistStopsTheStart(t *testing.T) {
|
||||
": no such file or directory")
|
||||
}
|
||||
|
||||
func TestCoreRuleSetExclusionsFileThatCannotBeReadStopsTheStart(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
path := filepath.Join(t.TempDir(), "crs-exclusions.conf")
|
||||
out := &output{}
|
||||
|
||||
status := run(t.Context(), map[string]string{
|
||||
listenAddr: localhost + ":0", stateDir: t.TempDir(), rulesDir: t.TempDir(),
|
||||
"SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE": path,
|
||||
}, out)
|
||||
if status != 1 {
|
||||
t.Fatalf("exit status %d, want 1", status)
|
||||
}
|
||||
|
||||
line := out.line(t, "msg", "cannot use the Core Rule Set")
|
||||
want := "SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE: read the exclusions file: open " +
|
||||
path + ": no such file or directory"
|
||||
|
||||
if line["error"] != want {
|
||||
t.Errorf("start refused with %v, want the error %q", line, want)
|
||||
}
|
||||
}
|
||||
|
||||
func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
+288
@@ -0,0 +1,288 @@
|
||||
# Ordinary gitea requests, one path and query per line, that the Core Rule Set
|
||||
# must let through with deploy/gitea/crs-exclusions.conf. The tests of
|
||||
# internal/waf inspect each, and script/gitea-setup sends each to gitea through
|
||||
# the setup.
|
||||
|
||||
# Searches with ordinary words and symbols.
|
||||
/owner/repo/search?q=package.json
|
||||
/owner/repo/search?q=.gitignore
|
||||
/owner/repo/search?q=docker-compose.yml
|
||||
/owner/repo/search?q=python3
|
||||
/owner/repo/search?q=ssh%20key
|
||||
/owner/repo/search?q=feat%2Fdocker-support
|
||||
/owner/repo/search?q=ls%20-la
|
||||
/owner/repo/search?q=sed%20-i
|
||||
/owner/repo/search?q=%2Fbin%2Fsh
|
||||
/owner/repo/search?q=%60make%20check%60
|
||||
/owner/repo/search?q=%60whoami%60
|
||||
/owner/repo/search?q=fetch%28
|
||||
/owner/repo/search?q=%24%7BVAR%7D
|
||||
/owner/repo/search?q=%24%28pwd%29
|
||||
/owner/repo/search?q=process.env
|
||||
/owner/repo/search?q=%3Cimg%20src%3D
|
||||
/owner/repo/search?q=%3Cscript%3E
|
||||
/owner/repo/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/owner/repo/search?q=union%20select
|
||||
/owner/repo/search?q=http%3A%2F%2F127.0.0.1%3A8080
|
||||
/owner/repo/search?q=..%2F
|
||||
/owner/repo/search?q=%2Fetc%2Fpasswd
|
||||
/owner/repo/search?q=eval%28
|
||||
/owner/repo/search?q=chmod%20%2Bx
|
||||
/owner/repo/search?q=echo%20%24PATH
|
||||
/owner/repo/search?q=java.lang.Runtime
|
||||
/owner/repo/search?q=document.cookie
|
||||
/owner/repo/search?q=alert%281%29
|
||||
/owner/repo/search?q=__proto__
|
||||
/owner/repo/search?q=C%3A%5CWindows%5CSystem32
|
||||
/owner/repo/search?q=Invoke-WebRequest
|
||||
/owner/repo/search?q=if%20%28a%20%3D%3D%20b%29%20%7B
|
||||
/owner/repo/search?q=sudo%20apt%20install
|
||||
/owner/repo/search?q=~%2F.ssh%2Fconfig
|
||||
/owner/repo/search?q=.htaccess
|
||||
/explore/repos?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/explore/repos?q=%3Cimg%20src%3D
|
||||
/explore/code?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/explore/code?q=%3Cimg%20src%3D
|
||||
/owner/repo/issues?type=all&state=open&q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/owner/repo/issues?type=all&state=open&q=%3Cimg%20src%3D
|
||||
/owner/repo/pulls?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/owner/repo/pulls?q=%3Cimg%20src%3D
|
||||
/issues?type=your_repositories&q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/issues?type=your_repositories&q=%3Cimg%20src%3D
|
||||
/owner/repo/commits/branch/main/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/owner/repo/commits/branch/main/search?q=%3Cimg%20src%3D
|
||||
/owner/repo/wiki/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/owner/repo/wiki/search?q=%3Cimg%20src%3D
|
||||
/owner/repo/branches?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/owner/repo/branches?q=%3Cimg%20src%3D
|
||||
/owner/repo/releases?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/owner/repo/releases?q=%3Cimg%20src%3D
|
||||
/owner/-/packages?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/owner/-/packages?q=%3Cimg%20src%3D
|
||||
/user/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/user/search?q=%3Cimg%20src%3D
|
||||
/api/v1/repos/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/api/v1/repos/search?q=%3Cimg%20src%3D
|
||||
/api/v1/repos/issues/search?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/api/v1/repos/issues/search?q=%3Cimg%20src%3D
|
||||
/api/v1/repos/owner/repo/issues?q=SELECT%20%2A%20FROM%20users%20WHERE
|
||||
/api/v1/repos/owner/repo/issues?q=%3Cimg%20src%3D
|
||||
|
||||
# Pages, raw files and edit pages of files, and the API's call for their
|
||||
# contents.
|
||||
/owner/repo/src/branch/main/package.json
|
||||
/owner/repo/raw/branch/main/package.json
|
||||
/owner/repo/_edit/main/package.json
|
||||
/api/v1/repos/owner/repo/contents/package.json
|
||||
/owner/repo/src/branch/main/.gitignore
|
||||
/owner/repo/raw/branch/main/.gitignore
|
||||
/owner/repo/_edit/main/.gitignore
|
||||
/api/v1/repos/owner/repo/contents/.gitignore
|
||||
/owner/repo/src/branch/main/docker-compose.yml
|
||||
/owner/repo/raw/branch/main/docker-compose.yml
|
||||
/owner/repo/_edit/main/docker-compose.yml
|
||||
/api/v1/repos/owner/repo/contents/docker-compose.yml
|
||||
/owner/repo/src/branch/main/Dockerfile
|
||||
/owner/repo/raw/branch/main/Dockerfile
|
||||
/owner/repo/_edit/main/Dockerfile
|
||||
/api/v1/repos/owner/repo/contents/Dockerfile
|
||||
/owner/repo/src/branch/main/.htaccess
|
||||
/owner/repo/raw/branch/main/.htaccess
|
||||
/owner/repo/_edit/main/.htaccess
|
||||
/api/v1/repos/owner/repo/contents/.htaccess
|
||||
/owner/repo/src/branch/main/rootfs/etc/passwd
|
||||
/owner/repo/raw/branch/main/rootfs/etc/passwd
|
||||
/owner/repo/_edit/main/rootfs/etc/passwd
|
||||
/api/v1/repos/owner/repo/contents/rootfs/etc/passwd
|
||||
/owner/repo/src/branch/main/bin/docker-entrypoint
|
||||
/owner/repo/raw/branch/main/bin/docker-entrypoint
|
||||
/owner/repo/_edit/main/bin/docker-entrypoint
|
||||
/api/v1/repos/owner/repo/contents/bin/docker-entrypoint
|
||||
/owner/repo/src/branch/main/scripts/python3-setup.sh
|
||||
/owner/repo/raw/branch/main/scripts/python3-setup.sh
|
||||
/owner/repo/_edit/main/scripts/python3-setup.sh
|
||||
/api/v1/repos/owner/repo/contents/scripts/python3-setup.sh
|
||||
/owner/repo/src/branch/main/src/java.base/share/classes/java/lang/Runtime.java
|
||||
/owner/repo/raw/branch/main/src/java.base/share/classes/java/lang/Runtime.java
|
||||
/owner/repo/_edit/main/src/java.base/share/classes/java/lang/Runtime.java
|
||||
/api/v1/repos/owner/repo/contents/src/java.base/share/classes/java/lang/Runtime.java
|
||||
/owner/repo/src/branch/main/java.lang.String.md
|
||||
/owner/repo/raw/branch/main/java.lang.String.md
|
||||
/owner/repo/_edit/main/java.lang.String.md
|
||||
/api/v1/repos/owner/repo/contents/java.lang.String.md
|
||||
/owner/repo/src/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
|
||||
/owner/repo/raw/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
|
||||
/owner/repo/_edit/main/META-INF/services/org.apache.commons.logging.LogFactory
|
||||
/api/v1/repos/owner/repo/contents/META-INF/services/org.apache.commons.logging.LogFactory
|
||||
/owner/repo/src/branch/main/process.env.d.ts
|
||||
/owner/repo/raw/branch/main/process.env.d.ts
|
||||
/owner/repo/_edit/main/process.env.d.ts
|
||||
/api/v1/repos/owner/repo/contents/process.env.d.ts
|
||||
/owner/repo/src/branch/main/document.write.js
|
||||
/owner/repo/raw/branch/main/document.write.js
|
||||
/owner/repo/_edit/main/document.write.js
|
||||
/api/v1/repos/owner/repo/contents/document.write.js
|
||||
/owner/repo/src/branch/main/test/fixtures/__proto__.json
|
||||
/owner/repo/raw/branch/main/test/fixtures/__proto__.json
|
||||
/owner/repo/_edit/main/test/fixtures/__proto__.json
|
||||
/api/v1/repos/owner/repo/contents/test/fixtures/__proto__.json
|
||||
/owner/repo/src/branch/main/install-script.sh
|
||||
/owner/repo/raw/branch/main/install-script.sh
|
||||
/owner/repo/_edit/main/install-script.sh
|
||||
/api/v1/repos/owner/repo/contents/install-script.sh
|
||||
/owner/repo/src/branch/main/src/backend/catalog/information_schema.sql
|
||||
/owner/repo/raw/branch/main/src/backend/catalog/information_schema.sql
|
||||
/owner/repo/_edit/main/src/backend/catalog/information_schema.sql
|
||||
/api/v1/repos/owner/repo/contents/src/backend/catalog/information_schema.sql
|
||||
/owner/repo/src/branch/main/docs/a%20b.xhtml
|
||||
/owner/repo/raw/branch/main/docs/a%20b.xhtml
|
||||
/owner/repo/_edit/main/docs/a%20b.xhtml
|
||||
/api/v1/repos/owner/repo/contents/docs/a%20b.xhtml
|
||||
/owner/repo/src/branch/main/%5Bdraft%5D.md
|
||||
/owner/repo/raw/branch/main/%5Bdraft%5D.md
|
||||
/owner/repo/_edit/main/%5Bdraft%5D.md
|
||||
/api/v1/repos/owner/repo/contents/%5Bdraft%5D.md
|
||||
|
||||
# File names in the query string: the commit graph of a file, the API's lists
|
||||
# of its commits and of its contents, an issue opened from a template, and the
|
||||
# sign-in page on the way to its edit page.
|
||||
/owner/repo/graph?file=package.json
|
||||
/api/v1/repos/owner/repo/commits?path=package.json
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22package.json%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fpackage.json
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fpackage.json
|
||||
/owner/repo/graph?file=.gitignore
|
||||
/api/v1/repos/owner/repo/commits?path=.gitignore
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.gitignore%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.gitignore
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.gitignore
|
||||
/owner/repo/graph?file=docker-compose.yml
|
||||
/api/v1/repos/owner/repo/commits?path=docker-compose.yml
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docker-compose.yml%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocker-compose.yml
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocker-compose.yml
|
||||
/owner/repo/graph?file=.htaccess
|
||||
/api/v1/repos/owner/repo/commits?path=.htaccess
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22.htaccess%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F.htaccess
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F.htaccess
|
||||
/owner/repo/graph?file=rootfs%2Fetc%2Fpasswd
|
||||
/api/v1/repos/owner/repo/commits?path=rootfs%2Fetc%2Fpasswd
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22rootfs%2Fetc%2Fpasswd%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Frootfs%2Fetc%2Fpasswd
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Frootfs%2Fetc%2Fpasswd
|
||||
/owner/repo/graph?file=bin%2Fdocker-entrypoint
|
||||
/api/v1/repos/owner/repo/commits?path=bin%2Fdocker-entrypoint
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22bin%2Fdocker-entrypoint%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fbin%2Fdocker-entrypoint
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fbin%2Fdocker-entrypoint
|
||||
/owner/repo/graph?file=scripts%2Fpython3-setup.sh
|
||||
/api/v1/repos/owner/repo/commits?path=scripts%2Fpython3-setup.sh
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22scripts%2Fpython3-setup.sh%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fscripts%2Fpython3-setup.sh
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fscripts%2Fpython3-setup.sh
|
||||
/owner/repo/graph?file=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
||||
/api/v1/repos/owner/repo/commits?path=src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fjava.base%2Fshare%2Fclasses%2Fjava%2Flang%2FRuntime.java
|
||||
/owner/repo/graph?file=java.lang.String.md
|
||||
/api/v1/repos/owner/repo/commits?path=java.lang.String.md
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22java.lang.String.md%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fjava.lang.String.md
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fjava.lang.String.md
|
||||
/owner/repo/graph?file=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
||||
/api/v1/repos/owner/repo/commits?path=META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22META-INF%2Fservices%2Forg.apache.commons.logging.LogFactory%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2FMETA-INF%2Fservices%2Forg.apache.commons.logging.LogFactory
|
||||
/owner/repo/graph?file=process.env.d.ts
|
||||
/api/v1/repos/owner/repo/commits?path=process.env.d.ts
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22process.env.d.ts%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fprocess.env.d.ts
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fprocess.env.d.ts
|
||||
/owner/repo/graph?file=document.write.js
|
||||
/api/v1/repos/owner/repo/commits?path=document.write.js
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22document.write.js%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocument.write.js
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocument.write.js
|
||||
/owner/repo/graph?file=test%2Ffixtures%2F__proto__.json
|
||||
/api/v1/repos/owner/repo/commits?path=test%2Ffixtures%2F__proto__.json
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22test%2Ffixtures%2F__proto__.json%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Ftest%2Ffixtures%2F__proto__.json
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Ftest%2Ffixtures%2F__proto__.json
|
||||
/owner/repo/graph?file=install-script.sh
|
||||
/api/v1/repos/owner/repo/commits?path=install-script.sh
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22install-script.sh%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Finstall-script.sh
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Finstall-script.sh
|
||||
/owner/repo/graph?file=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
||||
/api/v1/repos/owner/repo/commits?path=src%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22src%2Fbackend%2Fcatalog%2Finformation_schema.sql%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fsrc%2Fbackend%2Fcatalog%2Finformation_schema.sql
|
||||
/owner/repo/graph?file=docs%2Fa%20b.xhtml
|
||||
/api/v1/repos/owner/repo/commits?path=docs%2Fa%20b.xhtml
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22docs%2Fa%20b.xhtml%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2Fdocs%2Fa%20b.xhtml
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2Fdocs%2Fa%20b.xhtml
|
||||
/owner/repo/graph?file=%5Bdraft%5D.md
|
||||
/api/v1/repos/owner/repo/commits?path=%5Bdraft%5D.md
|
||||
/api/v1/repos/owner/repo/file-contents?body=%7B%22files%22%3A%5B%22%5Bdraft%5D.md%22%5D%7D
|
||||
/owner/repo/issues/new?template=.gitea%2FISSUE_TEMPLATE%2F%5Bdraft%5D.md
|
||||
/user/login?redirect_to=%2Fowner%2Frepo%2F_edit%2Fmain%2F%5Bdraft%5D.md
|
||||
|
||||
# Branch and tag names in the query string: deleting a branch, starting a
|
||||
# release from a tag, a branch protection rule, a directory listing's last
|
||||
# commits, and a container image push that reuses a layer.
|
||||
/owner/repo/branches?name=docker-build
|
||||
/owner/repo/releases/new?tag=docker-build
|
||||
/owner/repo/settings/branches/edit?rule_name=docker-build
|
||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fdocker-build
|
||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fdocker-build
|
||||
/owner/repo/branches?name=feat%2Fdocker-support
|
||||
/owner/repo/releases/new?tag=feat%2Fdocker-support
|
||||
/owner/repo/settings/branches/edit?rule_name=feat%2Fdocker-support
|
||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Ffeat%2Fdocker-support
|
||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Ffeat%2Fdocker-support
|
||||
/owner/repo/branches?name=renovate%2Fdocker-build-push-action-6.x
|
||||
/owner/repo/releases/new?tag=renovate%2Fdocker-build-push-action-6.x
|
||||
/owner/repo/settings/branches/edit?rule_name=renovate%2Fdocker-build-push-action-6.x
|
||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Frenovate%2Fdocker-build-push-action-6.x
|
||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Frenovate%2Fdocker-build-push-action-6.x
|
||||
/owner/repo/branches?name=python3-migration
|
||||
/owner/repo/releases/new?tag=python3-migration
|
||||
/owner/repo/settings/branches/edit?rule_name=python3-migration
|
||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fpython3-migration
|
||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fpython3-migration
|
||||
/owner/repo/branches?name=ansible-roles
|
||||
/owner/repo/releases/new?tag=ansible-roles
|
||||
/owner/repo/settings/branches/edit?rule_name=ansible-roles
|
||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fansible-roles
|
||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fansible-roles
|
||||
/owner/repo/branches?name=whoami
|
||||
/owner/repo/releases/new?tag=whoami
|
||||
/owner/repo/settings/branches/edit?rule_name=whoami
|
||||
/owner/repo/lastcommit/main/src?refSubUrl=branch%2Fwhoami
|
||||
/v2/owner/app/blobs/uploads/?mount=sha256%3A0000&from=owner%2Fwhoami
|
||||
|
||||
# Pages a signed-in user opens.
|
||||
/
|
||||
/owner/repo
|
||||
/owner/repo/issues
|
||||
/owner/repo/pulls
|
||||
/owner/repo/releases
|
||||
/owner/repo/wiki
|
||||
/owner/repo/activity
|
||||
/owner/repo/branches
|
||||
/owner/repo/tags
|
||||
/owner/repo/settings
|
||||
/owner/repo/actions
|
||||
/owner/repo/compare/main...main
|
||||
/owner/repo/commits/branch/main
|
||||
/owner/repo/archive/main.zip
|
||||
/notifications
|
||||
/user/settings
|
||||
/user/settings/applications
|
||||
/explore/repos
|
||||
/-/admin
|
||||
+51
-12
@@ -1,8 +1,9 @@
|
||||
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
||||
// method, the URL with its query and the headers of a request, and on its
|
||||
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
|
||||
// makes to it, as "Attack detection" under "Settings" in SPEC.md describes
|
||||
// them. It reads no response.
|
||||
// body while SWWAF_WAF_BODY_LIMIT is set, with the seven changes smallwebwaf
|
||||
// makes to it and the exclusions in SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE, as
|
||||
// "Attack detection" under "Settings" in SPEC.md describes them. It reads no
|
||||
// response.
|
||||
//
|
||||
// smallwebwaf writes only to its state directory, so Coraza is built with
|
||||
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
||||
@@ -11,10 +12,13 @@
|
||||
package waf
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"fmt"
|
||||
"io"
|
||||
"net/http"
|
||||
"net/netip"
|
||||
"os"
|
||||
"regexp"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -29,9 +33,9 @@ import (
|
||||
// paranoia level for %d, and bodyDirectives for %s while
|
||||
// SWWAF_WAF_BODY_LIMIT is set. Each rule smallwebwaf adds has an id from
|
||||
// 900000 to 900999, the ids the Core Rule Set keeps for the rules that set
|
||||
// it up, which SWWAF_WAF_DISABLED_RULES refuses, so that no setting
|
||||
// switches one off. Coraza joins a line ending in \ to the next, without
|
||||
// the spaces at the start of the next.
|
||||
// it up, which SWWAF_WAF_DISABLED_RULES and the exclusions file refuse, so
|
||||
// that no setting switches one off. Coraza joins a line ending in \ to the
|
||||
// next, without the spaces at the start of the next.
|
||||
const directives = `
|
||||
# The engine only detects. smallwebwaf compares the request's anomaly
|
||||
# score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
|
||||
@@ -100,6 +104,13 @@ SecRuleUpdateTargetById 932260 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
|
||||
# Inspect.
|
||||
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
|
||||
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
|
||||
|
||||
# The seventh: a container image push that reuses a layer names the image
|
||||
# it takes the layer from in the query parameter from, which is not
|
||||
# checked for command names. 932260 refused the push of a layer from
|
||||
# owner/python3-app.
|
||||
SecRule REQUEST_FILENAME "@rx ^/v2/.+/blobs/uploads/$" \
|
||||
"id:900500,phase:1,pass,nolog,ctl:ruleRemoveTargetById=932260;ARGS:from"
|
||||
`
|
||||
|
||||
// bodyDirectives have the Core Rule Set read the part of a request body
|
||||
@@ -160,8 +171,20 @@ type Params struct {
|
||||
// BodyLimit is the most of a request body the Core Rule Set reads
|
||||
// (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
|
||||
BodyLimit int64
|
||||
// ExclusionsFile is the path of a file of exclusions loaded after the
|
||||
// rest (SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE), "" while there is none.
|
||||
ExclusionsFile string
|
||||
}
|
||||
|
||||
// smallwebwafRuleID matches an id from 900000 to 900999, which an
|
||||
// exclusions file may not name: those are the ids of smallwebwaf's own
|
||||
// rules.
|
||||
var smallwebwafRuleID = regexp.MustCompile(`\b900\d{3}\b`)
|
||||
|
||||
// errSmallwebwafRuleID is the error of an exclusions file that names an
|
||||
// id of smallwebwaf's own rules.
|
||||
var errSmallwebwafRuleID = errors.New("an id of smallwebwaf's own rules")
|
||||
|
||||
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
|
||||
// for concurrent use.
|
||||
type CoreRuleSet struct {
|
||||
@@ -169,9 +192,11 @@ type CoreRuleSet struct {
|
||||
bodyLimit int64
|
||||
}
|
||||
|
||||
// New returns the Core Rule Set with the six changes, at params'
|
||||
// paranoia level, without the rules it switches off, and reading request
|
||||
// bodies up to params' limit.
|
||||
// New returns the Core Rule Set with the seven changes, at params'
|
||||
// paranoia level, without the rules it switches off, reading request
|
||||
// bodies up to params' limit, and with params' exclusions file loaded
|
||||
// last. An exclusions file that cannot be read, does not parse or names
|
||||
// an id of smallwebwaf's own rules is an error.
|
||||
func New(params Params) (*CoreRuleSet, error) {
|
||||
body := ""
|
||||
if params.BodyLimit > 0 {
|
||||
@@ -189,9 +214,23 @@ func New(params Params) (*CoreRuleSet, error) {
|
||||
text += "SecRuleRemoveById " + strings.Join(ids, " ") + "\n"
|
||||
}
|
||||
|
||||
waf, err := coraza.NewWAF(coraza.NewWAFConfig().
|
||||
WithRootFS(coreruleset.FS).
|
||||
WithDirectives(text))
|
||||
config := coraza.NewWAFConfig().WithRootFS(coreruleset.FS).WithDirectives(text)
|
||||
|
||||
if params.ExclusionsFile != "" {
|
||||
exclusions, err := os.ReadFile(params.ExclusionsFile)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("read the exclusions file: %w", err)
|
||||
}
|
||||
|
||||
if id := smallwebwafRuleID.Find(exclusions); id != nil {
|
||||
return nil, fmt.Errorf("the exclusions file names %s: %w", id,
|
||||
errSmallwebwafRuleID)
|
||||
}
|
||||
|
||||
config = config.WithDirectives(string(exclusions))
|
||||
}
|
||||
|
||||
waf, err := coraza.NewWAF(config)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("load the Core Rule Set: %w", err)
|
||||
}
|
||||
|
||||
+122
-1
@@ -5,6 +5,7 @@ import (
|
||||
"net/http/httptest"
|
||||
"net/netip"
|
||||
"net/url"
|
||||
"os"
|
||||
"path/filepath"
|
||||
"reflect"
|
||||
"strconv"
|
||||
@@ -119,7 +120,7 @@ func atDefaults(t *testing.T) *waf.CoreRuleSet {
|
||||
}
|
||||
|
||||
// wantChange checks that crs lets through passes, a gitea request one of
|
||||
// the six changes is for, and still finds result in refused, a request
|
||||
// the seven changes is for, and still finds result in refused, a request
|
||||
// like it that the change is not for.
|
||||
func wantChange(
|
||||
t *testing.T, crs *waf.CoreRuleSet, passes, refused request, result waf.Result,
|
||||
@@ -362,6 +363,126 @@ func TestEachDisabledRuleIsSwitchedOff(t *testing.T) {
|
||||
wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{})
|
||||
}
|
||||
|
||||
func TestImagePushThatReusesALayerIsNotCheckedForCommandNames(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
crs := atDefaults(t)
|
||||
reuse := "?mount=sha256:0000&from=owner/python3-app"
|
||||
|
||||
wantChange(t, crs, get("/v2/owner/app/blobs/uploads/"+reuse), get("/upload/"+reuse),
|
||||
matched(932260))
|
||||
|
||||
// A command after the image's name is still refused.
|
||||
wantResult(t, crs,
|
||||
get("/v2/owner/app/blobs/uploads/?from=owner/app;cat%20/etc/passwd"),
|
||||
matched(930120, 932160))
|
||||
}
|
||||
|
||||
// withExclusions returns the Core Rule Set at the defaults, with the
|
||||
// exclusions file at path.
|
||||
func withExclusions(path string) (*waf.CoreRuleSet, error) {
|
||||
return waf.New(waf.Params{
|
||||
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, ExclusionsFile: path,
|
||||
})
|
||||
}
|
||||
|
||||
// exclusionsFile writes text to a file of exclusions and returns its path.
|
||||
func exclusionsFile(t *testing.T, text string) string {
|
||||
t.Helper()
|
||||
|
||||
path := filepath.Join(t.TempDir(), "exclusions.conf")
|
||||
|
||||
err := os.WriteFile(path, []byte(text), 0o600)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
return path
|
||||
}
|
||||
|
||||
func TestExclusionsFileIsLoadedAfterTheCoreRuleSet(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
crs, err := withExclusions(exclusionsFile(t,
|
||||
"SecRuleUpdateTargetById 942100 \"!ARGS:id\"\n"))
|
||||
if err != nil {
|
||||
t.Fatalf("load the exclusions: %v", err)
|
||||
}
|
||||
|
||||
wantChange(t, crs, get("/?id=1'%20OR%20'1'='1"), get("/?q=1'%20OR%20'1'='1"),
|
||||
matched(942100))
|
||||
}
|
||||
|
||||
func TestExclusionsFileThatCannotBeUsedIsAnError(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, path := range []string{
|
||||
filepath.Join(t.TempDir(), "missing.conf"),
|
||||
exclusionsFile(t, "SecNoSuchDirective On\n"),
|
||||
// smallwebwaf's own rules may not be switched off or changed.
|
||||
exclusionsFile(t, "SecRuleRemoveById 900300\n"),
|
||||
exclusionsFile(t, "SecRuleUpdateTargetById 900500 \"!ARGS:x\"\n"),
|
||||
} {
|
||||
_, err := withExclusions(path)
|
||||
if err == nil {
|
||||
t.Errorf("%s loaded", path)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// giteaRequests returns the targets of the recorded ordinary gitea requests
|
||||
// in testdata, which script/gitea-setup sends too.
|
||||
func giteaRequests(t *testing.T) []string {
|
||||
t.Helper()
|
||||
|
||||
text, err := os.ReadFile(filepath.Join("testdata", "gitea-requests.txt"))
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
|
||||
var targets []string
|
||||
|
||||
for line := range strings.Lines(string(text)) {
|
||||
line = strings.TrimSpace(line)
|
||||
if line != "" && !strings.HasPrefix(line, "#") {
|
||||
targets = append(targets, line)
|
||||
}
|
||||
}
|
||||
|
||||
return targets
|
||||
}
|
||||
|
||||
func TestGiteaExclusionsLetOrdinaryGiteaRequestsThroughAndRefuseAttacks(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
crs, err := withExclusions(filepath.Join("..", "..", "deploy", "gitea",
|
||||
"crs-exclusions.conf"))
|
||||
if err != nil {
|
||||
t.Fatalf("load gitea's exclusions: %v", err)
|
||||
}
|
||||
|
||||
const threshold = 5 // SWWAF_WAF_ANOMALY_THRESHOLD's default
|
||||
|
||||
// Each with the Referer of a page that names the same, as a page
|
||||
// opened from it has.
|
||||
for _, target := range giteaRequests(t) {
|
||||
r := get(target, "Referer: http://git.example"+target)
|
||||
if got := inspect(t, crs, r); got.Score >= threshold {
|
||||
t.Errorf("%s refused: %+v", target, got)
|
||||
}
|
||||
}
|
||||
|
||||
for _, attack := range []string{
|
||||
"/owner/repo?id=1'%20OR%20'1'='1",
|
||||
"/owner/repo?id=..%2Fcustom%2Fconf%2Fapp.ini",
|
||||
"/owner/repo?id=%3Cscript%3Ealert(1)%3C%2Fscript%3E",
|
||||
} {
|
||||
if got := inspect(t, crs, get(attack)); got.Score < threshold {
|
||||
t.Errorf("%s let through: %+v", attack, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies.
|
||||
const bodyLimit = 8 << 10
|
||||
|
||||
|
||||
@@ -4,7 +4,8 @@
|
||||
# default but these: SWRP_UPSTREAM_URL names gitea's container, SWRP_WAF_BACKEND
|
||||
# names servewaf's, and SWWAF_LOG_REQUEST_HEADERS adds the header that labels a
|
||||
# request. Then send through rproxy the gitea requests that reading gitea's
|
||||
# source found the Core Rule Set refuses, which docs/gitea.md records, and a
|
||||
# source found the Core Rule Set refuses without the exclusions in
|
||||
# deploy/gitea/crs-exclusions.conf, and a
|
||||
# short sample of gitea's ordinary use: signing in, pages of files and commits,
|
||||
# a diff, the API, a push, clone and fetch over HTTP, and two container image
|
||||
# pushes. Last, print each request servewaf logged, with what it did with it and
|
||||
|
||||
@@ -5,6 +5,11 @@
|
||||
# rproxy, and that servewaf checks them;
|
||||
# - that gitea logs the visitor's address, not rproxy's;
|
||||
# - that a request from a container on the shared network skips the checks;
|
||||
# - that the Core Rule Set refuses none of the ordinary gitea requests in
|
||||
# internal/waf/testdata/gitea-requests.txt, nor a fetch, nor API calls with
|
||||
# JSON bodies;
|
||||
# - that it refuses SQL injection, path traversal and script injection in a
|
||||
# query string;
|
||||
# - that with servewaf stopped, gitea still answers through rproxy.
|
||||
# It publishes no port of traefik's, since this host's ports 80 and 443 may be
|
||||
# taken. It reaches traefik at its address on the shared network instead, so
|
||||
@@ -136,6 +141,65 @@ passed_through() {
|
||||
"\"client_ip\":\"$gateway\"" '"action":"forward"' '"counts":' '"waf_score":'
|
||||
}
|
||||
|
||||
# refused: print each request servewaf logged as refused, for any reason.
|
||||
refused() {
|
||||
compose logs --no-color --no-log-prefix servewaf 2>&1 |
|
||||
grep -F '"servewaf_line":"report"' | grep -v -F '"action":"forward"' || true
|
||||
}
|
||||
|
||||
# ordinary_requests: sign in as the owner, save a file and open an issue
|
||||
# through the API with JSON bodies, then send, signed in, each request in
|
||||
# internal/waf/testdata/gitea-requests.txt, with that request's own address as
|
||||
# its Referer, as a page opened from it has. One a second keeps them under the
|
||||
# default limit of 120 requests a minute, since a request refused for a limit is
|
||||
# never inspected. Fail if servewaf refuses any of them.
|
||||
ordinary_requests() {
|
||||
send /user/login --cookie-jar "$WORK/cookies" >/dev/null
|
||||
status="$(send /user/login --cookie "$WORK/cookies" --cookie-jar "$WORK/cookies" \
|
||||
--header "Origin: https://$SITE" --data-urlencode user_name=owner \
|
||||
--data-urlencode "password=$PASSWORD")"
|
||||
[ "$status" = 303 ] || fail "signing in was answered $status"
|
||||
|
||||
content="$(printf 'declare const env: Record<string, string>;\n' | base64)"
|
||||
status="$(send /api/v1/repos/owner/repo/contents/process.env.d.ts \
|
||||
--user "owner:$PASSWORD" --header "Content-Type: application/json" \
|
||||
--data "{\"message\":\"Add process.env.d.ts\",\"content\":\"$content\"}")"
|
||||
[ "$status" = 201 ] || fail "saving a file through the API was answered $status"
|
||||
status="$(send /api/v1/repos/owner/repo/issues --user "owner:$PASSWORD" \
|
||||
--header "Content-Type: application/json" \
|
||||
--data '{"title":"ls -la fails","body":"SELECT * FROM users; <img src=x>"}')"
|
||||
[ "$status" = 201 ] || fail "opening an issue through the API was answered $status"
|
||||
|
||||
grep -v -e '^#' -e '^$' "$ROOT/internal/waf/testdata/gitea-requests.txt" |
|
||||
while IFS= read -r target; do
|
||||
send "$target" --globoff --cookie "$WORK/cookies" \
|
||||
--header "Referer: https://$SITE$target" >/dev/null
|
||||
sleep 1
|
||||
done
|
||||
|
||||
# The last request's report is logged once its answer has ended.
|
||||
send /api/v1/settings/api >/dev/null
|
||||
wait_for "servewaf logged no report of the last ordinary request" logged servewaf \
|
||||
'"servewaf_line":"report"' '"path":"/api/v1/settings/api"'
|
||||
refusals="$(refused)"
|
||||
[ -z "$refusals" ] || fail "servewaf refused ordinary requests:
|
||||
$refusals"
|
||||
}
|
||||
|
||||
# attacks: send SQL injection, path traversal and script injection in a query
|
||||
# string, and fail unless the Core Rule Set refuses each.
|
||||
attacks() {
|
||||
# The traversal climbs one directory: the default rule file refuses two or
|
||||
# more first, before the Core Rule Set runs.
|
||||
for query in "id=1'%20OR%20'1'%3D'1" "id=..%2Fcustom%2Fconf%2Fapp.ini" \
|
||||
"id=%3Cscript%3Ealert(1)%3C%2Fscript%3E"; do
|
||||
status="$(send "/owner/repo?$query")"
|
||||
[ "$status" = 403 ] || fail "the attack ?$query was answered $status"
|
||||
wait_for "servewaf logged no refusal of ?$query" logged servewaf \
|
||||
'"servewaf_line":"report"' '"action":"waf_blocked"' "\"query\":\"$query\""
|
||||
done
|
||||
}
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
trap cleanup EXIT
|
||||
@@ -204,6 +268,12 @@ main() {
|
||||
esac
|
||||
echo "gitea-setup: a request from the shared network skips the checks"
|
||||
|
||||
site_git -C "$WORK/clone" fetch --quiet || fail "the fetch failed"
|
||||
ordinary_requests
|
||||
echo "gitea-setup: the Core Rule Set refuses no ordinary gitea request"
|
||||
attacks
|
||||
echo "gitea-setup: the Core Rule Set refuses the attacks"
|
||||
|
||||
compose stop servewaf >/dev/null 2>&1 || fail "servewaf did not stop"
|
||||
status="$(send /owner/repo)"
|
||||
[ "$status" = 200 ] ||
|
||||
|
||||
@@ -1,51 +0,0 @@
|
||||
#!/bin/sh
|
||||
# script/publish: build the image on the Docker host and push it to the
|
||||
# registry under four tags: the UTC date (YYYY-MM-DD), the short commit
|
||||
# hash, the branch name and latest. The build is script/docker's, so the
|
||||
# tests and the linter run first. The host and the registry name are
|
||||
# environment variables with defaults, for running it against another.
|
||||
#
|
||||
# The working tree must be clean: the hash tag names a commit, and an
|
||||
# image built from uncommitted changes would carry a hash that does not
|
||||
# describe it.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||
|
||||
: "${DOCKER_HOST:=ssh://root@fsn1app1}"
|
||||
: "${PUBLISH_IMAGE:=git.eeqj.de/sneak/smallwebwaf}"
|
||||
export DOCKER_HOST
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
|
||||
if [ -n "$(git status --porcelain)" ]; then
|
||||
echo "script/publish: the working tree is not clean; commit or stash first" >&2
|
||||
exit 1
|
||||
fi
|
||||
|
||||
date="$(date -u +%Y-%m-%d)"
|
||||
hash="$(git rev-parse --short HEAD)"
|
||||
branch="$(git rev-parse --abbrev-ref HEAD)"
|
||||
if [ "$branch" = "HEAD" ]; then
|
||||
echo "script/publish: detached HEAD; check out a branch first" >&2
|
||||
exit 1
|
||||
fi
|
||||
# A tag is [A-Za-z0-9_.-]; a branch name such as feature/x is not.
|
||||
branch="$(printf '%s' "$branch" | tr -c 'A-Za-z0-9_.-' '-')"
|
||||
|
||||
local_tag="$("$SCRIPT_DIR/projectname")"
|
||||
"$SCRIPT_DIR/docker"
|
||||
|
||||
for tag in "$date" "$hash" "$branch" latest; do
|
||||
docker tag "$local_tag" "$PUBLISH_IMAGE:$tag"
|
||||
done
|
||||
for tag in "$date" "$hash" "$branch" latest; do
|
||||
docker push "$PUBLISH_IMAGE:$tag"
|
||||
done
|
||||
|
||||
echo "pushed $PUBLISH_IMAGE with tags: $date $hash $branch latest"
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Reference in New Issue
Block a user