sneak/smallwebwaf
1
0
Fork 0
Code Issues 58 Pull Requests 3 Actions 7 Packages Projects Releases Wiki Activity

Compare commits

base: sneak/smallwebwaf:issue-30-gitea-requests
Branches Tags
sneak/smallwebwaf:main sneak/smallwebwaf:issue-30-gitea-requests sneak/smallwebwaf:next sneak/smallwebwaf:issue-130-client-timeout-after-early-answer
..
compare: sneak/smallwebwaf:df8c0ebb2918be71ddf6ba14a3b37d112a135a67
Branches Tags
sneak/smallwebwaf:issue-30-gitea-requests sneak/smallwebwaf:next sneak/smallwebwaf:issue-130-client-timeout-after-early-answer sneak/smallwebwaf:main
1 Commits
issue-30-gitea-requests .. df8c0ebb29
Author SHA1 Message Date
clawbot df8c0ebb29 The Core Rule Set, run by Coraza, on each request's method, URL and headers (closes #25)
check / check (push) Canceled after 0s
Details 
Coraza v3.8.1 runs the Core Rule Set 4.25.0 (coraza-coreruleset v4.25.0)
after the rule files, with the six changes and the default
SWWAF_WAF_DISABLED_RULES that SPEC.md gives; no body, no response.
SWWAF_WAF_MODE, SWWAF_WAF_PARANOIA_LEVEL, SWWAF_WAF_ANOMALY_THRESHOLD and
SWWAF_WAF_EXEMPT_PATHS as specified; SWWAF_WAF_DISABLED_RULES refuses
900000 to 900999, smallwebwaf's own rules among them. A request with more
query parameters than Coraza reads, 1000, adds 5 (rule 900300). In block
mode a match is refused with 403, an offence counted toward the error
burst; in detect mode it is let through. Both log waf_rule_ids, waf_score
and duration_waf, raise waf_block, and count
smallwebwaf_waf_matches_total.

Judgement call: waf_block is raised in block mode too.
Deviation: no engine-error path; with no body read, Coraza cannot fail.

Model: opus-5-5
 2026-10-08 06:57:17 +00:00
21 changed files with 156 additions and 1501 deletions
Expand all files Collapse all files
Dockerfile
+5 -9
View File
@@ -36,12 +36,11 @@ RUN go mod tidy -diff || \
{ echo "go.mod or go.sum is not tidy: run make tidy" >&2; exit 1; }
# Go's build cache is kept on a tmpfs, out of the image: nothing uses it
# after this step, and writing it into the image takes seconds. The tests
# are built with the no_fs_access tag, as the binary is in the build stage.
# after this step, and writing it into the image takes seconds.
RUN --mount=type=tmpfs,target=/root/.cache/go-build \
go test -tags no_fs_access -timeout 90s -race -cover ./... || \
go test -timeout 90s -race -cover ./... || \
{ echo "--- Rerunning with -v for details ---"; \
go test -tags no_fs_access -timeout 90s -race -v ./...; exit 1; }
go test -timeout 90s -race -v ./...; exit 1; }
# Tidy stage: `go mod tidy` in the test phase's Go, so that the files it
# writes pass the test phase's check. Nothing else depends on it, so only
@@ -85,10 +84,7 @@ COPY . .
# The VERSION build arg when one is given, otherwise
# `git describe --tags --always` on the .git in the build context. With
# .git present, a version that is still empty, dev or unknown fails the
# build: git is missing or could not read the checkout. The no_fs_access
# tag keeps Coraza from writing the files of a multipart body to the
# system's temporary directory, since smallwebwaf writes only to its state
# directory.
# build: git is missing or could not read the checkout.
ARG VERSION
RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
if [ -e .git ]; then \
@@ -97,7 +93,7 @@ RUN VERSION="${VERSION:-$(git describe --tags --always)}"; \
exit 1 ;; \
esac; \
fi; \
CGO_ENABLED=0 go build -tags no_fs_access -trimpath \
CGO_ENABLED=0 go build -trimpath \
-ldflags="-s -w -X main.Version=${VERSION}" \
-o /usr/local/bin/smallwebwaf ./cmd/smallwebwaf
Makefile
+2 -6
View File
@@ -1,12 +1,11 @@
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
run example-app gitea-requests
run example-app
# Makefile targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
# of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
# which test checks. build and run are for working on the code by hand;
# example-app checks the image with an app built on it; gitea-requests
# reports the gitea requests the Core Rule Set refuses at the defaults.
# example-app checks the image with an app built on it.
bootstrap:
@script/bootstrap
@@ -46,6 +45,3 @@ run:
example-app:
@script/example-app
gitea-requests:
@script/gitea-requests
README.md
+56 -115
View File
@@ -64,10 +64,11 @@ cannot read, serves Prometheus metrics to a scraper that holds the metrics
token, lets an admin who holds the admin token list, add and lift bans and ask
what it knows of a client, and in `observe` mode passes on the requests it would
refuse, logging what it would have done with them. It comes as the image the
app's own image is built on. The Core Rule Set reads a request's body too once
`SWWAF_WAF_BODY_LIMIT` is set. The rest of the design comes next, in the order
of the build order in [`SPEC.md`](SPEC.md). The survey of existing tools that
led to the design is in [`EVALUATION.md`](EVALUATION.md).
app's own image is built on. The Core Rule Set reads no request body yet:
`SWWAF_WAF_BODY_LIMIT`, which switches that on, comes with
https://git.eeqj.de/sneak/smallwebwaf/issues/116. The rest of the design comes
after that, in the order of the build order in [`SPEC.md`](SPEC.md). The survey
of existing tools that led to the design is in [`EVALUATION.md`](EVALUATION.md).
## Getting started
@@ -97,9 +98,7 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
- Passes each request to the app and the app's answer back unchanged: method,
path, query, headers, body and status. Bodies stream through in both
directions and are never held whole in memory, but for the part of a request
body the Core Rule Set reads, at most `SWWAF_WAF_BODY_LIMIT` and one byte
more, which is held until the app is sent it. A WebSocket, or any other
directions and are never held whole in memory. A WebSocket, or any other
upgraded connection, passes through, and the timeouts do not cut it.
- Works out the client's address. A TCP peer outside `SWWAF_TRUSTED_PROXIES` is
the client, and the forwarded headers it sends are replaced, not passed on.
@@ -197,41 +196,33 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
rate limits and before the rule files, which it does not need. A request for
one is refused with `SWWAF_BAN_RESPONSE` and bans the client's netblock for a
clear sign of attack, as a `ban` rule's match does. A trap path is matched
against the path a `path` rule sees: without the query, as the client sent it
and once percent-decoded, the whole of it, character for character.
`/wp-login.php` matches `/wp-login.php?redirect_to=x` and `/%77p-login.php`,
but not `/wp-login.php/`, `/WP-LOGIN.PHP`, `/blog/wp-login.php` or
`/%2577p-login.php`. The ban's notes give the path as the client sent it. A
client in `SWWAF_ALLOW_NETS` is not checked.
against the path a `path` rule sees: as the client sent it, before any
decoding and without the query, the whole of it, character for character.
`/wp-login.php` matches `/wp-login.php?redirect_to=x`, but not
`/wp-login.php/`, `/WP-LOGIN.PHP`, `/blog/wp-login.php` or `/%77p-login.php`.
A client in `SWWAF_ALLOW_NETS` is not checked.
- Inspects each request with the OWASP Core Rule Set 4.25.0, run by Coraza,
after the rule files, unless `SWWAF_WAF_MODE` is `off`: its method, its URL
with the query, and its headers, and, once `SWWAF_WAF_BODY_LIMIT` is set, its
body when it is form data, multipart, JSON or XML, as that setting below
describes; no response is inspected. Each of its rules that matches adds to
the request's anomaly score, up to the paranoia level
`SWWAF_WAF_PARANOIA_LEVEL` sets. A request with more than 1000 query
parameters, or more than 1000 fields in a form data or JSON body it reads,
adds 5 (rule 900300), as a rule rated critical does, since Coraza reads only
the first 1000. A score at or over `SWWAF_WAF_ANOMALY_THRESHOLD`, 5 by
default, is a match: in `block` mode, the default, the request is refused with
`403`, and in `detect` mode it goes on to the app. Either way its log line
names the rules and the score (see `waf_rule_ids` and `waf_score` in "Request
log" below), and it raises a `waf_block` alert. A refusal bans no one by
itself, since the Core Rule Set takes some ordinary requests for attacks, but
it is an offence the client's history counts, and it counts toward the error
burst; a match in `detect` mode is neither. A request a rule file refuses, one
for a path `SWWAF_WAF_EXEMPT_PATHS` exempts, and one from a client in
`SWWAF_ALLOW_NETS` are not inspected. `smallwebwaf` changes the Core Rule Set
in six ways, so that gitea's ordinary requests get through, and no setting
undoes them:
with the query, and its headers, but no request body and no response. Each of
its rules that matches adds to the request's anomaly score, up to the paranoia
level `SWWAF_WAF_PARANOIA_LEVEL` sets. A request with more than 1000 query
parameters adds 5 (rule 900300), as a rule rated critical does, since Coraza
reads only the first 1000. A score at or over `SWWAF_WAF_ANOMALY_THRESHOLD`, 5
by default, is a match: in `block` mode, the default, the request is refused
with `403`, and in `detect` mode it goes on to the app. Either way its log
line names the rules and the score (see `waf_rule_ids` and `waf_score` in
"Request log" below), and it raises a `waf_block` alert. A refusal bans no one
by itself, since the Core Rule Set takes some ordinary requests for attacks,
but it is an offence the client's history counts, and it counts toward the
error burst; a match in `detect` mode is neither. A request a rule file
refuses, one for a path `SWWAF_WAF_EXEMPT_PATHS` exempts, and one from a
client in `SWWAF_ALLOW_NETS` are not inspected. `smallwebwaf` changes the Core
Rule Set in six ways, so that gitea's ordinary requests get through, and no
setting undoes them:
- `PUT`, `PATCH` and `DELETE` are allowed besides `GET`, `HEAD`, `POST` and
`OPTIONS`; any other method stays refused.
- The headers `Expect` and `Content-Encoding` are allowed; the others the
Core Rule Set refuses, such as `Proxy` and `Content-Range`, stay refused.
Once `SWWAF_WAF_BODY_LIMIT` is set, `Content-Encoding` is refused again
(rule 920450) on form data, multipart, JSON and XML, the kinds of body the
Core Rule Set reads, since a compressed body cannot be inspected; so it is
on a JSON or XML body too large to be read.
- The query parameter `redirect_uri` is not checked for a URL naming an IP
address or `localhost` (rules 931100 and 934110), which Git Credential
Manager, git-credential-oauth and tea ask to be sent back to.
@@ -241,14 +232,7 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
command names (932260), so that a file such as `.gitignore` or a branch
such as `docker-build` gets through there. So does what only these rules
refuse, such as `|cat /etc/passwd`; path traversal and SQL injection are
still refused there. These names, and `redirect_uri` above, are matched
without regard to case, as Coraza matches them, so `Path` or `PATH` is
treated as `path`. Once `SWWAF_WAF_BODY_LIMIT` is set, they are left out
in the same way among the fields of a form data or multipart body, which
Coraza holds with the query parameters, and gitea posts some of them in
its forms: `redirect_uri` when an OAuth sign-in is granted, and `ref` when
a workflow is run by hand. A field of a JSON body is named by its path,
such as `json.path`, and keeps these rules.
still refused there.
- The cookies `gitea_flash` and `redirect_to` are not read, and `Referer` is
not checked for a Unix command given without arguments (932340) or for
Java starting a process (944110); it is checked by every other rule.
@@ -711,39 +695,12 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
Core Rule Set does not inspect, each starting with `/`, matched as
`SWWAF_RATE_LIMIT_EXEMPT_PATHS` matches its own: a request whose path holds
`..`, a backslash or an encoded slash is inspected whatever its prefix.
- `SWWAF_WAF_BODY_LIMIT` (default `off`): `off` has the Core Rule Set read no
request body. A size, such as `128K`, at most `1G`, has it read a body of form
data or multipart up to that size, the rest of a longer one passing on to the
app as it arrives, without being held, and a JSON or XML body no larger than
that size, since those cannot be read in part. A body is JSON when its type is
`application/json` or `text/json`, or an `application/` or `text/` type ending
in `+json`, and XML when its type is `application/xml` or `text/xml`, or an
`application/` or `text/` type ending in `+xml`. Any other body reaches the
app uninspected, and so does a larger JSON or XML body: the Core Rule Set
would read any other body as form data, where binary content such as a git
push trips rules written for text. A body it reads that Coraza cannot parse
(rule 900440), and a multipart body that fails Coraza's strict checks (rule
900450), add 5 to the score, as a rule rated critical does, since no rule
reads what comes after the fault. So does a multipart body the limit cuts
before the colon of a part's header line, or between the carriage return and
the line feed that end a part's header line or the empty line after its
headers, since Coraza takes the line the limit cuts for a malformed header.
The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out to send the part
that is read, and a request whose body passes `SWWAF_REQUEST_MAX_BYTES` within
it is refused before anything reaches the app. Of a file in a multipart body,
Coraza counts the bytes and writes nothing. Body inspection suits apps whose
forms carry no code. In front of gitea it refuses issue and comment text, wiki
pages and files saved in the web editor that hold shell commands or code
(932125, 932235, 932250 and others), package descriptions that show code, PyPI
uploads (922130), and attachments named like `debug.log` or `config.yml`
(932180), until the rule ids the request log names are added to
`SWWAF_WAF_DISABLED_RULES`.
- `SWWAF_TRAP_PATHS` (default empty): paths the app never serves and only
scanners ask for, such as `/wp-login.php,/xmlrpc.php` in front of gitea; a
request for one bans its client for a clear sign of attack, as a `ban` rule
does, with or without rule files (see "What it does so far" above). A path
that does not start with `/`, or holds a `?`, as one written with a query
does, stops the start.
that does not start with `/`, or holds a `?`, which no path a `path` rule sees
holds, stops the start.
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most requests of a client in
a minute that `smallwebwaf` may refuse after a rule file match, a trap path or
a Core Rule Set match, or for a missing or wrong token; one more breaks the
@@ -1026,13 +983,13 @@ which every line has.
from when the request's headers had been read to when its line is written, and
`duration_checks` over the same start to when the checks were done; the health
check runs none, and its line has no `duration_checks`. `duration_waf`, the
part of the checks the Core Rule Set took, reading the part of the body it
reads included, is there with `waf_score`. `duration_upstream_connect`,
`duration_upstream_first_byte` and `duration_upstream_total` are there for a
request passed to the app, and run from when it was handed to the app: until
there was a connection to it, new or kept open from an earlier request, until
the first byte of its answer arrived, and until the end. The first two are
left out when that never happened, as for an app that cannot be reached.
part of the checks the Core Rule Set took, is there with `waf_score`.
`duration_upstream_connect`, `duration_upstream_first_byte` and
`duration_upstream_total` are there for a request passed to the app, and run
from when it was handed to the app: until there was a connection to it, new or
kept open from an earlier request, until the first byte of its answer arrived,
and until the end. The first two are left out when that never happened, as for
an app that cannot be reached.
No body is logged, and no header but those above. `smallwebwaf`'s own messages
(start, the settings, stop, errors) share the stream as JSON lines marked
@@ -1523,15 +1480,14 @@ scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
- The id is letters, digits, `-` and `_`, and no two rules share one. The
request log, the metrics and a ban's notes name the rule by it.
- The target is what the regex is matched against: `path`, the path without the
query, and `uri`, the path and the query together, each both as sent and once
percent-decoded, so that an encoded probe does not slip past; `query`, as the
client sent it, before any decoding; `method`; `host`; `user_agent`;
`referer`; or `header:<Name>`, any one request header but `Host` and
`Transfer-Encoding`, which Go's HTTP server takes out of every request; the
request's host is the target `host`. A header sent more than once is matched
with its values joined by `, `, and one not sent as empty text. No body is
read.
- The target is what the regex is matched against: `path` or `query`, as the
client sent it, before any decoding; `uri`, the path and the query together,
both as sent and once percent-decoded, so that an encoded probe does not slip
past; `method`; `host`; `user_agent`; `referer`; or `header:<Name>`, any one
request header but `Host` and `Transfer-Encoding`, which Go's HTTP server
takes out of every request; the request's host is the target `host`. A header
sent more than once is matched with its values joined by `, `, and one not
sent as empty text. No body is read.
- The action is `log`, `block` or `ban` (see "What it does so far" above). Keep
`ban` for requests no real visitor sends, and anchor a path at the site root
with `^/`: a file of the same name deeper in a site can be ordinary content,
@@ -2256,17 +2212,15 @@ alerts, the metrics nor `reputation.json` hold it. Given as a file, with
which bans the client, for a DNSBL zone's verdict, for AbuseIPDB's score, for
a rate limit, which bans the client, for a trap path, which bans the client,
for a `block` or `ban` rule, the latter banning the client, for a Core Rule
Set match in `block` mode, for a body that passes the size limit or
`SWWAF_CLIENT_REQUEST_TIMEOUT` while the Core Rule Set reads it, and for an
announced body over the size limit; in `observe` mode, only for the last two,
with what it would have refused for noted in the log line. A request under
`/_smallwebwaf/` that `check` lets through is answered by `answerAdmin`
instead of reaching the app. Once the answer to a request passed to the app
has ended, `countBytes` counts its bytes for the byte limits, and once any
request but the health check has ended, `countRefusal` counts it for the error
burst if it was refused after a rule file match, a trap path or a Core Rule
Set match or for its token, and `countAnomalies` counts it for the anomaly
thresholds.
Set match in `block` mode, and for an announced body over the size limit; in
`observe` mode, only for the size limit, with what it would have refused for
noted in the log line. A request under `/_smallwebwaf/` that `check` lets
through is answered by `answerAdmin` instead of reaching the app. Once the
answer to a request passed to the app has ended, `countBytes` counts its bytes
for the byte limits, and once any request but the health check has ended,
`countRefusal` counts it for the error burst if it was refused after a rule
file match, a trap path or a Core Rule Set match or for its token, and
`countAnomalies` counts it for the anomaly thresholds.
- `internal/metrics`: the metrics, counted as the other parts tell it what
happened, and served in the Prometheus text format.
- `internal/bans`: the ban ledger: each netblock's bans with their notes, how
@@ -2275,8 +2229,8 @@ alerts, the metrics nor `reputation.json` hold it. Given as a file, with
- `internal/rules`: reads the rule files at start and again as they change, and
tells which of their rules a request matches.
- `internal/waf`: the Core Rule Set with the six changes, as Coraza's own
directives, and what it finds in a request's method, URL, headers and the part
of its body it reads: the rules that matched and the anomaly score.
directives, and what it finds in a request's method, URL and headers: the
rules that matched and the anomaly score.
- `internal/lookup`: looks up each client's AS number and country through GeoJS,
keeps the answers, and hands each new one to the proxy, which adds it to the
client's history and to the notes of its bans; or in the lookup database,
@@ -2374,19 +2328,6 @@ so that they run in minimal containers.
the same volume still refuses the banned client; then removes the containers,
the volume and both images. It needs network access, for nixpkgs' binary
cache, and `script/check` does not run it; `make example-app` does.
- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind it,
with every setting of `smallwebwaf` at its default but `SWWAF_UPSTREAM_URL`,
which names gitea's container. It sends through it the gitea requests that the
notes specific to gitea in [`SPEC.md`](SPEC.md) name for a file name in a
query parameter, for a path and for a container image push, and a short sample
of gitea's ordinary use: signing in, pages of files and commits, a diff, the
API, a push, clone and fetch over HTTP, and two container image pushes. It
prints a line for each request `smallwebwaf` logged, with what it did with it
and the ids of the Core Rule Set's rules that matched it, then removes the
containers, the network and the images it made, but gitea's own image. It
needs network access, and a docker daemon that pushes to a registry on
`localhost` over plain HTTP, as docker does by default; `script/check` does
not run it, and `make gitea-requests` does.
## TODO
SPEC.md
+9 -48
View File
@@ -618,12 +618,10 @@ The settings, by group:
`|cat /etc/passwd`, `wget http://…` and `nc -e /bin/sh …`, and
`file:///etc/passwd`, pass as well. Path traversal (`../`), SQL and
script injection and PHP, Java and Node.js code are still refused
there, and every other parameter keeps all three rules. These names,
and `redirect_uri` in the change before, are matched without regard to
case, as Coraza matches them, so `Path` or `PATH` is treated as
`path`. An app that uses one of these parameters as a file on the
server, or passes it to a shell, gets no help from the three rules
there (see "Risks the design has to handle").
there, and every other parameter keeps all three rules. An app that
uses one of these parameters as a file on the server, or passes it to
a shell, gets no help from the three rules there (see "Risks the
design has to handle").
- The Core Rule Set reads the request without the `gitea_flash` and
`redirect_to` cookies, and does not check `Referer` for a Unix command
given without arguments (932340) or for Java starting a process
@@ -825,8 +823,7 @@ a file edited, added or removed there takes effect while it runs.
- `id`: a short name of letters, digits, `-` and `_`, unique across all files.
It appears in the request log, metrics, alerts and ban notes.
- `target`: what the regex is matched against.
- `path`: the URL path without the query, both as received and once
percent-decoded, so an encoded probe cannot slip past.
- `path`: the URL path as received, before any decoding.
- `query`: the raw query string.
- `uri`: path and query together, both as received and once percent-decoded,
so an encoded probe cannot slip past.
@@ -1477,42 +1474,8 @@ holds any token file.
Saving such a rule stores it, but the branch settings page gitea then
returns to gets the 403 answer of `smallwebwaf`, so the save looks as
if it failed.
- A file name in a query parameter, those the fourth change leaves out
of 930120, 932160 and 932260 included, in which the Core Rule Set's
rules for Java, script, PowerShell and SQL find what they look for:
`runtime` in OpenJDK's
`src/java.base/share/classes/java/lang/Runtime.java` (944110), a Java
class, as in `java.lang.String.md` (944130), `document.write.js`
(941180), `process.env.d.ts` (934100), `__proto__` (934130), the
PowerShell command `Install-Script`, as in `install-script.sh`
(932120), or a database's name, as in PostgreSQL's
`information_schema.sql` (942140). For such a file, the API's list of
its commits (`path`) is refused, and so are the API's call for the
contents of files that names it in its query (`body`), the commit
graph of that file alone (`file`), and the hidden lines of its diff
(`path`), which stay hidden. A visitor whom gitea sends to sign in on
the way to a page of such a file that needs signing in, such as its
edit page, gets the 403 answer of `smallwebwaf` in place of the
sign-in page (`redirect_to`), and can still sign in by opening the
sign-in page itself.
- A path: a file name ending in `~`, or an `.xhtml` file whose path
holds a space.
- A path that holds what the Core Rule Set takes for a Java class
(944130), as `java.lang.String.md` and
`META-INF/services/org.apache.commons.logging.LogFactory` do, or for
Node.js code (934100), as `process.env.d.ts` does, or that has a
directory named `...` (930100, 930110), such as `.../notes.txt`. The
page of such a file, its raw file, its history, its blame and its edit
page are refused; the directory that holds it lists it, and a clone
carries it.
- A container image push that reuses a layer of an image docker has
pushed to that gitea before, when that image's name, after its owner,
starts with an entry on the list of commands (932260), such as
`owner/python3-app` or `owner/docker-base`. docker asks gitea to take
the layer from that image, naming it in the query parameter `from`,
which 932260 checks. The push fails with the 403 answer of
`smallwebwaf`, and so does each new try, since docker asks in the same
way.
- Artifact uploads from `actions/upload-artifact@v3` send the header
`Content-Range`, which the Core Rule Set refuses (920450). The action
sends two files at a time, does not retry a 403 and sends no further file
@@ -1600,12 +1563,10 @@ holds any token file.
- Core Rule Set false positives against real apps (a search for code, and any
body once `SWWAF_WAF_BODY_LIMIT` is set): a match refuses only that request
and bans no one by itself; the request log names the rule, and exclusions by
rule id and path fix it. `script/gitea-requests` runs gitea behind the image
at the defaults, sends through it the requests the notes specific to gitea
name for a file name in a query parameter, for a path and for a container
image push, and a short sample of gitea's ordinary use, and reports each
request the Core Rule Set refuses, so that those notes can be checked again
when gitea or the Core Rule Set changes version.
rule id and path fix it. Further gitea requests the Core Rule Set may refuse
at the defaults, found by reading gitea's source rather than a running gitea,
are collected in https://git.eeqj.de/sneak/smallwebwaf/issues/30 and checked
when milestone 1 runs in front of a real gitea.
- Attacks carried in request bodies: not refused by default, since on a code
forge bodies are full of code the Core Rule Set takes for attacks. Most of
what shows in URLs and headers is still refused, the client that sends an
internal/bans/bans.go
+1 -2
View File
@@ -123,8 +123,7 @@ type Notes struct {
LimitPercentSetting string `json:"limit_percent_setting,omitempty"`
// RuleID and Target are, for a ban for a clear sign of attack, the id
// of the rule file rule that matched, and its target; TrapPath is, for
// one for a request for a path in SWWAF_TRAP_PATHS, that path as the
// client sent it.
// one for a request for a path in SWWAF_TRAP_PATHS, that path.
RuleID string `json:"rule_id,omitempty"`
Target string `json:"target,omitempty"`
TrapPath string `json:"trap_path,omitempty"`
internal/config/config.go
+3 -33
View File
@@ -244,16 +244,14 @@ type Config struct {
// level, from 1 to 4 (SWWAF_WAF_PARANOIA_LEVEL), and
// WAFAnomalyThreshold the anomaly score at which a request is a match
// (SWWAF_WAF_ANOMALY_THRESHOLD), 0 while it is off. WAFDisabledRules
// are the ids of its rules switched off (SWWAF_WAF_DISABLED_RULES),
// are the ids of its rules switched off (SWWAF_WAF_DISABLED_RULES), and
// WAFExemptPaths the path prefixes it does not inspect
// (SWWAF_WAF_EXEMPT_PATHS), and WAFBodyLimit the most of a request body
// it reads (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
// (SWWAF_WAF_EXEMPT_PATHS).
WAFMode string
WAFParanoiaLevel int
WAFAnomalyThreshold int
WAFDisabledRules []int
WAFExemptPaths []string
WAFBodyLimit int64
// TrapPaths are the paths a request for which is a clear sign of
// attack (SWWAF_TRAP_PATHS), each starting with / and without a ?.
TrapPaths []string
@@ -398,7 +396,6 @@ var (
errNeedsDBPath = errors.New("it names the file to look clients up in")
errDBPathUnused = errors.New("only file reads it")
errNotOver4K = errors.New("is not a size of more than 4K, such as 32K")
errOver1G = errors.New("is more than 1G, the most Coraza reads")
errNotDurationAboveZero = errors.New(
"is not a duration above zero, such as 1h or 7d")
errNotNumberAboveZero = errors.New(
@@ -562,7 +559,6 @@ func FromEnvironment(lookupEnv func(string) (string, bool)) (*Config, error) {
WAFDisabledRules: env.ruleIDs("SWWAF_WAF_DISABLED_RULES",
"920340,920420,920440,920640,930130,930140"),
WAFExemptPaths: env.pathPrefixes("SWWAF_WAF_EXEMPT_PATHS", ""),
WAFBodyLimit: env.wafBodyLimit("SWWAF_WAF_BODY_LIMIT", off),
TrapPaths: env.trapPaths("SWWAF_TRAP_PATHS"),
ErrorBurstThreshold: env.count("SWWAF_ERROR_BURST_THRESHOLD", "30"),
LogRemoteURL: env.logRemoteURL("SWWAF_LOG_REMOTE_URL"),
@@ -768,15 +764,6 @@ func (e *environment) size(name, defaultValue string) int64 {
return size
}
// wafBodyLimit reads the setting that is the most of a request body the
// Core Rule Set reads.
func (e *environment) wafBodyLimit(name, defaultValue string) int64 {
limit, err := parseWAFBodyLimit(e.value(name, defaultValue))
e.check(name, err)
return limit
}
// headerSize reads the setting that is the largest request line and
// headers.
func (e *environment) headerSize(name, defaultValue string) int64 {
@@ -1445,22 +1432,6 @@ func parseHeaderSize(value string) (int64, error) {
return size, nil
}
// parseWAFBodyLimit reads the most of a request body the Core Rule Set
// reads: a size as parseSize reads it, or off, but at most 1G, since
// Coraza, which runs the Core Rule Set, refuses to load with more.
func parseWAFBodyLimit(value string) (int64, error) {
limit, err := parseSize(value)
if err != nil {
return 0, err
}
if limit > gibibyte {
return 0, fmt.Errorf("%q %w", value, errOver1G)
}
return limit, nil
}
// splitUnit splits a size into its number and the bytes its suffix
// stands for.
func splitUnit(value string) (string, int64) {
@@ -1665,8 +1636,7 @@ func parsePathPrefixes(value string) ([]string, error) {
// parseTrapPaths reads a comma-separated list of trap paths. Each is
// matched against a request's path as a path rule is, without the query,
// so a path that does not start with / would never match, and one that
// holds a ? is written with a query.
// so a path that does not start with / or holds a ? would never match.
func parseTrapPaths(value string) ([]string, error) {
paths, err := parseList(value)
if err != nil {
internal/config/config_test.go
+3 -32
View File
@@ -96,7 +96,6 @@ const (
wafAnomalyThreshold = "SWWAF_WAF_ANOMALY_THRESHOLD"
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
trapPaths = "SWWAF_TRAP_PATHS"
errorBurstThreshold = "SWWAF_ERROR_BURST_THRESHOLD"
logRemoteURL = "SWWAF_LOG_REMOTE_URL"
@@ -581,20 +580,15 @@ func TestCoreRuleSetSettings(t *testing.T) {
environment{
wafMode: config.WAFModeDetect, wafParanoiaLevel: "4", wafAnomalyThreshold: "10",
wafDisabledRules: "942100, 920350", wafExemptPaths: "/api/, /static/",
wafBodyLimit: "128K",
},
config.Config{
WAFMode: config.WAFModeDetect, WAFParanoiaLevel: 4, WAFAnomalyThreshold: 10,
WAFDisabledRules: []int{942100, 920350},
WAFExemptPaths: []string{"/api/", "/static/"},
WAFBodyLimit: 128 << 10,
},
},
{
environment{
wafMode: off, wafAnomalyThreshold: off, wafDisabledRules: "",
wafBodyLimit: off,
},
environment{wafMode: off, wafAnomalyThreshold: off, wafDisabledRules: ""},
config.Config{
WAFMode: config.WAFModeOff, WAFParanoiaLevel: 1, WAFAnomalyThreshold: 0,
WAFDisabledRules: []int{}, WAFExemptPaths: []string{},
@@ -607,7 +601,6 @@ func TestCoreRuleSetSettings(t *testing.T) {
WAFMode: cfg.WAFMode, WAFParanoiaLevel: cfg.WAFParanoiaLevel,
WAFAnomalyThreshold: cfg.WAFAnomalyThreshold,
WAFDisabledRules: cfg.WAFDisabledRules, WAFExemptPaths: cfg.WAFExemptPaths,
WAFBodyLimit: cfg.WAFBodyLimit,
}
if !reflect.DeepEqual(got, tc.want) {
t.Errorf("%v gave\n%+v\nwant\n%+v", tc.env, got, tc.want)
@@ -615,15 +608,6 @@ func TestCoreRuleSetSettings(t *testing.T) {
}
}
func TestWAFBodyLimitOf1G(t *testing.T) {
t.Parallel()
cfg := fromEnvironment(t, environment{wafBodyLimit: "1G"})
if cfg.WAFBodyLimit != 1<<30 {
t.Errorf("1G read as %d", cfg.WAFBodyLimit)
}
}
func TestInvalidCoreRuleSetSettingStopsTheStart(t *testing.T) {
t.Parallel()
@@ -652,28 +636,16 @@ func TestInvalidCoreRuleSetSettingStopsTheStart(t *testing.T) {
`"-942100" is not the id of a Core Rule Set rule, ` +
`a whole number such as 942100`,
},
// The paranoia level, the allowed methods, the headers refused, a
// request with more query parameters than Coraza keeps, and a body
// Coraza cannot parse or that fails its strict checks.
// The paranoia level, the allowed methods, the headers refused, and
// a request with more query parameters than Coraza keeps.
{wafDisabledRules, "942100,900000", `"900000"` + setupRule},
{wafDisabledRules, "942100,900200", `"900200"` + setupRule},
{wafDisabledRules, "942100,900250", `"900250"` + setupRule},
{wafDisabledRules, "942100,900300", `"900300"` + setupRule},
{wafDisabledRules, "942100,900440", `"900440"` + setupRule},
{wafDisabledRules, "942100,900450", `"900450"` + setupRule},
{
wafExemptPaths, "api/",
`"api/" is not a path prefix starting with /, such as /assets/`,
},
{
wafBodyLimit, "128KB",
`"128KB" is not a size such as 512K, 100M or 5G, or off`,
},
{wafBodyLimit, "2G", `"2G" is more than 1G, the most Coraza reads`},
{
wafBodyLimit, "1073741825",
`"1073741825" is more than 1G, the most Coraza reads`,
},
} {
t.Run(tc.name+"="+tc.value, func(t *testing.T) {
t.Parallel()
@@ -2431,7 +2403,6 @@ func TestLogsEachSettingWithItsValue(t *testing.T) {
wafAnomalyThreshold: "5",
wafDisabledRules: defaultWAFDisabledRules,
wafExemptPaths: "",
wafBodyLimit: off,
trapPaths: "",
errorBurstThreshold: "30",
logRemoteURL: "",
internal/proxy/bodies.go
+1 -12
View File
@@ -21,9 +21,6 @@ type requestBody struct {
// SWWAF_REQUEST_MAX_BYTES.
body io.ReadCloser
rq *request
// readByCoreRuleSet is what the Core Rule Set read of the body before
// the request went to the app, and Read gives first.
readByCoreRuleSet []byte
// waiting is true while a Read waits for the client to send more.
waiting atomic.Bool
// received is true once the client has sent the whole body.
@@ -32,16 +29,8 @@ type requestBody struct {
bytes atomic.Int64
}
// Read reads from the client's body, after what the Core Rule Set read of
// it, which has been counted already.
// Read reads from the client's body.
func (b *requestBody) Read(p []byte) (int, error) {
if len(b.readByCoreRuleSet) > 0 {
n := copy(p, b.readByCoreRuleSet)
b.readByCoreRuleSet = b.readByCoreRuleSet[n:]
return n, nil
}
b.waiting.Store(true)
n, err := b.body.Read(p)
b.waiting.Store(false)
internal/proxy/coreruleset.go
+2 -44
View File
@@ -1,9 +1,6 @@
package proxy
import (
"errors"
"net/http"
"os"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
@@ -19,8 +16,7 @@ import (
// SWWAF_WAF_ANOMALY_THRESHOLD is a match: it raises the waf_block alert,
// and in block mode refuses the request, which is an offence its client's
// history counts, and so returns ActionWAFBlocked. It returns "" for a
// request it does not refuse, and for one whose body meets a size or time
// limit while the Core Rule Set reads it, which it notes nothing of.
// request it does not refuse.
func (rq *request) checkCoreRuleSet() string {
cfg := rq.h.config
if cfg.WAFMode == config.WAFModeOff || pathExempt(rq.in.URL, cfg.WAFExemptPaths) {
@@ -28,12 +24,7 @@ func (rq *request) checkCoreRuleSet() string {
}
start := time.Now()
result := rq.inspect()
if rq.refused.Load() != nil {
return "" // the refusal for that limit, which check returns
}
result := rq.h.coreRuleSet.Inspect(rq.in, rq.client)
rq.line.DurationWAF = new(requestlog.Milliseconds(time.Since(start)))
rq.line.WAFRuleIDs = result.RuleIDs
rq.line.WAFScore = &result.Score
@@ -58,39 +49,6 @@ func (rq *request) checkCoreRuleSet() string {
return requestlog.ActionWAFBlocked
}
// inspect runs the Core Rule Set on the request, which reads the part of
// its body it inspects within SWWAF_CLIENT_REQUEST_TIMEOUT, and keeps that
// part for the app. A client that runs out of time is refused with 408
// here, and a body over SWWAF_REQUEST_MAX_BYTES with 413 as it is read;
// check returns the refusal. A body that breaks off for any other reason
// is passed on as far as it came, and the request to the app fails there,
// as it would have without the Core Rule Set.
func (rq *request) inspect() waf.Result {
if rq.body == nil {
// Nothing is read of no body, so nothing can go wrong reading it.
result, _, _ := rq.h.coreRuleSet.Inspect(rq.in, rq.client, http.NoBody)
return result
}
_ = rq.rc.SetReadDeadline(rq.clientRequestDeadline())
result, read, err := rq.h.coreRuleSet.Inspect(rq.in, rq.client, rq.body)
// The timeouts that run while the request goes to the app take over.
_ = rq.rc.SetReadDeadline(time.Time{})
rq.body.readByCoreRuleSet = read
if errors.Is(err, os.ErrDeadlineExceeded) {
rq.refuse(refusal{
status: http.StatusRequestTimeout,
action: requestlog.ActionTimedOut,
limit: "SWWAF_CLIENT_REQUEST_TIMEOUT",
})
}
return result
}
// alertWAFBlock raises the waf_block alert for the request, which the Core
// Rule Set scored at result, at or over SWWAF_WAF_ANOMALY_THRESHOLD. Its
// detail gives the rule ids, the score, the method and the path with the
internal/proxy/coreruleset_test.go
-198
View File
@@ -1,14 +1,11 @@
package proxy_test
import (
"io"
"net/http"
"net/netip"
"slices"
"strconv"
"strings"
"testing"
"time"
"sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
@@ -21,14 +18,10 @@ const (
wafAnomalyThreshold = "SWWAF_WAF_ANOMALY_THRESHOLD"
wafDisabledRules = "SWWAF_WAF_DISABLED_RULES"
wafExemptPaths = "SWWAF_WAF_EXEMPT_PATHS"
wafBodyLimit = "SWWAF_WAF_BODY_LIMIT"
block = "block"
detect = "detect"
)
// formData is the type of a form's body.
const formData = "application/x-www-form-urlencoded"
// sqlInjection asks for / with an SQL injection in its query, which only
// the Core Rule Set's rule 942100 matches, with a score of 5, the default
// SWWAF_WAF_ANOMALY_THRESHOLD.
@@ -231,197 +224,6 @@ func TestDisabledRulesSwitchOffWhatGiteaWouldBeRefused(t *testing.T) {
}
}
func TestAttackInAFormBodyIsRefusedOnlyWhileBodiesAreRead(t *testing.T) {
t.Parallel()
const body = "id=1'%20OR%20'1'='1"
header := "Content-Type: " + formData + "\r\nContent-Length: " +
strconv.Itoa(len(body))
s, _, _ := startWithClock(t, "", map[string]string{wafMode: block})
line, _ := s.requestWithBody(http.MethodPost, client, "/", header, body,
http.StatusOK, requestlog.ActionForward)
wantWAF(t, line, new(0))
s, _, _ = startWithClock(t, "", map[string]string{
wafMode: block, wafBodyLimit: sizeLimitSetting,
})
line, _ = s.requestWithBody(http.MethodPost, client, "/", header, body,
http.StatusForbidden, requestlog.ActionWAFBlocked)
wantWAF(t, line, new(5), 942100)
}
func TestBodiesReachTheAppAsSentWhileBodiesAreRead(t *testing.T) {
t.Parallel()
// The app answers with the body it was sent, once it has the whole of
// it: Go's server reads no more of a body once the answer has begun.
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
body, _ := io.ReadAll(r.Body)
_, _ = w.Write(body)
})
addr, out := startProxy(t, app.URL, map[string]string{
wafMode: block, wafBodyLimit: sizeLimitSetting,
})
longer := "a=" + strings.Repeat("b", 64*sizeLimit)
for i, tc := range []struct {
name, contentType, body string
// announced sends the body's length in Content-Length; otherwise
// the body is sent in chunks with no length given.
announced bool
}{
{"form data within the limit", formData, "a=b", true},
{"form data longer than the limit", formData, longer, true},
{"form data longer than the limit, not announced", formData, longer, false},
{
"JSON larger than the limit", "application/json",
`{"a":"` + strings.Repeat("b", 2*sizeLimit) + `"}`, true,
},
{
"a binary body", "application/octet-stream",
strings.Repeat("\x00\xff", sizeLimit), true,
},
} {
// A reader whose length the client cannot tell is sent in chunks.
var body io.Reader = strings.NewReader(tc.body)
if !tc.announced {
body = io.MultiReader(body)
}
req := newRequest(t, http.MethodPost, addr, "/", body)
req.Header.Set("Content-Type", tc.contentType)
got := do(t, req)
if got.status != http.StatusOK || string(got.body) != tc.body {
t.Errorf("%s: the app got %d bytes, answered %d, want the %d sent, 200",
tc.name, len(got.body), got.status, len(tc.body))
}
line := out.requestLines(t, i+1)[i]
wantLine(t, line, http.StatusOK, requestlog.ActionForward)
if line.RequestBytes != int64(len(tc.body)) {
t.Errorf("%s: log line has request_bytes %d, want %d", tc.name,
line.RequestBytes, len(tc.body))
}
}
}
func TestFormBodyLongerThanTheLimitStreamsOnToTheApp(t *testing.T) {
t.Parallel()
const (
first = "a=" // and twice the limit of b's, then the rest
rest = 64 * sizeLimit
)
// past is closed once the app has received twice what the Core Rule
// Set reads, and got is the length of the whole body it received.
past := make(chan struct{})
got := make(chan int64, 1)
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
n, _ := io.CopyN(io.Discard, r.Body, 2*sizeLimit)
close(past)
m, _ := io.Copy(io.Discard, r.Body)
got <- n + m
})
addr, out := startProxy(t, app.URL, map[string]string{
wafMode: block, wafBodyLimit: sizeLimitSetting,
})
// The client sends the rest only once the app has received the first
// part: were smallwebwaf to hold the body until the end, it would
// never come.
body, sender := io.Pipe()
go func() {
_, _ = io.WriteString(sender, first+strings.Repeat("b", 2*sizeLimit))
select {
case <-past:
case <-time.After(waitLimit):
t.Error("the app got no more than the Core Rule Set reads " +
"before the whole body was sent")
_ = sender.CloseWithError(io.ErrUnexpectedEOF)
return
}
_, _ = io.WriteString(sender, strings.Repeat("b", rest))
_ = sender.Close()
}()
req := newRequest(t, http.MethodPost, addr, "/", body)
req.Header.Set("Content-Type", formData)
wantStatus(t, do(t, req), http.StatusOK)
want := int64(len(first) + 2*sizeLimit + rest)
if n := <-got; n != want {
t.Errorf("the app got %d bytes, want %d", n, want)
}
wantLine(t, out.requestLine(t), http.StatusOK, requestlog.ActionForward)
}
func TestClientTooSlowToSendWhatTheCoreRuleSetReads(t *testing.T) {
t.Parallel()
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
addr, out := startProxy(t, app.URL, map[string]string{
wafMode: block, wafBodyLimit: sizeLimitSetting,
clientRequestTimeout: shortTimeoutSetting, metricsToken: token,
})
conn := dial(t, addr)
send(t, conn, "POST /comment HTTP/1.1\r\nHost: app\r\nContent-Type: "+formData+
"\r\nContent-Length: 100\r\n\r\ncontent=the first bytes")
wantStatus(t, readResponse(t, conn), http.StatusRequestTimeout)
line := out.requestLine(t)
wantLine(t, line, http.StatusRequestTimeout, requestlog.ActionTimedOut)
wantNotSentToTheApp(t, line)
wantLimitHits(t, addr, clientRequestTimeout, 1)
}
func TestBodyOverTheSizeLimitWhileTheCoreRuleSetReadsIt(t *testing.T) {
t.Parallel()
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
addr, out := startProxy(t, app.URL, map[string]string{
wafMode: block, wafBodyLimit: "4K",
requestMaxBytes: sizeLimitSetting, metricsToken: token,
})
// Sent in chunks, its length is not announced, and is found to be over
// the limit as the Core Rule Set reads it.
body := io.MultiReader(strings.NewReader("a=" + strings.Repeat("b", 2*sizeLimit)))
req := newRequest(t, http.MethodPost, addr, "/", body)
req.Header.Set("Content-Type", formData)
wantStatus(t, do(t, req), http.StatusRequestEntityTooLarge)
line := out.requestLine(t)
wantLine(t, line, http.StatusRequestEntityTooLarge, requestlog.ActionTooLarge)
wantNotSentToTheApp(t, line)
wantLimitHits(t, addr, requestMaxBytes, 1)
}
// wantNotSentToTheApp checks that the request of line was not sent to the
// app at all.
func wantNotSentToTheApp(t *testing.T, line logLine) {
t.Helper()
_, sent := line.fields["duration_upstream_total"]
if sent {
t.Error("log line has duration_upstream_total, for a request sent to the app")
}
}
func TestResponsesAreNotInspected(t *testing.T) {
t.Parallel()
internal/proxy/lookup_test.go
-69
View File
@@ -86,75 +86,6 @@ func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(t *testing.T)
}
}
func TestRequestThatWaitedForItsLookupIsCountedWhenItGoesOn(t *testing.T) {
t.Parallel()
// The country list has a new client's request wait for its answer,
// which the stand-in for GeoJS gives only once released.
geojsURL, asked, release := startHeldGeoJS(t)
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 58, 0, time.UTC)}
addr, out, server := startProxyWithClock(t, app.URL, geojsURL, clk.Now,
map[string]string{
trustedProxies: trustLocalhost,
lookupTimeout: "1h",
deniedCountries: "kp",
})
requestFromDE := func() *http.Request {
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Header.Set(forwardedFor, fromDE)
return req
}
// fromDE's first request arrives two seconds before a minute ends,
// and waits for GeoJS.
first, httpClient := requestFromDE(), newClient(t)
firstEnded := make(chan error, 1)
go func() {
res, err := httpClient.Do(first)
if err == nil {
err = readAnswer(res).err
}
firstEnded <- err
}()
waitUntil(func() bool { return len(asked()) == 1 })
// Half a second into the next minute, the answer is kept, as an edit
// of lookups.json would keep it, while the first request still waits
// for GeoJS. The second request goes on at once, and is counted.
clk.advance(2500 * time.Millisecond)
server.GeoJS.Load([]lookup.Answer{{
Client: netip.MustParsePrefix(fromDE + "/32"), ASN: asnDE, ASName: asNameDE,
Country: "DE", Answered: clk.Now(), Used: clk.Now(),
}})
wantStatus(t, do(t, requestFromDE()), http.StatusOK)
// GeoJS answers, and the first request goes on.
release()
err := <-firstEnded
if err != nil {
t.Fatalf("first request: %v", err)
}
wantLine(t, out.requestLines(t, 2)[1], http.StatusOK, requestlog.ActionForward)
// The third request's count holds all three: the first was counted
// when it went on, in the minute under way, and the second's count
// was kept.
clk.advance(500 * time.Millisecond)
wantStatus(t, do(t, requestFromDE()), http.StatusOK)
if got := out.requestLines(t, 3)[2].Counts.Minute; got != 3 {
t.Errorf("the third request's count in the minute is %v, want 3", got)
}
}
func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert(t *testing.T) {
t.Parallel()
internal/proxy/proxy.go
+4 -5
View File
@@ -232,8 +232,8 @@ func newReputation(
}
// newCoreRuleSet returns the Core Rule Set at SWWAF_WAF_PARANOIA_LEVEL,
// without the rules SWWAF_WAF_DISABLED_RULES switches off, reading bodies
// up to SWWAF_WAF_BODY_LIMIT, or nil while SWWAF_WAF_MODE is off.
// without the rules SWWAF_WAF_DISABLED_RULES switches off, or nil while
// SWWAF_WAF_MODE is off.
func newCoreRuleSet(cfg *config.Config) *waf.CoreRuleSet {
if cfg.WAFMode == config.WAFModeOff {
return nil
@@ -241,12 +241,11 @@ func newCoreRuleSet(cfg *config.Config) *waf.CoreRuleSet {
coreRuleSet, err := waf.New(waf.Params{
ParanoiaLevel: cfg.WAFParanoiaLevel, DisabledRules: cfg.WAFDisabledRules,
BodyLimit: cfg.WAFBodyLimit,
})
if err != nil {
// The Core Rule Set is built in, and the settings cannot break it:
// the paranoia level is from 1 to 4, the body limit at most 1G, and
// the id of no rule switches nothing off.
// the paranoia level is from 1 to 4, and the id of no rule switches
// nothing off.
panic(err)
}
internal/proxy/request.go
+7 -20
View File
@@ -188,23 +188,16 @@ func requestHeaders(r *http.Request, names []string) map[string]string {
}
// check is the one place where a request can be refused once its client
// is known, before anything reaches the app, and before its body is read,
// but for the part the Core Rule Set reads. It returns nil to let the
// request through. The checks of checkClient come first, answered with
// SWWAF_BAN_RESPONSE, or 403 for a block rule or the Core Rule Set, and
// then the size limit, so that a request the rate limits count is counted
// even when it is refused for its size. In observe mode a request
// checkClient refuses goes on to the size limit like any other. A size or
// time limit the Core Rule Set's reading of the body meets ends the
// request in either mode. ctx is the request's own context.
// is known, before its body is read or anything reaches the app. It
// returns nil to let the request through. The checks of checkClient come
// first, answered with SWWAF_BAN_RESPONSE, or 403 for a block rule or the
// Core Rule Set, and then the size limit, so that a request the rate
// limits count is counted even when it is refused for its size. In
// observe mode a request checkClient refuses goes on to the size limit
// like any other. ctx is the request's own context.
func (rq *request) check(ctx context.Context) *refusal {
action := rq.checkClient(ctx)
refused := rq.refused.Load()
if refused != nil {
return refused
}
switch {
case action == "":
case rq.h.config.Observe:
@@ -261,12 +254,6 @@ func (rq *request) checkClient(ctx context.Context) string {
rq.lookUp(ctx)
// The lookup can wait for GeoJS, so the checks after it read the time
// again: a request counted at the time it arrived could be counted
// after a later one of its client's, which the rate limits would take
// for the clock set back.
now = rq.h.now()
if rq.countryDenied() {
return requestlog.ActionCountryDenied
}
internal/proxy/rules.go
+5 -7
View File
@@ -10,15 +10,13 @@ import (
)
// trapPath reports whether the request asks for a path in
// SWWAF_TRAP_PATHS: its path as a path rule sees it, without the query,
// as sent or once percent-decoded, is one of them. Such a request is a
// clear sign of attack, as a ban rule's match is: it bans the client's
// netblock, or in observe mode raises the alert for the ban it would have
// made. The ban's notes give the path as sent.
// SWWAF_TRAP_PATHS: its path as a path rule sees it, before any decoding
// and without the query, is one of them. Such a request is a clear sign of
// attack, as a ban rule's match is: it bans the client's netblock, or in
// observe mode raises the alert for the ban it would have made.
func (rq *request) trapPath(now time.Time) bool {
path := rules.Path(rq.in)
if !slices.Contains(rq.h.config.TrapPaths, path) &&
!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
if !slices.Contains(rq.h.config.TrapPaths, path) {
return false
}
internal/proxy/trappaths_test.go
+4 -19
View File
@@ -32,9 +32,10 @@ func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
})
start := clk.Now()
// Only the path itself, the whole of it in the same case, is a trap
// path.
for _, path := range []string{"/wp-login.php/", "/WP-LOGIN.PHP", "/blog/xmlrpc.php"} {
// Only the path itself, as the client sent it, is a trap path.
for _, path := range []string{
"/wp-login.php/", "/WP-LOGIN.PHP", "/blog/xmlrpc.php", "/%77p-login.php",
} {
s.request(otherClient, path, http.StatusOK, requestlog.ActionForward)
}
@@ -83,22 +84,6 @@ func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
}
}
func TestTrapPathMatchesThePathDecodedOnce(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{trapPaths: trapPathList})
// %2577 decodes once to %77, not to w.
s.request(otherClient, "/%2577p-login.php", http.StatusOK, requestlog.ActionForward)
s.request(client, "/%77p-login.php", http.StatusForbidden, requestlog.ActionBanned)
// The ban's notes give the path as the client sent it.
got := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
if len(got) != 1 || got[0].Notes.TrapPath != "/%77p-login.php" {
t.Fatalf("bans %+v, want one whose notes give the trap path /%%77p-login.php", got)
}
}
func TestTrapPathsNeedNoRuleFiles(t *testing.T) {
t.Parallel()
internal/rules/rules.go
+16 -26
View File
@@ -50,13 +50,6 @@ const quietTime = 2 * time.Second
// header:<Name>.
const headerTarget = "header:"
// pathTarget and uriTarget are the targets matched both as received and
// once percent-decoded: the path, and the path and query together.
const (
pathTarget = "path"
uriTarget = "uri"
)
// escapeLength is the length of a percent escape, such as %2e.
const escapeLength = 3
@@ -381,7 +374,7 @@ func parse(line string) (Rule, bool, error) {
// isTarget reports whether target is one a rule may have.
func isTarget(target string) bool {
switch target {
case pathTarget, "query", uriTarget, "method", "host", "user_agent", "referer":
case "path", "query", "uri", "method", "host", "user_agent", "referer":
return true
}
@@ -391,38 +384,35 @@ func isTarget(target string) bool {
}
// matches reports whether the rule's regex matches its target in r. For
// path and uri it is matched against the target as received, and against
// it once percent-decoded, so that an encoded probe cannot slip past.
// uri it is matched against the path and query as received, and against
// them once percent-decoded, so that an encoded probe cannot slip past.
func (rule Rule) matches(r *http.Request) bool {
target := value(rule.Target, r)
if rule.Target == "uri" {
uri := pathAndQuery(r)
if rule.Target == pathTarget || rule.Target == uriTarget {
return rule.regex.MatchString(target) || rule.regex.MatchString(DecodeOnce(target))
return rule.regex.MatchString(uri) || rule.regex.MatchString(decodeOnce(uri))
}
return rule.regex.MatchString(target)
return rule.regex.MatchString(value(rule.Target, r))
}
// Path returns r's path as the client sent it, before any decoding or
// re-encoding, up to the first ?: what a path rule is matched against, as
// it is and once percent-decoded.
// re-encoding, up to the first ?: what a path rule is matched against.
func Path(r *http.Request) string {
path, _, _ := strings.Cut(pathAndQuery(r), "?")
return path
}
// value returns what a rule with target is matched against in r: the
// path, as Path gives it, the path and query, as pathAndQuery gives
// them, and the query as the client sent it, before any decoding or
// re-encoding, after the first ?, and a header's values joined by ", ",
// as HTTP joins those of a header sent more than once.
// value returns what a rule with target, other than uri, is matched
// against in r: the path, as Path gives it, and the query as the client
// sent it, before any decoding or re-encoding, after the first ?, and a
// header's values joined by ", ", as HTTP joins those of a header sent
// more than once.
func value(target string, r *http.Request) string {
switch target {
case pathTarget:
case "path":
return Path(r)
case uriTarget:
return pathAndQuery(r)
case "query":
_, query, _ := strings.Cut(pathAndQuery(r), "?")
@@ -474,11 +464,11 @@ func header(r *http.Request, name string) string {
return strings.Join(r.Header.Values(name), ", ")
}
// DecodeOnce returns s with each percent escape, such as %2e, replaced by
// decodeOnce returns s with each percent escape, such as %2e, replaced by
// the byte it stands for. A % that is not followed by two hex digits is
// left as it is, so that a malformed escape cannot keep the rest of s
// from being decoded.
func DecodeOnce(s string) string {
func decodeOnce(s string) string {
var decoded strings.Builder
for i := 0; i < len(s); i++ {
internal/rules/rules_test.go
+2 -4
View File
@@ -49,9 +49,7 @@ func TestEachTargetMatchesWhatItNames(t *testing.T) {
want bool
}{
{"path as received", `path log ^/%2eenv$`, "/%2eenv", nil, true},
{"path decoded", `path log ^/\.env$`, "/%2Eenv", nil, true},
{"path decoded only once", `path log ^/\.env$`, "/%252eenv", nil, false},
{"path decoded without the query", `path log ^/a\?b$`, "/a%3fb?c", nil, true},
{"path not decoded", `path log ^/\.env$`, "/%2eenv", nil, false},
{"path without the query", `path log ^/a$`, "/a?b=c", nil, true},
{"query as received", `query log ^b=%2e$`, "/a?b=%2e", nil, true},
{"uri as received", `uri log ^/a\?b=%2e$`, "/a?b=%2e", nil, true},
@@ -420,7 +418,7 @@ func TestDefaultFileBansProbesAtTheSiteRootAlone(t *testing.T) {
// Probes sent by a browser, by the rule that refuses them.
for rule, targets := range map[string][]string{
"env-file": {"/.env", "/.env.production", "/.ENV", "/%2eenv"},
"env-file": {"/.env", "/.env.production", "/.ENV"},
"vcs-dir": {"/.git/config", "/.git", "/.svn/entries"},
"secrets-dir": {"/.aws/credentials", "/.ssh/id_rsa"},
"secret-file": {"/.htpasswd", "/.DS_Store", "/.git-credentials"},
internal/waf/waf.go
+30 -139
View File
@@ -1,18 +1,12 @@
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
// method, the URL with its query and the headers of a request, and on its
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
// makes to it, as "Attack detection" under "Configuration surface" in
// SPEC.md describes them. It reads no response.
//
// smallwebwaf writes only to its state directory, so Coraza is built with
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
// file in a multipart body, Coraza then counts the bytes instead of
// writing them to the system's temporary directory.
// method, the URL with its query and the headers of a request, with the
// six changes smallwebwaf makes to it, as "Attack detection" under
// "Configuration surface" in SPEC.md describes them. It reads no request
// body and no response.
package waf
import (
"fmt"
"io"
"net/http"
"net/netip"
"slices"
@@ -26,16 +20,15 @@ import (
)
// directives are the Core Rule Set as smallwebwaf runs it, with the
// paranoia level for %d, and bodyDirectives for %s while
// SWWAF_WAF_BODY_LIMIT is set. Each rule smallwebwaf adds has an id from
// 900000 to 900999, the ids the Core Rule Set keeps for the rules that set
// it up, which SWWAF_WAF_DISABLED_RULES refuses, so that no setting
// switches one off. Coraza joins a line ending in \ to the next, without
// the spaces at the start of the next.
// paranoia level for %d. Each rule smallwebwaf adds has an id from 900000
// to 900999, the ids the Core Rule Set keeps for the rules that set it
// up, which SWWAF_WAF_DISABLED_RULES refuses, so that no setting switches
// one off. Coraza joins a line ending in \ to the next, without the spaces
// at the start of the next.
const directives = `
# The engine only detects. smallwebwaf compares the request's anomaly
# score with SWWAF_WAF_ANOMALY_THRESHOLD itself, in block and detect mode
# alike. It reads no body, unless bodyDirectives switch that on.
# alike. It reads no body.
SecRuleEngine DetectionOnly
SecRequestBodyAccess Off
SecResponseBodyAccess Off
@@ -51,33 +44,19 @@ SecAction "id:900200,phase:1,pass,nolog,\
setvar:'tx.allowed_methods=GET HEAD POST OPTIONS PUT PATCH DELETE'"
# The second: Expect and Content-Encoding are taken off the Core Rule Set's
# list of the headers it refuses. Content-Encoding goes back on it for a
# body the Core Rule Set reads (900260 in bodyDirectives).
# list of the headers it refuses. Content-Encoding stays refused on a body
# the Core Rule Set reads, and it reads none.
SecAction "id:900250,phase:1,pass,nolog,\
setvar:'tx.restricted_headers_basic=/proxy/ /lock-token/ /content-range/ \
/if/ /x-http-method-override/ /x-http-method/ /x-method-override/ \
/x-middleware-subrequest/'"
%s
# Coraza keeps the first 1000 query parameters of a request, and the first
# 1000 fields of a form data or JSON body, and drops the rest, which no
# rule then reads, so a request with more adds 5 to the score, as a rule
# the Core Rule Set rates critical does. Coraza's recommended
# configuration refuses such a request in its rules 200004 and 200005.
# This rule runs once the body is read, and before the Core Rule Set adds
# up the score in the same phase.
SecArgumentsLimit 1000
SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:2,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
# The sixth: only the rules for requests are loaded, and no response is
# inspected.
Include @owasp_crs/REQUEST-*.conf
# The third: redirect_uri is not checked for a URL naming an IP address or
# localhost. Coraza matches a parameter name here, and in the fourth,
# without regard to case. ARGS holds the fields of a form data or multipart
# body Coraza reads as well as the query parameters, so a field of one of
# these names is left out too.
# localhost.
SecRuleUpdateTargetById 931100 "!ARGS:redirect_uri"
SecRuleUpdateTargetById 934110 "!ARGS:redirect_uri"
@@ -100,47 +79,15 @@ SecRuleUpdateTargetById 932260 "!ARGS:path|!ARGS:files|!ARGS:skip-to|\
# Inspect.
SecRuleUpdateTargetById 932340 "!REQUEST_HEADERS:Referer"
SecRuleUpdateTargetById 944110 "!REQUEST_HEADERS:Referer"
`
// bodyDirectives have the Core Rule Set read the part of a request body
// Inspect gives it, which is at most one byte longer than the limit, up to
// the limit, %d bytes, and read JSON and XML as Coraza's recommended
// configuration has it in its rules 200000, 200001 and 200006, with
// text/json, and any application or text type ending in +xml or +json,
// besides; form data and multipart Coraza knows by itself. %% stands for
// a % Coraza reads.
const bodyDirectives = `
SecRequestBodyAccess On
SecRequestBodyLimit %d
SecRequestBodyLimitAction ProcessPartial
SecRule REQUEST_HEADERS:Content-Type \
"@rx ^(?:application|text)/(?:[a-z0-9.-]+[+])?xml" \
"id:900410,phase:1,pass,nolog,t:none,t:lowercase,ctl:requestBodyProcessor=XML"
SecRule REQUEST_HEADERS:Content-Type \
"@rx ^(?:application|text)/(?:[a-z0-9.-]+[+])?json" \
"id:900420,phase:1,pass,nolog,t:none,t:lowercase,ctl:requestBodyProcessor=JSON"
# The rest of the second change: Content-Encoding is refused again on a
# body of a kind the Core Rule Set reads, since a compressed body cannot be
# inspected.
SecRule REQBODY_PROCESSOR "@rx ^(?:URLENCODED|MULTIPART|JSON|XML)$" \
"id:900260,phase:1,pass,nolog,\
setvar:'tx.restricted_headers_basic=%%{tx.restricted_headers_basic} \
/content-encoding/'"
# A body Coraza fails to parse (900440), and a multipart body that fails
# its strict checks (900450), each add 5 to the score, as a rule the Core
# Rule Set rates critical does: no rule reads what comes after the fault,
# which the app may still read. Coraza's recommended configuration refuses
# them in its rules 200002 and 200003. A multipart body the limit cuts
# before the colon of a part's header line, or between the carriage return
# and the line feed that end a part's header line or the empty line after
# its headers, adds 5 too, since Coraza takes the line the limit cuts for a
# malformed header. Coraza parses any form data body.
SecRule REQBODY_ERROR "!@eq 0" "id:900440,phase:2,pass,severity:'CRITICAL',\
setvar:'tx.inbound_anomaly_score_pl1=+5'"
SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
# Coraza keeps the first 1000 query parameters of a request and drops the
# rest, which no rule then reads, so a request with more adds 5 to the
# score, as a rule the Core Rule Set rates critical does. Coraza's
# recommended configuration refuses such a request in its rule 200004.
# This rule comes after the Core Rule Set's, which set the score to 0 in
# the same phase.
SecArgumentsLimit 1000
SecRule ARGUMENTS_LIMIT_REACHED "@eq 1" "id:900300,phase:1,pass,\
severity:'CRITICAL',setvar:'tx.inbound_anomaly_score_pl1=+5'"
`
@@ -157,28 +104,18 @@ type Params struct {
// DisabledRules are the ids of the rules switched off
// (SWWAF_WAF_DISABLED_RULES).
DisabledRules []int
// BodyLimit is the most of a request body the Core Rule Set reads
// (SWWAF_WAF_BODY_LIMIT), 0 while it is off and it reads none.
BodyLimit int64
}
// CoreRuleSet is the Core Rule Set, ready to inspect requests. It is safe
// for concurrent use.
type CoreRuleSet struct {
waf coraza.WAF
bodyLimit int64
waf coraza.WAF
}
// New returns the Core Rule Set with the six changes, at params'
// paranoia level, without the rules it switches off, and reading request
// bodies up to params' limit.
// paranoia level and without the rules it switches off.
func New(params Params) (*CoreRuleSet, error) {
body := ""
if params.BodyLimit > 0 {
body = fmt.Sprintf(bodyDirectives, params.BodyLimit)
}
text := fmt.Sprintf(directives, params.ParanoiaLevel, body)
text := fmt.Sprintf(directives, params.ParanoiaLevel)
if len(params.DisabledRules) > 0 {
ids := make([]string, len(params.DisabledRules))
@@ -196,7 +133,7 @@ func New(params Params) (*CoreRuleSet, error) {
return nil, fmt.Errorf("load the Core Rule Set: %w", err)
}
return &CoreRuleSet{waf: waf, bodyLimit: params.BodyLimit}, nil
return &CoreRuleSet{waf: waf}, nil
}
// Result is what the Core Rule Set found in a request.
@@ -210,15 +147,10 @@ type Result struct {
// Inspect runs the Core Rule Set on r, a request from client: on its
// method, its URL with the query, and its headers, the Cookie header
// without the cookies in cookiesNotRead, and on body, r's body as the
// caller has it, as readBody reads it. It returns what it found, what it
// read of body, which the app is still to be sent, and the error that
// ended the reading early, if one did.
func (c *CoreRuleSet) Inspect(
r *http.Request, client netip.Addr, body io.Reader,
) (Result, []byte, error) {
// without the cookies in cookiesNotRead.
func (c *CoreRuleSet) Inspect(r *http.Request, client netip.Addr) Result {
tx := c.waf.NewTransaction()
// Closing would remove the files Coraza wrote, and it writes none.
// With no body read, there is nothing whose closing can fail.
defer func() { _ = tx.Close() }()
tx.ProcessConnection(client.String(), 0, "", 0)
@@ -245,11 +177,7 @@ func (c *CoreRuleSet) Inspect(
}
tx.ProcessRequestHeaders()
read, err := c.readBody(tx, body)
// This reads the body in memory and runs the rest of the rules, and
// cannot fail.
// With no body read, this runs the rest of the rules, and cannot fail.
_, _ = tx.ProcessRequestBody()
var ids []int
@@ -263,44 +191,7 @@ func (c *CoreRuleSet) Inspect(
}
}
return Result{RuleIDs: ids, Score: score(tx)}, read, err
}
// readBody reads body, the body of the request in tx, which has run on
// the request's headers, while SWWAF_WAF_BODY_LIMIT is set and the body is
// of a kind the Core Rule Set reads: form data and multipart, of which it
// reads the first c.bodyLimit bytes, and JSON and XML, which it reads only
// when they are no longer than that, since they cannot be read in part.
// readBody reads one byte past the limit, to tell which they are, gives
// the Core Rule Set what it reads, and returns what it read and the error
// that ended the reading early, if one did.
func (c *CoreRuleSet) readBody(tx types.Transaction, body io.Reader) ([]byte, error) {
if c.bodyLimit == 0 {
return nil, nil
}
inPart := false
// How the body is read is a variable of the transaction, which only
// Coraza's interface for plugins reads.
state := tx.(plugintypes.TransactionState) //nolint:forcetypeassert // every one is
switch state.Variables().RequestBodyProcessor().Get() {
case "URLENCODED", "MULTIPART":
inPart = true
case "JSON", "XML":
default:
return nil, nil
}
read, err := io.ReadAll(io.LimitReader(body, c.bodyLimit+1))
if inPart || (err == nil && int64(len(read)) <= c.bodyLimit) {
// Coraza holds what it reads of the body in memory, up to the
// limit, so this cannot fail.
_, _, _ = tx.WriteRequestBody(read)
}
return read, err
return Result{RuleIDs: ids, Score: score(tx)}
}
// score returns the anomaly score the Core Rule Set added up in tx, a
internal/waf/waf_test.go
+4 -382
View File
@@ -4,10 +4,7 @@ import (
"net/http"
"net/http/httptest"
"net/netip"
"net/url"
"path/filepath"
"reflect"
"strconv"
"strings"
"testing"
@@ -51,28 +48,12 @@ func get(target string, headers ...string) request {
func inspect(t *testing.T, crs *waf.CoreRuleSet, r request) waf.Result {
t.Helper()
result, _ := inspectBody(t, crs, r, "")
return result
}
// inspectBody is inspect for r with body, which is announced with its
// Content-Length unless it is "", and returns what crs read of body too.
func inspectBody(
t *testing.T, crs *waf.CoreRuleSet, r request, body string,
) (waf.Result, string) {
t.Helper()
req := httptest.NewRequestWithContext(t.Context(), r.method,
"http://git.example"+r.target, strings.NewReader(body))
"http://git.example"+r.target, http.NoBody)
req.Header.Set("User-Agent", "Mozilla/5.0 (X11; Linux x86_64; rv:131.0) "+
"Gecko/20100101 Firefox/131.0")
req.Header.Set("Accept", "text/html")
if body != "" {
req.Header.Set("Content-Length", strconv.Itoa(len(body)))
}
for _, header := range r.headers {
// Go's server keeps Host and Transfer-Encoding out of the headers.
name, value, _ := strings.Cut(header, ": ")
@@ -86,12 +67,7 @@ func inspectBody(
}
}
result, read, err := crs.Inspect(req, netip.MustParseAddr("203.0.113.9"), req.Body)
if err != nil {
t.Fatalf("read the body: %v", err)
}
return result, string(read)
return crs.Inspect(req, netip.MustParseAddr("203.0.113.9"))
}
// wantResult checks what crs finds in r.
@@ -163,9 +139,7 @@ func TestExpectAndContentEncodingAreAllowed(t *testing.T) {
request{http.MethodPost, push, []string{pushType, length, "Expect: 100-continue"}},
waf.Result{})
wantResult(t, crs,
request{http.MethodPost, fetch, []string{
fetchType, length, "Content-Encoding: gzip",
}},
request{http.MethodPost, fetch, []string{fetchType, length, "Content-Encoding: gzip"}},
waf.Result{})
// Every other header on the Core Rule Set's list stays refused.
@@ -198,7 +172,7 @@ func TestTransferEncodingIsRead(t *testing.T) {
waf.Result{})
}
func TestMoreParametersThanCorazaKeepsIsAMatch(t *testing.T) {
func TestMoreQueryParametersThanCorazaKeepsIsAMatch(t *testing.T) {
t.Parallel()
const attack = "id=1'%20OR%20'1'='1"
@@ -209,23 +183,6 @@ func TestMoreParametersThanCorazaKeepsIsAMatch(t *testing.T) {
// after it is not, but the request is a match all the same.
wantResult(t, crs, get("/?"+strings.Repeat("a=1&", 999)+attack), matched(942100))
wantResult(t, crs, get("/?"+strings.Repeat("a=1&", 1000)+attack), matched(900300))
// So it is with the fields of a form data or JSON body.
crs = readingBodies(t)
for _, tc := range []struct{ header, body string }{
{formData, strings.Repeat("a=1&", 999) + attack},
{jsonBody, `{"a":[` + strings.Repeat("1,", 998) + `1],"id":"` + injection + `"}`},
} {
wantBody(t, crs, post(tc.header), tc.body, matched(942100), tc.body)
}
for _, tc := range []struct{ header, body string }{
{formData, strings.Repeat("a=1&", 1000) + attack},
{jsonBody, `{"a":[` + strings.Repeat("1,", 999) + `1],"id":"` + injection + `"}`},
} {
wantBody(t, crs, post(tc.header), tc.body, matched(900300), tc.body)
}
}
func TestRedirectURIMayNameALocalAddress(t *testing.T) {
@@ -274,16 +231,6 @@ func TestParametersGiteaSendsNamesInSkipTheListsOfFilesPathsAndCommands(t *testi
wantResult(t, crs, get("/?path=1'%20OR%20'1'='1"), matched(942100))
}
func TestParameterNamesAreMatchedWithoutRegardToCase(t *testing.T) {
t.Parallel()
crs := atDefaults(t)
wantChange(t, crs, get("/?Path=.gitignore"), get("/?q=.gitignore"), matched(930120))
wantChange(t, crs, get("/?REDIRECT_URI=http://127.0.0.1:52341/"),
get("/?next=http://127.0.0.1:52341/"), matched(931100, 934110))
}
func TestCookiesGiteaFlashAndRedirectToAreNotRead(t *testing.T) {
t.Parallel()
@@ -361,328 +308,3 @@ func TestEachDisabledRuleIsSwitchedOff(t *testing.T) {
waf.Result{RuleIDs: []int{911100, 920350}, Score: 8})
wantResult(t, newCoreRuleSet(t, 1, 920350, 911100), r, waf.Result{})
}
// bodyLimit is SWWAF_WAF_BODY_LIMIT in the tests that read bodies.
const bodyLimit = 8 << 10
// The Content-Type headers of the kinds of body the Core Rule Set reads.
const (
formData = "Content-Type: application/x-www-form-urlencoded"
multipart = "Content-Type: multipart/form-data; boundary=b"
jsonBody = "Content-Type: application/json"
xmlBody = "Content-Type: application/xml"
)
// injection is an SQL injection, which rule 942100 matches.
const injection = "1' OR '1'='1"
// readingBodies returns the Core Rule Set as smallwebwaf runs it by
// default, but reading bodies up to bodyLimit.
func readingBodies(t *testing.T) *waf.CoreRuleSet {
t.Helper()
crs, err := waf.New(waf.Params{
ParanoiaLevel: 1, DisabledRules: defaultDisabledRules, BodyLimit: bodyLimit,
})
if err != nil {
t.Fatalf("load the Core Rule Set: %v", err)
}
return crs
}
// post is a POST request for / with a body of the type contentType, a
// Content-Type header, gives, and headers besides.
func post(contentType string, headers ...string) request {
return request{http.MethodPost, "/", append([]string{contentType}, headers...)}
}
// field is a part of a multipart body: the field name, holding value.
func field(name, value string) string {
return "--b\r\nContent-Disposition: form-data; name=\"" + name + "\"\r\n\r\n" +
value + "\r\n"
}
// end ends a multipart body.
const end = "--b--\r\n"
// padded returns head and tail with as many a's between them as make n
// bytes in all.
func padded(head, tail string, n int) string {
return head + strings.Repeat("a", n-len(head)-len(tail)) + tail
}
// wantBody checks what crs finds in r with body, and that what it read of
// body is read.
func wantBody(
t *testing.T, crs *waf.CoreRuleSet, r request, body string, want waf.Result,
read string,
) {
t.Helper()
got, gotRead := inspectBody(t, crs, r, body)
if !reflect.DeepEqual(got, want) || gotRead != read {
t.Errorf("%q with a body of %d bytes, %.40q: %+v, reading %d bytes, "+
"want %+v, reading %d", r.headers, len(body), body, got, len(gotRead),
want, len(read))
}
}
func TestBodiesAreReadOnlyWhileBodyLimitIsSet(t *testing.T) {
t.Parallel()
off, on := atDefaults(t), readingBodies(t)
for _, tc := range []struct{ header, body string }{
{formData, "q=" + url.QueryEscape(injection)},
{multipart, field("q", injection) + end},
{jsonBody, `{"q":"` + injection + `"}`},
{xmlBody, "<q>" + injection + "</q>"},
} {
wantBody(t, off, post(tc.header), tc.body, waf.Result{}, "")
wantBody(t, on, post(tc.header), tc.body, matched(942100), tc.body)
}
}
func TestFormDataAndMultipartAreReadUpToTheLimit(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
pad := strings.Repeat("a", bodyLimit)
for _, tc := range []struct{ header, attackFirst, attackLast string }{
{
formData, "q=" + url.QueryEscape(injection) + "&pad=" + pad,
"pad=" + pad + "&q=" + url.QueryEscape(injection),
},
{
multipart, field("q", injection) + field("pad", pad) + end,
field("pad", pad) + field("q", injection) + end,
},
} {
wantBody(t, crs, post(tc.header), tc.attackFirst, matched(942100),
tc.attackFirst[:bodyLimit+1])
wantBody(t, crs, post(tc.header), tc.attackLast, waf.Result{},
tc.attackLast[:bodyLimit+1])
}
// To the byte: a system file's path is found when it ends at the limit,
// and not when its last letter is past it, which is still read.
atLimit := padded("pad=", "&q=/etc/passwd", bodyLimit)
wantBody(t, crs, post(formData), atLimit, matched(930120, 932160), atLimit)
pastLimit := padded("pad=", "&q=/etc/passwd", bodyLimit+1)
wantBody(t, crs, post(formData), pastLimit, waf.Result{}, pastLimit)
}
func TestJSONAndXMLAreReadOnlyWhenNoLargerThanTheLimit(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
for _, tc := range []struct{ header, head, tail string }{
{jsonBody, `{"q":"` + injection + `","pad":"`, `"}`},
{xmlBody, "<r><q>" + injection + "</q><pad>", "</pad></r>"},
} {
fits := padded(tc.head, tc.tail, bodyLimit)
wantBody(t, crs, post(tc.header), fits, matched(942100), fits)
larger := padded(tc.head, tc.tail, bodyLimit+1)
wantBody(t, crs, post(tc.header), larger, waf.Result{}, larger)
}
}
func TestOtherBodiesAreNotRead(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
// Read as form data, which the Core Rule Set does with a body of a type
// it does not know, this would be an SQL injection.
body := "q=" + url.QueryEscape(injection)
for _, header := range []string{
"Content-Type: application/octet-stream",
"Content-Type: text/plain",
"Content-Type: application/x-git-receive-pack-request",
} {
wantBody(t, crs, post(header), body, waf.Result{}, "")
}
}
func TestContentEncodingIsRefusedOnTheKindsOfBodyTheCoreRuleSetReads(t *testing.T) {
t.Parallel()
const gzip = "Content-Encoding: gzip"
crs := readingBodies(t)
for _, tc := range []struct{ header, body string }{
{formData, "a=1"},
{multipart, field("a", "1") + end},
{jsonBody, `{"a":1}`},
{xmlBody, "<a>1</a>"},
} {
wantBody(t, crs, post(tc.header, gzip), tc.body, matched(920450), tc.body)
}
// Whatever its size: a JSON body larger than the limit is not read, but
// Content-Encoding on it is refused all the same.
larger := strings.Repeat("a", bodyLimit+1)
wantBody(t, crs, post(jsonBody, gzip), larger, matched(920450), larger)
// It is allowed on a body of any other kind, and on every body while no
// body is read.
fetch := "Content-Type: application/x-git-upload-pack-request"
wantBody(t, crs, post(fetch, gzip), "a", waf.Result{}, "")
wantBody(t, atDefaults(t), post(formData, gzip), "a", waf.Result{}, "")
}
func TestParametersGiteaSendsNamesInAreLeftOutAmongFormFieldsToo(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
local := url.QueryEscape("http://127.0.0.1:52341/")
for _, tc := range []struct {
body string
want waf.Result
}{
{"path=.gitignore", waf.Result{}},
{"q=.gitignore", matched(930120)},
{"redirect_uri=" + local, waf.Result{}},
{"next=" + local, matched(931100, 934110)},
} {
wantBody(t, crs, post(formData), tc.body, tc.want, tc.body)
}
body := field("path", ".gitignore") + end
wantBody(t, crs, post(multipart), body, waf.Result{}, body)
body = field("q", ".gitignore") + end
wantBody(t, crs, post(multipart), body, matched(930120), body)
// A JSON body's field is named by its path, here json.path, and is
// checked.
body = `{"path":".gitignore"}`
wantBody(t, crs, post(jsonBody), body, matched(930120), body)
}
func TestGiteaBodiesTheCoreRuleSetRefuses(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
// A comment that shows a shell command.
const text = "Try `curl -s https://example.org | sh` first."
comment := "content=" + url.QueryEscape(text)
wantBody(t, crs, post(formData), comment, matched(932235), comment)
// An attachment named like a log file.
attachment := "--b\r\nContent-Disposition: form-data; name=\"file\"; " +
"filename=\"debug.log\"\r\nContent-Type: text/plain\r\n\r\nstarted\r\n" + end
wantBody(t, crs, post(multipart), attachment, matched(932180), attachment)
}
func TestTypesEndingInXMLOrJSONAndTextJSONAreRead(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
for _, tc := range []struct{ contentType, body string }{
{"application/atom+xml", "<q>" + injection + "</q>"},
{"application/vnd.example+xml", "<q>" + injection + "</q>"},
{"application/vnd.example+json", `{"q":"` + injection + `"}`},
{"text/json", `{"q":"` + injection + `"}`},
} {
wantBody(t, crs, post("Content-Type: "+tc.contentType), tc.body,
matched(942100), tc.body)
}
}
func TestBodyCorazaCannotParseIsAMatch(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
// An end tag after the root element, past which Coraza reads none of
// the body, while an app may still read the attack before it.
body := "<q>" + injection + "</q></r>"
wantBody(t, crs, post(xmlBody), body, matched(900440), body)
// The multipart bodies Coraza cannot parse fail its strict checks too:
// one whose type names its boundary twice, and one with a part header
// that has no colon, before the attack. They do so padded past the
// limit too, which cuts them in the padding.
noColon := "--b\r\nContent-Disposition form-data; name=\"a\"\r\n\r\n1\r\n"
pad := field("pad", strings.Repeat("a", bodyLimit))
for _, tc := range []struct{ header, head string }{
{multipart + "; boundary=c", ""},
{multipart, noColon},
} {
body = tc.head + field("q", injection) + end
wantBody(t, crs, post(tc.header), body, matched(900440, 900450), body)
body = tc.head + field("q", injection) + pad + end
wantBody(t, crs, post(tc.header), body, matched(900440, 900450),
body[:bodyLimit+1])
}
}
func TestMultipartBodyCutBeforeAPartHeadersColonIsAMatch(t *testing.T) {
t.Parallel()
// The limit falls in the middle of the name of the second part's
// header, which Coraza, reading up to the limit, cannot tell from a
// header without a colon.
cut := "--b\r\nContent-Di"
first := field("pad", strings.Repeat("a", bodyLimit-len(field("pad", ""))-len(cut)))
body := first + cut + "sposition: form-data; name=\"q\"\r\n\r\n1\r\n" + end
wantBody(t, readingBodies(t), post(multipart), body, matched(900440, 900450),
body[:bodyLimit+1])
}
func TestMultipartBodyCutBeforeALineFeedInAPartsHeadersIsAMatch(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
headerLine := "--b\r\nContent-Disposition: form-data; name=\"q\"\r"
// The limit falls between the carriage return and the line feed that
// end the second part's header line, and then between those that end
// the empty line after it. Coraza, reading up to the limit, takes the
// line ending in a lone carriage return for a malformed header.
for _, cut := range []int{len(headerLine), len(headerLine + "\n\r")} {
first := field("pad", strings.Repeat("a", bodyLimit-len(field("pad", ""))-cut))
body := first + field("q", "1") + end
wantBody(t, crs, post(multipart), body, matched(900440, 900450),
body[:bodyLimit+1])
}
}
// TestCorazaWritesNoFile is not parallel, since it sets TMPDIR, the
// system's temporary directory, for the whole test process.
func TestCorazaWritesNoFile(t *testing.T) {
// The system's temporary directory is one that does not exist, so that
// Coraza could write nothing there: built without no_fs_access, it
// refuses to load, and could not write a file of a multipart body.
t.Setenv("TMPDIR", filepath.Join(t.TempDir(), "missing"))
body := "--b\r\nContent-Disposition: form-data; name=\"file\"; " +
"filename=\"notes.txt\"\r\nContent-Type: text/plain\r\n\r\n" +
strings.Repeat("a", 1000) + "\r\n" + field("q", injection) + end
wantBody(t, readingBodies(t), post(multipart), body, matched(942100), body)
}
func TestBodyLimitOf1GLoads(t *testing.T) {
t.Parallel()
_, err := waf.New(waf.Params{ParanoiaLevel: 1, BodyLimit: 1 << 30})
if err != nil {
t.Errorf("load the Core Rule Set reading bodies up to 1G: %v", err)
}
}
script/build
+2 -2
View File
@@ -1,7 +1,7 @@
#!/bin/sh
# script/build: build bin/smallwebwaf on the host, with Go installed, for
# working on the code by hand. The version it reports comes from git, as
# in script/docker, and the no_fs_access tag is the Dockerfile's.
# in script/docker.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
@@ -11,7 +11,7 @@ main() {
cd "$ROOT"
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
go build -tags no_fs_access -trimpath -ldflags "-X main.Version=$version" \
go build -trimpath -ldflags "-X main.Version=$version" \
-o bin/smallwebwaf ./cmd/smallwebwaf
}
script/gitea-requests
-329
View File
@@ -1,329 +0,0 @@
#!/bin/sh
# script/gitea-requests: build the image and run gitea 1.27.3 behind it,
# with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
# which names gitea's container. Then send through it the gitea requests
# that reading gitea's source found the Core Rule Set refuses, and a short
# sample of gitea's ordinary use: signing in, pages of files and commits,
# a diff, the API, a push, clone and fetch over HTTP, and two container
# image pushes. Last, print each request smallwebwaf logged, with what it
# did with it and the ids of the Core Rule Set's rules that matched it.
# Every request comes from one client, docker's gateway, so more than
# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
# the requests after would show as banned. The containers, the network, the images and the files the script makes
# are removed however it ends; gitea's own image stays, as the base images
# of a build do. It needs network access, and a docker daemon that pushes
# to a registry on localhost over plain HTTP, as docker does by default.
# script/check does not run this.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
# gitea 1.27.3, 2026-08-29
GITEA_IMAGE="gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef"
# Named after this run, so that runs in other clones on the same host
# never touch each other's.
NAME="$("$SCRIPT_DIR/projectname")-gitea-$$"
IMAGE="$NAME-image"
NETWORK="$NAME"
SMALLWEBWAF="$NAME-smallwebwaf"
GITEA="$NAME-gitea"
# gitea's one user, who owns the repository and the container images.
PASSWORD="owner-password"
BROWSER="Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
# The files the script makes, and smallwebwaf's address once it runs,
# localhost and the port docker publishes it on: a host name, since the
# Core Rule Set takes a Host that is an IP address for a warning (920350).
WORK=""
address=""
cleanup() {
docker rm --force --volumes "$SMALLWEBWAF" "$GITEA" >/dev/null 2>&1 || true
docker network rm "$NETWORK" >/dev/null 2>&1 || true
docker rmi --force "$IMAGE" >/dev/null 2>&1 || true
if [ -n "$address" ]; then
docker rmi --force "$address/owner/python3-app:1" "$address/owner/app:1" \
>/dev/null 2>&1 || true
fi
rm -rf "$WORK"
}
fail() {
echo "gitea-requests: $*" >&2
report
exit 1
}
# wait_for <what fails> <command>...: run the command every second until
# it succeeds, for at most a minute.
wait_for() {
failure="$1"
shift
tries=0
until "$@"; do
tries=$((tries + 1))
[ "$tries" -lt 60 ] || fail "$failure"
sleep 1
done
}
healthy() {
status="$(docker inspect --format '{{.State.Health.Status}}' "$SMALLWEBWAF")"
[ "$status" = healthy ]
}
# logged <text>...: a line of smallwebwaf's output holds every text, in
# any order.
logged() {
lines="$(docker logs "$SMALLWEBWAF" 2>&1)"
for text in "$@"; do
lines="$(printf '%s\n' "$lines" | grep -F "$text")" || return 1
done
}
# send <label> <path> <curl option>...: send a request for path to
# smallwebwaf, with label as its X-Request-ID, which the report gives,
# and keep the body of the answer in $WORK/answer. The Core Rule Set
# reads that header too, so a label names no file the cases name.
send() {
label="$1"
path="$2"
shift 2
curl --silent --show-error --max-time 30 --output "$WORK/answer" \
--header "X-Request-ID: $label" "$@" "http://$address$path"
}
# page <label> <path> <curl option>...: send the request as a browser
# does, with the cookies gitea has set. Its Referer is the repository's
# page, which links to the others, and never a page the Core Rule Set
# refused: its address would have the next request refused too.
page() {
label="$1"
path="$2"
shift 2
send "$label" "$path" --compressed --user-agent "$BROWSER" \
--header "Accept: text/html,application/xhtml+xml,*/*;q=0.8" \
--header "Accept-Language: en-US,en;q=0.5" \
--cookie "$WORK/cookies" --cookie-jar "$WORK/cookies" \
--referer "http://$address/owner/repo" "$@"
}
# api <label> <path> <curl option>...: send the request as a client of
# gitea's API does, as the owner.
api() {
label="$1"
path="$2"
shift 2
send "$label" "$path" --user "owner:$PASSWORD" \
--header "Accept: application/json" "$@"
}
# repo_git <git argument>...: run git as the owner, without the host's
# own git settings, so that nothing it does is kept outside $WORK.
repo_git() {
GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 git \
-c user.name=owner -c user.email=owner@example.invalid "$@"
}
# lines: forty lines of text, enough for a diff to hide some of them.
lines() {
i=1
while [ "$i" -le 40 ]; do
echo "line $i"
i=$((i + 1))
done
}
# make_repository: create the repository owner/repo, holding a file of
# each name the cases name, and push it, which is the push over HTTP of
# the sample. The second commit adds a line at the end of Runtime.java,
# so that its diff hides the lines above.
make_repository() {
status="$(api create-repository /api/v1/user/repos \
--header "Content-Type: application/json" --data '{"name":"repo"}' \
--write-out '%{http_code}')"
[ "$status" = 201 ] || fail "creating the repository was answered $status"
dir="$WORK/repo"
repo_git init --quiet --initial-branch=main "$dir"
for file in \
src/java.base/share/classes/java/lang/Runtime.java \
java.lang.String.md \
META-INF/services/org.apache.commons.logging.LogFactory \
document.write.js \
process.env.d.ts \
test/fixtures/__proto__.json \
install-script.sh \
src/backend/catalog/information_schema.sql \
.../notes.txt \
README.md; do
mkdir -p "$dir/$(dirname "$file")"
lines >"$dir/$file"
repo_git -C "$dir" add "$file"
done
repo_git -C "$dir" commit --quiet --message "Add the files"
echo "line 41" >>"$dir/src/java.base/share/classes/java/lang/Runtime.java"
repo_git -C "$dir" commit --quiet --all --message "Add a line"
repo_git -C "$dir" -c "http.extraHeader=X-Request-ID: git-push" push --quiet \
"http://owner:$PASSWORD@$address/owner/repo.git" main ||
fail "the push of the repository failed"
}
# browse: sign in, then open pages as a person does. The cases are those
# of https://git.eeqj.de/sneak/smallwebwaf/issues/30.
browse() {
# Case: gitea sends a visitor who is not signed in to the sign-in
# page, with the page asked for in redirect_to, when that page needs
# signing in, as a file's edit page does.
page issue-30-case /owner/repo/_edit/main/document.write.js --location
page sign-in /user/login
status="$(page sign-in /user/login --header "Origin: http://$address" \
--data-urlencode user_name=owner --data-urlencode "password=$PASSWORD" \
--write-out '%{http_code}')"
[ "$status" = 303 ] || fail "signing in was answered $status"
page browse /
page browse /owner/repo
page browse /owner/repo/src/branch/main/README.md
page browse /owner/repo/commits/branch/main
page browse "/owner/repo/commit/$(repo_git -C "$WORK/repo" rev-parse HEAD)"
# Case: expanding the lines the diff hides, by the link the diff's
# page gives for it.
excerpt="$(sed -n 's/.*data-fetch-url="\([^"]*blob_excerpt[^"]*\)".*/\1/p' \
"$WORK/answer" | head -n 1 | sed 's/&amp;/\&/g')"
[ -n "$excerpt" ] || fail "the diff's page has no link to its hidden lines"
page issue-30-case "$excerpt"
# Cases: the commit graph of one file, and the pages of files whose
# path the Core Rule Set takes for an attack.
page issue-30-case "/owner/repo/graph?file=src/java.base/share/classes/java/lang/Runtime.java"
page issue-30-case /owner/repo/src/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
page issue-30-case /owner/repo/src/branch/main/java.lang.String.md
page issue-30-case /owner/repo/src/branch/main/process.env.d.ts
page issue-30-case /owner/repo/src/branch/main/.../notes.txt
}
# call_api: the API's calls the cases name: the commits of a file, for a
# file of each name the Core Rule Set takes for an attack in the query
# parameter path, and the file-contents call, which takes its list of
# files as JSON in the query parameter body.
call_api() {
for file in \
src/java.base/share/classes/java/lang/Runtime.java \
java.lang.String.md \
document.write.js \
process.env.d.ts \
test/fixtures/__proto__.json \
install-script.sh \
src/backend/catalog/information_schema.sql; do
api issue-30-case /api/v1/repos/owner/repo/commits --get --data-urlencode "path=$file"
done
api issue-30-case /api/v1/repos/owner/repo/file-contents --get \
--data-urlencode 'body={"files":["src/java.base/share/classes/java/lang/Runtime.java"]}'
}
# push_images: push a container image to owner/python3-app, then the same
# image to owner/app. For the second, the case, docker asks gitea to take
# its layer from owner/python3-app, naming it in the query parameter
# from. docker sends no X-Request-ID, so its requests have the ids
# smallwebwaf gives them. The image is one small file, imported rather
# than built.
push_images() {
echo "$NAME" >"$WORK/layer.txt"
tar -cf - -C "$WORK" layer.txt |
docker import - "$address/owner/python3-app:1" >/dev/null
docker tag "$address/owner/python3-app:1" "$address/owner/app:1"
echo "$PASSWORD" | DOCKER_CONFIG="$WORK/docker" docker login --username owner \
--password-stdin "$address" >/dev/null 2>&1 || fail "docker login failed"
for image in "$address/owner/python3-app:1" "$address/owner/app:1"; do
DOCKER_CONFIG="$WORK/docker" docker push --quiet "$image" >/dev/null ||
echo "gitea-requests: the push of $image failed" >&2
done
}
# clone_and_fetch: the clone and fetch over HTTP of the sample.
clone_and_fetch() {
repo_git -c "http.extraHeader=X-Request-ID: git-clone" clone --quiet \
"http://owner:$PASSWORD@$address/owner/repo.git" "$WORK/clone" ||
fail "the clone failed"
repo_git -C "$WORK/clone" -c "http.extraHeader=X-Request-ID: git-fetch" fetch --quiet ||
fail "the fetch failed"
}
# field <name> <line>: the text of the field name in line, a line of the
# request log, or nothing when it has none.
field() {
printf '%s\n' "$2" | sed -n "s/.*\"$1\":\"\([^\"]*\)\".*/\1/p"
}
# report: print a line for each request smallwebwaf logged, but its
# health checks: what it did with the request, the ids of the Core Rule
# Set's rules that matched it, its X-Request-ID, and the request.
# waf_blocked is a refusal by the Core Rule Set, and forward a request
# passed on to gitea, whatever gitea answered.
report() {
docker logs "$SMALLWEBWAF" 2>/dev/null | grep '"type":"request"' |
grep -v '"action":"admin"' | sed 's/\\u0026/\&/g' >"$WORK/requests"
while IFS= read -r line; do
ids="$(printf '%s\n' "$line" | sed -n 's/.*"waf_rule_ids":\[\([0-9,]*\)\].*/\1/p')"
query="$(field query "$line")"
printf '%-11s %-13s %-17s %s %s%s\n' "$(field action "$line")" "${ids:--}" \
"$(field request_id "$line")" "$(field method "$line")" \
"$(field path "$line")" "${query:+?$query}"
done <"$WORK/requests"
echo "gitea-requests: $(grep -c . "$WORK/requests") requests," \
"$(grep -c '"action":"waf_blocked"' "$WORK/requests") refused by the Core Rule Set"
}
main() {
cd "$ROOT"
trap cleanup EXIT
trap 'exit 1' HUP INT TERM
WORK="$(mktemp -d)"
docker build --no-cache -t "$IMAGE" .
# smallwebwaf first, so that gitea can be given the address it is
# reached at, and gitea's image pulled before, so that gitea starts
# within the minute smallwebwaf's health check waits for it.
docker pull --quiet "$GITEA_IMAGE" >/dev/null
docker network create "$NETWORK" >/dev/null
docker run --detach --name "$SMALLWEBWAF" --network "$NETWORK" \
--publish 127.0.0.1::8080 \
--env SWWAF_UPSTREAM_URL="http://$GITEA:3000" \
"$IMAGE" >/dev/null
published="$(docker port "$SMALLWEBWAF" 8080/tcp)"
address="localhost:${published##*:}"
docker run --detach --name "$GITEA" --network "$NETWORK" \
--env GITEA__security__INSTALL_LOCK=true \
--env GITEA__server__ROOT_URL="http://$address/" \
"$GITEA_IMAGE" >/dev/null
# smallwebwaf's health check passes once gitea takes connections.
wait_for "the health check did not pass" healthy
docker exec --user git "$GITEA" gitea admin user create --admin \
--username owner --password "$PASSWORD" --email owner@example.invalid \
--must-change-password=false >/dev/null
make_repository
browse
call_api
push_images
clone_and_fetch
# The fetch's last request is the last of all; once it is logged, so
# is every other.
wait_for "smallwebwaf logged no line for the fetch" \
logged '"request_id":"git-fetch"' '"method":"POST"'
report
}
main "$@"