Author SHA1 Message Date
clawbot fbe87a7c5e Core Rule Set reads request bodies up to SWWAF_WAF_BODY_LIMIT (closes #116)
check / check (push) Canceled after 0s
SWWAF_WAF_BODY_LIMIT (default off, at most 1G) has the Core Rule Set read
form data and multipart up to the limit, the rest streaming on, and JSON
and XML no larger than it, with text/json and the application and text
types ending in +json or +xml. The part read is held for the app. A size
or time limit met while reading ends the request. Content-Encoding is
refused again on these kinds. A body Coraza cannot parse, or a multipart
body failing its strict checks, adds 5, as does a multipart body the limit
cuts before the colon of a part's header. Coraza is built with
no_fs_access, so writes no file. Rule 900300 moves to phase 2.

Judgement call: Content-Encoding is refused on a JSON or XML body too
large to read, as SPEC.md allows.

Model: opus-5-5
2026-10-08 10:04:58 +00:00
14 changed files with 64 additions and 578 deletions
+2 -6
View File
@@ -1,12 +1,11 @@
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
run example-app gitea-requests
run example-app
# Makefile targets are thin shims; the implementations live in script/
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
# of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
# which test checks. build and run are for working on the code by hand;
# example-app checks the image with an app built on it; gitea-requests
# reports the gitea requests the Core Rule Set refuses at the defaults.
# example-app checks the image with an app built on it.
bootstrap:
@script/bootstrap
@@ -46,6 +45,3 @@ run:
example-app:
@script/example-app
gitea-requests:
@script/gitea-requests
+26 -43
View File
@@ -197,12 +197,11 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
rate limits and before the rule files, which it does not need. A request for
one is refused with `SWWAF_BAN_RESPONSE` and bans the client's netblock for a
clear sign of attack, as a `ban` rule's match does. A trap path is matched
against the path a `path` rule sees: without the query, as the client sent it
and once percent-decoded, the whole of it, character for character.
`/wp-login.php` matches `/wp-login.php?redirect_to=x` and `/%77p-login.php`,
but not `/wp-login.php/`, `/WP-LOGIN.PHP`, `/blog/wp-login.php` or
`/%2577p-login.php`. The ban's notes give the path as the client sent it. A
client in `SWWAF_ALLOW_NETS` is not checked.
against the path a `path` rule sees: as the client sent it, before any
decoding and without the query, the whole of it, character for character.
`/wp-login.php` matches `/wp-login.php?redirect_to=x`, but not
`/wp-login.php/`, `/WP-LOGIN.PHP`, `/blog/wp-login.php` or `/%77p-login.php`.
A client in `SWWAF_ALLOW_NETS` is not checked.
- Inspects each request with the OWASP Core Rule Set 4.25.0, run by Coraza,
after the rule files, unless `SWWAF_WAF_MODE` is `off`: its method, its URL
with the query, and its headers, and, once `SWWAF_WAF_BODY_LIMIT` is set, its
@@ -725,25 +724,23 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
(rule 900440), and a multipart body that fails Coraza's strict checks (rule
900450), add 5 to the score, as a rule rated critical does, since no rule
reads what comes after the fault. So does a multipart body the limit cuts
before the colon of a part's header line, or between the carriage return and
the line feed that end a part's header line or the empty line after its
headers, since Coraza takes the line the limit cuts for a malformed header.
The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out to send the part
that is read, and a request whose body passes `SWWAF_REQUEST_MAX_BYTES` within
it is refused before anything reaches the app. Of a file in a multipart body,
Coraza counts the bytes and writes nothing. Body inspection suits apps whose
forms carry no code. In front of gitea it refuses issue and comment text, wiki
pages and files saved in the web editor that hold shell commands or code
(932125, 932235, 932250 and others), package descriptions that show code, PyPI
uploads (922130), and attachments named like `debug.log` or `config.yml`
(932180), until the rule ids the request log names are added to
`SWWAF_WAF_DISABLED_RULES`.
before the colon of a part's header line, which Coraza cannot tell from a
header without one. The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs
out to send the part that is read, and a request whose body passes
`SWWAF_REQUEST_MAX_BYTES` within it is refused before anything reaches the
app. Of a file in a multipart body, Coraza counts the bytes and writes
nothing. Body inspection suits apps whose forms carry no code. In front of
gitea it refuses issue and comment text, wiki pages and files saved in the web
editor that hold shell commands or code (932125, 932235, 932250 and others),
package descriptions that show code, PyPI uploads (922130), and attachments
named like `debug.log` or `config.yml` (932180), until the rule ids the
request log names are added to `SWWAF_WAF_DISABLED_RULES`.
- `SWWAF_TRAP_PATHS` (default empty): paths the app never serves and only
scanners ask for, such as `/wp-login.php,/xmlrpc.php` in front of gitea; a
request for one bans its client for a clear sign of attack, as a `ban` rule
does, with or without rule files (see "What it does so far" above). A path
that does not start with `/`, or holds a `?`, as one written with a query
does, stops the start.
that does not start with `/`, or holds a `?`, which no path a `path` rule sees
holds, stops the start.
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most requests of a client in
a minute that `smallwebwaf` may refuse after a rule file match, a trap path or
a Core Rule Set match, or for a missing or wrong token; one more breaks the
@@ -1523,15 +1520,14 @@ scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
- The id is letters, digits, `-` and `_`, and no two rules share one. The
request log, the metrics and a ban's notes name the rule by it.
- The target is what the regex is matched against: `path`, the path without the
query, and `uri`, the path and the query together, each both as sent and once
percent-decoded, so that an encoded probe does not slip past; `query`, as the
client sent it, before any decoding; `method`; `host`; `user_agent`;
`referer`; or `header:<Name>`, any one request header but `Host` and
`Transfer-Encoding`, which Go's HTTP server takes out of every request; the
request's host is the target `host`. A header sent more than once is matched
with its values joined by `, `, and one not sent as empty text. No body is
read.
- The target is what the regex is matched against: `path` or `query`, as the
client sent it, before any decoding; `uri`, the path and the query together,
both as sent and once percent-decoded, so that an encoded probe does not slip
past; `method`; `host`; `user_agent`; `referer`; or `header:<Name>`, any one
request header but `Host` and `Transfer-Encoding`, which Go's HTTP server
takes out of every request; the request's host is the target `host`. A header
sent more than once is matched with its values joined by `, `, and one not
sent as empty text. No body is read.
- The action is `log`, `block` or `ban` (see "What it does so far" above). Keep
`ban` for requests no real visitor sends, and anchor a path at the site root
with `^/`: a file of the same name deeper in a site can be ordinary content,
@@ -2374,19 +2370,6 @@ so that they run in minimal containers.
the same volume still refuses the banned client; then removes the containers,
the volume and both images. It needs network access, for nixpkgs' binary
cache, and `script/check` does not run it; `make example-app` does.
- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind it,
with every setting of `smallwebwaf` at its default but `SWWAF_UPSTREAM_URL`,
which names gitea's container. It sends through it the gitea requests that the
notes specific to gitea in [`SPEC.md`](SPEC.md) name for a file name in a
query parameter, for a path and for a container image push, and a short sample
of gitea's ordinary use: signing in, pages of files and commits, a diff, the
API, a push, clone and fetch over HTTP, and two container image pushes. It
prints a line for each request `smallwebwaf` logged, with what it did with it
and the ids of the Core Rule Set's rules that matched it, then removes the
containers, the network and the images it made, but gitea's own image. It
needs network access, and a docker daemon that pushes to a registry on
`localhost` over plain HTTP, as docker does by default; `script/check` does
not run it, and `make gitea-requests` does.
## TODO
+5 -42
View File
@@ -825,8 +825,7 @@ a file edited, added or removed there takes effect while it runs.
- `id`: a short name of letters, digits, `-` and `_`, unique across all files.
It appears in the request log, metrics, alerts and ban notes.
- `target`: what the regex is matched against.
- `path`: the URL path without the query, both as received and once
percent-decoded, so an encoded probe cannot slip past.
- `path`: the URL path as received, before any decoding.
- `query`: the raw query string.
- `uri`: path and query together, both as received and once percent-decoded,
so an encoded probe cannot slip past.
@@ -1477,42 +1476,8 @@ holds any token file.
Saving such a rule stores it, but the branch settings page gitea then
returns to gets the 403 answer of `smallwebwaf`, so the save looks as
if it failed.
- A file name in a query parameter, those the fourth change leaves out
of 930120, 932160 and 932260 included, in which the Core Rule Set's
rules for Java, script, PowerShell and SQL find what they look for:
`runtime` in OpenJDK's
`src/java.base/share/classes/java/lang/Runtime.java` (944110), a Java
class, as in `java.lang.String.md` (944130), `document.write.js`
(941180), `process.env.d.ts` (934100), `__proto__` (934130), the
PowerShell command `Install-Script`, as in `install-script.sh`
(932120), or a database's name, as in PostgreSQL's
`information_schema.sql` (942140). For such a file, the API's list of
its commits (`path`) is refused, and so are the API's call for the
contents of files that names it in its query (`body`), the commit
graph of that file alone (`file`), and the hidden lines of its diff
(`path`), which stay hidden. A visitor whom gitea sends to sign in on
the way to a page of such a file that needs signing in, such as its
edit page, gets the 403 answer of `smallwebwaf` in place of the
sign-in page (`redirect_to`), and can still sign in by opening the
sign-in page itself.
- A path: a file name ending in `~`, or an `.xhtml` file whose path
holds a space.
- A path that holds what the Core Rule Set takes for a Java class
(944130), as `java.lang.String.md` and
`META-INF/services/org.apache.commons.logging.LogFactory` do, or for
Node.js code (934100), as `process.env.d.ts` does, or that has a
directory named `...` (930100, 930110), such as `.../notes.txt`. The
page of such a file, its raw file, its history, its blame and its edit
page are refused; the directory that holds it lists it, and a clone
carries it.
- A container image push that reuses a layer of an image docker has
pushed to that gitea before, when that image's name, after its owner,
starts with an entry on the list of commands (932260), such as
`owner/python3-app` or `owner/docker-base`. docker asks gitea to take
the layer from that image, naming it in the query parameter `from`,
which 932260 checks. The push fails with the 403 answer of
`smallwebwaf`, and so does each new try, since docker asks in the same
way.
- Artifact uploads from `actions/upload-artifact@v3` send the header
`Content-Range`, which the Core Rule Set refuses (920450). The action
sends two files at a time, does not retry a 403 and sends no further file
@@ -1600,12 +1565,10 @@ holds any token file.
- Core Rule Set false positives against real apps (a search for code, and any
body once `SWWAF_WAF_BODY_LIMIT` is set): a match refuses only that request
and bans no one by itself; the request log names the rule, and exclusions by
rule id and path fix it. `script/gitea-requests` runs gitea behind the image
at the defaults, sends through it the requests the notes specific to gitea
name for a file name in a query parameter, for a path and for a container
image push, and a short sample of gitea's ordinary use, and reports each
request the Core Rule Set refuses, so that those notes can be checked again
when gitea or the Core Rule Set changes version.
rule id and path fix it. Further gitea requests the Core Rule Set may refuse
at the defaults, found by reading gitea's source rather than a running gitea,
are collected in https://git.eeqj.de/sneak/smallwebwaf/issues/30 and checked
when milestone 1 runs in front of a real gitea.
- Attacks carried in request bodies: not refused by default, since on a code
forge bodies are full of code the Core Rule Set takes for attacks. Most of
what shows in URLs and headers is still refused, the client that sends an
+1 -2
View File
@@ -123,8 +123,7 @@ type Notes struct {
LimitPercentSetting string `json:"limit_percent_setting,omitempty"`
// RuleID and Target are, for a ban for a clear sign of attack, the id
// of the rule file rule that matched, and its target; TrapPath is, for
// one for a request for a path in SWWAF_TRAP_PATHS, that path as the
// client sent it.
// one for a request for a path in SWWAF_TRAP_PATHS, that path.
RuleID string `json:"rule_id,omitempty"`
Target string `json:"target,omitempty"`
TrapPath string `json:"trap_path,omitempty"`
+1 -2
View File
@@ -1665,8 +1665,7 @@ func parsePathPrefixes(value string) ([]string, error) {
// parseTrapPaths reads a comma-separated list of trap paths. Each is
// matched against a request's path as a path rule is, without the query,
// so a path that does not start with / would never match, and one that
// holds a ? is written with a query.
// so a path that does not start with / or holds a ? would never match.
func parseTrapPaths(value string) ([]string, error) {
paths, err := parseList(value)
if err != nil {
-69
View File
@@ -86,75 +86,6 @@ func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(t *testing.T)
}
}
func TestRequestThatWaitedForItsLookupIsCountedWhenItGoesOn(t *testing.T) {
t.Parallel()
// The country list has a new client's request wait for its answer,
// which the stand-in for GeoJS gives only once released.
geojsURL, asked, release := startHeldGeoJS(t)
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 58, 0, time.UTC)}
addr, out, server := startProxyWithClock(t, app.URL, geojsURL, clk.Now,
map[string]string{
trustedProxies: trustLocalhost,
lookupTimeout: "1h",
deniedCountries: "kp",
})
requestFromDE := func() *http.Request {
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Header.Set(forwardedFor, fromDE)
return req
}
// fromDE's first request arrives two seconds before a minute ends,
// and waits for GeoJS.
first, httpClient := requestFromDE(), newClient(t)
firstEnded := make(chan error, 1)
go func() {
res, err := httpClient.Do(first)
if err == nil {
err = readAnswer(res).err
}
firstEnded <- err
}()
waitUntil(func() bool { return len(asked()) == 1 })
// Half a second into the next minute, the answer is kept, as an edit
// of lookups.json would keep it, while the first request still waits
// for GeoJS. The second request goes on at once, and is counted.
clk.advance(2500 * time.Millisecond)
server.GeoJS.Load([]lookup.Answer{{
Client: netip.MustParsePrefix(fromDE + "/32"), ASN: asnDE, ASName: asNameDE,
Country: "DE", Answered: clk.Now(), Used: clk.Now(),
}})
wantStatus(t, do(t, requestFromDE()), http.StatusOK)
// GeoJS answers, and the first request goes on.
release()
err := <-firstEnded
if err != nil {
t.Fatalf("first request: %v", err)
}
wantLine(t, out.requestLines(t, 2)[1], http.StatusOK, requestlog.ActionForward)
// The third request's count holds all three: the first was counted
// when it went on, in the minute under way, and the second's count
// was kept.
clk.advance(500 * time.Millisecond)
wantStatus(t, do(t, requestFromDE()), http.StatusOK)
if got := out.requestLines(t, 3)[2].Counts.Minute; got != 3 {
t.Errorf("the third request's count in the minute is %v, want 3", got)
}
}
func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert(t *testing.T) {
t.Parallel()
-6
View File
@@ -261,12 +261,6 @@ func (rq *request) checkClient(ctx context.Context) string {
rq.lookUp(ctx)
// The lookup can wait for GeoJS, so the checks after it read the time
// again: a request counted at the time it arrived could be counted
// after a later one of its client's, which the rate limits would take
// for the clock set back.
now = rq.h.now()
if rq.countryDenied() {
return requestlog.ActionCountryDenied
}
+5 -7
View File
@@ -10,15 +10,13 @@ import (
)
// trapPath reports whether the request asks for a path in
// SWWAF_TRAP_PATHS: its path as a path rule sees it, without the query,
// as sent or once percent-decoded, is one of them. Such a request is a
// clear sign of attack, as a ban rule's match is: it bans the client's
// netblock, or in observe mode raises the alert for the ban it would have
// made. The ban's notes give the path as sent.
// SWWAF_TRAP_PATHS: its path as a path rule sees it, before any decoding
// and without the query, is one of them. Such a request is a clear sign of
// attack, as a ban rule's match is: it bans the client's netblock, or in
// observe mode raises the alert for the ban it would have made.
func (rq *request) trapPath(now time.Time) bool {
path := rules.Path(rq.in)
if !slices.Contains(rq.h.config.TrapPaths, path) &&
!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
if !slices.Contains(rq.h.config.TrapPaths, path) {
return false
}
+4 -19
View File
@@ -32,9 +32,10 @@ func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
})
start := clk.Now()
// Only the path itself, the whole of it in the same case, is a trap
// path.
for _, path := range []string{"/wp-login.php/", "/WP-LOGIN.PHP", "/blog/xmlrpc.php"} {
// Only the path itself, as the client sent it, is a trap path.
for _, path := range []string{
"/wp-login.php/", "/WP-LOGIN.PHP", "/blog/xmlrpc.php", "/%77p-login.php",
} {
s.request(otherClient, path, http.StatusOK, requestlog.ActionForward)
}
@@ -83,22 +84,6 @@ func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
}
}
func TestTrapPathMatchesThePathDecodedOnce(t *testing.T) {
t.Parallel()
s, _, server := startWithClock(t, "", map[string]string{trapPaths: trapPathList})
// %2577 decodes once to %77, not to w.
s.request(otherClient, "/%2577p-login.php", http.StatusOK, requestlog.ActionForward)
s.request(client, "/%77p-login.php", http.StatusForbidden, requestlog.ActionBanned)
// The ban's notes give the path as the client sent it.
got := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
if len(got) != 1 || got[0].Notes.TrapPath != "/%77p-login.php" {
t.Fatalf("bans %+v, want one whose notes give the trap path /%%77p-login.php", got)
}
}
func TestTrapPathsNeedNoRuleFiles(t *testing.T) {
t.Parallel()
+16 -26
View File
@@ -50,13 +50,6 @@ const quietTime = 2 * time.Second
// header:<Name>.
const headerTarget = "header:"
// pathTarget and uriTarget are the targets matched both as received and
// once percent-decoded: the path, and the path and query together.
const (
pathTarget = "path"
uriTarget = "uri"
)
// escapeLength is the length of a percent escape, such as %2e.
const escapeLength = 3
@@ -381,7 +374,7 @@ func parse(line string) (Rule, bool, error) {
// isTarget reports whether target is one a rule may have.
func isTarget(target string) bool {
switch target {
case pathTarget, "query", uriTarget, "method", "host", "user_agent", "referer":
case "path", "query", "uri", "method", "host", "user_agent", "referer":
return true
}
@@ -391,38 +384,35 @@ func isTarget(target string) bool {
}
// matches reports whether the rule's regex matches its target in r. For
// path and uri it is matched against the target as received, and against
// it once percent-decoded, so that an encoded probe cannot slip past.
// uri it is matched against the path and query as received, and against
// them once percent-decoded, so that an encoded probe cannot slip past.
func (rule Rule) matches(r *http.Request) bool {
target := value(rule.Target, r)
if rule.Target == "uri" {
uri := pathAndQuery(r)
if rule.Target == pathTarget || rule.Target == uriTarget {
return rule.regex.MatchString(target) || rule.regex.MatchString(DecodeOnce(target))
return rule.regex.MatchString(uri) || rule.regex.MatchString(decodeOnce(uri))
}
return rule.regex.MatchString(target)
return rule.regex.MatchString(value(rule.Target, r))
}
// Path returns r's path as the client sent it, before any decoding or
// re-encoding, up to the first ?: what a path rule is matched against, as
// it is and once percent-decoded.
// re-encoding, up to the first ?: what a path rule is matched against.
func Path(r *http.Request) string {
path, _, _ := strings.Cut(pathAndQuery(r), "?")
return path
}
// value returns what a rule with target is matched against in r: the
// path, as Path gives it, the path and query, as pathAndQuery gives
// them, and the query as the client sent it, before any decoding or
// re-encoding, after the first ?, and a header's values joined by ", ",
// as HTTP joins those of a header sent more than once.
// value returns what a rule with target, other than uri, is matched
// against in r: the path, as Path gives it, and the query as the client
// sent it, before any decoding or re-encoding, after the first ?, and a
// header's values joined by ", ", as HTTP joins those of a header sent
// more than once.
func value(target string, r *http.Request) string {
switch target {
case pathTarget:
case "path":
return Path(r)
case uriTarget:
return pathAndQuery(r)
case "query":
_, query, _ := strings.Cut(pathAndQuery(r), "?")
@@ -474,11 +464,11 @@ func header(r *http.Request, name string) string {
return strings.Join(r.Header.Values(name), ", ")
}
// DecodeOnce returns s with each percent escape, such as %2e, replaced by
// decodeOnce returns s with each percent escape, such as %2e, replaced by
// the byte it stands for. A % that is not followed by two hex digits is
// left as it is, so that a malformed escape cannot keep the rest of s
// from being decoded.
func DecodeOnce(s string) string {
func decodeOnce(s string) string {
var decoded strings.Builder
for i := 0; i < len(s); i++ {
+2 -4
View File
@@ -49,9 +49,7 @@ func TestEachTargetMatchesWhatItNames(t *testing.T) {
want bool
}{
{"path as received", `path log ^/%2eenv$`, "/%2eenv", nil, true},
{"path decoded", `path log ^/\.env$`, "/%2Eenv", nil, true},
{"path decoded only once", `path log ^/\.env$`, "/%252eenv", nil, false},
{"path decoded without the query", `path log ^/a\?b$`, "/a%3fb?c", nil, true},
{"path not decoded", `path log ^/\.env$`, "/%2eenv", nil, false},
{"path without the query", `path log ^/a$`, "/a?b=c", nil, true},
{"query as received", `query log ^b=%2e$`, "/a?b=%2e", nil, true},
{"uri as received", `uri log ^/a\?b=%2e$`, "/a?b=%2e", nil, true},
@@ -420,7 +418,7 @@ func TestDefaultFileBansProbesAtTheSiteRootAlone(t *testing.T) {
// Probes sent by a browser, by the rule that refuses them.
for rule, targets := range map[string][]string{
"env-file": {"/.env", "/.env.production", "/.ENV", "/%2eenv"},
"env-file": {"/.env", "/.env.production", "/.ENV"},
"vcs-dir": {"/.git/config", "/.git", "/.svn/entries"},
"secrets-dir": {"/.aws/credentials", "/.ssh/id_rsa"},
"secret-file": {"/.htpasswd", "/.DS_Store", "/.git-credentials"},
+2 -4
View File
@@ -134,10 +134,8 @@ SecRule REQBODY_PROCESSOR "@rx ^(?:URLENCODED|MULTIPART|JSON|XML)$" \
# Rule Set rates critical does: no rule reads what comes after the fault,
# which the app may still read. Coraza's recommended configuration refuses
# them in its rules 200002 and 200003. A multipart body the limit cuts
# before the colon of a part's header line, or between the carriage return
# and the line feed that end a part's header line or the empty line after
# its headers, adds 5 too, since Coraza takes the line the limit cuts for a
# malformed header. Coraza parses any form data body.
# before the colon of a part's header line adds 5 too, since Coraza cannot
# tell it from a header without one. Coraza parses any form data body.
SecRule REQBODY_ERROR "!@eq 0" "id:900440,phase:2,pass,severity:'CRITICAL',\
setvar:'tx.inbound_anomaly_score_pl1=+5'"
SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
-19
View File
@@ -645,25 +645,6 @@ func TestMultipartBodyCutBeforeAPartHeadersColonIsAMatch(t *testing.T) {
body[:bodyLimit+1])
}
func TestMultipartBodyCutBeforeALineFeedInAPartsHeadersIsAMatch(t *testing.T) {
t.Parallel()
crs := readingBodies(t)
headerLine := "--b\r\nContent-Disposition: form-data; name=\"q\"\r"
// The limit falls between the carriage return and the line feed that
// end the second part's header line, and then between those that end
// the empty line after it. Coraza, reading up to the limit, takes the
// line ending in a lone carriage return for a malformed header.
for _, cut := range []int{len(headerLine), len(headerLine + "\n\r")} {
first := field("pad", strings.Repeat("a", bodyLimit-len(field("pad", ""))-cut))
body := first + field("q", "1") + end
wantBody(t, crs, post(multipart), body, matched(900440, 900450),
body[:bodyLimit+1])
}
}
// TestCorazaWritesNoFile is not parallel, since it sets TMPDIR, the
// system's temporary directory, for the whole test process.
func TestCorazaWritesNoFile(t *testing.T) {
-329
View File
@@ -1,329 +0,0 @@
#!/bin/sh
# script/gitea-requests: build the image and run gitea 1.27.3 behind it,
# with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
# which names gitea's container. Then send through it the gitea requests
# that reading gitea's source found the Core Rule Set refuses, and a short
# sample of gitea's ordinary use: signing in, pages of files and commits,
# a diff, the API, a push, clone and fetch over HTTP, and two container
# image pushes. Last, print each request smallwebwaf logged, with what it
# did with it and the ids of the Core Rule Set's rules that matched it.
# Every request comes from one client, docker's gateway, so more than
# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
# the requests after would show as banned. The containers, the network, the images and the files the script makes
# are removed however it ends; gitea's own image stays, as the base images
# of a build do. It needs network access, and a docker daemon that pushes
# to a registry on localhost over plain HTTP, as docker does by default.
# script/check does not run this.
set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
# gitea 1.27.3, 2026-08-29
GITEA_IMAGE="gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef"
# Named after this run, so that runs in other clones on the same host
# never touch each other's.
NAME="$("$SCRIPT_DIR/projectname")-gitea-$$"
IMAGE="$NAME-image"
NETWORK="$NAME"
SMALLWEBWAF="$NAME-smallwebwaf"
GITEA="$NAME-gitea"
# gitea's one user, who owns the repository and the container images.
PASSWORD="owner-password"
BROWSER="Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
# The files the script makes, and smallwebwaf's address once it runs,
# localhost and the port docker publishes it on: a host name, since the
# Core Rule Set takes a Host that is an IP address for a warning (920350).
WORK=""
address=""
cleanup() {
docker rm --force --volumes "$SMALLWEBWAF" "$GITEA" >/dev/null 2>&1 || true
docker network rm "$NETWORK" >/dev/null 2>&1 || true
docker rmi --force "$IMAGE" >/dev/null 2>&1 || true
if [ -n "$address" ]; then
docker rmi --force "$address/owner/python3-app:1" "$address/owner/app:1" \
>/dev/null 2>&1 || true
fi
rm -rf "$WORK"
}
fail() {
echo "gitea-requests: $*" >&2
report
exit 1
}
# wait_for <what fails> <command>...: run the command every second until
# it succeeds, for at most a minute.
wait_for() {
failure="$1"
shift
tries=0
until "$@"; do
tries=$((tries + 1))
[ "$tries" -lt 60 ] || fail "$failure"
sleep 1
done
}
healthy() {
status="$(docker inspect --format '{{.State.Health.Status}}' "$SMALLWEBWAF")"
[ "$status" = healthy ]
}
# logged <text>...: a line of smallwebwaf's output holds every text, in
# any order.
logged() {
lines="$(docker logs "$SMALLWEBWAF" 2>&1)"
for text in "$@"; do
lines="$(printf '%s\n' "$lines" | grep -F "$text")" || return 1
done
}
# send <label> <path> <curl option>...: send a request for path to
# smallwebwaf, with label as its X-Request-ID, which the report gives,
# and keep the body of the answer in $WORK/answer. The Core Rule Set
# reads that header too, so a label names no file the cases name.
send() {
label="$1"
path="$2"
shift 2
curl --silent --show-error --max-time 30 --output "$WORK/answer" \
--header "X-Request-ID: $label" "$@" "http://$address$path"
}
# page <label> <path> <curl option>...: send the request as a browser
# does, with the cookies gitea has set. Its Referer is the repository's
# page, which links to the others, and never a page the Core Rule Set
# refused: its address would have the next request refused too.
page() {
label="$1"
path="$2"
shift 2
send "$label" "$path" --compressed --user-agent "$BROWSER" \
--header "Accept: text/html,application/xhtml+xml,*/*;q=0.8" \
--header "Accept-Language: en-US,en;q=0.5" \
--cookie "$WORK/cookies" --cookie-jar "$WORK/cookies" \
--referer "http://$address/owner/repo" "$@"
}
# api <label> <path> <curl option>...: send the request as a client of
# gitea's API does, as the owner.
api() {
label="$1"
path="$2"
shift 2
send "$label" "$path" --user "owner:$PASSWORD" \
--header "Accept: application/json" "$@"
}
# repo_git <git argument>...: run git as the owner, without the host's
# own git settings, so that nothing it does is kept outside $WORK.
repo_git() {
GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 git \
-c user.name=owner -c user.email=owner@example.invalid "$@"
}
# lines: forty lines of text, enough for a diff to hide some of them.
lines() {
i=1
while [ "$i" -le 40 ]; do
echo "line $i"
i=$((i + 1))
done
}
# make_repository: create the repository owner/repo, holding a file of
# each name the cases name, and push it, which is the push over HTTP of
# the sample. The second commit adds a line at the end of Runtime.java,
# so that its diff hides the lines above.
make_repository() {
status="$(api create-repository /api/v1/user/repos \
--header "Content-Type: application/json" --data '{"name":"repo"}' \
--write-out '%{http_code}')"
[ "$status" = 201 ] || fail "creating the repository was answered $status"
dir="$WORK/repo"
repo_git init --quiet --initial-branch=main "$dir"
for file in \
src/java.base/share/classes/java/lang/Runtime.java \
java.lang.String.md \
META-INF/services/org.apache.commons.logging.LogFactory \
document.write.js \
process.env.d.ts \
test/fixtures/__proto__.json \
install-script.sh \
src/backend/catalog/information_schema.sql \
.../notes.txt \
README.md; do
mkdir -p "$dir/$(dirname "$file")"
lines >"$dir/$file"
repo_git -C "$dir" add "$file"
done
repo_git -C "$dir" commit --quiet --message "Add the files"
echo "line 41" >>"$dir/src/java.base/share/classes/java/lang/Runtime.java"
repo_git -C "$dir" commit --quiet --all --message "Add a line"
repo_git -C "$dir" -c "http.extraHeader=X-Request-ID: git-push" push --quiet \
"http://owner:$PASSWORD@$address/owner/repo.git" main ||
fail "the push of the repository failed"
}
# browse: sign in, then open pages as a person does. The cases are those
# of https://git.eeqj.de/sneak/smallwebwaf/issues/30.
browse() {
# Case: gitea sends a visitor who is not signed in to the sign-in
# page, with the page asked for in redirect_to, when that page needs
# signing in, as a file's edit page does.
page issue-30-case /owner/repo/_edit/main/document.write.js --location
page sign-in /user/login
status="$(page sign-in /user/login --header "Origin: http://$address" \
--data-urlencode user_name=owner --data-urlencode "password=$PASSWORD" \
--write-out '%{http_code}')"
[ "$status" = 303 ] || fail "signing in was answered $status"
page browse /
page browse /owner/repo
page browse /owner/repo/src/branch/main/README.md
page browse /owner/repo/commits/branch/main
page browse "/owner/repo/commit/$(repo_git -C "$WORK/repo" rev-parse HEAD)"
# Case: expanding the lines the diff hides, by the link the diff's
# page gives for it.
excerpt="$(sed -n 's/.*data-fetch-url="\([^"]*blob_excerpt[^"]*\)".*/\1/p' \
"$WORK/answer" | head -n 1 | sed 's/&amp;/\&/g')"
[ -n "$excerpt" ] || fail "the diff's page has no link to its hidden lines"
page issue-30-case "$excerpt"
# Cases: the commit graph of one file, and the pages of files whose
# path the Core Rule Set takes for an attack.
page issue-30-case "/owner/repo/graph?file=src/java.base/share/classes/java/lang/Runtime.java"
page issue-30-case /owner/repo/src/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
page issue-30-case /owner/repo/src/branch/main/java.lang.String.md
page issue-30-case /owner/repo/src/branch/main/process.env.d.ts
page issue-30-case /owner/repo/src/branch/main/.../notes.txt
}
# call_api: the API's calls the cases name: the commits of a file, for a
# file of each name the Core Rule Set takes for an attack in the query
# parameter path, and the file-contents call, which takes its list of
# files as JSON in the query parameter body.
call_api() {
for file in \
src/java.base/share/classes/java/lang/Runtime.java \
java.lang.String.md \
document.write.js \
process.env.d.ts \
test/fixtures/__proto__.json \
install-script.sh \
src/backend/catalog/information_schema.sql; do
api issue-30-case /api/v1/repos/owner/repo/commits --get --data-urlencode "path=$file"
done
api issue-30-case /api/v1/repos/owner/repo/file-contents --get \
--data-urlencode 'body={"files":["src/java.base/share/classes/java/lang/Runtime.java"]}'
}
# push_images: push a container image to owner/python3-app, then the same
# image to owner/app. For the second, the case, docker asks gitea to take
# its layer from owner/python3-app, naming it in the query parameter
# from. docker sends no X-Request-ID, so its requests have the ids
# smallwebwaf gives them. The image is one small file, imported rather
# than built.
push_images() {
echo "$NAME" >"$WORK/layer.txt"
tar -cf - -C "$WORK" layer.txt |
docker import - "$address/owner/python3-app:1" >/dev/null
docker tag "$address/owner/python3-app:1" "$address/owner/app:1"
echo "$PASSWORD" | DOCKER_CONFIG="$WORK/docker" docker login --username owner \
--password-stdin "$address" >/dev/null 2>&1 || fail "docker login failed"
for image in "$address/owner/python3-app:1" "$address/owner/app:1"; do
DOCKER_CONFIG="$WORK/docker" docker push --quiet "$image" >/dev/null ||
echo "gitea-requests: the push of $image failed" >&2
done
}
# clone_and_fetch: the clone and fetch over HTTP of the sample.
clone_and_fetch() {
repo_git -c "http.extraHeader=X-Request-ID: git-clone" clone --quiet \
"http://owner:$PASSWORD@$address/owner/repo.git" "$WORK/clone" ||
fail "the clone failed"
repo_git -C "$WORK/clone" -c "http.extraHeader=X-Request-ID: git-fetch" fetch --quiet ||
fail "the fetch failed"
}
# field <name> <line>: the text of the field name in line, a line of the
# request log, or nothing when it has none.
field() {
printf '%s\n' "$2" | sed -n "s/.*\"$1\":\"\([^\"]*\)\".*/\1/p"
}
# report: print a line for each request smallwebwaf logged, but its
# health checks: what it did with the request, the ids of the Core Rule
# Set's rules that matched it, its X-Request-ID, and the request.
# waf_blocked is a refusal by the Core Rule Set, and forward a request
# passed on to gitea, whatever gitea answered.
report() {
docker logs "$SMALLWEBWAF" 2>/dev/null | grep '"type":"request"' |
grep -v '"action":"admin"' | sed 's/\\u0026/\&/g' >"$WORK/requests"
while IFS= read -r line; do
ids="$(printf '%s\n' "$line" | sed -n 's/.*"waf_rule_ids":\[\([0-9,]*\)\].*/\1/p')"
query="$(field query "$line")"
printf '%-11s %-13s %-17s %s %s%s\n' "$(field action "$line")" "${ids:--}" \
"$(field request_id "$line")" "$(field method "$line")" \
"$(field path "$line")" "${query:+?$query}"
done <"$WORK/requests"
echo "gitea-requests: $(grep -c . "$WORK/requests") requests," \
"$(grep -c '"action":"waf_blocked"' "$WORK/requests") refused by the Core Rule Set"
}
main() {
cd "$ROOT"
trap cleanup EXIT
trap 'exit 1' HUP INT TERM
WORK="$(mktemp -d)"
docker build --no-cache -t "$IMAGE" .
# smallwebwaf first, so that gitea can be given the address it is
# reached at, and gitea's image pulled before, so that gitea starts
# within the minute smallwebwaf's health check waits for it.
docker pull --quiet "$GITEA_IMAGE" >/dev/null
docker network create "$NETWORK" >/dev/null
docker run --detach --name "$SMALLWEBWAF" --network "$NETWORK" \
--publish 127.0.0.1::8080 \
--env SWWAF_UPSTREAM_URL="http://$GITEA:3000" \
"$IMAGE" >/dev/null
published="$(docker port "$SMALLWEBWAF" 8080/tcp)"
address="localhost:${published##*:}"
docker run --detach --name "$GITEA" --network "$NETWORK" \
--env GITEA__security__INSTALL_LOCK=true \
--env GITEA__server__ROOT_URL="http://$address/" \
"$GITEA_IMAGE" >/dev/null
# smallwebwaf's health check passes once gitea takes connections.
wait_for "the health check did not pass" healthy
docker exec --user git "$GITEA" gitea admin user create --admin \
--username owner --password "$PASSWORD" --email owner@example.invalid \
--must-change-password=false >/dev/null
make_repository
browse
call_api
push_images
clone_and_fetch
# The fetch's last request is the last of all; once it is logged, so
# is every other.
wait_for "smallwebwaf logged no line for the fetch" \
logged '"request_id":"git-fetch"' '"method":"POST"'
report
}
main "$@"