Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
fbe87a7c5e |
@@ -1,12 +1,11 @@
|
||||
.PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \
|
||||
run example-app gitea-requests
|
||||
run example-app
|
||||
|
||||
# Makefile targets are thin shims; the implementations live in script/
|
||||
# per the scripts-to-rule-them-all pattern (see the Entrypoints section
|
||||
# of README.md). tidy writes go.mod and go.sum as `go mod tidy` does,
|
||||
# which test checks. build and run are for working on the code by hand;
|
||||
# example-app checks the image with an app built on it; gitea-requests
|
||||
# reports the gitea requests the Core Rule Set refuses at the defaults.
|
||||
# example-app checks the image with an app built on it.
|
||||
|
||||
bootstrap:
|
||||
@script/bootstrap
|
||||
@@ -46,6 +45,3 @@ run:
|
||||
|
||||
example-app:
|
||||
@script/example-app
|
||||
|
||||
gitea-requests:
|
||||
@script/gitea-requests
|
||||
|
||||
@@ -197,12 +197,11 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
||||
rate limits and before the rule files, which it does not need. A request for
|
||||
one is refused with `SWWAF_BAN_RESPONSE` and bans the client's netblock for a
|
||||
clear sign of attack, as a `ban` rule's match does. A trap path is matched
|
||||
against the path a `path` rule sees: without the query, as the client sent it
|
||||
and once percent-decoded, the whole of it, character for character.
|
||||
`/wp-login.php` matches `/wp-login.php?redirect_to=x` and `/%77p-login.php`,
|
||||
but not `/wp-login.php/`, `/WP-LOGIN.PHP`, `/blog/wp-login.php` or
|
||||
`/%2577p-login.php`. The ban's notes give the path as the client sent it. A
|
||||
client in `SWWAF_ALLOW_NETS` is not checked.
|
||||
against the path a `path` rule sees: as the client sent it, before any
|
||||
decoding and without the query, the whole of it, character for character.
|
||||
`/wp-login.php` matches `/wp-login.php?redirect_to=x`, but not
|
||||
`/wp-login.php/`, `/WP-LOGIN.PHP`, `/blog/wp-login.php` or `/%77p-login.php`.
|
||||
A client in `SWWAF_ALLOW_NETS` is not checked.
|
||||
- Inspects each request with the OWASP Core Rule Set 4.25.0, run by Coraza,
|
||||
after the rule files, unless `SWWAF_WAF_MODE` is `off`: its method, its URL
|
||||
with the query, and its headers, and, once `SWWAF_WAF_BODY_LIMIT` is set, its
|
||||
@@ -725,25 +724,23 @@ effective settings are logged at start, unless `SWWAF_LOG_LEVEL` is `warn` or
|
||||
(rule 900440), and a multipart body that fails Coraza's strict checks (rule
|
||||
900450), add 5 to the score, as a rule rated critical does, since no rule
|
||||
reads what comes after the fault. So does a multipart body the limit cuts
|
||||
before the colon of a part's header line, or between the carriage return and
|
||||
the line feed that end a part's header line or the empty line after its
|
||||
headers, since Coraza takes the line the limit cuts for a malformed header.
|
||||
The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs out to send the part
|
||||
that is read, and a request whose body passes `SWWAF_REQUEST_MAX_BYTES` within
|
||||
it is refused before anything reaches the app. Of a file in a multipart body,
|
||||
Coraza counts the bytes and writes nothing. Body inspection suits apps whose
|
||||
forms carry no code. In front of gitea it refuses issue and comment text, wiki
|
||||
pages and files saved in the web editor that hold shell commands or code
|
||||
(932125, 932235, 932250 and others), package descriptions that show code, PyPI
|
||||
uploads (922130), and attachments named like `debug.log` or `config.yml`
|
||||
(932180), until the rule ids the request log names are added to
|
||||
`SWWAF_WAF_DISABLED_RULES`.
|
||||
before the colon of a part's header line, which Coraza cannot tell from a
|
||||
header without one. The client has until `SWWAF_CLIENT_REQUEST_TIMEOUT` runs
|
||||
out to send the part that is read, and a request whose body passes
|
||||
`SWWAF_REQUEST_MAX_BYTES` within it is refused before anything reaches the
|
||||
app. Of a file in a multipart body, Coraza counts the bytes and writes
|
||||
nothing. Body inspection suits apps whose forms carry no code. In front of
|
||||
gitea it refuses issue and comment text, wiki pages and files saved in the web
|
||||
editor that hold shell commands or code (932125, 932235, 932250 and others),
|
||||
package descriptions that show code, PyPI uploads (922130), and attachments
|
||||
named like `debug.log` or `config.yml` (932180), until the rule ids the
|
||||
request log names are added to `SWWAF_WAF_DISABLED_RULES`.
|
||||
- `SWWAF_TRAP_PATHS` (default empty): paths the app never serves and only
|
||||
scanners ask for, such as `/wp-login.php,/xmlrpc.php` in front of gitea; a
|
||||
request for one bans its client for a clear sign of attack, as a `ban` rule
|
||||
does, with or without rule files (see "What it does so far" above). A path
|
||||
that does not start with `/`, or holds a `?`, as one written with a query
|
||||
does, stops the start.
|
||||
that does not start with `/`, or holds a `?`, which no path a `path` rule sees
|
||||
holds, stops the start.
|
||||
- `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most requests of a client in
|
||||
a minute that `smallwebwaf` may refuse after a rule file match, a trap path or
|
||||
a Core Rule Set match, or for a missing or wrong token; one more breaks the
|
||||
@@ -1523,15 +1520,14 @@ scanner-agent user_agent ban (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
|
||||
|
||||
- The id is letters, digits, `-` and `_`, and no two rules share one. The
|
||||
request log, the metrics and a ban's notes name the rule by it.
|
||||
- The target is what the regex is matched against: `path`, the path without the
|
||||
query, and `uri`, the path and the query together, each both as sent and once
|
||||
percent-decoded, so that an encoded probe does not slip past; `query`, as the
|
||||
client sent it, before any decoding; `method`; `host`; `user_agent`;
|
||||
`referer`; or `header:<Name>`, any one request header but `Host` and
|
||||
`Transfer-Encoding`, which Go's HTTP server takes out of every request; the
|
||||
request's host is the target `host`. A header sent more than once is matched
|
||||
with its values joined by `, `, and one not sent as empty text. No body is
|
||||
read.
|
||||
- The target is what the regex is matched against: `path` or `query`, as the
|
||||
client sent it, before any decoding; `uri`, the path and the query together,
|
||||
both as sent and once percent-decoded, so that an encoded probe does not slip
|
||||
past; `method`; `host`; `user_agent`; `referer`; or `header:<Name>`, any one
|
||||
request header but `Host` and `Transfer-Encoding`, which Go's HTTP server
|
||||
takes out of every request; the request's host is the target `host`. A header
|
||||
sent more than once is matched with its values joined by `, `, and one not
|
||||
sent as empty text. No body is read.
|
||||
- The action is `log`, `block` or `ban` (see "What it does so far" above). Keep
|
||||
`ban` for requests no real visitor sends, and anchor a path at the site root
|
||||
with `^/`: a file of the same name deeper in a site can be ordinary content,
|
||||
@@ -2374,19 +2370,6 @@ so that they run in minimal containers.
|
||||
the same volume still refuses the banned client; then removes the containers,
|
||||
the volume and both images. It needs network access, for nixpkgs' binary
|
||||
cache, and `script/check` does not run it; `make example-app` does.
|
||||
- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind it,
|
||||
with every setting of `smallwebwaf` at its default but `SWWAF_UPSTREAM_URL`,
|
||||
which names gitea's container. It sends through it the gitea requests that the
|
||||
notes specific to gitea in [`SPEC.md`](SPEC.md) name for a file name in a
|
||||
query parameter, for a path and for a container image push, and a short sample
|
||||
of gitea's ordinary use: signing in, pages of files and commits, a diff, the
|
||||
API, a push, clone and fetch over HTTP, and two container image pushes. It
|
||||
prints a line for each request `smallwebwaf` logged, with what it did with it
|
||||
and the ids of the Core Rule Set's rules that matched it, then removes the
|
||||
containers, the network and the images it made, but gitea's own image. It
|
||||
needs network access, and a docker daemon that pushes to a registry on
|
||||
`localhost` over plain HTTP, as docker does by default; `script/check` does
|
||||
not run it, and `make gitea-requests` does.
|
||||
|
||||
## TODO
|
||||
|
||||
|
||||
@@ -825,8 +825,7 @@ a file edited, added or removed there takes effect while it runs.
|
||||
- `id`: a short name of letters, digits, `-` and `_`, unique across all files.
|
||||
It appears in the request log, metrics, alerts and ban notes.
|
||||
- `target`: what the regex is matched against.
|
||||
- `path`: the URL path without the query, both as received and once
|
||||
percent-decoded, so an encoded probe cannot slip past.
|
||||
- `path`: the URL path as received, before any decoding.
|
||||
- `query`: the raw query string.
|
||||
- `uri`: path and query together, both as received and once percent-decoded,
|
||||
so an encoded probe cannot slip past.
|
||||
@@ -1477,42 +1476,8 @@ holds any token file.
|
||||
Saving such a rule stores it, but the branch settings page gitea then
|
||||
returns to gets the 403 answer of `smallwebwaf`, so the save looks as
|
||||
if it failed.
|
||||
- A file name in a query parameter, those the fourth change leaves out
|
||||
of 930120, 932160 and 932260 included, in which the Core Rule Set's
|
||||
rules for Java, script, PowerShell and SQL find what they look for:
|
||||
`runtime` in OpenJDK's
|
||||
`src/java.base/share/classes/java/lang/Runtime.java` (944110), a Java
|
||||
class, as in `java.lang.String.md` (944130), `document.write.js`
|
||||
(941180), `process.env.d.ts` (934100), `__proto__` (934130), the
|
||||
PowerShell command `Install-Script`, as in `install-script.sh`
|
||||
(932120), or a database's name, as in PostgreSQL's
|
||||
`information_schema.sql` (942140). For such a file, the API's list of
|
||||
its commits (`path`) is refused, and so are the API's call for the
|
||||
contents of files that names it in its query (`body`), the commit
|
||||
graph of that file alone (`file`), and the hidden lines of its diff
|
||||
(`path`), which stay hidden. A visitor whom gitea sends to sign in on
|
||||
the way to a page of such a file that needs signing in, such as its
|
||||
edit page, gets the 403 answer of `smallwebwaf` in place of the
|
||||
sign-in page (`redirect_to`), and can still sign in by opening the
|
||||
sign-in page itself.
|
||||
- A path: a file name ending in `~`, or an `.xhtml` file whose path
|
||||
holds a space.
|
||||
- A path that holds what the Core Rule Set takes for a Java class
|
||||
(944130), as `java.lang.String.md` and
|
||||
`META-INF/services/org.apache.commons.logging.LogFactory` do, or for
|
||||
Node.js code (934100), as `process.env.d.ts` does, or that has a
|
||||
directory named `...` (930100, 930110), such as `.../notes.txt`. The
|
||||
page of such a file, its raw file, its history, its blame and its edit
|
||||
page are refused; the directory that holds it lists it, and a clone
|
||||
carries it.
|
||||
- A container image push that reuses a layer of an image docker has
|
||||
pushed to that gitea before, when that image's name, after its owner,
|
||||
starts with an entry on the list of commands (932260), such as
|
||||
`owner/python3-app` or `owner/docker-base`. docker asks gitea to take
|
||||
the layer from that image, naming it in the query parameter `from`,
|
||||
which 932260 checks. The push fails with the 403 answer of
|
||||
`smallwebwaf`, and so does each new try, since docker asks in the same
|
||||
way.
|
||||
- Artifact uploads from `actions/upload-artifact@v3` send the header
|
||||
`Content-Range`, which the Core Rule Set refuses (920450). The action
|
||||
sends two files at a time, does not retry a 403 and sends no further file
|
||||
@@ -1600,12 +1565,10 @@ holds any token file.
|
||||
- Core Rule Set false positives against real apps (a search for code, and any
|
||||
body once `SWWAF_WAF_BODY_LIMIT` is set): a match refuses only that request
|
||||
and bans no one by itself; the request log names the rule, and exclusions by
|
||||
rule id and path fix it. `script/gitea-requests` runs gitea behind the image
|
||||
at the defaults, sends through it the requests the notes specific to gitea
|
||||
name for a file name in a query parameter, for a path and for a container
|
||||
image push, and a short sample of gitea's ordinary use, and reports each
|
||||
request the Core Rule Set refuses, so that those notes can be checked again
|
||||
when gitea or the Core Rule Set changes version.
|
||||
rule id and path fix it. Further gitea requests the Core Rule Set may refuse
|
||||
at the defaults, found by reading gitea's source rather than a running gitea,
|
||||
are collected in https://git.eeqj.de/sneak/smallwebwaf/issues/30 and checked
|
||||
when milestone 1 runs in front of a real gitea.
|
||||
- Attacks carried in request bodies: not refused by default, since on a code
|
||||
forge bodies are full of code the Core Rule Set takes for attacks. Most of
|
||||
what shows in URLs and headers is still refused, the client that sends an
|
||||
|
||||
@@ -123,8 +123,7 @@ type Notes struct {
|
||||
LimitPercentSetting string `json:"limit_percent_setting,omitempty"`
|
||||
// RuleID and Target are, for a ban for a clear sign of attack, the id
|
||||
// of the rule file rule that matched, and its target; TrapPath is, for
|
||||
// one for a request for a path in SWWAF_TRAP_PATHS, that path as the
|
||||
// client sent it.
|
||||
// one for a request for a path in SWWAF_TRAP_PATHS, that path.
|
||||
RuleID string `json:"rule_id,omitempty"`
|
||||
Target string `json:"target,omitempty"`
|
||||
TrapPath string `json:"trap_path,omitempty"`
|
||||
|
||||
@@ -1665,8 +1665,7 @@ func parsePathPrefixes(value string) ([]string, error) {
|
||||
|
||||
// parseTrapPaths reads a comma-separated list of trap paths. Each is
|
||||
// matched against a request's path as a path rule is, without the query,
|
||||
// so a path that does not start with / would never match, and one that
|
||||
// holds a ? is written with a query.
|
||||
// so a path that does not start with / or holds a ? would never match.
|
||||
func parseTrapPaths(value string) ([]string, error) {
|
||||
paths, err := parseList(value)
|
||||
if err != nil {
|
||||
|
||||
@@ -86,75 +86,6 @@ func TestEveryClientIsLookedUpWithoutWaitingWhileNoSettingNeedsIt(t *testing.T)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRequestThatWaitedForItsLookupIsCountedWhenItGoesOn(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
// The country list has a new client's request wait for its answer,
|
||||
// which the stand-in for GeoJS gives only once released.
|
||||
geojsURL, asked, release := startHeldGeoJS(t)
|
||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {})
|
||||
clk := &clock{now: time.Date(2026, 10, 6, 0, 0, 58, 0, time.UTC)}
|
||||
addr, out, server := startProxyWithClock(t, app.URL, geojsURL, clk.Now,
|
||||
map[string]string{
|
||||
trustedProxies: trustLocalhost,
|
||||
lookupTimeout: "1h",
|
||||
deniedCountries: "kp",
|
||||
})
|
||||
|
||||
requestFromDE := func() *http.Request {
|
||||
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
|
||||
req.Header.Set(forwardedFor, fromDE)
|
||||
|
||||
return req
|
||||
}
|
||||
|
||||
// fromDE's first request arrives two seconds before a minute ends,
|
||||
// and waits for GeoJS.
|
||||
first, httpClient := requestFromDE(), newClient(t)
|
||||
firstEnded := make(chan error, 1)
|
||||
|
||||
go func() {
|
||||
res, err := httpClient.Do(first)
|
||||
if err == nil {
|
||||
err = readAnswer(res).err
|
||||
}
|
||||
|
||||
firstEnded <- err
|
||||
}()
|
||||
|
||||
waitUntil(func() bool { return len(asked()) == 1 })
|
||||
|
||||
// Half a second into the next minute, the answer is kept, as an edit
|
||||
// of lookups.json would keep it, while the first request still waits
|
||||
// for GeoJS. The second request goes on at once, and is counted.
|
||||
clk.advance(2500 * time.Millisecond)
|
||||
server.GeoJS.Load([]lookup.Answer{{
|
||||
Client: netip.MustParsePrefix(fromDE + "/32"), ASN: asnDE, ASName: asNameDE,
|
||||
Country: "DE", Answered: clk.Now(), Used: clk.Now(),
|
||||
}})
|
||||
wantStatus(t, do(t, requestFromDE()), http.StatusOK)
|
||||
|
||||
// GeoJS answers, and the first request goes on.
|
||||
release()
|
||||
|
||||
err := <-firstEnded
|
||||
if err != nil {
|
||||
t.Fatalf("first request: %v", err)
|
||||
}
|
||||
|
||||
wantLine(t, out.requestLines(t, 2)[1], http.StatusOK, requestlog.ActionForward)
|
||||
|
||||
// The third request's count holds all three: the first was counted
|
||||
// when it went on, in the minute under way, and the second's count
|
||||
// was kept.
|
||||
clk.advance(500 * time.Millisecond)
|
||||
wantStatus(t, do(t, requestFromDE()), http.StatusOK)
|
||||
|
||||
if got := out.requestLines(t, 3)[2].Counts.Minute; got != 3 {
|
||||
t.Errorf("the third request's count in the minute is %v, want 3", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestASNumberAndNameInTheLogLineTheHistoryTheBanNotesAndTheAlert(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
@@ -261,12 +261,6 @@ func (rq *request) checkClient(ctx context.Context) string {
|
||||
|
||||
rq.lookUp(ctx)
|
||||
|
||||
// The lookup can wait for GeoJS, so the checks after it read the time
|
||||
// again: a request counted at the time it arrived could be counted
|
||||
// after a later one of its client's, which the rate limits would take
|
||||
// for the clock set back.
|
||||
now = rq.h.now()
|
||||
|
||||
if rq.countryDenied() {
|
||||
return requestlog.ActionCountryDenied
|
||||
}
|
||||
|
||||
@@ -10,15 +10,13 @@ import (
|
||||
)
|
||||
|
||||
// trapPath reports whether the request asks for a path in
|
||||
// SWWAF_TRAP_PATHS: its path as a path rule sees it, without the query,
|
||||
// as sent or once percent-decoded, is one of them. Such a request is a
|
||||
// clear sign of attack, as a ban rule's match is: it bans the client's
|
||||
// netblock, or in observe mode raises the alert for the ban it would have
|
||||
// made. The ban's notes give the path as sent.
|
||||
// SWWAF_TRAP_PATHS: its path as a path rule sees it, before any decoding
|
||||
// and without the query, is one of them. Such a request is a clear sign of
|
||||
// attack, as a ban rule's match is: it bans the client's netblock, or in
|
||||
// observe mode raises the alert for the ban it would have made.
|
||||
func (rq *request) trapPath(now time.Time) bool {
|
||||
path := rules.Path(rq.in)
|
||||
if !slices.Contains(rq.h.config.TrapPaths, path) &&
|
||||
!slices.Contains(rq.h.config.TrapPaths, rules.DecodeOnce(path)) {
|
||||
if !slices.Contains(rq.h.config.TrapPaths, path) {
|
||||
return false
|
||||
}
|
||||
|
||||
|
||||
@@ -32,9 +32,10 @@ func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
|
||||
})
|
||||
start := clk.Now()
|
||||
|
||||
// Only the path itself, the whole of it in the same case, is a trap
|
||||
// path.
|
||||
for _, path := range []string{"/wp-login.php/", "/WP-LOGIN.PHP", "/blog/xmlrpc.php"} {
|
||||
// Only the path itself, as the client sent it, is a trap path.
|
||||
for _, path := range []string{
|
||||
"/wp-login.php/", "/WP-LOGIN.PHP", "/blog/xmlrpc.php", "/%77p-login.php",
|
||||
} {
|
||||
s.request(otherClient, path, http.StatusOK, requestlog.ActionForward)
|
||||
}
|
||||
|
||||
@@ -83,22 +84,6 @@ func TestTrapPathBansAsABanRuleDoes(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestTrapPathMatchesThePathDecodedOnce(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
s, _, server := startWithClock(t, "", map[string]string{trapPaths: trapPathList})
|
||||
|
||||
// %2577 decodes once to %77, not to w.
|
||||
s.request(otherClient, "/%2577p-login.php", http.StatusOK, requestlog.ActionForward)
|
||||
s.request(client, "/%77p-login.php", http.StatusForbidden, requestlog.ActionBanned)
|
||||
|
||||
// The ban's notes give the path as the client sent it.
|
||||
got := server.Ledger.Bans(netip.MustParsePrefix(client + "/32"))
|
||||
if len(got) != 1 || got[0].Notes.TrapPath != "/%77p-login.php" {
|
||||
t.Fatalf("bans %+v, want one whose notes give the trap path /%%77p-login.php", got)
|
||||
}
|
||||
}
|
||||
|
||||
func TestTrapPathsNeedNoRuleFiles(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
+16
-26
@@ -50,13 +50,6 @@ const quietTime = 2 * time.Second
|
||||
// header:<Name>.
|
||||
const headerTarget = "header:"
|
||||
|
||||
// pathTarget and uriTarget are the targets matched both as received and
|
||||
// once percent-decoded: the path, and the path and query together.
|
||||
const (
|
||||
pathTarget = "path"
|
||||
uriTarget = "uri"
|
||||
)
|
||||
|
||||
// escapeLength is the length of a percent escape, such as %2e.
|
||||
const escapeLength = 3
|
||||
|
||||
@@ -381,7 +374,7 @@ func parse(line string) (Rule, bool, error) {
|
||||
// isTarget reports whether target is one a rule may have.
|
||||
func isTarget(target string) bool {
|
||||
switch target {
|
||||
case pathTarget, "query", uriTarget, "method", "host", "user_agent", "referer":
|
||||
case "path", "query", "uri", "method", "host", "user_agent", "referer":
|
||||
return true
|
||||
}
|
||||
|
||||
@@ -391,38 +384,35 @@ func isTarget(target string) bool {
|
||||
}
|
||||
|
||||
// matches reports whether the rule's regex matches its target in r. For
|
||||
// path and uri it is matched against the target as received, and against
|
||||
// it once percent-decoded, so that an encoded probe cannot slip past.
|
||||
// uri it is matched against the path and query as received, and against
|
||||
// them once percent-decoded, so that an encoded probe cannot slip past.
|
||||
func (rule Rule) matches(r *http.Request) bool {
|
||||
target := value(rule.Target, r)
|
||||
if rule.Target == "uri" {
|
||||
uri := pathAndQuery(r)
|
||||
|
||||
if rule.Target == pathTarget || rule.Target == uriTarget {
|
||||
return rule.regex.MatchString(target) || rule.regex.MatchString(DecodeOnce(target))
|
||||
return rule.regex.MatchString(uri) || rule.regex.MatchString(decodeOnce(uri))
|
||||
}
|
||||
|
||||
return rule.regex.MatchString(target)
|
||||
return rule.regex.MatchString(value(rule.Target, r))
|
||||
}
|
||||
|
||||
// Path returns r's path as the client sent it, before any decoding or
|
||||
// re-encoding, up to the first ?: what a path rule is matched against, as
|
||||
// it is and once percent-decoded.
|
||||
// re-encoding, up to the first ?: what a path rule is matched against.
|
||||
func Path(r *http.Request) string {
|
||||
path, _, _ := strings.Cut(pathAndQuery(r), "?")
|
||||
|
||||
return path
|
||||
}
|
||||
|
||||
// value returns what a rule with target is matched against in r: the
|
||||
// path, as Path gives it, the path and query, as pathAndQuery gives
|
||||
// them, and the query as the client sent it, before any decoding or
|
||||
// re-encoding, after the first ?, and a header's values joined by ", ",
|
||||
// as HTTP joins those of a header sent more than once.
|
||||
// value returns what a rule with target, other than uri, is matched
|
||||
// against in r: the path, as Path gives it, and the query as the client
|
||||
// sent it, before any decoding or re-encoding, after the first ?, and a
|
||||
// header's values joined by ", ", as HTTP joins those of a header sent
|
||||
// more than once.
|
||||
func value(target string, r *http.Request) string {
|
||||
switch target {
|
||||
case pathTarget:
|
||||
case "path":
|
||||
return Path(r)
|
||||
case uriTarget:
|
||||
return pathAndQuery(r)
|
||||
case "query":
|
||||
_, query, _ := strings.Cut(pathAndQuery(r), "?")
|
||||
|
||||
@@ -474,11 +464,11 @@ func header(r *http.Request, name string) string {
|
||||
return strings.Join(r.Header.Values(name), ", ")
|
||||
}
|
||||
|
||||
// DecodeOnce returns s with each percent escape, such as %2e, replaced by
|
||||
// decodeOnce returns s with each percent escape, such as %2e, replaced by
|
||||
// the byte it stands for. A % that is not followed by two hex digits is
|
||||
// left as it is, so that a malformed escape cannot keep the rest of s
|
||||
// from being decoded.
|
||||
func DecodeOnce(s string) string {
|
||||
func decodeOnce(s string) string {
|
||||
var decoded strings.Builder
|
||||
|
||||
for i := 0; i < len(s); i++ {
|
||||
|
||||
@@ -49,9 +49,7 @@ func TestEachTargetMatchesWhatItNames(t *testing.T) {
|
||||
want bool
|
||||
}{
|
||||
{"path as received", `path log ^/%2eenv$`, "/%2eenv", nil, true},
|
||||
{"path decoded", `path log ^/\.env$`, "/%2Eenv", nil, true},
|
||||
{"path decoded only once", `path log ^/\.env$`, "/%252eenv", nil, false},
|
||||
{"path decoded without the query", `path log ^/a\?b$`, "/a%3fb?c", nil, true},
|
||||
{"path not decoded", `path log ^/\.env$`, "/%2eenv", nil, false},
|
||||
{"path without the query", `path log ^/a$`, "/a?b=c", nil, true},
|
||||
{"query as received", `query log ^b=%2e$`, "/a?b=%2e", nil, true},
|
||||
{"uri as received", `uri log ^/a\?b=%2e$`, "/a?b=%2e", nil, true},
|
||||
@@ -420,7 +418,7 @@ func TestDefaultFileBansProbesAtTheSiteRootAlone(t *testing.T) {
|
||||
|
||||
// Probes sent by a browser, by the rule that refuses them.
|
||||
for rule, targets := range map[string][]string{
|
||||
"env-file": {"/.env", "/.env.production", "/.ENV", "/%2eenv"},
|
||||
"env-file": {"/.env", "/.env.production", "/.ENV"},
|
||||
"vcs-dir": {"/.git/config", "/.git", "/.svn/entries"},
|
||||
"secrets-dir": {"/.aws/credentials", "/.ssh/id_rsa"},
|
||||
"secret-file": {"/.htpasswd", "/.DS_Store", "/.git-credentials"},
|
||||
|
||||
+2
-4
@@ -134,10 +134,8 @@ SecRule REQBODY_PROCESSOR "@rx ^(?:URLENCODED|MULTIPART|JSON|XML)$" \
|
||||
# Rule Set rates critical does: no rule reads what comes after the fault,
|
||||
# which the app may still read. Coraza's recommended configuration refuses
|
||||
# them in its rules 200002 and 200003. A multipart body the limit cuts
|
||||
# before the colon of a part's header line, or between the carriage return
|
||||
# and the line feed that end a part's header line or the empty line after
|
||||
# its headers, adds 5 too, since Coraza takes the line the limit cuts for a
|
||||
# malformed header. Coraza parses any form data body.
|
||||
# before the colon of a part's header line adds 5 too, since Coraza cannot
|
||||
# tell it from a header without one. Coraza parses any form data body.
|
||||
SecRule REQBODY_ERROR "!@eq 0" "id:900440,phase:2,pass,severity:'CRITICAL',\
|
||||
setvar:'tx.inbound_anomaly_score_pl1=+5'"
|
||||
SecRule MULTIPART_STRICT_ERROR "!@eq 0" "id:900450,phase:2,pass,\
|
||||
|
||||
@@ -645,25 +645,6 @@ func TestMultipartBodyCutBeforeAPartHeadersColonIsAMatch(t *testing.T) {
|
||||
body[:bodyLimit+1])
|
||||
}
|
||||
|
||||
func TestMultipartBodyCutBeforeALineFeedInAPartsHeadersIsAMatch(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
crs := readingBodies(t)
|
||||
headerLine := "--b\r\nContent-Disposition: form-data; name=\"q\"\r"
|
||||
|
||||
// The limit falls between the carriage return and the line feed that
|
||||
// end the second part's header line, and then between those that end
|
||||
// the empty line after it. Coraza, reading up to the limit, takes the
|
||||
// line ending in a lone carriage return for a malformed header.
|
||||
for _, cut := range []int{len(headerLine), len(headerLine + "\n\r")} {
|
||||
first := field("pad", strings.Repeat("a", bodyLimit-len(field("pad", ""))-cut))
|
||||
body := first + field("q", "1") + end
|
||||
|
||||
wantBody(t, crs, post(multipart), body, matched(900440, 900450),
|
||||
body[:bodyLimit+1])
|
||||
}
|
||||
}
|
||||
|
||||
// TestCorazaWritesNoFile is not parallel, since it sets TMPDIR, the
|
||||
// system's temporary directory, for the whole test process.
|
||||
func TestCorazaWritesNoFile(t *testing.T) {
|
||||
|
||||
@@ -1,329 +0,0 @@
|
||||
#!/bin/sh
|
||||
# script/gitea-requests: build the image and run gitea 1.27.3 behind it,
|
||||
# with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
|
||||
# which names gitea's container. Then send through it the gitea requests
|
||||
# that reading gitea's source found the Core Rule Set refuses, and a short
|
||||
# sample of gitea's ordinary use: signing in, pages of files and commits,
|
||||
# a diff, the API, a push, clone and fetch over HTTP, and two container
|
||||
# image pushes. Last, print each request smallwebwaf logged, with what it
|
||||
# did with it and the ids of the Core Rule Set's rules that matched it.
|
||||
# Every request comes from one client, docker's gateway, so more than
|
||||
# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
|
||||
# the requests after would show as banned. The containers, the network, the images and the files the script makes
|
||||
# are removed however it ends; gitea's own image stays, as the base images
|
||||
# of a build do. It needs network access, and a docker daemon that pushes
|
||||
# to a registry on localhost over plain HTTP, as docker does by default.
|
||||
# script/check does not run this.
|
||||
set -eu
|
||||
|
||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
|
||||
|
||||
# gitea 1.27.3, 2026-08-29
|
||||
GITEA_IMAGE="gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef"
|
||||
|
||||
# Named after this run, so that runs in other clones on the same host
|
||||
# never touch each other's.
|
||||
NAME="$("$SCRIPT_DIR/projectname")-gitea-$$"
|
||||
IMAGE="$NAME-image"
|
||||
NETWORK="$NAME"
|
||||
SMALLWEBWAF="$NAME-smallwebwaf"
|
||||
GITEA="$NAME-gitea"
|
||||
|
||||
# gitea's one user, who owns the repository and the container images.
|
||||
PASSWORD="owner-password"
|
||||
|
||||
BROWSER="Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0"
|
||||
|
||||
# The files the script makes, and smallwebwaf's address once it runs,
|
||||
# localhost and the port docker publishes it on: a host name, since the
|
||||
# Core Rule Set takes a Host that is an IP address for a warning (920350).
|
||||
WORK=""
|
||||
address=""
|
||||
|
||||
cleanup() {
|
||||
docker rm --force --volumes "$SMALLWEBWAF" "$GITEA" >/dev/null 2>&1 || true
|
||||
docker network rm "$NETWORK" >/dev/null 2>&1 || true
|
||||
docker rmi --force "$IMAGE" >/dev/null 2>&1 || true
|
||||
if [ -n "$address" ]; then
|
||||
docker rmi --force "$address/owner/python3-app:1" "$address/owner/app:1" \
|
||||
>/dev/null 2>&1 || true
|
||||
fi
|
||||
rm -rf "$WORK"
|
||||
}
|
||||
|
||||
fail() {
|
||||
echo "gitea-requests: $*" >&2
|
||||
report
|
||||
exit 1
|
||||
}
|
||||
|
||||
# wait_for <what fails> <command>...: run the command every second until
|
||||
# it succeeds, for at most a minute.
|
||||
wait_for() {
|
||||
failure="$1"
|
||||
shift
|
||||
tries=0
|
||||
until "$@"; do
|
||||
tries=$((tries + 1))
|
||||
[ "$tries" -lt 60 ] || fail "$failure"
|
||||
sleep 1
|
||||
done
|
||||
}
|
||||
|
||||
healthy() {
|
||||
status="$(docker inspect --format '{{.State.Health.Status}}' "$SMALLWEBWAF")"
|
||||
[ "$status" = healthy ]
|
||||
}
|
||||
|
||||
# logged <text>...: a line of smallwebwaf's output holds every text, in
|
||||
# any order.
|
||||
logged() {
|
||||
lines="$(docker logs "$SMALLWEBWAF" 2>&1)"
|
||||
for text in "$@"; do
|
||||
lines="$(printf '%s\n' "$lines" | grep -F "$text")" || return 1
|
||||
done
|
||||
}
|
||||
|
||||
# send <label> <path> <curl option>...: send a request for path to
|
||||
# smallwebwaf, with label as its X-Request-ID, which the report gives,
|
||||
# and keep the body of the answer in $WORK/answer. The Core Rule Set
|
||||
# reads that header too, so a label names no file the cases name.
|
||||
send() {
|
||||
label="$1"
|
||||
path="$2"
|
||||
shift 2
|
||||
curl --silent --show-error --max-time 30 --output "$WORK/answer" \
|
||||
--header "X-Request-ID: $label" "$@" "http://$address$path"
|
||||
}
|
||||
|
||||
# page <label> <path> <curl option>...: send the request as a browser
|
||||
# does, with the cookies gitea has set. Its Referer is the repository's
|
||||
# page, which links to the others, and never a page the Core Rule Set
|
||||
# refused: its address would have the next request refused too.
|
||||
page() {
|
||||
label="$1"
|
||||
path="$2"
|
||||
shift 2
|
||||
send "$label" "$path" --compressed --user-agent "$BROWSER" \
|
||||
--header "Accept: text/html,application/xhtml+xml,*/*;q=0.8" \
|
||||
--header "Accept-Language: en-US,en;q=0.5" \
|
||||
--cookie "$WORK/cookies" --cookie-jar "$WORK/cookies" \
|
||||
--referer "http://$address/owner/repo" "$@"
|
||||
}
|
||||
|
||||
# api <label> <path> <curl option>...: send the request as a client of
|
||||
# gitea's API does, as the owner.
|
||||
api() {
|
||||
label="$1"
|
||||
path="$2"
|
||||
shift 2
|
||||
send "$label" "$path" --user "owner:$PASSWORD" \
|
||||
--header "Accept: application/json" "$@"
|
||||
}
|
||||
|
||||
# repo_git <git argument>...: run git as the owner, without the host's
|
||||
# own git settings, so that nothing it does is kept outside $WORK.
|
||||
repo_git() {
|
||||
GIT_CONFIG_GLOBAL=/dev/null GIT_CONFIG_NOSYSTEM=1 git \
|
||||
-c user.name=owner -c user.email=owner@example.invalid "$@"
|
||||
}
|
||||
|
||||
# lines: forty lines of text, enough for a diff to hide some of them.
|
||||
lines() {
|
||||
i=1
|
||||
while [ "$i" -le 40 ]; do
|
||||
echo "line $i"
|
||||
i=$((i + 1))
|
||||
done
|
||||
}
|
||||
|
||||
# make_repository: create the repository owner/repo, holding a file of
|
||||
# each name the cases name, and push it, which is the push over HTTP of
|
||||
# the sample. The second commit adds a line at the end of Runtime.java,
|
||||
# so that its diff hides the lines above.
|
||||
make_repository() {
|
||||
status="$(api create-repository /api/v1/user/repos \
|
||||
--header "Content-Type: application/json" --data '{"name":"repo"}' \
|
||||
--write-out '%{http_code}')"
|
||||
[ "$status" = 201 ] || fail "creating the repository was answered $status"
|
||||
|
||||
dir="$WORK/repo"
|
||||
repo_git init --quiet --initial-branch=main "$dir"
|
||||
for file in \
|
||||
src/java.base/share/classes/java/lang/Runtime.java \
|
||||
java.lang.String.md \
|
||||
META-INF/services/org.apache.commons.logging.LogFactory \
|
||||
document.write.js \
|
||||
process.env.d.ts \
|
||||
test/fixtures/__proto__.json \
|
||||
install-script.sh \
|
||||
src/backend/catalog/information_schema.sql \
|
||||
.../notes.txt \
|
||||
README.md; do
|
||||
mkdir -p "$dir/$(dirname "$file")"
|
||||
lines >"$dir/$file"
|
||||
repo_git -C "$dir" add "$file"
|
||||
done
|
||||
repo_git -C "$dir" commit --quiet --message "Add the files"
|
||||
echo "line 41" >>"$dir/src/java.base/share/classes/java/lang/Runtime.java"
|
||||
repo_git -C "$dir" commit --quiet --all --message "Add a line"
|
||||
|
||||
repo_git -C "$dir" -c "http.extraHeader=X-Request-ID: git-push" push --quiet \
|
||||
"http://owner:$PASSWORD@$address/owner/repo.git" main ||
|
||||
fail "the push of the repository failed"
|
||||
}
|
||||
|
||||
# browse: sign in, then open pages as a person does. The cases are those
|
||||
# of https://git.eeqj.de/sneak/smallwebwaf/issues/30.
|
||||
browse() {
|
||||
# Case: gitea sends a visitor who is not signed in to the sign-in
|
||||
# page, with the page asked for in redirect_to, when that page needs
|
||||
# signing in, as a file's edit page does.
|
||||
page issue-30-case /owner/repo/_edit/main/document.write.js --location
|
||||
|
||||
page sign-in /user/login
|
||||
status="$(page sign-in /user/login --header "Origin: http://$address" \
|
||||
--data-urlencode user_name=owner --data-urlencode "password=$PASSWORD" \
|
||||
--write-out '%{http_code}')"
|
||||
[ "$status" = 303 ] || fail "signing in was answered $status"
|
||||
|
||||
page browse /
|
||||
page browse /owner/repo
|
||||
page browse /owner/repo/src/branch/main/README.md
|
||||
page browse /owner/repo/commits/branch/main
|
||||
page browse "/owner/repo/commit/$(repo_git -C "$WORK/repo" rev-parse HEAD)"
|
||||
|
||||
# Case: expanding the lines the diff hides, by the link the diff's
|
||||
# page gives for it.
|
||||
excerpt="$(sed -n 's/.*data-fetch-url="\([^"]*blob_excerpt[^"]*\)".*/\1/p' \
|
||||
"$WORK/answer" | head -n 1 | sed 's/&/\&/g')"
|
||||
[ -n "$excerpt" ] || fail "the diff's page has no link to its hidden lines"
|
||||
page issue-30-case "$excerpt"
|
||||
|
||||
# Cases: the commit graph of one file, and the pages of files whose
|
||||
# path the Core Rule Set takes for an attack.
|
||||
page issue-30-case "/owner/repo/graph?file=src/java.base/share/classes/java/lang/Runtime.java"
|
||||
page issue-30-case /owner/repo/src/branch/main/META-INF/services/org.apache.commons.logging.LogFactory
|
||||
page issue-30-case /owner/repo/src/branch/main/java.lang.String.md
|
||||
page issue-30-case /owner/repo/src/branch/main/process.env.d.ts
|
||||
page issue-30-case /owner/repo/src/branch/main/.../notes.txt
|
||||
}
|
||||
|
||||
# call_api: the API's calls the cases name: the commits of a file, for a
|
||||
# file of each name the Core Rule Set takes for an attack in the query
|
||||
# parameter path, and the file-contents call, which takes its list of
|
||||
# files as JSON in the query parameter body.
|
||||
call_api() {
|
||||
for file in \
|
||||
src/java.base/share/classes/java/lang/Runtime.java \
|
||||
java.lang.String.md \
|
||||
document.write.js \
|
||||
process.env.d.ts \
|
||||
test/fixtures/__proto__.json \
|
||||
install-script.sh \
|
||||
src/backend/catalog/information_schema.sql; do
|
||||
api issue-30-case /api/v1/repos/owner/repo/commits --get --data-urlencode "path=$file"
|
||||
done
|
||||
api issue-30-case /api/v1/repos/owner/repo/file-contents --get \
|
||||
--data-urlencode 'body={"files":["src/java.base/share/classes/java/lang/Runtime.java"]}'
|
||||
}
|
||||
|
||||
# push_images: push a container image to owner/python3-app, then the same
|
||||
# image to owner/app. For the second, the case, docker asks gitea to take
|
||||
# its layer from owner/python3-app, naming it in the query parameter
|
||||
# from. docker sends no X-Request-ID, so its requests have the ids
|
||||
# smallwebwaf gives them. The image is one small file, imported rather
|
||||
# than built.
|
||||
push_images() {
|
||||
echo "$NAME" >"$WORK/layer.txt"
|
||||
tar -cf - -C "$WORK" layer.txt |
|
||||
docker import - "$address/owner/python3-app:1" >/dev/null
|
||||
docker tag "$address/owner/python3-app:1" "$address/owner/app:1"
|
||||
|
||||
echo "$PASSWORD" | DOCKER_CONFIG="$WORK/docker" docker login --username owner \
|
||||
--password-stdin "$address" >/dev/null 2>&1 || fail "docker login failed"
|
||||
for image in "$address/owner/python3-app:1" "$address/owner/app:1"; do
|
||||
DOCKER_CONFIG="$WORK/docker" docker push --quiet "$image" >/dev/null ||
|
||||
echo "gitea-requests: the push of $image failed" >&2
|
||||
done
|
||||
}
|
||||
|
||||
# clone_and_fetch: the clone and fetch over HTTP of the sample.
|
||||
clone_and_fetch() {
|
||||
repo_git -c "http.extraHeader=X-Request-ID: git-clone" clone --quiet \
|
||||
"http://owner:$PASSWORD@$address/owner/repo.git" "$WORK/clone" ||
|
||||
fail "the clone failed"
|
||||
repo_git -C "$WORK/clone" -c "http.extraHeader=X-Request-ID: git-fetch" fetch --quiet ||
|
||||
fail "the fetch failed"
|
||||
}
|
||||
|
||||
# field <name> <line>: the text of the field name in line, a line of the
|
||||
# request log, or nothing when it has none.
|
||||
field() {
|
||||
printf '%s\n' "$2" | sed -n "s/.*\"$1\":\"\([^\"]*\)\".*/\1/p"
|
||||
}
|
||||
|
||||
# report: print a line for each request smallwebwaf logged, but its
|
||||
# health checks: what it did with the request, the ids of the Core Rule
|
||||
# Set's rules that matched it, its X-Request-ID, and the request.
|
||||
# waf_blocked is a refusal by the Core Rule Set, and forward a request
|
||||
# passed on to gitea, whatever gitea answered.
|
||||
report() {
|
||||
docker logs "$SMALLWEBWAF" 2>/dev/null | grep '"type":"request"' |
|
||||
grep -v '"action":"admin"' | sed 's/\\u0026/\&/g' >"$WORK/requests"
|
||||
while IFS= read -r line; do
|
||||
ids="$(printf '%s\n' "$line" | sed -n 's/.*"waf_rule_ids":\[\([0-9,]*\)\].*/\1/p')"
|
||||
query="$(field query "$line")"
|
||||
printf '%-11s %-13s %-17s %s %s%s\n' "$(field action "$line")" "${ids:--}" \
|
||||
"$(field request_id "$line")" "$(field method "$line")" \
|
||||
"$(field path "$line")" "${query:+?$query}"
|
||||
done <"$WORK/requests"
|
||||
echo "gitea-requests: $(grep -c . "$WORK/requests") requests," \
|
||||
"$(grep -c '"action":"waf_blocked"' "$WORK/requests") refused by the Core Rule Set"
|
||||
}
|
||||
|
||||
main() {
|
||||
cd "$ROOT"
|
||||
trap cleanup EXIT
|
||||
trap 'exit 1' HUP INT TERM
|
||||
WORK="$(mktemp -d)"
|
||||
|
||||
docker build --no-cache -t "$IMAGE" .
|
||||
|
||||
# smallwebwaf first, so that gitea can be given the address it is
|
||||
# reached at, and gitea's image pulled before, so that gitea starts
|
||||
# within the minute smallwebwaf's health check waits for it.
|
||||
docker pull --quiet "$GITEA_IMAGE" >/dev/null
|
||||
docker network create "$NETWORK" >/dev/null
|
||||
docker run --detach --name "$SMALLWEBWAF" --network "$NETWORK" \
|
||||
--publish 127.0.0.1::8080 \
|
||||
--env SWWAF_UPSTREAM_URL="http://$GITEA:3000" \
|
||||
"$IMAGE" >/dev/null
|
||||
published="$(docker port "$SMALLWEBWAF" 8080/tcp)"
|
||||
address="localhost:${published##*:}"
|
||||
|
||||
docker run --detach --name "$GITEA" --network "$NETWORK" \
|
||||
--env GITEA__security__INSTALL_LOCK=true \
|
||||
--env GITEA__server__ROOT_URL="http://$address/" \
|
||||
"$GITEA_IMAGE" >/dev/null
|
||||
# smallwebwaf's health check passes once gitea takes connections.
|
||||
wait_for "the health check did not pass" healthy
|
||||
docker exec --user git "$GITEA" gitea admin user create --admin \
|
||||
--username owner --password "$PASSWORD" --email owner@example.invalid \
|
||||
--must-change-password=false >/dev/null
|
||||
|
||||
make_repository
|
||||
browse
|
||||
call_api
|
||||
push_images
|
||||
clone_and_fetch
|
||||
|
||||
# The fetch's last request is the last of all; once it is logged, so
|
||||
# is every other.
|
||||
wait_for "smallwebwaf logged no line for the fetch" \
|
||||
logged '"request_id":"git-fetch"' '"method":"POST"'
|
||||
report
|
||||
}
|
||||
|
||||
main "$@"
|
||||
Reference in New Issue
Block a user