Compare commits
6
Commits
1e70698cfc
..
next
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
eafcc3fd73 | ||
|
|
da0c182a47 | ||
|
|
4cc492004a | ||
|
|
69178da59a | ||
|
|
d1d21a0469 | ||
|
|
7acc6ae153 |
@@ -96,11 +96,14 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
|||||||
## What it does so far
|
## What it does so far
|
||||||
|
|
||||||
- Passes each request to the app and the app's answer back unchanged: method,
|
- Passes each request to the app and the app's answer back unchanged: method,
|
||||||
path, query, headers, body and status. Bodies stream through in both
|
path, query, headers, body and status. The one exception is a path starting
|
||||||
directions and are never held whole in memory, but for the part of a request
|
with `//` that holds a character Go escapes, such as `|`: Go cannot send it as
|
||||||
body the Core Rule Set reads, at most `SWWAF_WAF_BODY_LIMIT` and one byte
|
it is, and the app receives it decoded and escaped again, `%2F` as `/`. Bodies
|
||||||
more, which is held until the app is sent it. A WebSocket, or any other
|
stream through in both directions and are never held whole in memory, but for
|
||||||
upgraded connection, passes through, and the timeouts do not cut it.
|
the part of a request body the Core Rule Set reads, at most
|
||||||
|
`SWWAF_WAF_BODY_LIMIT` and one byte more, which is held until the app is sent
|
||||||
|
it. A WebSocket, or any other upgraded connection, passes through, and the
|
||||||
|
timeouts do not cut it.
|
||||||
- Works out the client's address. A TCP peer outside `SWWAF_TRUSTED_PROXIES` is
|
- Works out the client's address. A TCP peer outside `SWWAF_TRUSTED_PROXIES` is
|
||||||
the client, and the forwarded headers it sends are replaced, not passed on.
|
the client, and the forwarded headers it sends are replaced, not passed on.
|
||||||
For a peer inside it, `X-Forwarded-For` is read from the right, and the first
|
For a peer inside it, `X-Forwarded-For` is read from the right, and the first
|
||||||
@@ -110,16 +113,16 @@ in `bin/state` unless `SWWAF_STATE_DIR` is set, and the default rule file of
|
|||||||
`X-Forwarded-Proto`, and `X-Forwarded-For` with the peer added at the end. It
|
`X-Forwarded-Proto`, and `X-Forwarded-For` with the peer added at the end. It
|
||||||
also gets the request's id in `X-Request-ID`, the same id as in the request's
|
also gets the request's id in `X-Request-ID`, the same id as in the request's
|
||||||
log line (see `request_id` in "Request log" below).
|
log line (see `request_id` in "Request log" below).
|
||||||
- Enforces the timeouts and the size limits below. A limit passed before the
|
- Enforces the timeouts and the size limits below. Once a request's line and
|
||||||
response has started gets `smallwebwaf`'s own answer: `408` for a client too
|
headers have been read, a limit passed before the response has started gets
|
||||||
slow to send its request, `413` for a request body that is too large, `504`
|
`smallwebwaf`'s own answer: `408` for a client too slow to send its body,
|
||||||
for an app too slow to answer, and `502` for a response that is too large or
|
`413` for a request body that is too large, `504` for an app too slow to
|
||||||
an app that cannot be reached. A request that announces a body over the limit
|
answer, and `502` for a response that is too large or an app that cannot be
|
||||||
is refused before anything reaches the app. While a request body is still on
|
reached. A request that announces a body over the limit is refused before
|
||||||
its way, a request timeout that runs out answers `408` if `smallwebwaf` was
|
anything reaches the app. While a request body is still on its way, a request
|
||||||
waiting for the client to send more, and `504` if it was waiting for the app
|
timeout that runs out answers `408` if `smallwebwaf` was waiting for the
|
||||||
to take what it had. Once the response has started, a limit can only cut the
|
client to send more, and `504` if it was waiting for the app to take what it
|
||||||
connection.
|
had. Once the response has started, a limit can only cut the connection.
|
||||||
- Counts each client's requests over a minute, an hour and a day. A request that
|
- Counts each client's requests over a minute, an hour and a day. A request that
|
||||||
takes the client over one of the rate limits below is refused with
|
takes the client over one of the rate limits below is refused with
|
||||||
`SWWAF_BAN_RESPONSE`, `403` by default, before anything reaches the app, and
|
`SWWAF_BAN_RESPONSE`, `403` by default, before anything reaches the app, and
|
||||||
@@ -1042,9 +1045,11 @@ less severe than `SWWAF_LOG_LEVEL` are not written.
|
|||||||
Go's HTTP server, on which `smallwebwaf` is built, reads a request's line and
|
Go's HTTP server, on which `smallwebwaf` is built, reads a request's line and
|
||||||
headers before `smallwebwaf` sees the request, and some requests end there,
|
headers before `smallwebwaf` sees the request, and some requests end there,
|
||||||
without a line in the log: headers over `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`,
|
without a line in the log: headers over `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES`,
|
||||||
which it answers `431`, headers slower than `SWWAF_CLIENT_REQUEST_TIMEOUT`,
|
which it answers `431`, and requests it cannot read at all, which it answers
|
||||||
whose connection it closes without an answer, and requests it cannot read at
|
itself, mostly with `400`. A client too slow to send its request line and
|
||||||
all, which it answers itself, mostly with `400`.
|
headers within `SWWAF_CLIENT_REQUEST_TIMEOUT` gets either no answer or an error
|
||||||
|
answer from the server, mostly `400 Bad Request`; either way the server closes
|
||||||
|
its connection, nothing reaches the app, and no line is written in the log.
|
||||||
|
|
||||||
### Sending the log to a syslog server
|
### Sending the log to a syslog server
|
||||||
|
|
||||||
@@ -1444,11 +1449,12 @@ whole service, its `asn`, for an AS number, its `name`, for a named netblock, or
|
|||||||
the `start` of a window in which it has requests or bytes; a list's `url`,
|
the `start` of a window in which it has requests or bytes; a list's `url`,
|
||||||
`fetched` or `lines`, which is `[]` for an empty list; a verdict's `zone`,
|
`fetched` or `lines`, which is `[]` for an empty list; a verdict's `zone`,
|
||||||
`client`, `listed`, which is `false` for a client the zone does not list, or
|
`client`, `listed`, which is `false` for a client the zone does not list, or
|
||||||
`fetched`. So does a ban whose `cause` is not `limit`, `attack`, `admin` or
|
`fetched`. So does a ban whose `expires` is not after its `start` or whose
|
||||||
`crowdsec`, alerts waiting for a destination that is not `webhook`, `slack` or
|
`cause` is not `limit`, `attack`, `admin` or `crowdsec`, alerts waiting for a
|
||||||
`ntfy`, an anomaly counter whose `scope` is not `client`, `net`, `asn`, `total`
|
destination that is not `webhook`, `slack` or `ntfy`, an anomaly counter whose
|
||||||
or `watch`, and a copy of a list with a line that would make its fetch fail. An
|
`scope` is not `client`, `net`, `asn`, `total` or `watch`, and a copy of a list
|
||||||
answer's `asn` or `as_name` left out reads as empty.
|
with a line that would make its fetch fail. An answer's `asn` or `as_name` left
|
||||||
|
out reads as empty.
|
||||||
|
|
||||||
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
|
While it runs, `smallwebwaf` watches `SWWAF_STATE_DIR` and takes in your edit of
|
||||||
a state file as soon as you save it: what the file then holds replaces what
|
a state file as soon as you save it: what the file then holds replaces what
|
||||||
@@ -1458,14 +1464,15 @@ writes a file it takes in any edit made since, so your edit is not overwritten;
|
|||||||
a change `smallwebwaf` made after you opened the file, such as a new ban, is
|
a change `smallwebwaf` made after you opened the file, such as a new ban, is
|
||||||
lost when you save over it. An edit that would stop the start, because it does
|
lost when you save over it. An edit that would stop the start, because it does
|
||||||
not parse, has another `version`, leaves out a field an entry needs, gives a ban
|
not parse, has another `version`, leaves out a field an entry needs, gives a ban
|
||||||
another `cause`, names another destination, gives an anomaly counter another
|
an `expires` not after its `start` or another `cause`, names another
|
||||||
`scope` or gives a list's copy a line that would make its fetch fail, does not
|
destination, gives an anomaly counter another `scope` or gives a list's copy a
|
||||||
stop the running `smallwebwaf`: it keeps what it holds, and at the file's next
|
line that would make its fetch fail, does not stop the running `smallwebwaf`: it
|
||||||
write renames your file to `<name>.bad`, such as `bans.json.bad`, writes the
|
keeps what it holds, and at the file's next write renames your file to
|
||||||
file again from memory, logs the file and where the error is, and raises a
|
`<name>.bad`, such as `bans.json.bad`, writes the file again from memory, logs
|
||||||
`file_error` alert for it. It waits for that write because an editor's file can
|
the file and where the error is, and raises a `file_error` alert for it. It
|
||||||
be read before the editor has finished writing it. Mend the `.bad` file and move
|
waits for that write because an editor's file can be read before the editor has
|
||||||
it back. A file you remove is written again at its next write.
|
finished writing it. Mend the `.bad` file and move it back. A file you remove is
|
||||||
|
written again at its next write.
|
||||||
|
|
||||||
To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start`
|
To ban a netblock, add an entry to `bans.json` with its `netblock`, its `start`
|
||||||
and its `expires`, `null` for a ban that never ends; its `reason` and its
|
and its `expires`, `null` for a ban that never ends; its `reason` and its
|
||||||
@@ -2153,13 +2160,14 @@ addresses sends, so that one client costs at most one check every
|
|||||||
`SWWAF_REPUTATION_CACHE_TTL`.
|
`SWWAF_REPUTATION_CACHE_TTL`.
|
||||||
|
|
||||||
Only a client whose history counts an offence is checked, so that the checks are
|
Only a client whose history counts an offence is checked, so that the checks are
|
||||||
spent on suspects: so far, one that has broken a rate limit or a byte limit,
|
spent on suspects: so far, one that has broken a rate limit, a byte limit or the
|
||||||
matched a ban rule, or had a request refused by a block rule or the Core Rule
|
error burst, matched a ban rule, asked for a trap path, or had a request refused
|
||||||
Set. A client dropped from the table of clients loses its history, and with it
|
by a block rule, by the Core Rule Set or for a missing or wrong token. A client
|
||||||
its offences. A client is checked in the background, at its first request after
|
dropped from the table of clients loses its history, and with it its offences. A
|
||||||
its offence that reaches the check: no request waits, a request refused under
|
client is checked in the background, at its first request after its offence that
|
||||||
its ban is not checked, and the request that has it checked, and any other from
|
reaches the check: no request waits, a request refused under its ban is not
|
||||||
it before the answer comes, goes on as from a client without a score.
|
checked, and the request that has it checked, and any other from it before the
|
||||||
|
answer comes, goes on as from a client without a score.
|
||||||
|
|
||||||
A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE`, 75 by default, is a hit, and
|
A score at or over `SWWAF_ABUSEIPDB_MIN_SCORE`, 75 by default, is a hit, and
|
||||||
`SWWAF_REPUTATION_ACTION` says what is done with its client, as for a DNSBL
|
`SWWAF_REPUTATION_ACTION` says what is done with its client, as for a DNSBL
|
||||||
@@ -2376,10 +2384,13 @@ so that they run in minimal containers.
|
|||||||
that the health check passes, that a request reaches the app through
|
that the health check passes, that a request reaches the app through
|
||||||
`smallwebwaf`, that a second request in a minute bans the client, that a probe
|
`smallwebwaf`, that a second request in a minute bans the client, that a probe
|
||||||
for `/.env` bans another client, whose next request makes the ban permanent,
|
for `/.env` bans another client, whose next request makes the ban permanent,
|
||||||
that `sv stop` and `docker stop` stop it in order, and that a new container on
|
that `sv stop` and `docker stop` stop it in order, that a new container on the
|
||||||
the same volume still refuses the banned client; then removes the containers,
|
same volume still refuses the banned client, that a container with its state
|
||||||
the volume and both images. It needs network access, for nixpkgs' binary
|
files on a new volume mounted at the directory `SWWAF_STATE_DIR_FILE` names
|
||||||
cache, and `script/check` does not run it; `make example-app` does.
|
becomes healthy, and that one whose `SWWAF_STATE_DIR` does not exist logs
|
||||||
|
`smallwebwaf`'s message about it; then removes the containers, the volumes and
|
||||||
|
both images. It needs network access, for nixpkgs' binary cache, and
|
||||||
|
`script/check` does not run it; `make example-app` does.
|
||||||
- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind it,
|
- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind it,
|
||||||
with every setting of `smallwebwaf` at its default but `SWWAF_UPSTREAM_URL`,
|
with every setting of `smallwebwaf` at its default but `SWWAF_UPSTREAM_URL`,
|
||||||
which names gitea's container. Of the requests that three of the notes
|
which names gitea's container. Of the requests that three of the notes
|
||||||
|
|||||||
@@ -225,9 +225,9 @@ address or one /64, so a permanent ban does not reach neighbours who did
|
|||||||
nothing.
|
nothing.
|
||||||
|
|
||||||
An offence is a request `smallwebwaf` holds against the client: one that carries
|
An offence is a request `smallwebwaf` holds against the client: one that carries
|
||||||
a clear sign of attack, breaks a limit, or is refused by a rule file or the Core
|
a clear sign of attack, breaks a limit, or is refused by a rule file, by the
|
||||||
Rule Set. Offences are counted by kind in the client's history. Two kinds lead
|
Core Rule Set or for a missing or wrong token. Offences are counted by kind in
|
||||||
to a ban, each by its own rule:
|
the client's history. Two kinds lead to a ban, each by its own rule:
|
||||||
|
|
||||||
- A clear sign of attack: a request that matches a rule file rule whose action
|
- A clear sign of attack: a request that matches a rule file rule whose action
|
||||||
is `ban`, or asks for a path in `SWWAF_TRAP_PATHS`. Examples are a probe for a
|
is `ban`, or asks for a path in `SWWAF_TRAP_PATHS`. Examples are a probe for a
|
||||||
@@ -414,10 +414,10 @@ The settings, by group:
|
|||||||
to send its request line and headers, and then, from the end of the
|
to send its request line and headers, and then, from the end of the
|
||||||
headers, its body.
|
headers, its body.
|
||||||
- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest
|
- `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` (default `32K`): the largest
|
||||||
request line and headers a client may send. Over it, `smallwebwaf` answers
|
request line and headers a client may send. Over it, Go's HTTP server
|
||||||
`431` and closes the connection, and nothing reaches the app. It must be
|
answers `431` and closes the connection, and nothing reaches the app. It
|
||||||
more than `4K`, and cannot be `off`: Go's HTTP server always has such a
|
must be more than `4K`, and cannot be `off`: Go's HTTP server always has
|
||||||
limit, and reads 4 KiB past the one it is given before it refuses.
|
such a limit, and reads 4 KiB past the one it is given before it refuses.
|
||||||
- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open
|
- `SWWAF_CLIENT_IDLE_TIMEOUT` (default `120s`): how long a kept-open
|
||||||
connection may wait for its next request before `smallwebwaf` closes it.
|
connection may wait for its next request before `smallwebwaf` closes it.
|
||||||
It is longer than the 90 seconds after which traefik, by default, closes a
|
It is longer than the 90 seconds after which traefik, by default, closes a
|
||||||
@@ -435,12 +435,13 @@ The settings, by group:
|
|||||||
the client sends it and the app receives it.
|
the client sends it and the app receives it.
|
||||||
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body, as
|
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body, as
|
||||||
the app sends it and the client receives it.
|
the app sends it and the client receives it.
|
||||||
- When a limit is passed before the response has started, `smallwebwaf`
|
- When a limit is passed after a request's line and headers have been read,
|
||||||
answers itself: `413` for a request body that is too large, `408` for a
|
but before the response has started, `smallwebwaf` answers itself: `413`
|
||||||
client that is too slow, `502` for a response that is too large, `504` for
|
for a request body that is too large, `408` for a client too slow to send
|
||||||
an app that is too slow. A request that announces a body larger than its
|
its body, `502` for a response that is too large, `504` for an app that is
|
||||||
limit is refused before anything reaches the app. Once the response has
|
too slow. A request that announces a body larger than its limit is refused
|
||||||
started it can only be cut off, and the connection is closed.
|
before anything reaches the app. Once the response has started it can only
|
||||||
|
be cut off, and the connection is closed.
|
||||||
- Since a request body streams through, each side can hold up the other: a
|
- Since a request body streams through, each side can hold up the other: a
|
||||||
slow client slows the send to the app, and an app slow to take the body
|
slow client slows the send to the app, and an app slow to take the body
|
||||||
slows the client's send. So while a request body is still on its way, a
|
slows the client's send. So while a request body is still on its way, a
|
||||||
@@ -450,10 +451,11 @@ The settings, by group:
|
|||||||
waiting for the app to take what it had.
|
waiting for the app to take what it had.
|
||||||
- Go's HTTP server, on which `smallwebwaf` is built, reads a request's line
|
- Go's HTTP server, on which `smallwebwaf` is built, reads a request's line
|
||||||
and headers before `smallwebwaf` sees the request. A client that takes
|
and headers before `smallwebwaf` sees the request. A client that takes
|
||||||
longer than `SWWAF_CLIENT_REQUEST_TIMEOUT` to send them gets no answer:
|
longer than `SWWAF_CLIENT_REQUEST_TIMEOUT` to send them gets either no
|
||||||
the server closes its connection. Headers over
|
answer or an error answer from the server, mostly `400 Bad Request`;
|
||||||
`SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` are answered `431` by the server
|
either way the server closes its connection and nothing reaches the app.
|
||||||
itself. Neither request gets a line in the request log.
|
Headers over `SWWAF_CLIENT_REQUEST_HEADER_MAX_BYTES` are answered `431` by
|
||||||
|
the server itself. Neither request gets a line in the request log.
|
||||||
- A WebSocket connection leaves these limits behind once it is upgraded: it
|
- A WebSocket connection leaves these limits behind once it is upgraded: it
|
||||||
stays open until either side closes it.
|
stays open until either side closes it.
|
||||||
- Lookup of AS number and country (R7). On by default through GeoJS, which needs
|
- Lookup of AS number and country (R7). On by default through GeoJS, which needs
|
||||||
|
|||||||
@@ -23,7 +23,7 @@ import (
|
|||||||
// limit while the Core Rule Set reads it, which it notes nothing of.
|
// limit while the Core Rule Set reads it, which it notes nothing of.
|
||||||
func (rq *request) checkCoreRuleSet() string {
|
func (rq *request) checkCoreRuleSet() string {
|
||||||
cfg := rq.h.config
|
cfg := rq.h.config
|
||||||
if cfg.WAFMode == config.WAFModeOff || pathExempt(rq.in.URL, cfg.WAFExemptPaths) {
|
if cfg.WAFMode == config.WAFModeOff || pathExempt(rq.in, cfg.WAFExemptPaths) {
|
||||||
return ""
|
return ""
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -74,6 +74,47 @@ func TestPassesRequestAndAnswerUnchanged(t *testing.T) {
|
|||||||
wantRequestFields(t, line, addr, len(requestBody), len(answerBody))
|
wantRequestFields(t, line, addr, len(requestBody), len(answerBody))
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestPassesThePathAsSent(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
for _, path := range []string{
|
||||||
|
// An encoded slash and a character Go escapes: |, ^, and é sent
|
||||||
|
// as its UTF-8 bytes.
|
||||||
|
"/a%2Fb|c", "/feat%2Fx/file^1", "/x%2Fy/café",
|
||||||
|
// A path starting with //, which Go would send as a scheme and
|
||||||
|
// host if it were put in URL.Opaque.
|
||||||
|
"//a%2Fb",
|
||||||
|
} {
|
||||||
|
t.Run(path, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
saw := make(chan string, 1)
|
||||||
|
app := startApp(t, func(_ http.ResponseWriter, r *http.Request) {
|
||||||
|
saw <- r.RequestURI
|
||||||
|
})
|
||||||
|
addr, out := startProxy(t, app.URL, nil)
|
||||||
|
|
||||||
|
conn := dial(t, addr)
|
||||||
|
send(t, conn, "GET "+path+"?q=1 HTTP/1.1\r\nHost: "+appHost+"\r\n\r\n")
|
||||||
|
|
||||||
|
got := readResponse(t, conn)
|
||||||
|
if got.status != http.StatusOK {
|
||||||
|
t.Fatalf("status %d, want %d", got.status, http.StatusOK)
|
||||||
|
}
|
||||||
|
|
||||||
|
target := <-saw
|
||||||
|
if target != path+"?q=1" {
|
||||||
|
t.Errorf("app saw %s, want %s", target, path+"?q=1")
|
||||||
|
}
|
||||||
|
|
||||||
|
line := out.requestLine(t)
|
||||||
|
if line.Path != path {
|
||||||
|
t.Errorf("log line has path %s, want %s", line.Path, path)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// wantAppSaw checks that the app received the test's request unchanged.
|
// wantAppSaw checks that the app received the test's request unchanged.
|
||||||
func wantAppSaw(t *testing.T, saw appSaw, body []byte) {
|
func wantAppSaw(t *testing.T, saw appSaw, body []byte) {
|
||||||
t.Helper()
|
t.Helper()
|
||||||
@@ -168,18 +209,13 @@ func TestAddsNoContentTypeTheAppDidNotSend(t *testing.T) {
|
|||||||
})
|
})
|
||||||
addr, _ := startProxy(t, app.URL, nil)
|
addr, _ := startProxy(t, app.URL, nil)
|
||||||
|
|
||||||
// Go's server guesses a Content-Type for an answer without one only
|
got := get(t, addr, "/")
|
||||||
// when the first part of the body goes out with the headers, which
|
if got.header.Values("Content-Type") != nil {
|
||||||
// happens to some answers and not others: the test asks many times.
|
t.Errorf("client got Content-Type %q, want none",
|
||||||
for range 50 {
|
got.header.Get("Content-Type"))
|
||||||
got := get(t, addr, "/")
|
|
||||||
if got.header.Values("Content-Type") != nil {
|
|
||||||
t.Fatalf("client got Content-Type %q, want none",
|
|
||||||
got.header.Get("Content-Type"))
|
|
||||||
}
|
|
||||||
}
|
}
|
||||||
|
|
||||||
got := get(t, addr, "/typed")
|
got = get(t, addr, "/typed")
|
||||||
if got.header.Get("Content-Type") != typed {
|
if got.header.Get("Content-Type") != typed {
|
||||||
t.Errorf("client got Content-Type %q, want %q",
|
t.Errorf("client got Content-Type %q, want %q",
|
||||||
got.header.Get("Content-Type"), typed)
|
got.header.Get("Content-Type"), typed)
|
||||||
|
|||||||
@@ -185,6 +185,9 @@ func TestRateLimitCountsPathsThatAreNotExempt(t *testing.T) {
|
|||||||
"/assets/x%2Fy",
|
"/assets/x%2Fy",
|
||||||
"/assets/x%2fy",
|
"/assets/x%2fy",
|
||||||
`/assets/x\y`,
|
`/assets/x\y`,
|
||||||
|
// The same with a character Go escapes, |, for which Go's
|
||||||
|
// URL.EscapedPath gives /assets/x/y%7Cz, with no encoded slash.
|
||||||
|
"/assets/x%2Fy|z",
|
||||||
} {
|
} {
|
||||||
t.Run(sent, func(t *testing.T) {
|
t.Run(sent, func(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|||||||
@@ -8,7 +8,6 @@ import (
|
|||||||
"net/http/httptrace"
|
"net/http/httptrace"
|
||||||
"net/http/httputil"
|
"net/http/httputil"
|
||||||
"net/netip"
|
"net/netip"
|
||||||
"net/url"
|
|
||||||
"os"
|
"os"
|
||||||
"slices"
|
"slices"
|
||||||
"strings"
|
"strings"
|
||||||
@@ -22,6 +21,7 @@ import (
|
|||||||
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
"sneak.berlin/go/smallwebwaf/internal/lookup"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
"sneak.berlin/go/smallwebwaf/internal/ratelimit"
|
||||||
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
"sneak.berlin/go/smallwebwaf/internal/requestlog"
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/rules"
|
||||||
)
|
)
|
||||||
|
|
||||||
// flushAfterEachWrite has ReverseProxy pass on each part of the app's
|
// flushAfterEachWrite has ReverseProxy pass on each part of the app's
|
||||||
@@ -137,7 +137,7 @@ func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
|
|||||||
Method: r.Method,
|
Method: r.Method,
|
||||||
Scheme: scheme(r, peerTrusted),
|
Scheme: scheme(r, peerTrusted),
|
||||||
Host: r.Host,
|
Host: r.Host,
|
||||||
Path: r.URL.EscapedPath(),
|
Path: rules.Path(r),
|
||||||
Query: r.URL.RawQuery,
|
Query: r.URL.RawQuery,
|
||||||
Protocol: r.Proto,
|
Protocol: r.Proto,
|
||||||
Referer: r.Referer(),
|
Referer: r.Referer(),
|
||||||
@@ -284,7 +284,7 @@ func (rq *request) checkClient(ctx context.Context) string {
|
|||||||
}
|
}
|
||||||
|
|
||||||
rq.counted = !isInside(rq.client, cfg.RateLimitExemptNets) &&
|
rq.counted = !isInside(rq.client, cfg.RateLimitExemptNets) &&
|
||||||
!pathExempt(rq.in.URL, cfg.RateLimitExemptPaths)
|
!pathExempt(rq.in, cfg.RateLimitExemptPaths)
|
||||||
if rq.counted {
|
if rq.counted {
|
||||||
rq.limitPercent, rq.bytesPercent = rq.limitPercentages()
|
rq.limitPercent, rq.bytesPercent = rq.limitPercentages()
|
||||||
rq.line.LimitPercent, rq.line.LimitPercentSetting = rq.limitPercent.logged()
|
rq.line.LimitPercent, rq.line.LimitPercentSetting = rq.limitPercent.logged()
|
||||||
@@ -307,7 +307,7 @@ func (rq *request) checkClient(ctx context.Context) string {
|
|||||||
return rq.checkCoreRuleSet()
|
return rq.checkCoreRuleSet()
|
||||||
}
|
}
|
||||||
|
|
||||||
// pathExempt reports whether a request for u is exempt under prefixes,
|
// pathExempt reports whether r is exempt under prefixes,
|
||||||
// SWWAF_RATE_LIMIT_EXEMPT_PATHS or SWWAF_WAF_EXEMPT_PATHS: whether its
|
// SWWAF_RATE_LIMIT_EXEMPT_PATHS or SWWAF_WAF_EXEMPT_PATHS: whether its
|
||||||
// path as sent, the path the app receives, not percent-decoded, starts
|
// path as sent, the path the app receives, not percent-decoded, starts
|
||||||
// with one of prefixes, so that /%61ssets/x is not under /assets/ for an
|
// with one of prefixes, so that /%61ssets/x is not under /assets/ for an
|
||||||
@@ -316,10 +316,9 @@ func (rq *request) checkClient(ctx context.Context) string {
|
|||||||
// encoded slash (%2F or %2f), never is, since an app may act on it as a
|
// encoded slash (%2F or %2f), never is, since an app may act on it as a
|
||||||
// path outside every prefix: /assets/..%2Flogin as /login, or /assets%2Fx
|
// path outside every prefix: /assets/..%2Flogin as /login, or /assets%2Fx
|
||||||
// as one path segment, as Go's router does.
|
// as one path segment, as Go's router does.
|
||||||
func pathExempt(u *url.URL, prefixes []string) bool {
|
func pathExempt(r *http.Request, prefixes []string) bool {
|
||||||
decoded := u.Path
|
decoded := r.URL.Path
|
||||||
// EscapedPath is the path as the app receives it, not decoded.
|
sent := rules.Path(r)
|
||||||
sent := u.EscapedPath()
|
|
||||||
|
|
||||||
if strings.Contains(decoded, "..") || strings.Contains(decoded, `\`) ||
|
if strings.Contains(decoded, "..") || strings.Contains(decoded, `\`) ||
|
||||||
strings.Contains(strings.ToLower(sent), "%2f") {
|
strings.Contains(strings.ToLower(sent), "%2f") {
|
||||||
@@ -375,6 +374,17 @@ func (rq *request) rewrite(pr *httputil.ProxyRequest) {
|
|||||||
// ReverseProxy drops query parameters it cannot parse; the app gets
|
// ReverseProxy drops query parameters it cannot parse; the app gets
|
||||||
// the query as the client sent it.
|
// the query as the client sent it.
|
||||||
pr.Out.URL.RawQuery = pr.In.URL.RawQuery
|
pr.Out.URL.RawQuery = pr.In.URL.RawQuery
|
||||||
|
// ReverseProxy sends the path URL.EscapedPath gives, which, for a path
|
||||||
|
// holding a character Go escapes, such as |, is the whole path decoded
|
||||||
|
// and escaped again, %2F turned into /. Opaque is sent as it is, so the
|
||||||
|
// app gets the path as the client sent it. Left out are a target that
|
||||||
|
// is no path, such as *, and a path starting with //, which Go would
|
||||||
|
// send in Opaque as a scheme and host.
|
||||||
|
path := rules.Path(pr.In)
|
||||||
|
if strings.HasPrefix(path, "/") && !strings.HasPrefix(path, "//") {
|
||||||
|
pr.Out.URL.Opaque = path
|
||||||
|
}
|
||||||
|
|
||||||
setForwardedHeaders(pr.In, pr.Out, rq.peer, rq.peerTrusted)
|
setForwardedHeaders(pr.In, pr.Out, rq.peer, rq.peerTrusted)
|
||||||
pr.Out.Header.Set(requestIDHeader, rq.line.RequestID)
|
pr.Out.Header.Set(requestIDHeader, rq.line.RequestID)
|
||||||
pr.Out.Header.Del(asnHeader)
|
pr.Out.Header.Del(asnHeader)
|
||||||
|
|||||||
@@ -0,0 +1,55 @@
|
|||||||
|
package proxy
|
||||||
|
|
||||||
|
import (
|
||||||
|
"io"
|
||||||
|
"net/http"
|
||||||
|
"net/http/httptest"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"sneak.berlin/go/smallwebwaf/internal/config"
|
||||||
|
)
|
||||||
|
|
||||||
|
func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
srv := httptest.NewServer(http.HandlerFunc(
|
||||||
|
func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
rq := &request{
|
||||||
|
h: &handler{config: &config.Config{}},
|
||||||
|
out: &responseWriter{ResponseWriter: w},
|
||||||
|
}
|
||||||
|
app := &http.Response{
|
||||||
|
StatusCode: http.StatusOK,
|
||||||
|
Header: http.Header{},
|
||||||
|
Body: http.NoBody,
|
||||||
|
}
|
||||||
|
|
||||||
|
err := rq.modifyResponse(app)
|
||||||
|
if err != nil {
|
||||||
|
t.Errorf("modifyResponse: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
// The headers go out with this, the first part of the body,
|
||||||
|
// which Go's server would guess a Content-Type from.
|
||||||
|
_, _ = io.WriteString(w, "<!doctype html><title>page</title>")
|
||||||
|
}))
|
||||||
|
t.Cleanup(srv.Close)
|
||||||
|
|
||||||
|
req, err := http.NewRequestWithContext(t.Context(), http.MethodGet, srv.URL,
|
||||||
|
http.NoBody)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("new request: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
res, err := srv.Client().Do(req)
|
||||||
|
if err != nil {
|
||||||
|
t.Fatalf("get: %v", err)
|
||||||
|
}
|
||||||
|
|
||||||
|
_ = res.Body.Close()
|
||||||
|
|
||||||
|
if res.Header.Values("Content-Type") != nil {
|
||||||
|
t.Errorf("client got Content-Type %q, want none",
|
||||||
|
res.Header.Get("Content-Type"))
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -61,6 +61,7 @@ var (
|
|||||||
// errMissing is for an entry without a field it needs.
|
// errMissing is for an entry without a field it needs.
|
||||||
errMissing = errors.New("has no")
|
errMissing = errors.New("has no")
|
||||||
errCause = errors.New("is not limit, attack, admin or crowdsec")
|
errCause = errors.New("is not limit, attack, admin or crowdsec")
|
||||||
|
errExpires = errors.New("is not after its start")
|
||||||
errDestination = errors.New("is not webhook, slack or ntfy")
|
errDestination = errors.New("is not webhook, slack or ntfy")
|
||||||
errScope = errors.New("is not client, net, asn, total or watch")
|
errScope = errors.New("is not client, net, asn, total or watch")
|
||||||
errWaitingList = errors.New(`waiting is a list, but now lists the alerts by ` +
|
errWaitingList = errors.New(`waiting is a list, but now lists the alerts by ` +
|
||||||
@@ -646,8 +647,10 @@ func (e BanEntry) ban() bans.Ban {
|
|||||||
// client, a start, from which the length of the netblock's next ban is
|
// client, a start, from which the length of the netblock's next ban is
|
||||||
// worked out, or an expires, which would make it permanent. A permanent
|
// worked out, or an expires, which would make it permanent. A permanent
|
||||||
// ban's expires is null, which Bans cannot tell from a missing one, so
|
// ban's expires is null, which Bans cannot tell from a missing one, so
|
||||||
// each expires is read again as written. A cause other than limit,
|
// each expires is read again as written. An expires not after the start
|
||||||
// attack, admin or crowdsec, most likely misspelt, is refused too.
|
// is refused too, since the netblock's next ban for a broken limit, three
|
||||||
|
// times as long as this one, would then refuse nothing; and so is a cause
|
||||||
|
// other than limit, attack, admin or crowdsec, most likely misspelt.
|
||||||
func (f *bansFile) check(data []byte) error {
|
func (f *bansFile) check(data []byte) error {
|
||||||
var written struct {
|
var written struct {
|
||||||
Bans []struct {
|
Bans []struct {
|
||||||
@@ -668,6 +671,8 @@ func (f *bansFile) check(data []byte) error {
|
|||||||
return missing(i, "start")
|
return missing(i, "start")
|
||||||
case written.Bans[i].Expires == nil:
|
case written.Bans[i].Expires == nil:
|
||||||
return missing(i, "expires")
|
return missing(i, "expires")
|
||||||
|
case entry.Expires != nil && !entry.Expires.After(entry.Start):
|
||||||
|
return fmt.Errorf("entry %d's expires %w", i+1, errExpires)
|
||||||
case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
|
case entry.Cause != "" && entry.Cause != bans.CauseLimit &&
|
||||||
entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin &&
|
entry.Cause != bans.CauseAttack && entry.Cause != bans.CauseAdmin &&
|
||||||
entry.Cause != bans.CauseCrowdSec:
|
entry.Cause != bans.CauseCrowdSec:
|
||||||
|
|||||||
@@ -865,6 +865,21 @@ func TestBanWithAnotherCauseStopsTheStart(t *testing.T) {
|
|||||||
`: entry 4's cause "atack" is not limit, attack, admin or crowdsec`)
|
`: entry 4's cause "atack" is not limit, attack, admin or crowdsec`)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
func TestBanThatDoesNotEndAfterItsStartStopsTheStart(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
// The first ban ends a second after its start; the second ends as it
|
||||||
|
// starts, or an hour before.
|
||||||
|
for _, expires := range []string{`"2026-10-06T00:00:00Z"`, `"2026-10-05T23:00:00Z"`} {
|
||||||
|
wantRefused(t, bansJSON, `{"version": 1, "bans": [`+
|
||||||
|
`{"netblock": "203.0.113.9/32", "start": "2026-10-06T00:00:00Z", `+
|
||||||
|
`"expires": "2026-10-06T00:00:01Z"}, `+
|
||||||
|
`{"netblock": "203.0.113.10/32", "start": "2026-10-06T00:00:00Z", `+
|
||||||
|
`"expires": `+expires+`}]}`,
|
||||||
|
`: entry 2's expires is not after its start`)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
func TestUnknownVersionStopsTheStart(t *testing.T) {
|
func TestUnknownVersionStopsTheStart(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
|
|||||||
+49
-6
@@ -5,12 +5,15 @@
|
|||||||
# served through smallwebwaf, that a second one in a minute bans the
|
# served through smallwebwaf, that a second one in a minute bans the
|
||||||
# client, that a probe for /.env bans another client, which its next
|
# client, that a probe for /.env bans another client, which its next
|
||||||
# request bans for good, that `sv stop` stops smallwebwaf in order, that
|
# request bans for good, that `sv stop` stops smallwebwaf in order, that
|
||||||
# `docker stop` stops the container without having to kill it, and that
|
# `docker stop` stops the container without having to kill it, that a
|
||||||
# a new container on the same volume still refuses the banned client. The
|
# new container on the same volume still refuses the banned client, that
|
||||||
|
# a container with its state files on a new volume mounted at the
|
||||||
|
# directory SWWAF_STATE_DIR_FILE names becomes healthy, and that one whose
|
||||||
|
# SWWAF_STATE_DIR does not exist logs smallwebwaf's message about it. The
|
||||||
# containers run with SWWAF_LOOKUP_SOURCE=off, so that no address is sent
|
# containers run with SWWAF_LOOKUP_SOURCE=off, so that no address is sent
|
||||||
# to GeoJS. The containers, the volume and both images are removed however
|
# to GeoJS. The containers, the volumes and both images are removed
|
||||||
# the script ends. Building the app needs network access, for nixpkgs'
|
# however the script ends. Building the app needs network access, for
|
||||||
# binary cache. script/check does not run this.
|
# nixpkgs' binary cache. script/check does not run this.
|
||||||
set -eu
|
set -eu
|
||||||
|
|
||||||
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
|
||||||
@@ -23,10 +26,15 @@ IMAGE="$NAME-base"
|
|||||||
APP_IMAGE="$NAME-app"
|
APP_IMAGE="$NAME-app"
|
||||||
CONTAINER="$NAME"
|
CONTAINER="$NAME"
|
||||||
VOLUME="$NAME-state"
|
VOLUME="$NAME-state"
|
||||||
|
# The volumes of the container whose state directory SWWAF_STATE_DIR_FILE
|
||||||
|
# names: its state files, and the file that names their directory.
|
||||||
|
MOVED_VOLUME="$NAME-moved-state"
|
||||||
|
SETTINGS_VOLUME="$NAME-settings"
|
||||||
|
|
||||||
cleanup() {
|
cleanup() {
|
||||||
docker rm --force "$CONTAINER" >/dev/null 2>&1 || true
|
docker rm --force "$CONTAINER" >/dev/null 2>&1 || true
|
||||||
docker volume rm --force "$VOLUME" >/dev/null 2>&1 || true
|
docker volume rm --force "$VOLUME" "$MOVED_VOLUME" "$SETTINGS_VOLUME" \
|
||||||
|
>/dev/null 2>&1 || true
|
||||||
docker rmi --force "$APP_IMAGE" "$IMAGE" >/dev/null 2>&1 || true
|
docker rmi --force "$APP_IMAGE" "$IMAGE" >/dev/null 2>&1 || true
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -76,6 +84,27 @@ start_container() {
|
|||||||
address="$(docker port "$CONTAINER" 8080/tcp)"
|
address="$(docker port "$CONTAINER" 8080/tcp)"
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# start_moved_container: run the app's container with the state files on
|
||||||
|
# a new volume at /srv/state, which SWWAF_STATE_DIR_FILE names in a file
|
||||||
|
# only the smallwebwaf user can read, as under "Settings given as files"
|
||||||
|
# in README.md, and wait until it is healthy. smallwebwaf can write the
|
||||||
|
# new volume only once the run script has given it to that user.
|
||||||
|
start_moved_container() {
|
||||||
|
docker volume create "$MOVED_VOLUME" >/dev/null
|
||||||
|
docker volume create "$SETTINGS_VOLUME" >/dev/null
|
||||||
|
docker run --rm --volume "$SETTINGS_VOLUME:/settings" \
|
||||||
|
--entrypoint bash "$IMAGE" -c 'echo /srv/state > /settings/state-dir &&
|
||||||
|
chown smallwebwaf:smallwebwaf /settings/state-dir &&
|
||||||
|
chmod 0400 /settings/state-dir'
|
||||||
|
docker run --detach --name "$CONTAINER" \
|
||||||
|
--volume "$MOVED_VOLUME:/srv/state" \
|
||||||
|
--volume "$SETTINGS_VOLUME:/etc/smallwebwaf/settings:ro" \
|
||||||
|
--env SWWAF_STATE_DIR_FILE=/etc/smallwebwaf/settings/state-dir \
|
||||||
|
--env SWWAF_LOOKUP_SOURCE=off \
|
||||||
|
"$APP_IMAGE" >/dev/null
|
||||||
|
wait_for "the health check did not pass" healthy
|
||||||
|
}
|
||||||
|
|
||||||
# refused: a request to the container gets 403, SWWAF_BAN_RESPONSE's
|
# refused: a request to the container gets 403, SWWAF_BAN_RESPONSE's
|
||||||
# default.
|
# default.
|
||||||
refused() {
|
refused() {
|
||||||
@@ -140,6 +169,20 @@ main() {
|
|||||||
wait_for "smallwebwaf logged no request refused under the ban" \
|
wait_for "smallwebwaf logged no request refused under the ban" \
|
||||||
logged '"action":"banned"'
|
logged '"action":"banned"'
|
||||||
echo "example-app: a new container on the same volume keeps the ban"
|
echo "example-app: a new container on the same volume keeps the ban"
|
||||||
|
|
||||||
|
docker rm --force "$CONTAINER" >/dev/null
|
||||||
|
start_moved_container
|
||||||
|
echo "example-app: the run script gives the state files' directory named" \
|
||||||
|
"by SWWAF_STATE_DIR_FILE to the smallwebwaf user"
|
||||||
|
|
||||||
|
docker rm --force "$CONTAINER" >/dev/null
|
||||||
|
docker run --detach --name "$CONTAINER" \
|
||||||
|
--env SWWAF_STATE_DIR=/srv/missing \
|
||||||
|
--env SWWAF_LOOKUP_SOURCE=off \
|
||||||
|
"$APP_IMAGE" >/dev/null
|
||||||
|
wait_for "smallwebwaf did not report the missing state directory" \
|
||||||
|
logged "SWWAF_STATE_DIR cannot be written"
|
||||||
|
echo "example-app: smallwebwaf reports a state directory that does not exist"
|
||||||
}
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
+23
-3
@@ -5,12 +5,32 @@ set -euo pipefail
|
|||||||
# exits; the wait spaces out the restarts. The state directory and every
|
# exits; the wait spaces out the restarts. The state directory and every
|
||||||
# file in it are given to the smallwebwaf user, so that a volume mounted
|
# file in it are given to the smallwebwaf user, so that a volume mounted
|
||||||
# there needs no change of owner; chown -R changes a symbolic link itself,
|
# there needs no change of owner; chown -R changes a symbolic link itself,
|
||||||
# never what it points to. exec, so that the signal `sv stop` sends
|
# never what it points to. When SWWAF_STATE_DIR, in either form, cannot be
|
||||||
# reaches smallwebwaf itself.
|
# read or is not the absolute path of a directory, no owner is changed,
|
||||||
|
# and smallwebwaf starts and stops with a message naming the setting. A
|
||||||
|
# change of owner that fails, as on a volume mounted read-only, does not
|
||||||
|
# end the script either: smallwebwaf reports a directory it cannot write.
|
||||||
|
# exec, so that the signal `sv stop` sends reaches smallwebwaf itself.
|
||||||
main() {
|
main() {
|
||||||
sleep 1
|
sleep 1
|
||||||
chown -R smallwebwaf:smallwebwaf "${SWWAF_STATE_DIR:-/var/lib/smallwebwaf}"
|
local dir
|
||||||
|
if dir="$(state_dir)" && [[ "$dir" == /* && -d "$dir" ]]; then
|
||||||
|
chown -R smallwebwaf:smallwebwaf "$dir" || true
|
||||||
|
fi
|
||||||
exec chpst -u smallwebwaf:smallwebwaf /usr/local/bin/smallwebwaf
|
exec chpst -u smallwebwaf:smallwebwaf /usr/local/bin/smallwebwaf
|
||||||
}
|
}
|
||||||
|
|
||||||
|
# state_dir prints SWWAF_STATE_DIR, /var/lib/smallwebwaf when it is unset,
|
||||||
|
# or what is in the file SWWAF_STATE_DIR_FILE names, less every newline at
|
||||||
|
# its end. It fails when both are set or the file cannot be read.
|
||||||
|
state_dir() {
|
||||||
|
if [[ ! -v SWWAF_STATE_DIR_FILE ]]; then
|
||||||
|
printf '%s\n' "${SWWAF_STATE_DIR-/var/lib/smallwebwaf}"
|
||||||
|
elif [[ ! -v SWWAF_STATE_DIR ]]; then
|
||||||
|
cat -- "$SWWAF_STATE_DIR_FILE" 2>/dev/null
|
||||||
|
else
|
||||||
|
return 1
|
||||||
|
fi
|
||||||
|
}
|
||||||
|
|
||||||
main "$@"
|
main "$@"
|
||||||
|
|||||||
Reference in New Issue
Block a user