Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
3490284e16 |
@@ -10,9 +10,10 @@ refuses the ones that look like abuse. `smallwebwaf` is also a reverse proxy: it
|
||||
accepts requests on the app's behalf, passes them to the app, and passes the
|
||||
app's answers back.
|
||||
|
||||
It needs no setting. Every setting has a default chosen for a service on the
|
||||
open internet. [`SPEC.md`](SPEC.md) says how `smallwebwaf` behaves and why. The
|
||||
reference documents under [`docs/`](docs/) are listed under "Documents" below.
|
||||
It needs no setting. Every `SWWAF_` setting has a default chosen for a service
|
||||
on the open internet. [`SPEC.md`](SPEC.md) says how `smallwebwaf` behaves and
|
||||
why. The reference documents under [`docs/`](docs/) are listed under "Documents"
|
||||
below.
|
||||
|
||||
## What it protects against
|
||||
|
||||
|
||||
@@ -12,10 +12,9 @@ to them.
|
||||
|
||||
- It protects one self-hosted app on the open internet from scrapers, scanners
|
||||
and other abusers. It does so from the app's first request, with no setting at
|
||||
all. Every setting but `SWRP_WAF_BACKEND`, which only `smallwebwaf rproxy`
|
||||
reads, has a default chosen for a service facing the internet in 2026. A
|
||||
setting that needs an alert destination, an account key or a token stays off
|
||||
until the operator supplies it.
|
||||
all. Every `SWWAF_` setting has a default chosen for a service facing the
|
||||
internet in 2026. A setting that needs an alert destination, an account key or
|
||||
a token stays off until the operator supplies it.
|
||||
- It limits each client's requests and bytes per minute, hour and day. It bounds
|
||||
the size and duration of every request and answer. It detects attacks.
|
||||
- It bans abusers: briefly at first, for seven days on a clear sign of attack,
|
||||
@@ -23,10 +22,10 @@ to them.
|
||||
- It looks up each client's AS number and country. It can refuse whole countries
|
||||
and netblocks, lower its limits for listed AS numbers and countries, consult
|
||||
reputation sources, and send alerts.
|
||||
- Everything is set by environment variables whose names start with `SWWAF_`, or
|
||||
with `SWRP_` for `smallwebwaf rproxy`, since `smallwebwaf` shares its
|
||||
container, and so its environment, with the app. The only other inputs are
|
||||
files: the rule files, the state files, a lookup database, and a `SWWAF_`
|
||||
- Everything is set by environment variables whose names start with `SWWAF_`,
|
||||
since `smallwebwaf` shares its container, and so its environment, with the
|
||||
app. `smallwebwaf rproxy` uses `SWRP_` in their place. The only other inputs
|
||||
are files: the rule files, the state files, a lookup database, and a `SWWAF_`
|
||||
setting given as a file.
|
||||
|
||||
## Non-goals
|
||||
@@ -1060,14 +1059,17 @@ request. It is the visitor's request as the visitor sent it:
|
||||
- every header the visitor sent, including its own `X-Forwarded-For`;
|
||||
- the first 64 KiB of the body, or all of it when shorter.
|
||||
|
||||
A body whose length the visitor did not announce comes chunked. Any other body
|
||||
comes with a `Content-Length` of the part it carries.
|
||||
The path always starts with `/`, and `Host` is never empty. `rproxy` refuses any
|
||||
other request itself (see "Requests `rproxy` refuses itself").
|
||||
|
||||
A body whose length the visitor did not announce comes chunked, whatever the
|
||||
method. Any other body comes with a `Content-Length` of the part it carries.
|
||||
|
||||
`rproxy` sets these headers, replacing any the visitor sent, and adds no others:
|
||||
|
||||
- `X-Smallwebwaf-Client`: the visitor's address, which `rproxy` has worked out.
|
||||
`servewaf` takes it as the client, and groups an IPv6 address as "What happens
|
||||
to a request" says.
|
||||
- `X-Smallwebwaf-Client`: the visitor's address, which `rproxy` has worked out,
|
||||
without any IPv6 zone. `servewaf` takes it as the client, and groups an IPv6
|
||||
address as "What happens to a request" says.
|
||||
- `X-Smallwebwaf-Peer`: the address of `rproxy`'s own TCP peer, for the log
|
||||
line's `peer_ip`.
|
||||
- `X-Smallwebwaf-Content-Length`: the body length the visitor announced. It is
|
||||
@@ -1253,23 +1255,48 @@ and health check.
|
||||
`servewaf`.
|
||||
2. `rproxy` works out the visitor's address. It believes `X-Forwarded-For` only
|
||||
from `SWRP_TRUSTED_PROXIES`, and reads it as step 2 of "What happens to a
|
||||
request" says.
|
||||
request" says. It removes any IPv6 zone.
|
||||
3. It makes a fresh request id, whatever id the request came with. `servewaf`
|
||||
and the app get it as `X-Request-ID`, and `rproxy`'s log line gives it.
|
||||
4. It reads up to the first 64 KiB of the body into memory.
|
||||
5. It sends `servewaf` the question (see "The question"). The question never
|
||||
carries the visitor's `Transfer-Encoding` or `Trailer`, which Go's HTTP
|
||||
server and client handle themselves.
|
||||
6. It does what the answer says (see "The answer"):
|
||||
4. It refuses a request it cannot put to `servewaf` as the visitor sent it (see
|
||||
"Requests `rproxy` refuses itself").
|
||||
5. It reads up to the first 64 KiB of the body into memory.
|
||||
6. It sends `servewaf` the question (see "The question"). It speaks HTTP/1.1 to
|
||||
`servewaf`, whatever `servewaf` offers. Go's HTTP/2 client would refuse to
|
||||
send a visitor's `Upgrade` header, and most `Connection` headers. The
|
||||
question never carries the visitor's `Transfer-Encoding` or `Trailer`, which
|
||||
Go's HTTP server and client handle themselves.
|
||||
7. It does what the answer says (see "The answer"):
|
||||
- `allow`: it passes the request to the app.
|
||||
- `deny`: the visitor gets the answer's status, headers and body, less
|
||||
`X-Smallwebwaf-Disposition`.
|
||||
- `close`: it closes the visitor's connection without an answer.
|
||||
7. Once the answer has ended, it writes its log line. `docs/request-log.md`
|
||||
8. Once the answer has ended, it writes its log line. `docs/request-log.md`
|
||||
gives the fields.
|
||||
8. It then sends the report, if `servewaf` answered `allow`, `deny` or `close`
|
||||
9. It then sends the report, if `servewaf` answered `allow`, `deny` or `close`
|
||||
(see "The report"). The visitor does not wait for it. A report that fails is
|
||||
dropped, with a process warning.
|
||||
dropped, with a process warning. A status above `599` from the app goes to
|
||||
the visitor as it is, and into the report as `599`, the highest `servewaf`
|
||||
takes.
|
||||
|
||||
### Requests `rproxy` refuses itself
|
||||
|
||||
How a visitor sends a request never lets it past `servewaf`.
|
||||
`SWRP_DEFAULT_DISPOSITION` applies only when `servewaf` itself is down, slow or
|
||||
answers badly.
|
||||
|
||||
`rproxy` answers `400`, with no question and no report, to a request with:
|
||||
|
||||
- a target that does not start with `/`, such as `x:admin`,
|
||||
`http://site.example/` or `site.example:443`;
|
||||
- no `Host`, or an empty one.
|
||||
|
||||
Go's HTTP server answers `OPTIONS *` itself, before `rproxy` sees it.
|
||||
|
||||
`rproxy` closes the connection without an answer when it cannot read the first
|
||||
64 KiB of the body. That happens for a malformed chunk, and for a visitor that
|
||||
goes away or stops its sending side. There is no question and no report, and the
|
||||
log line has status `0`.
|
||||
|
||||
### Without an answer from `servewaf`
|
||||
|
||||
@@ -1290,6 +1317,10 @@ Either way `rproxy` writes a process warning, its log line has
|
||||
A path under `/_smallwebwaf/` that `allow` would pass on is answered `404`
|
||||
instead, so that the app never sees that prefix.
|
||||
|
||||
A visitor that goes away while `servewaf` is being asked is no failure of
|
||||
`servewaf`. `rproxy` closes the connection, with no warning, no default and no
|
||||
report. Its log line has status `0` and no `disposition`.
|
||||
|
||||
### Passing a request to the app
|
||||
|
||||
`rproxy` sends the app at `SWRP_UPSTREAM_URL` the visitor's request: the part of
|
||||
@@ -1327,10 +1358,9 @@ The largest request line and headers is 32 KiB. A kept-open connection is closed
|
||||
after 120 seconds idle. Neither has a setting.
|
||||
|
||||
`SWRP_MAX_REQUEST_TIME_MS` bounds the request line and headers. From their end,
|
||||
it then bounds everything until the app has been sent the whole request,
|
||||
including the wait for `servewaf`. A visitor too slow to send the first 64 KiB
|
||||
is answered `408`, with no question and no report. A visitor that goes away
|
||||
before then is not answered, with no question and no report either.
|
||||
it then bounds the time until the app has been sent the whole request. The wait
|
||||
for `servewaf` counts toward it. A visitor too slow to send the first 64 KiB is
|
||||
answered `408`, with no question and no report.
|
||||
|
||||
`SWRP_MAX_RESPONSE_TIME_MS` runs from the end of the request to the last byte of
|
||||
the answer reaching the visitor. The request ends when the app has been sent all
|
||||
|
||||
+2
-1
@@ -204,7 +204,8 @@ it to `servewaf`'s two. It has these fields and no others:
|
||||
- `duration_response`: milliseconds from then until the end of the answer.
|
||||
- `disposition`: what was done with the request: `allow`, `deny` or `close`. It
|
||||
is `servewaf`'s answer, or the value of `SWRP_DEFAULT_DISPOSITION`. It is left
|
||||
out when `servewaf` was not asked, as for the health check.
|
||||
out when `servewaf` was not asked, as for the health check, and when the
|
||||
visitor went away while it was being asked.
|
||||
- `default_disposition`: `true` when `SWRP_DEFAULT_DISPOSITION` gave
|
||||
`disposition`, and otherwise left out.
|
||||
|
||||
|
||||
+3
-2
@@ -347,7 +347,7 @@ A few limits are fixed rather than settings:
|
||||
|
||||
## `servewaf`
|
||||
|
||||
`smallwebwaf servewaf` uses every setting here but these, which it ignores:
|
||||
`smallwebwaf servewaf` uses every `SWWAF_` setting but these, which it ignores:
|
||||
|
||||
- `SWWAF_UPSTREAM_URL` and `SWWAF_TRUSTED_PROXIES`;
|
||||
- the five timeouts under "Size and time limits";
|
||||
@@ -372,7 +372,8 @@ None can be given as a file, since `rproxy` never touches disk. "`rproxy`" in
|
||||
A time here is a whole number of milliseconds above zero, such as `2000`.
|
||||
|
||||
- `SWRP_WAF_BACKEND` (required, no default): `servewaf`'s URL. An `http` or
|
||||
`https` URL with a host and an optional port, and nothing more.
|
||||
`https` URL with a host and an optional port, and nothing more. `rproxy`
|
||||
speaks HTTP/1.1 to it.
|
||||
- `SWRP_TIMEOUT_MS` (default `2000`): how long `rproxy` waits for `servewaf`'s
|
||||
answer, to a question and to a report. A time; never `off`.
|
||||
- `SWRP_DEFAULT_DISPOSITION` (default `allow`): what is done with a request
|
||||
|
||||
@@ -17,14 +17,15 @@ const (
|
||||
countryHeader = "X-Client-Country"
|
||||
)
|
||||
|
||||
// peerAddress is the address of the request's TCP peer, normally traefik.
|
||||
// peerAddress is the address of the request's TCP peer, normally traefik, without
|
||||
// any IPv6 zone, which servewaf refuses.
|
||||
func peerAddress(r *http.Request) netip.Addr {
|
||||
addrPort, err := netip.ParseAddrPort(r.RemoteAddr)
|
||||
if err != nil {
|
||||
return netip.Addr{}
|
||||
}
|
||||
|
||||
return addrPort.Addr().Unmap()
|
||||
return addrPort.Addr().Unmap().WithZone("")
|
||||
}
|
||||
|
||||
// clientAddress works out who the visitor is. A peer outside the trusted
|
||||
@@ -32,7 +33,8 @@ func peerAddress(r *http.Request) netip.Addr {
|
||||
// peer inside them, X-Forwarded-For is read from the right, and the first
|
||||
// address outside them is the visitor; if every address in it is inside, the
|
||||
// leftmost is, and with no header, the peer. An entry that is not an address
|
||||
// ends the reading, since nothing to its left can be believed.
|
||||
// ends the reading, since nothing to its left can be believed. The visitor's
|
||||
// address never has an IPv6 zone, which servewaf refuses.
|
||||
func clientAddress(
|
||||
peer netip.Addr, forwardedFor []string, trusted []netip.Prefix,
|
||||
) netip.Addr {
|
||||
@@ -48,7 +50,7 @@ func clientAddress(
|
||||
break
|
||||
}
|
||||
|
||||
client = addr.Unmap()
|
||||
client = addr.Unmap().WithZone("")
|
||||
if !isInside(client, trusted) {
|
||||
break
|
||||
}
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"log/slog"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
@@ -115,9 +116,10 @@ func TestABodyNotAnnouncedOverItsLimitIsRefused(t *testing.T) {
|
||||
for _, tc := range []struct {
|
||||
name, limit string
|
||||
size int
|
||||
// reachesApp is true when the part of the body the limit lets through
|
||||
// goes on to the app before the limit is met.
|
||||
reachesApp bool
|
||||
// mayReachApp is true when the part of the body the limit lets through
|
||||
// goes on to the app before the limit is met. The app may still not have
|
||||
// started on the request when it is cut off.
|
||||
mayReachApp bool
|
||||
}{
|
||||
{"within the first part", "1K", 2 << 10, false},
|
||||
{"past the first part", "128K", 129 << 10, true},
|
||||
@@ -145,8 +147,8 @@ func TestABodyNotAnnouncedOverItsLimitIsRefused(t *testing.T) {
|
||||
t.Errorf("status %d, want 413", status)
|
||||
}
|
||||
|
||||
if asked.Load() != tc.reachesApp {
|
||||
t.Errorf("the app was asked: %v, want %v", asked.Load(), tc.reachesApp)
|
||||
if asked.Load() && !tc.mayReachApp {
|
||||
t.Error("the app was asked")
|
||||
}
|
||||
})
|
||||
}
|
||||
@@ -267,6 +269,115 @@ func TestAVisitorThatGoesAwayBeforeTheFirstPartIsNotAsked(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name, request string
|
||||
stopsSending bool
|
||||
}{
|
||||
{
|
||||
"a malformed chunk", post + "Transfer-Encoding: chunked\r\n\r\nnot a chunk\r\n",
|
||||
false,
|
||||
},
|
||||
{"a visitor that stops sending", post + announcedLength(100) + "some of it", true},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
servewaf := startServeWAF(t, allow)
|
||||
app := startApp(t, greet)
|
||||
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, tc.request)
|
||||
|
||||
if tc.stopsSending {
|
||||
tcpConn, ok := conn.(*net.TCPConn)
|
||||
if !ok {
|
||||
t.Fatalf("the connection is a %T", conn)
|
||||
}
|
||||
|
||||
err := tcpConn.CloseWrite()
|
||||
if err != nil {
|
||||
t.Fatalf("stop sending: %v", err)
|
||||
}
|
||||
}
|
||||
|
||||
text, err := io.ReadAll(conn)
|
||||
if err != nil || len(text) != 0 {
|
||||
t.Errorf("read %q (%v), want the connection closed without an answer",
|
||||
text, err)
|
||||
}
|
||||
|
||||
line := out.requestLine(t)
|
||||
if line["status"] != 0.0 || line["disposition"] != nil {
|
||||
t.Errorf("line %v, want no answer and no disposition", line)
|
||||
}
|
||||
|
||||
if len(servewaf.questions) != 0 {
|
||||
t.Error("servewaf was asked")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
// deadlineRecorder notes the time limit rproxy sets on sending its answer.
|
||||
type deadlineRecorder struct {
|
||||
*httptest.ResponseRecorder
|
||||
|
||||
writeDeadline time.Time
|
||||
}
|
||||
|
||||
// SetWriteDeadline notes deadline.
|
||||
func (w *deadlineRecorder) SetWriteDeadline(deadline time.Time) error {
|
||||
w.writeDeadline = deadline
|
||||
|
||||
return nil
|
||||
}
|
||||
|
||||
func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const limit = 5 * time.Minute
|
||||
|
||||
for _, tc := range []struct {
|
||||
name, target string
|
||||
status int
|
||||
}{
|
||||
{"servewaf's deny", "/", http.StatusForbidden},
|
||||
{"rproxy's own 400", "x:admin", http.StatusBadRequest},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
servewaf := startServeWAF(t, deny)
|
||||
app := startApp(t, greet)
|
||||
_, _, server := startServer(t, servewaf.URL, app.URL, map[string]string{
|
||||
maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
|
||||
})
|
||||
|
||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, tc.target,
|
||||
http.NoBody)
|
||||
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()}
|
||||
|
||||
before := time.Now()
|
||||
|
||||
server.Handler.ServeHTTP(w, req)
|
||||
|
||||
if w.Code != tc.status {
|
||||
t.Errorf("status %d, want %d", w.Code, tc.status)
|
||||
}
|
||||
|
||||
if w.writeDeadline.Before(before.Add(limit)) ||
|
||||
w.writeDeadline.After(time.Now().Add(limit)) {
|
||||
t.Errorf("the time limit on sending ends at %s, want %s from the answer",
|
||||
w.writeDeadline, limit)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAVisitorTooSlowToSendTheRestIsAnswered408(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
|
||||
@@ -6,6 +6,7 @@ import (
|
||||
"io"
|
||||
"maps"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"slices"
|
||||
"strconv"
|
||||
"strings"
|
||||
@@ -17,6 +18,14 @@ import (
|
||||
// questionBodyMaxBytes is the most of a body a question carries.
|
||||
const questionBodyMaxBytes = proxy.QuestionBodyMaxBytes
|
||||
|
||||
// chunked is the encoding of a body whose length the visitor did not announce.
|
||||
const chunked = "chunked"
|
||||
|
||||
// pathGoReencodes is a path Go's client would decode and escape again, %2F turned
|
||||
// into / and | into %7C, unless told to send it as it is. A test sets it as the
|
||||
// visitor's URL.Opaque, so that the visitor sends it unchanged too.
|
||||
const pathGoReencodes = "/some%2Fpath/with%20space|pipe"
|
||||
|
||||
func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -24,9 +33,10 @@ func TestQuestionIsTheVisitorsRequestWithRProxysHeaders(t *testing.T) {
|
||||
app := startApp(t, greet)
|
||||
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||
|
||||
const target = "/some%2Fpath/with%20space?b=2&a=1&bad=%zz;x"
|
||||
const target = pathGoReencodes + "?b=2&a=1&bad=%zz;x"
|
||||
|
||||
req := newRequest(t, http.MethodDelete, addr, target, http.NoBody)
|
||||
req.URL.Opaque = pathGoReencodes
|
||||
req.Host = site
|
||||
req.Header.Set("X-Test", "from the visitor")
|
||||
req.Header.Set(forwardedForHeader, "198.51.100.1")
|
||||
@@ -199,7 +209,7 @@ func wantQuestionBody(
|
||||
return
|
||||
}
|
||||
|
||||
if len(values) != 0 || !slices.Equal(asked.transferEncoding, []string{"chunked"}) {
|
||||
if len(values) != 0 || !slices.Equal(asked.transferEncoding, []string{chunked}) {
|
||||
t.Errorf("announced %v and encoded %v, want none and chunked", values,
|
||||
asked.transferEncoding)
|
||||
}
|
||||
@@ -226,3 +236,130 @@ func TestDenyNeedsOnlyTheFirstPartOfALargeBody(t *testing.T) {
|
||||
t.Errorf("report %+v, want the first part's bytes", report)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAQuestionCarriesItsBodyWhateverTheMethod(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name, framing, body string
|
||||
encoding []string
|
||||
}{
|
||||
{
|
||||
chunked, "Transfer-Encoding: chunked\r\n\r\n4\r\nbody\r\n0\r\n\r\n", "body",
|
||||
[]string{chunked},
|
||||
},
|
||||
{
|
||||
"chunked and empty", "Transfer-Encoding: chunked\r\n\r\n0\r\n\r\n", "",
|
||||
[]string{chunked},
|
||||
},
|
||||
{"with its length", announcedLength(4) + "body", "body", nil},
|
||||
} {
|
||||
for _, method := range []string{
|
||||
http.MethodConnect, http.MethodGet, http.MethodDelete, http.MethodPost,
|
||||
} {
|
||||
t.Run(method+" "+tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
servewaf := startServeWAF(t, deny)
|
||||
addr, _ := startRProxy(t, servewaf.URL, nowhere, nil)
|
||||
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, method+" /x HTTP/1.1\r\nHost: site.example\r\n"+tc.framing)
|
||||
|
||||
if status := readStatus(t, conn); status != http.StatusForbidden {
|
||||
t.Errorf("status %d, want servewaf's 403", status)
|
||||
}
|
||||
|
||||
asked := servewaf.question(t)
|
||||
if string(asked.body) != tc.body ||
|
||||
!slices.Equal(asked.transferEncoding, tc.encoding) {
|
||||
t.Errorf("the question carries %q, encoded %v, want %q, encoded %v",
|
||||
asked.body, asked.transferEncoding, tc.body, tc.encoding)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestARequestThatCannotGoToServeWAFAsSentIsRefused(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const host = "Host: site.example\r\n\r\n"
|
||||
|
||||
for name, request := range map[string]string{
|
||||
"a target of a scheme": "GET x:admin HTTP/1.1\r\n" + host,
|
||||
"a target of a URL": "GET http://site.example/ HTTP/1.1\r\n" + host,
|
||||
"a host Go will not send": "GET http://a<b/ HTTP/1.1\r\n" + host,
|
||||
"a target of a host": "CONNECT site.example:443 HTTP/1.1\r\n" + host,
|
||||
"a target of *": "GET * HTTP/1.1\r\n" + host,
|
||||
"an empty Host": "GET / HTTP/1.1\r\nHost:\r\n\r\n",
|
||||
"HTTP/1.0 with no Host": "GET / HTTP/1.0\r\n\r\n",
|
||||
} {
|
||||
t.Run(name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
servewaf := startServeWAF(t, allow)
|
||||
app := startApp(t, greet)
|
||||
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, request)
|
||||
|
||||
if status := readStatus(t, conn); status != http.StatusBadRequest {
|
||||
t.Errorf("status %d, want 400", status)
|
||||
}
|
||||
|
||||
line := out.requestLine(t)
|
||||
if line["status"] != float64(http.StatusBadRequest) ||
|
||||
line["disposition"] != nil || line["default_disposition"] != nil {
|
||||
t.Errorf("line %v, want status 400 and no disposition", line)
|
||||
}
|
||||
|
||||
if len(servewaf.questions) != 0 {
|
||||
t.Error("servewaf was asked")
|
||||
}
|
||||
|
||||
servewaf.wantNoReport(t)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestAQuestionNamesNoZone(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, tc := range []struct {
|
||||
name, remoteAddr, forwardedFor, client, peer string
|
||||
}{
|
||||
{"from a link-local peer", "[fe80::1%eth0]:1234", "", "fe80::1", "fe80::1"},
|
||||
{
|
||||
"from X-Forwarded-For through a trusted peer", localhost + ":1234",
|
||||
"2001:db8::1%eth0", "2001:db8::1", localhost,
|
||||
},
|
||||
} {
|
||||
t.Run(tc.name, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
servewaf := startServeWAF(t, deny)
|
||||
app := startApp(t, greet)
|
||||
_, out, server := startServer(t, servewaf.URL, app.URL, map[string]string{
|
||||
trustedProxies: localhost + "/32",
|
||||
})
|
||||
|
||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
|
||||
req.RemoteAddr = tc.remoteAddr
|
||||
req.Header.Set(forwardedForHeader, tc.forwardedFor)
|
||||
|
||||
server.Handler.ServeHTTP(httptest.NewRecorder(), req)
|
||||
|
||||
asked := servewaf.question(t)
|
||||
if asked.header.Get(proxy.ClientHeader) != tc.client ||
|
||||
asked.header.Get(proxy.PeerHeader) != tc.peer {
|
||||
t.Errorf("the question's headers are %v", asked.header)
|
||||
}
|
||||
|
||||
if line := out.requestLine(t); line["client_ip"] != tc.client {
|
||||
t.Errorf("line %v, want client_ip %s", line, tc.client)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
@@ -25,6 +25,10 @@ import (
|
||||
// soon as it arrives.
|
||||
const flushAfterEachWrite time.Duration = -1
|
||||
|
||||
// reportStatusMax is the highest status servewaf takes in a report. An app can
|
||||
// send a higher one, up to 999, and the visitor gets that as it is.
|
||||
const reportStatusMax = 599
|
||||
|
||||
// request is one request on its way through rproxy, from the moment its headers
|
||||
// have been read to its log line and report.
|
||||
type request struct {
|
||||
@@ -111,9 +115,10 @@ func (rq *request) answerHealth() {
|
||||
|
||||
// readFirstPartOfBody reads up to the first 64 KiB of the body into memory,
|
||||
// within SWRP_MAX_REQUEST_TIME_MS, and reports whether the request goes on. A
|
||||
// visitor too slow to send it is answered 408, and one that went away is not
|
||||
// answered. A body over SWRP_MAX_REQUEST_SIZE goes on with the part read, so
|
||||
// that servewaf is asked first; passToApp then refuses it.
|
||||
// visitor too slow to send it is answered 408. For a malformed chunk, or a
|
||||
// visitor that went away or stopped its sending side, the connection is closed
|
||||
// without an answer. A body over SWRP_MAX_REQUEST_SIZE goes on with the part
|
||||
// read, so that servewaf is asked first; passToApp then refuses it.
|
||||
func (rq *request) readFirstPartOfBody() bool {
|
||||
if rq.body == nil {
|
||||
return true
|
||||
@@ -135,19 +140,20 @@ func (rq *request) readFirstPartOfBody() bool {
|
||||
|
||||
return false
|
||||
default:
|
||||
return false
|
||||
// This panic has Go's server close the connection without an answer, and
|
||||
// log nothing; the deferred finish still runs.
|
||||
panic(http.ErrAbortHandler)
|
||||
}
|
||||
}
|
||||
|
||||
// setPathAsSent has out, the URL of a request made from in, send the path as the
|
||||
// visitor sent it. Go sends the path URL.EscapedPath gives, which, for a path
|
||||
// holding a character Go escapes, such as |, is the whole path decoded and
|
||||
// escaped again, %2F turned into /. Opaque is sent as it is. Left out are a
|
||||
// target that is no path, such as *, and a path starting with //, which Go would
|
||||
// send in Opaque as a scheme and host.
|
||||
// escaped again, %2F turned into /. Opaque is sent as it is. Left out is a path
|
||||
// starting with //, which Go would send in Opaque as a scheme and host.
|
||||
func setPathAsSent(out *url.URL, in *http.Request) {
|
||||
path := rules.Path(in)
|
||||
if strings.HasPrefix(path, "/") && !strings.HasPrefix(path, "//") {
|
||||
if !strings.HasPrefix(path, "//") {
|
||||
out.Opaque = path
|
||||
}
|
||||
}
|
||||
@@ -357,7 +363,7 @@ func (rq *request) finish(ctx context.Context) {
|
||||
|
||||
report := &proxy.Report{
|
||||
RequestID: rq.id,
|
||||
Status: rq.out.status,
|
||||
Status: min(rq.out.status, reportStatusMax),
|
||||
RequestBytes: requestBytes,
|
||||
ResponseBytes: responseBytes,
|
||||
DurationTotal: requestlog.Milliseconds(end.Sub(rq.start)),
|
||||
|
||||
+41
-15
@@ -1,7 +1,8 @@
|
||||
// Package rproxy is the reverse proxy in front of one app that asks servewaf
|
||||
// about each request. It passes a request servewaf allows to the app and the
|
||||
// app's answer back, within the size and time limits. When servewaf gives no
|
||||
// answer, SWRP_DEFAULT_DISPOSITION decides. It keeps nothing between requests
|
||||
// answer, SWRP_DEFAULT_DISPOSITION decides. A request it cannot put to servewaf
|
||||
// as the visitor sent it, it refuses itself. It keeps nothing between requests
|
||||
// and never touches disk. "rproxy" in SPEC.md says what it does.
|
||||
package rproxy
|
||||
|
||||
@@ -69,11 +70,12 @@ func New(params Params) *Server {
|
||||
backend := params.Config.WAFBackend
|
||||
errorLog := slog.NewLogLogger(params.ProcessLog.Handler(), slog.LevelWarn)
|
||||
h := &handler{
|
||||
config: params.Config,
|
||||
requestLog: params.RequestLog,
|
||||
processLog: params.ProcessLog,
|
||||
errorLog: errorLog,
|
||||
transport: newTransport(),
|
||||
config: params.Config,
|
||||
requestLog: params.RequestLog,
|
||||
processLog: params.ProcessLog,
|
||||
errorLog: errorLog,
|
||||
transport: newTransport(),
|
||||
serveWAFTransport: newServeWAFTransport(),
|
||||
reportURL: (&url.URL{
|
||||
Scheme: backend.Scheme, Host: backend.Host, Path: proxy.ReportPath,
|
||||
}).String(),
|
||||
@@ -110,7 +112,10 @@ type handler struct {
|
||||
requestLog io.Writer
|
||||
processLog *slog.Logger
|
||||
errorLog *log.Logger
|
||||
transport http.RoundTripper
|
||||
// transport carries requests to the app, and serveWAFTransport questions and
|
||||
// reports to servewaf.
|
||||
transport http.RoundTripper
|
||||
serveWAFTransport http.RoundTripper
|
||||
// reportURL is where the reports go.
|
||||
reportURL string
|
||||
// reports waits for the reports being sent. reportsMu guards starting one
|
||||
@@ -120,9 +125,9 @@ type handler struct {
|
||||
reportsStopped bool
|
||||
}
|
||||
|
||||
// newTransport returns what carries requests to the app and to servewaf. It
|
||||
// never goes through a proxy named in the environment, and leaves answers
|
||||
// compressed or not as they were sent.
|
||||
// newTransport returns what carries requests to the app. It never goes through
|
||||
// a proxy named in the environment, and leaves answers compressed or not as they
|
||||
// were sent.
|
||||
func newTransport() *http.Transport {
|
||||
return &http.Transport{
|
||||
MaxIdleConns: idleConns,
|
||||
@@ -132,10 +137,23 @@ func newTransport() *http.Transport {
|
||||
}
|
||||
}
|
||||
|
||||
// ServeHTTP handles one request: it answers the health check itself, and for
|
||||
// any other request reads the first part of the body, asks servewaf, and does
|
||||
// what the answer says. Once the answer has ended, it writes the request's log
|
||||
// line and sends its report.
|
||||
// newServeWAFTransport returns what carries questions and reports to servewaf.
|
||||
// It speaks HTTP/1.1 only. Go's HTTP/2 client refuses to send a visitor's Upgrade
|
||||
// header, or a Connection header other than close or keep-alive, and that would
|
||||
// read as servewaf giving no answer.
|
||||
func newServeWAFTransport() *http.Transport {
|
||||
transport := newTransport()
|
||||
transport.Protocols = new(http.Protocols)
|
||||
transport.Protocols.SetHTTP1(true)
|
||||
|
||||
return transport
|
||||
}
|
||||
|
||||
// ServeHTTP handles one request: it answers the health check itself, refuses a
|
||||
// request it cannot put to servewaf as the visitor sent it, and for any other
|
||||
// request reads the first part of the body, asks servewaf, and does what the
|
||||
// answer says. Once the answer has ended, it writes the request's log line and
|
||||
// sends its report.
|
||||
func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
rq := h.newRequest(w, r)
|
||||
defer rq.finish(r.Context())
|
||||
@@ -146,6 +164,14 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
|
||||
// Go's client sends only a target that starts with / as it came. For an empty
|
||||
// Host, it sends servewaf's own address.
|
||||
if r.Host == "" || !strings.HasPrefix(r.RequestURI, "/") {
|
||||
rq.answer(http.StatusBadRequest)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
if !rq.readFirstPartOfBody() {
|
||||
return
|
||||
}
|
||||
@@ -163,7 +189,7 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
|
||||
}
|
||||
|
||||
rq.answerWith(answer)
|
||||
default: // proxy.DispositionClose
|
||||
default: // proxy.DispositionClose, or none when the visitor has gone
|
||||
// This panic has Go's server close the connection without an answer,
|
||||
// and log nothing; the deferred finish still runs.
|
||||
panic(http.ErrAbortHandler)
|
||||
|
||||
+215
-19
@@ -117,6 +117,14 @@ func (o *output) requestLine(t *testing.T) map[string]any {
|
||||
return nil
|
||||
}
|
||||
|
||||
// has reports whether what rproxy has written so far holds text.
|
||||
func (o *output) has(text string) bool {
|
||||
o.mu.Lock()
|
||||
defer o.mu.Unlock()
|
||||
|
||||
return strings.Contains(o.buf.String(), text)
|
||||
}
|
||||
|
||||
// question is what the stand-in for servewaf received as a question.
|
||||
type question struct {
|
||||
method, target, host string
|
||||
@@ -438,9 +446,10 @@ func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
|
||||
})
|
||||
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||
|
||||
const target = "/some%2Fpath/with%20space?b=2&a=1&bad=%zz;x"
|
||||
const target = pathGoReencodes + "?b=2&a=1&bad=%zz;x"
|
||||
|
||||
req := newRequest(t, http.MethodGet, addr, target, http.NoBody)
|
||||
req.URL.Opaque = pathGoReencodes
|
||||
req.Host = site
|
||||
req.Header.Set(asnHeader, "AS1")
|
||||
req.Header.Set(countryHeader, "XX")
|
||||
@@ -471,6 +480,82 @@ func TestPassesAnAllowedRequestToTheAppAndItsAnswerBack(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
removed := []string{
|
||||
"X-Forwarded-Port", "X-Forwarded-Server", "X-Forwarded-Uri",
|
||||
"X-Forwarded-Method", "X-Forwarded-Prefix", "X-Forwarded-Tls-Client-Cert",
|
||||
"X-Forwarded-Tls-Client-Cert-Info", "X-Real-Ip",
|
||||
}
|
||||
|
||||
servewaf := startServeWAF(t, allow)
|
||||
seen := make(chan http.Header, 1)
|
||||
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
|
||||
seen <- r.Header.Clone()
|
||||
|
||||
greet(w, r)
|
||||
})
|
||||
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||
trustedProxies: "10.0.0.0/8",
|
||||
})
|
||||
|
||||
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
|
||||
req.Host = site
|
||||
req.Header.Set(forwardedForHeader, "198.51.100.1")
|
||||
req.Header.Set("X-Forwarded-Host", "claimed.example")
|
||||
req.Header.Set(forwardedProtoHeader, "https")
|
||||
|
||||
for _, name := range removed {
|
||||
req.Header.Set(name, "claimed")
|
||||
}
|
||||
|
||||
wantGreeting(t, visit(t, req))
|
||||
|
||||
header := <-seen
|
||||
|
||||
for name, want := range map[string]string{
|
||||
forwardedForHeader: localhost,
|
||||
"X-Forwarded-Host": site,
|
||||
forwardedProtoHeader: "http",
|
||||
} {
|
||||
if got := header.Values(name); !slices.Equal(got, []string{want}) {
|
||||
t.Errorf("the app saw %s %q, want %q", name, got, want)
|
||||
}
|
||||
}
|
||||
|
||||
for _, name := range removed {
|
||||
if got := header.Values(name); len(got) != 0 {
|
||||
t.Errorf("the app saw %s %q, want none", name, got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
servewaf := startServeWAF(t, allow)
|
||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.Header()["Content-Type"] = nil
|
||||
_, _ = io.WriteString(w, "<html>")
|
||||
})
|
||||
_, _, server := startServer(t, servewaf.URL, app.URL, nil)
|
||||
|
||||
req := httptest.NewRequestWithContext(t.Context(), http.MethodGet, "/", http.NoBody)
|
||||
w := httptest.NewRecorder()
|
||||
|
||||
server.Handler.ServeHTTP(w, req)
|
||||
|
||||
// Go's server guesses a type when the first part of the body goes before the
|
||||
// headers are flushed, which is a matter of timing here. So this looks for what
|
||||
// stops it: the key with no value.
|
||||
values, there := w.Header()["Content-Type"]
|
||||
if !there || len(values) != 0 {
|
||||
t.Errorf("the answer's Content-Type is %q, there: %v, want the key with no value",
|
||||
values, there)
|
||||
}
|
||||
}
|
||||
|
||||
func TestRemovesTheVisitorsLookupHeadersWhenServeWAFGivesNone(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -588,6 +673,14 @@ func noAnswers() map[string]func(t *testing.T) string {
|
||||
"too slow": answeringWith(func(_ http.ResponseWriter, r *http.Request) {
|
||||
<-r.Context().Done()
|
||||
}),
|
||||
"too slow after its headers": answeringWith(
|
||||
func(w http.ResponseWriter, r *http.Request) {
|
||||
w.Header().Set(dispositionHeader, dispositionDeny)
|
||||
w.WriteHeader(http.StatusForbidden)
|
||||
_ = http.NewResponseController(w).Flush()
|
||||
|
||||
<-r.Context().Done()
|
||||
}),
|
||||
"no disposition": answeringWith(func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(http.StatusOK)
|
||||
}),
|
||||
@@ -662,6 +755,45 @@ func TestWithoutAnAnswerTheDefaultDispositionDecides(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestAVisitorThatGoesAwayWhileServeWAFIsAskedGetsNoDefault(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
for _, disposition := range []string{dispositionAllow, dispositionDeny} {
|
||||
t.Run("with the default "+disposition, func(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
servewaf := startServeWAF(t, func(_ http.ResponseWriter, r *http.Request) {
|
||||
<-r.Context().Done()
|
||||
})
|
||||
app := startApp(t, func(http.ResponseWriter, *http.Request) {
|
||||
t.Error("the app was asked")
|
||||
})
|
||||
addr, out := startRProxy(t, servewaf.URL, app.URL, map[string]string{
|
||||
timeoutSetting: "60000",
|
||||
defaultDisposition: disposition,
|
||||
})
|
||||
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
||||
servewaf.question(t)
|
||||
|
||||
_ = conn.Close()
|
||||
|
||||
line := out.requestLine(t)
|
||||
if line["status"] != 0.0 || line["disposition"] != nil ||
|
||||
line["default_disposition"] != nil {
|
||||
t.Errorf("line %v, want status 0 and no disposition", line)
|
||||
}
|
||||
|
||||
if out.has("servewaf gave no answer") {
|
||||
t.Error("rproxy warned that servewaf gave no answer")
|
||||
}
|
||||
|
||||
servewaf.wantNoReport(t)
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestSendsNoReportWithoutAnAnswer(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -716,24 +848,18 @@ func TestHealthCheckIsAnsweredWhileServeWAFIsDown(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestReportComesAfterTheAnswerWithoutTheVisitorWaiting(t *testing.T) {
|
||||
func TestTheVisitorDoesNotWaitForTheReport(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
release := make(chan struct{})
|
||||
reports := make(chan proxy.Report, 1)
|
||||
backend := httptest.NewServer(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get(proxy.ClientHeader) != "" {
|
||||
allow(w, r)
|
||||
deny(w, r)
|
||||
|
||||
return
|
||||
}
|
||||
|
||||
var report proxy.Report
|
||||
|
||||
_ = json.NewDecoder(r.Body).Decode(&report)
|
||||
reports <- report
|
||||
|
||||
select {
|
||||
case <-release:
|
||||
case <-r.Context().Done():
|
||||
@@ -743,23 +869,37 @@ func TestReportComesAfterTheAnswerWithoutTheVisitorWaiting(t *testing.T) {
|
||||
}))
|
||||
t.Cleanup(backend.Close)
|
||||
|
||||
app := startApp(t, greet)
|
||||
addr, _ := startRProxy(t, backend.URL, app.URL, map[string]string{
|
||||
timeoutSetting: "60000",
|
||||
})
|
||||
|
||||
t.Cleanup(func() { close(release) })
|
||||
|
||||
// Go's server sends a deny's answer only once the handler has returned, so
|
||||
// this answer comes only if the handler does not wait for the report, which
|
||||
// the stand-in leaves unanswered.
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
|
||||
|
||||
if status := readStatus(t, conn); status != http.StatusForbidden {
|
||||
t.Errorf("status %d, want servewaf's 403", status)
|
||||
}
|
||||
}
|
||||
|
||||
func TestReportGivesWhatHappened(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const body = "the visitor's body"
|
||||
|
||||
servewaf := startServeWAF(t, allow)
|
||||
app := startApp(t, greet)
|
||||
addr, out := startRProxy(t, backend.URL, app.URL, nil)
|
||||
addr, out := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||
|
||||
// The stand-in does not answer the report until the visitor has its answer.
|
||||
wantGreeting(t, visit(t, newRequest(t, http.MethodPost, addr, "/",
|
||||
strings.NewReader(body))))
|
||||
close(release)
|
||||
|
||||
var report proxy.Report
|
||||
|
||||
select {
|
||||
case report = <-reports:
|
||||
case <-time.After(waitLimit):
|
||||
t.Fatal("no report")
|
||||
}
|
||||
report := servewaf.report(t)
|
||||
|
||||
if report.RequestID == "" || report.Status != http.StatusOK ||
|
||||
report.RequestBytes != int64(len(body)) ||
|
||||
@@ -887,6 +1027,62 @@ func TestReportOfAWebSocketCountsWhatItCarried(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
func TestAWebSocketPassesOnThatTheVisitorStoppedSending(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const goodbye = "sent after the visitor stopped sending\n"
|
||||
|
||||
servewaf := startServeWAF(t, allow)
|
||||
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
|
||||
conn, buffered, err := http.NewResponseController(w).Hijack()
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
|
||||
defer func() {
|
||||
_ = conn.Close()
|
||||
}()
|
||||
|
||||
_, _ = buffered.WriteString("HTTP/1.1 101 Switching Protocols\r\n" +
|
||||
"Connection: Upgrade\r\nUpgrade: websocket\r\n\r\n")
|
||||
_ = buffered.Flush()
|
||||
|
||||
_, _ = io.Copy(io.Discard, buffered)
|
||||
|
||||
_, _ = buffered.WriteString(goodbye)
|
||||
_ = buffered.Flush()
|
||||
})
|
||||
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
|
||||
|
||||
conn := dial(t, addr)
|
||||
send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+
|
||||
"Connection: Upgrade\r\nUpgrade: websocket\r\n\r\n")
|
||||
|
||||
reader := bufio.NewReader(conn)
|
||||
|
||||
res, err := http.ReadResponse(reader, nil)
|
||||
if err != nil {
|
||||
t.Fatalf("read the answer to the upgrade: %v", err)
|
||||
}
|
||||
|
||||
_ = res.Body.Close()
|
||||
|
||||
tcpConn, ok := conn.(*net.TCPConn)
|
||||
if !ok {
|
||||
t.Fatalf("the connection is a %T", conn)
|
||||
}
|
||||
|
||||
err = tcpConn.CloseWrite()
|
||||
if err != nil {
|
||||
t.Fatalf("stop sending: %v", err)
|
||||
}
|
||||
|
||||
text, err := io.ReadAll(reader)
|
||||
if err != nil || string(text) != goodbye {
|
||||
t.Errorf("read %q (%v), want %q", text, err, goodbye)
|
||||
}
|
||||
}
|
||||
|
||||
// echoAfterUpgrade is an app that switches protocols on request, and then sends
|
||||
// back each line it receives.
|
||||
func echoAfterUpgrade(w http.ResponseWriter, _ *http.Request) {
|
||||
|
||||
@@ -36,10 +36,16 @@ type servewafAnswer struct {
|
||||
|
||||
// askServeWAF asks servewaf about the request, and notes what is to be done
|
||||
// with it. It returns servewaf's answer, or nil when servewaf gave none, and
|
||||
// SWRP_DEFAULT_DISPOSITION decides. ctx is the request's own context.
|
||||
// SWRP_DEFAULT_DISPOSITION decides. When the visitor went away while servewaf
|
||||
// was being asked, servewaf has not failed, and nothing is noted. ctx is the
|
||||
// request's own context.
|
||||
func (rq *request) askServeWAF(ctx context.Context) *servewafAnswer {
|
||||
answer, err := rq.ask(ctx)
|
||||
if err != nil {
|
||||
if ctx.Err() != nil {
|
||||
return nil
|
||||
}
|
||||
|
||||
rq.h.processLog.Warn("servewaf gave no answer", "error", err.Error())
|
||||
|
||||
rq.disposition = rq.h.config.DefaultDisposition
|
||||
@@ -68,7 +74,7 @@ func (rq *request) ask(ctx context.Context) (*servewafAnswer, error) {
|
||||
return nil, err
|
||||
}
|
||||
|
||||
res, err := rq.h.transport.RoundTrip(question)
|
||||
res, err := rq.h.serveWAFTransport.RoundTrip(question)
|
||||
if err != nil {
|
||||
return nil, fmt.Errorf("send the question: %w", err)
|
||||
}
|
||||
@@ -120,10 +126,10 @@ func (rq *request) newQuestion(ctx context.Context) (*http.Request, error) {
|
||||
question.Host = in.Host
|
||||
|
||||
// The length NewRequestWithContext set is that of what is held. A body whose
|
||||
// length the visitor did not announce goes chunked.
|
||||
if in.ContentLength < 0 {
|
||||
question.ContentLength = -1
|
||||
}
|
||||
// length the visitor did not announce goes chunked. Named here, since Go's
|
||||
// client would otherwise send a CONNECT's body with neither length nor chunks,
|
||||
// which servewaf would read as further requests.
|
||||
question.TransferEncoding = in.TransferEncoding
|
||||
|
||||
question.Header = in.Header.Clone()
|
||||
question.Header.Set(proxy.ClientHeader, rq.client.String())
|
||||
@@ -196,7 +202,7 @@ func (h *handler) postReport(ctx context.Context, report *proxy.Report) error {
|
||||
|
||||
req.Header.Set("Content-Type", "application/json")
|
||||
|
||||
res, err := h.transport.RoundTrip(req)
|
||||
res, err := h.serveWAFTransport.RoundTrip(req)
|
||||
if err != nil {
|
||||
return fmt.Errorf("send the report: %w", err)
|
||||
}
|
||||
|
||||
@@ -1,14 +1,18 @@
|
||||
package smallwebwaf_test
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"encoding/pem"
|
||||
"io"
|
||||
"maps"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"os"
|
||||
"os/exec"
|
||||
"path/filepath"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"syscall"
|
||||
@@ -112,12 +116,12 @@ func startServeWAF(t *testing.T, env map[string]string) (string, *output, func()
|
||||
}
|
||||
|
||||
// serveWAFSettings are servewaf's settings with the default rule file, to listen
|
||||
// at addr.
|
||||
func serveWAFSettings(t *testing.T, addr string) map[string]string {
|
||||
// at any free port.
|
||||
func serveWAFSettings(t *testing.T) map[string]string {
|
||||
t.Helper()
|
||||
|
||||
return map[string]string{
|
||||
listenAddr: addr,
|
||||
listenAddr: localhost + ":0",
|
||||
stateDir: t.TempDir(),
|
||||
rulesDir: filepath.Join("..", "..", "share", "rules.d"),
|
||||
}
|
||||
@@ -126,7 +130,7 @@ func serveWAFSettings(t *testing.T, addr string) map[string]string {
|
||||
func TestABanThroughOneSiteRefusesTheClientOnTheOther(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
addr, serveWAFOut, stop := startServeWAF(t, serveWAFSettings(t, localhost+":0"))
|
||||
addr, serveWAFOut, stop := startServeWAF(t, serveWAFSettings(t))
|
||||
defer stop()
|
||||
|
||||
backend := "http://" + addr
|
||||
@@ -181,7 +185,7 @@ func serveWAFLine(t *testing.T, out *output, id, kind string) map[string]any {
|
||||
func TestTheSiteAnswersWhileServeWAFIsStoppedAndStartedAgain(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
env := serveWAFSettings(t, localhost+":0")
|
||||
env := serveWAFSettings(t)
|
||||
addr, _, stop := startServeWAF(t, env)
|
||||
site, out := startRProxy(t, "http://"+addr, startApp(t))
|
||||
|
||||
@@ -261,6 +265,244 @@ func answersNormally(ctx context.Context, url, from string) bool {
|
||||
return err == nil && res.StatusCode == http.StatusOK && string(body) == greeting
|
||||
}
|
||||
|
||||
// sendRaw sends request, as it is, to rproxy at site, and returns the status of
|
||||
// the answer.
|
||||
func sendRaw(t *testing.T, site, request string) int {
|
||||
t.Helper()
|
||||
|
||||
conn, err := (&net.Dialer{}).DialContext(t.Context(), "tcp",
|
||||
strings.TrimPrefix(site, "http://"))
|
||||
if err != nil {
|
||||
t.Fatalf("dial: %v", err)
|
||||
}
|
||||
|
||||
defer func() {
|
||||
_ = conn.Close()
|
||||
}()
|
||||
|
||||
err = conn.SetDeadline(time.Now().Add(waitLimit))
|
||||
if err != nil {
|
||||
t.Fatalf("set the deadline: %v", err)
|
||||
}
|
||||
|
||||
_, err = io.WriteString(conn, request)
|
||||
if err != nil {
|
||||
t.Fatalf("send: %v", err)
|
||||
}
|
||||
|
||||
res, err := http.ReadResponse(bufio.NewReader(conn), nil)
|
||||
if err != nil {
|
||||
t.Fatalf("read the answer: %v", err)
|
||||
}
|
||||
|
||||
_ = res.Body.Close()
|
||||
|
||||
return res.StatusCode
|
||||
}
|
||||
|
||||
func TestAConnectBodyStaysABodyAtServeWAF(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
addr, serveWAFOut, stop := startServeWAF(t, serveWAFSettings(t))
|
||||
defer stop()
|
||||
|
||||
site, _ := startRProxy(t, "http://"+addr, startApp(t))
|
||||
|
||||
// Read as a further request, this body would ban otherClient on every site.
|
||||
const smuggledID = "smuggled-request"
|
||||
|
||||
smuggled := "GET " + bannedProbe + " HTTP/1.1\r\nHost: site.example\r\n" +
|
||||
proxy.ClientHeader + ": " + otherClient + "\r\n" +
|
||||
"X-Request-ID: " + smuggledID + "\r\n\r\n"
|
||||
|
||||
status := sendRaw(t, site, "CONNECT /tunnel HTTP/1.1\r\nHost: site.example\r\n"+
|
||||
"X-Forwarded-For: "+placed+"\r\nTransfer-Encoding: chunked\r\n\r\n"+
|
||||
strconv.FormatInt(int64(len(smuggled)), 16)+"\r\n"+smuggled+"\r\n0\r\n\r\n")
|
||||
if status != http.StatusForbidden {
|
||||
t.Errorf("status %d, want the 403 of the Core Rule Set, which refuses CONNECT",
|
||||
status)
|
||||
}
|
||||
|
||||
// Read as a further request, it would act straight after the answer.
|
||||
for range 20 {
|
||||
wantStatus(t, site+"/", otherClient, http.StatusOK)
|
||||
time.Sleep(pollInterval)
|
||||
}
|
||||
|
||||
line := serveWAFOut.findLine(t, func(fields map[string]any) bool {
|
||||
return fields["request_id"] == smuggledID
|
||||
})
|
||||
if line != nil {
|
||||
t.Errorf("servewaf read the body as a request: %v", line)
|
||||
}
|
||||
}
|
||||
|
||||
func TestATargetThatCannotGoToServeWAFIsRefusedForABannedClient(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
addr, _, stop := startServeWAF(t, serveWAFSettings(t))
|
||||
defer stop()
|
||||
|
||||
site, out := startRProxy(t, "http://"+addr, startApp(t))
|
||||
|
||||
wantStatus(t, site+bannedProbe, placed, http.StatusForbidden)
|
||||
|
||||
status := sendRaw(t, site, "GET x:admin HTTP/1.1\r\nHost: site.example\r\n"+
|
||||
"X-Forwarded-For: "+placed+"\r\n\r\n")
|
||||
if status != http.StatusBadRequest {
|
||||
t.Errorf("status %d, want 400", status)
|
||||
}
|
||||
|
||||
line := out.line(t, "path", "admin")
|
||||
if line["disposition"] != nil || line["default_disposition"] != nil {
|
||||
t.Errorf("rproxy's line is %v, want no disposition", line)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAClientWithAZoneIsTheClientWithout(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const zoned = "2001:db8::1%eth0"
|
||||
|
||||
addr, _, stop := startServeWAF(t, serveWAFSettings(t))
|
||||
defer stop()
|
||||
|
||||
site, out := startRProxy(t, "http://"+addr, startApp(t))
|
||||
|
||||
wantStatus(t, site+bannedProbe, zoned, http.StatusForbidden)
|
||||
wantStatus(t, site+"/", zoned, http.StatusForbidden)
|
||||
wantStatus(t, site+"/", "2001:db8::1", http.StatusForbidden)
|
||||
|
||||
line := out.line(t, "path", bannedProbe)
|
||||
if line["client_ip"] != "2001:db8::1" || line["default_disposition"] != nil {
|
||||
t.Errorf("rproxy's line is %v", line)
|
||||
}
|
||||
}
|
||||
|
||||
func TestAStatusServeWAFDoesNotTakeIsReportedAs599(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
const status = 799
|
||||
|
||||
app := httptest.NewServer(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, _ *http.Request) {
|
||||
w.WriteHeader(status)
|
||||
}))
|
||||
t.Cleanup(app.Close)
|
||||
|
||||
addr, serveWAFOut, stop := startServeWAF(t, serveWAFSettings(t))
|
||||
defer stop()
|
||||
|
||||
site, out := startRProxy(t, "http://"+addr, app.URL)
|
||||
|
||||
wantStatus(t, site+"/", placed, status)
|
||||
|
||||
line := out.line(t, "path", "/")
|
||||
if line["status"] != float64(status) {
|
||||
t.Errorf("rproxy's line is %v, want the status the visitor got", line)
|
||||
}
|
||||
|
||||
id, _ := line["request_id"].(string)
|
||||
|
||||
report := serveWAFLine(t, serveWAFOut, id, "report")
|
||||
if report["status"] != 599.0 {
|
||||
t.Errorf("servewaf's line is %v, want status 599", report)
|
||||
}
|
||||
}
|
||||
|
||||
// startRProxyProcess runs `smallwebwaf rproxy` in a process of its own, with the
|
||||
// settings in env, each as NAME=value. It returns where rproxy listens, the
|
||||
// process, and what rproxy writes.
|
||||
func startRProxyProcess(t *testing.T, env ...string) (string, *exec.Cmd, *output) {
|
||||
t.Helper()
|
||||
|
||||
binary, err := os.Executable()
|
||||
if err != nil {
|
||||
t.Fatalf("find the test binary: %v", err)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(t.Context(), 3*waitLimit)
|
||||
t.Cleanup(cancel)
|
||||
|
||||
out := &output{}
|
||||
cmd := exec.CommandContext(ctx, binary) //nolint:gosec // this test binary
|
||||
cmd.Stdout = out
|
||||
|
||||
cmd.Env = append(os.Environ(),
|
||||
commandForMain+"=rproxy",
|
||||
"SWRP_LISTEN_ADDR="+localhost+":0")
|
||||
cmd.Env = append(cmd.Env, env...)
|
||||
|
||||
err = cmd.Start()
|
||||
if err != nil {
|
||||
t.Fatalf("start: %v", err)
|
||||
}
|
||||
|
||||
t.Cleanup(func() {
|
||||
_ = cmd.Process.Kill()
|
||||
_ = cmd.Wait()
|
||||
})
|
||||
|
||||
addr, _ := out.line(t, "msg", "starting")["address"].(string)
|
||||
|
||||
return addr, cmd, out
|
||||
}
|
||||
|
||||
func TestRProxyAsksAServeWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
protocols := make(chan string, 10)
|
||||
backend := httptest.NewUnstartedServer(http.HandlerFunc(
|
||||
func(w http.ResponseWriter, r *http.Request) {
|
||||
if r.Header.Get(proxy.ClientHeader) != "" {
|
||||
protocols <- r.Proto
|
||||
|
||||
w.Header().Set(proxy.DispositionHeader, proxy.DispositionDeny)
|
||||
w.WriteHeader(http.StatusForbidden)
|
||||
}
|
||||
}))
|
||||
backend.EnableHTTP2 = true
|
||||
backend.StartTLS()
|
||||
t.Cleanup(backend.Close)
|
||||
|
||||
// rproxy has no setting for the certificates it trusts, so it is told of this
|
||||
// one as Go's client is told of any, in a process of its own.
|
||||
certificates := filepath.Join(t.TempDir(), "servewaf.pem")
|
||||
|
||||
err := os.WriteFile(certificates, pem.EncodeToMemory(&pem.Block{
|
||||
Type: "CERTIFICATE", Bytes: backend.Certificate().Raw,
|
||||
}), 0o600)
|
||||
if err != nil {
|
||||
t.Fatalf("write the certificate: %v", err)
|
||||
}
|
||||
|
||||
addr, _, _ := startRProxyProcess(t,
|
||||
wafBackend+"="+backend.URL,
|
||||
"SWRP_UPSTREAM_URL="+startApp(t),
|
||||
"SSL_CERT_FILE="+certificates)
|
||||
|
||||
for name, headers := range map[string]string{
|
||||
"no such header": "",
|
||||
"Upgrade": "Upgrade: anything\r\n",
|
||||
"Connection": "Connection: anything\r\n",
|
||||
"a WebSocket upgrade": "Connection: Upgrade\r\nUpgrade: websocket\r\n",
|
||||
} {
|
||||
status := sendRaw(t, addr, "GET / HTTP/1.1\r\nHost: site.example\r\n"+headers+"\r\n")
|
||||
if status != http.StatusForbidden {
|
||||
t.Errorf("with %s: status %d, want servewaf's 403", name, status)
|
||||
}
|
||||
|
||||
select {
|
||||
case protocol := <-protocols:
|
||||
if protocol != "HTTP/1.1" {
|
||||
t.Errorf("with %s: asked over %s, want HTTP/1.1", name, protocol)
|
||||
}
|
||||
case <-time.After(waitLimit):
|
||||
t.Errorf("with %s: servewaf was not asked", name)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -321,35 +563,8 @@ func TestRProxyAddressInUseStopsTheStart(t *testing.T) {
|
||||
func TestMainRunsRProxy(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
binary, err := os.Executable()
|
||||
if err != nil {
|
||||
t.Fatalf("find the test binary: %v", err)
|
||||
}
|
||||
|
||||
ctx, cancel := context.WithTimeout(t.Context(), 3*waitLimit)
|
||||
defer cancel()
|
||||
|
||||
out := &output{}
|
||||
cmd := exec.CommandContext(ctx, binary) //nolint:gosec // this test binary
|
||||
cmd.Stdout = out
|
||||
|
||||
cmd.Env = append(os.Environ(),
|
||||
commandForMain+"=rproxy",
|
||||
"SWRP_LISTEN_ADDR="+localhost+":0",
|
||||
// Nothing listens there; the health check does not ask.
|
||||
wafBackend+"=http://127.0.0.1:9")
|
||||
|
||||
err = cmd.Start()
|
||||
if err != nil {
|
||||
t.Fatalf("start: %v", err)
|
||||
}
|
||||
|
||||
t.Cleanup(func() {
|
||||
_ = cmd.Process.Kill()
|
||||
_ = cmd.Wait()
|
||||
})
|
||||
|
||||
addr, _ := out.line(t, "msg", "starting")["address"].(string)
|
||||
// Nothing listens there; the health check does not ask.
|
||||
addr, cmd, out := startRProxyProcess(t, wafBackend+"=http://127.0.0.1:9")
|
||||
|
||||
req, err := http.NewRequestWithContext(t.Context(), http.MethodGet,
|
||||
"http://"+addr+proxy.HealthPath, http.NoBody)
|
||||
|
||||
Reference in New Issue
Block a user