1 Commits
Author SHA1 Message Date
sneak 3490284e16 rproxy, the proxy in front of each app, asks servewaf and fails open (closes #171)
check / check (push) Successful in 5m39s
`smallwebwaf rproxy` reads the first 64 KiB of each body, asks servewaf over
HTTP/1.1, and passes what servewaf allows to the app within its own size and
time limits. It reports each request servewaf answered. SWRP_DEFAULT_DISPOSITION
decides when servewaf gives no answer. A request it cannot put to servewaf as
sent, it refuses itself.

Duplication: internal/rproxy copies passing-on code from internal/proxy until
#172 removes the originals.
Rule suppressed: gosec G704 on servewaf's URL, a setting.
Rule suppressed: gosec G705 on the answer written to the visitor.
Judgement call: the SWRP_ size and time limits take off; SWRP_TIMEOUT_MS does not.
Judgement call: every target not starting with / gets 400, not only x:admin.

Model: opus-5-5
2026-11-05 01:12:42 +00:00
9 changed files with 69 additions and 580 deletions
+10 -33
View File
@@ -1059,12 +1059,8 @@ request. It is the visitor's request as the visitor sent it:
- every header the visitor sent, including its own `X-Forwarded-For`;
- the first 64 KiB of the body, or all of it when shorter.
`rproxy` refuses itself any request it cannot send this way (see "Requests
`rproxy` refuses itself"). So a question always has:
- a path starting with one `/`;
- a `Host` that is not empty and holds no IPv6 zone;
- at most one `User-Agent`.
The path always starts with `/`, and `Host` is never empty. `rproxy` refuses any
other request itself (see "Requests `rproxy` refuses itself").
A body whose length the visitor did not announce comes chunked, whatever the
method. Any other body comes with a `Content-Length` of the part it carries.
@@ -1268,16 +1264,15 @@ and health check.
6. It sends `servewaf` the question (see "The question"). It speaks HTTP/1.1 to
`servewaf`, whatever `servewaf` offers. Go's HTTP/2 client would refuse to
send a visitor's `Upgrade` header, and most `Connection` headers. The
question never carries the visitor's `Trailer`.
question never carries the visitor's `Transfer-Encoding` or `Trailer`, which
Go's HTTP server and client handle themselves.
7. It does what the answer says (see "The answer"):
- `allow`: it passes the request to the app.
- `deny`: the visitor gets the answer's status, headers and body, less
`X-Smallwebwaf-Disposition`.
- `close`: it closes the visitor's connection without an answer.
8. Once the answer has ended, it writes its log line. `docs/request-log.md`
gives the fields. An app's answer can end just after the line: Go's server
sends the closing chunk of a chunked answer, and may send all of an empty
one, only once `rproxy` has finished with the request.
gives the fields.
9. It then sends the report, if `servewaf` answered `allow`, `deny` or `close`
(see "The report"). The visitor does not wait for it. A report that fails is
dropped, with a process warning. A status above `599` from the app goes to
@@ -1286,23 +1281,15 @@ and health check.
### Requests `rproxy` refuses itself
How a visitor sends a request never lets it past `servewaf`.
`SWRP_DEFAULT_DISPOSITION` applies only when `servewaf` itself is down, slow or
answers badly. A visitor can make `servewaf` slow. A request that keeps
`servewaf` busy past `SWRP_TIMEOUT_MS`, like any request while `servewaf` is
overloaded, gets `SWRP_DEFAULT_DISPOSITION`. `deny` closes this, at the cost of
answering `503` whenever `servewaf` is slow.
answers badly.
`rproxy` answers `400`, with no question and no report, to a request with:
- a target that does not start with `/`, such as `x:admin`,
`http://site.example/` or `site.example:443`;
- a target that starts with `//`;
- no `Host`, or an empty one;
- a `Host` holding an IPv6 zone: one that starts with `[` and holds a `%`, such
as `[fe80::1%25eth0]:80`;
- more than one `User-Agent`.
Go's HTTP client could not send any of these as the visitor sent it.
- no `Host`, or an empty one.
Go's HTTP server answers `OPTIONS *` itself, before `rproxy` sees it.
@@ -1351,19 +1338,9 @@ same way. The request reaches the app unchanged but for these headers:
- `X-Client-ASN` and `X-Client-Country`, which carry what `servewaf` answered. A
header it did not give is left out. Any of the two the visitor sent is
removed.
- The headers that concern only the connection to `rproxy` never reach the app:
- `Connection`, and any header it names;
- `Keep-Alive`;
- `Proxy-Authenticate`;
- `Proxy-Authorization`;
- `Proxy-Connection`;
- `Te`, though one holding `trailers` goes on as `Te: trailers`;
- `Trailer`;
- `Upgrade`.
A WebSocket upgrade passes through, with `Connection: Upgrade` and its
`Upgrade`. The report of a WebSocket comes when it closes, with what it carried
each way.
A WebSocket upgrade passes through. The report of a WebSocket comes when it
closes, with what it carried each way.
### `rproxy`'s limits
+1 -1
View File
@@ -1,7 +1,7 @@
// Package config reads smallwebwaf's settings, and is the one place they are
// read. Every setting is an environment variable. The settings of
// `smallwebwaf` and `smallwebwaf servewaf` start with SWWAF_, every one of them
// has a default, and most can be given as a file instead. Those of `smallwebwaf
// has a default, and each can be given as a file instead. Those of `smallwebwaf
// rproxy` start with SWRP_. Each of them but SWRP_WAF_BACKEND has a default, and
// none can be given as a file, since rproxy never touches disk.
package config
+14 -85
View File
@@ -14,7 +14,6 @@ import (
"time"
"sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rproxy"
)
@@ -323,22 +322,13 @@ func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
}
}
// deadlineRecorder notes the time limits rproxy sets on reading the visitor's body
// and on sending its answer.
// deadlineRecorder notes the time limit rproxy sets on sending its answer.
type deadlineRecorder struct {
*httptest.ResponseRecorder
readDeadline time.Time
writeDeadline time.Time
}
// SetReadDeadline notes deadline.
func (w *deadlineRecorder) SetReadDeadline(deadline time.Time) error {
w.readDeadline = deadline
return nil
}
// SetWriteDeadline notes deadline.
func (w *deadlineRecorder) SetWriteDeadline(deadline time.Time) error {
w.writeDeadline = deadline
@@ -346,51 +336,6 @@ func (w *deadlineRecorder) SetWriteDeadline(deadline time.Time) error {
return nil
}
func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
t.Parallel()
const (
limit = 5 * time.Minute
// announced is over what Go's server reads of a body left unread, so that
// the app's own server answers without waiting for the rest.
announced = 100 * questionBodyMaxBytes
)
servewaf := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNoContent)
})
_, _, server := startServer(t, servewaf.URL, app.URL, map[string]string{
maxRequestTime: strconv.FormatInt(limit.Milliseconds(), 10),
})
// The visitor sends the first part and a little more, and then nothing.
stalled, stop := io.Pipe()
t.Cleanup(func() { _ = stop.Close() })
req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/",
io.MultiReader(bytes.NewReader(make([]byte, questionBodyMaxBytes+1)), stalled))
req.ContentLength = announced
req.Header.Set(contentLengthHeader, strconv.Itoa(announced))
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()}
before := time.Now()
server.Handler.ServeHTTP(w, req)
if w.Code != http.StatusNoContent {
t.Errorf("status %d, want the app's 204", w.Code)
}
if w.readDeadline.Before(before.Add(limit)) ||
w.readDeadline.After(time.Now().Add(limit)) {
t.Errorf("the reading of the body ends at %s, want %s from the request",
w.readDeadline, limit)
}
}
func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
t.Parallel()
@@ -688,38 +633,22 @@ func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(t *testing
}
}
func TestAVisitorThatStopsSendingGetsAnAnswerRProxyGivesByTheDeadline(t *testing.T) {
func TestAVisitorThatStopsSendingAfterADenyGetsTheAnswerByTheDeadline(t *testing.T) {
t.Parallel()
for _, tc := range []struct {
name, start string
status int
}{
{"a deny", post, http.StatusForbidden},
{
"the health check",
"GET " + proxy.HealthPath + " HTTP/1.1\r\nHost: site.example\r\n",
http.StatusOK,
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
servewaf := startServeWAF(t, deny)
app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
maxRequestTime: "500",
})
servewaf := startServeWAF(t, deny)
app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
maxRequestTime: "500",
})
// Go's server waits for a little of the body left before it sends an answer,
// and the time for sending the request ends that wait.
conn := dial(t, addr)
send(t, conn, post+announcedLength(questionBodyMaxBytes+100)+
strings.Repeat("x", questionBodyMaxBytes))
// Go's server waits for a little of the body left before it sends an answer,
// and the time for sending the request ends that wait.
conn := dial(t, addr)
send(t, conn, tc.start+announcedLength(questionBodyMaxBytes+100)+
strings.Repeat("x", questionBodyMaxBytes))
if status := readStatus(t, conn); status != tc.status {
t.Errorf("status %d, want %d", status, tc.status)
}
})
if status := readStatus(t, conn); status != http.StatusForbidden {
t.Errorf("status %d, want servewaf's 403", status)
}
}
-5
View File
@@ -292,13 +292,8 @@ func TestARequestThatCannotGoToServeWAFAsSentIsRefused(t *testing.T) {
"a host Go will not send": "GET http://a<b/ HTTP/1.1\r\n" + host,
"a target of a host": "CONNECT site.example:443 HTTP/1.1\r\n" + host,
"a target of *": "GET * HTTP/1.1\r\n" + host,
"a target of //": "GET //x/y HTTP/1.1\r\n" + host,
"a target of // escaped": "GET //x%2Fadmin| HTTP/1.1\r\n" + host,
"an empty Host": "GET / HTTP/1.1\r\nHost:\r\n\r\n",
"HTTP/1.0 with no Host": "GET / HTTP/1.0\r\n\r\n",
"a Host with a zone": "GET / HTTP/1.1\r\nHost: [fe80::1%25eth0]:80\r\n\r\n",
"two User-Agents": "GET / HTTP/1.1\r\nUser-Agent: a\r\nUser-Agent: b\r\n" +
host,
} {
t.Run(name, func(t *testing.T) {
t.Parallel()
+10 -30
View File
@@ -11,7 +11,7 @@ import (
"net/netip"
"net/url"
"os"
"strconv"
"strings"
"sync"
"sync/atomic"
"time"
@@ -29,9 +29,6 @@ const flushAfterEachWrite time.Duration = -1
// send a higher one, up to 999, and the visitor gets that as it is.
const reportStatusMax = 599
// plainText is the Content-Type of the answers rproxy gives itself.
const plainText = "text/plain; charset=utf-8"
// request is one request on its way through rproxy, from the moment its headers
// have been read to its log line and report.
type request struct {
@@ -112,9 +109,8 @@ func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
// answerHealth answers the health check.
func (rq *request) answerHealth() {
rq.prepareOwnAnswer(rq.requestDeadline())
rq.out.Header().Set("Content-Type", plainText)
rq.send(http.StatusOK, []byte("ok\n"))
rq.out.Header().Set("Content-Type", "text/plain; charset=utf-8")
_, _ = io.WriteString(rq.out, "ok\n")
}
// readFirstPartOfBody reads up to the first 64 KiB of the body into memory,
@@ -153,9 +149,13 @@ func (rq *request) readFirstPartOfBody() bool {
// setPathAsSent has out, the URL of a request made from in, send the path as the
// visitor sent it. Go sends the path URL.EscapedPath gives, which, for a path
// holding a character Go escapes, such as |, is the whole path decoded and
// escaped again, %2F turned into /. Opaque is sent as it is.
// escaped again, %2F turned into /. Opaque is sent as it is. Left out is a path
// starting with //, which Go would send in Opaque as a scheme and host.
func setPathAsSent(out *url.URL, in *http.Request) {
out.Opaque = rules.Path(in)
path := rules.Path(in)
if !strings.HasPrefix(path, "//") {
out.Opaque = path
}
}
// forward passes the request to the app, the part of the body held and then the
@@ -297,27 +297,7 @@ func (rq *request) answer(status int) {
}
rq.prepareOwnAnswer(deadline)
header := rq.out.Header()
header.Set("Content-Type", plainText)
header.Set("X-Content-Type-Options", "nosniff")
rq.send(status, []byte(http.StatusText(status)+"\n"))
}
// send gives the visitor an answer rproxy holds whole: status, the headers set so
// far and body. Go's server would otherwise send so short an answer only once the
// handler has returned, after finish has written the log line and sent the report.
// Without Content-Length, the flush would send the body in chunks, the last of
// which would still wait for the handler. A HEAD's answer has no body, so any
// Content-Length it has stays as servewaf gave it.
func (rq *request) send(status int, body []byte) {
if rq.in.Method != http.MethodHead {
rq.out.Header().Set("Content-Length", strconv.Itoa(len(body)))
}
rq.out.WriteHeader(status)
_, _ = rq.out.Write(body)
_ = rq.rc.Flush()
http.Error(rq.out, http.StatusText(status), status)
}
// prepareOwnAnswer readies the connection for an answer rproxy gives itself: the
+3 -1
View File
@@ -164,7 +164,9 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
return
}
if !canBeAsked(r) {
// Go's client sends only a target that starts with / as it came. For an empty
// Host, it sends servewaf's own address.
if r.Host == "" || !strings.HasPrefix(r.RequestURI, "/") {
rq.answer(http.StatusBadRequest)
return
+20 -309
View File
@@ -4,7 +4,6 @@ import (
"bufio"
"bytes"
"encoding/json"
"errors"
"io"
"log/slog"
"maps"
@@ -12,7 +11,6 @@ import (
"net/http"
"net/http/httptest"
"slices"
"strconv"
"strings"
"sync"
"testing"
@@ -127,34 +125,6 @@ func (o *output) has(text string) bool {
return strings.Contains(o.buf.String(), text)
}
// waitFor waits until what rproxy has written holds text.
func (o *output) waitFor(t *testing.T, text string) {
t.Helper()
deadline := time.Now().Add(waitLimit)
for time.Now().Before(deadline) {
if o.has(text) {
return
}
time.Sleep(pollInterval)
}
t.Fatalf("rproxy did not write %q", text)
}
// heldLog is a request log whose Write waits until release is closed.
type heldLog struct {
release chan struct{}
}
// Write waits until release is closed.
func (l heldLog) Write(p []byte) (int, error) {
<-l.release
return len(p), nil
}
// question is what the stand-in for servewaf received as a question.
type question struct {
method, target, host string
@@ -261,28 +231,6 @@ func (s *serveWAF) question(t *testing.T) question {
}
}
// startServeWAFAnsweringReports starts a stand-in for servewaf that answers each
// question as answer does and each report as report does, and returns its URL.
func startServeWAFAnsweringReports(
t *testing.T, answer, report http.HandlerFunc,
) string {
t.Helper()
backend := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get(proxy.ClientHeader) != "" {
answer(w, r)
return
}
report(w, r)
}))
t.Cleanup(backend.Close)
return backend.URL
}
// allow answers a question with allow.
func allow(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(dispositionHeader, dispositionAllow)
@@ -332,20 +280,6 @@ func startServer(
) (string, *output, *rproxy.Server) {
t.Helper()
out := &output{}
addr, server := startServerLogging(t, backend, appURL, env, out, out)
return addr, out, server
}
// startServerLogging is startServer with rproxy's request lines going to
// requestLog and its process lines to processLog.
func startServerLogging(
t *testing.T, backend, appURL string, env map[string]string,
requestLog, processLog io.Writer,
) (string, *rproxy.Server) {
t.Helper()
settings := map[string]string{
"SWRP_WAF_BACKEND": backend, "SWRP_UPSTREAM_URL": appURL,
}
@@ -360,10 +294,11 @@ func startServerLogging(
t.Fatalf("settings %v: %v", settings, err)
}
out := &output{}
server := rproxy.New(rproxy.Params{
Config: cfg,
RequestLog: requestLog,
ProcessLog: requestlog.NewProcessLogger(processLog, "test", slog.LevelInfo),
RequestLog: out,
ProcessLog: requestlog.NewProcessLogger(out, "test", slog.LevelInfo),
})
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
@@ -381,7 +316,7 @@ func startServerLogging(
server.WaitForReports()
})
return listener.Addr().String(), server
return listener.Addr().String(), out, server
}
// answered is the visitor's answer.
@@ -596,45 +531,6 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
}
}
func TestATrustedProxysForwardedHeadersPassOn(t *testing.T) {
t.Parallel()
servewaf := startServeWAF(t, allow)
seen := make(chan http.Header, 1)
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
seen <- r.Header.Clone()
greet(w, r)
})
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
trustedProxies: localhost + "/32",
})
sent := map[string]string{
"Forwarded": "for=198.51.100.1",
"X-Forwarded-Host": "public.example",
forwardedProtoHeader: "https",
"X-Real-Ip": "198.51.100.1",
}
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Host = site
for name, value := range sent {
req.Header.Set(name, value)
}
wantGreeting(t, visit(t, req))
header := <-seen
for name, want := range sent {
if got := header.Values(name); !slices.Equal(got, []string{want}) {
t.Errorf("the app saw %s %q, want %q", name, got, want)
}
}
}
func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
t.Parallel()
@@ -730,87 +626,6 @@ func TestDenyGivesTheVisitorServeWAFsAnswer(t *testing.T) {
}
}
func TestADeniedHeadKeepsServeWAFsContentLength(t *testing.T) {
t.Parallel()
servewaf := startServeWAF(t, deny)
app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
if got.status != http.StatusForbidden ||
got.header.Get(contentLengthHeader) != strconv.Itoa(len(deniedBody)) {
t.Errorf("answered %+v, want servewaf's 403 and its length", got)
}
}
func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
t.Parallel()
const host = " HTTP/1.1\r\nHost: site.example\r\n\r\n"
for _, tc := range []struct {
name, request, body string
status int
}{
{"a deny", "GET /" + host, deniedBody, http.StatusForbidden},
{"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest},
{"the health check", "GET " + proxy.HealthPath + host, "ok\n", http.StatusOK},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
log := heldLog{release: make(chan struct{})}
servewaf := startServeWAF(t, deny)
app := startApp(t, greet)
addr, _ := startServerLogging(t, servewaf.URL, app.URL, nil, log, io.Discard)
t.Cleanup(func() { close(log.release) })
conn := dial(t, addr)
send(t, conn, tc.request)
res, err := http.ReadResponse(bufio.NewReader(conn), nil)
if err != nil {
t.Fatalf("read the answer: %v", err)
}
defer func() {
_ = res.Body.Close()
}()
body, err := io.ReadAll(res.Body)
if err != nil || res.StatusCode != tc.status || string(body) != tc.body {
t.Errorf("answered %d %q (%v), want %d %q", res.StatusCode, body, err,
tc.status, tc.body)
}
})
}
}
func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
t.Parallel()
servewaf := startServeWAF(t, allow)
app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
conn := dial(t, addr)
send(t, conn, "GET x:admin HTTP/1.1\r\nHost: site.example\r\n\r\n")
res, err := http.ReadResponse(bufio.NewReader(conn), nil)
if err != nil {
t.Fatalf("read the answer: %v", err)
}
_ = res.Body.Close()
if res.Header.Get("Content-Type") != "text/plain; charset=utf-8" ||
res.Header.Get("X-Content-Type-Options") != "nosniff" {
t.Errorf("the answer's headers are %v", res.Header)
}
}
func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
t.Parallel()
@@ -883,8 +698,6 @@ func noAnswers() map[string]func(t *testing.T) string {
w.Header().Set(contentLengthHeader, "100")
w.WriteHeader(http.StatusForbidden)
_, _ = io.WriteString(w, deniedBody)
// Go's server drops on this panic what it has not flushed.
_ = http.NewResponseController(w).Flush()
panic(http.ErrAbortHandler)
}),
@@ -1039,97 +852,41 @@ func TestTheVisitorDoesNotWaitForTheReport(t *testing.T) {
t.Parallel()
release := make(chan struct{})
backend := startServeWAFAnsweringReports(t, deny,
backend := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get(proxy.ClientHeader) != "" {
deny(w, r)
return
}
select {
case <-release:
case <-r.Context().Done():
}
w.WriteHeader(http.StatusNoContent)
})
}))
t.Cleanup(backend.Close)
app := startApp(t, greet)
addr, _ := startRProxy(t, backend, app.URL, map[string]string{
addr, _ := startRProxy(t, backend.URL, app.URL, map[string]string{
timeoutSetting: "60000",
})
t.Cleanup(func() { close(release) })
// Go's server reads the next request on a connection only once the handler has
// returned, so the second answer comes only if the handler does not wait for
// the first report, which the stand-in leaves unanswered.
// Go's server sends a deny's answer only once the handler has returned, so
// this answer comes only if the handler does not wait for the report, which
// the stand-in leaves unanswered.
conn := dial(t, addr)
reader := bufio.NewReader(conn)
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
for range 2 {
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
res, err := http.ReadResponse(reader, nil)
if err != nil {
t.Fatalf("read the answer: %v", err)
}
_, _ = io.Copy(io.Discard, res.Body)
_ = res.Body.Close()
if res.StatusCode != http.StatusForbidden {
t.Errorf("status %d, want servewaf's 403", res.StatusCode)
}
if status := readStatus(t, conn); status != http.StatusForbidden {
t.Errorf("status %d, want servewaf's 403", status)
}
}
func TestAReportServeWAFLeavesUnansweredIsDroppedAtTheTimeLimit(t *testing.T) {
t.Parallel()
release := make(chan struct{})
backend := startServeWAFAnsweringReports(t, allow,
func(w http.ResponseWriter, _ *http.Request) {
<-release
w.WriteHeader(http.StatusNoContent)
})
app := startApp(t, greet)
addr, out := startRProxy(t, backend, app.URL, map[string]string{
timeoutSetting: "1000",
})
t.Cleanup(func() { close(release) })
wantGreeting(t, get(t, addr, "/"))
out.waitFor(t, "a report is dropped")
}
func TestAReportServeWAFRefusesIsDroppedWithAWarning(t *testing.T) {
t.Parallel()
backend := startServeWAFAnsweringReports(t, allow,
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusBadRequest)
})
app := startApp(t, greet)
addr, out := startRProxy(t, backend, app.URL, nil)
wantGreeting(t, get(t, addr, "/"))
out.waitFor(t, "a report is dropped")
}
func TestAReportDueAfterWaitForReportsIsDropped(t *testing.T) {
t.Parallel()
servewaf := startServeWAF(t, allow)
app := startApp(t, greet)
addr, out, server := startServer(t, servewaf.URL, app.URL, nil)
server.WaitForReports()
wantGreeting(t, get(t, addr, "/"))
out.requestLine(t)
servewaf.wantNoReport(t)
}
func TestReportGivesWhatHappened(t *testing.T) {
t.Parallel()
@@ -1326,52 +1083,6 @@ func TestAWebSocketPassesOnThatTheVisitorStoppedSending(t *testing.T) {
}
}
// This test does not run in parallel. Go closes a connection nothing refers to at
// its next garbage collection, which other tests' allocations would soon start, and
// that would hide a connection left open.
//
//nolint:paralleltest // see above
func TestASwitchNotPassedOnClosesTheAppsConnection(t *testing.T) {
closed := make(chan error, 1)
servewaf := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
conn, buffered, err := http.NewResponseController(w).Hijack()
if err != nil {
closed <- err
return
}
defer func() {
_ = conn.Close()
}()
// Not the protocol the visitor asked for, so ReverseProxy refuses the switch.
_, _ = buffered.WriteString("HTTP/1.1 101 Switching Protocols\r\n" +
"Connection: Upgrade\r\nUpgrade: another\r\n\r\n")
_ = buffered.Flush()
_ = conn.SetReadDeadline(time.Now().Add(waitLimit))
_, err = buffered.ReadByte()
closed <- err
})
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
conn := dial(t, addr)
send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+
"Connection: Upgrade\r\nUpgrade: websocket\r\n\r\n")
if status := readStatus(t, conn); status != http.StatusBadGateway {
t.Errorf("status %d, want 502", status)
}
err := <-closed
if !errors.Is(err, io.EOF) {
t.Errorf("the app's read gave %v, want its connection closed", err)
}
}
// echoAfterUpgrade is an app that switches protocols on request, and then sends
// back each line it receives.
func echoAfterUpgrade(w http.ResponseWriter, _ *http.Request) {
+6 -22
View File
@@ -9,7 +9,6 @@ import (
"io"
"maps"
"net/http"
"strings"
"sneak.berlin/go/smallwebwaf/internal/proxy"
)
@@ -47,9 +46,6 @@ func (rq *request) askServeWAF(ctx context.Context) *servewafAnswer {
return nil
}
// A request that keeps servewaf busy past SWRP_TIMEOUT_MS, like any request
// while servewaf is overloaded, gets SWRP_DEFAULT_DISPOSITION here. deny closes
// this, at the cost of answering 503 whenever servewaf is slow.
rq.h.processLog.Warn("servewaf gave no answer", "error", err.Error())
rq.disposition = rq.h.config.DefaultDisposition
@@ -104,20 +100,6 @@ func (rq *request) ask(ctx context.Context) (*servewafAnswer, error) {
}
}
// canBeAsked reports whether Go's client can send servewaf r as the visitor sent
// it. It cannot send these as they came:
// - a target that does not start with /, or that starts with //;
// - an empty Host, for which it sends servewaf's own address;
// - a Host holding an IPv6 zone, which it removes;
// - more than one User-Agent, of which it sends the first.
func canBeAsked(r *http.Request) bool {
target, host := r.RequestURI, r.Host
hostHasZone := strings.HasPrefix(host, "[") && strings.Contains(host, "%")
return strings.HasPrefix(target, "/") && !strings.HasPrefix(target, "//") &&
host != "" && !hostHasZone && len(r.Header.Values("User-Agent")) <= 1
}
// newQuestion returns the question about the request, as "The question" in
// SPEC.md gives it: the visitor's request as the visitor sent it, with the part
// of the body it holds and rproxy's headers.
@@ -143,9 +125,10 @@ func (rq *request) newQuestion(ctx context.Context) (*http.Request, error) {
question.URL = &target
question.Host = in.Host
// The length NewRequestWithContext set is that of what is held. This line makes
// a body whose length the visitor did not announce go chunked, as "The question"
// in SPEC.md requires.
// The length NewRequestWithContext set is that of what is held. A body whose
// length the visitor did not announce goes chunked. Named here, since Go's
// client would otherwise send a CONNECT's body with neither length nor chunks,
// which servewaf would read as further requests.
question.TransferEncoding = in.TransferEncoding
question.Header = in.Header.Clone()
@@ -177,7 +160,8 @@ func (rq *request) answerWith(answer *servewafAnswer) {
header := rq.out.Header()
maps.Copy(header, answer.header)
header.Del(proxy.DispositionHeader)
rq.send(answer.status, answer.body)
rq.out.WriteHeader(answer.status)
_, _ = rq.out.Write(answer.body)
}
// sendReport sends servewaf report on a goroutine of its own, so that the
+5 -94
View File
@@ -26,10 +26,6 @@ import (
const (
// wafBackend is the setting that names servewaf to rproxy.
wafBackend = "SWRP_WAF_BACKEND"
// rproxyListenAddr is the setting that says where rproxy listens.
rproxyListenAddr = "SWRP_LISTEN_ADDR"
// rproxyTimeout is the setting that says how long rproxy waits for servewaf.
rproxyTimeout = "SWRP_TIMEOUT_MS"
// bannedProbe is a path the default rule file bans a client for.
bannedProbe = "/.env"
// otherClient is a client the tests do not ban.
@@ -62,7 +58,7 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
go func() {
exited <- runRProxy(ctx, map[string]string{
rproxyListenAddr: localhost + ":0",
"SWRP_LISTEN_ADDR": localhost + ":0",
"SWRP_UPSTREAM_URL": appURL,
"SWRP_TRUSTED_PROXIES": localhost + "/32",
wafBackend: backend,
@@ -507,91 +503,6 @@ func TestRProxyAsksAServeWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
}
}
func TestRProxyProcessLinesNameTheHostAndComeAtInfo(t *testing.T) {
t.Parallel()
hostname, err := os.Hostname()
if err != nil {
t.Fatalf("the host's name: %v", err)
}
// Nothing listens at the backend's address; no request asks it.
_, out := startRProxy(t, "http://127.0.0.1:9", startApp(t))
line := out.line(t, "msg", "starting")
if line["instance"] != hostname || line["level"] != "INFO" {
t.Errorf("rproxy's line is %v, want instance %s at INFO", line, hostname)
}
}
func TestRProxyStoppingWaitsForAReportBeingSent(t *testing.T) {
t.Parallel()
reportGotThere := make(chan struct{})
release := make(chan struct{})
backend := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get(proxy.ClientHeader) != "" {
w.Header().Set(proxy.DispositionHeader, proxy.DispositionAllow)
return
}
close(reportGotThere)
<-release
w.WriteHeader(http.StatusNoContent)
}))
t.Cleanup(backend.Close)
ctx, stop := context.WithCancel(t.Context())
defer stop()
env := map[string]string{
rproxyListenAddr: localhost + ":0",
"SWRP_UPSTREAM_URL": startApp(t),
rproxyTimeout: "60000",
wafBackend: backend.URL,
}
out := &output{}
exited := make(chan int, 1)
go func() {
exited <- runRProxy(ctx, env, out)
}()
addr, _ := out.line(t, "msg", "starting")["address"].(string)
wantStatus(t, "http://"+addr+"/", placed, http.StatusOK)
select {
case <-reportGotThere:
case <-time.After(waitLimit):
t.Fatal("no report")
}
stop()
select {
case <-exited:
t.Error("rproxy stopped with a report on its way")
close(release)
return
case <-time.After(10 * pollInterval):
}
close(release)
select {
case status := <-exited:
if status != 0 {
t.Errorf("rproxy's exit status %d, want 0; output:\n%s", status, out.text())
}
case <-time.After(waitLimit):
t.Error("rproxy still running, the report having been answered")
}
}
func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
t.Parallel()
@@ -601,8 +512,8 @@ func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
}{
{"no backend", wafBackend, nil},
{
"another invalid", rproxyTimeout,
map[string]string{wafBackend: "http://servewaf:8080", rproxyTimeout: "soon"},
"another invalid", "SWRP_TIMEOUT_MS",
map[string]string{wafBackend: "http://servewaf:8080", "SWRP_TIMEOUT_MS": "soon"},
},
} {
t.Run(tc.name, func(t *testing.T) {
@@ -639,8 +550,8 @@ func TestRProxyAddressInUseStopsTheStart(t *testing.T) {
out := &output{}
status := runRProxy(t.Context(), map[string]string{
wafBackend: "http://servewaf:8080",
rproxyListenAddr: taken.Addr().String(),
wafBackend: "http://servewaf:8080",
"SWRP_LISTEN_ADDR": taken.Addr().String(),
}, out)
if status != 1 {
t.Errorf("exit status %d, want 1", status)