1 Commits
Author SHA1 Message Date
sneak 3490284e16 rproxy, the proxy in front of each app, asks servewaf and fails open (closes #171)
check / check (push) Successful in 5m39s
`smallwebwaf rproxy` reads the first 64 KiB of each body, asks servewaf over
HTTP/1.1, and passes what servewaf allows to the app within its own size and
time limits. It reports each request servewaf answered. SWRP_DEFAULT_DISPOSITION
decides when servewaf gives no answer. A request it cannot put to servewaf as
sent, it refuses itself.

Duplication: internal/rproxy copies passing-on code from internal/proxy until
#172 removes the originals.
Rule suppressed: gosec G704 on servewaf's URL, a setting.
Rule suppressed: gosec G705 on the answer written to the visitor.
Judgement call: the SWRP_ size and time limits take off; SWRP_TIMEOUT_MS does not.
Judgement call: every target not starting with / gets 400, not only x:admin.

Model: opus-5-5
2026-11-05 01:12:42 +00:00
9 changed files with 69 additions and 580 deletions

No files matched your search

+10 -33
View File
@@ -1059,12 +1059,8 @@ request. It is the visitor's request as the visitor sent it:
- every header the visitor sent, including its own `X-Forwarded-For`; - every header the visitor sent, including its own `X-Forwarded-For`;
- the first 64 KiB of the body, or all of it when shorter. - the first 64 KiB of the body, or all of it when shorter.
`rproxy` refuses itself any request it cannot send this way (see "Requests The path always starts with `/`, and `Host` is never empty. `rproxy` refuses any
`rproxy` refuses itself"). So a question always has: other request itself (see "Requests `rproxy` refuses itself").
- a path starting with one `/`;
- a `Host` that is not empty and holds no IPv6 zone;
- at most one `User-Agent`.
A body whose length the visitor did not announce comes chunked, whatever the A body whose length the visitor did not announce comes chunked, whatever the
method. Any other body comes with a `Content-Length` of the part it carries. method. Any other body comes with a `Content-Length` of the part it carries.
@@ -1268,16 +1264,15 @@ and health check.
6. It sends `servewaf` the question (see "The question"). It speaks HTTP/1.1 to 6. It sends `servewaf` the question (see "The question"). It speaks HTTP/1.1 to
`servewaf`, whatever `servewaf` offers. Go's HTTP/2 client would refuse to `servewaf`, whatever `servewaf` offers. Go's HTTP/2 client would refuse to
send a visitor's `Upgrade` header, and most `Connection` headers. The send a visitor's `Upgrade` header, and most `Connection` headers. The
question never carries the visitor's `Trailer`. question never carries the visitor's `Transfer-Encoding` or `Trailer`, which
Go's HTTP server and client handle themselves.
7. It does what the answer says (see "The answer"): 7. It does what the answer says (see "The answer"):
- `allow`: it passes the request to the app. - `allow`: it passes the request to the app.
- `deny`: the visitor gets the answer's status, headers and body, less - `deny`: the visitor gets the answer's status, headers and body, less
`X-Smallwebwaf-Disposition`. `X-Smallwebwaf-Disposition`.
- `close`: it closes the visitor's connection without an answer. - `close`: it closes the visitor's connection without an answer.
8. Once the answer has ended, it writes its log line. `docs/request-log.md` 8. Once the answer has ended, it writes its log line. `docs/request-log.md`
gives the fields. An app's answer can end just after the line: Go's server gives the fields.
sends the closing chunk of a chunked answer, and may send all of an empty
one, only once `rproxy` has finished with the request.
9. It then sends the report, if `servewaf` answered `allow`, `deny` or `close` 9. It then sends the report, if `servewaf` answered `allow`, `deny` or `close`
(see "The report"). The visitor does not wait for it. A report that fails is (see "The report"). The visitor does not wait for it. A report that fails is
dropped, with a process warning. A status above `599` from the app goes to dropped, with a process warning. A status above `599` from the app goes to
@@ -1286,23 +1281,15 @@ and health check.
### Requests `rproxy` refuses itself ### Requests `rproxy` refuses itself
How a visitor sends a request never lets it past `servewaf`.
`SWRP_DEFAULT_DISPOSITION` applies only when `servewaf` itself is down, slow or `SWRP_DEFAULT_DISPOSITION` applies only when `servewaf` itself is down, slow or
answers badly. A visitor can make `servewaf` slow. A request that keeps answers badly.
`servewaf` busy past `SWRP_TIMEOUT_MS`, like any request while `servewaf` is
overloaded, gets `SWRP_DEFAULT_DISPOSITION`. `deny` closes this, at the cost of
answering `503` whenever `servewaf` is slow.
`rproxy` answers `400`, with no question and no report, to a request with: `rproxy` answers `400`, with no question and no report, to a request with:
- a target that does not start with `/`, such as `x:admin`, - a target that does not start with `/`, such as `x:admin`,
`http://site.example/` or `site.example:443`; `http://site.example/` or `site.example:443`;
- a target that starts with `//`; - no `Host`, or an empty one.
- no `Host`, or an empty one;
- a `Host` holding an IPv6 zone: one that starts with `[` and holds a `%`, such
as `[fe80::1%25eth0]:80`;
- more than one `User-Agent`.
Go's HTTP client could not send any of these as the visitor sent it.
Go's HTTP server answers `OPTIONS *` itself, before `rproxy` sees it. Go's HTTP server answers `OPTIONS *` itself, before `rproxy` sees it.
@@ -1351,19 +1338,9 @@ same way. The request reaches the app unchanged but for these headers:
- `X-Client-ASN` and `X-Client-Country`, which carry what `servewaf` answered. A - `X-Client-ASN` and `X-Client-Country`, which carry what `servewaf` answered. A
header it did not give is left out. Any of the two the visitor sent is header it did not give is left out. Any of the two the visitor sent is
removed. removed.
- The headers that concern only the connection to `rproxy` never reach the app:
- `Connection`, and any header it names;
- `Keep-Alive`;
- `Proxy-Authenticate`;
- `Proxy-Authorization`;
- `Proxy-Connection`;
- `Te`, though one holding `trailers` goes on as `Te: trailers`;
- `Trailer`;
- `Upgrade`.
A WebSocket upgrade passes through, with `Connection: Upgrade` and its A WebSocket upgrade passes through. The report of a WebSocket comes when it
`Upgrade`. The report of a WebSocket comes when it closes, with what it carried closes, with what it carried each way.
each way.
### `rproxy`'s limits ### `rproxy`'s limits
+1 -1
View File
@@ -1,7 +1,7 @@
// Package config reads smallwebwaf's settings, and is the one place they are // Package config reads smallwebwaf's settings, and is the one place they are
// read. Every setting is an environment variable. The settings of // read. Every setting is an environment variable. The settings of
// `smallwebwaf` and `smallwebwaf servewaf` start with SWWAF_, every one of them // `smallwebwaf` and `smallwebwaf servewaf` start with SWWAF_, every one of them
// has a default, and most can be given as a file instead. Those of `smallwebwaf // has a default, and each can be given as a file instead. Those of `smallwebwaf
// rproxy` start with SWRP_. Each of them but SWRP_WAF_BACKEND has a default, and // rproxy` start with SWRP_. Each of them but SWRP_WAF_BACKEND has a default, and
// none can be given as a file, since rproxy never touches disk. // none can be given as a file, since rproxy never touches disk.
package config package config
+14 -85
View File
@@ -14,7 +14,6 @@ import (
"time" "time"
"sneak.berlin/go/smallwebwaf/internal/config" "sneak.berlin/go/smallwebwaf/internal/config"
"sneak.berlin/go/smallwebwaf/internal/proxy"
"sneak.berlin/go/smallwebwaf/internal/requestlog" "sneak.berlin/go/smallwebwaf/internal/requestlog"
"sneak.berlin/go/smallwebwaf/internal/rproxy" "sneak.berlin/go/smallwebwaf/internal/rproxy"
) )
@@ -323,22 +322,13 @@ func TestABodyThatCannotBeReadClosesTheConnectionWithoutAnAnswer(t *testing.T) {
} }
} }
// deadlineRecorder notes the time limits rproxy sets on reading the visitor's body // deadlineRecorder notes the time limit rproxy sets on sending its answer.
// and on sending its answer.
type deadlineRecorder struct { type deadlineRecorder struct {
*httptest.ResponseRecorder *httptest.ResponseRecorder
readDeadline time.Time
writeDeadline time.Time writeDeadline time.Time
} }
// SetReadDeadline notes deadline.
func (w *deadlineRecorder) SetReadDeadline(deadline time.Time) error {
w.readDeadline = deadline
return nil
}
// SetWriteDeadline notes deadline. // SetWriteDeadline notes deadline.
func (w *deadlineRecorder) SetWriteDeadline(deadline time.Time) error { func (w *deadlineRecorder) SetWriteDeadline(deadline time.Time) error {
w.writeDeadline = deadline w.writeDeadline = deadline
@@ -346,51 +336,6 @@ func (w *deadlineRecorder) SetWriteDeadline(deadline time.Time) error {
return nil return nil
} }
func TestABodyLeftUnreadIsReadOnlyUntilTheRequestTimeLimit(t *testing.T) {
t.Parallel()
const (
limit = 5 * time.Minute
// announced is over what Go's server reads of a body left unread, so that
// the app's own server answers without waiting for the rest.
announced = 100 * questionBodyMaxBytes
)
servewaf := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusNoContent)
})
_, _, server := startServer(t, servewaf.URL, app.URL, map[string]string{
maxRequestTime: strconv.FormatInt(limit.Milliseconds(), 10),
})
// The visitor sends the first part and a little more, and then nothing.
stalled, stop := io.Pipe()
t.Cleanup(func() { _ = stop.Close() })
req := httptest.NewRequestWithContext(t.Context(), http.MethodPost, "/",
io.MultiReader(bytes.NewReader(make([]byte, questionBodyMaxBytes+1)), stalled))
req.ContentLength = announced
req.Header.Set(contentLengthHeader, strconv.Itoa(announced))
w := &deadlineRecorder{ResponseRecorder: httptest.NewRecorder()}
before := time.Now()
server.Handler.ServeHTTP(w, req)
if w.Code != http.StatusNoContent {
t.Errorf("status %d, want the app's 204", w.Code)
}
if w.readDeadline.Before(before.Add(limit)) ||
w.readDeadline.After(time.Now().Add(limit)) {
t.Errorf("the reading of the body ends at %s, want %s from the request",
w.readDeadline, limit)
}
}
func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) { func TestAnAnswerRProxyGivesItselfHasTheResponseTimeToBeSent(t *testing.T) {
t.Parallel() t.Parallel()
@@ -688,38 +633,22 @@ func TestTheRequestTimeLimitDoesNotCutAnAnswerOnceTheAppHasTheRequest(t *testing
} }
} }
func TestAVisitorThatStopsSendingGetsAnAnswerRProxyGivesByTheDeadline(t *testing.T) { func TestAVisitorThatStopsSendingAfterADenyGetsTheAnswerByTheDeadline(t *testing.T) {
t.Parallel() t.Parallel()
for _, tc := range []struct { servewaf := startServeWAF(t, deny)
name, start string app := startApp(t, greet)
status int addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
}{ maxRequestTime: "500",
{"a deny", post, http.StatusForbidden}, })
{
"the health check",
"GET " + proxy.HealthPath + " HTTP/1.1\r\nHost: site.example\r\n",
http.StatusOK,
},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
servewaf := startServeWAF(t, deny) // Go's server waits for a little of the body left before it sends an answer,
app := startApp(t, greet) // and the time for sending the request ends that wait.
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{ conn := dial(t, addr)
maxRequestTime: "500", send(t, conn, post+announcedLength(questionBodyMaxBytes+100)+
}) strings.Repeat("x", questionBodyMaxBytes))
// Go's server waits for a little of the body left before it sends an answer, if status := readStatus(t, conn); status != http.StatusForbidden {
// and the time for sending the request ends that wait. t.Errorf("status %d, want servewaf's 403", status)
conn := dial(t, addr)
send(t, conn, tc.start+announcedLength(questionBodyMaxBytes+100)+
strings.Repeat("x", questionBodyMaxBytes))
if status := readStatus(t, conn); status != tc.status {
t.Errorf("status %d, want %d", status, tc.status)
}
})
} }
} }
-5
View File
@@ -292,13 +292,8 @@ func TestARequestThatCannotGoToServeWAFAsSentIsRefused(t *testing.T) {
"a host Go will not send": "GET http://a<b/ HTTP/1.1\r\n" + host, "a host Go will not send": "GET http://a<b/ HTTP/1.1\r\n" + host,
"a target of a host": "CONNECT site.example:443 HTTP/1.1\r\n" + host, "a target of a host": "CONNECT site.example:443 HTTP/1.1\r\n" + host,
"a target of *": "GET * HTTP/1.1\r\n" + host, "a target of *": "GET * HTTP/1.1\r\n" + host,
"a target of //": "GET //x/y HTTP/1.1\r\n" + host,
"a target of // escaped": "GET //x%2Fadmin| HTTP/1.1\r\n" + host,
"an empty Host": "GET / HTTP/1.1\r\nHost:\r\n\r\n", "an empty Host": "GET / HTTP/1.1\r\nHost:\r\n\r\n",
"HTTP/1.0 with no Host": "GET / HTTP/1.0\r\n\r\n", "HTTP/1.0 with no Host": "GET / HTTP/1.0\r\n\r\n",
"a Host with a zone": "GET / HTTP/1.1\r\nHost: [fe80::1%25eth0]:80\r\n\r\n",
"two User-Agents": "GET / HTTP/1.1\r\nUser-Agent: a\r\nUser-Agent: b\r\n" +
host,
} { } {
t.Run(name, func(t *testing.T) { t.Run(name, func(t *testing.T) {
t.Parallel() t.Parallel()
+10 -30
View File
@@ -11,7 +11,7 @@ import (
"net/netip" "net/netip"
"net/url" "net/url"
"os" "os"
"strconv" "strings"
"sync" "sync"
"sync/atomic" "sync/atomic"
"time" "time"
@@ -29,9 +29,6 @@ const flushAfterEachWrite time.Duration = -1
// send a higher one, up to 999, and the visitor gets that as it is. // send a higher one, up to 999, and the visitor gets that as it is.
const reportStatusMax = 599 const reportStatusMax = 599
// plainText is the Content-Type of the answers rproxy gives itself.
const plainText = "text/plain; charset=utf-8"
// request is one request on its way through rproxy, from the moment its headers // request is one request on its way through rproxy, from the moment its headers
// have been read to its log line and report. // have been read to its log line and report.
type request struct { type request struct {
@@ -112,9 +109,8 @@ func (h *handler) newRequest(w http.ResponseWriter, r *http.Request) *request {
// answerHealth answers the health check. // answerHealth answers the health check.
func (rq *request) answerHealth() { func (rq *request) answerHealth() {
rq.prepareOwnAnswer(rq.requestDeadline()) rq.out.Header().Set("Content-Type", "text/plain; charset=utf-8")
rq.out.Header().Set("Content-Type", plainText) _, _ = io.WriteString(rq.out, "ok\n")
rq.send(http.StatusOK, []byte("ok\n"))
} }
// readFirstPartOfBody reads up to the first 64 KiB of the body into memory, // readFirstPartOfBody reads up to the first 64 KiB of the body into memory,
@@ -153,9 +149,13 @@ func (rq *request) readFirstPartOfBody() bool {
// setPathAsSent has out, the URL of a request made from in, send the path as the // setPathAsSent has out, the URL of a request made from in, send the path as the
// visitor sent it. Go sends the path URL.EscapedPath gives, which, for a path // visitor sent it. Go sends the path URL.EscapedPath gives, which, for a path
// holding a character Go escapes, such as |, is the whole path decoded and // holding a character Go escapes, such as |, is the whole path decoded and
// escaped again, %2F turned into /. Opaque is sent as it is. // escaped again, %2F turned into /. Opaque is sent as it is. Left out is a path
// starting with //, which Go would send in Opaque as a scheme and host.
func setPathAsSent(out *url.URL, in *http.Request) { func setPathAsSent(out *url.URL, in *http.Request) {
out.Opaque = rules.Path(in) path := rules.Path(in)
if !strings.HasPrefix(path, "//") {
out.Opaque = path
}
} }
// forward passes the request to the app, the part of the body held and then the // forward passes the request to the app, the part of the body held and then the
@@ -297,27 +297,7 @@ func (rq *request) answer(status int) {
} }
rq.prepareOwnAnswer(deadline) rq.prepareOwnAnswer(deadline)
http.Error(rq.out, http.StatusText(status), status)
header := rq.out.Header()
header.Set("Content-Type", plainText)
header.Set("X-Content-Type-Options", "nosniff")
rq.send(status, []byte(http.StatusText(status)+"\n"))
}
// send gives the visitor an answer rproxy holds whole: status, the headers set so
// far and body. Go's server would otherwise send so short an answer only once the
// handler has returned, after finish has written the log line and sent the report.
// Without Content-Length, the flush would send the body in chunks, the last of
// which would still wait for the handler. A HEAD's answer has no body, so any
// Content-Length it has stays as servewaf gave it.
func (rq *request) send(status int, body []byte) {
if rq.in.Method != http.MethodHead {
rq.out.Header().Set("Content-Length", strconv.Itoa(len(body)))
}
rq.out.WriteHeader(status)
_, _ = rq.out.Write(body)
_ = rq.rc.Flush()
} }
// prepareOwnAnswer readies the connection for an answer rproxy gives itself: the // prepareOwnAnswer readies the connection for an answer rproxy gives itself: the
+3 -1
View File
@@ -164,7 +164,9 @@ func (h *handler) ServeHTTP(w http.ResponseWriter, r *http.Request) {
return return
} }
if !canBeAsked(r) { // Go's client sends only a target that starts with / as it came. For an empty
// Host, it sends servewaf's own address.
if r.Host == "" || !strings.HasPrefix(r.RequestURI, "/") {
rq.answer(http.StatusBadRequest) rq.answer(http.StatusBadRequest)
return return
+20 -309
View File
@@ -4,7 +4,6 @@ import (
"bufio" "bufio"
"bytes" "bytes"
"encoding/json" "encoding/json"
"errors"
"io" "io"
"log/slog" "log/slog"
"maps" "maps"
@@ -12,7 +11,6 @@ import (
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"slices" "slices"
"strconv"
"strings" "strings"
"sync" "sync"
"testing" "testing"
@@ -127,34 +125,6 @@ func (o *output) has(text string) bool {
return strings.Contains(o.buf.String(), text) return strings.Contains(o.buf.String(), text)
} }
// waitFor waits until what rproxy has written holds text.
func (o *output) waitFor(t *testing.T, text string) {
t.Helper()
deadline := time.Now().Add(waitLimit)
for time.Now().Before(deadline) {
if o.has(text) {
return
}
time.Sleep(pollInterval)
}
t.Fatalf("rproxy did not write %q", text)
}
// heldLog is a request log whose Write waits until release is closed.
type heldLog struct {
release chan struct{}
}
// Write waits until release is closed.
func (l heldLog) Write(p []byte) (int, error) {
<-l.release
return len(p), nil
}
// question is what the stand-in for servewaf received as a question. // question is what the stand-in for servewaf received as a question.
type question struct { type question struct {
method, target, host string method, target, host string
@@ -261,28 +231,6 @@ func (s *serveWAF) question(t *testing.T) question {
} }
} }
// startServeWAFAnsweringReports starts a stand-in for servewaf that answers each
// question as answer does and each report as report does, and returns its URL.
func startServeWAFAnsweringReports(
t *testing.T, answer, report http.HandlerFunc,
) string {
t.Helper()
backend := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get(proxy.ClientHeader) != "" {
answer(w, r)
return
}
report(w, r)
}))
t.Cleanup(backend.Close)
return backend.URL
}
// allow answers a question with allow. // allow answers a question with allow.
func allow(w http.ResponseWriter, _ *http.Request) { func allow(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(dispositionHeader, dispositionAllow) w.Header().Set(dispositionHeader, dispositionAllow)
@@ -332,20 +280,6 @@ func startServer(
) (string, *output, *rproxy.Server) { ) (string, *output, *rproxy.Server) {
t.Helper() t.Helper()
out := &output{}
addr, server := startServerLogging(t, backend, appURL, env, out, out)
return addr, out, server
}
// startServerLogging is startServer with rproxy's request lines going to
// requestLog and its process lines to processLog.
func startServerLogging(
t *testing.T, backend, appURL string, env map[string]string,
requestLog, processLog io.Writer,
) (string, *rproxy.Server) {
t.Helper()
settings := map[string]string{ settings := map[string]string{
"SWRP_WAF_BACKEND": backend, "SWRP_UPSTREAM_URL": appURL, "SWRP_WAF_BACKEND": backend, "SWRP_UPSTREAM_URL": appURL,
} }
@@ -360,10 +294,11 @@ func startServerLogging(
t.Fatalf("settings %v: %v", settings, err) t.Fatalf("settings %v: %v", settings, err)
} }
out := &output{}
server := rproxy.New(rproxy.Params{ server := rproxy.New(rproxy.Params{
Config: cfg, Config: cfg,
RequestLog: requestLog, RequestLog: out,
ProcessLog: requestlog.NewProcessLogger(processLog, "test", slog.LevelInfo), ProcessLog: requestlog.NewProcessLogger(out, "test", slog.LevelInfo),
}) })
listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0") listener, err := (&net.ListenConfig{}).Listen(t.Context(), "tcp", localhost+":0")
@@ -381,7 +316,7 @@ func startServerLogging(
server.WaitForReports() server.WaitForReports()
}) })
return listener.Addr().String(), server return listener.Addr().String(), out, server
} }
// answered is the visitor's answer. // answered is the visitor's answer.
@@ -596,45 +531,6 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
} }
} }
func TestATrustedProxysForwardedHeadersPassOn(t *testing.T) {
t.Parallel()
servewaf := startServeWAF(t, allow)
seen := make(chan http.Header, 1)
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
seen <- r.Header.Clone()
greet(w, r)
})
addr, _ := startRProxy(t, servewaf.URL, app.URL, map[string]string{
trustedProxies: localhost + "/32",
})
sent := map[string]string{
"Forwarded": "for=198.51.100.1",
"X-Forwarded-Host": "public.example",
forwardedProtoHeader: "https",
"X-Real-Ip": "198.51.100.1",
}
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
req.Host = site
for name, value := range sent {
req.Header.Set(name, value)
}
wantGreeting(t, visit(t, req))
header := <-seen
for name, want := range sent {
if got := header.Values(name); !slices.Equal(got, []string{want}) {
t.Errorf("the app saw %s %q, want %q", name, got, want)
}
}
}
func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) { func TestAnAnswerWithoutAContentTypeGetsNone(t *testing.T) {
t.Parallel() t.Parallel()
@@ -730,87 +626,6 @@ func TestDenyGivesTheVisitorServeWAFsAnswer(t *testing.T) {
} }
} }
func TestADeniedHeadKeepsServeWAFsContentLength(t *testing.T) {
t.Parallel()
servewaf := startServeWAF(t, deny)
app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
if got.status != http.StatusForbidden ||
got.header.Get(contentLengthHeader) != strconv.Itoa(len(deniedBody)) {
t.Errorf("answered %+v, want servewaf's 403 and its length", got)
}
}
func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
t.Parallel()
const host = " HTTP/1.1\r\nHost: site.example\r\n\r\n"
for _, tc := range []struct {
name, request, body string
status int
}{
{"a deny", "GET /" + host, deniedBody, http.StatusForbidden},
{"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest},
{"the health check", "GET " + proxy.HealthPath + host, "ok\n", http.StatusOK},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
log := heldLog{release: make(chan struct{})}
servewaf := startServeWAF(t, deny)
app := startApp(t, greet)
addr, _ := startServerLogging(t, servewaf.URL, app.URL, nil, log, io.Discard)
t.Cleanup(func() { close(log.release) })
conn := dial(t, addr)
send(t, conn, tc.request)
res, err := http.ReadResponse(bufio.NewReader(conn), nil)
if err != nil {
t.Fatalf("read the answer: %v", err)
}
defer func() {
_ = res.Body.Close()
}()
body, err := io.ReadAll(res.Body)
if err != nil || res.StatusCode != tc.status || string(body) != tc.body {
t.Errorf("answered %d %q (%v), want %d %q", res.StatusCode, body, err,
tc.status, tc.body)
}
})
}
}
func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
t.Parallel()
servewaf := startServeWAF(t, allow)
app := startApp(t, greet)
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
conn := dial(t, addr)
send(t, conn, "GET x:admin HTTP/1.1\r\nHost: site.example\r\n\r\n")
res, err := http.ReadResponse(bufio.NewReader(conn), nil)
if err != nil {
t.Fatalf("read the answer: %v", err)
}
_ = res.Body.Close()
if res.Header.Get("Content-Type") != "text/plain; charset=utf-8" ||
res.Header.Get("X-Content-Type-Options") != "nosniff" {
t.Errorf("the answer's headers are %v", res.Header)
}
}
func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) { func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
t.Parallel() t.Parallel()
@@ -883,8 +698,6 @@ func noAnswers() map[string]func(t *testing.T) string {
w.Header().Set(contentLengthHeader, "100") w.Header().Set(contentLengthHeader, "100")
w.WriteHeader(http.StatusForbidden) w.WriteHeader(http.StatusForbidden)
_, _ = io.WriteString(w, deniedBody) _, _ = io.WriteString(w, deniedBody)
// Go's server drops on this panic what it has not flushed.
_ = http.NewResponseController(w).Flush()
panic(http.ErrAbortHandler) panic(http.ErrAbortHandler)
}), }),
@@ -1039,97 +852,41 @@ func TestTheVisitorDoesNotWaitForTheReport(t *testing.T) {
t.Parallel() t.Parallel()
release := make(chan struct{}) release := make(chan struct{})
backend := startServeWAFAnsweringReports(t, deny, backend := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) { func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get(proxy.ClientHeader) != "" {
deny(w, r)
return
}
select { select {
case <-release: case <-release:
case <-r.Context().Done(): case <-r.Context().Done():
} }
w.WriteHeader(http.StatusNoContent) w.WriteHeader(http.StatusNoContent)
}) }))
t.Cleanup(backend.Close)
app := startApp(t, greet) app := startApp(t, greet)
addr, _ := startRProxy(t, backend, app.URL, map[string]string{ addr, _ := startRProxy(t, backend.URL, app.URL, map[string]string{
timeoutSetting: "60000", timeoutSetting: "60000",
}) })
t.Cleanup(func() { close(release) }) t.Cleanup(func() { close(release) })
// Go's server reads the next request on a connection only once the handler has // Go's server sends a deny's answer only once the handler has returned, so
// returned, so the second answer comes only if the handler does not wait for // this answer comes only if the handler does not wait for the report, which
// the first report, which the stand-in leaves unanswered. // the stand-in leaves unanswered.
conn := dial(t, addr) conn := dial(t, addr)
reader := bufio.NewReader(conn) send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n")
for range 2 { if status := readStatus(t, conn); status != http.StatusForbidden {
send(t, conn, "GET / HTTP/1.1\r\nHost: site.example\r\n\r\n") t.Errorf("status %d, want servewaf's 403", status)
res, err := http.ReadResponse(reader, nil)
if err != nil {
t.Fatalf("read the answer: %v", err)
}
_, _ = io.Copy(io.Discard, res.Body)
_ = res.Body.Close()
if res.StatusCode != http.StatusForbidden {
t.Errorf("status %d, want servewaf's 403", res.StatusCode)
}
} }
} }
func TestAReportServeWAFLeavesUnansweredIsDroppedAtTheTimeLimit(t *testing.T) {
t.Parallel()
release := make(chan struct{})
backend := startServeWAFAnsweringReports(t, allow,
func(w http.ResponseWriter, _ *http.Request) {
<-release
w.WriteHeader(http.StatusNoContent)
})
app := startApp(t, greet)
addr, out := startRProxy(t, backend, app.URL, map[string]string{
timeoutSetting: "1000",
})
t.Cleanup(func() { close(release) })
wantGreeting(t, get(t, addr, "/"))
out.waitFor(t, "a report is dropped")
}
func TestAReportServeWAFRefusesIsDroppedWithAWarning(t *testing.T) {
t.Parallel()
backend := startServeWAFAnsweringReports(t, allow,
func(w http.ResponseWriter, _ *http.Request) {
w.WriteHeader(http.StatusBadRequest)
})
app := startApp(t, greet)
addr, out := startRProxy(t, backend, app.URL, nil)
wantGreeting(t, get(t, addr, "/"))
out.waitFor(t, "a report is dropped")
}
func TestAReportDueAfterWaitForReportsIsDropped(t *testing.T) {
t.Parallel()
servewaf := startServeWAF(t, allow)
app := startApp(t, greet)
addr, out, server := startServer(t, servewaf.URL, app.URL, nil)
server.WaitForReports()
wantGreeting(t, get(t, addr, "/"))
out.requestLine(t)
servewaf.wantNoReport(t)
}
func TestReportGivesWhatHappened(t *testing.T) { func TestReportGivesWhatHappened(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1326,52 +1083,6 @@ func TestAWebSocketPassesOnThatTheVisitorStoppedSending(t *testing.T) {
} }
} }
// This test does not run in parallel. Go closes a connection nothing refers to at
// its next garbage collection, which other tests' allocations would soon start, and
// that would hide a connection left open.
//
//nolint:paralleltest // see above
func TestASwitchNotPassedOnClosesTheAppsConnection(t *testing.T) {
closed := make(chan error, 1)
servewaf := startServeWAF(t, allow)
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
conn, buffered, err := http.NewResponseController(w).Hijack()
if err != nil {
closed <- err
return
}
defer func() {
_ = conn.Close()
}()
// Not the protocol the visitor asked for, so ReverseProxy refuses the switch.
_, _ = buffered.WriteString("HTTP/1.1 101 Switching Protocols\r\n" +
"Connection: Upgrade\r\nUpgrade: another\r\n\r\n")
_ = buffered.Flush()
_ = conn.SetReadDeadline(time.Now().Add(waitLimit))
_, err = buffered.ReadByte()
closed <- err
})
addr, _ := startRProxy(t, servewaf.URL, app.URL, nil)
conn := dial(t, addr)
send(t, conn, "GET /socket HTTP/1.1\r\nHost: app\r\n"+
"Connection: Upgrade\r\nUpgrade: websocket\r\n\r\n")
if status := readStatus(t, conn); status != http.StatusBadGateway {
t.Errorf("status %d, want 502", status)
}
err := <-closed
if !errors.Is(err, io.EOF) {
t.Errorf("the app's read gave %v, want its connection closed", err)
}
}
// echoAfterUpgrade is an app that switches protocols on request, and then sends // echoAfterUpgrade is an app that switches protocols on request, and then sends
// back each line it receives. // back each line it receives.
func echoAfterUpgrade(w http.ResponseWriter, _ *http.Request) { func echoAfterUpgrade(w http.ResponseWriter, _ *http.Request) {
+6 -22
View File
@@ -9,7 +9,6 @@ import (
"io" "io"
"maps" "maps"
"net/http" "net/http"
"strings"
"sneak.berlin/go/smallwebwaf/internal/proxy" "sneak.berlin/go/smallwebwaf/internal/proxy"
) )
@@ -47,9 +46,6 @@ func (rq *request) askServeWAF(ctx context.Context) *servewafAnswer {
return nil return nil
} }
// A request that keeps servewaf busy past SWRP_TIMEOUT_MS, like any request
// while servewaf is overloaded, gets SWRP_DEFAULT_DISPOSITION here. deny closes
// this, at the cost of answering 503 whenever servewaf is slow.
rq.h.processLog.Warn("servewaf gave no answer", "error", err.Error()) rq.h.processLog.Warn("servewaf gave no answer", "error", err.Error())
rq.disposition = rq.h.config.DefaultDisposition rq.disposition = rq.h.config.DefaultDisposition
@@ -104,20 +100,6 @@ func (rq *request) ask(ctx context.Context) (*servewafAnswer, error) {
} }
} }
// canBeAsked reports whether Go's client can send servewaf r as the visitor sent
// it. It cannot send these as they came:
// - a target that does not start with /, or that starts with //;
// - an empty Host, for which it sends servewaf's own address;
// - a Host holding an IPv6 zone, which it removes;
// - more than one User-Agent, of which it sends the first.
func canBeAsked(r *http.Request) bool {
target, host := r.RequestURI, r.Host
hostHasZone := strings.HasPrefix(host, "[") && strings.Contains(host, "%")
return strings.HasPrefix(target, "/") && !strings.HasPrefix(target, "//") &&
host != "" && !hostHasZone && len(r.Header.Values("User-Agent")) <= 1
}
// newQuestion returns the question about the request, as "The question" in // newQuestion returns the question about the request, as "The question" in
// SPEC.md gives it: the visitor's request as the visitor sent it, with the part // SPEC.md gives it: the visitor's request as the visitor sent it, with the part
// of the body it holds and rproxy's headers. // of the body it holds and rproxy's headers.
@@ -143,9 +125,10 @@ func (rq *request) newQuestion(ctx context.Context) (*http.Request, error) {
question.URL = &target question.URL = &target
question.Host = in.Host question.Host = in.Host
// The length NewRequestWithContext set is that of what is held. This line makes // The length NewRequestWithContext set is that of what is held. A body whose
// a body whose length the visitor did not announce go chunked, as "The question" // length the visitor did not announce goes chunked. Named here, since Go's
// in SPEC.md requires. // client would otherwise send a CONNECT's body with neither length nor chunks,
// which servewaf would read as further requests.
question.TransferEncoding = in.TransferEncoding question.TransferEncoding = in.TransferEncoding
question.Header = in.Header.Clone() question.Header = in.Header.Clone()
@@ -177,7 +160,8 @@ func (rq *request) answerWith(answer *servewafAnswer) {
header := rq.out.Header() header := rq.out.Header()
maps.Copy(header, answer.header) maps.Copy(header, answer.header)
header.Del(proxy.DispositionHeader) header.Del(proxy.DispositionHeader)
rq.send(answer.status, answer.body) rq.out.WriteHeader(answer.status)
_, _ = rq.out.Write(answer.body)
} }
// sendReport sends servewaf report on a goroutine of its own, so that the // sendReport sends servewaf report on a goroutine of its own, so that the
+5 -94
View File
@@ -26,10 +26,6 @@ import (
const ( const (
// wafBackend is the setting that names servewaf to rproxy. // wafBackend is the setting that names servewaf to rproxy.
wafBackend = "SWRP_WAF_BACKEND" wafBackend = "SWRP_WAF_BACKEND"
// rproxyListenAddr is the setting that says where rproxy listens.
rproxyListenAddr = "SWRP_LISTEN_ADDR"
// rproxyTimeout is the setting that says how long rproxy waits for servewaf.
rproxyTimeout = "SWRP_TIMEOUT_MS"
// bannedProbe is a path the default rule file bans a client for. // bannedProbe is a path the default rule file bans a client for.
bannedProbe = "/.env" bannedProbe = "/.env"
// otherClient is a client the tests do not ban. // otherClient is a client the tests do not ban.
@@ -62,7 +58,7 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
go func() { go func() {
exited <- runRProxy(ctx, map[string]string{ exited <- runRProxy(ctx, map[string]string{
rproxyListenAddr: localhost + ":0", "SWRP_LISTEN_ADDR": localhost + ":0",
"SWRP_UPSTREAM_URL": appURL, "SWRP_UPSTREAM_URL": appURL,
"SWRP_TRUSTED_PROXIES": localhost + "/32", "SWRP_TRUSTED_PROXIES": localhost + "/32",
wafBackend: backend, wafBackend: backend,
@@ -507,91 +503,6 @@ func TestRProxyAsksAServeWAFThatSpeaksHTTP2OverHTTP1(t *testing.T) {
} }
} }
func TestRProxyProcessLinesNameTheHostAndComeAtInfo(t *testing.T) {
t.Parallel()
hostname, err := os.Hostname()
if err != nil {
t.Fatalf("the host's name: %v", err)
}
// Nothing listens at the backend's address; no request asks it.
_, out := startRProxy(t, "http://127.0.0.1:9", startApp(t))
line := out.line(t, "msg", "starting")
if line["instance"] != hostname || line["level"] != "INFO" {
t.Errorf("rproxy's line is %v, want instance %s at INFO", line, hostname)
}
}
func TestRProxyStoppingWaitsForAReportBeingSent(t *testing.T) {
t.Parallel()
reportGotThere := make(chan struct{})
release := make(chan struct{})
backend := httptest.NewServer(http.HandlerFunc(
func(w http.ResponseWriter, r *http.Request) {
if r.Header.Get(proxy.ClientHeader) != "" {
w.Header().Set(proxy.DispositionHeader, proxy.DispositionAllow)
return
}
close(reportGotThere)
<-release
w.WriteHeader(http.StatusNoContent)
}))
t.Cleanup(backend.Close)
ctx, stop := context.WithCancel(t.Context())
defer stop()
env := map[string]string{
rproxyListenAddr: localhost + ":0",
"SWRP_UPSTREAM_URL": startApp(t),
rproxyTimeout: "60000",
wafBackend: backend.URL,
}
out := &output{}
exited := make(chan int, 1)
go func() {
exited <- runRProxy(ctx, env, out)
}()
addr, _ := out.line(t, "msg", "starting")["address"].(string)
wantStatus(t, "http://"+addr+"/", placed, http.StatusOK)
select {
case <-reportGotThere:
case <-time.After(waitLimit):
t.Fatal("no report")
}
stop()
select {
case <-exited:
t.Error("rproxy stopped with a report on its way")
close(release)
return
case <-time.After(10 * pollInterval):
}
close(release)
select {
case status := <-exited:
if status != 0 {
t.Errorf("rproxy's exit status %d, want 0; output:\n%s", status, out.text())
}
case <-time.After(waitLimit):
t.Error("rproxy still running, the report having been answered")
}
}
func TestRProxyInvalidSettingStopsTheStart(t *testing.T) { func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
t.Parallel() t.Parallel()
@@ -601,8 +512,8 @@ func TestRProxyInvalidSettingStopsTheStart(t *testing.T) {
}{ }{
{"no backend", wafBackend, nil}, {"no backend", wafBackend, nil},
{ {
"another invalid", rproxyTimeout, "another invalid", "SWRP_TIMEOUT_MS",
map[string]string{wafBackend: "http://servewaf:8080", rproxyTimeout: "soon"}, map[string]string{wafBackend: "http://servewaf:8080", "SWRP_TIMEOUT_MS": "soon"},
}, },
} { } {
t.Run(tc.name, func(t *testing.T) { t.Run(tc.name, func(t *testing.T) {
@@ -639,8 +550,8 @@ func TestRProxyAddressInUseStopsTheStart(t *testing.T) {
out := &output{} out := &output{}
status := runRProxy(t.Context(), map[string]string{ status := runRProxy(t.Context(), map[string]string{
wafBackend: "http://servewaf:8080", wafBackend: "http://servewaf:8080",
rproxyListenAddr: taken.Addr().String(), "SWRP_LISTEN_ADDR": taken.Addr().String(),
}, out) }, out)
if status != 1 { if status != 1 {
t.Errorf("exit status %d, want 1", status) t.Errorf("exit status %d, want 1", status)