Compare commits

..
6 Commits
Author SHA1 Message Date
sneak 965936ab4d Caches of outside APIs live in /var/cache/sww (closes #314)
check / check (push) Successful in 4m32s
The answers of GeoJS, the lists fetched by URL, the DNS blocklists,
AbuseIPDB and routewatch each go in a cache file in SWW_CACHE_DIR
(default /var/cache/sww): lookups.json, lists.json, dnsbl.json,
abuseipdb-scores.json and routewatch.json. reputation.json is gone. The
AbuseIPDB checks spent today are a count, not a cache, and stay in
SWW_STATE_DIR as abuseipdb-checks-spent.json.

A cache file never stops the start: one that does not parse is set
aside as an edit is, one that cannot be read is written over, and its
cache starts empty. An unwritable SWW_CACHE_DIR stops the start, as an
unwritable SWW_STATE_DIR does. Each directory is watched on its own.
The run script makes the cache directory and gives it to sww.

Model: opus-5-5
2026-11-11 07:23:39 +00:00
sneak eba697af48 rproxy: close the connection when a short answer is cut off (closes #318)
check / check (push) Failing after 3m39s
When SWRP_MAX_RESPONSE_TIME_MS ran out before the last of a short answer
was written, the flush failed. If the visitor had stopped short of its
announced body, Go's server then read the rest until
SWRP_MAX_REQUEST_TIME_MS ran out, keeping the connection open with the
answer cut off.

finish now flushes once more, and if that fails at its deadline, it ends
the reading of the body at once. ReverseProxy ignores the failed flush when
the app's answer came in one read, and panics when it was still waiting for
more; finish runs after both. Go's server never reuses a connection after a
failed write, so it closes it.

Model: opus-5-5
2026-11-11 07:54:10 +01:00
sneak b9cfbc7fcb rproxy tests: a cut-off answer must close the connection, and the stop-short tests no longer race the clock
A new test fails on the current code. The visitor stops a little short of
its announced body, and the response time limit runs out before rproxy can
write the app's short body. The connection then stays open until the
request time limit instead of closing. The app sends that body in one part,
and in two; with two, ReverseProxy panics instead of returning.

The test that failed under load used a 300 ms response time limit, which a
busy host can use up between the app's headers and its body. It and the
test that answers a visitor that stops sending now use limits far longer
than the test's 10-second read deadline.

Model: opus-5-5
2026-11-11 07:54:10 +01:00
clawbot f077dd7fd0 Say that rproxy passes a refused question's 400 to the visitor (closes #326)
check / check (push) Successful in 4m25s
The comment above `readQuestion` in `internal/waf/waf.go` said `rproxy` treats a refused question (`400`, no disposition) as no answer. It passes the `400` to the visitor, and `SWRP_DEFAULT_DISPOSITION` does not apply, as `SPEC.md` and `internal/rproxy/waf.go` say. Comment only.

Model: opus-5-5
2026-11-11 07:50:59 +01:00
clawbot 81ec6f9bf5 Request log: write & < > as sent (closes #327)
check / check (push) Failing after 3m30s
Request lines of `waf` and `rproxy` now hold `&`, `<` and `>` as sent, where `json.Marshal` wrote `\u0026`, `\u003c` and `\u003e`, and are still valid JSON. `writeJSONLine` encodes with a `json.Encoder` that has HTML escaping off, and still writes each line in one write. This follows the `AGENTS.md` rule that the request log records query strings exactly as received, and lets `grep` find a logged query string.

Alert webhooks and the report `rproxy` posts to `waf` are unchanged: neither is a request line, and both are read only by a JSON parser.

Model: opus-5-5
2026-11-11 07:50:54 +01:00
clawbot 7b52574e5b rproxy: remove every X-Forwarded- header from an untrusted peer (closes #325)
check / check (push) Failing after 3m22s
For a peer outside `SWRP_TRUSTED_PROXIES`, `rproxy` now removes every header whose name starts with `X-Forwarded-`, in any letter case, before it sets `X-Forwarded-For`, `X-Forwarded-Host` and `X-Forwarded-Proto`; `Forwarded` and `X-Real-IP` are removed as before. Until now only traefik's names were removed, so a client connecting directly could send `X-Forwarded-Ssl: on` or `X-Forwarded-Scheme: https` and make an app take a plain request for a secure one. A trusted peer's headers pass on unchanged. `SPEC.md` says so.

Model: opus-5-5
2026-11-11 07:50:50 +01:00
11 changed files with 234 additions and 65 deletions

No files matched your search

+2 -2
View File
@@ -673,8 +673,8 @@ stops the running process (see [`docs/state-files.md`](docs/state-files.md)).
`internal/routewatch`, `internal/anomaly` and `internal/alerts`: one check or `internal/routewatch`, `internal/anomaly` and `internal/alerts`: one check or
source each. source each.
- `internal/state`, `internal/requestlog`, `internal/remotelog` and - `internal/state`, `internal/requestlog`, `internal/remotelog` and
`internal/metrics`: the state files, the log lines, the syslog sender and the `internal/metrics`: the state and cache files, the log lines, the syslog
metrics. sender and the metrics.
## Rationale ## Rationale
+15 -14
View File
@@ -62,13 +62,13 @@ never as a time.
- Packages: - Packages:
- `cmd/sww`: the binary, which only calls `internal/sww`. - `cmd/sww`: the binary, which only calls `internal/sww`.
- `internal/sww`: the process, as the command it is given. `sww waf` reads - `internal/sww`: the process, as the command it is given. `sww waf` reads
the settings, the rule files, the lookup database and the state files, the settings, the rule files, the lookup database and the state and cache
listens, and serves until `SIGTERM` or `SIGINT`. Then it stops, writing files, listens, and serves until `SIGTERM` or `SIGINT`. Then it stops,
the state files (see "`waf`"). `sww rproxy` reads only that command's writing the state and cache files (see "`waf`"). `sww rproxy` reads only
settings, and runs `internal/rproxy` until `SIGTERM` or `SIGINT` (see that command's settings, and runs `internal/rproxy` until `SIGTERM` or
"`rproxy`"). `sww healthcheck` is the image's health check. With no `SIGINT` (see "`rproxy`"). `sww healthcheck` is the image's health check.
command, or one it does not know, the binary prints the commands and exits With no command, or one it does not know, the binary prints the commands
with status 2. and exits with status 2.
- `internal/config`: reads the settings, the one place they are read. - `internal/config`: reads the settings, the one place they are read.
- `internal/waf`: `waf`'s server. It runs the checks in `check`, and holds - `internal/waf`: `waf`'s server. It runs the checks in `check`, and holds
`waf`'s question, answer and report. It answers `waf`'s own endpoints in `waf`'s question, answer and report. It answers `waf`'s own endpoints in
@@ -106,7 +106,7 @@ never as a time.
container, which `script/example-app` checks with `waf` in another. container, which `script/example-app` checks with `waf` in another.
- Libraries, checked against the owner's Go dependency defaults: - Libraries, checked against the owner's Go dependency defaults:
- the standard library for the reverse proxy, HTTP clients, DNS, logging - the standard library for the reverse proxy, HTTP clients, DNS, logging
(`log/slog`) and the state files (`encoding/json`, `os.Rename`); (`log/slog`) and the state and cache files (`encoding/json`, `os.Rename`);
- `github.com/corazawaf/coraza/v3` and - `github.com/corazawaf/coraza/v3` and
`github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0`, which carries `github.com/corazawaf/coraza-coreruleset/v4` at `v4.25.0`, which carries
the Core Rule Set 4.25.0. The changes `smallwebwaf` makes to it, and the the Core Rule Set 4.25.0. The changes `smallwebwaf` makes to it, and the
@@ -367,7 +367,7 @@ below are `waf`'s settings; "`rproxy`" gives the use of `rproxy`'s. Conventions:
one, since `rproxy` never touches disk. one, since `rproxy` never touches disk.
- Settings, files included, are read once at start. Changing one means - Settings, files included, are read once at start. Changing one means
restarting the container. The files `waf` watches while it runs are its state restarting the container. The files `waf` watches while it runs are its state
files, its rule files and the lookup database. and cache files, its rule files and the lookup database.
- An invalid setting stops the process at start with one log line, marked - An invalid setting stops the process at start with one log line, marked
`"type":"process"` with `"msg":"invalid setting"`. The line names the variable `"type":"process"` with `"msg":"invalid setting"`. The line names the variable
and what is wrong with the value; "Each container's processes" says what and what is wrong with the value; "Each container's processes" says what
@@ -1932,10 +1932,10 @@ each app. `rproxy` asks `waf` about each visitor's request. `waf` answers
This section is the agreement between the two. The header names, the fields and This section is the agreement between the two. The header names, the fields and
the statuses below are exact. the statuses below are exact.
`waf` reads the settings, the rule files, the lookup database and the state `waf` reads the settings, the rule files, the lookup database and the state and
files, and listens at `SWW_LISTEN_ADDR`. On `SIGTERM` or `SIGINT` it stops cache files, and listens at `SWW_LISTEN_ADDR`. On `SIGTERM` or `SIGINT` it stops
taking requests and gives those in progress five seconds to finish. It then taking requests and gives those in progress five seconds to finish. It then
writes the state files. writes the state and cache files.
### The question ### The question
@@ -2351,8 +2351,9 @@ but for these headers:
on. on.
- For a peer outside `SWRP_TRUSTED_PROXIES`, its forwarded headers are its own - For a peer outside `SWRP_TRUSTED_PROXIES`, its forwarded headers are its own
claims and are replaced. `X-Forwarded-For` names the peer, `X-Forwarded-Host` claims and are replaced. `X-Forwarded-For` names the peer, `X-Forwarded-Host`
the host it asked for, and `X-Forwarded-Proto` plain `http`. The other headers the host it asked for, and `X-Forwarded-Proto` plain `http`. Every other
in which traefik tells an app about a visitor are removed. header whose name starts with `X-Forwarded-` is removed, and so are
`Forwarded` and `X-Real-IP`.
- `X-Request-ID`, which carries the fresh id. - `X-Request-ID`, which carries the fresh id.
- `X-Client-ASN` and `X-Client-Country`, which carry what `waf` answered. A - `X-Client-ASN` and `X-Client-Country`, which carry what `waf` answered. A
header it did not give is left out. Any of the two the visitor sent is header it did not give is left out. Any of the two the visitor sent is
+10 -3
View File
@@ -5,6 +5,7 @@
package requestlog package requestlog
import ( import (
"bytes"
"encoding/json" "encoding/json"
"fmt" "fmt"
"io" "io"
@@ -252,14 +253,20 @@ func WriteRProxyLine(w io.Writer, line *RProxyLine) error {
return writeJSONLine(w, line) return writeJSONLine(w, line)
} }
// writeJSONLine writes line to w as one JSON line. // writeJSONLine writes line to w as one JSON line. It writes "&", "<" and
// ">" as they are, so a query string reads as it was sent.
func writeJSONLine(w io.Writer, line any) error { func writeJSONLine(w io.Writer, line any) error {
encoded, err := json.Marshal(line) var encoded bytes.Buffer
encoder := json.NewEncoder(&encoded)
encoder.SetEscapeHTML(false)
err := encoder.Encode(line)
if err != nil { if err != nil {
return fmt.Errorf("encode the request log line: %w", err) return fmt.Errorf("encode the request log line: %w", err)
} }
_, err = w.Write(append(encoded, '\n')) _, err = w.Write(encoded.Bytes())
if err != nil { if err != nil {
return fmt.Errorf("write the request log line: %w", err) return fmt.Errorf("write the request log line: %w", err)
} }
+38
View File
@@ -67,6 +67,44 @@ func TestWriteWritesOneJSONLineMarkedRequest(t *testing.T) {
} }
} }
func TestRequestLinesHoldTheQueryAndRefererAsSent(t *testing.T) {
t.Parallel()
var out bytes.Buffer
err := requestlog.Write(&out, &requestlog.Line{
Query: "a=1&b=<2>",
Referer: "https://example.com/?x=1&y=<2>",
})
if err != nil {
t.Fatalf("write: %v", err)
}
err = requestlog.WriteRProxyLine(&out, &requestlog.RProxyLine{Query: "a=1&b=<2>"})
if err != nil {
t.Fatalf("write the rproxy line: %v", err)
}
lines := strings.Split(strings.TrimSuffix(out.String(), "\n"), "\n")
if len(lines) != 2 {
t.Fatalf("wrote %q, want two lines", out.String())
}
for _, line := range lines {
if !json.Valid([]byte(line)) {
t.Errorf("%s is not valid JSON", line)
}
if !strings.Contains(line, `"query":"a=1&b=<2>"`) {
t.Errorf("%s does not hold the query as sent", line)
}
}
if !strings.Contains(lines[0], `"referer":"https://example.com/?x=1&y=<2>"`) {
t.Errorf("%s does not hold the referer as sent", lines[0])
}
}
func TestWriteRProxyLineWritesOneJSONLineWithOnlyItsFields(t *testing.T) { func TestWriteRProxyLineWritesOneJSONLineWithOnlyItsFields(t *testing.T) {
t.Parallel() t.Parallel()
+9 -9
View File
@@ -105,17 +105,17 @@ func setForwardedHeaders(in, out *http.Request, peer netip.Addr, trusted bool) {
return return
} }
// ReverseProxy has removed Forwarded and the three set below; these are the // ReverseProxy has removed Forwarded and the three set below. Go's server
// other headers in which traefik tells the app about the visitor and its // gives header names in canonical form, so the prefix matches the name in
// request. // any letter case.
for _, name := range []string{ for name := range out.Header {
"X-Forwarded-Port", "X-Forwarded-Server", "X-Forwarded-Uri", if strings.HasPrefix(name, "X-Forwarded-") {
"X-Forwarded-Method", "X-Forwarded-Prefix", "X-Forwarded-Tls-Client-Cert", delete(out.Header, name)
"X-Forwarded-Tls-Client-Cert-Info", "X-Real-Ip", }
} {
out.Header.Del(name)
} }
out.Header.Del("X-Real-IP")
out.Header.Set("X-Forwarded-For", forwardedFor) out.Header.Set("X-Forwarded-For", forwardedFor)
out.Header.Set("X-Forwarded-Host", in.Host) out.Header.Set("X-Forwarded-Host", in.Host)
out.Header.Set("X-Forwarded-Proto", "http") out.Header.Set("X-Forwarded-Proto", "http")
+84 -5
View File
@@ -12,6 +12,7 @@ import (
"os" "os"
"strconv" "strconv"
"strings" "strings"
"sync"
"sync/atomic" "sync/atomic"
"testing" "testing"
"time" "time"
@@ -807,8 +808,10 @@ func TestAVisitorThatStopsSendingAfterTheAppHasAnsweredGetsTheAnswer(t *testing.
app := startApp(t, func(w http.ResponseWriter, _ *http.Request) { app := startApp(t, func(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, greeting) _, _ = io.WriteString(w, greeting)
}) })
// The request time limit is far longer than dial's deadline, so that a busy
// host cannot use it up before the app's answer arrives.
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "3000", maxRequestTime: "120000",
}) })
// Far more announced than sent, so that Go's server answers for the app // Far more announced than sent, so that Go's server answers for the app
@@ -869,11 +872,11 @@ func TestAVisitorThatStopsALittleShortGetsTheAppsAnswerAtOnce(t *testing.T) {
standIn := startWAF(t, allow) standIn := startWAF(t, allow)
app := startApp(t, answerBeforeTheBody) app := startApp(t, answerBeforeTheBody)
// The request time limit is longer than the 10 seconds the status and headers // Both time limits are far longer than dial's deadline, so that the answer
// of an answer have to be sent, and than dial's deadline. // has to come before either runs out, and a busy host cannot use them up.
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{ addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "15000", maxRequestTime: "120000",
maxResponseTime: "300", maxResponseTime: "60000",
}) })
conn := dial(t, addr) conn := dial(t, addr)
@@ -910,6 +913,82 @@ func TestAVisitorThatStopsALittleShortIsReadUntilTheRequestTimeLimit(t *testing.
} }
} }
func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOff(
t *testing.T,
) {
t.Parallel()
const limit = 300 * time.Millisecond
// With two parts, rproxy fails to send the first and then waits for the
// second, which the app sends only once rproxy has ended the request to it.
for _, tc := range []struct {
name string
parts []string
}{
{"with the body in one part", []string{greeting}},
{"with the body in two parts", []string{greeting[:5], greeting[5:]}},
} {
t.Run(tc.name, func(t *testing.T) {
t.Parallel()
headersRead := make(chan struct{})
// letAppSendBody also runs when the test ends, as the app's server waits
// for the app.
letAppSendBody := sync.OnceFunc(func() { close(headersRead) })
standIn := startWAF(t, allow)
// The app sends its body once the visitor has the status and headers and
// the response time limit has run out since, so that rproxy cannot send
// it. After each part, it reads the request's body, which ends when rproxy
// ends the request to it.
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
rc := http.NewResponseController(w)
_ = rc.EnableFullDuplex()
w.Header().Set(contentLengthHeader, strconv.Itoa(len(greeting)))
_ = rc.Flush()
<-headersRead
time.Sleep(limit)
for _, part := range tc.parts {
_, _ = io.WriteString(w, part)
_ = rc.Flush()
_, _ = io.Copy(io.Discard, r.Body)
}
})
t.Cleanup(letAppSendBody)
// The request time limit is far longer than dial's deadline.
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "120000",
maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
})
conn := dial(t, addr)
sendAllButALittle(t, conn)
res, err := http.ReadResponse(bufio.NewReader(conn), nil)
if err != nil {
t.Fatalf("read the answer: %v", err)
}
letAppSendBody()
// If the connection stays open, dial's deadline ends this read with a
// timeout.
body, err := io.ReadAll(res.Body)
_ = res.Body.Close()
if len(body) != 0 || !errors.Is(err, io.ErrUnexpectedEOF) {
t.Errorf("read %q (%v), want the connection closed", body, err)
}
})
}
}
func TestAVisitorThatStopsSendingWhileTheAppWorksGetsNoAnswer(t *testing.T) { func TestAVisitorThatStopsSendingWhileTheAppWorksGetsNoAnswer(t *testing.T) {
t.Parallel() t.Parallel()
+18 -7
View File
@@ -384,8 +384,19 @@ func (rq *request) refuse(status int) {
func (rq *request) finish(ctx context.Context) { func (rq *request) finish(ctx context.Context) {
rq.stopTimers() rq.stopTimers()
if rq.refused.Load() == 0 { // After a cut-off answer, the reading of a body that has not arrived whole
rq.stopPassingOnBody() // ends now. Go's server would otherwise keep the connection open while it
// reads the rest. A flush fails if the time to send the answer ran out, and
// every flush after a failed one fails too. ReverseProxy ignores the failure,
// but panics if it was still waiting for more of the app's answer; finish runs
// either way. Before a status, a flush would send a 200; after a 101,
// ReverseProxy has taken over the connection.
switch {
case rq.out.status != 0 && rq.out.status != http.StatusSwitchingProtocols &&
errors.Is(rq.rc.Flush(), os.ErrDeadlineExceeded):
rq.stopPassingOnBody(time.Now())
case rq.refused.Load() == 0:
rq.stopPassingOnBody(rq.requestDeadline())
} }
requestEnd := rq.endRequest() requestEnd := rq.endRequest()
@@ -473,10 +484,10 @@ func (rq *request) stopReadingBody(deadline time.Time) {
} }
// stopPassingOnBody ends the transport's reading of a visitor's body that has // stopPassingOnBody ends the transport's reading of a visitor's body that has
// not arrived whole, and lets Go's server read the rest until // not arrived whole, and lets Go's server read the rest until deadline. Go's
// SWRP_MAX_REQUEST_TIME_MS runs out. Go's server reads it once the handler // server reads it once the handler returns, but with no deadline at all if
// returns, but with no deadline at all if another read of it is still waiting. // another read of it is still waiting.
func (rq *request) stopPassingOnBody() { func (rq *request) stopPassingOnBody(deadline time.Time) {
if rq.body == nil || rq.body.received.Load() { if rq.body == nil || rq.body.received.Load() {
return return
} }
@@ -489,7 +500,7 @@ func (rq *request) stopPassingOnBody() {
rq.body.stopped = true rq.body.stopped = true
rq.body.mu.Unlock() rq.body.mu.Unlock()
rq.stopReadingBody(rq.requestDeadline()) rq.stopReadingBody(deadline)
} }
// startRequestTimer starts SWRP_MAX_REQUEST_TIME_MS, which runs from the // startRequestTimer starts SWRP_MAX_REQUEST_TIME_MS, which runs from the
+7 -1
View File
@@ -559,6 +559,8 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
"X-Forwarded-Port", "X-Forwarded-Server", "X-Forwarded-Uri", "X-Forwarded-Port", "X-Forwarded-Server", "X-Forwarded-Uri",
"X-Forwarded-Method", "X-Forwarded-Prefix", "X-Forwarded-Tls-Client-Cert", "X-Forwarded-Method", "X-Forwarded-Prefix", "X-Forwarded-Tls-Client-Cert",
"X-Forwarded-Tls-Client-Cert-Info", "X-Real-Ip", "X-Forwarded-Tls-Client-Cert-Info", "X-Real-Ip",
"X-Forwarded-Scheme", "X-Forwarded-Ssl", "X-Forwarded-Protocol",
"X-Forwarded-Path", "X-Forwarded-Client-Cert",
} }
standIn := startWAF(t, allow) standIn := startWAF(t, allow)
@@ -582,6 +584,9 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
req.Header.Set(name, "claimed") req.Header.Set(name, "claimed")
} }
// A made-up name, sent in lower case.
req.Header["x-forwarded-made-up"] = []string{"claimed"}
wantGreeting(t, visit(t, req)) wantGreeting(t, visit(t, req))
header := <-seen header := <-seen
@@ -596,7 +601,7 @@ func TestAPeerNotTrustedHasItsForwardedHeadersReplaced(t *testing.T) {
} }
} }
for _, name := range removed { for _, name := range append(removed, "X-Forwarded-Made-Up") {
if got := header.Values(name); len(got) != 0 { if got := header.Values(name); len(got) != 0 {
t.Errorf("the app saw %s %q, want none", name, got) t.Errorf("the app saw %s %q, want none", name, got)
} }
@@ -622,6 +627,7 @@ func TestATrustedProxysForwardedHeadersPassOn(t *testing.T) {
"X-Forwarded-Host": "public.example", "X-Forwarded-Host": "public.example",
forwardedProtoHeader: "https", forwardedProtoHeader: "https",
"X-Real-Ip": "198.51.100.1", "X-Real-Ip": "198.51.100.1",
"X-Forwarded-Ssl": "on",
} }
req := newRequest(t, http.MethodGet, addr, "/", http.NoBody) req := newRequest(t, http.MethodGet, addr, "/", http.NoBody)
+18 -8
View File
@@ -283,7 +283,7 @@ func Load(params Params) (*Files, error) {
f.readCache(abuseIPDBScoresJSON) f.readCache(abuseIPDBScoresJSON)
f.readCache(routeWatchJSON) f.readCache(routeWatchJSON)
params.ProcessLog.Info("read the state files", "directory", params.Dir, params.ProcessLog.Info("read the state and cache files", "directory", params.Dir,
"cache_directory", params.CacheDir, "bans", bansRead, "clients", clientsRead, "cache_directory", params.CacheDir, "bans", bansRead, "clients", clientsRead,
"lookups", lookupsRead, "lists", listsRead, "alerts_waiting", alertsRead) "lookups", lookupsRead, "lists", listsRead, "alerts_waiting", alertsRead)
@@ -344,7 +344,7 @@ func (f *Files) WriteAll() error {
func (f *Files) Watch(ctx context.Context) { func (f *Files) Watch(ctx context.Context) {
watcher, err := fsnotify.NewWatcher() watcher, err := fsnotify.NewWatcher()
if err != nil { if err != nil {
f.params.ProcessLog.Error("cannot watch the state files for edits", f.params.ProcessLog.Error("cannot watch the state and cache files for edits",
"error", err.Error()) "error", err.Error())
return return
@@ -371,12 +371,18 @@ func (f *Files) Watch(ctx context.Context) {
watched = append(watched, dir.path) watched = append(watched, dir.path)
} }
if len(watched) == 0 { watching := "watching the state and cache files for edits"
switch {
case len(watched) == 0:
return return
case len(watched) == 1 && watched[0] == f.params.Dir:
watching = "watching the state files for edits"
case len(watched) == 1:
watching = "watching the cache files for edits"
} }
f.params.ProcessLog.Info("watching the state files for edits", f.params.ProcessLog.Info(watching, "directories", watched)
"directories", watched)
for { for {
select { select {
@@ -389,7 +395,7 @@ func (f *Files) Watch(ctx context.Context) {
f.fileChanged(name) f.fileChanged(name)
} }
case err = <-watcher.Errors: case err = <-watcher.Errors:
f.params.ProcessLog.Warn("watching the state files failed", f.params.ProcessLog.Warn("watching the state and cache files failed",
"error", err.Error()) "error", err.Error())
} }
} }
@@ -443,11 +449,15 @@ func (f *Files) takeInEdit(name string, data []byte) error {
return err return err
} }
tookIn := "took in an edit of a state file"
if isCacheFile(name) {
tookIn = "took in an edit of a cache file"
}
// Counted before it is logged, so that the count is there once the // Counted before it is logged, so that the count is there once the
// log line is. // log line is.
f.params.Metrics.StateFileEditTakenIn(name) f.params.Metrics.StateFileEditTakenIn(name)
f.params.ProcessLog.Info("took in an edit of a state file", f.params.ProcessLog.Info(tookIn, "file", filepath.Join(f.dir(name), name))
"file", filepath.Join(f.dir(name), name))
return nil return nil
} }
+29 -12
View File
@@ -62,10 +62,11 @@ const (
// The AS number and AS name the tests' clients are looked up in. // The AS number and AS name the tests' clients are looked up in.
asn = "AS64496" asn = "AS64496"
asName = "Example Net" asName = "Example Net"
// What the process log says once Watch watches the directory, and as // What the process log says once Watch watches both directories, and as
// it takes in an edit. // it takes in an edit of a state file, and of a cache file.
watching = "watching the state files for edits" watching = "watching the state and cache files for edits"
tookIn = "took in an edit of a state file" tookIn = "took in an edit of a state file"
cacheTookIn = "took in an edit of a cache file"
// maxLogLines is how many lines of the process log wait for a test to // maxLogLines is how many lines of the process log wait for a test to
// read them. // read them.
maxLogLines = 64 maxLogLines = 64
@@ -2063,11 +2064,18 @@ func TestDirectoryThatCannotBeWatchedLeavesTheOtherWatched(t *testing.T) {
for _, tc := range []struct { for _, tc := range []struct {
// gone is the directory removed before Watch runs, and failed what // gone is the directory removed before Watch runs, and failed what
// the log says of it. edited is a file in the other directory. // the log says of it. watching is what the log says once Watch
gone, failed, edited string // watches the other directory, and edited is a file in it.
gone, failed, watching, edited string
}{ }{
{"state", "cannot watch the state files for edits", lookupsJSON}, {
{"cache", "cannot watch the cache files for edits", bansJSON}, "state", "cannot watch the state files for edits",
"watching the cache files for edits", lookupsJSON,
},
{
"cache", "cannot watch the cache files for edits",
"watching the state files for edits", bansJSON,
},
} { } {
t.Run(tc.gone, func(t *testing.T) { t.Run(tc.gone, func(t *testing.T) {
t.Parallel() t.Parallel()
@@ -2095,7 +2103,7 @@ func TestDirectoryThatCannotBeWatchedLeavesTheOtherWatched(t *testing.T) {
t.Errorf("logged %v, want an error naming %s", line, gone) t.Errorf("logged %v, want an error naming %s", line, gone)
} }
lines.waitFor(t, watching) lines.waitFor(t, tc.watching)
edit(t, other, tc.edited, `{"version": 1}`) edit(t, other, tc.edited, `{"version": 1}`)
wantTakenIn(t, lines, other, tc.edited) wantTakenIn(t, lines, other, tc.edited)
}) })
@@ -2460,12 +2468,21 @@ func (l processLog) waitFor(t *testing.T, msg string) map[string]any {
return nil return nil
} }
// wantTakenIn waits for the next edit taken in, and checks that it is of // wantTakenIn waits for the next edit taken in, of a state file or a cache
// the state file name in dir. // file as name is one, and checks that it is of the file name in dir.
func wantTakenIn(t *testing.T, lines processLog, dir, name string) { func wantTakenIn(t *testing.T, lines processLog, dir, name string) {
t.Helper() t.Helper()
line := lines.waitFor(t, tookIn) var msg string
switch name {
case lookupsJSON, listsJSON, dnsblJSON, abuseIPDBScoresJSON, routeWatchJSON:
msg = cacheTookIn
default:
msg = tookIn
}
line := lines.waitFor(t, msg)
if line["file"] != filepath.Join(dir, name) { if line["file"] != filepath.Join(dir, name) {
t.Fatalf("took in %v, want an edit of %s", line, name) t.Fatalf("took in %v, want an edit of %s", line, name)
} }
+4 -4
View File
@@ -213,10 +213,10 @@ func (h *handler) answerQuestion(w http.ResponseWriter, r *http.Request) {
// readQuestion returns the visitor's address and the body a question carries, // readQuestion returns the visitor's address and the body a question carries,
// and whether waf takes the question. It answers a question it does not // and whether waf takes the question. It answers a question it does not
// take with 400 and no disposition, which rproxy treats as it treats no // take with 400 and no disposition. rproxy passes that 400 to the visitor,
// answer. The id of a question it takes is held from here on, without a // and SWRP_DEFAULT_DISPOSITION does not apply. The id of a question waf takes
// decision yet, so that another question with that id is refused while this // is held from here on, without a decision yet, so that another question with
// one is being handled. // that id is refused while this one is being handled.
// //
// The body is read before any check, since Go's server does not send an answer // The body is read before any check, since Go's server does not send an answer
// until it has read the body. A question whose body stops arriving gets no // until it has read the body. A question whose body stops arriving gets no