Compare commits

..
2 Commits
Author SHA1 Message Date
sneak 387744ab50 rproxy: close the connection when a short answer is cut off (closes #318)
check / check (push) Successful in 3m44s
When SWRP_MAX_RESPONSE_TIME_MS ran out before the last of a short
answer was written, the flush failed and ReverseProxy ignored it. If
the visitor had stopped short of its announced body, Go's server then
read the rest until SWRP_MAX_REQUEST_TIME_MS ran out, keeping the
connection open with the answer cut off. Go does that read even after
an aborting panic.

forward now checks for a failed flush once ReverseProxy returns. It
then records a 504 refusal, ends the reading of the body at once, and
panics so that Go's server closes the connection, as SPEC.md says a
cut-off answer does.

Model: opus-5-5
2026-11-11 02:34:56 +00:00
sneak 78047e52a9 rproxy tests: a cut-off answer must close the connection, and the stop-short tests no longer race the clock
A new test fails on the current code: when the response time limit
runs out before rproxy can write the app's short body, and the visitor
has stopped a little short of its announced length, the connection
stays open until the request time limit instead of closing.

The test that failed under load used a 300 ms response time limit,
which a busy host can use up between the app's headers and its body.
It and the test that answers a visitor that stops sending now use
limits far longer than the test's 10-second read deadline.

Model: opus-5-5
2026-11-11 02:34:56 +00:00
15 changed files with 78 additions and 357 deletions

No files matched your search

-3
View File
@@ -37,6 +37,3 @@ These are the rules for agents working on this repository.
- `192.168.0.0/16`. - `192.168.0.0/16`.
- `SPEC.md` states that default. - `SPEC.md` states that default.
- Forwarded headers count only when the peer is a trusted proxy. - Forwarded headers count only when the peer is a trusted proxy.
- Every time is in UTC, whatever `TZ` or the host's time zone says.
- Every day is a date written `YYYY-MM-DD`, never a time.
- `SPEC.md` states this near its start.
-5
View File
@@ -12,10 +12,6 @@ it is built, except what the last section lists. `README.md` says how to run it.
under `docs/` give the settings, the formats and the lists. This document links under `docs/` give the settings, the formats and the lists. This document links
to them. to them.
The whole application stores, logs and handles every time in UTC, whatever `TZ`
or the host's time zone says. It keeps every day as a date written `YYYY-MM-DD`,
never as a time.
## Purpose ## Purpose
- It protects self-hosted apps on the open internet from scrapers, scanners and - It protects self-hosted apps on the open internet from scrapers, scanners and
@@ -1779,7 +1775,6 @@ in. `docs/state-files.md` lists the files, what each holds, and their sizes.
- a top-level `version` number; - a top-level `version` number;
- entries sorted by client address; - entries sorted by client address;
- times in RFC 3339 UTC; - times in RFC 3339 UTC;
- days as `YYYY-MM-DD`;
- durations and sizes as plain numbers, with the unit in the field name. - durations and sizes as plain numbers, with the unit in the field name.
- `bans.json`, `reputation.json` and `alerts.json` are indented, so that a - `bans.json`, `reputation.json` and `alerts.json` are indented, so that a
person can open `bans.json` in an editor, find an address, and remove or add person can open `bans.json` in an editor, find an address, and remove or add
-1
View File
@@ -3,7 +3,6 @@ module sneak.berlin/go/smallwebwaf
go 1.26.0 go 1.26.0
require ( require (
cloud.google.com/go v0.123.0
github.com/corazawaf/coraza-coreruleset/v4 v4.25.0 github.com/corazawaf/coraza-coreruleset/v4 v4.25.0
github.com/corazawaf/coraza/v3 v3.8.2-0.20261022102553-d028be399c68 github.com/corazawaf/coraza/v3 v3.8.2-0.20261022102553-d028be399c68
github.com/fsnotify/fsnotify v1.10.1 github.com/fsnotify/fsnotify v1.10.1
-2
View File
@@ -1,5 +1,3 @@
cloud.google.com/go v0.123.0 h1:2NAUJwPR47q+E35uaJeYoNhuNEM9kM8SjgRgdeOJUSE=
cloud.google.com/go v0.123.0/go.mod h1:xBoMV08QcqUGuPW65Qfm1o9Y4zKZBpGS+7bImXLTAZU=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM= github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw= github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs= github.com/cespare/xxhash/v2 v2.3.0 h1:UL815xU9SqsFlibzuggzjXhog7bL6oX9BbNZnL2UFvs=
+1 -1
View File
@@ -26,7 +26,7 @@ type FileParams struct {
// Path is the lookup database, the IPinfo Lite file in its .mmdb form // Path is the lookup database, the IPinfo Lite file in its .mmdb form
// (SWW_LOOKUP_DB_PATH). // (SWW_LOOKUP_DB_PATH).
Path string Path string
// Now tells the time, normally time.Now in UTC. // Now tells the time, normally time.Now.
Now func() time.Time Now func() time.Time
// ProcessLog receives each reading of the file, and why a replacement // ProcessLog receives each reading of the file, and why a replacement
// of it cannot be read. // of it cannot be read.
+1 -1
View File
@@ -74,7 +74,7 @@ type Params struct {
// Answered, unless nil, is given each answer GeoJS gives, once it is // Answered, unless nil, is given each answer GeoJS gives, once it is
// kept. // kept.
Answered func(Answer) Answered func(Answer)
// Now tells the time, normally time.Now in UTC. // Now tells the time, normally time.Now.
Now func() time.Time Now func() time.Time
// ProcessLog receives GeoJS's failures. // ProcessLog receives GeoJS's failures.
ProcessLog *slog.Logger ProcessLog *slog.Logger
+8 -7
View File
@@ -14,7 +14,6 @@ import (
"sync" "sync"
"time" "time"
"cloud.google.com/go/civil"
"github.com/hashicorp/golang-lru/v2/simplelru" "github.com/hashicorp/golang-lru/v2/simplelru"
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
) )
@@ -28,6 +27,8 @@ const (
AbuseIPDBSource = "abuseipdb" AbuseIPDBSource = "abuseipdb"
// maxAnswerBytes is the most of an answer of AbuseIPDB that is read. // maxAnswerBytes is the most of an answer of AbuseIPDB that is read.
maxAnswerBytes = 64 << 10 maxAnswerBytes = 64 << 10
// day is the length of the day the checks are counted in, in UTC.
day = 24 * time.Hour
) )
var ( var (
@@ -49,9 +50,9 @@ type Score struct {
// AbuseIPDB: the day, in UTC, of the checks Spent counts, zero before the // AbuseIPDB: the day, in UTC, of the checks Spent counts, zero before the
// first, and the scores still in use. // first, and the scores still in use.
type Checks struct { type Checks struct {
Day civil.Date `json:"day,omitzero"` Day time.Time `json:"day,omitzero"`
Spent int `json:"spent"` Spent int `json:"spent"`
Scores []Score `json:"scores"` Scores []Score `json:"scores"`
} }
// AbuseIPDBParams are what NewAbuseIPDB needs. // AbuseIPDBParams are what NewAbuseIPDB needs.
@@ -92,7 +93,7 @@ type AbuseIPDB struct {
// checking are the clients whose check is under way. // checking are the clients whose check is under way.
checking map[netip.Prefix]bool checking map[netip.Prefix]bool
// day is the day, in UTC, of the checks spent counts. // day is the day, in UTC, of the checks spent counts.
day civil.Date day time.Time
spent int spent int
// checks and failures count the checks made and those that failed, // checks and failures count the checks made and those that failed,
// and retryAt is when a client may be checked again after the last // and retryAt is when a client may be checked again after the last
@@ -142,7 +143,7 @@ func (a *AbuseIPDB) Hit(
return kept.Score, kept.Score >= a.params.MinScore return kept.Score, kept.Score >= a.params.MinScore
} }
if today := civil.DateOf(now.UTC()); a.day != today { if today := now.Truncate(day); !a.day.Equal(today) {
a.day, a.spent = today, 0 a.day, a.spent = today, 0
} }
@@ -189,7 +190,7 @@ func (a *AbuseIPDB) BudgetLeft() int {
a.mu.Lock() a.mu.Lock()
defer a.mu.Unlock() defer a.mu.Unlock()
if a.day != civil.DateOf(a.params.Now().UTC()) { if !a.day.Equal(a.params.Now().Truncate(day)) {
return a.params.DailyBudget return a.params.DailyBudget
} }
-37
View File
@@ -202,43 +202,6 @@ func TestDailyBudgetKeptAcrossARestartAndWholeAgainAsTheDayEnds(t *testing.T) {
}) })
} }
func TestChecksAreCountedInUTCDaysWhateverTheClocksZone(t *testing.T) {
t.Parallel()
synctest.Test(t, func(t *testing.T) {
// A clock 14 hours ahead of UTC, whose day starts at 10:00 UTC.
ahead := time.FixedZone("UTC+14", 14*60*60)
p := abuseIPDBParams()
p.DailyBudget = 1
p.Now = func() time.Time { return time.Now().In(ahead) }
abuseIPDB := &abuseIPDBStandIn{scores: map[string]int64{suspect: 100}}
checker := newAbuseIPDB(abuseIPDB, p)
// At 09:00 UTC the one check is spent.
time.Sleep(9 * time.Hour)
hitFrom(t, checker, suspect, true)
synctest.Wait()
wantBudgetLeft(t, checker, 0)
// At 11:00 UTC the clock's day has changed, but the UTC day has not.
time.Sleep(2 * time.Hour)
wantBudgetLeft(t, checker, 0)
encoded, err := json.Marshal(checker.Snapshot())
if err != nil {
t.Fatalf("encode: %v", err)
}
if !strings.Contains(string(encoded), `"day":"2000-01-01"`) {
t.Errorf("reputation.json would hold %s, want the day 2000-01-01", encoded)
}
// At 00:00 UTC the budget is whole again.
time.Sleep(13 * time.Hour)
wantBudgetLeft(t, checker, 1)
})
}
func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) { func TestFailedCheckGivesNoScoreAndNoClientIsCheckedForAMinute(t *testing.T) {
t.Parallel() t.Parallel()
+36 -55
View File
@@ -920,72 +920,53 @@ func TestAVisitorThatStopsALittleShortHasTheConnectionClosedWhenTheAnswerIsCutOf
const limit = 300 * time.Millisecond const limit = 300 * time.Millisecond
// With two parts, rproxy fails to send the first and then waits for the headersRead := make(chan struct{})
// second, which the app sends only once rproxy has ended the request to it. // letAppSendBody also runs when the test ends, as the app's server waits for
for _, tc := range []struct { // the app.
name string letAppSendBody := sync.OnceFunc(func() { close(headersRead) })
parts []string standIn := startWAF(t, allow)
}{ // The app sends its body once the visitor has the status and headers and the
{"with the body in one part", []string{greeting}}, // response time limit has run out since, so that rproxy cannot send it.
{"with the body in two parts", []string{greeting[:5], greeting[5:]}}, app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
} { rc := http.NewResponseController(w)
t.Run(tc.name, func(t *testing.T) { _ = rc.EnableFullDuplex()
t.Parallel()
headersRead := make(chan struct{}) w.Header().Set(contentLengthHeader, strconv.Itoa(len(greeting)))
// letAppSendBody also runs when the test ends, as the app's server waits
// for the app.
letAppSendBody := sync.OnceFunc(func() { close(headersRead) })
standIn := startWAF(t, allow)
// The app sends its body once the visitor has the status and headers and
// the response time limit has run out since, so that rproxy cannot send
// it. After each part, it reads the request's body, which ends when rproxy
// ends the request to it.
app := startApp(t, func(w http.ResponseWriter, r *http.Request) {
rc := http.NewResponseController(w)
_ = rc.EnableFullDuplex()
w.Header().Set(contentLengthHeader, strconv.Itoa(len(greeting))) _ = rc.Flush()
_ = rc.Flush() <-headersRead
time.Sleep(limit)
<-headersRead _, _ = io.WriteString(w, greeting)
time.Sleep(limit) _ = rc.Flush()
_, _ = io.Copy(io.Discard, r.Body)
})
for _, part := range tc.parts { t.Cleanup(letAppSendBody)
_, _ = io.WriteString(w, part)
_ = rc.Flush()
_, _ = io.Copy(io.Discard, r.Body)
}
})
t.Cleanup(letAppSendBody) // The request time limit is far longer than dial's deadline.
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{
maxRequestTime: "120000",
maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
})
// The request time limit is far longer than dial's deadline. conn := dial(t, addr)
addr, _ := startRProxy(t, standIn.URL, app.URL, map[string]string{ sendAllButALittle(t, conn)
maxRequestTime: "120000",
maxResponseTime: strconv.FormatInt(limit.Milliseconds(), 10),
})
conn := dial(t, addr) res, err := http.ReadResponse(bufio.NewReader(conn), nil)
sendAllButALittle(t, conn) if err != nil {
t.Fatalf("read the answer: %v", err)
}
res, err := http.ReadResponse(bufio.NewReader(conn), nil) letAppSendBody()
if err != nil {
t.Fatalf("read the answer: %v", err)
}
letAppSendBody() // If the connection stays open, dial's deadline ends this read with a timeout.
body, err := io.ReadAll(res.Body)
_ = res.Body.Close()
// If the connection stays open, dial's deadline ends this read with a if len(body) != 0 || !errors.Is(err, io.ErrUnexpectedEOF) {
// timeout. t.Errorf("read %q (%v), want the connection closed", body, err)
body, err := io.ReadAll(res.Body)
_ = res.Body.Close()
if len(body) != 0 || !errors.Is(err, io.ErrUnexpectedEOF) {
t.Errorf("read %q (%v), want the connection closed", body, err)
}
})
} }
} }
+15 -12
View File
@@ -195,6 +195,20 @@ func (rq *request) forward(ctx context.Context) {
rq.startRequestTimer() rq.startRequestTimer()
rq.upstreamStart = time.Now() rq.upstreamStart = time.Now()
reverseProxy.ServeHTTP(rq.out, out) reverseProxy.ServeHTTP(rq.out, out)
// ReverseProxy ignores a failed flush, so it misses a short answer cut off when
// SWRP_MAX_RESPONSE_TIME_MS runs out; once a flush has failed, every later one
// fails too. Go's server would then keep the connection open while it reads
// the rest of a body that has not arrived whole. So that reading ends now, and
// the refusal keeps finish from extending it to SWRP_MAX_REQUEST_TIME_MS. The
// panic has Go's server close the connection, and log nothing; the deferred
// finish still runs.
if rq.out.status != http.StatusSwitchingProtocols &&
errors.Is(rq.rc.Flush(), os.ErrDeadlineExceeded) {
rq.refuse(http.StatusGatewayTimeout)
rq.stopPassingOnBody(time.Now())
panic(http.ErrAbortHandler)
}
} }
// rewrite makes the request the app receives: the visitor's request, unchanged, // rewrite makes the request the app receives: the visitor's request, unchanged,
@@ -384,18 +398,7 @@ func (rq *request) refuse(status int) {
func (rq *request) finish(ctx context.Context) { func (rq *request) finish(ctx context.Context) {
rq.stopTimers() rq.stopTimers()
// After a cut-off answer, the reading of a body that has not arrived whole if rq.refused.Load() == 0 {
// ends now. Go's server would otherwise keep the connection open while it
// reads the rest. A flush fails if the time to send the answer ran out, and
// every flush after a failed one fails too. ReverseProxy ignores the failure,
// but panics if it was still waiting for more of the app's answer; finish runs
// either way. Before a status, a flush would send a 200; after a 101,
// ReverseProxy has taken over the connection.
switch {
case rq.out.status != 0 && rq.out.status != http.StatusSwitchingProtocols &&
errors.Is(rq.rc.Flush(), os.ErrDeadlineExceeded):
rq.stopPassingOnBody(time.Now())
case rq.refused.Load() == 0:
rq.stopPassingOnBody(rq.requestDeadline()) rq.stopPassingOnBody(rq.requestDeadline())
} }
+1 -4
View File
@@ -1117,7 +1117,6 @@ func TestAnAnswerThatCannotBePassedOnIsAnswered502(t *testing.T) {
t.Error("the app was asked") t.Error("the app was asked")
}) })
addr, out := startRProxy(t, backend(t), app.URL, map[string]string{ addr, out := startRProxy(t, backend(t), app.URL, map[string]string{
timeoutSetting: "60000",
defaultDisposition: setting, defaultDisposition: setting,
}) })
@@ -1141,9 +1140,7 @@ func TestAnAnswerThatCannotBePassedOnGetsNoReport(t *testing.T) {
t.Parallel() t.Parallel()
standIn := startWAF(t, overOneMiB(dispositionDeny)) standIn := startWAF(t, overOneMiB(dispositionDeny))
addr, out := startRProxy(t, standIn.URL, nowhere, map[string]string{ addr, out := startRProxy(t, standIn.URL, nowhere, nil)
timeoutSetting: "60000",
})
if got := get(t, addr, "/"); got.status != http.StatusBadGateway { if got := get(t, addr, "/"); got.status != http.StatusBadGateway {
t.Errorf("answered %+v, want 502", got) t.Errorf("answered %+v, want 502", got)
+2 -81
View File
@@ -171,11 +171,9 @@ type alertsFile struct {
// check refuses the first entry without a field it needs, which would // check refuses the first entry without a field it needs, which would
// otherwise be read as something the entry does not say. data is the // otherwise be read as something the entry does not say. data is the
// file, for a field that may be null or "" but not left out, which the // file, for a field that may be null or "" but not left out, which the
// struct cannot tell apart. Its toUTC then converts every time in it to // struct cannot tell apart.
// UTC, since an admin may write a time with an offset.
type stateFile interface { type stateFile interface {
check(data []byte) error check(data []byte) error
toUTC()
} }
// Load checks that files can be written in Dir, and reads the state files // Load checks that files can be written in Dir, and reads the state files
@@ -920,80 +918,6 @@ func missing(i int, field string) error {
return fmt.Errorf("entry %d %w %q", i+1, errMissing, field) return fmt.Errorf("entry %d %w %q", i+1, errMissing, field)
} }
// toUTC converts every time of each ban to UTC.
func (f *bansFile) toUTC() {
for i := range f.Bans {
ban := &f.Bans[i]
convertToUTC(&ban.Start, ban.Expires, ban.Lifted, &ban.Notes.Request.Time)
}
}
// toUTC converts every time of each client to UTC.
func (f *clientsFile) toUTC() {
for i := range f.Clients {
client := &f.Clients[i]
convertToUTC(&client.Minute.Start, &client.Hour.Start, &client.Day.Start,
&client.MinuteBytes.Start, &client.HourBytes.Start,
&client.DayBytes.Start, &client.MinuteRefusals.Start,
&client.History.FirstSeen, &client.History.LastSeen,
&client.History.LookedUp)
}
}
// toUTC converts both times of each answer to UTC.
func (f *lookupsFile) toUTC() {
for i := range f.Lookups {
convertToUTC(&f.Lookups[i].Answered, &f.Lookups[i].Used)
}
}
// toUTC converts every time of each list, verdict and AbuseIPDB score to
// UTC.
func (f *reputationFile) toUTC() {
for i := range f.Lists {
convertToUTC(&f.Lists[i].Tried, &f.Lists[i].Fetched)
}
for i := range f.Verdicts {
convertToUTC(&f.Verdicts[i].Fetched)
}
for i := range f.AbuseIPDB.Scores {
convertToUTC(&f.AbuseIPDB.Scores[i].Fetched)
}
}
// toUTC converts every time of each cooldown, of the hour, of each alert
// waiting and of each anomaly counter to UTC.
func (f *alertsFile) toUTC() {
for i := range f.Cooldowns {
convertToUTC(&f.Cooldowns[i].Sent)
}
convertToUTC(&f.Hour.Start)
for _, waiting := range f.Waiting {
for i := range waiting {
convertToUTC(&waiting[i].Time)
}
}
for i := range f.AnomalyCounters {
counter := &f.AnomalyCounters[i]
convertToUTC(&counter.Minute.Start, &counter.Hour.Start,
&counter.MinuteBytes.Start, &counter.HourBytes.Start)
}
}
// convertToUTC converts each of times that is not nil to UTC.
func convertToUTC(times ...*time.Time) {
for _, at := range times {
if at != nil {
*at = at.UTC()
}
}
}
// encodeOnePerLine encodes a state file whose entries, under key, are one // encodeOnePerLine encodes a state file whose entries, under key, are one
// to a line, so that grep shows everything about one client. // to a line, so that grep shows everything about one client.
func encodeOnePerLine[E any](key string, entries []E) ([]byte, error) { func encodeOnePerLine[E any](key string, entries []E) ([]byte, error) {
@@ -1031,8 +955,7 @@ func checkWritable(dir string) error {
} }
// parse reads data, what the state file at path holds, into file, a // parse reads data, what the state file at path holds, into file, a
// pointer to that file's struct, checks its entries, and converts every // pointer to that file's struct, and checks its entries.
// time in it to UTC.
func parse(path string, data []byte, file stateFile) error { func parse(path string, data []byte, file stateFile) error {
// The version is read first, so that a file of another version is // The version is read first, so that a file of another version is
// refused for that, and not for an entry this version cannot read. // refused for that, and not for an entry this version cannot read.
@@ -1062,8 +985,6 @@ func parse(path string, data []byte, file stateFile) error {
return fmt.Errorf("%s%s: %w", path, position(data, err), err) return fmt.Errorf("%s%s: %w", path, position(data, err), err)
} }
file.toUTC()
return nil return nil
} }
+7 -67
View File
@@ -14,7 +14,6 @@ import (
"os" "os"
"path/filepath" "path/filepath"
"reflect" "reflect"
"regexp"
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
@@ -22,7 +21,6 @@ import (
"testing/synctest" "testing/synctest"
"time" "time"
"cloud.google.com/go/civil"
"sneak.berlin/go/smallwebwaf/internal/alerts" "sneak.berlin/go/smallwebwaf/internal/alerts"
"sneak.berlin/go/smallwebwaf/internal/anomaly" "sneak.berlin/go/smallwebwaf/internal/anomaly"
"sneak.berlin/go/smallwebwaf/internal/bans" "sneak.berlin/go/smallwebwaf/internal/bans"
@@ -260,7 +258,7 @@ const filledReputationJSON = `{
} }
], ],
"abuseipdb": { "abuseipdb": {
"day": "2026-10-06", "day": "2026-10-06T00:00:00Z",
"spent": 3, "spent": 3,
"scores": [ "scores": [
{ {
@@ -1308,7 +1306,7 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
`"tried": "2026-10-06T00:00:00Z", "fetched": "2026-10-06T00:00:00Z", `+ `"tried": "2026-10-06T00:00:00Z", "fetched": "2026-10-06T00:00:00Z", `+
`"lines": ["198.51.100.7"]}], "verdicts": [{"zone": "`+dnsblZone+`", `+ `"lines": ["198.51.100.7"]}], "verdicts": [{"zone": "`+dnsblZone+`", `+
`"client": "198.51.100.7", "listed": true, "fetched": "2026-10-06T00:00:00Z"}], `+ `"client": "198.51.100.7", "listed": true, "fetched": "2026-10-06T00:00:00Z"}], `+
`"abuseipdb": {"day": "2026-10-06", "spent": 9, "scores": [`+ `"abuseipdb": {"day": "2026-10-06T00:00:00Z", "spent": 9, "scores": [`+
`{"client": "198.51.100.7/32", "score": 80, "fetched": "2026-10-06T00:00:00Z"}]}}`) `{"client": "198.51.100.7/32", "score": 80, "fetched": "2026-10-06T00:00:00Z"}]}}`)
wantTakenIn(t, lines, dir, reputationJSON) wantTakenIn(t, lines, dir, reputationJSON)
@@ -1323,7 +1321,7 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
}}) }})
checks := reputation.Checks{ checks := reputation.Checks{
Day: civil.DateOf(midnight()), Spent: 9, Day: midnight(), Spent: 9,
Scores: []reputation.Score{{Client: client, Score: 80, Fetched: midnight()}}, Scores: []reputation.Score{{Client: client, Score: 80, Fetched: midnight()}},
} }
if got := params.AbuseIPDB.Snapshot(); !reflect.DeepEqual(got, checks) { if got := params.AbuseIPDB.Snapshot(); !reflect.DeepEqual(got, checks) {
@@ -1352,62 +1350,6 @@ func TestEditOfEachFileTakenIn(t *testing.T) {
} }
} }
func TestTimesEditedWithAnOffsetWrittenBackInUTC(t *testing.T) {
t.Parallel()
dir := t.TempDir()
params := newParams(dir)
fill(params)
// A lifted ban, so that bans.json holds every time a ban has.
params.Ledger.Lift(netip.MustParseAddr("192.0.2.1"), midnight())
files := load(t, params)
err := files.WriteAll()
if err != nil {
t.Fatalf("write: %v", err)
}
// An admin saves each file with every time in it two hours ahead of
// UTC, midnight as "2026-10-06T02:00:00+02:00".
stamp := regexp.MustCompile(`"\d{4}-\d\d-\d\dT[^"]*Z"`)
ahead := time.FixedZone("UTC+2", 2*60*60)
written := map[string]string{}
for _, name := range []string{
bansJSON, clientsJSON, lookupsJSON, reputationJSON, alertsJSON,
} {
written[name] = readFile(t, filepath.Join(dir, name))
edited := stamp.ReplaceAllStringFunc(written[name], func(quoted string) string {
at, err := time.Parse(time.RFC3339Nano, strings.Trim(quoted, `"`))
if err != nil {
t.Fatalf("%s holds %s: %v", name, quoted, err)
}
return `"` + at.In(ahead).Format(time.RFC3339Nano) + `"`
})
if !strings.Contains(edited, "+02:00") {
t.Fatalf("%s holds no time:\n%s", name, edited)
}
edit(t, dir, name, edited)
}
// Each write takes its edit in first, and writes the same times back in
// UTC.
err = files.WriteAll()
if err != nil {
t.Fatalf("write: %v", err)
}
for name, want := range written {
if got := readFile(t, filepath.Join(dir, name)); got != want {
t.Errorf("%s written back as\n%s\nwant\n%s", name, got, want)
}
}
}
func TestOwnWritesAreNotTakenIn(t *testing.T) { func TestOwnWritesAreNotTakenIn(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1895,12 +1837,10 @@ func fill(params state.Params) {
}, },
{Zone: dnsblZone, Client: client.Addr(), Listed: true, Fetched: now.Add(-time.Hour)}, {Zone: dnsblZone, Client: client.Addr(), Listed: true, Fetched: now.Add(-time.Hour)},
}) })
params.AbuseIPDB.Load(reputation.Checks{ params.AbuseIPDB.Load(reputation.Checks{Day: now, Spent: 3, Scores: []reputation.Score{
Day: civil.DateOf(now), Spent: 3, Scores: []reputation.Score{ {Client: netip.MustParsePrefix("2001:db8::/64"), Fetched: now.Add(-2 * time.Hour)},
{Client: netip.MustParsePrefix("2001:db8::/64"), Fetched: now.Add(-2 * time.Hour)}, {Client: client, Score: 100, Fetched: now.Add(-time.Hour)},
{Client: client, Score: 100, Fetched: now.Add(-time.Hour)}, }})
},
})
// An alert waiting, a repeat of it the cooldown holds back, another // An alert waiting, a repeat of it the cooldown holds back, another
// alert waiting, and one past the two an hour, for the hour's summary. // alert waiting, and one past the two an hour, for the hour's summary.
+4 -30
View File
@@ -92,31 +92,6 @@ func (o *output) line(t *testing.T, key, value string) map[string]any {
return nil return nil
} }
// lineBeforeExit returns the first line whose msg is msg, waiting for it
// while sww runs. It waits as long as that takes, so that a slow start
// cannot fail the test, and fails the test if sww exits first.
func (o *output) lineBeforeExit(
t *testing.T, msg string, exited <-chan int,
) map[string]any {
t.Helper()
for {
fields := o.findLine(t, func(fields map[string]any) bool {
return fields["msg"] == msg
})
if fields != nil {
return fields
}
select {
case status := <-exited:
t.Fatalf("exit status %d while waiting for a line with msg %q; output:\n%s",
status, msg, o.text())
case <-time.After(pollInterval):
}
}
}
// findLine returns the first line written so far for which matches is true, or // findLine returns the first line written so far for which matches is true, or
// nil. // nil.
func (o *output) findLine( func (o *output) findLine(
@@ -261,12 +236,11 @@ func TestServesUntilToldToStop(t *testing.T) {
}() }()
// The default rule file is read. // The default rule file is read.
readRules := out.lineBeforeExit(t, "read the rule files", exited) if rules := out.line(t, "msg", "read the rule files")["rules"]; rules != 12.0 {
if rules := readRules["rules"]; rules != 12.0 {
t.Errorf("read %v rules from the default rule file, want 12", rules) t.Errorf("read %v rules from the default rule file, want 12", rules)
} }
starting := out.lineBeforeExit(t, "starting", exited) starting := out.line(t, "msg", "starting")
wantStartingLine(t, starting, dir) wantStartingLine(t, starting, dir)
addr, _ := starting["address"].(string) addr, _ := starting["address"].(string)
@@ -312,7 +286,7 @@ func TestLogLevelHoldsBackTheLessSevereProcessLines(t *testing.T) {
}, out) }, out)
}() }()
out.lineBeforeExit(t, "fetching a list failed", exited) out.line(t, "msg", "fetching a list failed")
stop() stop()
select { select {
@@ -1287,7 +1261,7 @@ func runUntilStopped(
exited <- run(ctx, env, out) exited <- run(ctx, env, out)
}() }()
addr, _ := out.lineBeforeExit(t, "starting", exited)["address"].(string) addr, _ := out.line(t, "msg", "starting")["address"].(string)
use("http://" + addr + "/") use("http://" + addr + "/")
stop() stop()
+3 -51
View File
@@ -9,14 +9,10 @@ import (
"os" "os"
"os/exec" "os/exec"
"path/filepath" "path/filepath"
"regexp"
"strings" "strings"
"syscall" "syscall"
"testing" "testing"
"time" "time"
// The time zone database, so that the process TestMainRunsWAF starts
// finds the zone TZ names on any host.
_ "time/tzdata"
"sneak.berlin/go/smallwebwaf/internal/sww" "sneak.berlin/go/smallwebwaf/internal/sww"
"sneak.berlin/go/smallwebwaf/internal/waf" "sneak.berlin/go/smallwebwaf/internal/waf"
@@ -41,14 +37,6 @@ func TestMain(m *testing.M) {
func TestMainRunsWAF(t *testing.T) { func TestMainRunsWAF(t *testing.T) {
t.Parallel() t.Parallel()
// Kiritimati is 14 hours ahead of UTC; every time sww gives stays in UTC.
const zone = "Pacific/Kiritimati"
_, err := time.LoadLocation(zone)
if err != nil {
t.Fatalf("load %s: %v", zone, err)
}
binary, err := os.Executable() binary, err := os.Executable()
if err != nil { if err != nil {
t.Fatalf("find the test binary: %v", err) t.Fatalf("find the test binary: %v", err)
@@ -57,16 +45,14 @@ func TestMainRunsWAF(t *testing.T) {
ctx, cancel := context.WithTimeout(t.Context(), 3*waitLimit) ctx, cancel := context.WithTimeout(t.Context(), 3*waitLimit)
defer cancel() defer cancel()
dir := t.TempDir()
out := &output{} out := &output{}
cmd := exec.CommandContext(ctx, binary) //nolint:gosec // this test binary cmd := exec.CommandContext(ctx, binary) //nolint:gosec // this test binary
cmd.Stdout = out cmd.Stdout = out
cmd.Env = append(os.Environ(), cmd.Env = append(os.Environ(),
"TZ="+zone,
commandForMain+"="+wafCommand, commandForMain+"="+wafCommand,
listenAddr+"="+localhost+":0", listenAddr+"="+localhost+":0",
stateDir+"="+dir, stateDir+"="+t.TempDir(),
rulesDir+"="+t.TempDir(), rulesDir+"="+t.TempDir(),
lookupSource+"=off") lookupSource+"=off")
@@ -80,11 +66,9 @@ func TestMainRunsWAF(t *testing.T) {
_ = cmd.Wait() _ = cmd.Wait()
}) })
starting := out.line(t, "msg", "starting") addr, _ := out.line(t, "msg", "starting")["address"].(string)
addr, _ := starting["address"].(string)
ask(t, addr, "/", http.StatusOK, waf.DispositionAllow) ask(t, addr, "/", http.StatusOK, waf.DispositionAllow)
// The files are written as sww stops.
err = cmd.Process.Signal(syscall.SIGTERM) err = cmd.Process.Signal(syscall.SIGTERM)
if err != nil { if err != nil {
t.Fatalf("signal: %v", err) t.Fatalf("signal: %v", err)
@@ -92,39 +76,7 @@ func TestMainRunsWAF(t *testing.T) {
err = cmd.Wait() err = cmd.Wait()
if err != nil { if err != nil {
t.Fatalf("exit: %v; output:\n%s", err, out.text()) t.Errorf("exit: %v; output:\n%s", err, out.text())
}
wantUTC(t, "the process line", starting["time"])
wantUTC(t, "the request line", out.line(t, "type", "request")["time"])
path := filepath.Join(dir, "clients.json")
clients, err := os.ReadFile(path) //nolint:gosec // a file in the test's directory
if err != nil {
t.Fatalf("read clients.json: %v", err)
}
stamp := regexp.MustCompile(`"\d{4}-\d\d-\d\dT[^"]*"`)
times := stamp.FindAllString(string(clients), -1)
if len(times) == 0 {
t.Errorf("clients.json holds no time:\n%s", clients)
}
for _, stored := range times {
wantUTC(t, "clients.json", strings.Trim(stored, `"`))
}
}
// wantUTC checks that value, a time read from the log line or file that
// where names, is in UTC.
func wantUTC(t *testing.T, where string, value any) {
t.Helper()
text, _ := value.(string)
if !strings.HasSuffix(text, "Z") {
t.Errorf("%s gives the time %v, want one in UTC", where, value)
} }
} }