Compare commits

..
Author SHA1 Message Date
clawbot fde9b9c0b8 requestlog: list checks_timed_out among the values of would_action
check / check (push) Successful in 4m38s
The comment on WouldAction lists every action observe mode can log as
would_action, and left out ActionChecksTimedOut, which this change adds.

Model: opus-5-5
2026-11-17 04:48:26 +00:00
clawbot 7904647254 SPEC: the worst-case request is refused only if it takes over 500 ms
"Answering in time" said waf refuses the slowest request a visitor can
send after 500 milliseconds. Its inspection takes under a second, which
can be under 500 milliseconds, and then it is not refused.

Model: opus-5-5
2026-11-17 04:48:10 +00:00
clawbot b0a06aac63 waf: rename the heuristic list last, a name that said nothing
The list that runs tag_deny the second time was named last(). The bare
word does not say what it holds or when it runs, a local variable in
rulefiles.go shadows it, and its doc read "the heuristics of last run".
It is now afterBesideLookup, beside the other lists named for when they
run.

Model: opus-5-5
2026-11-17 04:47:47 +00:00
clawbot 0e5a5b468c waf: rename afterLookUp, which differed from afterLookup by one capital
The method that runs the heuristics of afterLookup was named afterLookUp,
beside the list afterLookup() in the same package. It is now
runAfterLookup, like runBesideLookup.

Model: opus-5-5
2026-11-17 04:47:28 +00:00
sneak 098bbe8878 waf: run the trap paths, rule files and Core Rule Set during the GeoJS wait (closes #405)
check / check (push) Failing after 1m56s
While the lookup waits for GeoJS, the trap paths, the rule files and the
Core Rule Set run at the same time as the checks after the lookup. The
first side to refuse answers the request; what the other side finds leaves
no trace. Those three checks have 500 ms together: past it, the request is
refused with 503 as checks_timed_out, with no ban and no offence.

Tests hold GeoJS and the Core Rule Set with stand-ins under synctest. The
test helper lifts the 500 ms limit for tests that run the real Core Rule
Set, which the race detector slows past it.

Model: opus-5-5
2026-11-17 04:27:26 +00:00
15 changed files with 31 additions and 134 deletions

No files matched your search

-14
View File
@@ -161,20 +161,6 @@ RUN git checkout --quiet --detach b4b2c785308b1ce785b6155c7fe5f16879080193 \
# ubuntu 26.04, 2026-09-27
FROM ubuntu@sha256:f144425ff09be612d6d9ad965196e9cdc23dae1f42110a8a11a3e9a8198759f7
# The image's own labels, in place of the ones it would inherit from
# Ubuntu's. script/docker and script/cibuild pass the build arguments;
# a build that passes none labels the image unknown rather than leave
# Ubuntu's values.
ARG VERSION=unknown
ARG CREATED=unknown
ARG REVISION=unknown
LABEL org.opencontainers.image.title="smallwebwaf" \
org.opencontainers.image.description="A web application firewall for self-hosted web apps: sww rproxy in front of an app, and sww waf, which decides." \
org.opencontainers.image.version="${VERSION}" \
org.opencontainers.image.created="${CREATED}" \
org.opencontainers.image.revision="${REVISION}" \
org.opencontainers.image.source="https://git.eeqj.de/sneak/smallwebwaf"
# runit's install creates its _runit-log user with minsysusers, which
# reads this file in place of runit's /usr/lib/sysusers.d/runit.conf.
# runit's line leaves out the shell, and minsysusers prints a Perl
-4
View File
@@ -9,10 +9,6 @@
ARG SWW_IMAGE=smallwebwaf
FROM ${SWW_IMAGE}
# Its own title and description, in place of the smallwebwaf image's.
LABEL org.opencontainers.image.title="smallwebwaf-example-app" \
org.opencontainers.image.description="An example app built on the smallwebwaf image."
# Packages the app needs, from the nixpkgs in the image.
RUN nix-env -iA nixpkgs.busybox
+5 -8
View File
@@ -31,12 +31,10 @@ import (
// this package: Go's resolver counts the queries under way in one
// sync.WaitGroup for the whole process, and the process fails when
// queries from two bubbles, or from a bubble and from outside one, are
// under way at once. The one test outside a bubble is parallel, so it
// starts only once every test that is not parallel has ended; no other
// parallel test of this package queries a resolver. TestMain has the
// resolver make its configuration, which it makes on its first query,
// outside every bubble, since the configuration holds a channel, which
// the bubble it was made in would keep to itself.
// under way at once. TestMain has the resolver make its configuration,
// which it makes on its first query, outside every bubble, since the
// configuration holds a channel, which the bubble it was made in would
// keep to itself.
const (
// zone and otherZone are the DNSBL zones the tests name.
@@ -527,9 +525,8 @@ func TestAtMost100000VerdictsKeptTheOneFetchedLongestAgoDroppedFirst(
}
}
//nolint:paralleltest // one at a time, as the comment at the top of this file says
func TestQueriesGoToTheResolverSWWDNSBLResolverNames(t *testing.T) {
t.Parallel()
resolver := &resolverStandIn{answers: map[string]answer{
listedName: {addrs: []string{listing}},
}}
-53
View File
@@ -71,59 +71,6 @@ func TestServerHasTheFixedLimits(t *testing.T) {
}
}
// getOfSize is a GET whose request line and headers, the blank line that ends
// them included, are size bytes long.
func getOfSize(size int) string {
const start, end = "GET / HTTP/1.1\r\nHost: site.example\r\nX-Pad: ", "\r\n\r\n"
return start + strings.Repeat("x", size-len(start)-len(end)) + end
}
func TestARequestLineAndHeadersOf32KiBReachTheApp(t *testing.T) {
t.Parallel()
standIn := startWAF(t, allow)
app := startApp(t, greet)
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr)
send(t, conn, getOfSize(32<<10))
if status := readStatus(t, conn); status != http.StatusOK {
t.Errorf("status %d, want the app's 200", status)
}
}
func TestARequestLineAndHeadersOver32KiBAreRefusedAndTheConnectionClosed(
t *testing.T,
) {
t.Parallel()
standIn := startWAF(t, allow)
app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked")
})
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
conn := dial(t, addr)
send(t, conn, getOfSize(32<<10+1))
// io.ReadAll returns without an error only once rproxy has closed the
// connection.
text, err := io.ReadAll(conn)
if err != nil || !strings.HasPrefix(string(text), "HTTP/1.1 431 ") {
t.Errorf(
"read %q (%v), want 431 and the connection closed",
text,
err,
)
}
if len(standIn.questions) != 0 {
t.Error("waf was asked")
}
}
func TestABodyAnnouncedOverItsLimitIsRefusedOnceWAFAllowsIt(t *testing.T) {
t.Parallel()
+1 -3
View File
@@ -324,8 +324,7 @@ func greet(w http.ResponseWriter, r *http.Request) {
// startRProxy starts rproxy with waf at backend and the app at appURL, and
// the settings in env on top of the defaults. It returns where rproxy listens
// and what it writes. SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS is a minute unless env
// sets it, so that a busy host cannot make rproxy drop a question or a report.
// and what it writes.
func startRProxy(
t *testing.T, backend, appURL string, env map[string]string,
) (string, *output) {
@@ -358,7 +357,6 @@ func startServerLogging(
settings := map[string]string{
"SWW_RPROXY_WAF_BACKEND": backend, "SWW_RPROXY_UPSTREAM_URL": appURL,
wafBackendTimeout: "60000",
}
maps.Copy(settings, env)
+3 -9
View File
@@ -51,9 +51,7 @@ func runRProxy(ctx context.Context, env map[string]string, out *output) int {
// startRProxy runs rproxy with waf at backend and the app at appURL, and
// returns its URL and what it writes. It believes X-Forwarded-For from the
// tests. It waits a minute for waf's answer, so that a slow answer on a busy
// host is not taken for no answer. It stops as the test ends, and that it
// stopped in order is checked.
// tests. It stops as the test ends, and that it stopped in order is checked.
func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
t.Helper()
@@ -67,7 +65,6 @@ func startRProxy(t *testing.T, backend, appURL string) (string, *output) {
"SWW_RPROXY_UPSTREAM_URL": appURL,
"SWW_RPROXY_TRUSTED_PROXIES": localhost + "/32",
wafBackend: backend,
wafBackendTimeout: "60000",
}, out)
}()
@@ -552,9 +549,7 @@ func TestAStatusWAFDoesNotTakeIsReportedAs599(t *testing.T) {
// startRProxyProcess runs `sww rproxy` in a process of its own, with the
// settings in env, each as NAME=value. It returns where rproxy listens and a
// function that sends rproxy SIGTERM and fails the test unless it exits with
// status 0 within waitLimit. It waits a minute for waf's answers unless env
// sets SWW_RPROXY_WAF_BACKEND_TIMEOUT_MS, so that a busy host cannot make it
// drop a question.
// status 0 within waitLimit.
func startRProxyProcess(t *testing.T, env ...string) (string, func()) {
t.Helper()
@@ -570,8 +565,7 @@ func startRProxyProcess(t *testing.T, env ...string) (string, func()) {
cmd.Env = append(os.Environ(),
commandForMain+"=rproxy",
"SWW_RPROXY_LISTEN_ADDR="+localhost+":0",
wafBackendTimeout+"=60000")
"SWW_RPROXY_LISTEN_ADDR="+localhost+":0")
cmd.Env = append(cmd.Env, env...)
exited := startProcess(t, cmd)
+4 -10
View File
@@ -67,9 +67,6 @@ type Params struct {
Viper *viper.Viper
// Stdout receives the request log and the process's own messages.
Stdout io.Writer
// BesideLookupTimeLimit is the waf.Params.BesideLookupTimeLimit RunWAF
// gives waf. Main leaves it zero, waf's own limit; tests lengthen it.
BesideLookupTimeLimit time.Duration
}
// Main runs the command os.Args names, and returns the process's exit status.
@@ -149,7 +146,7 @@ func RunWAF(ctx context.Context, params Params) int {
return runWAFWithSettings(
ctx,
params,
params.Version,
cfg,
stdout,
remote,
@@ -162,7 +159,7 @@ func RunWAF(ctx context.Context, params Params) int {
// SWW_WAF_LOG_REMOTE_URL is set.
func runWAFWithSettings(
ctx context.Context,
params Params,
version string,
cfg *config.WAF,
stdout io.Writer,
remote *remotelog.Sender,
@@ -196,7 +193,6 @@ func runWAFWithSettings(
ruleFiles,
alertQueue,
crs,
params.BesideLookupTimeLimit,
)
if err != nil {
return cannotStart(processLog, "cannot use the lookup database", err)
@@ -225,7 +221,7 @@ func runWAFWithSettings(
}
processLog.Info("starting",
"version", params.Version,
"version", version,
"address", listener.Addr().String(),
"settings", cfg)
@@ -253,7 +249,7 @@ func cannotStart(processLog *slog.Logger, message string, err error) int {
func newServer(
cfg *config.WAF, stdout io.Writer, processLog *slog.Logger,
now func() time.Time, ruleFiles *rules.Files, alertQueue *alerts.Queue,
crs *coreruleset.CoreRuleSet, besideLookupTimeLimit time.Duration,
crs *coreruleset.CoreRuleSet,
) (*waf.Server, error) {
var lookupFile *lookup.File
@@ -282,8 +278,6 @@ func newServer(
Now: now,
Rules: ruleFiles,
Alerts: alertQueue,
BesideLookupTimeLimit: besideLookupTimeLimit,
})
server.Metrics.AddAlerts(alertQueue)
-3
View File
@@ -173,9 +173,6 @@ func run(ctx context.Context, env map[string]string, out *output) int {
Version: testVersion,
Viper: v,
Stdout: out,
// The race detector and a busy host can slow the Core Rule Set's
// inspections past waf's own limit.
BesideLookupTimeLimit: time.Hour,
})
}
+1 -4
View File
@@ -90,8 +90,6 @@ func TestMainRunsWAF(t *testing.T) {
cmd := exec.CommandContext(t.Context(), binary)
cmd.Stdout = out
// The Core Rule Set is off. Main keeps waf's own limit for its checks,
// and on a busy host the Core Rule Set's inspection can pass it.
cmd.Env = append(os.Environ(),
"TZ="+zone,
commandForMain+"="+wafCommand,
@@ -99,8 +97,7 @@ func TestMainRunsWAF(t *testing.T) {
stateDir+"="+dir,
cacheDir+"="+dir,
ruleFilesDir+"="+t.TempDir(),
lookupSource+"=off",
"SWW_WAF_CORE_RULE_SET_MODE=off")
lookupSource+"=off")
exited := startProcess(t, cmd)
+1 -1
View File
@@ -73,10 +73,10 @@ func newWAFHoldingTheCoreRuleSet(
server, out, queue := newWAFWith(t, func(params waf.Params) *waf.Server {
params.CoreRuleSet = crs
params.BesideLookupTimeLimit = waf.BesideLookupTimeLimit
return waf.New(params)
}, silentGeoJSURL, time.Now, settings)
server.SetBesideLookupTimeLimit(waf.BesideLookupTimeLimit)
return server, out, queue
}
+9
View File
@@ -1,4 +1,13 @@
package waf
import "time"
// BesideLookupTimeLimit is besideLookupTimeLimit.
const BesideLookupTimeLimit = besideLookupTimeLimit
// SetBesideLookupTimeLimit gives the heuristics of besideLookup limit on s in
// place of besideLookupTimeLimit.
func (s *Server) SetBesideLookupTimeLimit(limit time.Duration) {
h := s.Handler.(*handler) //nolint:forcetypeassert // New makes it one
h.besideLookupLimit = limit
}
+2 -12
View File
@@ -4,7 +4,6 @@
package waf
import (
"cmp"
"fmt"
"io"
"log/slog"
@@ -84,11 +83,6 @@ type Params struct {
// failing, a query to a DNSBL zone or a check with AbuseIPDB failing,
// or the day's AbuseIPDB checks used up.
Alerts *alerts.Queue
// BesideLookupTimeLimit is how long the heuristics of besideLookup have
// for a request. Zero means besideLookupTimeLimit. Tests that run the
// Core Rule Set lengthen it, since the race detector and a busy host can
// slow an inspection past besideLookupTimeLimit.
BesideLookupTimeLimit time.Duration
}
// CoreRuleSetInspector inspects a request with the Core Rule Set, as
@@ -174,10 +168,7 @@ func newHandler(params Params) *handler {
coreRuleSet: params.CoreRuleSet,
alerts: params.Alerts,
besideLookupLimit: cmp.Or(
params.BesideLookupTimeLimit,
besideLookupTimeLimit,
),
besideLookupLimit: besideLookupTimeLimit,
}
if params.Config.RouteWatchURL != "" {
h.routeWatch = routewatch.New(routewatch.Params{
@@ -301,8 +292,7 @@ type handler struct {
// waitForGeoJS is true when a setting needs each request's answer from
// GeoJS before the request goes on.
waitForGeoJS bool
// besideLookupLimit is how long the heuristics of besideLookup have for a
// request, as Params.BesideLookupTimeLimit says.
// besideLookupLimit is besideLookupTimeLimit. Only tests change it.
besideLookupLimit time.Duration
// awaitingReport holds the decision on each answered question by its id
// until its report comes, and nil for a question still being handled.
+5 -3
View File
@@ -510,11 +510,13 @@ func newWAFWith(
Now: now,
Rules: ruleFiles,
Alerts: alertQueue,
// The tests of the limit set it back, and hold the Core Rule Set
// with a stand-in.
BesideLookupTimeLimit: time.Hour,
})
// The race detector and a busy host can slow the Core Rule Set's
// inspections past the time limit. The tests of the limit set it back, and
// hold the Core Rule Set with a stand-in.
server.SetBesideLookupTimeLimit(time.Hour)
return server, out, alertQueue
}
-5
View File
@@ -20,13 +20,8 @@ main() {
# the version a build stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
revision="$(git rev-parse HEAD 2>/dev/null || true)"
[ -n "$revision" ] || revision="unknown"
created="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
docker build --no-cache \
--build-arg VERSION="$version" \
--build-arg REVISION="$revision" \
--build-arg CREATED="$created" \
-t "$("$SCRIPT_DIR/projectname")" .
}
-5
View File
@@ -16,13 +16,8 @@ main() {
# the version a build stage derives from the .git in the context.
version="$(git describe --tags --always --dirty 2>/dev/null || true)"
[ -n "$version" ] || version="unknown"
revision="$(git rev-parse HEAD 2>/dev/null || true)"
[ -n "$revision" ] || revision="unknown"
created="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
docker build --no-cache \
--build-arg VERSION="$version" \
--build-arg REVISION="$revision" \
--build-arg CREATED="$created" \
-t "$("$SCRIPT_DIR/projectname")" .
}