Compare commits

..
Author SHA1 Message Date
clawbot eca8f4d8cc waf: the server's types and constructor in waf.go, active_ban in its own file (closes #435)
check / check (push) Failing after 3m45s
Moves only. waf.go now holds the package comment, Params, Server, New,
newHandler, handler and ServeHTTP. The code that answers questions and
takes reports moves to questionsandreports.go, and activeBan to
activeban.go. refusedWhateverTheAnswer moves to lookup.go, beside its
only caller. proxy.go is gone. The only changed lines are imports and
the file comment of bans.go, split between it and activeban.go.

Model: opus-5-5
2026-11-20 01:54:31 +00:00
14 changed files with 45 additions and 572 deletions

No files matched your search

-1
View File
@@ -308,7 +308,6 @@ Every setting the code reads, with its default:
| `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. | | `SWW_RPROXY_MAX_RESPONSE_SIZE` | `1G` | The largest response body, a size or `off`. The visitor gets `502` in place of a larger one, or it is cut off if it has already started. |
| `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. | | `SWW_RPROXY_MAX_REQUEST_TIME_MS` | `60000` | Two time limits, each this long, in milliseconds or `off`: one for the visitor to send its request line and headers, then one for the whole request to reach the app. |
| `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. | | `SWW_RPROXY_MAX_RESPONSE_TIME_MS` | `1800000` | How long the answer may take, from the end of the request to its last byte reaching the visitor, in milliseconds or `off`. |
| `SWW_RPROXY_CONTACT_EMAIL` | unset | The email address the refusal pages link to. |
| `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. | | `SWW_WAF_LISTEN_ADDR` | `:8080` | Where `waf` listens; leave the host part empty. |
| `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `host1/waf`. | | `SWW_WAF_INSTANCE_NAME` | the container's host name | The name in every line `waf` logs, and in every alert and metric, such as `host1/waf`. |
| `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. | | `SWW_WAF_MODE` | `enforce` | `observe` refuses nothing and makes no ban; each log line says what `enforce` would have done. |
+2 -3
View File
@@ -2431,9 +2431,8 @@ but the last below carries `X-Sww-Disposition`.
carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the carries `X-Client-ASN` and `X-Client-Country`, which `rproxy` adds to the
request it passes on. A header whose value is unknown is left out. request it passes on. A header whose value is unknown is left out.
- `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to - `X-Sww-Disposition: deny`, with the status, headers and body the visitor is to
get. `rproxy` gives them to the visitor, less this header; a `403` or `429` get. `rproxy` gives them to the visitor, less this header. The status is one
gets `rproxy`'s own page as its body, linking `SWW_RPROXY_CONTACT_EMAIL` when of:
set. The status is one of:
- `SWW_WAF_BAN_RESPONSE`, for a refusal; - `SWW_WAF_BAN_RESPONSE`, for a refusal;
- `403`, for a `block` rule or the Core Rule Set; - `403`, for a `block` rule or the Core Rule Set;
- `414` or `431`, for a request over the header limits; - `414` or `431`, for a request over the header limits;
-217
View File
@@ -1,217 +0,0 @@
services:
exampleapp:
# Your app, built from the Dockerfile in this directory, listening
# on port 8081. Publish no port of it: it is reached through rproxy.
build: .
rproxy:
image: git.eeqj.de/sneak/smallwebwaf:latest
environment:
SWW_RPROXY_WAF_BACKEND: http://waf:8080
SWW_RPROXY_UPSTREAM_URL: http://exampleapp:8081
# The address the refusal pages link to.
# SWW_RPROXY_CONTACT_EMAIL: ops@example.com
# The netblocks whose X-Forwarded-For rproxy believes. By default
# the private ranges, which cover a TLS-terminating proxy on the
# Docker network. A proxy elsewhere needs its address added; a
# list you give replaces the default, so repeat the private
# ranges if they are still wanted. Set but empty, rproxy believes
# no forwarded header.
# SWW_RPROXY_TRUSTED_PROXIES: 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16,203.0.113.5,2001:db8:1::/64
# By default rproxy passes a request to the app while waf gives
# no answer. Uncomment this to answer 503 instead.
# SWW_RPROXY_WAF_BACKEND_NO_ANSWER_DISPOSITION: deny
# The largest request body (default 100M) and response body
# (default 1G): a size with K, M or G, or `off`. Raise them for
# uploads and downloads larger than that, such as git pushes and
# clones or image layers.
# SWW_RPROXY_MAX_REQUEST_SIZE: 2G
# SWW_RPROXY_MAX_RESPONSE_SIZE: 4G
# How long a request may take to reach the app (default 60000,
# one minute) and how long its answer may take to reach the
# visitor (default 1800000, 30 minutes), in milliseconds or
# `off`. Raise the first for slow uploads, the second for long
# downloads.
# SWW_RPROXY_MAX_REQUEST_TIME_MS: "600000"
# SWW_RPROXY_MAX_RESPONSE_TIME_MS: "7200000"
# traefik terminates TLS and forwards to rproxy:8080 over its own
# network, which the `networks` block at the end of this file names.
# No port is published.
networks:
- default
- traefik
labels:
traefik.enable: "true"
# rproxy is on two networks; traefik reaches it on this one.
traefik.docker.network: traefik
traefik.http.routers.exampleapp.rule: Host(`app.example.com`)
traefik.http.routers.exampleapp.entrypoints: websecure
traefik.http.routers.exampleapp.tls.certresolver: letsencrypt
traefik.http.services.exampleapp.loadbalancer.server.port: "8080"
waf:
image: git.eeqj.de/sneak/smallwebwaf:latest
environment:
SWW_COMMAND: waf
# `observe` refuses nothing and makes no ban; each log line says
# what `enforce` (the default) would have done. Start here to
# see what the defaults would refuse on your site.
# SWW_WAF_MODE: observe
# waf's own messages below `info` (the default), while finding
# out why something is refused.
# SWW_WAF_LOG_LEVEL: debug
# Netblocks whose clients skip every check, such as monitoring.
# SWW_WAF_ALLOW_NETS: 192.0.2.10,2001:db8:2::/48
# Netblocks whose clients are refused.
# SWW_WAF_DENY_NETS: 198.51.100.128/25,2001:db8:3::/48
# Netblocks exempt from rate limiting and the byte limits, such
# as the office and an automation host that calls an API. Their
# requests are still checked for attacks.
# SWW_WAF_RATE_LIMIT_EXEMPT_NETS: 203.0.113.0/24,198.51.100.7
# Refuse every request from a country, by its two-letter code.
# This refuses North Korea, Cuba, Russia and Iran. A client the
# lookup cannot place gets through; see SWW_WAF_LOOKUP_SOURCE.
# SWW_WAF_TAG_DENY: country=KP,country=CU,country=RU,country=IR
# Or from a network, by its AS number: here two hosting networks
# that send only scrapers, M247 (AS9009) and WorldStream
# (AS49981). Countries and networks go in the one list together.
# SWW_WAF_TAG_DENY: asn=9009,asn=49981,country=KP,country=RU
# Or let only these countries through; a client the lookup
# cannot place is refused too.
# SWW_WAF_EXCLUSIVELY_ALLOWED_COUNTRIES: us,de
# Address lists that each tag their clients' requests, as
# name=URL pairs; this one tags Tor exit nodes. A tag is refused
# by adding its name to SWW_WAF_TAG_DENY, such as
# `torexit,country=KP`, or given a share of the limits in
# SWW_WAF_TAG_LIMIT_PERCENT, such as `torexit:25`.
# SWW_WAF_TAG_LISTS: torexit=https://check.torproject.org/torbulkexitlist
# A percentage, from 1 to 99, of every limit for a tag: here
# half for DigitalOcean (AS14061), a quarter for OVH (AS16276)
# and a quarter for China. The lowest of a client's tags applies.
# SWW_WAF_TAG_LIMIT_PERCENT: asn=14061:50,asn=16276:25,country=CN:25
# The most requests a client may make in a minute, an hour and a
# day (defaults 120, 1500 and 6000), a count or `off`; the next
# one bans it. Lower them for a small site, raise them for a
# busy API.
# SWW_WAF_RATE_LIMIT_PER_MINUTE: "60"
# SWW_WAF_RATE_LIMIT_PER_HOUR: "600"
# SWW_WAF_RATE_LIMIT_PER_DAY: "3000"
# Path prefixes exempt from rate limiting and the byte limits,
# such as static assets that one page loads many of.
# SWW_WAF_RATE_LIMIT_EXEMPT_PATHS: /assets/,/static/
# Paths only scanners ask for; one request bans the client. Pick
# paths your app never serves.
# SWW_WAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php,/wp-admin/
# How a refused client is answered: 403 (the default), 429 or
# `close`, which drops the connection.
# SWW_WAF_BAN_RESPONSE: "429"
# The ban for a first broken limit (default 1h) and for a first
# clear sign of attack (default 7d): a duration with m, h or d.
# A repeat within SWW_WAF_LIMIT_BAN_REPEAT_WINDOW (default 24h)
# gets a ban three times as long.
# SWW_WAF_LIMIT_BAN_DURATION: 30m
# SWW_WAF_ATTACK_BAN_DURATION: 30d
# The Core Rule Set, which finds attacks in requests. `block`
# (the default) refuses a match, `detect` only logs and alerts
# it, `off` runs no rule. Use `detect` while finding out what it
# would refuse of your app's ordinary requests.
# SWW_WAF_CORE_RULE_SET_MODE: detect
# Path prefixes the Core Rule Set does not inspect, for a part of
# the app that it keeps matching, such as an editor that posts
# HTML or code.
# SWW_WAF_CORE_RULE_SET_EXEMPT_PATHS: /api/editor/,/admin/
# The apps each hostname serves, as hostname:app pairs separated
# by semicolons, so that each hostname gets its own apps' fixes
# to the attack detection and no other's. "Attack detection" in
# the README lists the apps.
# SWW_WAF_APPS: git.example.com:gitea;blog.example.com:wordpress
# Blocklists of addresses and netblocks, text files fetched once
# a day, whose clients are refused.
# SWW_WAF_BLOCKLIST_URLS: https://www.spamhaus.org/drop/drop.txt,https://www.spamhaus.org/drop/dropv6.txt
# With the key of an AbuseIPDB account, a client that has
# committed an offence is checked there in the background; one
# scored 75 or more (SWW_WAF_ABUSEIPDB_MIN_SCORE) gets what
# SWW_WAF_DNSBL_ABUSEIPDB_ACTION says: a quarter of every limit
# by default, or `deny`. At most 900 checks a day
# (SWW_WAF_ABUSEIPDB_DAILY_BUDGET), under the free plan's 1000.
# SWW_WAF_ABUSEIPDB_KEY: api-key-.....
# SWW_WAF_DNSBL_ABUSEIPDB_ACTION: deny
# By default waf looks each client's address up at GeoJS, a web
# service, for its country and AS number. Set `off`, it makes no
# external requests for lookups. Every setting at its default
# works without them; one that needs a lookup, such as a country
# or ASN setting above, then stops the start. Set `file`, it
# reads an IPinfo Lite database instead, mounted by its
# directory as under `volumes` below, so that a replaced file is
# seen.
# SWW_WAF_LOOKUP_SOURCE: "off"
# SWW_WAF_LOOKUP_SOURCE: file
# SWW_WAF_LOOKUP_DB_PATH: /var/lib/ipinfo/ipinfo_lite.mmdb
# Alerts about bans, attacks and failures, to an ntfy topic, a
# Slack incoming webhook or any webhook. The ntfy token is the
# access token ntfy issued for the topic.
# SWW_WAF_ALERT_NTFY_URL: https://ntfy.sh/my-alerts
# SWW_WAF_ALERT_NTFY_TOKEN: hunter2
# SWW_WAF_ALERT_SLACK_WEBHOOK_URL: https://hooks.slack.com/services/...
# Bearer tokens you make up, at least 32 characters each. The
# admin token switches on the ban and client endpoints, through
# which a ban is lifted, your own included, as
# docs/admin-endpoints.md describes. The metrics token switches
# on /_sww/metrics.
# SWW_WAF_ADMIN_TOKEN: hunter2
# SWW_WAF_METRICS_TOKEN: hunter2
# Any setting can be read from a file instead, by adding _FILE
# to its name: the file's contents are the value, less one
# newline at the end. That keeps a secret out of this file and
# out of `docker inspect`. Mount the files as under `volumes`
# below, and set each setting in one form only.
# SWW_WAF_ALERT_NTFY_TOKEN_FILE: /etc/sww/tokens/ntfy
# SWW_WAF_ADMIN_TOKEN_FILE: /etc/sww/tokens/admin
# SWW_WAF_METRICS_TOKEN_FILE: /etc/sww/tokens/metrics
# SWW_WAF_ABUSEIPDB_KEY_FILE: /etc/sww/tokens/abuseipdb
volumes:
# waf's state files: its clients and bans, kept across restarts.
- waf-state:/var/lib/sww
# Token files, each owned by uid 65532 (the sww user) with mode
# 0400, for the _FILE settings above.
# - /srv/sww/tokens:/etc/sww/tokens:ro
# The IPinfo Lite database, for SWW_WAF_LOOKUP_SOURCE=file.
# - /srv/ipinfo:/var/lib/ipinfo:ro
volumes:
waf-state:
# traefik's network, made outside this file.
networks:
traefik:
external: true
-2
View File
@@ -483,8 +483,6 @@ start, and an invalid value stops the start. Any can be given as a file, as a
answer `rproxy` gives itself always has at least 10 seconds to be sent, and so answer `rproxy` gives itself always has at least 10 seconds to be sent, and so
do the status and headers of the app's answer. A time in milliseconds or do the status and headers of the app's answer. A time in milliseconds or
`off`. `off`.
- `SWW_RPROXY_CONTACT_EMAIL` (default unset): the email address the refusal
pages link to.
- `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as - `SWW_RPROXY_LISTEN_ADDR` (default `:8080`): where `rproxy` listens, such as
`:9000`. `:9000`.
- `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An - `SWW_RPROXY_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app. An
-4
View File
@@ -46,9 +46,6 @@ type RProxy struct {
// (SWW_RPROXY_MAX_RESPONSE_TIME_MS). // (SWW_RPROXY_MAX_RESPONSE_TIME_MS).
MaxRequestTime time.Duration MaxRequestTime time.Duration
MaxResponseTime time.Duration MaxResponseTime time.Duration
// ContactEmail is the address the pages a refused visitor gets link to,
// or "" for no link (SWW_RPROXY_CONTACT_EMAIL). It is taken as given.
ContactEmail string
// settings are the values read, as given or by default, for the log line // settings are the values read, as given or by default, for the log line
// at start. // at start.
@@ -111,7 +108,6 @@ func RProxyFromEnvironment(v *viper.Viper) (*RProxy, error) {
"SWW_RPROXY_MAX_RESPONSE_TIME_MS", "SWW_RPROXY_MAX_RESPONSE_TIME_MS",
"1800000", "1800000",
), ),
ContactEmail: env.value("SWW_RPROXY_CONTACT_EMAIL", ""),
} }
if env.err != nil { if env.err != nil {
-2
View File
@@ -25,7 +25,6 @@ const (
rproxyMaxResponse = "SWW_RPROXY_MAX_RESPONSE_SIZE" rproxyMaxResponse = "SWW_RPROXY_MAX_RESPONSE_SIZE"
rproxyRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS" rproxyRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
rproxyAnswerTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS" rproxyAnswerTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
rproxyContactEmail = "SWW_RPROXY_CONTACT_EMAIL"
backendURL = "http://waf:8080" backendURL = "http://waf:8080"
rproxyNoAnswerAddr = ":8080" rproxyNoAnswerAddr = ":8080"
rproxyNoAnswerApp = "http://127.0.0.1:8081" rproxyNoAnswerApp = "http://127.0.0.1:8081"
@@ -251,7 +250,6 @@ func TestRProxySettingsAreLoggedByName(t *testing.T) {
rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, wafBackendTimeout, rproxyListen, rproxyUpstream, rproxyTrusted, wafBackend, wafBackendTimeout,
rproxyNoAnswer, rproxyNoAnswer,
rproxyMaxRequest, rproxyMaxResponse, rproxyRequestTime, rproxyAnswerTime, rproxyMaxRequest, rproxyMaxResponse, rproxyRequestTime, rproxyAnswerTime,
rproxyContactEmail,
} { } {
if !slices.Contains(names, name) { if !slices.Contains(names, name) {
t.Errorf("%s is not logged, only %v", name, names) t.Errorf("%s is not logged, only %v", name, names)
-71
View File
@@ -1,71 +0,0 @@
package rproxy
import (
"embed"
"html"
"net/http"
"strings"
)
// pageFiles are the pages a refused visitor gets, as HTML with nothing
// external: refused.html for a 403, ratelimited.html for a 429.
//
//go:embed refused.html ratelimited.html
var pageFiles embed.FS
// contactSentence is the sentence of each page that becomes a link to
// SWW_RPROXY_CONTACT_EMAIL when that is set.
const contactSentence = "contact the site's operator"
// htmlContentType is the Content-Type of the pages.
const htmlContentType = "text/html; charset=utf-8"
// blockPages are the pages a refused visitor gets, ready to send, made once
// at start.
type blockPages struct {
refused []byte
rateLimited []byte
}
// newBlockPages makes the pages. With contactEmail set, each page's
// contactSentence is a mailto link to it.
func newBlockPages(contactEmail string) blockPages {
return blockPages{
refused: readPage("refused.html", contactEmail),
rateLimited: readPage("ratelimited.html", contactEmail),
}
}
// readPage reads the page in name from pageFiles, linking its
// contactSentence to contactEmail unless that is "". The files are part of
// the binary, so a missing one is a bug, and a panic.
func readPage(name, contactEmail string) []byte {
contents, err := pageFiles.ReadFile(name)
if err != nil {
panic(err)
}
if contactEmail == "" {
return contents
}
link := `<a href="mailto:` + html.EscapeString(contactEmail) + `">` +
contactSentence + `</a>`
return []byte(
strings.Replace(string(contents), contactSentence, link, 1),
)
}
// forStatus returns the page for a refusal with status, and whether there is
// one: the rate-limited page for a 429, the refused page for a 403.
func (p blockPages) forStatus(status int) ([]byte, bool) {
switch status {
case http.StatusForbidden:
return p.refused, true
case http.StatusTooManyRequests:
return p.rateLimited, true
default:
return nil, false
}
}
-27
View File
@@ -1,27 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex">
<title>Too many requests</title>
<style>
:root{color-scheme:light dark;--fg:#1a1a1a;--muted:#5c5c5c;--bg:#f6f6f4}
@media(prefers-color-scheme:dark){:root{--fg:#ebebe8;--muted:#9a9a96;--bg:#161616}}
html{background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif}
body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;padding:1.5rem}
main{max-width:34rem;width:100%}
p.code{margin:0 0 .5rem;color:var(--muted);font-size:.8rem;letter-spacing:.08em}
h1{margin:0 0 1rem;font-size:1.6rem;font-weight:600;line-height:1.25}
p{margin:0 0 1rem}
</style>
</head>
<body>
<main>
<p class="code">429</p>
<h1>Too many requests</h1>
<p>Your address has sent more requests than this site allows in a short time. Wait a while, then try again.</p>
<p>If you think this is a mistake, contact the site's operator.</p>
</main>
</body>
</html>
-27
View File
@@ -1,27 +0,0 @@
<!doctype html>
<html lang="en">
<head>
<meta charset="utf-8">
<meta name="viewport" content="width=device-width, initial-scale=1">
<meta name="robots" content="noindex">
<title>Request refused</title>
<style>
:root{color-scheme:light dark;--fg:#1a1a1a;--muted:#5c5c5c;--bg:#f6f6f4}
@media(prefers-color-scheme:dark){:root{--fg:#ebebe8;--muted:#9a9a96;--bg:#161616}}
html{background:var(--bg);color:var(--fg);font:16px/1.5 system-ui,-apple-system,"Segoe UI",Roboto,sans-serif}
body{margin:0;min-height:100vh;display:flex;align-items:center;justify-content:center;padding:1.5rem}
main{max-width:34rem;width:100%}
p.code{margin:0 0 .5rem;color:var(--muted);font-size:.8rem;letter-spacing:.08em}
h1{margin:0 0 1rem;font-size:1.6rem;font-weight:600;line-height:1.25}
p{margin:0 0 1rem}
</style>
</head>
<body>
<main>
<p class="code">403</p>
<h1>Request refused</h1>
<p>This site did not accept your request.</p>
<p>If you think this is a mistake, contact the site's operator.</p>
</main>
</body>
</html>
-3
View File
@@ -92,7 +92,6 @@ func New(params Params) *Server {
Path: wafprotocol.ReportPath, Path: wafprotocol.ReportPath,
}).String(), }).String(),
breakerClock: time.Now, breakerClock: time.Now,
pages: newBlockPages(params.Config.ContactEmail),
} }
s := &Server{h: h} s := &Server{h: h}
@@ -157,8 +156,6 @@ type handler struct {
breakerClock func() time.Time breakerClock func() time.Time
// reports are the goroutines sending reports to waf. // reports are the goroutines sending reports to waf.
reports reportGoroutines reports reportGoroutines
// pages are the pages a refused visitor gets.
pages blockPages
} }
// newTransport returns what carries requests to the app. It never goes through // newTransport returns what carries requests to the app. It never goes through
+30 -181
View File
@@ -11,7 +11,6 @@ import (
"net" "net"
"net/http" "net/http"
"net/http/httptest" "net/http/httptest"
"os"
"slices" "slices"
"strconv" "strconv"
"strings" "strings"
@@ -53,56 +52,8 @@ const (
maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE" maxResponseSize = "SWW_RPROXY_MAX_RESPONSE_SIZE"
maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS" maxRequestTime = "SWW_RPROXY_MAX_REQUEST_TIME_MS"
maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS" maxResponseTime = "SWW_RPROXY_MAX_RESPONSE_TIME_MS"
contactEmail = "SWW_RPROXY_CONTACT_EMAIL"
) )
// The pages rproxy gives a refused visitor in place of waf's body.
const (
// refusedPageFile and rateLimitedPageFile are the pages' files, beside
// the code.
refusedPageFile = "refused.html"
rateLimitedPageFile = "ratelimited.html"
// contactSentence is the sentence of each page that becomes a link to
// SWW_RPROXY_CONTACT_EMAIL.
contactSentence = "contact the site's operator"
htmlContentType = "text/html; charset=utf-8"
)
// page is the page in file, as rproxy serves it without a contact email.
func page(t *testing.T, file string) string {
t.Helper()
//nolint:gosec // a page beside the code
contents, err := os.ReadFile(file)
if err != nil {
t.Fatalf("read the page: %v", err)
}
return string(contents)
}
// wantPage checks that got is status with the page in file as its body, and
// the headers of a page.
func wantPage(t *testing.T, got answered, status int, file string) {
t.Helper()
if got.status != status || got.body != page(t, file) {
t.Errorf(
"answered %d %q, want %d and %s",
got.status,
got.body,
status,
file,
)
}
if got.header.Get("Content-Type") != htmlContentType ||
got.header.Get("X-Content-Type-Options") != "nosniff" ||
got.header.Get("Cache-Control") != "no-store" {
t.Errorf("the page's headers are %v", got.header)
}
}
// The headers the tests set and look at. // The headers the tests set and look at.
const ( const (
dispositionHeader = wafprotocol.DispositionHeader dispositionHeader = wafprotocol.DispositionHeader
@@ -355,20 +306,6 @@ func deny(w http.ResponseWriter, _ *http.Request) {
_, _ = io.WriteString(w, deniedBody) _, _ = io.WriteString(w, deniedBody)
} }
// answerAs answers a question with status and body, with disposition unless
// it is "", and a header of its own.
func answerAs(disposition string, status int, body string) http.HandlerFunc {
return func(w http.ResponseWriter, _ *http.Request) {
if disposition != "" {
w.Header().Set(dispositionHeader, disposition)
}
w.Header().Set(fromWAFHeader, addedHeaderValue)
w.WriteHeader(status)
_, _ = io.WriteString(w, body)
}
}
// startApp starts app as the app rproxy passes requests to. // startApp starts app as the app rproxy passes requests to.
func startApp(t *testing.T, app http.HandlerFunc) *httptest.Server { func startApp(t *testing.T, app http.HandlerFunc) *httptest.Server {
t.Helper() t.Helper()
@@ -830,7 +767,7 @@ func TestRemovesTheVisitorsLookupHeadersWhenWAFGivesNone(t *testing.T) {
} }
} }
func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) { func TestDenyGivesTheVisitorWAFsAnswer(t *testing.T) {
t.Parallel() t.Parallel()
standIn := startWAF(t, deny) standIn := startWAF(t, deny)
@@ -840,10 +777,9 @@ func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
addr, out := startRProxy(t, standIn.URL, app.URL, nil) addr, out := startRProxy(t, standIn.URL, app.URL, nil)
got := get(t, addr, "/") got := get(t, addr, "/")
wantPage(t, got, http.StatusForbidden, refusedPageFile) if got.status != http.StatusForbidden || got.body != deniedBody ||
got.header.Get(fromWAFHeader) != addedHeaderValue {
if got.header.Get(fromWAFHeader) != addedHeaderValue { t.Errorf("answered %+v, want waf's answer", got)
t.Errorf("answered %+v, want waf's headers", got)
} }
if len(got.header.Values(dispositionHeader)) != 0 { if len(got.header.Values(dispositionHeader)) != 0 {
@@ -852,7 +788,7 @@ func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
report := standIn.report(t) report := standIn.report(t)
if report.Status != http.StatusForbidden || if report.Status != http.StatusForbidden ||
report.ResponseBytes != int64(len(page(t, refusedPageFile))) || report.ResponseBytes != int64(len(deniedBody)) ||
report.DurationUpstreamTotalMS != 0 { report.DurationUpstreamTotalMS != 0 {
t.Errorf("report %+v, of a request denied", report) t.Errorf("report %+v, of a request denied", report)
} }
@@ -864,7 +800,7 @@ func TestDenyGivesTheVisitorTheRefusedPageWithWAFsHeaders(t *testing.T) {
} }
} }
func TestADeniedHeadGivesThePagesContentLength(t *testing.T) { func TestADeniedHeadKeepsWAFsContentLength(t *testing.T) {
t.Parallel() t.Parallel()
standIn := startWAF(t, deny) standIn := startWAF(t, deny)
@@ -872,13 +808,13 @@ func TestADeniedHeadGivesThePagesContentLength(t *testing.T) {
addr, _ := startRProxy(t, standIn.URL, app.URL, nil) addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody)) got := visit(t, newRequest(t, http.MethodHead, addr, "/", http.NoBody))
if got.status != http.StatusForbidden || got.body != "" || if got.status != http.StatusForbidden ||
got.header.Get( got.header.Get(
contentLengthHeader, contentLengthHeader,
) != strconv.Itoa( ) != strconv.Itoa(
len(page(t, refusedPageFile)), len(deniedBody),
) { ) {
t.Errorf("answered %+v, want waf's 403 and the page's length", got) t.Errorf("answered %+v, want waf's 403 and its length", got)
} }
} }
@@ -887,13 +823,11 @@ func TestAnAnswerRProxyHoldsWholeIsSentBeforeItsLogLine(t *testing.T) {
const host = " HTTP/1.1\r\nHost: site.example\r\n\r\n" const host = " HTTP/1.1\r\nHost: site.example\r\n\r\n"
refusedPage := page(t, refusedPageFile)
for _, tc := range []struct { for _, tc := range []struct {
name, request, body string name, request, body string
status int status int
}{ }{
{"a deny", "GET /" + host, refusedPage, http.StatusForbidden}, {"a deny", "GET /" + host, deniedBody, http.StatusForbidden},
{"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest}, {"rproxy's own answer", "GET x:admin" + host, "Bad Request\n", http.StatusBadRequest},
{"the health check", "GET " + wafprotocol.HealthPath + host, "ok\n", http.StatusOK}, {"the health check", "GET " + wafprotocol.HealthPath + host, "ok\n", http.StatusOK},
} { } {
@@ -965,83 +899,6 @@ func TestRProxysOwnAnswerIsPlainText(t *testing.T) {
} }
} }
func TestADenyOf429GivesTheVisitorTheRateLimitedPage(t *testing.T) {
t.Parallel()
standIn := startWAF(t, func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(dispositionHeader, dispositionDeny)
w.Header().Set("Content-Type", "text/plain; charset=utf-8")
w.WriteHeader(http.StatusTooManyRequests)
_, _ = io.WriteString(w, deniedBody)
})
app := startApp(t, greet)
addr, _ := startRProxy(t, standIn.URL, app.URL, nil)
wantPage(
t,
get(t, addr, "/"),
http.StatusTooManyRequests,
rateLimitedPageFile,
)
}
func TestThePagesLinkTheContactEmail(t *testing.T) {
t.Parallel()
const email = "ops@example.com"
link := `<a href="mailto:` + email + `">` + contactSentence + `</a>`
for _, tc := range []struct {
file string
status int
}{
{refusedPageFile, http.StatusForbidden},
{rateLimitedPageFile, http.StatusTooManyRequests},
} {
t.Run(tc.file, func(t *testing.T) {
t.Parallel()
standIn := startWAF(
t,
func(w http.ResponseWriter, _ *http.Request) {
w.Header().Set(dispositionHeader, dispositionDeny)
w.WriteHeader(tc.status)
},
)
app := startApp(t, greet)
addr, _ := startRProxy(
t,
standIn.URL,
app.URL,
map[string]string{contactEmail: email},
)
got := get(t, addr, "/")
want := strings.Replace(
page(t, tc.file),
contactSentence,
link,
1,
)
if got.status != tc.status || got.body != want {
t.Errorf(
"answered %d %q, want %d %q",
got.status,
got.body,
tc.status,
want,
)
}
if !strings.Contains(page(t, tc.file), contactSentence) {
t.Errorf("%s has no %q to link", tc.file, contactSentence)
}
})
}
}
func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) { func TestCloseClosesTheConnectionWithoutAnAnswer(t *testing.T) {
t.Parallel() t.Parallel()
@@ -1160,28 +1017,17 @@ func noAnswers() map[string]func(t *testing.T) string {
func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) { func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
t.Parallel() t.Parallel()
const unknownBody = "who knows\n"
refusedPage := page(t, refusedPageFile)
rateLimitedPage := page(t, rateLimitedPageFile)
// want is the body the visitor gets: waf's, but for a refusal, which gets
// rproxy's page for its status.
for _, tc := range []struct { for _, tc := range []struct {
name, disposition, body, want string name, disposition, body string
status int status int
}{ }{
{"a refusal", dispositionDeny, deniedBody, refusedPage, http.StatusForbidden}, {"a refusal", dispositionDeny, deniedBody, http.StatusForbidden},
{ {"a refusal, 429", dispositionDeny, deniedBody, http.StatusTooManyRequests},
"a refusal, 429", dispositionDeny, deniedBody, rateLimitedPage, {"waf's 400", "", "refused\n", http.StatusBadRequest},
http.StatusTooManyRequests, {"a 500", "", "broken\n", http.StatusInternalServerError},
}, {"a 503 denied", dispositionDeny, "busy\n", http.StatusServiceUnavailable},
{"waf's 400", "", "refused\n", "refused\n", http.StatusBadRequest}, {"no header", "", "who knows\n", http.StatusOK},
{"a 500", "", "broken\n", "broken\n", http.StatusInternalServerError}, {"an unknown disposition", "maybe", "who knows\n", http.StatusOK},
{"a 503 denied", dispositionDeny, "busy\n", "busy\n", http.StatusServiceUnavailable},
{"a 403 without a disposition", "", "nope\n", "nope\n", http.StatusForbidden},
{"no header", "", unknownBody, unknownBody, http.StatusOK},
{"an unknown disposition", "maybe", unknownBody, unknownBody, http.StatusOK},
} { } {
for _, setting := range []string{dispositionAllow, dispositionDeny} { for _, setting := range []string{dispositionAllow, dispositionDeny} {
subtest := tc.name + " with " + noAnswerSetting + "=" + setting subtest := tc.name + " with " + noAnswerSetting + "=" + setting
@@ -1190,7 +1036,15 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
standIn := startWAF( standIn := startWAF(
t, t,
answerAs(tc.disposition, tc.status, tc.body), func(w http.ResponseWriter, _ *http.Request) {
if tc.disposition != "" {
w.Header().Set(dispositionHeader, tc.disposition)
}
w.Header().Set(fromWAFHeader, addedHeaderValue)
w.WriteHeader(tc.status)
_, _ = io.WriteString(w, tc.body)
},
) )
app := startApp(t, func(http.ResponseWriter, *http.Request) { app := startApp(t, func(http.ResponseWriter, *http.Request) {
t.Error("the app was asked") t.Error("the app was asked")
@@ -1205,14 +1059,9 @@ func TestAnAnswerOtherThanAllowGoesToTheVisitorAsWAFGaveIt(t *testing.T) {
) )
got := get(t, addr, "/") got := get(t, addr, "/")
if got.status != tc.status || got.body != tc.want || if got.status != tc.status || got.body != tc.body ||
got.header.Get(fromWAFHeader) != addedHeaderValue { got.header.Get(fromWAFHeader) != addedHeaderValue {
t.Errorf( t.Errorf("answered %+v, want waf's answer", got)
"answered %+v, want %d %q",
got,
tc.status,
tc.want,
)
} }
if len(got.header.Values(dispositionHeader)) != 0 { if len(got.header.Values(dispositionHeader)) != 0 {
+1 -24
View File
@@ -7,7 +7,6 @@ import (
"io" "io"
"maps" "maps"
"net/http" "net/http"
"strconv"
"strings" "strings"
"sync" "sync"
@@ -188,9 +187,7 @@ func canBeAsked(r *http.Request) bool {
len(r.Header.Values("User-Agent")) <= 1 len(r.Header.Values("User-Agent")) <= 1
} }
// answerWith gives the visitor waf's answer, less its disposition. A deny of // answerWith gives the visitor waf's answer, less its disposition.
// 403 or 429 gets rproxy's page for that status as its body, in place of
// waf's.
func (rq *request) answerWith(answer *wafAnswer) { func (rq *request) answerWith(answer *wafAnswer) {
rq.prepareOwnAnswer(rq.limits.requestDeadline()) rq.prepareOwnAnswer(rq.limits.requestDeadline())
@@ -198,15 +195,6 @@ func (rq *request) answerWith(answer *wafAnswer) {
maps.Copy(header, answer.header) maps.Copy(header, answer.header)
header.Del(wafprotocol.DispositionHeader) header.Del(wafprotocol.DispositionHeader)
if rq.disposition == wafprotocol.DispositionDeny {
page, found := rq.h.pages.forStatus(answer.status)
if found {
rq.sendPage(answer.status, page)
return
}
}
// Go's server adds a Content-Type it guesses from the body to a response whose // Go's server adds a Content-Type it guesses from the body to a response whose
// headers have no Content-Type key. The key with no value sends none. // headers have no Content-Type key. The key with no value sends none.
if header.Get("Content-Type") == "" { if header.Get("Content-Type") == "" {
@@ -216,17 +204,6 @@ func (rq *request) answerWith(answer *wafAnswer) {
rq.send(answer.status, answer.body) rq.send(answer.status, answer.body)
} }
// sendPage sends page as the body of a refusal with status. The length is set
// here too, since send leaves a HEAD's as waf gave it, which is of waf's body.
func (rq *request) sendPage(status int, page []byte) {
header := rq.out.Header()
header.Set("Content-Type", htmlContentType)
header.Set("X-Content-Type-Options", "nosniff")
header.Set("Cache-Control", "no-store")
header.Set("Content-Length", strconv.Itoa(len(page)))
rq.send(status, page)
}
// sendReport sends waf report on a goroutine of its own, so that the // sendReport sends waf report on a goroutine of its own, so that the
// visitor does not wait for it. The report goes on after ctx, the request's own // visitor does not wait for it. The report goes on after ctx, the request's own
// context, has ended. // context, has ended.
+3 -8
View File
@@ -8,8 +8,6 @@ set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)" ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
set -x
main() { main() {
cd "$ROOT" cd "$ROOT"
# Own line: a failing command substitution inside an argument does # Own line: a failing command substitution inside an argument does
@@ -20,15 +18,12 @@ main() {
[ -n "$version" ] || version="unknown" [ -n "$version" ] || version="unknown"
revision="$(git rev-parse HEAD 2>/dev/null || true)" revision="$(git rev-parse HEAD 2>/dev/null || true)"
[ -n "$revision" ] || revision="unknown" [ -n "$revision" ] || revision="unknown"
created="$(date -u +%Y-%m-%d)" created="$(date -u +%Y-%m-%dT%H:%M:%SZ)"
docker build \ docker build --no-cache \
--build-arg VERSION="$version" \ --build-arg VERSION="$version" \
--build-arg REVISION="$revision" \ --build-arg REVISION="$revision" \
--build-arg CREATED="$created" \ --build-arg CREATED="$created" \
-t "git.eeqj.de/sneak/$("$SCRIPT_DIR/projectname")" \ -t "$("$SCRIPT_DIR/projectname")" .
-t "git.eeqj.de/sneak/$("$SCRIPT_DIR/projectname"):$version" \
-t "git.eeqj.de/sneak/$("$SCRIPT_DIR/projectname"):$(date -u +%Y-%m-%d)" \
.
} }
main "$@" main "$@"
+9 -2
View File
@@ -14,9 +14,16 @@ set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)" ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
PUBLISH_IMAGE="git.eeqj.de/sneak/smallwebwaf"
main() { main() {
if [ -z "${DOCKER_HOST:-}" ]; then
echo "script/publish: DOCKER_HOST is not set; set it to the Docker host that builds the image, such as ssh://user@build.example.com" >&2
exit 1
fi
if [ -z "${PUBLISH_IMAGE:-}" ]; then
echo "script/publish: PUBLISH_IMAGE is not set; set it to the image name in the registry, such as registry.example.com/smallwebwaf" >&2
exit 1
fi
cd "$ROOT" cd "$ROOT"
# Own line: a failing command substitution inside an argument does # Own line: a failing command substitution inside an argument does