Rule files, and bans for a clear sign of attack (closes #24)

Every *.rules file in SWWAF_RULES_DIR not named with a leading dot is
read at start, and again 2 seconds after the directory's last change.
Each request is checked against the rules after the rate limits: log
notes a match, block refuses with 403, ban refuses and bans the netblock
for SWWAF_ATTACK_BAN_DURATION, made permanent by its next request or
attack. path, query and uri are matched as the request line sent them;
header:Host and header:Transfer-Encoding are refused. Bans gain a cause.
The image ships 00-default.rules.

Judgement call: a header sent twice is matched with its values joined
by ", ".
Judgement call: SWWAF_MAX_BAN_DURATION does not cap a ban for an attack.
Not in this unit: offences for rule matches, with the error burst.

Model: opus-5-5

README.md

@@ -493,8 +493,10 @@ each request against their rules in that order, as "Rule files" in
 [`SPEC.md`](SPEC.md) describes. A file whose name starts with `.`, such as an
 editor's lock file `.#50-app.rules`, is not a rule file, as a shell's `*.rules`
 would not match it. A rule is a line of four fields separated by spaces or tabs:
-an id, a target, an action and a regex, which runs to the end of the line. Blank
-lines and lines that start with `#` are ignored.
+an id, a target, an action and a regex, which runs to the end of the line.
+Spaces and tabs at the end of a line are not part of its regex, so a line with
+only those after its action has no regex, and is not a rule. Blank lines and
+lines that start with `#` are ignored.
 
 ```
 # id           target      action  regex
@@ -520,18 +522,20 @@ scanner-agent  user_agent  ban     (?i)\b(sqlmap|nikto|nuclei|wpscan)\b
   and takes time linear in the text it reads. It matches anywhere in the target
   unless anchored with `^` and `$`; `(?i)` at its front makes it ignore case.
 
-A line that is not a rule, a rule for the `Host` or the `Transfer-Encoding`
-header, a regex that does not compile or an id used twice stops the start with a
-message naming the file and the line, and so does a `SWWAF_RULES_DIR` that does
-not exist. An empty directory is no error, and the log says that it holds no
-rules. While it runs, `smallwebwaf` watches the directory, and reads the rule
-files again once the directory has had no change for 2 seconds after one is
-edited, added or removed, so that a file saved in place, appended to or copied
-in with `scp` is read only once whole, unless its writing stops for longer. It
-also reads them 2 seconds after it starts watching, so that an edit saved while
-it started is not missed. If they then hold one of those errors, the rules stay
-as they were, the earlier version of the edited file included, the log names the
-file and the line, and the files are read again after the next change.
+A line that is not a rule, a `header:` name with a character no header name can
+have, such as `header:User-Agent:`, a rule for the `Host` or the
+`Transfer-Encoding` header, a regex that does not compile or an id used twice
+stops the start with a message naming the file and the line, and so does a
+`SWWAF_RULES_DIR` that does not exist. An empty directory is no error, and the
+log says that it holds no rules. While it runs, `smallwebwaf` watches the
+directory, and reads the rule files again once the directory has had no change
+for 2 seconds after one is edited, added or removed, so that a file saved in
+place, appended to or copied in with `scp` is read only once whole, unless its
+writing stops for longer. It also reads them 2 seconds after it starts watching,
+so that an edit saved while it started is not missed. If they then hold one of
+those errors, the rules stay as they were, the earlier version of the edited
+file included, the log names the file and the line, and the files are read again
+after the next change.
 
 The image ships one rule file, `share/rules.d/00-default.rules` here: rules that
 ban probes no real visitor sends, for secrets, version control directories,

internal/config/config.go

@@ -752,6 +752,22 @@ func parseCountries(value string) ([]string, error) {
 const headerNameChars = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz" +
 	"0123456789!#$%&'*+-.^_`|~"
 
+// IsHeaderName reports whether name can be a header name: one or more of
+// the characters RFC 9110 allows in one.
+func IsHeaderName(name string) bool {
+	if name == "" {
+		return false
+	}
+
+	for _, char := range name {
+		if !strings.ContainsRune(headerNameChars, char) {
+			return false
+		}
+	}
+
+	return true
+}
+
 // parseHeaderNames reads a comma-separated list of header names in either
 // case, and returns them in lower case. Host and Transfer-Encoding are
 // refused: Go's HTTP server takes them out of the request's headers.
@@ -764,10 +780,8 @@ func parseHeaderNames(value string) ([]string, error) {
 	headers := make([]string, 0, len(items))
 
 	for _, item := range items {
-		for _, char := range item {
-			if !strings.ContainsRune(headerNameChars, char) {
-				return nil, fmt.Errorf("%q %w", item, errNotHeaderName)
-			}
+		if !IsHeaderName(item) {
+			return nil, fmt.Errorf("%q %w", item, errNotHeaderName)
 		}
 
 		header := strings.ToLower(item)

internal/rules/rules.go

@@ -21,6 +21,8 @@ import (
 	"time"
 
 	"github.com/fsnotify/fsnotify"
+
+	"sneak.berlin/go/smallwebwaf/internal/config"
 )
 
 // The actions a rule takes when it matches.
@@ -65,6 +67,8 @@ var (
 	errNotID     = errors.New("is not an id of letters, digits, - and _")
 	errNotTarget = errors.New(
 		"is not path, query, uri, method, host, user_agent, referer or header:<Name>")
+	errNotHeaderName = errors.New(
+		"has a character after header: that no header name can have")
 	errHeaderTakenOut = errors.New(
 		"names a header that Go's HTTP server takes out of every request, " +
 			"so a rule never sees it")
@@ -109,7 +113,8 @@ type Files struct {
 
 // Load reads the rules of every *.rules file in Dir, in the order of the
 // files' names, unless Enabled is false. A Dir that cannot be read is an
-// error, and so is a line that is not a rule, a rule for the Host or the
+// error, and so is a line that is not a rule, a header name with a
+// character no header name can have, a rule for the Host or the
 // Transfer-Encoding header, which Go's HTTP server takes out of every
 // request, a regex that does not compile and an id used twice, each named
 // with its file and line.
@@ -308,9 +313,11 @@ func readFile(path string, rules []Rule, places map[string]string) ([]Rule, erro
 }
 
 // parse reads a line of a rule file. It returns false for a blank line
-// and for a comment, a line that starts with #.
+// and for a comment, a line that starts with #. Spaces and tabs at the
+// end of the line are not part of its regex, so a line with only those
+// after its action has no regex, and is not a rule.
 func parse(line string) (Rule, bool, error) {
-	line = strings.TrimLeft(line, " \t")
+	line = strings.Trim(line, " \t")
 	if line == "" || strings.HasPrefix(line, "#") {
 		return Rule{}, false, nil
 	}
@@ -321,12 +328,15 @@ func parse(line string) (Rule, bool, error) {
 	}
 
 	rule := Rule{ID: fields[1], Target: fields[2], Action: fields[3]}
+	headerName, isHeader := strings.CutPrefix(rule.Target, headerTarget)
 
 	switch {
 	case !idChars.MatchString(rule.ID):
 		return Rule{}, false, fmt.Errorf("the id %q %w", rule.ID, errNotID)
 	case !isTarget(rule.Target):
 		return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errNotTarget)
+	case isHeader && !config.IsHeaderName(headerName):
+		return Rule{}, false, fmt.Errorf("the target %q %w", rule.Target, errNotHeaderName)
 	case strings.EqualFold(rule.Target, headerTarget+"Host"):
 		return Rule{}, false, fmt.Errorf(
 			"the target %q %w; the request's host is the target host",

internal/rules/rules_test.go

@@ -145,6 +145,14 @@ after          path  log    ^/
 	wantMatched(t, files, get(t, "/"), "ban")
 }
 
+func TestSpacesAndTabsEndingALineAreNotPartOfItsRegex(t *testing.T) {
+	t.Parallel()
+
+	files := load(t, ruleFiles{testFile: "env-file path block ^/\\.env$ \t \n"})
+
+	wantMatched(t, files, get(t, "/.env"), "env-file")
+}
+
 func TestFilesReadInNameOrderThenLineOrder(t *testing.T) {
 	t.Parallel()
 
@@ -200,6 +208,12 @@ func TestFaultStopsTheStartNamingTheFileAndLine(t *testing.T) {
 			"is not a rule: an id, a target, an action and a regex, " +
 				"separated by spaces or tabs",
 		},
+		{
+			// Else its regex would be a space, found in nearly every user agent.
+			"a regex of only spaces and tabs", "scanner user_agent ban\t \n", 1,
+			"is not a rule: an id, a target, an action and a regex, " +
+				"separated by spaces or tabs",
+		},
 		{
 			"an id of other characters", "# ids\n\nenv.file path ban ^/\n", 3,
 			`the id "env.file" is not an id of letters, digits, - and _`,
@@ -214,6 +228,21 @@ func TestFaultStopsTheStartNamingTheFileAndLine(t *testing.T) {
 			`the target "header:" is not path, query, uri, method, host, ` +
 				"user_agent, referer or header:<Name>",
 		},
+		{
+			"a header name written with its colon", "sqlmap header:User-Agent: ban sqlmap\n", 1,
+			`the target "header:User-Agent:" has a character after header: ` +
+				"that no header name can have",
+		},
+		{
+			"a header name with a semicolon", "accept header:Accept;q log ^$\n", 1,
+			`the target "header:Accept;q" has a character after header: ` +
+				"that no header name can have",
+		},
+		{
+			"a header name with brackets", "x-header header:X(y) log ^$\n", 1,
+			`the target "header:X(y)" has a character after header: ` +
+				"that no header name can have",
+		},
 		{
 			"the Host header", "host-header header:host block ^$\n", 1,
 			`the target "header:host" names a header that Go's HTTP server ` +