Core Rule Set: WordPress's names and addresses pass on more fields (closes #245)

Each field skips only the rules its recorded requests set off:

- page on all of wp-admin, and the JSON fields title, excerpt and name
  on /wp/v2/ routes: 930120, 932120, 932250 and 932260;
- p, postId and slug on all of wp-admin, and the JSON slug on /wp/v2/
  routes: 932120 and 932260, since WordPress writes these as slugs;
- url on the block editor's link previews and embeds, and a commenter's
  url: 930120 and 932120.

Every field keeps 932235. Add 61 requests recorded from WordPress 7.1.4
with these fields.

Model: opus-5-5

AGENTS.md

@@ -0,0 +1,39 @@
+# Agents
+
+These are the rules for agents working on this repository.
+
+## Branches
+
+- `main` is the stable branch, and `next` is the development branch.
+- Every PR is cut from `next` and targets `next`.
+- Only sneak merges `next` into `main`.
+- Each issue lands on `next` as one squashed commit whose title ends
+  ` (closes #N)`. A PR may instead be exactly two commits: first tests that fail
+  against the current code, then the fix with no test assertion changed. Such a
+  PR lands as both commits.
+
+## Entrypoints
+
+- Work through the `Makefile` targets, which call the scripts in `script/`. The
+  README's [Entrypoints](README.md#entrypoints) section lists them.
+- Never run `go`, `golangci-lint` or prettier directly.
+- `make check` passes before every push.
+
+## Editing
+
+- Edit files by hand.
+- Format only with `make fmt`, and run it before committing any change.
+
+## Iron rules
+
+- The request log records query strings exactly as received.
+    - Secrets belong in POST data. An app that puts a secret in a query string
+      is fixed in the app.
+    - Never propose redacting a query string, Gitea's `?token=` included.
+- The default of `SWRP_TRUSTED_PROXIES` is the RFC 1918 set.
+    - The set is:
+        - `10.0.0.0/8`;
+        - `172.16.0.0/12`;
+        - `192.168.0.0/16`.
+    - `SPEC.md` states that default.
+    - Forwarded headers count only when the peer is a trusted proxy.

README.md

@@ -748,6 +748,7 @@ on the host.
 - [`EVALUATION.md`](EVALUATION.md): what already exists, what each tool covers
   and misses, and why none was adopted.
 - [`REPO_POLICIES.md`](REPO_POLICIES.md): the policies this repository follows.
+- [`AGENTS.md`](AGENTS.md): the rules for agents working on this repository.
 - [`docs/settings.md`](docs/settings.md): every setting and the values it takes.
 - [`docs/rules.md`](docs/rules.md): the rule file format and the default file.
 - [`docs/admin-endpoints.md`](docs/admin-endpoints.md): the endpoints under

SPEC.md

@@ -133,6 +133,8 @@ request log's `heuristic` field gives that name. `refusal_limit` and
 1. `GET /_sww/healthz` is answered at once by `rproxy`, before any check, so a
    health checker never needs an exemption and is never refused.
 2. The client is identified.
+    - `SWRP_TRUSTED_PROXIES` is `10.0.0.0/8`, `172.16.0.0/12` and
+      `192.168.0.0/16` by default. A list given replaces them.
     - If the TCP peer is inside `SWRP_TRUSTED_PROXIES`, `X-Forwarded-For` is
       walked from the right, and the first address outside those netblocks is
       the client. If every address is inside them, the leftmost is, so that a
@@ -870,13 +872,10 @@ changes hold in front of every app, and no setting undoes them:
             - the names `tag`, `category_name`, `plugin`, `checked[]`, `theme`
               and `stylesheet`;
             - the name of a plugin's settings page in `page`;
-            - the name of a template in `p` and `postId` of the site editor;
-            - the name of a plugin to install in `slug`;
             - the addresses `url`, `return`, `redirect_to`, `wp_http_referer`
               and `_wp_http_referer`.
-        - On every REST API route under `/wp/v2/`, `/wp-block-editor/v1/` and
+        - On every REST API route under `/wp/v2/` and on the route
-          `/oembed/1.0/`: `search`, `term`, `slug` and `url`. The block editor's
+          `/oembed/1.0/embed`: `search`, `term`, `slug` and `url`.
-          link previews and embeds send an address in `url`.
         - On `/`, `/index.php`, `/page/N/` and `/index.php/page/N/`: `s` and
           `tag`.
 
@@ -884,6 +883,18 @@ changes hold in front of every app, and no setting undoes them:
         (932235), because an address repeats the search it came from. The other
         fields of the groups keep that rule.
 
+    - On all of `/wp-admin/` and on `/wp-login.php`, these names skip the lists
+      of PowerShell commands and command names (932120, 932260):
+        - the name of a template in `p` and `postId` of the site editor;
+        - the name of a plugin to install in `slug`.
+
+        WordPress writes these names as slugs, such as `htaccess` and `ls-la`.
+
+    - `url` on the REST API routes `/wp-block-editor/v1/url-details` and
+      `/oembed/1.0/proxy`, for the lists of system files and PowerShell commands
+      (930120, 932120). The block editor's link previews and embeds send an
+      address in this field.
+
     - `_wp_http_referer` on `/wp-admin/upload.php`, for a Unix command given
       without arguments (932340). The media list's own address ends in `&ps`.
     - `file` on `/wp-admin/plugin-editor.php` and `/wp-admin/theme-editor.php`,
@@ -897,9 +908,11 @@ changes hold in front of every app, and no setting undoes them:
         - `newcontent` of a plugin's or a theme's file saved from its editor,
           which is PHP.
         - `pwd` when signing in, for the list of system files (930120).
-        - `title`, `excerpt`, `slug` and `name` of a post or a tag saved through
+        - `title`, `excerpt` and `name` in a JSON body sent to any REST API
-          a REST API route under `/wp/v2/`, for the rules on system files and
+          route under `/wp/v2/`, for the rules on system files and commands
-          commands (930120, 932120, 932250, 932260).
+          (930120, 932120, 932250, 932260).
+        - `slug` in a JSON body sent to any REST API route under `/wp/v2/`, for
+          the lists of PowerShell commands and command names (932120, 932260).
         - `url` on `/wp-comments-post.php`, the address of a commenter's site,
           for the lists of system files and PowerShell commands (930120,
           932120).

internal/coreruleset/coreruleset_test.go

@@ -862,7 +862,7 @@ func TestWordPressSearchesAndFileNamesAreExcludedOnlyThere(t *testing.T) {
 // TestWordPressGroupsSkipRulesOnTheirFields checks each field of the three
 // groups in wordpress.conf at one target of its group, against the same
 // values in a field no group names, then one field at each other target of
-// the group.
+// the group, then the names in the first group that skip fewer rules.
 func TestWordPressGroupsSkipRulesOnTheirFields(t *testing.T) {
 	t.Parallel()
 
@@ -882,7 +882,7 @@ func TestWordPressGroupsSkipRulesOnTheirFields(t *testing.T) {
 			"/wp-admin/options.php?",
 			[]string{
 				"s", "search", "q", "term", "tag", "category_name", "plugin",
-				"checked%5B%5D", "theme", "stylesheet", "page", "p", "postId", "slug",
+				"checked%5B%5D", "theme", "stylesheet", "page",
 			},
 			[]string{"url", "return", "redirect_to", "wp_http_referer", "_wp_http_referer"},
 		},
@@ -917,15 +917,25 @@ func TestWordPressGroupsSkipRulesOnTheirFields(t *testing.T) {
 		"/?rest_route=/wp/v2/tags&slug=python3",
 		"/wp-json/oembed/1.0/embed?url=python3",
 		"/?rest_route=/oembed/1.0/embed&url=python3",
-		"/wp-json/oembed/1.0/proxy?url=python3",
-		"/wp-json/wp-block-editor/v1/url-details?url=python3",
-		"/?rest_route=/wp-block-editor/v1/url-details&url=python3",
 		"/index.php?tag=python3",
 		"/page/2/?tag=python3",
 		"/index.php/page/2/?s=python3",
 	} {
 		wantResult(t, crs, get(target), coreruleset.Result{})
 	}
+
+	// The names WordPress writes as slugs skip only 932120 and 932260.
+	for _, field := range []string{"p", "postId", "slug"} {
+		for _, v := range wordpressWords() {
+			want := coreruleset.Result{}
+			if v.rule == 930120 || v.rule == 932250 {
+				want = matched(v.rule)
+			}
+
+			target := "/wp-admin/options.php?" + field + "=" + url.QueryEscape(v.value)
+			wantResult(t, crs, get(target), want)
+		}
+	}
 }
 
 func TestWordPressCommandAfterASeparatorIsRefusedOnSearchesAndNames(t *testing.T) {
@@ -954,8 +964,6 @@ func TestWordPressCommandAfterASeparatorIsRefusedOnSearchesAndNames(t *testing.T
 		"/wp-admin/site-editor.php?p=",
 		"/wp-admin/site-editor.php?postId=",
 		"/wp-admin/admin-ajax.php?slug=",
-		"/wp-json/wp-block-editor/v1/url-details?url=",
-		"/wp-json/oembed/1.0/proxy?url=",
 	} {
 		wantResult(t, crs, get(target+command), matched(932235))
 	}
@@ -967,7 +975,15 @@ func TestWordPressCommandAfterASeparatorIsRefusedOnSearchesAndNames(t *testing.T
 		wantBody(t, bodies, r, p.body, matched(932235), p.body)
 	}
 
-	// A commenter's address keeps 932260 too.
+	// The addresses of a link preview, an embed and a commenter's site keep
+	// 932260 too.
+	for _, target := range []string{
+		"/wp-json/wp-block-editor/v1/url-details?url=",
+		"/wp-json/oembed/1.0/proxy?url=",
+	} {
+		wantResult(t, crs, get(target+command), matched(932235, 932260))
+	}
+
 	comment := request{http.MethodPost, wpComment, []string{formData}}
 	wantBody(t, bodies, comment, "url="+command, matched(932235, 932260), "url="+command)
 }
@@ -1038,8 +1054,8 @@ func TestWordPressOtherFieldsAreExcludedOnlyThere(t *testing.T) {
 	}
 }
 
-// TestWordPressRESTFieldsAreExcludedOnlyThere checks search, term and the
+// TestWordPressRESTFieldsAreExcludedOnlyThere checks search, term, url and
-// header X-HTTP-Method-Override at the REST API's addresses.
+// the header X-HTTP-Method-Override at the REST API's addresses.
 func TestWordPressRESTFieldsAreExcludedOnlyThere(t *testing.T) {
 	t.Parallel()
 
@@ -1067,6 +1083,24 @@ func TestWordPressRESTFieldsAreExcludedOnlyThere(t *testing.T) {
 	wantChange(t, crs, get(directory+"?term=ls+-la"),
 		get("/wp-json/example/v1/things?term=ls+-la"), matched(932250))
 
+	// The block editor's link previews and embeds: url skips only 930120 and
+	// 932120.
+	for _, target := range []string{
+		"/wp-json/wp-block-editor/v1/url-details?url=",
+		"/?rest_route=/wp-block-editor/v1/url-details&url=",
+		"/wp-json/oembed/1.0/proxy?url=",
+		"/?rest_route=/oembed/1.0/proxy&url=",
+	} {
+		for _, v := range wordpressWords() {
+			want := coreruleset.Result{}
+			if v.rule == 932250 || v.rule == 932260 {
+				want = matched(v.rule)
+			}
+
+			wantResult(t, crs, get(target+url.QueryEscape(v.value)), want)
+		}
+	}
+
 	// The header X-HTTP-Method-Override at the address on the site root,
 	// and off the REST API.
 	headers := []string{jsonBody, "Content-Length: 2", wordpressOverride}
@@ -1093,26 +1127,24 @@ func TestWordPressBodyFieldsAreCheckedInTheQuery(t *testing.T) {
 }
 
 // TestWordPressBodyNamesAndAddressesAreExcludedOnlyThere checks the JSON
-// fields of a REST API save and a commenter's address, against the same
+// fields of a REST API save, a plugin to install and a commenter's address,
-// values in another field and off their paths.
+// against the same values in another field and off their paths.
 func TestWordPressBodyNamesAndAddressesAreExcludedOnlyThere(t *testing.T) {
 	t.Parallel()
 
 	crs := readingBodiesUpTo64K(t)
 
 	for _, v := range wordpressWords() {
-		for _, p := range wordpressBodyFields(url.QueryEscape(v.value)) {
+		for _, field := range []string{"title", "excerpt", "name"} {
-			if p.contentType != jsonBody {
+			body := fmt.Sprintf(`{%q:%q}`, field, v.value)
-				continue
-			}
 
 			for _, address := range wordpressRESTAddresses("/wp/v2/posts/1") {
 				r := request{http.MethodPost, address, []string{jsonBody, wordpressOverride}}
-				wantBody(t, crs, r, p.body, coreruleset.Result{}, p.body)
+				wantBody(t, crs, r, body, coreruleset.Result{}, body)
 			}
 
 			r := request{http.MethodPost, "/wp-json/example/v1/things", []string{jsonBody}}
-			wantBody(t, crs, r, p.body, matched(v.rule), p.body)
+			wantBody(t, crs, r, body, matched(v.rule), body)
 		}
 
 		other := fmt.Sprintf(`{"description":%q}`, v.value)
@@ -1120,6 +1152,23 @@ func TestWordPressBodyNamesAndAddressesAreExcludedOnlyThere(t *testing.T) {
 		wantBody(t, crs, r, other, matched(v.rule), other)
 	}
 
+	// A post's slug and a plugin to install skip only 932120 and 932260.
+	post := request{http.MethodPost, wpPost, []string{jsonBody, wordpressOverride}}
+	install := request{http.MethodPost, wpAjax, []string{formData}}
+
+	for _, v := range wordpressWords() {
+		want := coreruleset.Result{}
+		if v.rule == 930120 || v.rule == 932250 {
+			want = matched(v.rule)
+		}
+
+		slug := fmt.Sprintf(`{"slug":%q}`, v.value)
+		wantBody(t, crs, post, slug, want, slug)
+
+		plugin := "action=install-plugin&slug=" + url.QueryEscape(v.value)
+		wantBody(t, crs, install, plugin, want, plugin)
+	}
+
 	comment := request{http.MethodPost, wpComment, []string{formData}}
 	elsewhere := request{http.MethodPost, "/example/things", []string{formData}}
 

internal/coreruleset/wordpress.conf

@@ -102,17 +102,18 @@ SecRule REQUEST_FILENAME "@streq /wp-login.php" \
     "id:900704,phase:1,pass,nolog,chain"
     SecRule &ARGS_GET:pwd "@eq 0" "ctl:ruleRemoveTargetById=930120;ARGS:pwd"
 
-# Saving through the REST API, a post's title, excerpt and slug and a tag's name
+# Saving through the REST API, the JSON fields title, excerpt and name skip these
-# skip these rules in the JSON fields title, excerpt, slug and name:
+# rules. WordPress writes a slug such as htaccess or ls-la, so slug skips only
+# 932120 and 932260:
 #   930120: .htaccess
 #   932120: powershell
 #   932250: ls -la
 #   932260: python3
 SecRule TX:rest_route "@beginsWith /wp/v2/" "id:900712,phase:1,pass,nolog,chain"
     SecRule &ARGS_GET:/^json\.(?:title|excerpt|slug|name)$/ "@eq 0" \
-        "ctl:ruleRemoveTargetById=930120;ARGS:/^json\.(?:title|excerpt|slug|name)$/,\
+        "ctl:ruleRemoveTargetById=930120;ARGS:/^json\.(?:title|excerpt|name)$/,\
         ctl:ruleRemoveTargetById=932120;ARGS:/^json\.(?:title|excerpt|slug|name)$/,\
-        ctl:ruleRemoveTargetById=932250;ARGS:/^json\.(?:title|excerpt|slug|name)$/,\
+        ctl:ruleRemoveTargetById=932250;ARGS:/^json\.(?:title|excerpt|name)$/,\
         ctl:ruleRemoveTargetById=932260;ARGS:/^json\.(?:title|excerpt|slug|name)$/"
 
 # A comment's author gives the address of their site in url, which skips these
@@ -125,6 +126,15 @@ SecRule REQUEST_FILENAME "@streq /wp-comments-post.php" \
         "ctl:ruleRemoveTargetById=930120;ARGS:url,\
         ctl:ruleRemoveTargetById=932120;ARGS:url"
 
+# The block editor's link previews and embeds send an address in url, which
+# skips these rules there:
+#   930120: http://blog.example/.htaccess
+#   932120: http://blog.example/powershell-basics/
+SecRule TX:rest_route "@rx ^/(?:wp-block-editor/v1/url-details|oembed/1\.0/proxy)$" \
+    "id:900714,phase:1,pass,nolog,\
+    ctl:ruleRemoveTargetById=930120;ARGS:url,\
+    ctl:ruleRemoveTargetById=932120;ARGS:url"
+
 # The plugin and theme file editors name the file open in file, which skips
 # these rules there:
 #   930120: akismet/views/config.php
@@ -140,9 +150,9 @@ SecRule REQUEST_FILENAME "@rx ^/wp-admin/(?:plugin|theme)-editor\.php$" \
     ctl:ruleRemoveTargetById=932260;ARGS:file,\
     ctl:ruleRemoveTargetById=941180;ARGS:file"
 
-# Searches, the names of tags, categories, plugins, themes, templates and
+# Searches, the names of tags, categories, plugins, themes and plugins' settings
-# plugins' settings pages, and addresses can hold any word. In the fields of the
+# pages, and addresses can hold any word. In the fields of the next three rules,
-# next three rules, each request skips the rules that refused such words:
+# each request skips the rules that refused such words:
 #   930120: .htaccess
 #   932120: powershell
 #   932250: ls -la
@@ -151,20 +161,22 @@ SecRule REQUEST_FILENAME "@rx ^/wp-admin/(?:plugin|theme)-editor\.php$" \
 # group also skip:
 #   932235: python3, in the address of a page of results
 
-# All of wp-admin, and signing in. A plugin's settings page is named in page,
+# All of wp-admin, and signing in. A plugin's settings page is named in page.
-# a template in p and postId of the site editor, and a plugin to install in
+# The site editor names a template in p and postId, and a plugin to install is
-# slug. Names are matched in lower case, so postId is written postid.
+# named in slug. WordPress writes these three as slugs, such as htaccess or
+# ls-la, so they skip only 932120 and 932260. Names are matched in lower case,
+# so postId is written postid.
 SecRule REQUEST_FILENAME "@rx ^/(?:wp-admin/|wp-login\.php$)" \
     "id:900707,phase:1,pass,nolog,\
-    ctl:ruleRemoveTargetById=930120;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|p|postid|slug|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
+    ctl:ruleRemoveTargetById=930120;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
     ctl:ruleRemoveTargetById=932120;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|p|postid|slug|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
     ctl:ruleRemoveTargetById=932235;ARGS:/^(?:url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
-    ctl:ruleRemoveTargetById=932250;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|p|postid|slug|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
+    ctl:ruleRemoveTargetById=932250;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
     ctl:ruleRemoveTargetById=932260;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|p|postid|slug|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/"
 
-# Every route of WordPress's own REST API: under /wp/v2/, the block editor's
+# Every route of WordPress's own REST API, and the embed address each post links
-# link previews under /wp-block-editor/v1/, and the embeds under /oembed/1.0/.
+# to.
-SecRule TX:rest_route "@rx ^/(?:wp/v2/|wp-block-editor/v1/|oembed/1\.0/)" \
+SecRule TX:rest_route "@rx ^/(?:wp/v2/|oembed/1\.0/embed$)" \
     "id:900708,phase:1,pass,nolog,\
     ctl:ruleRemoveTargetById=930120;ARGS:/^(?:search|term|slug|url)$/,\
     ctl:ruleRemoveTargetById=932120;ARGS:/^(?:search|term|slug|url)$/,\