Core Rule Set: WordPress's names and addresses pass on more fields (closes #245)

The four rules on system files and commands (930120, 932120, 932250,
932260) now skip:

- page, p, postId and slug on all of wp-admin;
- url on the block editor's link previews and embeds: the REST group
  now covers /wp-block-editor/v1/ and /oembed/1.0/;
- the JSON fields title, excerpt, slug and name on /wp/v2/ routes.

A commenter's url skips 930120 and 932120 only. Every field keeps 932235.
Add 61 requests recorded from WordPress 7.1.4 with these fields.

Model: opus-5-5

AGENTS.md

@@ -1,39 +0,0 @@
-# Agents
-
-These are the rules for agents working on this repository.
-
-## Branches
-
-- `main` is the stable branch, and `next` is the development branch.
-- Every PR is cut from `next` and targets `next`.
-- Only sneak merges `next` into `main`.
-- Each issue lands on `next` as one squashed commit whose title ends
-  ` (closes #N)`. A PR may instead be exactly two commits: first tests that fail
-  against the current code, then the fix with no test assertion changed. Such a
-  PR lands as both commits.
-
-## Entrypoints
-
-- Work through the `Makefile` targets, which call the scripts in `script/`. The
-  README's [Entrypoints](README.md#entrypoints) section lists them.
-- Never run `go`, `golangci-lint` or prettier directly.
-- `make check` passes before every push.
-
-## Editing
-
-- Edit files by hand.
-- Format only with `make fmt`, and run it before committing any change.
-
-## Iron rules
-
-- The request log records query strings exactly as received.
-    - Secrets belong in POST data. An app that puts a secret in a query string
-      is fixed in the app.
-    - Never propose redacting a query string, Gitea's `?token=` included.
-- The default of `SWRP_TRUSTED_PROXIES` is the RFC 1918 set.
-    - The set is:
-        - `10.0.0.0/8`;
-        - `172.16.0.0/12`;
-        - `192.168.0.0/16`.
-    - `SPEC.md` states that default.
-    - Forwarded headers count only when the peer is a trusted proxy.

README.md

@@ -748,7 +748,6 @@ on the host.
 - [`EVALUATION.md`](EVALUATION.md): what already exists, what each tool covers
   and misses, and why none was adopted.
 - [`REPO_POLICIES.md`](REPO_POLICIES.md): the policies this repository follows.
-- [`AGENTS.md`](AGENTS.md): the rules for agents working on this repository.
 - [`docs/settings.md`](docs/settings.md): every setting and the values it takes.
 - [`docs/rules.md`](docs/rules.md): the rule file format and the default file.
 - [`docs/admin-endpoints.md`](docs/admin-endpoints.md): the endpoints under

SPEC.md

@@ -133,8 +133,6 @@ request log's `heuristic` field gives that name. `refusal_limit` and
 1. `GET /_sww/healthz` is answered at once by `rproxy`, before any check, so a
    health checker never needs an exemption and is never refused.
 2. The client is identified.
-    - `SWRP_TRUSTED_PROXIES` is `10.0.0.0/8`, `172.16.0.0/12` and
-      `192.168.0.0/16` by default. A list given replaces them.
     - If the TCP peer is inside `SWRP_TRUSTED_PROXIES`, `X-Forwarded-For` is
       walked from the right, and the first address outside those netblocks is
       the client. If every address is inside them, the leftmost is, so that a
@@ -872,10 +870,13 @@ changes hold in front of every app, and no setting undoes them:
             - the names `tag`, `category_name`, `plugin`, `checked[]`, `theme`
               and `stylesheet`;
             - the name of a plugin's settings page in `page`;
+            - the name of a template in `p` and `postId` of the site editor;
+            - the name of a plugin to install in `slug`;
             - the addresses `url`, `return`, `redirect_to`, `wp_http_referer`
               and `_wp_http_referer`.
-        - On every REST API route under `/wp/v2/` and on the route
+        - On every REST API route under `/wp/v2/`, `/wp-block-editor/v1/` and
-          `/oembed/1.0/embed`: `search`, `term`, `slug` and `url`.
+          `/oembed/1.0/`: `search`, `term`, `slug` and `url`. The block editor's
+          link previews and embeds send an address in `url`.
         - On `/`, `/index.php`, `/page/N/` and `/index.php/page/N/`: `s` and
           `tag`.
 
@@ -883,18 +884,6 @@ changes hold in front of every app, and no setting undoes them:
         (932235), because an address repeats the search it came from. The other
         fields of the groups keep that rule.
 
-    - On all of `/wp-admin/` and on `/wp-login.php`, these names skip the lists
-      of PowerShell commands and command names (932120, 932260):
-        - the name of a template in `p` and `postId` of the site editor;
-        - the name of a plugin to install in `slug`.
-
-        WordPress writes these names as slugs, such as `htaccess` and `ls-la`.
-
-    - `url` on the REST API routes `/wp-block-editor/v1/url-details` and
-      `/oembed/1.0/proxy`, for the lists of system files and PowerShell commands
-      (930120, 932120). The block editor's link previews and embeds send an
-      address in this field.
-
     - `_wp_http_referer` on `/wp-admin/upload.php`, for a Unix command given
       without arguments (932340). The media list's own address ends in `&ps`.
     - `file` on `/wp-admin/plugin-editor.php` and `/wp-admin/theme-editor.php`,
@@ -908,11 +897,9 @@ changes hold in front of every app, and no setting undoes them:
         - `newcontent` of a plugin's or a theme's file saved from its editor,
           which is PHP.
         - `pwd` when signing in, for the list of system files (930120).
-        - `title`, `excerpt` and `name` in a JSON body sent to any REST API
+        - `title`, `excerpt`, `slug` and `name` of a post or a tag saved through
-          route under `/wp/v2/`, for the rules on system files and commands
+          a REST API route under `/wp/v2/`, for the rules on system files and
-          (930120, 932120, 932250, 932260).
+          commands (930120, 932120, 932250, 932260).
-        - `slug` in a JSON body sent to any REST API route under `/wp/v2/`, for
-          the lists of PowerShell commands and command names (932120, 932260).
         - `url` on `/wp-comments-post.php`, the address of a commenter's site,
           for the lists of system files and PowerShell commands (930120,
           932120).

internal/coreruleset/coreruleset_test.go

@@ -862,7 +862,7 @@ func TestWordPressSearchesAndFileNamesAreExcludedOnlyThere(t *testing.T) {
 // TestWordPressGroupsSkipRulesOnTheirFields checks each field of the three
 // groups in wordpress.conf at one target of its group, against the same
 // values in a field no group names, then one field at each other target of
-// the group, then the names in the first group that skip fewer rules.
+// the group.
 func TestWordPressGroupsSkipRulesOnTheirFields(t *testing.T) {
 	t.Parallel()
 
@@ -882,7 +882,7 @@ func TestWordPressGroupsSkipRulesOnTheirFields(t *testing.T) {
 			"/wp-admin/options.php?",
 			[]string{
 				"s", "search", "q", "term", "tag", "category_name", "plugin",
-				"checked%5B%5D", "theme", "stylesheet", "page",
+				"checked%5B%5D", "theme", "stylesheet", "page", "p", "postId", "slug",
 			},
 			[]string{"url", "return", "redirect_to", "wp_http_referer", "_wp_http_referer"},
 		},
@@ -917,25 +917,15 @@ func TestWordPressGroupsSkipRulesOnTheirFields(t *testing.T) {
 		"/?rest_route=/wp/v2/tags&slug=python3",
 		"/wp-json/oembed/1.0/embed?url=python3",
 		"/?rest_route=/oembed/1.0/embed&url=python3",
+		"/wp-json/oembed/1.0/proxy?url=python3",
+		"/wp-json/wp-block-editor/v1/url-details?url=python3",
+		"/?rest_route=/wp-block-editor/v1/url-details&url=python3",
 		"/index.php?tag=python3",
 		"/page/2/?tag=python3",
 		"/index.php/page/2/?s=python3",
 	} {
 		wantResult(t, crs, get(target), coreruleset.Result{})
 	}
-
-	// The names WordPress writes as slugs skip only 932120 and 932260.
-	for _, field := range []string{"p", "postId", "slug"} {
-		for _, v := range wordpressWords() {
-			want := coreruleset.Result{}
-			if v.rule == 930120 || v.rule == 932250 {
-				want = matched(v.rule)
-			}
-
-			target := "/wp-admin/options.php?" + field + "=" + url.QueryEscape(v.value)
-			wantResult(t, crs, get(target), want)
-		}
-	}
 }
 
 func TestWordPressCommandAfterASeparatorIsRefusedOnSearchesAndNames(t *testing.T) {
@@ -964,6 +954,8 @@ func TestWordPressCommandAfterASeparatorIsRefusedOnSearchesAndNames(t *testing.T
 		"/wp-admin/site-editor.php?p=",
 		"/wp-admin/site-editor.php?postId=",
 		"/wp-admin/admin-ajax.php?slug=",
+		"/wp-json/wp-block-editor/v1/url-details?url=",
+		"/wp-json/oembed/1.0/proxy?url=",
 	} {
 		wantResult(t, crs, get(target+command), matched(932235))
 	}
@@ -975,15 +967,7 @@ func TestWordPressCommandAfterASeparatorIsRefusedOnSearchesAndNames(t *testing.T
 		wantBody(t, bodies, r, p.body, matched(932235), p.body)
 	}
 
-	// The addresses of a link preview, an embed and a commenter's site keep
+	// A commenter's address keeps 932260 too.
-	// 932260 too.
-	for _, target := range []string{
-		"/wp-json/wp-block-editor/v1/url-details?url=",
-		"/wp-json/oembed/1.0/proxy?url=",
-	} {
-		wantResult(t, crs, get(target+command), matched(932235, 932260))
-	}
-
 	comment := request{http.MethodPost, wpComment, []string{formData}}
 	wantBody(t, bodies, comment, "url="+command, matched(932235, 932260), "url="+command)
 }
@@ -1054,8 +1038,8 @@ func TestWordPressOtherFieldsAreExcludedOnlyThere(t *testing.T) {
 	}
 }
 
-// TestWordPressRESTFieldsAreExcludedOnlyThere checks search, term, url and
+// TestWordPressRESTFieldsAreExcludedOnlyThere checks search, term and the
-// the header X-HTTP-Method-Override at the REST API's addresses.
+// header X-HTTP-Method-Override at the REST API's addresses.
 func TestWordPressRESTFieldsAreExcludedOnlyThere(t *testing.T) {
 	t.Parallel()
 
@@ -1083,24 +1067,6 @@ func TestWordPressRESTFieldsAreExcludedOnlyThere(t *testing.T) {
 	wantChange(t, crs, get(directory+"?term=ls+-la"),
 		get("/wp-json/example/v1/things?term=ls+-la"), matched(932250))
 
-	// The block editor's link previews and embeds: url skips only 930120 and
-	// 932120.
-	for _, target := range []string{
-		"/wp-json/wp-block-editor/v1/url-details?url=",
-		"/?rest_route=/wp-block-editor/v1/url-details&url=",
-		"/wp-json/oembed/1.0/proxy?url=",
-		"/?rest_route=/oembed/1.0/proxy&url=",
-	} {
-		for _, v := range wordpressWords() {
-			want := coreruleset.Result{}
-			if v.rule == 932250 || v.rule == 932260 {
-				want = matched(v.rule)
-			}
-
-			wantResult(t, crs, get(target+url.QueryEscape(v.value)), want)
-		}
-	}
-
 	// The header X-HTTP-Method-Override at the address on the site root,
 	// and off the REST API.
 	headers := []string{jsonBody, "Content-Length: 2", wordpressOverride}
@@ -1127,24 +1093,26 @@ func TestWordPressBodyFieldsAreCheckedInTheQuery(t *testing.T) {
 }
 
 // TestWordPressBodyNamesAndAddressesAreExcludedOnlyThere checks the JSON
-// fields of a REST API save, a plugin to install and a commenter's address,
+// fields of a REST API save and a commenter's address, against the same
-// against the same values in another field and off their paths.
+// values in another field and off their paths.
 func TestWordPressBodyNamesAndAddressesAreExcludedOnlyThere(t *testing.T) {
 	t.Parallel()
 
 	crs := readingBodiesUpTo64K(t)
 
 	for _, v := range wordpressWords() {
-		for _, field := range []string{"title", "excerpt", "name"} {
+		for _, p := range wordpressBodyFields(url.QueryEscape(v.value)) {
-			body := fmt.Sprintf(`{%q:%q}`, field, v.value)
+			if p.contentType != jsonBody {
+				continue
+			}
 
 			for _, address := range wordpressRESTAddresses("/wp/v2/posts/1") {
 				r := request{http.MethodPost, address, []string{jsonBody, wordpressOverride}}
-				wantBody(t, crs, r, body, coreruleset.Result{}, body)
+				wantBody(t, crs, r, p.body, coreruleset.Result{}, p.body)
 			}
 
 			r := request{http.MethodPost, "/wp-json/example/v1/things", []string{jsonBody}}
-			wantBody(t, crs, r, body, matched(v.rule), body)
+			wantBody(t, crs, r, p.body, matched(v.rule), p.body)
 		}
 
 		other := fmt.Sprintf(`{"description":%q}`, v.value)
@@ -1152,23 +1120,6 @@ func TestWordPressBodyNamesAndAddressesAreExcludedOnlyThere(t *testing.T) {
 		wantBody(t, crs, r, other, matched(v.rule), other)
 	}
 
-	// A post's slug and a plugin to install skip only 932120 and 932260.
-	post := request{http.MethodPost, wpPost, []string{jsonBody, wordpressOverride}}
-	install := request{http.MethodPost, wpAjax, []string{formData}}
-
-	for _, v := range wordpressWords() {
-		want := coreruleset.Result{}
-		if v.rule == 930120 || v.rule == 932250 {
-			want = matched(v.rule)
-		}
-
-		slug := fmt.Sprintf(`{"slug":%q}`, v.value)
-		wantBody(t, crs, post, slug, want, slug)
-
-		plugin := "action=install-plugin&slug=" + url.QueryEscape(v.value)
-		wantBody(t, crs, install, plugin, want, plugin)
-	}
-
 	comment := request{http.MethodPost, wpComment, []string{formData}}
 	elsewhere := request{http.MethodPost, "/example/things", []string{formData}}
 

internal/coreruleset/wordpress.conf

@@ -102,18 +102,17 @@ SecRule REQUEST_FILENAME "@streq /wp-login.php" \
     "id:900704,phase:1,pass,nolog,chain"
     SecRule &ARGS_GET:pwd "@eq 0" "ctl:ruleRemoveTargetById=930120;ARGS:pwd"
 
-# Saving through the REST API, the JSON fields title, excerpt and name skip these
+# Saving through the REST API, a post's title, excerpt and slug and a tag's name
-# rules. WordPress writes a slug such as htaccess or ls-la, so slug skips only
+# skip these rules in the JSON fields title, excerpt, slug and name:
-# 932120 and 932260:
 #   930120: .htaccess
 #   932120: powershell
 #   932250: ls -la
 #   932260: python3
 SecRule TX:rest_route "@beginsWith /wp/v2/" "id:900712,phase:1,pass,nolog,chain"
     SecRule &ARGS_GET:/^json\.(?:title|excerpt|slug|name)$/ "@eq 0" \
-        "ctl:ruleRemoveTargetById=930120;ARGS:/^json\.(?:title|excerpt|name)$/,\
+        "ctl:ruleRemoveTargetById=930120;ARGS:/^json\.(?:title|excerpt|slug|name)$/,\
         ctl:ruleRemoveTargetById=932120;ARGS:/^json\.(?:title|excerpt|slug|name)$/,\
-        ctl:ruleRemoveTargetById=932250;ARGS:/^json\.(?:title|excerpt|name)$/,\
+        ctl:ruleRemoveTargetById=932250;ARGS:/^json\.(?:title|excerpt|slug|name)$/,\
         ctl:ruleRemoveTargetById=932260;ARGS:/^json\.(?:title|excerpt|slug|name)$/"
 
 # A comment's author gives the address of their site in url, which skips these
@@ -126,15 +125,6 @@ SecRule REQUEST_FILENAME "@streq /wp-comments-post.php" \
         "ctl:ruleRemoveTargetById=930120;ARGS:url,\
         ctl:ruleRemoveTargetById=932120;ARGS:url"
 
-# The block editor's link previews and embeds send an address in url, which
-# skips these rules there:
-#   930120: http://blog.example/.htaccess
-#   932120: http://blog.example/powershell-basics/
-SecRule TX:rest_route "@rx ^/(?:wp-block-editor/v1/url-details|oembed/1\.0/proxy)$" \
-    "id:900714,phase:1,pass,nolog,\
-    ctl:ruleRemoveTargetById=930120;ARGS:url,\
-    ctl:ruleRemoveTargetById=932120;ARGS:url"
-
 # The plugin and theme file editors name the file open in file, which skips
 # these rules there:
 #   930120: akismet/views/config.php
@@ -150,9 +140,9 @@ SecRule REQUEST_FILENAME "@rx ^/wp-admin/(?:plugin|theme)-editor\.php$" \
     ctl:ruleRemoveTargetById=932260;ARGS:file,\
     ctl:ruleRemoveTargetById=941180;ARGS:file"
 
-# Searches, the names of tags, categories, plugins, themes and plugins' settings
+# Searches, the names of tags, categories, plugins, themes, templates and
-# pages, and addresses can hold any word. In the fields of the next three rules,
+# plugins' settings pages, and addresses can hold any word. In the fields of the
-# each request skips the rules that refused such words:
+# next three rules, each request skips the rules that refused such words:
 #   930120: .htaccess
 #   932120: powershell
 #   932250: ls -la
@@ -161,22 +151,20 @@ SecRule REQUEST_FILENAME "@rx ^/wp-admin/(?:plugin|theme)-editor\.php$" \
 # group also skip:
 #   932235: python3, in the address of a page of results
 
-# All of wp-admin, and signing in. A plugin's settings page is named in page.
+# All of wp-admin, and signing in. A plugin's settings page is named in page,
-# The site editor names a template in p and postId, and a plugin to install is
+# a template in p and postId of the site editor, and a plugin to install in
-# named in slug. WordPress writes these three as slugs, such as htaccess or
+# slug. Names are matched in lower case, so postId is written postid.
-# ls-la, so they skip only 932120 and 932260. Names are matched in lower case,
-# so postId is written postid.
 SecRule REQUEST_FILENAME "@rx ^/(?:wp-admin/|wp-login\.php$)" \
     "id:900707,phase:1,pass,nolog,\
-    ctl:ruleRemoveTargetById=930120;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
+    ctl:ruleRemoveTargetById=930120;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|p|postid|slug|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
     ctl:ruleRemoveTargetById=932120;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|p|postid|slug|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
     ctl:ruleRemoveTargetById=932235;ARGS:/^(?:url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
-    ctl:ruleRemoveTargetById=932250;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
+    ctl:ruleRemoveTargetById=932250;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|p|postid|slug|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/,\
     ctl:ruleRemoveTargetById=932260;ARGS:/^(?:s|search|q|term|tag|category_name|plugin|checked\[\]|theme|stylesheet|page|p|postid|slug|url|return|redirect_to|wp_http_referer|_wp_http_referer)$/"
 
-# Every route of WordPress's own REST API, and the embed address each post links
+# Every route of WordPress's own REST API: under /wp/v2/, the block editor's
-# to.
+# link previews under /wp-block-editor/v1/, and the embeds under /oembed/1.0/.
-SecRule TX:rest_route "@rx ^/(?:wp/v2/|oembed/1\.0/embed$)" \
+SecRule TX:rest_route "@rx ^/(?:wp/v2/|wp-block-editor/v1/|oembed/1\.0/)" \
     "id:900708,phase:1,pass,nolog,\
     ctl:ruleRemoveTargetById=930120;ARGS:/^(?:search|term|slug|url)$/,\
     ctl:ruleRemoveTargetById=932120;ARGS:/^(?:search|term|slug|url)$/,\