Gitea setup: Core Rule Set blocks attacks, not ordinary Gitea traffic (closes #198)

The Gitea setup runs the Core Rule Set in block mode. Rules that refused
ordinary Gitea requests are switched off only where they misfired:

- deploy/gitea/crs-exclusions.conf, loaded through the new setting
  SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE before the Core Rule Set's rules,
  covers Gitea's search, file-name and branch-name fields, Referer, file
  paths, API attachment names and Actions artifact uploads;
- smallwebwaf's defaults gain a seventh change: the registry upload
  path's from parameter skips 932260.

A file that cannot be read, does not parse or names an id from 900000
to 900999 stops servewaf at start, in every mode. internal/waf's tests
and script/gitea-setup send the recorded requests in
internal/waf/testdata/gitea-requests.txt and three attacks.

Model: opus-5-5

docs/gitea.md

@@ -13,7 +13,8 @@ Four containers share one Docker network, the shared network:
   plain HTTP to HTTPS;
 - `rproxy`, in front of gitea;
 - gitea 1.27.3, as its image comes;
-- `servewaf`, with a volume for its state files.
+- `servewaf`, with a volume for its state files, and with
+  `deploy/gitea/crs-exclusions.conf` mounted read-only.
 
 To start it:
 
@@ -30,7 +31,8 @@ To start it:
    `docker run --rm <gitea's image> gitea generate secret SECRET_KEY`, with
    gitea's image as `deploy/gitea/compose.yaml` names it. Do this once per
    install, and keep the key.
-5. Run `docker compose up --detach` in `deploy/gitea`.
+5. Run `docker compose up --detach` in `deploy/gitea`, with
+   `crs-exclusions.conf` beside `compose.yaml`.
 6. Make gitea's first user with `gitea admin user create`, in gitea's container,
    as the user `git`.
 
@@ -46,6 +48,12 @@ under a name of its own, and checks:
   `rproxy`, and that `servewaf` checks them;
 - that gitea logs the visitor's address, not `rproxy`'s;
 - that a request from a container on the shared network skips the checks;
+- that the Core Rule Set refuses none of the ordinary gitea requests in
+  `internal/waf/testdata/gitea-requests.txt`, sent by a signed-in user;
+- that it refuses none of the API calls with JSON bodies;
+- that it refuses no fetch;
+- that it refuses SQL injection, path traversal and script injection in a query
+  string;
 - that with `servewaf` stopped, gitea still answers through `rproxy`.
 
 ## Uploads and downloads
@@ -79,11 +87,8 @@ GiB a day.
 
 ## The checks
 
-`SWWAF_WAF_MODE` is `detect`. The Core Rule Set then logs a match, with its rule
-ids, and refuses nothing. In `block` mode, the default, it refuses everyday
-gitea searches and file names. The section "What the Core Rule Set refuses in
-`block` mode" below lists them. Each such refusal counts toward the error burst,
-so 31 in one minute ban the address.
+The Core Rule Set runs in `block` mode, the default, with gitea's exclusions
+(see "The Core Rule Set's exclusions" below).
 
 Every other check stays at its default. A client may make 120 requests a minute,
 1,500 an hour and 6,000 a day. Archive downloads and blame and history pages are
@@ -153,188 +158,49 @@ within two minutes. Its address is then banned, its running jobs fail, and each
 repeat within a day makes the ban longer. Newer runners ask for work less often.
 Put the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
 
-## What the Core Rule Set refuses in `block` mode
+## The Core Rule Set's exclusions
 
-These notes record what the Core Rule Set does in front of gitea in `block`
-mode, at the other defaults. They are observations, not requirements.
-`script/gitea-requests` runs gitea behind `rproxy` and `servewaf` at the
-defaults. It sends through them a sample of gitea's ordinary use and the
-requests named below. It reports each request the Core Rule Set refuses, so that
-these notes can be checked again when gitea or the Core Rule Set changes
-version. In `detect` mode, as the setup runs it, the same requests are logged
-with their rule ids, and pass.
+`deploy/gitea/crs-exclusions.conf` switches single Core Rule Set rules off where
+they refused ordinary gitea requests. `servewaf` loads it from
+`SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE`, before the Core Rule Set's rules. Each
+exclusion covers one rule, in these places only:
 
-Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts
-toward). The request log names the rule in `waf_rule_ids`, which
-`SWWAF_WAF_DISABLED_RULES` can switch off.
+- `q`, the text of every search;
+- the query parameters that hold a file name: `file`, `path`, `template`, `body`
+  and `redirect_to`;
+- the query parameters that hold a branch or tag name: `name`, `tag`,
+  `rule_name` and `refSubUrl`;
+- `Referer`, on a page opened from search results or from a file's page;
+- the path of a file's page, raw file, blame, history and edit page, and of the
+  API's call for its contents;
+- `name`, the file name of an attachment uploaded through the API;
+- the header `Content-Range` of an Actions artifact upload.
 
-### What it lets through
+gitea reads these as text to search for, as names in a repository, or, for
+`Content-Range`, as the part of a file an upload carries. It never runs them as
+SQL, a command or a page's script. So an attack that only the excluded rules
+would catch there reaches gitea and does nothing. Every other field and path
+keeps every rule.
 
-The Core Rule Set lets gitea's ordinary use through, apart from the refusals in
-the sections after this one:
+To add an exclusion for a request the Core Rule Set refuses:
 
-- browsing and views of files in a repository, with their history, blame and the
-  file tree;
-- diffs, including their hidden lines and large files, and pull request review;
-- git's clone, fetch and push over HTTP;
-- signing in, including the return to the page a visitor came from, and sign-in
-  with Git Credential Manager, git-credential-oauth or tea;
-- the API's calls for a file and its commits;
-- pushing and pulling container images and packages;
-- Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`;
-- posting issues, pull requests, comments, wiki pages and files saved in the web
-  editor, code included, since no body is read;
-- the page gitea shows after a change, with its message naming what was done.
+1. Find the request in `servewaf`'s log. Its line gives the path and query, and
+   names the rules in `waf_rule_ids`.
+2. For each rule, add a line with `ctl:ruleRemoveTargetById` for the field it
+   matched. For a field on every path, copy one of the file's `SecAction` lines.
+   For some paths only, copy one of its `SecRule REQUEST_FILENAME` lines. Give
+   it a new id. An id from 900000 to 900999 stops the start.
+3. Above it, write a one-line comment naming the request it refused.
+4. Add the request to `internal/waf/testdata/gitea-requests.txt`, and run
+   `make check`.
+5. Restart `servewaf`.
 
-### A query string that reads as an attack
+The Core Rule Set still refuses the page of a file in a directory named `...`,
+which reads as path traversal.
 
-Most often this is a search. A search term is a query parameter, which the Core
-Rule Set checks against its lists of system files and commands (see the fourth
-change under "Attack detection" in `SPEC.md`). A release attachment uploaded
-through the API as `docker-compose.yml` is refused the same way. Examples:
+With `SWWAF_WAF_BODY_LIMIT` set, it also refuses issue text, comments, wiki
+pages and files saved in the web editor that hold shell commands or code.
 
-- a name on those lists, such as `package.json`, `.gitignore` or
-  `docker-compose.yml`;
-- text that starts with a command name, such as `python3` or `ssh key`, or has
-  one right after a `/` in its first word, such as `feat/docker-support`;
-- a shell command with its options or a system path (`ls -la`, `sed -i`,
-  `/bin/sh`), or a command in backticks;
-- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`) or an SQL
-  statement (`SELECT * FROM users WHERE`);
-- a URL naming an IP address or `localhost`.
-
-### A branch, tag or file name that starts with a command name
-
-The Core Rule Set's list of commands (932260) holds names such as `docker-`,
-`python3`, `ansible`, `base64` and `whoami`. A query parameter holding a branch,
-tag or file name that starts with one is refused. So is one that has such a name
-right after a `/`, such as `feat/docker-support`, `fix/docker-build` or
-`renovate/docker-build-push-action-6.x`. gitea's own pages send the name in a
-query parameter that 930120, 932160 and 932260 still check:
-
-- `refSubUrl`, the branch or tag of a directory listing. A listing sends it when
-  it asks for its entries' last commits in a second request. It does so whenever
-  gitea takes more than a second to work them out;
-- `name`, when a branch is deleted or restored on the branches page;
-- `tag`, when a release is started from a tag;
-- `template`, the file of an issue template, when an issue is opened from it;
-- `rule_name`, when a branch protection rule is opened for editing in the
-  repository's settings or has just been saved.
-
-What a visitor sees, for a branch named `docker-build`:
-
-- A directory listing that makes that second request leaves those last commits
-  out and shows an error.
-- The branches page shows an error instead of deleting or restoring the branch.
-- A release started from such a tag gets the `403` answer of `smallwebwaf` in
-  place of its form. So does an issue opened from such a template, and such a
-  rule opened for editing.
-- Saving such a rule stores it. gitea then returns to the branch settings page,
-  which gets the `403` answer, so the save looks as if it failed.
-
-### A file name in a query parameter that trips the code rules
-
-The Core Rule Set's rules for Java, script, PowerShell and SQL find what they
-look for in some file names. This holds too in the ten query parameters that
-930120, 932160 and 932260 do not check (see the fourth change under "Attack
-detection" in `SPEC.md`). Examples:
-
-- `runtime` in OpenJDK's `src/java.base/share/classes/java/lang/Runtime.java`
-  (944110);
-- a Java class, as in `java.lang.String.md` (944130);
-- `document.write.js` (941180);
-- `process.env.d.ts` (934100);
-- `__proto__` (934130);
-- the PowerShell command `Install-Script`, as in `install-script.sh` (932120);
-- a database's name, as in PostgreSQL's `information_schema.sql` (942140).
-
-For such a file, these requests are refused:
-
-- the API's list of its commits (`path`);
-- the API's call for the contents of files that names it in its query (`body`);
-- the commit graph of that file alone (`file`);
-- the hidden lines of its diff (`path`), which stay hidden;
-- the sign-in page, for a visitor who opens a page of the file that needs
-  signing in, such as its edit page (`redirect_to`). The visitor can still sign
-  in by opening the sign-in page itself.
-
-### A path
-
-- A file name ending in `~`, or an `.xhtml` file whose path holds a space.
-- A path that holds what the Core Rule Set takes for a Java class (944130), as
-  `java.lang.String.md` and
-  `META-INF/services/org.apache.commons.logging.LogFactory` do.
-- A path that holds what it takes for Node.js code (934100), as
-  `process.env.d.ts` does.
-- A path with a directory named `...` (930100, 930110), such as `.../notes.txt`.
-
-For such a file, these are refused:
-
-- the page of the file;
-- its raw file;
-- its history and its blame;
-- its edit page.
-
-The directory that holds it lists it, and a clone carries it.
-
-### A container image push that reuses a layer
-
-A push can reuse a layer of an image docker has pushed to that gitea before.
-docker then asks gitea to take the layer from that image, naming it in the query
-parameter `from`, which 932260 checks. The request is refused when the image's
-owner starts with an entry on the list of commands. It is refused too when the
-image's name after the owner starts with one. Examples are `owner/python3-app`,
-`owner/docker-base`, and any image of an owner named `ansible` or `docker-team`.
-
-What happens next depends on docker's image store. A docker that keeps its
-images in containerd's image store fails the push, and fails each new try the
-same way. A new install of Docker Engine 29 does this. A docker that keeps them
-in its own older image store uploads the layer instead. The push then succeeds
-after one refusal for each layer it reuses, and each of those refusals counts
-toward the error burst ("Attack detection" in `SPEC.md`).
-
-### Artifact uploads from `actions/upload-artifact@v3`
-
-This action sends the header `Content-Range`, which the Core Rule Set refuses
-(920450). The action sends two files at a time, does not retry a `403`, and
-sends no further file once one is refused. So an upload is refused once or
-twice, however many files it holds, and the step fails. One upload does not ban
-the runner. The error burst ("Attack detection" in `SPEC.md`) bans it after more
-than 30 such refusals from its address in a minute, as when many jobs upload at
-once. `actions/upload-artifact@v4` does not send the header.
-
-### What body inspection refuses
-
-With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In
-front of gitea it then refuses these, until their rule ids, named in the request
-log, are added to `SWWAF_WAF_DISABLED_RULES`:
-
-- issue and comment text, wiki pages and files saved in the web editor that hold
-  shell commands or code (932125, 932235, 932250 and others);
-- package descriptions that show code;
-- PyPI uploads (922130);
-- attachments named like `debug.log` or `config.yml` (932180).
-
-### What `script/gitea-requests` sends
-
-Of the requests above, the script sends:
-
-- for each file named under "A file name in a query parameter that trips the
-  code rules", the API's list of its commits;
-- for `Runtime.java` alone, the commit graph of that file, the hidden lines of
-  its diff, and the API's call for the contents of files that names it;
-- the sign-in page for a visitor who opens the edit page of `document.write.js`;
-- the pages of `java.lang.String.md`,
-  `META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
-  and `.../notes.txt`;
-- one container image push that reuses a layer, which it does not try again.
-
-It does not send the raw file, history, blame or edit page of those four files,
-or the pages of the directories `META-INF/services` and `...`. It also sends a
-short sample of gitea's ordinary use:
-
-- signing in;
-- pages of files and commits, and a diff;
-- the API;
-- a push, and clone and fetch over HTTP;
-- two container image pushes.
+`make gitea-requests` runs gitea behind `rproxy` and `servewaf` at the defaults,
+without these exclusions. It reports each request the Core Rule Set refuses
+there.

docs/settings.md

@@ -192,6 +192,13 @@ chooses between the two (see "The image's command" below).
   the Core Rule Set reads, a size of at most `64K`. A question carries no more
   of a body than 64 KiB, and the Core Rule Set reads at most 64 KiB less one
   byte of it, whatever the value (see "The answer" in [`SPEC.md`](../SPEC.md)).
+- `SWWAF_CORE_RULE_SET_EXCLUSIONS_FILE` (default unset): a file of Core Rule Set
+  exclusions, written as Coraza directives. It is loaded before the Core Rule
+  Set's rules, so it holds only `SecAction` and `SecRule` lines that exclude
+  with `ctl:ruleRemoveTargetById` or `ctl:ruleRemoveById`. A file that cannot be
+  read or does not parse stops the start, whatever `SWWAF_WAF_MODE` is. So does
+  a file that names a number from 900000 to 900999 outside a comment, alone or
+  in a range. It has no `_FILE` form.
 - `SWWAF_ERROR_BURST_THRESHOLD` (default `30`): the most refusals of one client
   in a minute (see "Attack detection" in `SPEC.md`). A count or `off`.