SPEC rework after review: each requirement once, gitea notes to docs/
check / check (push) Waiting to run
check / check (push) Waiting to run
Every requirement is stated once, where it belongs, and linked from elsewhere. The notes on gitea move to docs/gitea.md, which script/gitea-requests points at. The list of settings that take off is exact, the non-goal that contradicted the planned item is gone, and the sentences the review named are split or reordered. The package comments in internal/waf and internal/anomaly point at the renamed sections. Model: fable-5-1
This commit is contained in:
+198
@@ -0,0 +1,198 @@
|
|||||||
|
# smallwebwaf in front of gitea
|
||||||
|
|
||||||
|
These notes record what `smallwebwaf` does in front of gitea at the default
|
||||||
|
settings. They are observations, not requirements; the requirements are in
|
||||||
|
[`SPEC.md`](../SPEC.md). `script/gitea-requests` runs gitea behind the image and
|
||||||
|
sends through it a sample of gitea's ordinary use and the requests named below.
|
||||||
|
It reports each request the Core Rule Set refuses, so that these notes can be
|
||||||
|
checked again when gitea or the Core Rule Set changes version.
|
||||||
|
|
||||||
|
## What the defaults cover
|
||||||
|
|
||||||
|
- SSH access to gitea does not pass through `smallwebwaf` and is not protected
|
||||||
|
by it.
|
||||||
|
- A clone is a response. It fits the defaults of 30 minutes and 5 GiB for all
|
||||||
|
but the largest repositories and slowest links.
|
||||||
|
- A push is a request, and so is every other upload: a git push, an LFS object,
|
||||||
|
a container image layer, a package file or a release attachment. At the
|
||||||
|
defaults, an upload larger than 100 MiB or taking more than 60 seconds is cut
|
||||||
|
off. The client is answered `413` for a body that is too large, before
|
||||||
|
anything reaches gitea when the request announces its size. It is answered
|
||||||
|
`408` for a body it sends too slowly, or `504` if gitea is too slow to take
|
||||||
|
it. The upload fails, and no one is banned for it. A gitea that takes large
|
||||||
|
uploads needs `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and
|
||||||
|
`SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to fit. The Core Rule Set does not
|
||||||
|
read an upload's body, which streams through without being held in memory.
|
||||||
|
- Archive download and blame or history pages are what scrapers hammer. The
|
||||||
|
request limits do most of the work there.
|
||||||
|
- An Actions runner that reaches gitea through `smallwebwaf` sends requests all
|
||||||
|
day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
|
||||||
|
43,200 requests a day, and reports a running job's log and state every second.
|
||||||
|
So it passes the day limit of 50,000 after about an hour and a quarter of
|
||||||
|
jobs. It is then banned, the job it is running fails, and each repeat within a
|
||||||
|
day makes the ban longer. Newer runners ask less often, but a busy one, or
|
||||||
|
several on one address, can still pass it. Put the runners' addresses in
|
||||||
|
`SWWAF_RATE_LIMIT_EXEMPT_NETS`. That takes them out of the request and byte
|
||||||
|
limits, while the error burst, attack detection and bans still apply.
|
||||||
|
|
||||||
|
## What the Core Rule Set lets through
|
||||||
|
|
||||||
|
At the defaults, the Core Rule Set lets gitea's ordinary use through, apart from
|
||||||
|
the refusals in the next section:
|
||||||
|
|
||||||
|
- browsing and views of files in a repository, with their history, blame and the
|
||||||
|
file tree;
|
||||||
|
- diffs, including their hidden lines and large files, and pull request review;
|
||||||
|
- git's clone, fetch and push over HTTP;
|
||||||
|
- signing in, including the return to the page a visitor came from, and sign-in
|
||||||
|
with Git Credential Manager, git-credential-oauth or tea;
|
||||||
|
- the API's calls for a file and its commits;
|
||||||
|
- pushing and pulling container images and packages;
|
||||||
|
- Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`;
|
||||||
|
- posting issues, pull requests, comments, wiki pages and files saved in the web
|
||||||
|
editor, code included, since no body is read;
|
||||||
|
- the page gitea shows after a change, with its message naming what was done.
|
||||||
|
|
||||||
|
## What the Core Rule Set still refuses
|
||||||
|
|
||||||
|
Each refusal answers only that request, with `403`, and bans no one by itself.
|
||||||
|
It counts toward the error burst, which a person does not reach this way. The
|
||||||
|
request log names the rule in `waf_rule_ids`, which `SWWAF_WAF_DISABLED_RULES`
|
||||||
|
can switch off.
|
||||||
|
|
||||||
|
### A query string that reads as an attack
|
||||||
|
|
||||||
|
Most often this is a search. Every query parameter is checked against the Core
|
||||||
|
Rule Set's lists of system files and commands, except the ten named under
|
||||||
|
"Attack detection" in `SPEC.md`. A release attachment uploaded through the API
|
||||||
|
as `docker-compose.yml` is refused the same way. Examples:
|
||||||
|
|
||||||
|
- a name on those lists, such as `package.json`, `.gitignore` or
|
||||||
|
`docker-compose.yml`;
|
||||||
|
- text that starts with a command name, or whose first word has one right after
|
||||||
|
a `/`, such as `python3`, `ssh key` or `feat/docker-support`;
|
||||||
|
- a shell command with its options or a system path (`ls -la`, `sed -i`,
|
||||||
|
`/bin/sh`), or a command in backticks;
|
||||||
|
- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`) or an SQL
|
||||||
|
statement (`SELECT * FROM users WHERE`);
|
||||||
|
- a URL naming an IP address or `localhost`.
|
||||||
|
|
||||||
|
### A branch, tag or file name that starts with a command name
|
||||||
|
|
||||||
|
The Core Rule Set's list of commands (932260) holds names such as `docker-`,
|
||||||
|
`python3`, `ansible`, `base64` and `whoami`. A branch, tag or file name that
|
||||||
|
starts with one is refused. So is one that has such a name right after a `/`,
|
||||||
|
such as `feat/docker-support`, `fix/docker-build` or
|
||||||
|
`renovate/docker-build-push-action-6.x`. gitea's own pages send the name in a
|
||||||
|
query parameter that 930120, 932160 and 932260 still check:
|
||||||
|
|
||||||
|
- `refSubUrl`, the branch or tag of a directory listing. It is sent when the
|
||||||
|
listing asks for its entries' last commits in a second request, which it does
|
||||||
|
whenever gitea takes more than a second to work them out;
|
||||||
|
- `name`, when a branch is deleted or restored on the branches page;
|
||||||
|
- `tag`, when a release is started from a tag;
|
||||||
|
- `template`, the file of an issue template, when an issue is opened from it;
|
||||||
|
- `rule_name`, when a branch protection rule is opened for editing in the
|
||||||
|
repository's settings or has just been saved.
|
||||||
|
|
||||||
|
What a visitor sees, for a branch named `docker-build`:
|
||||||
|
|
||||||
|
- A directory listing that makes that second request leaves those last commits
|
||||||
|
out and shows an error.
|
||||||
|
- The branches page shows an error instead of deleting or restoring the branch.
|
||||||
|
- A release started from such a tag gets the `403` answer of `smallwebwaf` in
|
||||||
|
place of its form. So does an issue opened from such a template, and such a
|
||||||
|
rule opened for editing.
|
||||||
|
- Saving such a rule stores it. gitea then returns to the branch settings page,
|
||||||
|
which gets the `403` answer, so the save looks as if it failed.
|
||||||
|
|
||||||
|
### A file name in a query parameter that trips the code rules
|
||||||
|
|
||||||
|
The Core Rule Set's rules for Java, script, PowerShell and SQL find what they
|
||||||
|
look for in some file names. This holds in the parameters `smallwebwaf` leaves
|
||||||
|
out of 930120, 932160 and 932260 too (see the changes to the Core Rule Set under
|
||||||
|
"Attack detection" in `SPEC.md`). Examples:
|
||||||
|
|
||||||
|
- `runtime` in OpenJDK's `src/java.base/share/classes/java/lang/Runtime.java`
|
||||||
|
(944110);
|
||||||
|
- a Java class, as in `java.lang.String.md` (944130);
|
||||||
|
- `document.write.js` (941180);
|
||||||
|
- `process.env.d.ts` (934100);
|
||||||
|
- `__proto__` (934130);
|
||||||
|
- the PowerShell command `Install-Script`, as in `install-script.sh` (932120);
|
||||||
|
- a database's name, as in PostgreSQL's `information_schema.sql` (942140).
|
||||||
|
|
||||||
|
For such a file, these requests are refused:
|
||||||
|
|
||||||
|
- the API's list of its commits (`path`);
|
||||||
|
- the API's call for the contents of files that names it in its query (`body`);
|
||||||
|
- the commit graph of that file alone (`file`);
|
||||||
|
- the hidden lines of its diff (`path`), which stay hidden;
|
||||||
|
- the sign-in page gitea sends a visitor to on the way to a page of the file
|
||||||
|
that needs signing in, such as its edit page (`redirect_to`). The visitor can
|
||||||
|
still sign in by opening the sign-in page itself.
|
||||||
|
|
||||||
|
### A path
|
||||||
|
|
||||||
|
- A file name ending in `~`, or an `.xhtml` file whose path holds a space.
|
||||||
|
- A path that holds what the Core Rule Set takes for a Java class (944130), as
|
||||||
|
`java.lang.String.md` and
|
||||||
|
`META-INF/services/org.apache.commons.logging.LogFactory` do.
|
||||||
|
- A path that holds what it takes for Node.js code (934100), as
|
||||||
|
`process.env.d.ts` does.
|
||||||
|
- A path with a directory named `...` (930100, 930110), such as `.../notes.txt`.
|
||||||
|
|
||||||
|
For such a file, the page of the file, its raw file, its history, its blame and
|
||||||
|
its edit page are refused. The directory that holds it lists it, and a clone
|
||||||
|
carries it.
|
||||||
|
|
||||||
|
### A container image push that reuses a layer
|
||||||
|
|
||||||
|
A push can reuse a layer of an image docker has pushed to that gitea before.
|
||||||
|
docker then asks gitea to take the layer from that image, naming it in the query
|
||||||
|
parameter `from`, which 932260 checks. The request is refused when the image's
|
||||||
|
owner, or its name after the owner, starts with an entry on the list of
|
||||||
|
commands. Examples are `owner/python3-app`, `owner/docker-base`, and any image
|
||||||
|
of an owner named `ansible` or `docker-team`.
|
||||||
|
|
||||||
|
What happens next depends on docker's image store. A docker that keeps its
|
||||||
|
images in containerd's image store fails the push, and fails each new try the
|
||||||
|
same way. A new install of Docker Engine 29 does this. A docker that keeps them
|
||||||
|
in its own older image store uploads the layer instead. The push then succeeds
|
||||||
|
after one refusal for each layer it reuses, and each of those refusals counts
|
||||||
|
toward the error burst.
|
||||||
|
|
||||||
|
### Artifact uploads from `actions/upload-artifact@v3`
|
||||||
|
|
||||||
|
This action sends the header `Content-Range`, which the Core Rule Set refuses
|
||||||
|
(920450). The action sends two files at a time, does not retry a `403`, and
|
||||||
|
sends no further file once one is refused. So an upload is refused once or
|
||||||
|
twice, however many files it holds, and the step fails. One upload does not ban
|
||||||
|
the runner. More than 30 such refusals from its address within a minute, as when
|
||||||
|
many jobs upload at once, ban it through the error burst.
|
||||||
|
`actions/upload-artifact@v4` does not send the header.
|
||||||
|
|
||||||
|
## What `script/gitea-requests` sends
|
||||||
|
|
||||||
|
Of the requests above, the script sends:
|
||||||
|
|
||||||
|
- for each file named under "A file name in a query parameter that trips the
|
||||||
|
code rules", the API's list of its commits;
|
||||||
|
- for `Runtime.java` alone, the commit graph of that file, the hidden lines of
|
||||||
|
its diff, and the API's call for the contents of files that names it;
|
||||||
|
- the sign-in page gitea sends a visitor to on the way to the edit page of
|
||||||
|
`document.write.js`;
|
||||||
|
- the pages of `java.lang.String.md`,
|
||||||
|
`META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
|
||||||
|
and `.../notes.txt`;
|
||||||
|
- one container image push that reuses a layer, which it does not try again.
|
||||||
|
|
||||||
|
It does not send the raw file, history, blame or edit page of those four files,
|
||||||
|
or the pages of the directories `META-INF/services` and `...`. It also sends a
|
||||||
|
short sample of gitea's ordinary use:
|
||||||
|
|
||||||
|
- signing in;
|
||||||
|
- pages of files and commits, and a diff;
|
||||||
|
- the API;
|
||||||
|
- a push, and clone and fetch over HTTP;
|
||||||
|
- two container image pushes.
|
||||||
@@ -1,8 +1,8 @@
|
|||||||
// Package anomaly counts requests and bytes over a minute and an hour, per
|
// Package anomaly counts requests and bytes over a minute and an hour, per
|
||||||
// client, per surrounding netblock, per AS number, for the whole service
|
// client, per surrounding netblock, per AS number, for the whole service
|
||||||
// and per named netblock, and raises an anomaly alert for a count over its
|
// and per named netblock, and raises an anomaly alert for a count over its
|
||||||
// threshold, as "Anomaly thresholds" under "Configuration surface" in
|
// threshold, as "Anomaly thresholds" under "Settings" in SPEC.md
|
||||||
// SPEC.md describes. It refuses and bans nothing. At most 20,000 counters
|
// describes. It refuses and bans nothing. At most 20,000 counters
|
||||||
// are kept, in memory, and written to alerts.json and read from it by the
|
// are kept, in memory, and written to alerts.json and read from it by the
|
||||||
// state package.
|
// state package.
|
||||||
package anomaly
|
package anomaly
|
||||||
|
|||||||
+2
-2
@@ -1,8 +1,8 @@
|
|||||||
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
||||||
// method, the URL with its query and the headers of a request, and on its
|
// method, the URL with its query and the headers of a request, and on its
|
||||||
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
|
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
|
||||||
// makes to it, as "Attack detection" under "Configuration surface" in
|
// makes to it, as "Attack detection" under "Settings" in SPEC.md describes
|
||||||
// SPEC.md describes them. It reads no response.
|
// them. It reads no response.
|
||||||
//
|
//
|
||||||
// smallwebwaf writes only to its state directory, so Coraza is built with
|
// smallwebwaf writes only to its state directory, so Coraza is built with
|
||||||
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
||||||
|
|||||||
@@ -2,11 +2,12 @@
|
|||||||
# script/gitea-requests: build the image and run gitea 1.27.3 behind it,
|
# script/gitea-requests: build the image and run gitea 1.27.3 behind it,
|
||||||
# with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
|
# with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
|
||||||
# which names gitea's container. Then send through it the gitea requests
|
# which names gitea's container. Then send through it the gitea requests
|
||||||
# that reading gitea's source found the Core Rule Set refuses, and a short
|
# that reading gitea's source found the Core Rule Set refuses, which
|
||||||
# sample of gitea's ordinary use: signing in, pages of files and commits,
|
# docs/gitea.md records, and a short sample of gitea's ordinary use:
|
||||||
# a diff, the API, a push, clone and fetch over HTTP, and two container
|
# signing in, pages of files and commits, a diff, the API, a push, clone
|
||||||
# image pushes. Last, print each request smallwebwaf logged, with what it
|
# and fetch over HTTP, and two container image pushes. Last, print each
|
||||||
# did with it and the ids of the Core Rule Set's rules that matched it.
|
# request smallwebwaf logged, with what it did with it and the ids of the
|
||||||
|
# Core Rule Set's rules that matched it.
|
||||||
# Every request comes from one client, docker's gateway, so more than
|
# Every request comes from one client, docker's gateway, so more than
|
||||||
# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
|
# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
|
||||||
# the requests after would show as banned. The containers, the network,
|
# the requests after would show as banned. The containers, the network,
|
||||||
|
|||||||
Reference in New Issue
Block a user