deploy/gitea: Gitea behind traefik, rproxy and servewaf, with its test (closes #178)
check / check (push) Failing after 1m40s
check / check (push) Failing after 1m40s
deploy/gitea/compose.yaml runs traefik, stock Gitea 1.27.3, rproxy and servewaf on one network with a fixed subnet. Its containers skip servewaf's checks; the network's gateway is left out of that range. rproxy takes uploads of 2 GiB and 30 minutes, and cuts responses at 15 minutes. script/gitea-setup (make gitea-setup) brings the setup up and checks it. docs/gitea.md is rewritten around the setup. Judgement call: the Core Rule Set runs in detect mode, so it refuses nothing. Judgement call: traefik serves its own certificate; no certificate resolver. Partially verified: the image build failed three runs on Ubuntu's snapshot service, so the checks ran only against a stand-in image with the same binary. Model: opus-5-5
This commit is contained in:
+165
-49
@@ -1,51 +1,173 @@
|
||||
# smallwebwaf in front of gitea
|
||||
|
||||
These notes record what `smallwebwaf` does in front of gitea at the default
|
||||
settings. They are observations, not requirements; the requirements are in
|
||||
[`SPEC.md`](../SPEC.md). `script/gitea-requests` runs gitea behind `rproxy` and
|
||||
`servewaf`, and sends through them a sample of gitea's ordinary use and the
|
||||
`deploy/gitea/compose.yaml` runs gitea behind traefik, with `rproxy` and
|
||||
`servewaf` in front of it. This document says what that setup sets and why, and
|
||||
what the defaults would do without it. [`SPEC.md`](../SPEC.md) says how
|
||||
`smallwebwaf` behaves.
|
||||
|
||||
## The setup
|
||||
|
||||
Four containers share one Docker network, the shared network:
|
||||
|
||||
- traefik, which routes the site's HTTPS requests to `rproxy`, and redirects
|
||||
plain HTTP to HTTPS;
|
||||
- `rproxy`, in front of gitea;
|
||||
- gitea 1.27.3, as its image comes;
|
||||
- `servewaf`, with a volume for its state files.
|
||||
|
||||
To start it:
|
||||
|
||||
1. In `deploy/gitea/compose.yaml`, replace `git.example.org` with the site's
|
||||
host name.
|
||||
2. Replace each `10.99.0` with the first three numbers of a `/24` subnet that no
|
||||
other network on the host uses.
|
||||
3. Set `SMALLWEBWAF_IMAGE` to the `smallwebwaf` image, named by digest.
|
||||
4. Run `docker compose up --detach` in `deploy/gitea`.
|
||||
5. Make gitea's first user with `gitea admin user create`, in gitea's container,
|
||||
as the user `git`.
|
||||
|
||||
gitea starts at its defaults, with SQLite, instead of showing its install page
|
||||
to whoever opens it first. traefik answers with a certificate it made itself
|
||||
until you add a certificate resolver, as for your other sites. SSH to gitea does
|
||||
not pass through `smallwebwaf`; publish gitea's SSH port yourself if you use it.
|
||||
|
||||
`make gitea-setup` checks the setup. It builds the image, brings the setup up
|
||||
under a name of its own, and checks:
|
||||
|
||||
- that a page load, a push and a clone sent through traefik pass through
|
||||
`rproxy`, and that `servewaf` checks them;
|
||||
- that gitea logs the visitor's address, not `rproxy`'s;
|
||||
- that a request from a container on the shared network skips the checks;
|
||||
- that with `servewaf` stopped, gitea still answers through `rproxy`.
|
||||
|
||||
## Uploads and downloads
|
||||
|
||||
An upload is a request:
|
||||
|
||||
- a git push;
|
||||
- an LFS object;
|
||||
- a container image layer;
|
||||
- a package file;
|
||||
- a release attachment.
|
||||
|
||||
The setup lets an upload of up to 2 GiB, sent in up to 30 minutes, through:
|
||||
|
||||
- `SWRP_MAX_REQUEST_SIZE` is `2G`. At the default, an upload over 100 MiB is cut
|
||||
off.
|
||||
- `SWRP_MAX_REQUEST_TIME_MS` is `1800000`, 30 minutes. At the default, an upload
|
||||
that takes over 60 seconds is cut off.
|
||||
- traefik's `readTimeout` is 30 minutes. traefik's own default of 60 seconds for
|
||||
a whole request would cut off an upload first.
|
||||
|
||||
A clone or a download is a response:
|
||||
|
||||
- `SWRP_MAX_RESPONSE_TIME_MS` is `900000`, 15 minutes, half the default. A clone
|
||||
or download that takes longer is cut off.
|
||||
- `SWRP_MAX_RESPONSE_SIZE` stays at its default, 1 GiB. A repository whose clone
|
||||
is larger needs it raised, and the byte limits with it ("Byte limits, per
|
||||
client" in `SPEC.md`).
|
||||
|
||||
The byte limits stay at their defaults. Two uploads of 2 GiB from one address in
|
||||
one minute pass the minute's limit of 4 GiB, and ban the address.
|
||||
|
||||
## The checks
|
||||
|
||||
`SWWAF_WAF_MODE` is `detect`. The Core Rule Set then logs a match, with its rule
|
||||
ids, and refuses nothing. In `block` mode, the default, it refuses everyday
|
||||
gitea searches and file names, which "What the Core Rule Set refuses in `block`
|
||||
mode" below lists. Each such refusal counts toward the error burst, so 31 in one
|
||||
minute ban the address.
|
||||
|
||||
Every other check stays at its default. A client may make 500 requests a minute,
|
||||
20,000 an hour and 300,000 a day. Archive downloads and blame and history pages
|
||||
are what scrapers hammer, and these limits do most of the work there.
|
||||
|
||||
## The shared network
|
||||
|
||||
Containers on the shared network skip `servewaf`'s checks:
|
||||
|
||||
- The shared network's subnet is `10.99.0.0/24`. Docker gives its containers
|
||||
addresses in the upper half, `10.99.0.128/25`, which `SWWAF_ALLOW_NETS` names.
|
||||
- traefik is known on the shared network by the site's host name. So a container
|
||||
there that asks for the site reaches traefik from its own address. An
|
||||
automation fleet in such a container skips the checks.
|
||||
- The network's gateway, `10.99.0.1`, is the host's own address on it. Requests
|
||||
docker passes on from a published port can come from it, such as those of
|
||||
visitors over IPv6. So the gateway is left out of `SWWAF_ALLOW_NETS`.
|
||||
- A container on the shared network that asks for the site at the host's public
|
||||
address arrives from the gateway. It is checked like any visitor.
|
||||
- A fleet on another host can have its address added to `SWWAF_ALLOW_NETS`.
|
||||
Without that, the default request limits are sized for such a fleet.
|
||||
- traefik replaces the `X-Forwarded-For` a visitor sends, by default. Keep it
|
||||
so: otherwise a visitor could name an address on the shared network, and skip
|
||||
the checks.
|
||||
- Anything on the shared network can reach gitea and `servewaf` around `rproxy`.
|
||||
Put only containers you trust on it.
|
||||
|
||||
## gitea's settings
|
||||
|
||||
- `REVERSE_PROXY_TRUSTED_PROXIES` names `rproxy`'s fixed address, `10.99.0.2`.
|
||||
gitea then takes each visitor's address from the `X-Real-Ip` header, which
|
||||
traefik sets and `rproxy` passes on. Without it, gitea logs every visitor as
|
||||
`rproxy`.
|
||||
- `INSTALL_LOCK` is `true`, as "The setup" above says.
|
||||
|
||||
## Methods
|
||||
|
||||
`rproxy` passes on only these methods: `GET`, `HEAD`, `POST`, `OPTIONS`, `PUT`,
|
||||
`DELETE`, `PATCH` and `QUERY`. gitea 1.27.3 needs no other. Its routes use:
|
||||
|
||||
- pages and the API: `GET`, `HEAD`, `POST`, `PUT`, `PATCH` and `DELETE`;
|
||||
- git's push, clone and fetch over HTTP: `GET`, `POST` and `OPTIONS`;
|
||||
- LFS: `GET`, `POST` and `PUT`;
|
||||
- the container registry and the other package registries: `GET`, `HEAD`,
|
||||
`POST`, `PUT`, `PATCH` and `DELETE`;
|
||||
- Actions runners: `POST`.
|
||||
|
||||
## The log
|
||||
|
||||
`rproxy` and `servewaf` write each query string to the log whole. A token sent
|
||||
in the query string, as `?token=`, is then in the log. Send it in the
|
||||
`Authorization` header instead.
|
||||
|
||||
## Actions runners
|
||||
|
||||
A runner on the shared network skips the checks. One on another host is counted
|
||||
like any client. A runner older than version 0.4 (April 2026) asks for work
|
||||
every 2 seconds, 43,200 requests a day. While it runs a job, it reports the
|
||||
job's log and state every second instead, two requests a second. So one runner
|
||||
busy all day sends 120 requests a minute and 172,800 a day, under the request
|
||||
limits.
|
||||
|
||||
Several runners on one address can pass them:
|
||||
|
||||
- Two busy all day make 345,600 requests, and pass the day limit of 300,000.
|
||||
- Three busy at once make 21,600 an hour, and pass the hour limit of 20,000.
|
||||
- Five busy at once make 600 a minute, and pass the minute limit of 500.
|
||||
|
||||
Their address is then banned, their running jobs fail, and each repeat within a
|
||||
day makes the ban longer. Newer runners ask for work less often. Put the
|
||||
runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||
|
||||
## What the Core Rule Set refuses in `block` mode
|
||||
|
||||
These notes record what the Core Rule Set does in front of gitea in `block`
|
||||
mode, at the other defaults. They are observations, not requirements.
|
||||
`script/gitea-requests` runs gitea behind `rproxy` and `servewaf` at the
|
||||
defaults. It sends through them a sample of gitea's ordinary use and the
|
||||
requests named below. It reports each request the Core Rule Set refuses, so that
|
||||
these notes can be checked again when gitea or the Core Rule Set changes
|
||||
version.
|
||||
version. In `detect` mode, as the setup runs it, the same requests are logged
|
||||
with their rule ids, and pass.
|
||||
|
||||
## What the defaults cover
|
||||
Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts
|
||||
toward). The request log names the rule in `waf_rule_ids`, which
|
||||
`SWWAF_WAF_DISABLED_RULES` can switch off.
|
||||
|
||||
- SSH access to gitea does not pass through `smallwebwaf` and is not protected
|
||||
by it.
|
||||
- A clone is a response. It fits the defaults of 30 minutes and 1 GiB unless the
|
||||
repository is large or the link slow. A repository whose clone is larger than
|
||||
1 GiB needs `SWRP_MAX_RESPONSE_SIZE` raised, and the byte limits with it
|
||||
("Byte limits, per client" in `SPEC.md`).
|
||||
- A push is a request, and so is every other upload:
|
||||
- a git push;
|
||||
- an LFS object;
|
||||
- a container image layer;
|
||||
- a package file;
|
||||
- a release attachment.
|
||||
### What it lets through
|
||||
|
||||
At the defaults, an upload larger than 100 MiB or taking more than 60
|
||||
seconds is cut off ("`rproxy`'s limits" in `SPEC.md`). A gitea that takes
|
||||
large uploads needs `SWRP_MAX_REQUEST_SIZE` and `SWRP_MAX_REQUEST_TIME_MS`
|
||||
raised to fit.
|
||||
|
||||
- Archive download and blame or history pages are what scrapers hammer. The
|
||||
request limits do most of the work there.
|
||||
- An Actions runner that reaches gitea through `smallwebwaf` sends requests all
|
||||
day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
|
||||
43,200 requests a day. While it runs a job, it reports the job's log and state
|
||||
every second instead, two requests a second. So one runner busy all day sends
|
||||
120 requests a minute and 172,800 a day, under the request limits. Two such
|
||||
runners on one address, each busy all day, make 345,600 and pass the day limit
|
||||
of 300,000. Three busy at once make 21,600 an hour and pass the hour limit of
|
||||
20,000. Five busy at once make 600 a minute and pass the minute limit of 500.
|
||||
Their address is then banned, their running jobs fail, and each repeat within
|
||||
a day makes the ban longer. Newer runners ask for work less often. Put the
|
||||
runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`.
|
||||
|
||||
## What the Core Rule Set lets through
|
||||
|
||||
At the defaults, the Core Rule Set lets gitea's ordinary use through, apart from
|
||||
the refusals in the next section:
|
||||
The Core Rule Set lets gitea's ordinary use through, apart from the refusals in
|
||||
the sections after this one:
|
||||
|
||||
- browsing and views of files in a repository, with their history, blame and the
|
||||
file tree;
|
||||
@@ -60,12 +182,6 @@ the refusals in the next section:
|
||||
editor, code included, since no body is read;
|
||||
- the page gitea shows after a change, with its message naming what was done.
|
||||
|
||||
## What the Core Rule Set still refuses
|
||||
|
||||
Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts
|
||||
toward). The request log names the rule in `waf_rule_ids`, which
|
||||
`SWWAF_WAF_DISABLED_RULES` can switch off.
|
||||
|
||||
### A query string that reads as an attack
|
||||
|
||||
Most often this is a search. A search term is a query parameter, which the Core
|
||||
@@ -183,7 +299,7 @@ the runner. The error burst ("Attack detection" in `SPEC.md`) bans it after more
|
||||
than 30 such refusals from its address in a minute, as when many jobs upload at
|
||||
once. `actions/upload-artifact@v4` does not send the header.
|
||||
|
||||
## What body inspection refuses
|
||||
### What body inspection refuses
|
||||
|
||||
With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In
|
||||
front of gitea it then refuses these, until their rule ids, named in the request
|
||||
@@ -195,7 +311,7 @@ log, are added to `SWWAF_WAF_DISABLED_RULES`:
|
||||
- PyPI uploads (922130);
|
||||
- attachments named like `debug.log` or `config.yml` (932180).
|
||||
|
||||
## What `script/gitea-requests` sends
|
||||
### What `script/gitea-requests` sends
|
||||
|
||||
Of the requests above, the script sends:
|
||||
|
||||
|
||||
Reference in New Issue
Block a user