From 3543a9f616046b424d1ac70b91528be2680d25b6 Mon Sep 17 00:00:00 2001 From: sneak Date: Sat, 7 Nov 2026 01:57:48 +0000 Subject: [PATCH] Remove what is about deploying behind Gitea (closes #209) Delete the Gitea compose setup, its document, its two scripts and their make targets. Documents and tests no longer use Gitea as the example deployment. The Core Rule Set exclusions for Gitea's false positives in internal/waf stay, with their comments and tests. Model: opus-5-5 --- EVALUATION.md | 6 +- Makefile | 12 +- README.md | 105 +------ SPEC.md | 54 ++-- deploy/gitea/compose.yaml | 135 --------- docs/gitea.md | 340 ---------------------- docs/rules.md | 6 +- docs/settings.md | 13 +- internal/alerts/alerts_test.go | 20 +- internal/config/config.go | 2 +- internal/config/config_test.go | 12 +- internal/remotelog/remotelog_test.go | 8 +- internal/requestlog/requestlog_test.go | 6 +- internal/servewaf/alerts_test.go | 2 +- internal/servewaf/requestlog_test.go | 2 +- internal/smallwebwaf/smallwebwaf_test.go | 28 +- internal/state/state_test.go | 6 +- script/gitea-requests | 350 ----------------------- script/gitea-setup | 216 -------------- 19 files changed, 87 insertions(+), 1236 deletions(-) delete mode 100644 deploy/gitea/compose.yaml delete mode 100644 docs/gitea.md delete mode 100755 script/gitea-requests delete mode 100755 script/gitea-setup diff --git a/EVALUATION.md b/EVALUATION.md index 4ea3fc9..4214787 100644 --- a/EVALUATION.md +++ b/EVALUATION.md @@ -153,15 +153,15 @@ Closest existing options, and why each still falls short: - What it is: a single-binary reverse proxy that makes browsers solve a proof-of-work challenge before passing them to `TARGET`. Aimed at scrapers, which is likely a large share of unwanted traffic on a public - gitea. + code forge. - Covers: R10 well (one container, env vars for listener, target, difficulty, cookies). Policy rules can match on path, user agent, headers, IP ranges, and with the vendor's hosted data service also AS number and country, and can weigh a request toward a harder challenge. - Misses: R1, R3, R4, R5, R6, R9. Bot policy needs a YAML file, not env vars. AS number and country matching depend on the vendor's hosted - service. Breaks non-browser clients unless paths are exempted; for gitea, - git-over-HTTP and API paths must be allowed through by rule. + service. Breaks non-browser clients unless paths are exempted; for a code + forge, git-over-HTTP and API paths must be allowed through by rule. - Maturity: very active, widely deployed on code forges since 2025. - Value here: complementary. It can be chained (traefik, then the sidecar, then Anubis, then the app) if challenge pages are wanted. diff --git a/Makefile b/Makefile index d23221e..6f3d484 100644 --- a/Makefile +++ b/Makefile @@ -1,13 +1,11 @@ .PHONY: bootstrap setup test lint fmt fmt-check tidy check docker hooks build \ - run example-app gitea-requests gitea-setup + run example-app # Makefile targets are thin shims; the implementations live in script/ # per the scripts-to-rule-them-all pattern (see the Entrypoints section # of README.md). tidy writes go.mod and go.sum as `go mod tidy` does, # which test checks. build and run are for working on the code by hand; -# example-app checks the image with an app built on it; gitea-requests -# reports the gitea requests the Core Rule Set refuses at the defaults; -# gitea-setup checks the setup in deploy/gitea. +# example-app checks the image with an app built on it. bootstrap: @script/bootstrap @@ -47,9 +45,3 @@ run: example-app: @script/example-app - -gitea-requests: - @script/gitea-requests - -gitea-setup: - @script/gitea-setup diff --git a/README.md b/README.md index bf6cdb6..ea241da 100644 --- a/README.md +++ b/README.md @@ -100,8 +100,8 @@ Point traefik's labels at port 8080 of `rproxy` of the app or of `servewaf`. Anything on that network can reach the app around `rproxy`, and can ask `servewaf` questions, so the network holds only what you trust. Each further app adds its own container and its own `rproxy`, whose -`SWRP_WAF_BACKEND` names the same `servewaf`. The "Smallest setup" and "In front -of gitea" examples below show this shape. +`SWRP_WAF_BACKEND` names the same `servewaf`. The "Smallest setup" example below +shows this shape. ### `rproxy` in the app's container @@ -338,9 +338,8 @@ full, and the limits that are fixed rather than settings. Each example is a complete `compose.yaml`. `smallwebwaf` is the tag `make docker` gives the image; on another host, use the name you pushed it under, by digest. Docker Compose puts the services on a private network of its -own, where each reaches the others by service name. Only "In front of gitea" -uses traefik. The other three publish `rproxy`'s port on the host, for trying it -out. +own, where each reaches the others by service name. Each publishes `rproxy`'s +port on the host, for trying it out. ### Smallest setup @@ -426,88 +425,6 @@ volumes: servewaf-state: ``` -### In front of gitea - -gitea behind traefik, set up for a code forge: - -- the upload limits raised, in size and in time; -- the response size limit raised, for clones over 1 GiB; -- the Actions runners exempt from counting; -- the default request and byte limits kept. - -`traefik` is the network traefik is on, made outside this file. - -```yaml -services: - gitea: - # gitea 1.27.3, 2026-08-29 - image: gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef - environment: - GITEA__server__ROOT_URL: https://git.example/ - # The host and the port of the SSH clone URLs gitea shows. - GITEA__server__SSH_DOMAIN: git.example - GITEA__server__SSH_PORT: "2222" - # gitea's TCP peer is rproxy, on the private network. - GITEA__security__REVERSE_PROXY_TRUSTED_PROXIES: 10.0.0.0/8,172.16.0.0/12,192.168.0.0/16 - volumes: - - gitea-data:/data - networks: - - site - ports: - # SSH does not pass through smallwebwaf. - - "2222:22" - - rproxy: - image: smallwebwaf - environment: - SWRP_WAF_BACKEND: http://servewaf:8080 - SWRP_UPSTREAM_URL: http://gitea:3000 - # Pushes, LFS objects and image layers over 100 MiB, or slower - # than a minute, are cut off at the defaults. - SWRP_MAX_REQUEST_SIZE: 2G - SWRP_MAX_REQUEST_TIME_MS: "600000" - # A clone larger than 1 GiB is cut off at the default. - SWRP_MAX_RESPONSE_SIZE: 4G - networks: - - site - - traefik - labels: - traefik.enable: "true" - # rproxy is on two networks; traefik reaches it on this one. - traefik.docker.network: traefik - traefik.http.routers.git.rule: Host(`git.example`) - traefik.http.routers.git.entrypoints: websecure - traefik.http.routers.git.tls.certresolver: letsencrypt - traefik.http.services.git.loadbalancer.server.port: "8080" - - servewaf: - image: smallwebwaf - environment: - SMALLWEBWAF_COMMAND: servewaf - SWWAF_INSTANCE_NAME: fsn1/servewaf - # The Actions runners ask for work all day; never count them. - SWWAF_RATE_LIMIT_EXEMPT_NETS: 10.0.4.0/24 - # gitea never serves these; a request for one bans the client. - SWWAF_TRAP_PATHS: /wp-login.php,/xmlrpc.php,/wp-admin/ - volumes: - - servewaf-state:/var/lib/smallwebwaf - networks: - - site - -networks: - site: - traefik: - external: true - -volumes: - gitea-data: - servewaf-state: -``` - -`deploy/gitea` holds a complete setup of gitea, traefik included, as a Docker -Compose file. [`docs/gitea.md`](docs/gitea.md) explains it, and records what the -defaults do in front of gitea, request by request. - ### `rproxy` beside the app The app is built on the `smallwebwaf` image, with `rproxy` inside its container. @@ -771,18 +688,6 @@ on the host. - orderly stops with `sv stop` and `docker stop`; - bans surviving a new container on the same volume; - a setting given as a file. -- `script/gitea-requests`: builds the image and runs gitea 1.27.3 behind - `rproxy` and `servewaf`, three containers on one network. Every setting is at - its default, but for those that join the containers and - `SWWAF_LOG_REQUEST_HEADERS`, which adds the header that labels each request. - It sends through them a sample of gitea's ordinary use and the requests the - Core Rule Set is known to refuse, and prints what `servewaf` did with each. - Needs network access, and a docker daemon that pushes to a registry on - `localhost` over plain HTTP. `make gitea-requests`. -- `script/gitea-setup`: builds the image, brings up the setup in `deploy/gitea`, - and checks it as [`docs/gitea.md`](docs/gitea.md) describes. Then it removes - the containers, the network, the volumes and the image it built. Needs network - access. `make gitea-setup`. ## TODO @@ -806,8 +711,6 @@ on the host. - [`docs/state-files.md`](docs/state-files.md): the state files and what each holds. - [`docs/image.md`](docs/image.md): what the image holds and how it is pinned. -- [`docs/gitea.md`](docs/gitea.md): `smallwebwaf` in front of gitea: the setup - in `deploy/gitea`, and what the defaults do there. ## License diff --git a/SPEC.md b/SPEC.md index 875ba19..f9358e7 100644 --- a/SPEC.md +++ b/SPEC.md @@ -402,10 +402,10 @@ scrapers that send many requests slowly over hours or days. The hour limit is the main one. - Per minute: 120, 2 requests a second. It leaves room for a person opening two - pages at once with an empty cache. A gitea page opened with an empty cache + pages at once with an empty cache. A web app's page opened with an empty cache loads 20 to 60 files, and a handful once cached. -- Per hour: 1,500. A person reading code in gitea for a whole hour, a page every - 20 seconds, makes about 900 requests. A scraper making one request every 2 +- Per hour: 1,500. A person reading a site for a whole hour, a page every 20 + seconds, makes about 900 requests. A scraper making one request every 2 seconds makes 1,800 an hour, more than the limit. - Per day: 6,000, four hours at the hour limit. A scraper making one request every 10 seconds makes 8,640 a day, more than the limit. @@ -417,7 +417,7 @@ the blocklists. An abuser that trips none of them is not caught. Some legitimate clients pass the limits: -- An Actions runner asks gitea for work all day and can pass the limits. +- A CI runner asks its code forge for work all day and can pass the limits. - Everyone behind one shared address, such as an office, shares one client's limits. @@ -550,12 +550,11 @@ service's abuse comes from. By default the Core Rule Set reads the method, the URL with its query string, and the headers of every request. That is where automated attacks show: injection in query strings, path traversal, attacks carried in headers, -scanners' user agents. It reads no request body. In front of a code forge such -as gitea, bodies carry what people write and publish, such as issue text and -files saved in the web editor. The Core Rule Set takes the shell commands and -code in them for attacks. The default gives up refusing an attack carried in a -body. The rule files, the limits and the bans still apply to the client that -sends it. +scanners' user agents. It reads no request body. In front of a code forge, +bodies carry what people write and publish, such as issue text and files saved +in the web editor. The Core Rule Set takes the shell commands and code in them +for attacks. The default gives up refusing an attack carried in a body. The rule +files, the limits and the bans still apply to the client that sends it. - The Core Rule Set skips a path under `SWWAF_WAF_EXEMPT_PATHS`. - `SWWAF_WAF_PARANOIA_LEVEL` and `SWWAF_WAF_ANOMALY_THRESHOLD` are the Core Rule @@ -591,15 +590,14 @@ sends it. data, where binary content such as a git push trips rules written for text. A body the Core Rule Set cannot parse adds 5 to the score, and so does a multipart body cut by the limit in the middle of a part's header. Body - inspection suits apps whose forms carry no code; `docs/gitea.md` records what - it refuses in front of gitea. + inspection suits apps whose forms carry no code. - `SWWAF_TRAP_PATHS` names paths the app never serves and only scanners ask for, - such as `/wp-login.php` in front of gitea. This is the short form of a `path` - rule with the `ban` action, for deployments that mount no rule files. A trap - path is matched whole, without the query, against the path as sent and once - percent-decoded: `/wp-login.php` matches `/wp-login.php?x=1` and - `/%77p-login.php`, and does not match `/wp-login.php/` or - `/blog/wp-login.php`. + such as `/wp-login.php` in front of an app that is not WordPress. This is the + short form of a `path` rule with the `ban` action, for deployments that mount + no rule files. A trap path is matched whole, without the query, against the + path as sent and once percent-decoded: `/wp-login.php` matches + `/wp-login.php?x=1` and `/%77p-login.php`, and does not match `/wp-login.php/` + or `/blog/wp-login.php`. - `SWWAF_ERROR_BURST_THRESHOLD` counts a client's refusals per minute of these kinds, and of no other: - a rule file match; @@ -609,8 +607,8 @@ sends it. More than this breaks a limit (see "Bans"). A client trying one attack after another is refused many times a minute; a person rarely more than a few times. Answers the app gives are not counted, since they do not tell a - scanner from an ordinary client. In front of gitea, registry and git clients - get `404` and `401` by design, hundreds of times a minute. + scanner from an ordinary client. In front of a code forge, registry and git + clients get `404` and `401` by design, hundreds of times a minute. `smallwebwaf` changes the Core Rule Set 4.25.0 in six ways, since in front of gitea it would otherwise refuse ordinary requests. The changes hold in front of @@ -836,7 +834,7 @@ and the default file. runs. It reads the files again once the directory has had no change for 2 seconds, so that a file still being written is read whole. A file whose name starts with `.`, such as an editor's lock file, is not a rule file. -- Files are read in name order (`00-default.rules` before `50-gitea.rules`), +- Files are read in name order (`00-default.rules` before `50-site.rules`), rules in line order. - A rule's regex is Go's RE2 syntax. It has no backreferences or lookaround; in exchange matching time is linear in the input, so no rule can be made to stall @@ -865,14 +863,15 @@ and the default file. - The image ships a default file, kept short and limited to patterns that are wrong for every app, with its path rules anchored at the site root. In front - of gitea, `/.env` is a probe, while + of a code forge, `/.env` is a probe, while `///src/branch/main/.env.example` is a file in a repository that any visitor may open. Anchor the paths of your own `ban` rules at the site root too. Anything else, such as a rule banning WordPress probes in front of - gitea, belongs in a file the deployer adds to the image `servewaf` runs from, - or mounts. A rule has one target, so a `path` rule applies on every site - behind that `servewaf`. A directory mounted over `SWWAF_RULES_DIR` replaces - the default file; single files mounted into it add to it. + an app that is not WordPress, belongs in a file the deployer adds to the image + `servewaf` runs from, or mounts. A rule has one target, so a `path` rule + applies on every site behind that `servewaf`. A directory mounted over + `SWWAF_RULES_DIR` replaces the default file; single files mounted into it add + to it. - Rules meant to be edited while `servewaf` runs belong in a mounted directory, not a mounted file. Many editors save a file by replacing it, and a container keeps seeing the old one when the file alone is mounted. The lookup database @@ -1695,8 +1694,7 @@ with no change to upaas. upaas builds the app's Dockerfile and runs the one container as it runs any app. The traefik labels route to port 8080, and the `SWRP_` settings go with the app's environment variables. `servewaf` runs in a container of its own, on a network the app's container can reach, with its state -volume and the directory of any token file. `docs/gitea.md` records what the -defaults do in front of gitea, and `script/gitea-requests` checks those notes. +volume and the directory of any token file. ## Planned, not built diff --git a/deploy/gitea/compose.yaml b/deploy/gitea/compose.yaml deleted file mode 100644 index d5652a7..0000000 --- a/deploy/gitea/compose.yaml +++ /dev/null @@ -1,135 +0,0 @@ -# Gitea behind traefik, with rproxy and servewaf in front of it. -# docs/gitea.md says what each setting here does and why. Before starting it: -# - set SMALLWEBWAF_IMAGE to the smallwebwaf image, named by digest; -# - replace git.example.org with the site's host name; -# - replace each 10.99.0 with the first three numbers of a /24 subnet that -# no other network on the host uses. The /24 must lie inside 10.0.0.0/8, -# 172.16.0.0/12 or 192.168.0.0/16, because rproxy believes -# X-Forwarded-For only from those by default. Replace each fd10:99 with -# the first two hextets of an unused /64 subnet whose third and fourth -# hextets are 0, as in fd10:99::/64; -# - set GITEA_SECRET_KEY to the output of -# `docker run --rm gitea generate secret SECRET_KEY`, -# once per install, and keep it. -# Then run `docker compose up --detach` in this directory. -# script/gitea-setup checks this setup. - -name: gitea - -x-logging: &logging - driver: json-file - options: - max-size: 10m - max-file: "5" - -services: - traefik: - # traefik v3.7.17, 2026-11-03 - image: traefik@sha256:68ae5fe5b64ca166704b746a9e8afcd385e36b3f569a5691401bf652d57b3748 - # traefik replaces the X-Forwarded-For header a visitor sends. Keep it - # so: otherwise a visitor could name an address on the shared network, - # and skip servewaf's checks. - command: - - --providers.docker.exposedbydefault=false - - --entrypoints.web.address=:80 - - --entrypoints.web.http.redirections.entrypoint.to=websecure - - --entrypoints.websecure.address=:443 - # Until a certificate resolver is added here, as for your other - # sites, traefik answers with a certificate it made itself. - - --entrypoints.websecure.http.tls=true - # traefik's own limit of 60 seconds for a whole request would cut - # off an upload long before rproxy's limit of 30 minutes. - - --entrypoints.websecure.transport.respondingTimeouts.readTimeout=30m - ports: - - 80:80 - - 443:443 - volumes: - - /var/run/docker.sock:/var/run/docker.sock:ro - networks: - shared: - # Containers on the shared network find traefik under the - # site's name, and so reach it from their own address. - aliases: - - git.example.org - restart: unless-stopped - - gitea: - # gitea 1.27.3, 2026-08-29 - image: gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef - environment: - GITEA__server__ROOT_URL: https://git.example.org/ - # Gitea starts at its defaults, with SQLite, instead of showing its - # install page to whoever opens it first. - GITEA__security__INSTALL_LOCK: "true" - GITEA__security__SECRET_KEY: ${GITEA_SECRET_KEY:?set GITEA_SECRET_KEY, see docs/gitea.md} - # Accounts are made with `gitea admin user create`. - GITEA__service__DISABLE_REGISTRATION: "true" - # rproxy's addresses. Gitea then takes each visitor's address from - # the X-Real-Ip header traefik sets and rproxy passes on. - GITEA__security__REVERSE_PROXY_TRUSTED_PROXIES: 10.99.0.2,fd10:99::2 - volumes: - - gitea-data:/data - networks: - - shared - restart: unless-stopped - - rproxy: - image: ${SMALLWEBWAF_IMAGE:?set SMALLWEBWAF_IMAGE to the smallwebwaf image} - environment: - SWRP_WAF_BACKEND: http://servewaf:8080 - SWRP_UPSTREAM_URL: http://gitea:3000 - # Pushes, LFS objects, packages and release attachments of up to - # 2 GiB, sent in up to 30 minutes. - SWRP_MAX_REQUEST_SIZE: 2G - SWRP_MAX_REQUEST_TIME_MS: "1800000" - # 15 minutes for a clone or a download, half the default. - SWRP_MAX_RESPONSE_TIME_MS: "900000" - labels: - traefik.enable: "true" - traefik.http.routers.gitea.rule: Host(`git.example.org`) - traefik.http.routers.gitea.entrypoints: websecure - traefik.http.services.gitea.loadbalancer.server.port: "8080" - networks: - shared: - ipv4_address: 10.99.0.2 - ipv6_address: fd10:99::2 - logging: *logging - restart: unless-stopped - - servewaf: - image: ${SMALLWEBWAF_IMAGE:?set SMALLWEBWAF_IMAGE to the smallwebwaf image} - environment: - SMALLWEBWAF_COMMAND: servewaf - # The containers on the shared network skip the checks. Add the - # address of an automation fleet on another host here too; without - # that, it is counted like any client. - SWWAF_ALLOW_NETS: 10.99.0.128/25,fd10:99:0:0:8000::/65 - # In block mode the Core Rule Set refuses everyday Gitea searches - # and file names, and 31 refusals in a minute ban the address. - SWWAF_WAF_MODE: detect - volumes: - - servewaf-state:/var/lib/smallwebwaf - networks: - - shared - logging: *logging - restart: unless-stopped - -networks: - # The network traefik and Gitea share. Its containers get addresses in the - # upper half of each subnet, which servewaf lets skip the checks. Each - # subnet's gateway, the host's own address on it, stays out of that half: - # requests docker passes on from a published port can come from it. - shared: - enable_ipv6: true - ipam: - config: - - subnet: 10.99.0.0/24 - ip_range: 10.99.0.128/25 - gateway: 10.99.0.1 - - subnet: fd10:99::/64 - ip_range: fd10:99:0:0:8000::/65 - gateway: fd10:99::1 - -volumes: - gitea-data: - servewaf-state: diff --git a/docs/gitea.md b/docs/gitea.md deleted file mode 100644 index fde4cca..0000000 --- a/docs/gitea.md +++ /dev/null @@ -1,340 +0,0 @@ -# smallwebwaf in front of gitea - -`deploy/gitea/compose.yaml` runs gitea behind traefik, with `rproxy` and -`servewaf` in front of it. This document says what that setup sets and why, and -what the defaults would do without it. [`SPEC.md`](../SPEC.md) says how -`smallwebwaf` behaves. - -## The setup - -Four containers share one Docker network, the shared network: - -- traefik, which routes the site's HTTPS requests to `rproxy`, and redirects - plain HTTP to HTTPS; -- `rproxy`, in front of gitea; -- gitea 1.27.3, as its image comes; -- `servewaf`, with a volume for its state files. - -To start it: - -1. In `deploy/gitea/compose.yaml`, replace `git.example.org` with the site's - host name. -2. Replace each `10.99.0` with the first three numbers of a `/24` subnet that no - other network on the host uses. The `/24` must lie inside `10.0.0.0/8`, - `172.16.0.0/12` or `192.168.0.0/16`, because `rproxy` believes - `X-Forwarded-For` only from those by default. Replace each `fd10:99` with the - first two hextets of an unused `/64` subnet whose third and fourth hextets - are 0, as in `fd10:99::/64`. -3. Set `SMALLWEBWAF_IMAGE` to the `smallwebwaf` image, named by digest. -4. Set `GITEA_SECRET_KEY` to the output of - `docker run --rm gitea generate secret SECRET_KEY`, with - gitea's image as `deploy/gitea/compose.yaml` names it. Do this once per - install, and keep the key. -5. Run `docker compose up --detach` in `deploy/gitea`. -6. Make gitea's first user with `gitea admin user create`, in gitea's container, - as the user `git`. - -Sign-up is off: accounts are made with `gitea admin user create`. gitea starts -at its defaults, with SQLite. traefik answers with a certificate it made itself -until you add a certificate resolver, as for your other sites. SSH to gitea does -not pass through `smallwebwaf`; publish gitea's SSH port yourself if you use it. - -`make gitea-setup` checks the setup. It builds the image, brings the setup up -under a name of its own, and checks: - -- that a page load, a push and a clone sent through traefik pass through - `rproxy`, and that `servewaf` checks them; -- that gitea logs the visitor's address, not `rproxy`'s; -- that a request from a container on the shared network skips the checks; -- that with `servewaf` stopped, gitea still answers through `rproxy`. - -## Uploads and downloads - -An upload is a request: - -- a git push; -- an LFS object; -- a container image layer; -- a package file; -- a release attachment. - -The setup lets an upload of up to 2 GiB, sent in up to 30 minutes, through: - -- `SWRP_MAX_REQUEST_SIZE` is `2G`. At the default, an upload over 100 MiB is cut - off. -- `SWRP_MAX_REQUEST_TIME_MS` is `1800000`, 30 minutes. At the default, an upload - that takes over 60 seconds is cut off. -- traefik's `readTimeout` is 30 minutes. traefik's own default of 60 seconds for - a whole request would cut off an upload first. - -A clone or a download is a response: - -- `SWRP_MAX_RESPONSE_TIME_MS` is `900000`, 15 minutes, half the default. A clone - or download that takes longer is cut off. -- `SWRP_MAX_RESPONSE_SIZE` stays at its default, 1 GiB. A repository whose clone - is larger needs it raised. - -The byte limits stay at their defaults: no minute limit, 50 GiB an hour and 100 -GiB a day. - -## The checks - -`SWWAF_WAF_MODE` is `detect`. The Core Rule Set then logs a match, with its rule -ids, and refuses nothing. In `block` mode, the default, it refuses everyday -gitea searches and file names. The section "What the Core Rule Set refuses in -`block` mode" below lists them. Each such refusal counts toward the error burst, -so 31 in one minute ban the address. - -Every other check stays at its default. A client may make 120 requests a minute, -1,500 an hour and 6,000 a day. Archive downloads and blame and history pages are -what scrapers hammer, and these limits do most of the work there. - -## The shared network - -Containers on the shared network skip `servewaf`'s checks: - -- The shared network's subnets are `10.99.0.0/24` and `fd10:99::/64`. Docker - gives its containers addresses in the upper half of each, `10.99.0.128/25` and - `fd10:99:0:0:8000::/65`, which `SWWAF_ALLOW_NETS` names. -- traefik is known on the shared network by the site's host name. So a container - there that asks for the site reaches traefik from its own address. An - automation fleet in such a container skips the checks. -- Each subnet's gateway, `10.99.0.1` and `fd10:99::1`, is the host's own address - on it. Requests docker passes on from a published port can come from the - gateway, so the gateways are left out of `SWWAF_ALLOW_NETS`. The network has - IPv6, so a visitor over IPv6 reaches traefik from its own address, and - `servewaf` checks it as itself. -- A container on the shared network that asks for the site at the host's public - address arrives from the gateway. It is checked like any visitor. -- A fleet on another host can have its address added to `SWWAF_ALLOW_NETS`. - Without that, it is counted like any client. -- traefik replaces the `X-Forwarded-For` a visitor sends, by default. Keep it - so: otherwise a visitor could name an address on the shared network, and skip - the checks. -- Anything on the shared network can reach gitea and `servewaf` around `rproxy`. - Put only containers you trust on it. - -## gitea's settings - -- `REVERSE_PROXY_TRUSTED_PROXIES` names `rproxy`'s fixed addresses, `10.99.0.2` - and `fd10:99::2`. gitea then takes each visitor's address from the `X-Real-Ip` - header, which traefik sets and `rproxy` passes on. Without it, gitea logs - every visitor as `rproxy`. -- `INSTALL_LOCK` is `true`, as "The setup" above says. - -## Methods - -`rproxy` passes on only these methods: `GET`, `HEAD`, `POST`, `OPTIONS`, `PUT`, -`DELETE`, `PATCH` and `QUERY`. gitea 1.27.3 needs no other. Its routes use: - -- pages and the API: `GET`, `HEAD`, `POST`, `PUT`, `PATCH` and `DELETE`; -- git's push, clone and fetch over HTTP: `GET`, `POST` and `OPTIONS`; -- LFS: `GET`, `POST` and `PUT`; -- the container registry and the other package registries: `GET`, `HEAD`, - `POST`, `PUT`, `PATCH` and `DELETE`; -- Actions runners: `POST`. - -## The log - -`rproxy` and `servewaf` write each query string to the log whole. A token sent -in the query string, as `?token=`, is then in the log. Send it in the -`Authorization` header instead. - -## Actions runners - -A runner on the shared network skips the checks. One on another host is counted -like any client. A runner older than version 0.4 (April 2026) asks for work -every 2 seconds, 1,800 requests an hour. While it runs a job, it reports the -job's log and state every second instead, two requests a second. - -So one idle runner makes 1,800 requests an hour, more than the hour limit of -1,500. A busy one, at two requests a second, passes the minute limit of 120 -within two minutes. Its address is then banned, its running jobs fail, and each -repeat within a day makes the ban longer. Newer runners ask for work less often. -Put the runners' addresses in `SWWAF_RATE_LIMIT_EXEMPT_NETS`. - -## What the Core Rule Set refuses in `block` mode - -These notes record what the Core Rule Set does in front of gitea in `block` -mode, at the other defaults. They are observations, not requirements. -`script/gitea-requests` runs gitea behind `rproxy` and `servewaf` at the -defaults. It sends through them a sample of gitea's ordinary use and the -requests named below. It reports each request the Core Rule Set refuses, so that -these notes can be checked again when gitea or the Core Rule Set changes -version. In `detect` mode, as the setup runs it, the same requests are logged -with their rule ids, and pass. - -Each refusal answers only that request ("Bans" in `SPEC.md` says what it counts -toward). The request log names the rule in `waf_rule_ids`, which -`SWWAF_WAF_DISABLED_RULES` can switch off. - -### What it lets through - -The Core Rule Set lets gitea's ordinary use through, apart from the refusals in -the sections after this one: - -- browsing and views of files in a repository, with their history, blame and the - file tree; -- diffs, including their hidden lines and large files, and pull request review; -- git's clone, fetch and push over HTTP; -- signing in, including the return to the page a visitor came from, and sign-in - with Git Credential Manager, git-credential-oauth or tea; -- the API's calls for a file and its commits; -- pushing and pulling container images and packages; -- Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`; -- posting issues, pull requests, comments, wiki pages and files saved in the web - editor, code included, since no body is read; -- the page gitea shows after a change, with its message naming what was done. - -### A query string that reads as an attack - -Most often this is a search. A search term is a query parameter, which the Core -Rule Set checks against its lists of system files and commands (see the fourth -change under "Attack detection" in `SPEC.md`). A release attachment uploaded -through the API as `docker-compose.yml` is refused the same way. Examples: - -- a name on those lists, such as `package.json`, `.gitignore` or - `docker-compose.yml`; -- text that starts with a command name, such as `python3` or `ssh key`, or has - one right after a `/` in its first word, such as `feat/docker-support`; -- a shell command with its options or a system path (`ls -la`, `sed -i`, - `/bin/sh`), or a command in backticks; -- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`", "gitea", "first")) - wantFrame(t, frames, record(t, "<158>", "gitea", "second")) + wantFrame(t, frames, record(t, "<158>", "blog", "first")) + wantFrame(t, frames, record(t, "<158>", "blog", "second")) }) } diff --git a/internal/requestlog/requestlog_test.go b/internal/requestlog/requestlog_test.go index 3f959f8..5ee4672 100644 --- a/internal/requestlog/requestlog_test.go +++ b/internal/requestlog/requestlog_test.go @@ -94,7 +94,7 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) { var out bytes.Buffer - requestlog.NewProcessLogger(&out, "fsn1app1/gitea", slog.LevelInfo).Info("starting", + requestlog.NewProcessLogger(&out, "fsn1app1/blog", slog.LevelInfo).Info("starting", "version", "v1") var fields map[string]any @@ -104,7 +104,7 @@ func TestProcessLinesAreMarkedProcessAndGiveTheInstance(t *testing.T) { t.Fatalf("decode %q: %v", out.String(), err) } - if fields["type"] != "process" || fields["instance"] != "fsn1app1/gitea" || + if fields["type"] != "process" || fields["instance"] != "fsn1app1/blog" || fields["msg"] != "starting" || fields["level"] != "INFO" || fields["version"] != "v1" { t.Errorf("process line %v", fields) @@ -133,7 +133,7 @@ func TestProcessLoggerWritesTheMessagesAtItsLevelOrMoreSevere(t *testing.T) { var out bytes.Buffer - processLog := requestlog.NewProcessLogger(&out, "fsn1app1/gitea", level) + processLog := requestlog.NewProcessLogger(&out, "fsn1app1/blog", level) for _, at := range levels { processLog.Log(t.Context(), at, "message") } diff --git a/internal/servewaf/alerts_test.go b/internal/servewaf/alerts_test.go index de4d2f1..288c17e 100644 --- a/internal/servewaf/alerts_test.go +++ b/internal/servewaf/alerts_test.go @@ -18,7 +18,7 @@ const ( alertWebhookURL = "SWWAF_ALERT_WEBHOOK_URL" alertMaxPerHour = "SWWAF_ALERT_MAX_PER_HOUR" // alertInstance is the instance every alert of these tests gives. - alertInstance = "fsn1app1/gitea" + alertInstance = "fsn1app1/blog" ) func TestBanForABrokenLimitRaisesABanAlertWithItsNotes(t *testing.T) { diff --git a/internal/servewaf/requestlog_test.go b/internal/servewaf/requestlog_test.go index 4022cf2..dff7a2e 100644 --- a/internal/servewaf/requestlog_test.go +++ b/internal/servewaf/requestlog_test.go @@ -19,7 +19,7 @@ const ( // requestIDHeader carries the request's id. requestIDHeader = "X-Request-ID" // instance is the SWWAF_INSTANCE_NAME a test sets. - instance = "fsn1app1/gitea" + instance = "fsn1app1/blog" // ipv6Client is a client on IPv6, and ipv6Group the netblock the rate // limits count it as. ipv6Client = "2001:db8::7" diff --git a/internal/smallwebwaf/smallwebwaf_test.go b/internal/smallwebwaf/smallwebwaf_test.go index 5cf085d..2c0bc37 100644 --- a/internal/smallwebwaf/smallwebwaf_test.go +++ b/internal/smallwebwaf/smallwebwaf_test.go @@ -49,7 +49,7 @@ const ( adminSecret = "fedcba9876543210fedcba9876543210" // instance is the SWWAF_INSTANCE_NAME the tests set where they look at // it. - instance = "fsn1app1/gitea" + instance = "fsn1app1/blog" // greeting is what the tests' app answers. greeting = "hello from the app" // placed is the client the tests' lookup databases place. @@ -341,8 +341,8 @@ func TestEveryLogLineAndMetricCarriesTheInstanceName(t *testing.T) { // the comments. for line := range strings.Lines(metrics) { if !strings.HasPrefix(line, "#") && - !strings.Contains(line, `instance="fsn1app1/gitea"`) { - t.Errorf("series %q, without instance=\"fsn1app1/gitea\"", line) + !strings.Contains(line, `instance="fsn1app1/blog"`) { + t.Errorf("series %q, without instance=\"fsn1app1/blog\"", line) } } } @@ -610,9 +610,9 @@ func TestLookupDatabaseReplacedWhileRunningTakesEffect(t *testing.T) { rename(t, replacement, path) metrics := metricsWith(t, url+"_smallwebwaf/metrics", token, - `smallwebwaf_lookup_database_read_failures_total{instance="fsn1app1/gitea"} 1`) + `smallwebwaf_lookup_database_read_failures_total{instance="fsn1app1/blog"} 1`) lastRead = seriesValue(t, metrics, - `smallwebwaf_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/gitea"}`) + `smallwebwaf_lookup_database_last_read_timestamp_seconds{instance="fsn1app1/blog"}`) wantStatus(t, url, placed, http.StatusOK) }) @@ -668,7 +668,7 @@ func TestBlocklistTriesAndCopiesKeptInReputationJSONAcrossRestarts(t *testing.T) // too. rateLimitExemptNets: placed + "," + localhost, } - failures := `smallwebwaf_reputation_failures_total{instance="fsn1app1/gitea",` + + failures := `smallwebwaf_reputation_failures_total{instance="fsn1app1/blog",` + `source="` + torURL + `"} ` // tor.txt cannot be fetched at first, which is counted. @@ -909,8 +909,8 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) { // last. metrics := metricsText(t, url+"_smallwebwaf/metrics", token) for _, series := range []string{ - `smallwebwaf_remote_log_lines_sent_total{instance="fsn1app1/gitea"} 0`, - `smallwebwaf_remote_log_buffer_depth{instance="fsn1app1/gitea"} 1`, + `smallwebwaf_remote_log_lines_sent_total{instance="fsn1app1/blog"} 0`, + `smallwebwaf_remote_log_buffer_depth{instance="fsn1app1/blog"} 1`, } { if !strings.Contains(metrics, "\n"+series+"\n") { t.Errorf("no %q in the metrics:\n%s", series, metrics) @@ -918,7 +918,7 @@ func TestStalledRemoteLogEndpointHoldsUpNoRequest(t *testing.T) { } const dropped = "\nsmallwebwaf_remote_log_lines_dropped_total" + - `{instance="fsn1app1/gitea"} ` + `{instance="fsn1app1/blog"} ` if strings.Contains(metrics, dropped+"0\n") || !strings.Contains(metrics, dropped) { t.Errorf("no line dropped in the metrics:\n%s", metrics) } @@ -1011,7 +1011,7 @@ func TestBanIsAlertedAndAnAlertNotSentIsKeptAcrossARestart(t *testing.T) { runUntilStopped(t, env, func(url string) { webhook.waitFor(t, "permanent_ban", true) - const ofWebhook = `{destination="webhook",instance="fsn1app1/gitea"}` + const ofWebhook = `{destination="webhook",instance="fsn1app1/blog"}` metrics := metricsWith(t, url+"_smallwebwaf/metrics", token, "\nsmallwebwaf_alerts_sent_total"+ofWebhook+" 1\n") @@ -1062,13 +1062,13 @@ func TestBanIsAlertedToSlackAndNtfy(t *testing.T) { slackPost := slack.firstPost(t) err := json.Unmarshal([]byte(slackPost.body), &message) - if err != nil || !strings.HasPrefix(message.Text, "*fsn1app1/gitea: ban*\n") || + if err != nil || !strings.HasPrefix(message.Text, "*fsn1app1/blog: ban*\n") || !strings.Contains(message.Text, "\nclient: "+client+"\n") { t.Errorf("Slack was sent %s", slackPost.body) } ntfyPost := ntfy.firstPost(t) - if ntfyPost.header.Get("Title") != "fsn1app1/gitea: ban" || + if ntfyPost.header.Get("Title") != "fsn1app1/blog: ban" || ntfyPost.header.Get("Authorization") != "Bearer "+ntfyToken || !strings.Contains(ntfyPost.body, "\nclient: "+client+"\n") { t.Errorf("ntfy was sent %s, with the headers %v", ntfyPost.body, @@ -1080,9 +1080,9 @@ func TestBanIsAlertedToSlackAndNtfy(t *testing.T) { // test process cannot fail the test. sent := []string{ "\nsmallwebwaf_alerts_sent_total{destination=\"slack\"," + - "instance=\"fsn1app1/gitea\"} 1\n", + "instance=\"fsn1app1/blog\"} 1\n", "\nsmallwebwaf_alerts_sent_total{destination=\"ntfy\"," + - "instance=\"fsn1app1/gitea\"} 1\n", + "instance=\"fsn1app1/blog\"} 1\n", } metrics := metricsWith(t, url+"_smallwebwaf/metrics", token, sent...) diff --git a/internal/state/state_test.go b/internal/state/state_test.go index 891723d..3a0d769 100644 --- a/internal/state/state_test.go +++ b/internal/state/state_test.go @@ -143,7 +143,7 @@ const filledAlertsJSON = `{ "waiting": { "webhook": [ { - "instance": "fsn1app1/gitea", + "instance": "fsn1app1/blog", "time": "2026-10-06T00:00:00Z", "event": "ban", "client": "203.0.113.9", @@ -158,7 +158,7 @@ const filledAlertsJSON = `{ "suppressed_repeats": 0 }, { - "instance": "fsn1app1/gitea", + "instance": "fsn1app1/blog", "time": "2026-10-06T00:00:00Z", "event": "file_error", "client": "", @@ -1721,7 +1721,7 @@ func newParams(dir string) state.Params { Events: alerts.Events(), Cooldown: 15 * time.Minute, MaxPerHour: 2, - Instance: "fsn1app1/gitea", + Instance: "fsn1app1/blog", Now: midnight, }) diff --git a/script/gitea-requests b/script/gitea-requests deleted file mode 100755 index ca6837e..0000000 --- a/script/gitea-requests +++ /dev/null @@ -1,350 +0,0 @@ -#!/bin/sh -# script/gitea-requests: build the image and run gitea 1.27.3 behind rproxy -# and servewaf, three containers on one network. Every setting is at its -# default but these: SWRP_UPSTREAM_URL names gitea's container, SWRP_WAF_BACKEND -# names servewaf's, and SWWAF_LOG_REQUEST_HEADERS adds the header that labels a -# request. Then send through rproxy the gitea requests that reading gitea's -# source found the Core Rule Set refuses, which docs/gitea.md records, and a -# short sample of gitea's ordinary use: signing in, pages of files and commits, -# a diff, the API, a push, clone and fetch over HTTP, and two container image -# pushes. Last, print each request servewaf logged, with what it did with it and -# the ids of the Core Rule Set's rules that matched it. -# Every request comes from one client, docker's gateway, so more than -# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and -# the requests after would show as banned. The containers, the network, -# the images and the files the script makes are removed however it ends; -# gitea's own image stays, as the base images of a build do. It needs -# network access, and a docker daemon that pushes to a registry on -# localhost over plain HTTP, as docker does by default. script/check does -# not run this. -set -eu - -SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" -ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)" - -# gitea 1.27.3, 2026-08-29 -GITEA_IMAGE="gitea/gitea@sha256:87a67ee09d3ae0d1df5fda5dcda3e2a1f9236a45b0a59025d6e00e46adc43bef" - -# Named after this run, so that runs in other clones on the same host -# never touch each other's. -NAME="$("$SCRIPT_DIR/projectname")-gitea-$$" -IMAGE="$NAME-image" -NETWORK="$NAME" -RPROXY="$NAME-rproxy" -SERVEWAF="$NAME-servewaf" -GITEA="$NAME-gitea" - -# gitea's one user, who owns the repository and the container images. -PASSWORD="owner-password" - -BROWSER="Mozilla/5.0 (X11; Linux x86_64; rv:131.0) Gecko/20100101 Firefox/131.0" - -# The header that labels a request in the report. rproxy makes every request's -# id itself, so a label cannot be the X-Request-ID. -LABEL_HEADER="X-Case" - -# The default of SWWAF_LOG_REQUEST_HEADERS, and the header above. -LOGGED_HEADERS="accept,accept-language,accept-encoding,content-type,origin,range,x-case" - -# The files the script makes, and rproxy's address once it runs, localhost and -# the port docker publishes it on: a host name, since the Core Rule Set takes a -# Host that is an IP address for a warning (920350). -WORK="" -address="" - -cleanup() { - # So that a signal cannot stop it partway through. - trap '' HUP INT TERM - docker rm --force --volumes "$RPROXY" "$SERVEWAF" "$GITEA" >/dev/null 2>&1 || true - docker network rm "$NETWORK" >/dev/null 2>&1 || true - docker rmi --force "$IMAGE" >/dev/null 2>&1 || true - if [ -n "$address" ]; then - docker rmi --force "$address/owner/python3-app:1" "$address/owner/app:1" \ - >/dev/null 2>&1 || true - fi - rm -rf "$WORK" -} - -fail() { - echo "gitea-requests: $*" >&2 - report - exit 1 -} - -# wait_for ...: run the command every second until -# it succeeds, for at most a minute. -wait_for() { - failure="$1" - shift - tries=0 - until "$@"; do - tries=$((tries + 1)) - [ "$tries" -lt 60 ] || fail "$failure" - sleep 1 - done -} - -# healthy : docker says the container's health check passes. -healthy() { - status="$(docker inspect --format '{{.State.Health.Status}}' "$1")" - [ "$status" = healthy ] -} - -# logged ...: a line of servewaf's output holds every text, in any order. -logged() { - lines="$(docker logs "$SERVEWAF" 2>&1)" - for text in "$@"; do - lines="$(printf '%s\n' "$lines" | grep -F "$text")" || return 1 - done -} - -# send