sneak/smallwebwaf
1
0
Fork 0
Code Issues 42 Pull Requests 2 Actions 32 Packages Projects Releases Wiki Activity

SPEC: gitea path parameters, runners, uploads, GeoJS, tokens (closes #6)

Browse Source 
Fourth review of the spec update:

- The Core Rule Set's lists of system files, shell paths and command
  names no longer check the query parameters and cookie in which gitea
  sends file paths, branch names and sign-in redirects; searches stay
  checked.
- The gitea notes name uploads cut off by the size and time limits,
  upload-artifact v3's refused Content-Range header, and runners that
  pass the day limit, with RATE_LIMIT_EXEMPT_NETS for them.
- GeoJS is asked about at most 200 addresses at once; an answer without
  country_code counts as unknown.
- A set token shorter than 32 characters, or a RULES_DIR that does not
  exist, stops the start.

Model: opus-5-5
This commit is contained in:
sneak
2026-09-28 20:18:49 +00:00
parent 4020ff83c6
commit 44a95a6396
1 changed files with 106 additions and 51 deletions
Download Patch File Download Diff File Expand all files Collapse all files
SPEC.md
+106 -51
View File
@@ -301,8 +301,9 @@ The settings, by group:
sidecar forwards and for its own endpoints (see "Admin endpoints").
- `ADMIN_TOKEN`: bearer token for the ban endpoints and
`/_smallwebwaf/clients/<ip>`, a long random value, since they can be
reached from the internet. Unset by default, which switches them off; bans
are then managed by editing `bans.json`.
reached from the internet. A token shorter than 32 characters stops the
start with a message naming the variable. Unset by default, which switches
them off; bans are then managed by editing `bans.json`.
- `INSTANCE_NAME` (default: the host name in `UPSTREAM_URL`, for example
`gitea`): included in every log line, metric and alert. Set it, for
example to `fsn1app1/gitea`, when several sidecars report to one place.
@@ -351,8 +352,9 @@ The settings, by group:
`INSTANCE_NAME`): syslog header fields.
- Metrics
- `METRICS_TOKEN`: bearer token a scraper sends for `/_smallwebwaf/metrics`,
a long random value. Unset by default, which switches the metrics off,
since they would otherwise be open to anyone on the internet.
a long random value. A token shorter than 32 characters stops the start
with a message naming the variable. Unset by default, which switches the
metrics off, since they would otherwise be open to anyone on the internet.
- `METRICS_TOP_N` (default `50`): how many AS numbers and countries get
their own series; the rest are summed as `other`.
- Static lists
@@ -363,7 +365,10 @@ The settings, by group:
- Request rate limits, per client (R1, R2). Breaking one bans the client (see
"Bans"), so the defaults sit several times above what one busy person
produces: a browser loading a heavy page makes a few hundred requests, a git
clone a handful, and several people often share one address.
clone a handful, and several people often share one address. A machine that
talks to the app all day, such as a gitea Actions runner (see the notes
specific to gitea under "Deployment as a sidecar"), can still pass the day
limit; its address belongs in `RATE_LIMIT_EXEMPT_NETS`.
- `RATE_LIMIT_PER_MINUTE` (default `1000`), `RATE_LIMIT_PER_HOUR` (default
`10000`), `RATE_LIMIT_PER_DAY` (default `50000`).
- `RATE_LIMIT_EXEMPT_PATHS`: path prefixes not counted (static assets,
@@ -430,15 +435,15 @@ The settings, by group:
the file rather than the file itself: docker does not show a single
mounted file being replaced on the host.
- `geojs` asks the free GeoJS web service, which needs no account or key,
about several addresses in one request
about up to 200 addresses in one request
(`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), and reads each answer's
`country_code`, `asn` and `organization_name` (the AS name). A
`country_code` of `null`, which GeoJS gives for some ranges, and an `asn`
of `64512`, which it gives when it knows none, count as unknown. GeoJS is
told the address of every new visitor, whether or not a setting uses the
answer, since the request log, the metrics, the client's history and ban
notes carry the AS number and country. Private, loopback and link-local
addresses, which no source can place, are never sent.
`country_code`, `asn` and `organization_name` (the AS name). An answer
without a `country_code`, which GeoJS sends for ranges it cannot place,
and an `asn` of `64512`, which it gives when it knows none, count as
unknown. GeoJS is told the address of every new visitor, whether or not a
setting uses the answer, since the request log, the metrics, the client's
history and ban notes carry the AS number and country. Private, loopback
and link-local addresses, which no source can place, are never sent.
- Each GeoJS answer is kept for 7 days in memory and in `lookups.json`,
apart from the table of clients, so it survives a restart and outlasts a
client dropped from that table; after 7 days the client's next request
@@ -448,10 +453,11 @@ The settings, by group:
first answer when a setting needs it (see "Data flow for one request");
GeoJS normally answers in a fraction of that. At most one request to GeoJS
is under way at a time, the addresses that arrive meanwhile are asked
about together in the next one, and a request that takes longer than
`LOOKUP_TIMEOUT` is abandoned. A client whose answer has not come in time
counts as unknown until it comes: its later requests do not wait, and its
address is asked about again in the background.
about together in the next one, up to 200 per request with the rest in the
requests after it, and a request that takes longer than `LOOKUP_TIMEOUT`
is abandoned. A client whose answer has not come in time counts as unknown
until it comes: its later requests do not wait, and its address is asked
about again in the background.
- GeoJS publishes no rate limit, but its terms forbid "an excessive amount
of API requests", judged by GeoJS alone, and let it block a caller. While
GeoJS is slow, down or refusing the sidecar, clients with a kept answer
@@ -532,7 +538,7 @@ The settings, by group:
- `WAF_PARANOIA_LEVEL` (default `1`), `WAF_ANOMALY_THRESHOLD` (default `5`):
the Core Rule Set's own two tuning values, at the Core Rule Set's own
defaults.
- The sidecar also changes the Core Rule Set 4.25.0 in four ways that no
- The sidecar also changes the Core Rule Set 4.25.0 in five ways that no
setting undoes, since in front of gitea each would otherwise refuse
ordinary requests:
- PUT, PATCH and DELETE are allowed methods besides GET, HEAD, POST and
@@ -550,6 +556,21 @@ The settings, by group:
git-credential-oauth and tea, which gitea registers for OAuth sign-in
out of the box, ask to be sent back to `http://127.0.0.1` on the
user's own machine, and the server never fetches that address.
- The query parameters in which gitea sends file paths, branch and
workflow names and the page to return to after signing in (`path`,
`files`, `skip-to`, `sub_path`, `ref`, `sha`, `branch`, `workflow`,
`artifactName` and `redirect_to`), and the `redirect_to` cookie, are
not checked against the Core Rule Set's lists of system files
(930120), shell paths (932160) and command names (932260). In a
repository any name on those lists can be an ordinary file or branch,
such as `.gitignore`, `package.json`, `docker-compose.yml`,
`bin/docker-entrypoint` or a branch named `docker-build`, and gitea
reads these values as names within a repository or its own records, or
as a page of its own site. What only these three rules refuse, such as
`/etc/passwd` or `whoami` on its own, is therefore let through in
those parameters; path traversal (`../`), SQL and script injection and
PHP, Java and Node.js code are still refused there, and every other
parameter and cookie keeps all three rules.
- Responses are not inspected. A raw file from a repository, such as a
shell script, looks to the response rules like source code leaking
from the server.
@@ -749,8 +770,10 @@ removed there takes effect while the sidecar runs.
stops the process with a message naming the file and line. While running, the
same faults leave the rules as they were, including the earlier version of
that file, and the log and one `file_error` alert name the file and line; once
the file is fixed, it is read again. A missing or empty directory is not an
error: the log says that no rules were loaded.
the file is fixed, it is read again. A `RULES_DIR` that does not exist stops
the start with a message naming it. An empty directory is not an error: the
log says that no rules were loaded. To run without rule files, mount an empty
directory or set `RULES_ENABLED=false`.
- The image ships a default file in `RULES_DIR`. Mounting a directory over it
replaces the defaults; mounting single files into it adds to them. Docker does
not show a single mounted file being replaced on the host, which is how many
@@ -1012,9 +1035,10 @@ scrapers reach these endpoints through traefik, like any other request.
history, lookup result, reputation, offences, and bans with their notes; for
answering "why was this address refused" and "what has this netblock been
doing".
- A token is sent as `Authorization: Bearer <token>`. While a token is unset,
the endpoints that need it answer `404`, as does any other path under the
prefix.
- A token is sent as `Authorization: Bearer <token>`, and one set shorter than
32 characters stops the start (see "Configuration surface"). While a token is
unset, the endpoints that need it answer `404`, as does any other path under
the prefix.
- Apart from the health check, these requests go through every check that any
other request goes through, and are answered at the point where another
request would be forwarded to the app: a banned or refused client stays
@@ -1100,30 +1124,59 @@ networks:
a remote log endpoint, and an app-specific rule file.
- Notes specific to gitea:
- A clone is a response, and fits the defaults of 30 minutes and 5 GiB for
all but the largest repositories and slowest links. A push is a request:
at the defaults, one larger than 100 MiB or taking more than 60 seconds to
upload is cut off, so a gitea that takes large pushes needs
`CLIENT_REQUEST_MAX_BYTES`, `UPSTREAM_REQUEST_MAX_BYTES`,
`CLIENT_REQUEST_TIMEOUT` and `UPSTREAM_REQUEST_TIMEOUT` raised to fit. The
Core Rule Set does not read a pack upload, which streams through without
being held in memory.
all but the largest repositories and slowest links. A push is a request,
and so is every other upload: at the defaults, one larger than 100 MiB or
taking more than 60 seconds is cut off, whether it is a git push, an LFS
object, a container image layer, a package file or a release attachment.
The client is answered `413` for a body that is too large, before anything
reaches gitea when the request announces its size, or `408` for one that
is too slow; the upload fails, and no one is banned for it. A gitea that
takes large uploads needs `CLIENT_REQUEST_MAX_BYTES`,
`UPSTREAM_REQUEST_MAX_BYTES`, `CLIENT_REQUEST_TIMEOUT` and
`UPSTREAM_REQUEST_TIMEOUT` raised to fit. The Core Rule Set does not read
an upload's body, which streams through without being held in memory.
- At the defaults (see "Configuration surface", attack detection), the Core
Rule Set lets gitea's ordinary use through: browsing and views of files in
a repository, git's clone, fetch and push over HTTP, signing in, including
with Git Credential Manager, git-credential-oauth or tea, the API, pushing
and pulling container images and packages, and posting issues, pull
requests, comments, wiki pages and files saved in the web editor, code
included, since no body is read. It can still refuse a request whose URL
reads to it as an attack: a search, or another query string, holding a
shell command with its options or a system path (`ls -la`, `sed -i`,
`/bin/sh`), a command in backticks, script code (`fetch(`, `${VAR}`), HTML
(`<img src=`), an SQL statement (`SELECT * FROM users WHERE`), or a URL
naming an IP address or `localhost`; or a path such as a file name ending
in `~`, or an `.xhtml` file whose path holds a space. Such a refusal
answers only that request, with 403, and bans no one by itself: it counts
toward the error burst, which a person searching does not reach. The
request log names the rule in `waf_rule_ids`, which `WAF_DISABLED_RULES`
can switch off.
Rule Set lets gitea's ordinary use through, whatever its files and
branches are called: browsing and views of files in a repository, with
their history, blame and the file tree; diffs, including their hidden
lines and large files, and pull request review; git's clone, fetch and
push over HTTP; signing in, including the return to the page a visitor
came from and sign-in with Git Credential Manager, git-credential-oauth or
tea; the API's calls for a file and its commits; pushing and pulling
container images and packages; Actions runners, and artifacts uploaded
with `actions/upload-artifact@v4`; and posting issues, pull requests,
comments, wiki pages and files saved in the web editor, code included,
since no body is read. It can still refuse a query string that reads to it
as an attack, most often a search: one for a name on its lists of system
files and commands, such as `package.json`, `.gitignore` or
`docker-compose.yml`, or for text that starts with a command name, such as
`python3` or `ssh key`; or one holding a shell command with its options or
a system path (`ls -la`, `sed -i`, `/bin/sh`), a command in backticks,
script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`), an
SQL statement (`SELECT * FROM users WHERE`), or a URL naming an IP address
or `localhost`. The lists refuse such a name in any other query parameter
too, such as a release attachment uploaded through the API as
`docker-compose.yml`. A path can be refused as well: a file name ending in
`~`, or an `.xhtml` file whose path holds a space. Such a refusal answers
only that request, with 403, and bans no one by itself: it counts toward
the error burst, which a person searching does not reach. The request log
names the rule in `waf_rule_ids`, which `WAF_DISABLED_RULES` can switch
off.
- Artifact uploads from `actions/upload-artifact@v3` send the header
`Content-Range`, which the Core Rule Set refuses (920450). Each file of
the artifact is refused with 403 and the step fails; an artifact of more
than 30 files also bans the runner through the error burst.
`actions/upload-artifact@v4` does not send the header.
- An Actions runner that reaches gitea through the sidecar sends requests
all day. One older than version 0.4 (April 2026) asks for work every 2
seconds, 43,200 requests a day, and reports a running job's log and state
every second, so it passes the day limit of 50,000 after about an hour and
a quarter of jobs: it is banned, the job it is running fails, and each
repeat within a day makes the ban longer. Newer runners ask less often,
but a busy one, or several on one address, can still pass it. Put the
runners' addresses in `RATE_LIMIT_EXEMPT_NETS`, which takes them out of
the request and byte limits while the error burst, attack detection and
bans still apply.
- Archive download and blame or history pages are what scrapers hammer;
request limits do most of the work there.
@@ -1190,14 +1243,16 @@ networks:
no one by itself; the request log names the rule, and exclusions by rule id
and path fix it.
- Attacks carried in request bodies: not refused by default, since on a code
forge bodies are full of code the Core Rule Set takes for attacks. What shows
in URLs and headers is still refused, the client that sends an attack is still
subject to the rule files, the limits and the bans, and `WAF_BODY_LIMIT`
switches body inspection on for apps whose forms carry no code.
forge bodies are full of code the Core Rule Set takes for attacks. Most of
what shows in URLs and headers is still refused, the client that sends an
attack is still subject to the rule files, the limits and the bans, and
`WAF_BODY_LIMIT` switches body inspection on for apps whose forms carry no
code.
- The admin endpoints can be reached from the internet: all but the health check
need a token and are off while it is unset, and a missing or wrong token
counts toward the error burst, so a client guessing tokens is soon banned.
Tokens are meant to be long random values.
Tokens are meant to be long random values, and one shorter than 32 characters
stops the start.
- GeoJS, the default lookup source: every new visitor's address goes to a third
party, and a swarm of fresh addresses, when lookups peak, is when GeoJS may
slow down or block the sidecar. Keeping answers for 7 days and asking about