diff --git a/SPEC.md b/SPEC.md index 7b0711c..c7b7fdb 100644 --- a/SPEC.md +++ b/SPEC.md @@ -301,8 +301,9 @@ The settings, by group: sidecar forwards and for its own endpoints (see "Admin endpoints"). - `ADMIN_TOKEN`: bearer token for the ban endpoints and `/_smallwebwaf/clients/`, a long random value, since they can be - reached from the internet. Unset by default, which switches them off; bans - are then managed by editing `bans.json`. + reached from the internet. A token shorter than 32 characters stops the + start with a message naming the variable. Unset by default, which switches + them off; bans are then managed by editing `bans.json`. - `INSTANCE_NAME` (default: the host name in `UPSTREAM_URL`, for example `gitea`): included in every log line, metric and alert. Set it, for example to `fsn1app1/gitea`, when several sidecars report to one place. @@ -351,8 +352,9 @@ The settings, by group: `INSTANCE_NAME`): syslog header fields. - Metrics - `METRICS_TOKEN`: bearer token a scraper sends for `/_smallwebwaf/metrics`, - a long random value. Unset by default, which switches the metrics off, - since they would otherwise be open to anyone on the internet. + a long random value. A token shorter than 32 characters stops the start + with a message naming the variable. Unset by default, which switches the + metrics off, since they would otherwise be open to anyone on the internet. - `METRICS_TOP_N` (default `50`): how many AS numbers and countries get their own series; the rest are summed as `other`. - Static lists @@ -363,7 +365,10 @@ The settings, by group: - Request rate limits, per client (R1, R2). Breaking one bans the client (see "Bans"), so the defaults sit several times above what one busy person produces: a browser loading a heavy page makes a few hundred requests, a git - clone a handful, and several people often share one address. + clone a handful, and several people often share one address. A machine that + talks to the app all day, such as a gitea Actions runner (see the notes + specific to gitea under "Deployment as a sidecar"), can still pass the day + limit; its address belongs in `RATE_LIMIT_EXEMPT_NETS`. - `RATE_LIMIT_PER_MINUTE` (default `1000`), `RATE_LIMIT_PER_HOUR` (default `10000`), `RATE_LIMIT_PER_DAY` (default `50000`). - `RATE_LIMIT_EXEMPT_PATHS`: path prefixes not counted (static assets, @@ -430,15 +435,15 @@ The settings, by group: the file rather than the file itself: docker does not show a single mounted file being replaced on the host. - `geojs` asks the free GeoJS web service, which needs no account or key, - about several addresses in one request + about up to 200 addresses in one request (`https://get.geojs.io/v1/ip/geo.json?ip=a,b,c`), and reads each answer's - `country_code`, `asn` and `organization_name` (the AS name). A - `country_code` of `null`, which GeoJS gives for some ranges, and an `asn` - of `64512`, which it gives when it knows none, count as unknown. GeoJS is - told the address of every new visitor, whether or not a setting uses the - answer, since the request log, the metrics, the client's history and ban - notes carry the AS number and country. Private, loopback and link-local - addresses, which no source can place, are never sent. + `country_code`, `asn` and `organization_name` (the AS name). An answer + without a `country_code`, which GeoJS sends for ranges it cannot place, + and an `asn` of `64512`, which it gives when it knows none, count as + unknown. GeoJS is told the address of every new visitor, whether or not a + setting uses the answer, since the request log, the metrics, the client's + history and ban notes carry the AS number and country. Private, loopback + and link-local addresses, which no source can place, are never sent. - Each GeoJS answer is kept for 7 days in memory and in `lookups.json`, apart from the table of clients, so it survives a restart and outlasts a client dropped from that table; after 7 days the client's next request @@ -448,10 +453,11 @@ The settings, by group: first answer when a setting needs it (see "Data flow for one request"); GeoJS normally answers in a fraction of that. At most one request to GeoJS is under way at a time, the addresses that arrive meanwhile are asked - about together in the next one, and a request that takes longer than - `LOOKUP_TIMEOUT` is abandoned. A client whose answer has not come in time - counts as unknown until it comes: its later requests do not wait, and its - address is asked about again in the background. + about together in the next one, up to 200 per request with the rest in the + requests after it, and a request that takes longer than `LOOKUP_TIMEOUT` + is abandoned. A client whose answer has not come in time counts as unknown + until it comes: its later requests do not wait, and its address is asked + about again in the background. - GeoJS publishes no rate limit, but its terms forbid "an excessive amount of API requests", judged by GeoJS alone, and let it block a caller. While GeoJS is slow, down or refusing the sidecar, clients with a kept answer @@ -532,7 +538,7 @@ The settings, by group: - `WAF_PARANOIA_LEVEL` (default `1`), `WAF_ANOMALY_THRESHOLD` (default `5`): the Core Rule Set's own two tuning values, at the Core Rule Set's own defaults. - - The sidecar also changes the Core Rule Set 4.25.0 in four ways that no + - The sidecar also changes the Core Rule Set 4.25.0 in five ways that no setting undoes, since in front of gitea each would otherwise refuse ordinary requests: - PUT, PATCH and DELETE are allowed methods besides GET, HEAD, POST and @@ -550,6 +556,21 @@ The settings, by group: git-credential-oauth and tea, which gitea registers for OAuth sign-in out of the box, ask to be sent back to `http://127.0.0.1` on the user's own machine, and the server never fetches that address. + - The query parameters in which gitea sends file paths, branch and + workflow names and the page to return to after signing in (`path`, + `files`, `skip-to`, `sub_path`, `ref`, `sha`, `branch`, `workflow`, + `artifactName` and `redirect_to`), and the `redirect_to` cookie, are + not checked against the Core Rule Set's lists of system files + (930120), shell paths (932160) and command names (932260). In a + repository any name on those lists can be an ordinary file or branch, + such as `.gitignore`, `package.json`, `docker-compose.yml`, + `bin/docker-entrypoint` or a branch named `docker-build`, and gitea + reads these values as names within a repository or its own records, or + as a page of its own site. What only these three rules refuse, such as + `/etc/passwd` or `whoami` on its own, is therefore let through in + those parameters; path traversal (`../`), SQL and script injection and + PHP, Java and Node.js code are still refused there, and every other + parameter and cookie keeps all three rules. - Responses are not inspected. A raw file from a repository, such as a shell script, looks to the response rules like source code leaking from the server. @@ -749,8 +770,10 @@ removed there takes effect while the sidecar runs. stops the process with a message naming the file and line. While running, the same faults leave the rules as they were, including the earlier version of that file, and the log and one `file_error` alert name the file and line; once - the file is fixed, it is read again. A missing or empty directory is not an - error: the log says that no rules were loaded. + the file is fixed, it is read again. A `RULES_DIR` that does not exist stops + the start with a message naming it. An empty directory is not an error: the + log says that no rules were loaded. To run without rule files, mount an empty + directory or set `RULES_ENABLED=false`. - The image ships a default file in `RULES_DIR`. Mounting a directory over it replaces the defaults; mounting single files into it adds to them. Docker does not show a single mounted file being replaced on the host, which is how many @@ -1012,9 +1035,10 @@ scrapers reach these endpoints through traefik, like any other request. history, lookup result, reputation, offences, and bans with their notes; for answering "why was this address refused" and "what has this netblock been doing". -- A token is sent as `Authorization: Bearer `. While a token is unset, - the endpoints that need it answer `404`, as does any other path under the - prefix. +- A token is sent as `Authorization: Bearer `, and one set shorter than + 32 characters stops the start (see "Configuration surface"). While a token is + unset, the endpoints that need it answer `404`, as does any other path under + the prefix. - Apart from the health check, these requests go through every check that any other request goes through, and are answered at the point where another request would be forwarded to the app: a banned or refused client stays @@ -1100,30 +1124,59 @@ networks: a remote log endpoint, and an app-specific rule file. - Notes specific to gitea: - A clone is a response, and fits the defaults of 30 minutes and 5 GiB for - all but the largest repositories and slowest links. A push is a request: - at the defaults, one larger than 100 MiB or taking more than 60 seconds to - upload is cut off, so a gitea that takes large pushes needs - `CLIENT_REQUEST_MAX_BYTES`, `UPSTREAM_REQUEST_MAX_BYTES`, - `CLIENT_REQUEST_TIMEOUT` and `UPSTREAM_REQUEST_TIMEOUT` raised to fit. The - Core Rule Set does not read a pack upload, which streams through without - being held in memory. + all but the largest repositories and slowest links. A push is a request, + and so is every other upload: at the defaults, one larger than 100 MiB or + taking more than 60 seconds is cut off, whether it is a git push, an LFS + object, a container image layer, a package file or a release attachment. + The client is answered `413` for a body that is too large, before anything + reaches gitea when the request announces its size, or `408` for one that + is too slow; the upload fails, and no one is banned for it. A gitea that + takes large uploads needs `CLIENT_REQUEST_MAX_BYTES`, + `UPSTREAM_REQUEST_MAX_BYTES`, `CLIENT_REQUEST_TIMEOUT` and + `UPSTREAM_REQUEST_TIMEOUT` raised to fit. The Core Rule Set does not read + an upload's body, which streams through without being held in memory. - At the defaults (see "Configuration surface", attack detection), the Core - Rule Set lets gitea's ordinary use through: browsing and views of files in - a repository, git's clone, fetch and push over HTTP, signing in, including - with Git Credential Manager, git-credential-oauth or tea, the API, pushing - and pulling container images and packages, and posting issues, pull - requests, comments, wiki pages and files saved in the web editor, code - included, since no body is read. It can still refuse a request whose URL - reads to it as an attack: a search, or another query string, holding a - shell command with its options or a system path (`ls -la`, `sed -i`, - `/bin/sh`), a command in backticks, script code (`fetch(`, `${VAR}`), HTML - (`