Compare commits
3
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
eff99f0b13 | ||
|
|
e44ba7ee5e | ||
|
|
7fb8662e69 |
+210
@@ -0,0 +1,210 @@
|
|||||||
|
# smallwebwaf in front of gitea
|
||||||
|
|
||||||
|
These notes record what `smallwebwaf` does in front of gitea at the default
|
||||||
|
settings. They are observations, not requirements; the requirements are in
|
||||||
|
[`SPEC.md`](../SPEC.md). `script/gitea-requests` runs gitea behind the image and
|
||||||
|
sends through it a sample of gitea's ordinary use and the requests named below.
|
||||||
|
It reports each request the Core Rule Set refuses, so that these notes can be
|
||||||
|
checked again when gitea or the Core Rule Set changes version.
|
||||||
|
|
||||||
|
## What the defaults cover
|
||||||
|
|
||||||
|
- SSH access to gitea does not pass through `smallwebwaf` and is not protected
|
||||||
|
by it.
|
||||||
|
- A clone is a response. It fits the defaults of 30 minutes and 5 GiB for all
|
||||||
|
but the largest repositories and slowest links.
|
||||||
|
- A push is a request, and so is every other upload: a git push, an LFS object,
|
||||||
|
a container image layer, a package file or a release attachment. At the
|
||||||
|
defaults, an upload larger than 100 MiB or taking more than 60 seconds is cut
|
||||||
|
off. The client is answered `413` for a body that is too large, before
|
||||||
|
anything reaches gitea when the request announces its size. It is answered
|
||||||
|
`408` for a body it sends too slowly, or `504` if gitea is too slow to take
|
||||||
|
it. The upload fails, and no one is banned for it. A gitea that takes large
|
||||||
|
uploads needs `SWWAF_REQUEST_MAX_BYTES`, `SWWAF_CLIENT_REQUEST_TIMEOUT` and
|
||||||
|
`SWWAF_UPSTREAM_REQUEST_TIMEOUT` raised to fit. The Core Rule Set does not
|
||||||
|
read an upload's body, which streams through without being held in memory.
|
||||||
|
- Archive download and blame or history pages are what scrapers hammer. The
|
||||||
|
request limits do most of the work there.
|
||||||
|
- An Actions runner that reaches gitea through `smallwebwaf` sends requests all
|
||||||
|
day. One older than version 0.4 (April 2026) asks for work every 2 seconds,
|
||||||
|
43,200 requests a day, and reports a running job's log and state every second.
|
||||||
|
So it passes the day limit of 50,000 after about an hour and a quarter of
|
||||||
|
jobs. It is then banned, the job it is running fails, and each repeat within a
|
||||||
|
day makes the ban longer. Newer runners ask less often, but a busy one, or
|
||||||
|
several on one address, can still pass it. Put the runners' addresses in
|
||||||
|
`SWWAF_RATE_LIMIT_EXEMPT_NETS`. That takes them out of the request and byte
|
||||||
|
limits, while the error burst, attack detection and bans still apply.
|
||||||
|
|
||||||
|
## What the Core Rule Set lets through
|
||||||
|
|
||||||
|
At the defaults, the Core Rule Set lets gitea's ordinary use through, apart from
|
||||||
|
the refusals in the next section:
|
||||||
|
|
||||||
|
- browsing and views of files in a repository, with their history, blame and the
|
||||||
|
file tree;
|
||||||
|
- diffs, including their hidden lines and large files, and pull request review;
|
||||||
|
- git's clone, fetch and push over HTTP;
|
||||||
|
- signing in, including the return to the page a visitor came from, and sign-in
|
||||||
|
with Git Credential Manager, git-credential-oauth or tea;
|
||||||
|
- the API's calls for a file and its commits;
|
||||||
|
- pushing and pulling container images and packages;
|
||||||
|
- Actions runners, and artifacts uploaded with `actions/upload-artifact@v4`;
|
||||||
|
- posting issues, pull requests, comments, wiki pages and files saved in the web
|
||||||
|
editor, code included, since no body is read;
|
||||||
|
- the page gitea shows after a change, with its message naming what was done.
|
||||||
|
|
||||||
|
## What the Core Rule Set still refuses
|
||||||
|
|
||||||
|
Each refusal answers only that request, with `403`, and bans no one by itself.
|
||||||
|
It counts toward the error burst, which a person does not reach this way. The
|
||||||
|
request log names the rule in `waf_rule_ids`, which `SWWAF_WAF_DISABLED_RULES`
|
||||||
|
can switch off.
|
||||||
|
|
||||||
|
### A query string that reads as an attack
|
||||||
|
|
||||||
|
Most often this is a search. Every query parameter is checked against the Core
|
||||||
|
Rule Set's lists of system files and commands, except the ten named under
|
||||||
|
"Attack detection" in `SPEC.md`. A release attachment uploaded through the API
|
||||||
|
as `docker-compose.yml` is refused the same way. Examples:
|
||||||
|
|
||||||
|
- a name on those lists, such as `package.json`, `.gitignore` or
|
||||||
|
`docker-compose.yml`;
|
||||||
|
- text that starts with a command name, or whose first word has one right after
|
||||||
|
a `/`, such as `python3`, `ssh key` or `feat/docker-support`;
|
||||||
|
- a shell command with its options or a system path (`ls -la`, `sed -i`,
|
||||||
|
`/bin/sh`), or a command in backticks;
|
||||||
|
- script code (`fetch(`, `${VAR}`, `process.env`), HTML (`<img src=`) or an SQL
|
||||||
|
statement (`SELECT * FROM users WHERE`);
|
||||||
|
- a URL naming an IP address or `localhost`.
|
||||||
|
|
||||||
|
### A branch, tag or file name that starts with a command name
|
||||||
|
|
||||||
|
The Core Rule Set's list of commands (932260) holds names such as `docker-`,
|
||||||
|
`python3`, `ansible`, `base64` and `whoami`. A branch, tag or file name that
|
||||||
|
starts with one is refused. So is one that has such a name right after a `/`,
|
||||||
|
such as `feat/docker-support`, `fix/docker-build` or
|
||||||
|
`renovate/docker-build-push-action-6.x`. gitea's own pages send the name in a
|
||||||
|
query parameter that 930120, 932160 and 932260 still check:
|
||||||
|
|
||||||
|
- `refSubUrl`, the branch or tag of a directory listing. It is sent when the
|
||||||
|
listing asks for its entries' last commits in a second request, which it does
|
||||||
|
whenever gitea takes more than a second to work them out;
|
||||||
|
- `name`, when a branch is deleted or restored on the branches page;
|
||||||
|
- `tag`, when a release is started from a tag;
|
||||||
|
- `template`, the file of an issue template, when an issue is opened from it;
|
||||||
|
- `rule_name`, when a branch protection rule is opened for editing in the
|
||||||
|
repository's settings or has just been saved.
|
||||||
|
|
||||||
|
What a visitor sees, for a branch named `docker-build`:
|
||||||
|
|
||||||
|
- A directory listing that makes that second request leaves those last commits
|
||||||
|
out and shows an error.
|
||||||
|
- The branches page shows an error instead of deleting or restoring the branch.
|
||||||
|
- A release started from such a tag gets the `403` answer of `smallwebwaf` in
|
||||||
|
place of its form. So does an issue opened from such a template, and such a
|
||||||
|
rule opened for editing.
|
||||||
|
- Saving such a rule stores it. gitea then returns to the branch settings page,
|
||||||
|
which gets the `403` answer, so the save looks as if it failed.
|
||||||
|
|
||||||
|
### A file name in a query parameter that trips the code rules
|
||||||
|
|
||||||
|
The Core Rule Set's rules for Java, script, PowerShell and SQL find what they
|
||||||
|
look for in some file names. This holds in the parameters `smallwebwaf` leaves
|
||||||
|
out of 930120, 932160 and 932260 too (see the changes to the Core Rule Set under
|
||||||
|
"Attack detection" in `SPEC.md`). Examples:
|
||||||
|
|
||||||
|
- `runtime` in OpenJDK's `src/java.base/share/classes/java/lang/Runtime.java`
|
||||||
|
(944110);
|
||||||
|
- a Java class, as in `java.lang.String.md` (944130);
|
||||||
|
- `document.write.js` (941180);
|
||||||
|
- `process.env.d.ts` (934100);
|
||||||
|
- `__proto__` (934130);
|
||||||
|
- the PowerShell command `Install-Script`, as in `install-script.sh` (932120);
|
||||||
|
- a database's name, as in PostgreSQL's `information_schema.sql` (942140).
|
||||||
|
|
||||||
|
For such a file, these requests are refused:
|
||||||
|
|
||||||
|
- the API's list of its commits (`path`);
|
||||||
|
- the API's call for the contents of files that names it in its query (`body`);
|
||||||
|
- the commit graph of that file alone (`file`);
|
||||||
|
- the hidden lines of its diff (`path`), which stay hidden;
|
||||||
|
- the sign-in page gitea sends a visitor to on the way to a page of the file
|
||||||
|
that needs signing in, such as its edit page (`redirect_to`). The visitor can
|
||||||
|
still sign in by opening the sign-in page itself.
|
||||||
|
|
||||||
|
### A path
|
||||||
|
|
||||||
|
- A file name ending in `~`, or an `.xhtml` file whose path holds a space.
|
||||||
|
- A path that holds what the Core Rule Set takes for a Java class (944130), as
|
||||||
|
`java.lang.String.md` and
|
||||||
|
`META-INF/services/org.apache.commons.logging.LogFactory` do.
|
||||||
|
- A path that holds what it takes for Node.js code (934100), as
|
||||||
|
`process.env.d.ts` does.
|
||||||
|
- A path with a directory named `...` (930100, 930110), such as `.../notes.txt`.
|
||||||
|
|
||||||
|
For such a file, the page of the file, its raw file, its history, its blame and
|
||||||
|
its edit page are refused. The directory that holds it lists it, and a clone
|
||||||
|
carries it.
|
||||||
|
|
||||||
|
### A container image push that reuses a layer
|
||||||
|
|
||||||
|
A push can reuse a layer of an image docker has pushed to that gitea before.
|
||||||
|
docker then asks gitea to take the layer from that image, naming it in the query
|
||||||
|
parameter `from`, which 932260 checks. The request is refused when the image's
|
||||||
|
owner, or its name after the owner, starts with an entry on the list of
|
||||||
|
commands. Examples are `owner/python3-app`, `owner/docker-base`, and any image
|
||||||
|
of an owner named `ansible` or `docker-team`.
|
||||||
|
|
||||||
|
What happens next depends on docker's image store. A docker that keeps its
|
||||||
|
images in containerd's image store fails the push, and fails each new try the
|
||||||
|
same way. A new install of Docker Engine 29 does this. A docker that keeps them
|
||||||
|
in its own older image store uploads the layer instead. The push then succeeds
|
||||||
|
after one refusal for each layer it reuses, and each of those refusals counts
|
||||||
|
toward the error burst.
|
||||||
|
|
||||||
|
### Artifact uploads from `actions/upload-artifact@v3`
|
||||||
|
|
||||||
|
This action sends the header `Content-Range`, which the Core Rule Set refuses
|
||||||
|
(920450). The action sends two files at a time, does not retry a `403`, and
|
||||||
|
sends no further file once one is refused. So an upload is refused once or
|
||||||
|
twice, however many files it holds, and the step fails. One upload does not ban
|
||||||
|
the runner. More than 30 such refusals from its address within a minute, as when
|
||||||
|
many jobs upload at once, ban it through the error burst.
|
||||||
|
`actions/upload-artifact@v4` does not send the header.
|
||||||
|
|
||||||
|
## What body inspection refuses
|
||||||
|
|
||||||
|
With `SWWAF_WAF_BODY_LIMIT` set, the Core Rule Set reads request bodies too. In
|
||||||
|
front of gitea it then refuses these, until their rule ids, named in the request
|
||||||
|
log, are added to `SWWAF_WAF_DISABLED_RULES`:
|
||||||
|
|
||||||
|
- issue and comment text, wiki pages and files saved in the web editor that hold
|
||||||
|
shell commands or code (932125, 932235, 932250 and others);
|
||||||
|
- package descriptions that show code;
|
||||||
|
- PyPI uploads (922130);
|
||||||
|
- attachments named like `debug.log` or `config.yml` (932180).
|
||||||
|
|
||||||
|
## What `script/gitea-requests` sends
|
||||||
|
|
||||||
|
Of the requests above, the script sends:
|
||||||
|
|
||||||
|
- for each file named under "A file name in a query parameter that trips the
|
||||||
|
code rules", the API's list of its commits;
|
||||||
|
- for `Runtime.java` alone, the commit graph of that file, the hidden lines of
|
||||||
|
its diff, and the API's call for the contents of files that names it;
|
||||||
|
- the sign-in page gitea sends a visitor to on the way to the edit page of
|
||||||
|
`document.write.js`;
|
||||||
|
- the pages of `java.lang.String.md`,
|
||||||
|
`META-INF/services/org.apache.commons.logging.LogFactory`, `process.env.d.ts`
|
||||||
|
and `.../notes.txt`;
|
||||||
|
- one container image push that reuses a layer, which it does not try again.
|
||||||
|
|
||||||
|
It does not send the raw file, history, blame or edit page of those four files,
|
||||||
|
or the pages of the directories `META-INF/services` and `...`. It also sends a
|
||||||
|
short sample of gitea's ordinary use:
|
||||||
|
|
||||||
|
- signing in;
|
||||||
|
- pages of files and commits, and a diff;
|
||||||
|
- the API;
|
||||||
|
- a push, and clone and fetch over HTTP;
|
||||||
|
- two container image pushes.
|
||||||
@@ -0,0 +1,56 @@
|
|||||||
|
# What the image holds
|
||||||
|
|
||||||
|
The `smallwebwaf` image is the base of each app's own image. The requirements
|
||||||
|
are under "Deployment" in [`SPEC.md`](../SPEC.md); this file lists what the
|
||||||
|
image holds, how each part is pinned, and why.
|
||||||
|
|
||||||
|
## Contents
|
||||||
|
|
||||||
|
- Ubuntu 26.04 LTS, the newest long-term support release of Ubuntu, pinned by
|
||||||
|
digest. The image moves to the next LTS release when that ships.
|
||||||
|
- `ca-certificates`, `nix-bin` and `runit` from Ubuntu, installed from a dated
|
||||||
|
snapshot of its archive and checked by hash. The Ubuntu image has no CA
|
||||||
|
certificates; without them Nix cannot download packages and `smallwebwaf`
|
||||||
|
cannot reach GeoJS.
|
||||||
|
- Nix and nixpkgs, fixed at one release. Root uses Nix directly; no Nix daemon
|
||||||
|
runs. `nix-bin` creates no build users, so `build-users-group =` in
|
||||||
|
`/etc/nix/nix.conf` lets root build without them.
|
||||||
|
- runit, and `runsvinit` as the entrypoint, as the owner's code style guide asks
|
||||||
|
for service containers. No package carries `runsvinit`, so the image builds it
|
||||||
|
from its archived source (`github.com/peterbourgon/runsvinit`, unchanged
|
||||||
|
since 2015) at a fixed commit hash, with a `go.mod` the build writes.
|
||||||
|
`runsvinit` starts runit's `runsvdir`, which runs the `run` script of each
|
||||||
|
directory under `/etc/service` and runs it again whenever it exits. Ubuntu's
|
||||||
|
`sv` looks for services there too.
|
||||||
|
- The `smallwebwaf` binary, and a user of its own, `smallwebwaf` (uid and gid
|
||||||
|
65532).
|
||||||
|
- The service directory `/etc/service/smallwebwaf`. Its `run` script waits one
|
||||||
|
second (`sleep 1`), makes `SWWAF_STATE_DIR` and every file in it belong to the
|
||||||
|
`smallwebwaf` user, and starts `smallwebwaf` as that user with runit's
|
||||||
|
`chpst`.
|
||||||
|
- The directory `/var/lib/smallwebwaf` for the state files, and
|
||||||
|
`/etc/smallwebwaf/rules.d` with the default rule file.
|
||||||
|
- Port 8080 declared (`EXPOSE 8080`), and the health check (`HEALTHCHECK`) that
|
||||||
|
"The health check" in `SPEC.md` describes.
|
||||||
|
|
||||||
|
## Pinning
|
||||||
|
|
||||||
|
- Ubuntu's packages come from Ubuntu's snapshot service, the archive as it was
|
||||||
|
at the moment the Dockerfile names (`--snapshot 20261001T000000Z`), never
|
||||||
|
earlier than the pinned Ubuntu image's date. The Dockerfile names the SHA-256
|
||||||
|
hash of each of the snapshot's `InRelease` files, which cover every package
|
||||||
|
list and so every package, and the build checks them before installing. The
|
||||||
|
hashes are the amd64 archive's, so the image is built for amd64 only. The
|
||||||
|
snapshot is fetched over HTTPS with the CA certificates of the Go image that
|
||||||
|
builds `smallwebwaf`, itself pinned by digest, since the Ubuntu image has
|
||||||
|
none.
|
||||||
|
- nixpkgs comes from a release file on `releases.nixos.org` (`nixexprs.tar.xz`
|
||||||
|
of `nixos-26.05.11045.774debe7a0d1`), named with its SHA-256 hash, not from a
|
||||||
|
GitHub archive, whose bytes can change. It is set up for root as `nixpkgs`.
|
||||||
|
Root's Nix profile, `/nix/var/nix/profiles/default/bin`, is last on the
|
||||||
|
`PATH`, so that no package hides the image's own commands; busybox's `sv`, for
|
||||||
|
one, looks for services elsewhere. Unpacked, nixpkgs takes about 500 MiB.
|
||||||
|
- An app built on the image gets the same packages each time it is built.
|
||||||
|
Security fixes from Ubuntu and nixpkgs come with a newer `smallwebwaf` image,
|
||||||
|
which an app takes by changing the digest in its `FROM` line. The image moves
|
||||||
|
to each new nixpkgs release, every six months.
|
||||||
@@ -0,0 +1,83 @@
|
|||||||
|
# The metrics
|
||||||
|
|
||||||
|
`GET /_smallwebwaf/metrics` answers in the Prometheus text format. The
|
||||||
|
requirements are under "Metrics endpoint" in [`SPEC.md`](../SPEC.md); this file
|
||||||
|
lists the metrics. Every metric carries the label `instance`, set to
|
||||||
|
`SWWAF_INSTANCE_NAME`; Prometheus keeps it as `exported_instance` unless the
|
||||||
|
scrape sets `honor_labels: true`. No metric carries a client's address.
|
||||||
|
|
||||||
|
## Traffic
|
||||||
|
|
||||||
|
- `smallwebwaf_requests_total`, `smallwebwaf_request_bytes_total` and
|
||||||
|
`smallwebwaf_response_bytes_total`, by `status_class` such as `2xx` and by
|
||||||
|
`action`;
|
||||||
|
- `smallwebwaf_request_duration_seconds` and
|
||||||
|
`smallwebwaf_upstream_duration_seconds`, as histograms;
|
||||||
|
- `smallwebwaf_requests_in_flight`.
|
||||||
|
|
||||||
|
## Limits and bans
|
||||||
|
|
||||||
|
- `smallwebwaf_rate_limit_hits_total`, by `window` and by `kind` (`requests`,
|
||||||
|
`bytes` or `refusals`);
|
||||||
|
- `smallwebwaf_size_and_time_limit_hits_total`, by `limit`, the setting passed;
|
||||||
|
- `smallwebwaf_offences_total`, by `kind`;
|
||||||
|
- `smallwebwaf_bans_made_total`, by `cause`;
|
||||||
|
- `smallwebwaf_active_bans` and `smallwebwaf_permanent_bans`.
|
||||||
|
|
||||||
|
## Attack detection
|
||||||
|
|
||||||
|
- `smallwebwaf_rule_matches_total`, by `rule_id` and `action`, and
|
||||||
|
`smallwebwaf_rules_loaded`;
|
||||||
|
- `smallwebwaf_waf_matches_total`, by `mode` and `rule_id`, for the rules that
|
||||||
|
fired.
|
||||||
|
|
||||||
|
## Countries and AS numbers
|
||||||
|
|
||||||
|
For the `SWWAF_METRICS_TOP_N` busiest of each, the rest as `other`:
|
||||||
|
|
||||||
|
- `smallwebwaf_country_requests_total`,
|
||||||
|
`smallwebwaf_country_request_bytes_total`,
|
||||||
|
`smallwebwaf_country_response_bytes_total` and
|
||||||
|
`smallwebwaf_country_list_refusals_total`, by `country`;
|
||||||
|
- `smallwebwaf_asn_requests_total`, `smallwebwaf_asn_request_bytes_total` and
|
||||||
|
`smallwebwaf_asn_response_bytes_total`, by `asn`.
|
||||||
|
|
||||||
|
## Lookups
|
||||||
|
|
||||||
|
- `smallwebwaf_geojs_requests_total` and `smallwebwaf_geojs_failures_total`;
|
||||||
|
- `smallwebwaf_geojs_unanswered_total`: the requests that needed an answer and
|
||||||
|
went on without it;
|
||||||
|
- with the file source,
|
||||||
|
`smallwebwaf_lookup_database_last_read_timestamp_seconds` and
|
||||||
|
`smallwebwaf_lookup_database_read_failures_total`.
|
||||||
|
|
||||||
|
## Reputation sources
|
||||||
|
|
||||||
|
By `source`: each list's URL, each zone, and `abuseipdb`:
|
||||||
|
|
||||||
|
- `smallwebwaf_reputation_hits_total`, `smallwebwaf_reputation_queries_total`
|
||||||
|
and `smallwebwaf_reputation_failures_total`;
|
||||||
|
- `smallwebwaf_reputation_last_fetch_timestamp_seconds`, for a list;
|
||||||
|
- `smallwebwaf_reputation_daily_budget_remaining`, for AbuseIPDB.
|
||||||
|
|
||||||
|
## State files
|
||||||
|
|
||||||
|
By `file`:
|
||||||
|
|
||||||
|
- `smallwebwaf_state_file_writes_total`,
|
||||||
|
`smallwebwaf_state_file_write_failures_total`,
|
||||||
|
`smallwebwaf_state_file_last_write_timestamp_seconds` and
|
||||||
|
`smallwebwaf_state_file_size_bytes`;
|
||||||
|
- `smallwebwaf_state_file_edits_taken_in_total` and
|
||||||
|
`smallwebwaf_state_file_edits_set_aside_total`;
|
||||||
|
- `smallwebwaf_tracked_clients`: the clients in the table.
|
||||||
|
|
||||||
|
## Remote log, alerts and the process
|
||||||
|
|
||||||
|
- while a syslog server is set, `smallwebwaf_remote_log_lines_sent_total`,
|
||||||
|
`smallwebwaf_remote_log_lines_dropped_total` and
|
||||||
|
`smallwebwaf_remote_log_buffer_depth`;
|
||||||
|
- by `destination`, `smallwebwaf_alerts_sent_total`,
|
||||||
|
`smallwebwaf_alerts_failed_total`, `smallwebwaf_alerts_suppressed_total` and
|
||||||
|
`smallwebwaf_alerts_dropped_total`;
|
||||||
|
- Go's own `go_` metrics and the process's `process_` metrics.
|
||||||
@@ -0,0 +1,128 @@
|
|||||||
|
# The request log's fields
|
||||||
|
|
||||||
|
`smallwebwaf` writes one JSON object per line on stdout for every request. The
|
||||||
|
requirements are under "Request log" in [`SPEC.md`](../SPEC.md); this file lists
|
||||||
|
the fields. A field that does not apply to a request is left out.
|
||||||
|
|
||||||
|
## Example
|
||||||
|
|
||||||
|
```
|
||||||
|
{"type":"request","time":"2026-10-03T12:00:00.123Z","instance":"fsn1app1/gitea","client_ip":"203.0.113.9","method":"GET","scheme":"https","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","request_id":"7Q2NHZ4KJ3VXW5YB6R3MEFTD2A","peer_ip":"172.18.0.2","forwarded_for":"203.0.113.9","client_group":"203.0.113.9/32","asn":"AS64496","as_name":"Example Net","country":"DE","request_headers":{"accept":"*/*"},"response_content_type":"text/html; charset=utf-8","upstream_status":200,"action":"forward","counts":{"minute":1,"hour":12,"day":40,"minute_bytes":5120,"hour_bytes":61440,"day_bytes":204800},"duration_total":3.217,"duration_checks":0.041,"duration_upstream_connect":0.052,"duration_upstream_first_byte":2.874,"duration_upstream_total":3.104}
|
||||||
|
```
|
||||||
|
|
||||||
|
## Standard web log fields
|
||||||
|
|
||||||
|
- `time`: when the request arrived, RFC 3339 with milliseconds, in UTC;
|
||||||
|
- `instance`: `SWWAF_INSTANCE_NAME`;
|
||||||
|
- `client_ip`, `method`, `scheme`, `host`, `path`, `query`, `protocol`;
|
||||||
|
- `status`: what the client was sent, `0` if nothing;
|
||||||
|
- `request_bytes` and `response_bytes`: body bytes each way;
|
||||||
|
- `referer` and `user_agent`.
|
||||||
|
|
||||||
|
## Request detail
|
||||||
|
|
||||||
|
- `request_id`: the `X-Request-ID` a trusted proxy sent, or a new random one of
|
||||||
|
26 letters and digits. It goes to the app in `X-Request-ID`.
|
||||||
|
- `peer_ip`: the TCP peer, normally traefik.
|
||||||
|
- `forwarded_for`: the `X-Forwarded-For` header as received.
|
||||||
|
- `client_group`: the client as rate limiting counts it, an IPv4 address as a
|
||||||
|
/32 or an IPv6 group.
|
||||||
|
- `asn`, `as_name` and `country`: the client's, as looked up. Each is empty
|
||||||
|
when:
|
||||||
|
- lookups are off;
|
||||||
|
- the client is not looked up;
|
||||||
|
- the source cannot place the client;
|
||||||
|
- GeoJS had not answered by the time the request went on.
|
||||||
|
- `content_type` and `content_length`: the request's own.
|
||||||
|
- `request_headers`: the headers listed in `SWWAF_LOG_REQUEST_HEADERS`, by name
|
||||||
|
in lower case.
|
||||||
|
- `has_authorization` and `has_cookie`: true when the request had those headers.
|
||||||
|
Their values are never logged.
|
||||||
|
- `websocket`: true when the app switched the connection to another protocol.
|
||||||
|
|
||||||
|
## Response detail
|
||||||
|
|
||||||
|
- `upstream_status`: what the app answered. It differs from `status` when
|
||||||
|
`smallwebwaf` answered itself.
|
||||||
|
- `response_content_type`, `cache_control` and `location`: the answer's headers.
|
||||||
|
- `aborted`: true when the client went away early.
|
||||||
|
|
||||||
|
## Decision
|
||||||
|
|
||||||
|
- `action`: what was done:
|
||||||
|
- `forward`: passed to the app;
|
||||||
|
- `denied`: refused for `SWWAF_DENY_NETS`, or for a reputation source with
|
||||||
|
the action `deny`;
|
||||||
|
- `banned`: refused under a ban, or for a `ban` rule, a trap path or a
|
||||||
|
CrowdSec decision, each of which bans the client;
|
||||||
|
- `country_denied`, `rate_limited`, `rule_blocked` and `waf_blocked`:
|
||||||
|
refused by that check;
|
||||||
|
- `too_large` and `timed_out`: a size or time limit was passed;
|
||||||
|
- `upstream_error`: the app could not be reached, or its answer broke off;
|
||||||
|
- `admin`: answered at one of `smallwebwaf`'s own endpoints.
|
||||||
|
- `would_action`: in `observe` mode, what `enforce` mode would have done with a
|
||||||
|
request it would have refused. `action` then names what was done, normally
|
||||||
|
`forward`.
|
||||||
|
- `limit_percent` and `limit_percent_setting`: the client's percentage of the
|
||||||
|
rate limits and the setting that lowered it, when one did. `bytes_percent` and
|
||||||
|
`bytes_percent_setting` do the same for the byte limits.
|
||||||
|
- `counts`: the client's requests in the minute, hour and day as rate limiting
|
||||||
|
counts them, this request included, as `minute`, `hour` and `day`; its bytes
|
||||||
|
as `minute_bytes`, `hour_bytes` and `day_bytes`. A count can have a fraction,
|
||||||
|
since each window is counted in two buckets with the earlier one weighted.
|
||||||
|
`counts` is left out when rate limiting did not count the request.
|
||||||
|
- `limit_hit`: the limit broken: `minute`, `hour` or `day` for a rate limit;
|
||||||
|
`minute_bytes`, `hour_bytes` or `day_bytes` for a byte limit; or
|
||||||
|
`error_burst`. A request whose bytes broke a limit was passed on whole, so its
|
||||||
|
`action` stays `forward`.
|
||||||
|
- `rule_ids`: the rule file rules that matched, in order, up to the one that
|
||||||
|
refused.
|
||||||
|
- `waf_score` and `waf_rule_ids`: the Core Rule Set's score and the rules that
|
||||||
|
matched, for a request it inspected.
|
||||||
|
- `reputation`: the sources the client is on, in this order:
|
||||||
|
- blocklist URLs;
|
||||||
|
- the CrowdSec decision list's URL;
|
||||||
|
- DNS blocklist zones;
|
||||||
|
- `abuseipdb`.
|
||||||
|
- `offence`: the kind of offence recorded, when one was.
|
||||||
|
- `ban_expires`: when the client's ban ends, for a request that made one or was
|
||||||
|
refused under one: a time, or `permanent`.
|
||||||
|
|
||||||
|
## Timings
|
||||||
|
|
||||||
|
All in milliseconds:
|
||||||
|
|
||||||
|
- `duration_total`: from the request's headers being read to its log line;
|
||||||
|
- `duration_checks`: to the end of the checks, before forwarding;
|
||||||
|
- `duration_waf`: the part of that the Core Rule Set took;
|
||||||
|
- `duration_upstream_connect`: from the hand-over to the app until there was a
|
||||||
|
connection;
|
||||||
|
- `duration_upstream_first_byte`: until the first byte of the answer;
|
||||||
|
- `duration_upstream_total`: until the end of the answer.
|
||||||
|
|
||||||
|
## Every line
|
||||||
|
|
||||||
|
Every line has `type`, the standard web log fields, `request_id`, `peer_ip`,
|
||||||
|
`client_group`, `asn`, `as_name`, `country`, `action` and `duration_total`. The
|
||||||
|
process's own messages share the stream with `"type":"process"`; request lines
|
||||||
|
carry `"type":"request"`.
|
||||||
|
|
||||||
|
## Sending the log to a syslog server
|
||||||
|
|
||||||
|
While `SWWAF_LOG_REMOTE_URL` is set, every line on stdout is also sent to that
|
||||||
|
server, as the message of an RFC 5424 syslog record. Over UDP each record is one
|
||||||
|
datagram. Over TCP and TLS each record follows its length in bytes and a space.
|
||||||
|
stdout is unchanged. The record carries:
|
||||||
|
|
||||||
|
- the facility from `SWWAF_LOG_REMOTE_FACILITY`;
|
||||||
|
- the severity informational;
|
||||||
|
- the time;
|
||||||
|
- the host's name;
|
||||||
|
- the app name from `SWWAF_LOG_REMOTE_APP_NAME`.
|
||||||
|
|
||||||
|
Lines wait in a buffer of `SWWAF_LOG_REMOTE_BUFFER` lines, so a server that is
|
||||||
|
slow or down never holds up a request. When the buffer is full, the oldest line
|
||||||
|
is dropped. A failed connection is tried again a second later, and twice as long
|
||||||
|
after each further failure, up to a minute. A line too long for one UDP datagram
|
||||||
|
is dropped alone. As `smallwebwaf` stops, it spends at most two seconds sending
|
||||||
|
the lines still waiting, and drops the rest. stdout has carried them.
|
||||||
@@ -1,8 +1,8 @@
|
|||||||
// Package anomaly counts requests and bytes over a minute and an hour, per
|
// Package anomaly counts requests and bytes over a minute and an hour, per
|
||||||
// client, per surrounding netblock, per AS number, for the whole service
|
// client, per surrounding netblock, per AS number, for the whole service
|
||||||
// and per named netblock, and raises an anomaly alert for a count over its
|
// and per named netblock, and raises an anomaly alert for a count over its
|
||||||
// threshold, as "Anomaly thresholds" under "Configuration surface" in
|
// threshold, as "Anomaly thresholds" under "Settings" in SPEC.md
|
||||||
// SPEC.md describes. It refuses and bans nothing. At most 20,000 counters
|
// describes. It refuses and bans nothing. At most 20,000 counters
|
||||||
// are kept, in memory, and written to alerts.json and read from it by the
|
// are kept, in memory, and written to alerts.json and read from it by the
|
||||||
// state package.
|
// state package.
|
||||||
package anomaly
|
package anomaly
|
||||||
|
|||||||
+2
-2
@@ -1,8 +1,8 @@
|
|||||||
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
// Package waf runs the OWASP Core Rule Set 4.25.0, through Coraza, on the
|
||||||
// method, the URL with its query and the headers of a request, and on its
|
// method, the URL with its query and the headers of a request, and on its
|
||||||
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
|
// body while SWWAF_WAF_BODY_LIMIT is set, with the six changes smallwebwaf
|
||||||
// makes to it, as "Attack detection" under "Configuration surface" in
|
// makes to it, as "Attack detection" under "Settings" in SPEC.md describes
|
||||||
// SPEC.md describes them. It reads no response.
|
// them. It reads no response.
|
||||||
//
|
//
|
||||||
// smallwebwaf writes only to its state directory, so Coraza is built with
|
// smallwebwaf writes only to its state directory, so Coraza is built with
|
||||||
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
// its no_fs_access tag, as the Dockerfile and script/build build it: of a
|
||||||
|
|||||||
@@ -2,11 +2,12 @@
|
|||||||
# script/gitea-requests: build the image and run gitea 1.27.3 behind it,
|
# script/gitea-requests: build the image and run gitea 1.27.3 behind it,
|
||||||
# with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
|
# with every setting of smallwebwaf at its default but SWWAF_UPSTREAM_URL,
|
||||||
# which names gitea's container. Then send through it the gitea requests
|
# which names gitea's container. Then send through it the gitea requests
|
||||||
# that reading gitea's source found the Core Rule Set refuses, and a short
|
# that reading gitea's source found the Core Rule Set refuses, which
|
||||||
# sample of gitea's ordinary use: signing in, pages of files and commits,
|
# docs/gitea.md records, and a short sample of gitea's ordinary use:
|
||||||
# a diff, the API, a push, clone and fetch over HTTP, and two container
|
# signing in, pages of files and commits, a diff, the API, a push, clone
|
||||||
# image pushes. Last, print each request smallwebwaf logged, with what it
|
# and fetch over HTTP, and two container image pushes. Last, print each
|
||||||
# did with it and the ids of the Core Rule Set's rules that matched it.
|
# request smallwebwaf logged, with what it did with it and the ids of the
|
||||||
|
# Core Rule Set's rules that matched it.
|
||||||
# Every request comes from one client, docker's gateway, so more than
|
# Every request comes from one client, docker's gateway, so more than
|
||||||
# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
|
# SWWAF_ERROR_BURST_THRESHOLD, 30, refusals in a minute would ban it, and
|
||||||
# the requests after would show as banned. The containers, the network,
|
# the requests after would show as banned. The containers, the network,
|
||||||
|
|||||||
Reference in New Issue
Block a user