sneak/smallwebwaf
1
0
Fork 0
Code Issues 32 Pull Requests 3 Actions 23 Packages Projects Releases Wiki Activity

Pass-through proxy with timeouts, size limits and a request log (closes #13)
check / check (push) Successful in 1m29s
Details

Browse Source 
Milestone 1, the repo's first code. smallwebwaf passes each request to the app and the answer back unchanged, streaming bodies and WebSocket upgrades, within four timeouts (client and app, request and response) and two size limits, and writes one JSON line per request to stdout. Every setting has an SWWAF_ name and a default, and an invalid value stops the start. The repo gets the standard layout: script/ entrypoints, make targets that call them, a Dockerfile that runs the checks, and the Gitea workflow.

Disclosure: SPEC.md changed. Go's server reads the request line and headers before smallwebwaf sees the request, so slow headers are closed without an answer, and neither slow nor oversized headers get a log line.
Disclosure: standard library only.

Model: opus-5-5
This commit was merged in pull request #39.
This commit is contained in:
clawbot
2026-10-03 17:24:34 +02:00
parent fd77e76177
commit d76715b0df
47 changed files with 4917 additions and 74 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+187 -12
View File
@@ -1,18 +1,134 @@
# smallwebwaf
`smallwebwaf` is a simple, fast, logging web application firewall for people who
host their own services. It runs inside the container of the one application it
protects, between your reverse proxy (traefik) and the app: the app's Dockerfile
builds `FROM` the `smallwebwaf` image, traefik sends the app's requests to
`smallwebwaf` on port 8080, and `smallwebwaf` passes them on to the app on
`127.0.0.1:8081`. It needs no setting, and protects the app from the first
request with defaults chosen for a service on the open internet. It keeps its
state in memory and in JSON files you can read and edit, and writes a detailed
JSON log line for every request.
`smallwebwaf` is a simple, fast, logging web application firewall, written in Go
by [@sneak](https://sneak.berlin), for people who host their own services. It
runs inside the container of the one application it protects, between your
reverse proxy (traefik) and the app: the app's Dockerfile builds `FROM` the
`smallwebwaf` image, traefik sends the app's requests to `smallwebwaf` on port
8080, and `smallwebwaf` passes them on to the app on `127.0.0.1:8081`. It needs
no setting, and protects the app from the first request with defaults chosen for
a service on the open internet. It keeps its state in memory and in JSON files
you can read and edit, and writes a detailed JSON log line for every request.
Status: design stage. This repository currently holds the documents only; no
code has been written. The design is in [`SPEC.md`](SPEC.md), and the survey of
existing tools that led to it is in [`EVALUATION.md`](EVALUATION.md).
Status: the first milestone is built
(https://git.eeqj.de/sneak/smallwebwaf/issues/13). `smallwebwaf` passes each
request to the app and the app's answer back, unchanged, within its timeouts and
size limits, works out each client's address, and writes a JSON log line for
every request. Rate limits per client, the country lists and the image an app
builds on come with milestone 2
(https://git.eeqj.de/sneak/smallwebwaf/issues/14), and the rest of the design
after that, in the order of the build order in [`SPEC.md`](SPEC.md). The survey
of existing tools that led to the design is in [`EVALUATION.md`](EVALUATION.md).
## Getting started
`smallwebwaf` is one Go binary. Until milestone 2 brings its image, build and
run it from a clone, with Go installed:
```sh
git clone https://git.eeqj.de/sneak/smallwebwaf.git
cd smallwebwaf
make build
SWWAF_UPSTREAM_URL=http://127.0.0.1:3000 ./bin/smallwebwaf
```
It then listens on port 8080 and passes every request to the app at
`SWWAF_UPSTREAM_URL`, here an app on port 3000; with no setting at all, to an
app on `127.0.0.1:8081`. On `SIGTERM` or `SIGINT` it stops taking requests and
gives those in progress five seconds to finish.
## What milestone 1 does
- Passes each request to the app and the app's answer back unchanged: method,
path, query, headers, body and status. Bodies stream through in both
directions and are never held whole in memory. A WebSocket, or any other
upgraded connection, passes through, and the timeouts do not cut it.
- Works out the client's address. A TCP peer outside `SWWAF_TRUSTED_PROXIES` is
the client, and the forwarded headers it sends are replaced, not passed on.
For a peer inside it, `X-Forwarded-For` is read from the right, and the first
address outside `SWWAF_TRUSTED_PROXIES` is the client; if every address in it
is inside, the leftmost is, and with no header the peer is. The app sees what
it would see from traefik directly: the same `Host`, the same
`X-Forwarded-Proto`, and `X-Forwarded-For` with the peer added at the end.
- Enforces the four timeouts and the two size limits below. A limit passed
before the response has started gets `smallwebwaf`'s own answer: `408` for a
client too slow to send its request, `413` for a request body that is too
large, `504` for an app too slow to answer, and `502` for a response that is
too large or an app that cannot be reached. A request that announces a body
over the limit is refused before anything reaches the app. While a request
body is still on its way, a request timeout that runs out answers `408` if
`smallwebwaf` was waiting for the client to send more, and `504` if it was
waiting for the app to take what it had. Once the response has started, a
limit can only cut the connection.
- Writes a line in the request log for each request (see "Request log" below).
## Settings
Each setting is an environment variable, and each has a default, so none has to
be set. A setting that is set but invalid stops the start with a message naming
it, and the effective settings are logged at start.
- `SWWAF_LISTEN_ADDR` (default `:8080`): where `smallwebwaf` listens.
- `SWWAF_UPSTREAM_URL` (default `http://127.0.0.1:8081`): the app, as `http` or
`https`, a host and an optional port, and nothing more.
- `SWWAF_TRUSTED_PROXIES` (default `10.0.0.0/8,172.16.0.0/12,192.168.0.0/16`,
the private address ranges): the netblocks whose `X-Forwarded-For` is
believed. A list given replaces the default; set but empty, it trusts nothing.
- `SWWAF_CLIENT_REQUEST_TIMEOUT` (default `60s`): how long a client may take to
send its request line and headers, and then, from the end of the headers, its
body.
- `SWWAF_CLIENT_RESPONSE_TIMEOUT` (default `30m`): how long the response may
take to reach the client, from the end of the request to the last byte.
- `SWWAF_UPSTREAM_REQUEST_TIMEOUT` (default `60s`): how long connecting to the
app and sending it the whole request may take.
- `SWWAF_UPSTREAM_RESPONSE_TIMEOUT` (default `30m`): how long the app may take
to send its whole answer, from the end of the request to the last byte.
- `SWWAF_REQUEST_MAX_BYTES` (default `100M`): the largest request body.
- `SWWAF_RESPONSE_MAX_BYTES` (default `5G`): the largest response body.
Durations are in Go's syntax, with `d` for days (`90s`, `15m`, `7d`). Sizes are
bytes, with an optional `K`, `M` or `G`, which are powers of 1024 (`1K` is 1024
bytes). Netblocks are in CIDR form, and a bare address stands for itself alone.
`off` switches a timeout or a size limit off.
Two limits are fixed rather than settings: the request line and headers may take
up to 32 KiB, above which the answer is `431` and nothing reaches the app, and a
kept-open connection that sends nothing for 120 seconds is closed. That is
longer than the 90 seconds after which traefik closes a connection it is not
using, so traefik never sends a request on a connection `smallwebwaf` is
closing.
## Request log
`smallwebwaf` writes one JSON object per line on stdout for every request,
refused ones included:
```
{"type":"request","time":"2026-10-03T12:00:00.123Z","client_ip":"203.0.113.9","peer_ip":"172.18.0.2","method":"GET","host":"app.example","path":"/","query":"","protocol":"HTTP/1.1","status":200,"upstream_status":200,"request_bytes":0,"response_bytes":5120,"referer":"","user_agent":"curl/8.9.1","action":"forward","duration_total":3.217,"duration_upstream_total":3.104}
```
- `time` is when the request arrived, in UTC. `peer_ip` is the TCP peer,
normally traefik. `path` and `query` are as the client sent them.
- `status` is what the client was sent, `0` if nothing was; `upstream_status` is
what the app answered, and is left out when the app did not answer.
- `request_bytes` and `response_bytes` count body bytes.
- `action` is `forward` for a request passed to the app, `too_large` for a
request or response over its size limit, `timed_out` for one that ran out of
time, and `upstream_error` when the app could not be reached or its answer
broke off.
- `aborted` is there, and true, when the client went away early.
- `duration_total` and `duration_upstream_total` are in milliseconds.
No body and no other header is logged. `smallwebwaf`'s own messages (start, the
settings, stop, errors) share the stream as JSON lines marked
`"type":"process"`.
Go's HTTP server, on which `smallwebwaf` is built, reads a request's line and
headers before `smallwebwaf` sees the request, and some requests end there,
without a line in the log: headers over 32 KiB, which it answers `431`, headers
slower than `SWWAF_CLIENT_REQUEST_TIMEOUT`, whose connection it closes without
an answer, and requests it cannot read at all, which it answers itself, mostly
with `400`.
## Why
@@ -249,8 +365,67 @@ visitor on your local network, another container or your monitoring, has no
country: `SWWAF_EXCLUSIVELY_ALLOWED_COUNTRIES` refuses it unless you list it in
`SWWAF_ALLOW_NETS`. Such addresses are never sent to GeoJS.
## How the code is laid out
- `cmd/smallwebwaf`: the binary, which only calls `internal/smallwebwaf`.
- `internal/smallwebwaf`: the process: it reads the settings, listens, serves
requests until `SIGTERM` or `SIGINT`, and stops.
- `internal/config`: reads the settings, the one place they are read.
- `internal/proxy`: what happens to each request: it works out the client, runs
the checks, passes the request to the app and the answer back with the
standard library's `httputil.ReverseProxy` within the timeouts and size
limits, and writes the request's log line. Its `check` method is where
milestone 2's rate limits and country lists refuse a request.
- `internal/requestlog`: the lines on stdout: the request log line and the
process's own messages.
Only the Go standard library is used.
## Entrypoints
This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: the scripts in `script/` are the entrypoints for working on it, and
the `Makefile` targets are thin shims that call them. The scripts are POSIX sh,
so that they run in minimal containers.
- `script/bootstrap`: installs what the other scripts need on the host: `make`,
`git`, Go for `gofmt`, node and yarn, and prettier.
- `script/setup`: readies a fresh clone: runs `script/bootstrap`, then
`script/install-precommit`.
- `script/projectname`: prints the project's name, `smallwebwaf`, which
`script/docker` and the others tag their images with.
- `script/test`: runs the tests, as the `test` phase of the `Dockerfile`.
- `script/lint`: runs golangci-lint, as the `lint` phase of the `Dockerfile`.
- `script/fmt`: formats the Go code with `gofmt` and the Markdown with prettier.
- `script/fmt-check`: checks the formatting, and changes nothing.
- `script/check`: runs `script/test`, `script/lint` and `script/fmt-check`.
- `script/docker`: builds the image, whose build runs the tests and the linter
first.
- `script/cibuild`: what CI runs: `script/bootstrap`, `script/check`, then the
image build.
- `script/precommit`: run by the git pre-commit hook; runs `script/check`.
- `script/install-precommit`: installs that hook; `make hooks` runs it.
- `script/build`: builds `bin/smallwebwaf` on the host, with Go installed, for
working on the code by hand; `make build` runs it.
- `script/run`: builds `bin/smallwebwaf` with `script/build` and runs it;
`make run` runs it.
## TODO
- Milestone 2: rate limits per client, the country lists and the image an app
builds on (https://git.eeqj.de/sneak/smallwebwaf/issues/14).
- The licence (https://git.eeqj.de/sneak/smallwebwaf/issues/15).
- The rest of the design, in the order of the build order in
[`SPEC.md`](SPEC.md).
## Documents
- [`SPEC.md`](SPEC.md): the design.
- [`EVALUATION.md`](EVALUATION.md): what already exists, what each tool covers
and misses, and why none was adopted.
- [`REPO_POLICIES.md`](REPO_POLICIES.md): the policies this repository follows.
## Author
[@sneak](https://sneak.berlin)