The image apps build FROM, with its health check (closes #45)
check / check (push) Failing after 3s
check / check (push) Failing after 3s
The Dockerfile's last stage is now the image of "Deployment" in SPEC.md: Ubuntu 26.04 with ca-certificates, nix-bin and runit from a dated snapshot whose InRelease files are checked by hash, nixpkgs from its release file checked by SHA-256, runsvinit built at a fixed commit, and smallwebwaf as a runit service. smallwebwaf answers /_smallwebwaf/healthz, and `smallwebwaf healthcheck`, which takes no further argument, is the image's HEALTHCHECK. script/example-app builds an app on the image and checks it end to end. The Nix profile comes last on the PATH: first, busybox from nixpkgs replaced runit's own runsvdir and sv. SPEC.md is corrected to match what was built. Model: opus-5-5
This commit was merged in pull request #57.
This commit is contained in:
@@ -11,33 +11,38 @@ request with defaults chosen for a service on the open internet. It keeps its
|
||||
state in memory and in JSON files you can read and edit, and writes a detailed
|
||||
JSON log line for every request.
|
||||
|
||||
Status: the first milestone is built
|
||||
(https://git.eeqj.de/sneak/smallwebwaf/issues/13), and the rate limits and
|
||||
country lists of the second (https://git.eeqj.de/sneak/smallwebwaf/issues/14).
|
||||
`smallwebwaf` passes each request to the app and the app's answer back,
|
||||
unchanged, within its timeouts and size limits, works out each client's address,
|
||||
refuses a client that sends too many requests or comes from a country you
|
||||
refuse, and writes a JSON log line for every request. The image an app builds on
|
||||
comes with the rest of milestone 2, and the rest of the design after that, in
|
||||
the order of the build order in [`SPEC.md`](SPEC.md). The survey of existing
|
||||
tools that led to the design is in [`EVALUATION.md`](EVALUATION.md).
|
||||
Status: the first two milestones are built
|
||||
(https://git.eeqj.de/sneak/smallwebwaf/issues/13 and
|
||||
https://git.eeqj.de/sneak/smallwebwaf/issues/14). `smallwebwaf` passes each
|
||||
request to the app and the app's answer back, unchanged, within its timeouts and
|
||||
size limits, works out each client's address, refuses a client that sends too
|
||||
many requests or comes from a country you refuse, and writes a JSON log line for
|
||||
every request. It comes as the image the app's own image is built on. The rest
|
||||
of the design comes after that, in the order of the build order in
|
||||
[`SPEC.md`](SPEC.md). The survey of existing tools that led to the design is in
|
||||
[`EVALUATION.md`](EVALUATION.md).
|
||||
|
||||
## Getting started
|
||||
|
||||
`smallwebwaf` is one Go binary. Until milestone 2 brings its image, build and
|
||||
run it from a clone, with Go installed:
|
||||
Build the `smallwebwaf` image from a clone:
|
||||
|
||||
```sh
|
||||
git clone https://git.eeqj.de/sneak/smallwebwaf.git
|
||||
cd smallwebwaf
|
||||
make build
|
||||
SWWAF_UPSTREAM_URL=http://127.0.0.1:3000 ./bin/smallwebwaf
|
||||
make docker
|
||||
```
|
||||
|
||||
It then listens on port 8080 and passes every request to the app at
|
||||
`SWWAF_UPSTREAM_URL`, here an app on port 3000; with no setting at all, to an
|
||||
app on `127.0.0.1:8081`. On `SIGTERM` or `SIGINT` it stops taking requests and
|
||||
gives those in progress five seconds to finish.
|
||||
`make docker` runs the tests and the linter, then builds the image, tagged
|
||||
`smallwebwaf`, for amd64 and only on an amd64 host: the hashes the `Dockerfile`
|
||||
checks Ubuntu's package lists against are those of Ubuntu's amd64 archive. Push
|
||||
it to a registry your hosts pull from, and build each app's image on it, pinned
|
||||
by digest, as "How it works, in short" below shows. `make example-app` builds a
|
||||
small app on the image, the one in `deploy/example-app`, and checks that it
|
||||
works.
|
||||
|
||||
To work on the code, `make build` builds the binary alone, with Go installed,
|
||||
and `make run` builds and runs it, listening on port 8080 in front of an app at
|
||||
`SWWAF_UPSTREAM_URL`, by default `http://127.0.0.1:8081`.
|
||||
|
||||
## What it does so far
|
||||
|
||||
@@ -79,6 +84,8 @@ gives those in progress five seconds to finish.
|
||||
lookup" below); with neither set, no visitor's address leaves the host. A
|
||||
client on a private, loopback or link-local address has no country, and
|
||||
neither list checks it.
|
||||
- Answers `GET /_smallwebwaf/healthz` itself with `200` and `ok`, before any
|
||||
check and without asking the app, for the image's health check.
|
||||
- Writes a line in the request log for each request (see "Request log" below).
|
||||
|
||||
## Settings
|
||||
@@ -155,8 +162,9 @@ refused ones included:
|
||||
- `action` is `forward` for a request passed to the app, `country_denied` for
|
||||
one refused for its client's country, `rate_limited` for one refused for a
|
||||
rate limit, `too_large` for a request or response over its size limit,
|
||||
`timed_out` for one that ran out of time, and `upstream_error` when the app
|
||||
could not be reached or its answer broke off.
|
||||
`timed_out` for one that ran out of time, `upstream_error` when the app could
|
||||
not be reached or its answer broke off, and `admin` for one `smallwebwaf`
|
||||
answered at its own endpoint.
|
||||
- `limit_hit` is there for a request refused for a rate limit, and names the
|
||||
window whose limit it went over: `minute`, `hour` or `day`, the shortest if it
|
||||
went over several.
|
||||
@@ -353,9 +361,9 @@ main "$@"
|
||||
side by side, each as its own user, and start either again a second after it
|
||||
exits. Leave out `ENTRYPOINT` and `USER` from the app's Dockerfile.
|
||||
- `nix-env -iA nixpkgs.<name>` installs a package from the nixpkgs in the image,
|
||||
and the app finds it on its `PATH`. That nixpkgs is fixed at one commit, so
|
||||
the same `smallwebwaf` image always gives the app the same packages; newer
|
||||
ones come with a newer `smallwebwaf` image.
|
||||
and the app finds it on its `PATH`, after Ubuntu's own commands. That nixpkgs
|
||||
is fixed at one commit, so the same `smallwebwaf` image always gives the app
|
||||
the same packages; newer ones come with a newer `smallwebwaf` image.
|
||||
- Deploy it as you deploy any app, with traefik's labels on this one container
|
||||
pointing at port 8080. upaas needs no change for this.
|
||||
- The app has to trust `127.0.0.1` and `::1` for forwarded headers, besides the
|
||||
@@ -373,7 +381,10 @@ main "$@"
|
||||
the health check on `127.0.0.1`.
|
||||
- `smallwebwaf` keeps its state files in `/var/lib/smallwebwaf`. Mount a volume
|
||||
there to keep bans and client history when a deploy replaces the container;
|
||||
without one, it still starts.
|
||||
without one, it still starts. The state files come in milestone 3 or later;
|
||||
until then it writes nothing to disk and needs no volume.
|
||||
- `docker stop` has runit stop both processes. `smallwebwaf` then stops taking
|
||||
requests and gives those in progress five seconds to finish.
|
||||
|
||||
A rule file is one rule per line: a name, what to match against, what to do, and
|
||||
a regex.
|
||||
@@ -444,7 +455,8 @@ refusal comes with `SWWAF_ALLOW_NETS` in milestone 3 or later.
|
||||
|
||||
- `cmd/smallwebwaf`: the binary, which only calls `internal/smallwebwaf`.
|
||||
- `internal/smallwebwaf`: the process: it reads the settings, listens, serves
|
||||
requests until `SIGTERM` or `SIGINT`, and stops.
|
||||
requests until `SIGTERM` or `SIGINT`, and stops. Run as
|
||||
`smallwebwaf healthcheck`, it is the image's health check instead.
|
||||
- `internal/config`: reads the settings, the one place they are read.
|
||||
- `internal/proxy`: what happens to each request: it works out the client, runs
|
||||
the checks, passes the request to the app and the answer back with the
|
||||
@@ -458,6 +470,11 @@ refusal comes with `SWWAF_ALLOW_NETS` in milestone 3 or later.
|
||||
it over a rate limit.
|
||||
- `internal/requestlog`: the lines on stdout: the request log line and the
|
||||
process's own messages.
|
||||
- `Dockerfile`: the lint and test phases, then the image, whose last stage
|
||||
installs Ubuntu's packages, nixpkgs, `runsvinit` and `smallwebwaf`, with
|
||||
`share/smallwebwaf.run` as runit's `run` script for `smallwebwaf`.
|
||||
- `deploy/example-app`: an app built on the image, which `script/example-app`
|
||||
checks.
|
||||
|
||||
Besides the Go standard library, `github.com/hashicorp/golang-lru/v2` keeps the
|
||||
table of clients to 20,000 and the GeoJS answers to 100,000, dropping the least
|
||||
@@ -472,7 +489,7 @@ the `Makefile` targets are thin shims that call them. The scripts are POSIX sh,
|
||||
so that they run in minimal containers.
|
||||
|
||||
- `script/bootstrap`: installs what the other scripts need on the host: `make`,
|
||||
`git`, Go for `gofmt`, node and yarn, and prettier.
|
||||
`git`, `curl`, Go for `gofmt`, node and yarn, and prettier.
|
||||
- `script/setup`: readies a fresh clone: runs `script/bootstrap`, then
|
||||
`script/install-precommit`.
|
||||
- `script/projectname`: prints the project's name, `smallwebwaf`, which
|
||||
@@ -492,13 +509,16 @@ so that they run in minimal containers.
|
||||
working on the code by hand; `make build` runs it.
|
||||
- `script/run`: builds `bin/smallwebwaf` with `script/build` and runs it;
|
||||
`make run` runs it.
|
||||
- `script/example-app`: builds the image and, on it, the example app in
|
||||
`deploy/example-app`, runs it, and checks that the health check passes, that a
|
||||
request reaches the app through `smallwebwaf`, and that `sv stop` and
|
||||
`docker stop` stop it in order; then removes the container and both images. It
|
||||
needs network access, for nixpkgs' binary cache, and `script/check` does not
|
||||
run it; `make example-app` does.
|
||||
|
||||
## TODO
|
||||
|
||||
- Milestone 2: the image an app builds on
|
||||
(https://git.eeqj.de/sneak/smallwebwaf/issues/14); its rate limits and country
|
||||
lists are built.
|
||||
- The rest of the design, in the order of the build order in
|
||||
- Milestone 3 and the rest of the design, in the order of the build order in
|
||||
[`SPEC.md`](SPEC.md).
|
||||
|
||||
## Documents
|
||||
|
||||
Reference in New Issue
Block a user