Serve Prometheus metrics behind SWWAF_METRICS_TOKEN #76

Merged
clawbot merged 1 commits from issue-23-metrics into next 2026-10-06 11:40:28 +02:00
Collaborator

Builds #23 as its plan comment describes.

  • GET /_smallwebwaf/metrics serves the Prometheus text format, through github.com/prometheus/client_golang, to a request carrying SWWAF_METRICS_TOKEN as a bearer token; 401 without it, 404 while it is unset. A token shorter than 32 characters stops the start; the settings log masks it.
  • Every request under /_smallwebwaf/ but the health check now passes the checks first and is answered where it would be forwarded, 404 for any path but the metrics. None reaches the app any more: POST /_smallwebwaf/healthz, forwarded before, is now 404.
  • In clients.json a 401 counts as refused; the metrics answer and the 404s count as neither forwarded nor refused. Before, anything not forwarded counted as refused.
  • internal/metrics defines every metric; README "Metrics" lists them. Size and time limit hits are labelled with the setting's name. The metrics by country count only requests whose client's country is known, so only while a country list is set. The SWWAF_METRICS_TOP_N busiest countries since the start have series, the rest count as other; a country pushed out loses its series, so no series ever goes down.

Disclosures:

  • Judgement call: github.com/slok/go-http-metrics, listed beside client_golang in the package defaults, is not used: its metric names and labels do not fit "Metrics endpoint".
  • Deviation: go.mod and go.sum written by hand from the module proxy and sum.golang.org, as go runs only through make. The tests use client_golang's testutil, which adds github.com/kylelemons/godebug to go.mod; its hashes were already in go.sum.
  • Deviation: no metrics for state files read again after an edit or edits set aside; they come with #75, not merged yet.
  • Unverified: not scraped by a real Prometheus.

Model: opus-5-5

Builds https://git.eeqj.de/sneak/smallwebwaf/issues/23 as its plan comment describes. - `GET /_smallwebwaf/metrics` serves the Prometheus text format, through `github.com/prometheus/client_golang`, to a request carrying `SWWAF_METRICS_TOKEN` as a bearer token; `401` without it, `404` while it is unset. A token shorter than 32 characters stops the start; the settings log masks it. - Every request under `/_smallwebwaf/` but the health check now passes the checks first and is answered where it would be forwarded, `404` for any path but the metrics. None reaches the app any more: `POST /_smallwebwaf/healthz`, forwarded before, is now `404`. - In `clients.json` a `401` counts as refused; the metrics answer and the `404`s count as neither forwarded nor refused. Before, anything not forwarded counted as refused. - `internal/metrics` defines every metric; README "Metrics" lists them. Size and time limit hits are labelled with the setting's name. The metrics by country count only requests whose client's country is known, so only while a country list is set. The `SWWAF_METRICS_TOP_N` busiest countries since the start have series, the rest count as `other`; a country pushed out loses its series, so no series ever goes down. Disclosures: - Judgement call: `github.com/slok/go-http-metrics`, listed beside client_golang in the package defaults, is not used: its metric names and labels do not fit "Metrics endpoint". - Deviation: `go.mod` and `go.sum` written by hand from the module proxy and sum.golang.org, as `go` runs only through `make`. The tests use client_golang's `testutil`, which adds `github.com/kylelemons/godebug` to `go.mod`; its hashes were already in `go.sum`. - Deviation: no metrics for state files read again after an edit or edits set aside; they come with https://git.eeqj.de/sneak/smallwebwaf/pulls/75, not merged yet. - Unverified: not scraped by a real Prometheus. Model: opus-5-5
clawbot self-assigned this 2026-10-06 10:35:47 +02:00
clawbot added the needs-review label 2026-10-06 10:35:51 +02:00
Author
Collaborator

Review failed.

  1. internal/proxy/request.go:358 (addToHistory) and README.md:287: a 401 for a missing or wrong token is counted as neither forwarded nor refused in clients.json. SPEC.md "Data flow for one request" calls that request one smallwebwaf refused ("refused ... for a missing or wrong token"). The code's own meaning of refused, "refused before anything reached it", fits it too. As it stands, a client guessing tokens shows no refusals in its history. Acceptable: a 401 counts as refused, the metrics answer and the 404s stay neither, and the README sentence and the history test follow.
  2. README.md:345: the GeoJS bullet gives one description, "the requests whose client counted as coming from an unknown country because GeoJS had not answered in time", for three metrics. It fits only smallwebwaf_geojs_unanswered_total. Acceptable: say what each one counts: the requests to GeoJS, those that failed, and the requests left without an answer.
  3. internal/proxy/metrics_test.go: the limit label of smallwebwaf_size_and_time_limit_hits_total is tested only for a body announced over SWWAF_REQUEST_MAX_BYTES and for SWWAF_UPSTREAM_RESPONSE_TIMEOUT. The other places that set it are not tested: SWWAF_CLIENT_REQUEST_TIMEOUT, SWWAF_UPSTREAM_REQUEST_TIMEOUT and SWWAF_CLIENT_RESPONSE_TIMEOUT in request.go, SWWAF_RESPONSE_MAX_BYTES in request.go and bodies.go, and a body found over SWWAF_REQUEST_MAX_BYTES while it streams in bodies.go. A wrong or swapped name in any of them passes. Likewise, smallwebwaf_geojs_unanswered_total is not tested for a client that finds GeoJS left alone after a failure, or too many clients already waiting (answerOrWait, internal/lookup/lookup.go:226). Acceptable: a test for each case that fails when its label or count is wrong.

Judgement calls accepted:

  • A country pushed out of the SWWAF_METRICS_TOP_N busiest loses its series. No series goes down while it exists, and nothing is counted twice.
  • github.com/slok/go-http-metrics is not used.
  • go.mod and go.sum were written by hand.
  • The metrics for state-file edits are left for #75. Nothing on that PR or on #68 records them yet.

Model: opus-5-5

Review failed. 1. `internal/proxy/request.go:358` (`addToHistory`) and `README.md:287`: a `401` for a missing or wrong token is counted as neither forwarded nor refused in `clients.json`. `SPEC.md` "Data flow for one request" calls that request one `smallwebwaf` refused ("refused ... for a missing or wrong token"). The code's own meaning of refused, "refused before anything reached it", fits it too. As it stands, a client guessing tokens shows no refusals in its history. Acceptable: a `401` counts as refused, the metrics answer and the `404`s stay neither, and the README sentence and the history test follow. 2. `README.md:345`: the GeoJS bullet gives one description, "the requests whose client counted as coming from an unknown country because GeoJS had not answered in time", for three metrics. It fits only `smallwebwaf_geojs_unanswered_total`. Acceptable: say what each one counts: the requests to GeoJS, those that failed, and the requests left without an answer. 3. `internal/proxy/metrics_test.go`: the `limit` label of `smallwebwaf_size_and_time_limit_hits_total` is tested only for a body announced over `SWWAF_REQUEST_MAX_BYTES` and for `SWWAF_UPSTREAM_RESPONSE_TIMEOUT`. The other places that set it are not tested: `SWWAF_CLIENT_REQUEST_TIMEOUT`, `SWWAF_UPSTREAM_REQUEST_TIMEOUT` and `SWWAF_CLIENT_RESPONSE_TIMEOUT` in `request.go`, `SWWAF_RESPONSE_MAX_BYTES` in `request.go` and `bodies.go`, and a body found over `SWWAF_REQUEST_MAX_BYTES` while it streams in `bodies.go`. A wrong or swapped name in any of them passes. Likewise, `smallwebwaf_geojs_unanswered_total` is not tested for a client that finds GeoJS left alone after a failure, or too many clients already waiting (`answerOrWait`, `internal/lookup/lookup.go:226`). Acceptable: a test for each case that fails when its label or count is wrong. Judgement calls accepted: - A country pushed out of the `SWWAF_METRICS_TOP_N` busiest loses its series. No series goes down while it exists, and nothing is counted twice. - `github.com/slok/go-http-metrics` is not used. - `go.mod` and `go.sum` were written by hand. - The metrics for state-file edits are left for https://git.eeqj.de/sneak/smallwebwaf/pulls/75. Nothing on that PR or on https://git.eeqj.de/sneak/smallwebwaf/issues/68 records them yet. Model: opus-5-5
clawbot added needs-rework and removed needs-review labels 2026-10-06 10:54:26 +02:00
clawbot added 1 commit 2026-10-06 11:18:45 +02:00
GET /_smallwebwaf/metrics answers in the Prometheus text format for a
request carrying SWWAF_METRICS_TOKEN, 401 without it and 404 while it is
unset. Every request under /_smallwebwaf/ but the health check now goes
through the checks and is answered where it would be forwarded, 404 for
any path but the metrics, so none reaches the app. In the client's
history a 401 counts as refused, the metrics and the 404s as neither.
SWWAF_METRICS_TOP_N bounds the series by country, the rest counted as
other.

Deviation: go.mod and go.sum written by hand, as go runs only through
make.
Deviation: no metrics yet for state files read again after an edit or
edits set aside; that work is not merged.

Model: opus-5-5
clawbot force-pushed issue-23-metrics from 2776bb4b09 to 99702de60b 2026-10-06 11:18:45 +02:00 Compare
Author
Collaborator

Rework for #76 (comment):

  1. Fixed: the 401 is now answered as a refusal and counts as refused in clients.json; the metrics answer and the 404s stay neither; the README sentence, the code comments and the history test follow.
  2. Fixed: the README GeoJS bullet says what each of the three metrics counts.
  3. Fixed: each size and time limit's own test now checks its limit label at every place that sets it, and no series for a body at the limit; a new GeoJS test counts the requests left without an answer while GeoJS is left alone and once 10,000 clients wait, on the test's own clock. The separate size and time limit metrics test is folded into these.

#75 is not on next yet, so the metrics for state-file edits are still left for it.

Model: opus-5-5

Rework for https://git.eeqj.de/sneak/smallwebwaf/pulls/76#issuecomment-129104: 1. Fixed: the `401` is now answered as a refusal and counts as refused in `clients.json`; the metrics answer and the `404`s stay neither; the README sentence, the code comments and the history test follow. 2. Fixed: the README GeoJS bullet says what each of the three metrics counts. 3. Fixed: each size and time limit's own test now checks its `limit` label at every place that sets it, and no series for a body at the limit; a new GeoJS test counts the requests left without an answer while GeoJS is left alone and once 10,000 clients wait, on the test's own clock. The separate size and time limit metrics test is folded into these. https://git.eeqj.de/sneak/smallwebwaf/pulls/75 is not on `next` yet, so the metrics for state-file edits are still left for it. Model: opus-5-5
clawbot added needs-review and removed needs-rework labels 2026-10-06 11:19:04 +02:00
Author
Collaborator

Review passed.

Judgement calls accepted:

  • A 401 counts as refused in clients.json; the metrics answer and the 404s count as neither forwarded nor refused.
  • A country pushed out of the SWWAF_METRICS_TOP_N busiest loses its series.
  • The metrics by country count only requests whose client's country was looked up, so only while a country list is set.
  • github.com/slok/go-http-metrics is not used.
  • go.mod and go.sum were written by hand.
  • The metrics for state-file edits are left for #75.

Model: opus-5-5

Review passed. Judgement calls accepted: - A `401` counts as refused in `clients.json`; the metrics answer and the `404`s count as neither forwarded nor refused. - A country pushed out of the `SWWAF_METRICS_TOP_N` busiest loses its series. - The metrics by country count only requests whose client's country was looked up, so only while a country list is set. - `github.com/slok/go-http-metrics` is not used. - `go.mod` and `go.sum` were written by hand. - The metrics for state-file edits are left for https://git.eeqj.de/sneak/smallwebwaf/pulls/75. Model: opus-5-5
clawbot merged commit 234c5eac60 into next 2026-10-06 11:40:28 +02:00
clawbot deleted branch issue-23-metrics 2026-10-06 11:40:28 +02:00
clawbot removed the needs-review label 2026-10-06 11:40:28 +02:00
Sign in to join this conversation.