README states what refused reports, one sender and nginx's 413 do (closes #145, closes #146, closes #147)
check / check (push) Waiting to run

The README said the page keeps refused samples and sends them later.
They live only in the 100-round history, so at the 1-second interval
one refused or failed report loses the oldest 20 seconds of each host's
samples, and a pause longer than the history loses everything since the
last delivered report. Report limits now says one address at the
defaults can fill the 1 GiB cap in about 17 minutes with reports made to
compress badly, and which settings bound it. In the image, nginx's
default client_max_body_size of 1 MiB answers an over-limit body with
413 and an HTML page before the backend sees it; the backend's own 413
sentences now say they apply to it run on its own.

Model: opus-5-5
This commit is contained in:
2026-10-18 23:40:10 +00:00
parent 007190b541
commit bec2ba4e09
2 changed files with 63 additions and 14 deletions
+48 -14
View File
@@ -153,8 +153,24 @@ report a delta, so only new samples are sent; the mark advances only on a
delivered report, and while paused nothing is sent. Changing the interval clears delivered report, and while paused nothing is sent. Changing the interval clears
each host's history on the page; the samples in it not yet reported still go in each host's history on the page; the samples in it not yet reported still go in
the next report. Delivery failure is quiet — one debug-log line per outage, the next report. Delivery failure is quiet — one debug-log line per outage,
retried at the next interval, never blocking probing. The report-building step never blocking probing. The report-building step is a pure function of host
is a pure function of host state. state.
A sample not yet delivered is kept only as long as the history holds it, so
after a refused or failed report the next report carries the samples the history
still has. The history keeps each host's last 100 rounds: 100 seconds at the
1-second interval, 200 seconds at 2 seconds, 5 minutes at the default of 3
seconds and 100 minutes at 60 seconds. At 1 second, one refused or failed report
loses the oldest 20 seconds of each host's samples, as the next delivered report
comes 2 minutes after the last. At 2 seconds and longer one loses nothing;
samples are lost from the third refused report in a row at 2 seconds, from the
fifth at 3 seconds, and later still at the longer intervals.
No report is sent while the page is paused, but each paused round still adds an
entry to the history, so each report due during a pause counts as a refused one.
A pause longer than the history spans loses every sample recorded between the
last delivered report and the pause: a minute's worth for each host if the
report before the pause was delivered.
### Backend ### Backend
@@ -240,17 +256,19 @@ Files are created with `O_EXCL` to prevent overwrites.
The reports of a failed write stay in memory, and the next flush, within a The reports of a failed write stay in memory, and the next flush, within a
minute, tries again to write them. From the failed write until a write succeeds, minute, tries again to write them. From the failed write until a write succeeds,
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it, so `POST /api/v1/reports` refuses every report with 500 and keeps nothing of it.
the page keeps its samples and sends them again at its next report. Reports The page sends the refused samples again in its next report, as long as its
already answered 200 are lost only if the server stops before a write succeeds; history still holds them: at the 1-second interval one refused report already
the final flush at stop tries them too, and if it fails, the process exits loses some (see [Reporting](#reporting)). Reports already answered 200 are lost
non-zero. only if the server stops before a write succeeds; the final flush at stop tries
them too, and if it fails, the process exits non-zero.
#### Report limits #### Report limits
`POST /api/v1/reports` takes reports from anyone who can reach it, without `POST /api/v1/reports` takes reports from anyone who can reach it, without
credentials, so it is bounded instead. Each refusal below answers with the same credentials, so it is bounded instead. Each refusal below answers with the same
`{"status":"error"}` body as any other error. `{"status":"error"}` body as any other error, except nginx's refusal of a body
over the limit in the image.
- **JSON only.** A report must be sent with `Content-Type: application/json`, - **JSON only.** A report must be sent with `Content-Type: application/json`,
parameters such as `charset` allowed, as the page sends it; any other is parameters such as `charset` allowed, as the page sends it; any other is
@@ -258,6 +276,13 @@ credentials, so it is bounded instead. Each refusal below answers with the same
send a `text/plain` POST, or a form, without a CORS preflight; refusing them send a `text/plain` POST, or a form, without a CORS preflight; refusing them
keeps any site from making its visitors' browsers store reports (see keeps any site from making its visitors' browsers store reports (see
[CORS](#cors)). [CORS](#cors)).
- **Body limit.** A report body may be at most 1 MiB, the backend's limit for
any request. Run on its own, the backend refuses a larger body with 413:
before the route when the request declares a length over the limit, otherwise
at the route once reading the body passes it. In the image, nginx refuses it
first, with or without a declared length, as its `client_max_body_size`, which
`configs/nginx.conf` does not set, is 1 MiB by default. nginx answers 413 with
its own HTML error page, and the backend never sees the request.
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may - **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
`Retry-After: 60`. The minute slides: reports from the minute before still `Retry-After: 60`. The minute slides: reports from the minute before still
@@ -281,6 +306,12 @@ credentials, so it is bounded instead. Each refusal below answers with the same
how much of the newest reports is kept: the default of 1 GiB is small enough how much of the newest reports is kept: the default of 1 GiB is small enough
for any host; set it to the space you can give `DATA_DIR`. for any host; set it to the space you can give `DATA_DIR`.
At the defaults, one address may send 60 reports of up to 1 MiB a minute, and a
report made up to compress badly takes about as much room in the report files as
its request body, so in about 17 minutes one address can fill the 1 GiB cap with
its own reports, every report stored before them deleted to make room. Lowering
`REPORTS_PER_MINUTE` or raising `DATA_DIR_MAX_BYTES` makes that take longer.
#### CORS #### CORS
The page calls the API from the origin it is served from, so by default the The page calls the API from the origin it is served from, so by default the
@@ -302,12 +333,15 @@ the metrics record its duration and response size, labelled with its path,
method and status; they also count those requests in progress, and include Go's method and status; they also count those requests in progress, and include Go's
runtime and process metrics. No other request is recorded: not those to runtime and process metrics. No other request is recorded: not those to
`/metrics` itself, and not those answered before they reach either route, such `/metrics` itself, and not those answered before they reach either route, such
as a CORS preflight, or a request refused with 404 for a path no route has, 405 as a CORS preflight, or a request refused with 404 for a path no route has or
for a method its route does not take, or 413 for declaring a body length over 405 for a method its route does not take. Run on its own, the backend also
the 1 MiB limit. A report whose body goes over the limit without declaring its answers 413 before the route to a request declaring a body length over the 1 MiB
length reaches the route, is answered 413 there, and is recorded with that limit, which is not recorded; a report whose body goes over the limit without
status. Clients can make up any number of paths and methods, and each would add declaring its length reaches the route, is answered 413 there, and is recorded
labels to the metrics for as long as the server runs. with that status. In the image neither reaches the backend: nginx answers both
with 413 itself (see [Report limits](#report-limits)), and nothing of them is
recorded. Clients can make up any number of paths and methods, and each would
add labels to the metrics for as long as the server runs.
`/metrics` is rate limited, so that its password cannot be guessed quickly: each `/metrics` is rate limited, so that its password cannot be guessed quickly: each
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
+15
View File
@@ -21,6 +21,21 @@ Run `make frontend-viewport-test` in CI as its own step; it is not part of
# Completed Steps # Completed Steps
- 2026-10-18: `README.md` says what the page loses while reports are refused or
the page is paused ([#145](https://git.eeqj.de/sneak/netwatch/issues/145)),
how fast one address can replace the stored reports
([#146](https://git.eeqj.de/sneak/netwatch/issues/146)), and that in the image
nginx answers a body over 1 MiB itself
([#147](https://git.eeqj.de/sneak/netwatch/issues/147)). A sample not yet
delivered is kept only as long as the history holds it: at the 1-second
interval one refused or failed report loses the oldest 20 seconds of each
host's samples, and a pause longer than the history loses every sample since
the last delivered report. At the defaults, one address sending 1 MiB reports
made up to compress badly can fill the 1 GiB cap in about 17 minutes. nginx's
`client_max_body_size` is also 1 MiB by default, so in the image it answers a
larger body with 413 and its own HTML page, and the backend never sees or
records it; the backend's own 413 sentences now say they apply to it run on
its own. The reporter, the body limit and nginx are unchanged
- 2026-10-18: a browser that cannot format the header's clocks still probes - 2026-10-18: a browser that cannot format the header's clocks still probes
([#149](https://git.eeqj.de/sneak/netwatch/issues/149)). The clocks name the ([#149](https://git.eeqj.de/sneak/netwatch/issues/149)). The clocks name the
local time zone through `Intl.DateTimeFormat`'s `shortGeneric` option, which local time zone through `Intl.DateTimeFormat`'s `shortGeneric` option, which