README states what refused reports, one sender and nginx's 413 do (closes #145, closes #146, closes #147)
check / check (push) Waiting to run
check / check (push) Waiting to run
The README said the page keeps refused samples and sends them later. They live only in the 100-round history, so at the 1-second interval one refused or failed report loses the oldest 20 seconds of each host's samples, and a pause longer than the history loses everything since the last delivered report. Report limits now says one address at the defaults can fill the 1 GiB cap in about 17 minutes with reports made to compress badly, and which settings bound it. In the image, nginx's default client_max_body_size of 1 MiB answers an over-limit body with 413 and an HTML page before the backend sees it; the backend's own 413 sentences now say they apply to it run on its own. Model: opus-5-5
This commit is contained in:
@@ -153,8 +153,24 @@ report a delta, so only new samples are sent; the mark advances only on a
|
||||
delivered report, and while paused nothing is sent. Changing the interval clears
|
||||
each host's history on the page; the samples in it not yet reported still go in
|
||||
the next report. Delivery failure is quiet — one debug-log line per outage,
|
||||
retried at the next interval, never blocking probing. The report-building step
|
||||
is a pure function of host state.
|
||||
never blocking probing. The report-building step is a pure function of host
|
||||
state.
|
||||
|
||||
A sample not yet delivered is kept only as long as the history holds it, so
|
||||
after a refused or failed report the next report carries the samples the history
|
||||
still has. The history keeps each host's last 100 rounds: 100 seconds at the
|
||||
1-second interval, 200 seconds at 2 seconds, 5 minutes at the default of 3
|
||||
seconds and 100 minutes at 60 seconds. At 1 second, one refused or failed report
|
||||
loses the oldest 20 seconds of each host's samples, as the next delivered report
|
||||
comes 2 minutes after the last. At 2 seconds and longer one loses nothing;
|
||||
samples are lost from the third refused report in a row at 2 seconds, from the
|
||||
fifth at 3 seconds, and later still at the longer intervals.
|
||||
|
||||
No report is sent while the page is paused, but each paused round still adds an
|
||||
entry to the history, so each report due during a pause counts as a refused one.
|
||||
A pause longer than the history spans loses every sample recorded between the
|
||||
last delivered report and the pause: a minute's worth for each host if the
|
||||
report before the pause was delivered.
|
||||
|
||||
### Backend
|
||||
|
||||
@@ -240,17 +256,19 @@ Files are created with `O_EXCL` to prevent overwrites.
|
||||
|
||||
The reports of a failed write stay in memory, and the next flush, within a
|
||||
minute, tries again to write them. From the failed write until a write succeeds,
|
||||
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it, so
|
||||
the page keeps its samples and sends them again at its next report. Reports
|
||||
already answered 200 are lost only if the server stops before a write succeeds;
|
||||
the final flush at stop tries them too, and if it fails, the process exits
|
||||
non-zero.
|
||||
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it.
|
||||
The page sends the refused samples again in its next report, as long as its
|
||||
history still holds them: at the 1-second interval one refused report already
|
||||
loses some (see [Reporting](#reporting)). Reports already answered 200 are lost
|
||||
only if the server stops before a write succeeds; the final flush at stop tries
|
||||
them too, and if it fails, the process exits non-zero.
|
||||
|
||||
#### Report limits
|
||||
|
||||
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
||||
credentials, so it is bounded instead. Each refusal below answers with the same
|
||||
`{"status":"error"}` body as any other error.
|
||||
`{"status":"error"}` body as any other error, except nginx's refusal of a body
|
||||
over the limit in the image.
|
||||
|
||||
- **JSON only.** A report must be sent with `Content-Type: application/json`,
|
||||
parameters such as `charset` allowed, as the page sends it; any other is
|
||||
@@ -258,6 +276,13 @@ credentials, so it is bounded instead. Each refusal below answers with the same
|
||||
send a `text/plain` POST, or a form, without a CORS preflight; refusing them
|
||||
keeps any site from making its visitors' browsers store reports (see
|
||||
[CORS](#cors)).
|
||||
- **Body limit.** A report body may be at most 1 MiB, the backend's limit for
|
||||
any request. Run on its own, the backend refuses a larger body with 413:
|
||||
before the route when the request declares a length over the limit, otherwise
|
||||
at the route once reading the body passes it. In the image, nginx refuses it
|
||||
first, with or without a declared length, as its `client_max_body_size`, which
|
||||
`configs/nginx.conf` does not set, is 1 MiB by default. nginx answers 413 with
|
||||
its own HTML error page, and the backend never sees the request.
|
||||
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
|
||||
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
|
||||
`Retry-After: 60`. The minute slides: reports from the minute before still
|
||||
@@ -281,6 +306,12 @@ credentials, so it is bounded instead. Each refusal below answers with the same
|
||||
how much of the newest reports is kept: the default of 1 GiB is small enough
|
||||
for any host; set it to the space you can give `DATA_DIR`.
|
||||
|
||||
At the defaults, one address may send 60 reports of up to 1 MiB a minute, and a
|
||||
report made up to compress badly takes about as much room in the report files as
|
||||
its request body, so in about 17 minutes one address can fill the 1 GiB cap with
|
||||
its own reports, every report stored before them deleted to make room. Lowering
|
||||
`REPORTS_PER_MINUTE` or raising `DATA_DIR_MAX_BYTES` makes that take longer.
|
||||
|
||||
#### CORS
|
||||
|
||||
The page calls the API from the origin it is served from, so by default the
|
||||
@@ -302,12 +333,15 @@ the metrics record its duration and response size, labelled with its path,
|
||||
method and status; they also count those requests in progress, and include Go's
|
||||
runtime and process metrics. No other request is recorded: not those to
|
||||
`/metrics` itself, and not those answered before they reach either route, such
|
||||
as a CORS preflight, or a request refused with 404 for a path no route has, 405
|
||||
for a method its route does not take, or 413 for declaring a body length over
|
||||
the 1 MiB limit. A report whose body goes over the limit without declaring its
|
||||
length reaches the route, is answered 413 there, and is recorded with that
|
||||
status. Clients can make up any number of paths and methods, and each would add
|
||||
labels to the metrics for as long as the server runs.
|
||||
as a CORS preflight, or a request refused with 404 for a path no route has or
|
||||
405 for a method its route does not take. Run on its own, the backend also
|
||||
answers 413 before the route to a request declaring a body length over the 1 MiB
|
||||
limit, which is not recorded; a report whose body goes over the limit without
|
||||
declaring its length reaches the route, is answered 413 there, and is recorded
|
||||
with that status. In the image neither reaches the backend: nginx answers both
|
||||
with 413 itself (see [Report limits](#report-limits)), and nothing of them is
|
||||
recorded. Clients can make up any number of paths and methods, and each would
|
||||
add labels to the metrics for as long as the server runs.
|
||||
|
||||
`/metrics` is rate limited, so that its password cannot be guessed quickly: each
|
||||
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
|
||||
|
||||
@@ -21,6 +21,21 @@ Run `make frontend-viewport-test` in CI as its own step; it is not part of
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-10-18: `README.md` says what the page loses while reports are refused or
|
||||
the page is paused ([#145](https://git.eeqj.de/sneak/netwatch/issues/145)),
|
||||
how fast one address can replace the stored reports
|
||||
([#146](https://git.eeqj.de/sneak/netwatch/issues/146)), and that in the image
|
||||
nginx answers a body over 1 MiB itself
|
||||
([#147](https://git.eeqj.de/sneak/netwatch/issues/147)). A sample not yet
|
||||
delivered is kept only as long as the history holds it: at the 1-second
|
||||
interval one refused or failed report loses the oldest 20 seconds of each
|
||||
host's samples, and a pause longer than the history loses every sample since
|
||||
the last delivered report. At the defaults, one address sending 1 MiB reports
|
||||
made up to compress badly can fill the 1 GiB cap in about 17 minutes. nginx's
|
||||
`client_max_body_size` is also 1 MiB by default, so in the image it answers a
|
||||
larger body with 413 and its own HTML page, and the backend never sees or
|
||||
records it; the backend's own 413 sentences now say they apply to it run on
|
||||
its own. The reporter, the body limit and nginx are unchanged
|
||||
- 2026-10-18: `netwatch-server prepare-data-dir` no longer changes the owner of
|
||||
a file outside `/data` through a hard link in the volume
|
||||
([#143](https://git.eeqj.de/sneak/netwatch/issues/143)). A file with a second
|
||||
|
||||
Reference in New Issue
Block a user