README states what refused reports, one sender and nginx's 413 do (closes #145, closes #146, closes #147)
check / check (push) Waiting to run
check / check (push) Waiting to run
The README said the page keeps refused samples and sends them later. They live only in the 100-round history, so at the 1-second interval one refused or failed report loses the oldest 20 seconds of each host's samples, and a pause longer than the history loses everything since the last delivered report. Report limits now says one address at the defaults can fill the 1 GiB cap in about 17 minutes with reports made to compress badly, and which settings bound it. In the image, nginx's default client_max_body_size of 1 MiB answers an over-limit body with 413 and an HTML page before the backend sees it; the backend's own 413 sentences now say they apply to it run on its own. Model: opus-5-5
This commit is contained in:
@@ -153,8 +153,24 @@ report a delta, so only new samples are sent; the mark advances only on a
|
|||||||
delivered report, and while paused nothing is sent. Changing the interval clears
|
delivered report, and while paused nothing is sent. Changing the interval clears
|
||||||
each host's history on the page; the samples in it not yet reported still go in
|
each host's history on the page; the samples in it not yet reported still go in
|
||||||
the next report. Delivery failure is quiet — one debug-log line per outage,
|
the next report. Delivery failure is quiet — one debug-log line per outage,
|
||||||
retried at the next interval, never blocking probing. The report-building step
|
never blocking probing. The report-building step is a pure function of host
|
||||||
is a pure function of host state.
|
state.
|
||||||
|
|
||||||
|
A sample not yet delivered is kept only as long as the history holds it, so
|
||||||
|
after a refused or failed report the next report carries the samples the history
|
||||||
|
still has. The history keeps each host's last 100 rounds: 100 seconds at the
|
||||||
|
1-second interval, 200 seconds at 2 seconds, 5 minutes at the default of 3
|
||||||
|
seconds and 100 minutes at 60 seconds. At 1 second, one refused or failed report
|
||||||
|
loses the oldest 20 seconds of each host's samples, as the next delivered report
|
||||||
|
comes 2 minutes after the last. At 2 seconds and longer one loses nothing;
|
||||||
|
samples are lost from the third refused report in a row at 2 seconds, from the
|
||||||
|
fifth at 3 seconds, and later still at the longer intervals.
|
||||||
|
|
||||||
|
No report is sent while the page is paused, but each paused round still adds an
|
||||||
|
entry to the history, so each report due during a pause counts as a refused one.
|
||||||
|
A pause longer than the history spans loses every sample recorded between the
|
||||||
|
last delivered report and the pause: a minute's worth for each host if the
|
||||||
|
report before the pause was delivered.
|
||||||
|
|
||||||
### Backend
|
### Backend
|
||||||
|
|
||||||
@@ -240,17 +256,19 @@ Files are created with `O_EXCL` to prevent overwrites.
|
|||||||
|
|
||||||
The reports of a failed write stay in memory, and the next flush, within a
|
The reports of a failed write stay in memory, and the next flush, within a
|
||||||
minute, tries again to write them. From the failed write until a write succeeds,
|
minute, tries again to write them. From the failed write until a write succeeds,
|
||||||
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it, so
|
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it.
|
||||||
the page keeps its samples and sends them again at its next report. Reports
|
The page sends the refused samples again in its next report, as long as its
|
||||||
already answered 200 are lost only if the server stops before a write succeeds;
|
history still holds them: at the 1-second interval one refused report already
|
||||||
the final flush at stop tries them too, and if it fails, the process exits
|
loses some (see [Reporting](#reporting)). Reports already answered 200 are lost
|
||||||
non-zero.
|
only if the server stops before a write succeeds; the final flush at stop tries
|
||||||
|
them too, and if it fails, the process exits non-zero.
|
||||||
|
|
||||||
#### Report limits
|
#### Report limits
|
||||||
|
|
||||||
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
||||||
credentials, so it is bounded instead. Each refusal below answers with the same
|
credentials, so it is bounded instead. Each refusal below answers with the same
|
||||||
`{"status":"error"}` body as any other error.
|
`{"status":"error"}` body as any other error, except nginx's refusal of a body
|
||||||
|
over the limit in the image.
|
||||||
|
|
||||||
- **JSON only.** A report must be sent with `Content-Type: application/json`,
|
- **JSON only.** A report must be sent with `Content-Type: application/json`,
|
||||||
parameters such as `charset` allowed, as the page sends it; any other is
|
parameters such as `charset` allowed, as the page sends it; any other is
|
||||||
@@ -258,6 +276,13 @@ credentials, so it is bounded instead. Each refusal below answers with the same
|
|||||||
send a `text/plain` POST, or a form, without a CORS preflight; refusing them
|
send a `text/plain` POST, or a form, without a CORS preflight; refusing them
|
||||||
keeps any site from making its visitors' browsers store reports (see
|
keeps any site from making its visitors' browsers store reports (see
|
||||||
[CORS](#cors)).
|
[CORS](#cors)).
|
||||||
|
- **Body limit.** A report body may be at most 1 MiB, the backend's limit for
|
||||||
|
any request. Run on its own, the backend refuses a larger body with 413:
|
||||||
|
before the route when the request declares a length over the limit, otherwise
|
||||||
|
at the route once reading the body passes it. In the image, nginx refuses it
|
||||||
|
first, with or without a declared length, as its `client_max_body_size`, which
|
||||||
|
`configs/nginx.conf` does not set, is 1 MiB by default. nginx answers 413 with
|
||||||
|
its own HTML error page, and the backend never sees the request.
|
||||||
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
|
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
|
||||||
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
|
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
|
||||||
`Retry-After: 60`. The minute slides: reports from the minute before still
|
`Retry-After: 60`. The minute slides: reports from the minute before still
|
||||||
@@ -281,6 +306,12 @@ credentials, so it is bounded instead. Each refusal below answers with the same
|
|||||||
how much of the newest reports is kept: the default of 1 GiB is small enough
|
how much of the newest reports is kept: the default of 1 GiB is small enough
|
||||||
for any host; set it to the space you can give `DATA_DIR`.
|
for any host; set it to the space you can give `DATA_DIR`.
|
||||||
|
|
||||||
|
At the defaults, one address may send 60 reports of up to 1 MiB a minute, and a
|
||||||
|
report made up to compress badly takes about as much room in the report files as
|
||||||
|
its request body, so in about 17 minutes one address can fill the 1 GiB cap with
|
||||||
|
its own reports, every report stored before them deleted to make room. Lowering
|
||||||
|
`REPORTS_PER_MINUTE` or raising `DATA_DIR_MAX_BYTES` makes that take longer.
|
||||||
|
|
||||||
#### CORS
|
#### CORS
|
||||||
|
|
||||||
The page calls the API from the origin it is served from, so by default the
|
The page calls the API from the origin it is served from, so by default the
|
||||||
@@ -302,12 +333,15 @@ the metrics record its duration and response size, labelled with its path,
|
|||||||
method and status; they also count those requests in progress, and include Go's
|
method and status; they also count those requests in progress, and include Go's
|
||||||
runtime and process metrics. No other request is recorded: not those to
|
runtime and process metrics. No other request is recorded: not those to
|
||||||
`/metrics` itself, and not those answered before they reach either route, such
|
`/metrics` itself, and not those answered before they reach either route, such
|
||||||
as a CORS preflight, or a request refused with 404 for a path no route has, 405
|
as a CORS preflight, or a request refused with 404 for a path no route has or
|
||||||
for a method its route does not take, or 413 for declaring a body length over
|
405 for a method its route does not take. Run on its own, the backend also
|
||||||
the 1 MiB limit. A report whose body goes over the limit without declaring its
|
answers 413 before the route to a request declaring a body length over the 1 MiB
|
||||||
length reaches the route, is answered 413 there, and is recorded with that
|
limit, which is not recorded; a report whose body goes over the limit without
|
||||||
status. Clients can make up any number of paths and methods, and each would add
|
declaring its length reaches the route, is answered 413 there, and is recorded
|
||||||
labels to the metrics for as long as the server runs.
|
with that status. In the image neither reaches the backend: nginx answers both
|
||||||
|
with 413 itself (see [Report limits](#report-limits)), and nothing of them is
|
||||||
|
recorded. Clients can make up any number of paths and methods, and each would
|
||||||
|
add labels to the metrics for as long as the server runs.
|
||||||
|
|
||||||
`/metrics` is rate limited, so that its password cannot be guessed quickly: each
|
`/metrics` is rate limited, so that its password cannot be guessed quickly: each
|
||||||
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
|
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
|
||||||
|
|||||||
@@ -21,6 +21,21 @@ Run `make frontend-viewport-test` in CI as its own step; it is not part of
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-18: `README.md` says what the page loses while reports are refused or
|
||||||
|
the page is paused ([#145](https://git.eeqj.de/sneak/netwatch/issues/145)),
|
||||||
|
how fast one address can replace the stored reports
|
||||||
|
([#146](https://git.eeqj.de/sneak/netwatch/issues/146)), and that in the image
|
||||||
|
nginx answers a body over 1 MiB itself
|
||||||
|
([#147](https://git.eeqj.de/sneak/netwatch/issues/147)). A sample not yet
|
||||||
|
delivered is kept only as long as the history holds it: at the 1-second
|
||||||
|
interval one refused or failed report loses the oldest 20 seconds of each
|
||||||
|
host's samples, and a pause longer than the history loses every sample since
|
||||||
|
the last delivered report. At the defaults, one address sending 1 MiB reports
|
||||||
|
made up to compress badly can fill the 1 GiB cap in about 17 minutes. nginx's
|
||||||
|
`client_max_body_size` is also 1 MiB by default, so in the image it answers a
|
||||||
|
larger body with 413 and its own HTML page, and the backend never sees or
|
||||||
|
records it; the backend's own 413 sentences now say they apply to it run on
|
||||||
|
its own. The reporter, the body limit and nginx are unchanged
|
||||||
- 2026-10-18: `netwatch-server prepare-data-dir` no longer changes the owner of
|
- 2026-10-18: `netwatch-server prepare-data-dir` no longer changes the owner of
|
||||||
a file outside `/data` through a hard link in the volume
|
a file outside `/data` through a hard link in the volume
|
||||||
([#143](https://git.eeqj.de/sneak/netwatch/issues/143)). A file with a second
|
([#143](https://git.eeqj.de/sneak/netwatch/issues/143)). A file with a second
|
||||||
|
|||||||
Reference in New Issue
Block a user