README states what refused reports, one sender and nginx's 413 do (closes #145, closes #146, closes #147)
check / check (push) Waiting to run
check / check (push) Waiting to run
The README said the page keeps refused samples and sends them later. Those in the 100-round history are sent only while it holds them: at the 1-second interval one refused report loses the oldest 20 seconds of each host's samples, and a pause longer than the history loses every sample between the last delivered report and the pause. Samples an interval change cleared are kept apart, the last 100 per host, until delivered. Report limits now says one address at the defaults can fill the 1 GiB cap in about 17 minutes. In the image, nginx answers a body over 1 MiB itself, with 413 and an HTML page; the backend's own 413 sentences now apply to it run on its own. Model: opus-5-5
This commit is contained in:
@@ -153,8 +153,26 @@ report a delta, so only new samples are sent; the mark advances only on a
|
||||
delivered report, and while paused nothing is sent. Changing the interval clears
|
||||
each host's history on the page; the samples in it not yet reported still go in
|
||||
the next report. Delivery failure is quiet — one debug-log line per outage,
|
||||
retried at the next interval, never blocking probing. The report-building step
|
||||
is a pure function of host state.
|
||||
never blocking probing. The report-building step is a pure function of host
|
||||
state.
|
||||
|
||||
A sample in the history that no report has delivered is sent only as long as the
|
||||
history holds it, so after a refused or failed report the next report carries
|
||||
the samples the history still has. Samples an interval change cleared from the
|
||||
history are kept apart, the last 100 per host, until they are delivered. The
|
||||
history keeps each host's last 100 rounds: 100 seconds at the 1-second interval,
|
||||
200 seconds at 2 seconds, 5 minutes at the default of 3 seconds and 100 minutes
|
||||
at 60 seconds. At 1 second, one refused or failed report loses the oldest 20
|
||||
seconds of each host's samples, as the next delivered report comes 2 minutes
|
||||
after the last. At 2 seconds and longer one loses nothing; samples are lost from
|
||||
the third refused report in a row at 2 seconds, from the fifth at 3 seconds, and
|
||||
later still at the longer intervals.
|
||||
|
||||
No report is sent while the page is paused, but each paused round still adds an
|
||||
entry to the history, so each report due during a pause counts as a refused one.
|
||||
A pause longer than the history spans loses the samples in the history recorded
|
||||
between the last delivered report and the pause: up to a minute's worth for each
|
||||
host if the report before the pause was delivered.
|
||||
|
||||
### Backend
|
||||
|
||||
@@ -240,17 +258,19 @@ Files are created with `O_EXCL` to prevent overwrites.
|
||||
|
||||
The reports of a failed write stay in memory, and the next flush, within a
|
||||
minute, tries again to write them. From the failed write until a write succeeds,
|
||||
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it, so
|
||||
the page keeps its samples and sends them again at its next report. Reports
|
||||
already answered 200 are lost only if the server stops before a write succeeds;
|
||||
the final flush at stop tries them too, and if it fails, the process exits
|
||||
non-zero.
|
||||
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it.
|
||||
The page sends the refused samples again in its next report, as long as it still
|
||||
holds them: at the 1-second interval one refused report already loses some of
|
||||
those in the history (see [Reporting](#reporting)). Reports already answered 200
|
||||
are lost only if the server stops before a write succeeds; the final flush at
|
||||
stop tries them too, and if it fails, the process exits non-zero.
|
||||
|
||||
#### Report limits
|
||||
|
||||
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
||||
credentials, so it is bounded instead. Each refusal below answers with the same
|
||||
`{"status":"error"}` body as any other error.
|
||||
`{"status":"error"}` body as any other error, except nginx's refusal of a body
|
||||
over the limit in the image.
|
||||
|
||||
- **JSON only.** A report must be sent with `Content-Type: application/json`,
|
||||
parameters such as `charset` allowed, as the page sends it; any other is
|
||||
@@ -258,6 +278,16 @@ credentials, so it is bounded instead. Each refusal below answers with the same
|
||||
send a `text/plain` POST, or a form, without a CORS preflight; refusing them
|
||||
keeps any site from making its visitors' browsers store reports (see
|
||||
[CORS](#cors)).
|
||||
- **Body limit.** A report body may be at most 1 MiB, the backend's limit for
|
||||
any request. Run on its own, the backend answers 413 before the route to a
|
||||
request that declares a longer body. Without a declared length, the route
|
||||
answers 413 only when the report it reads runs past 1 MiB; nothing after the
|
||||
end of the report is read, so a longer body whose report ends sooner is
|
||||
stored, and one that stops being valid JSON sooner is answered 400. In the
|
||||
image, nginx refuses a body over 1 MiB first, with or without a declared
|
||||
length, as its `client_max_body_size`, which `configs/nginx.conf` does not
|
||||
set, is 1 MiB by default. nginx answers 413 with its own HTML error page, and
|
||||
the backend never sees the request.
|
||||
- **Rate limit.** Each client address, resolved through `TRUSTED_PROXIES`, may
|
||||
send `REPORTS_PER_MINUTE` reports a minute; past that it gets 429 with
|
||||
`Retry-After: 60`. The minute slides: reports from the minute before still
|
||||
@@ -281,6 +311,12 @@ credentials, so it is bounded instead. Each refusal below answers with the same
|
||||
how much of the newest reports is kept: the default of 1 GiB is small enough
|
||||
for any host; set it to the space you can give `DATA_DIR`.
|
||||
|
||||
At the defaults, one address may send 60 reports of up to 1 MiB a minute, and a
|
||||
report made up to compress badly takes about as much room in the report files as
|
||||
its request body, so in about 17 minutes one address can fill the 1 GiB cap with
|
||||
its own reports, every report stored before them deleted to make room. Lowering
|
||||
`REPORTS_PER_MINUTE` or raising `DATA_DIR_MAX_BYTES` makes that take longer.
|
||||
|
||||
#### CORS
|
||||
|
||||
The page calls the API from the origin it is served from, so by default the
|
||||
@@ -302,12 +338,16 @@ the metrics record its duration and response size, labelled with its path,
|
||||
method and status; they also count those requests in progress, and include Go's
|
||||
runtime and process metrics. No other request is recorded: not those to
|
||||
`/metrics` itself, and not those answered before they reach either route, such
|
||||
as a CORS preflight, or a request refused with 404 for a path no route has, 405
|
||||
for a method its route does not take, or 413 for declaring a body length over
|
||||
the 1 MiB limit. A report whose body goes over the limit without declaring its
|
||||
length reaches the route, is answered 413 there, and is recorded with that
|
||||
status. Clients can make up any number of paths and methods, and each would add
|
||||
labels to the metrics for as long as the server runs.
|
||||
as a CORS preflight, or a request refused with 404 for a path no route has or
|
||||
405 for a method its route does not take. Run on its own, the backend also
|
||||
answers 413 before the route to a request declaring a body length over the 1 MiB
|
||||
limit, which is not recorded. A body over the limit sent without a declared
|
||||
length reaches the route and is recorded with the status the route answers,
|
||||
which is 413 only when the report it reads runs past the limit: nothing after
|
||||
the end of the report is read. In the image neither reaches the backend: nginx
|
||||
answers both with 413 itself (see [Report limits](#report-limits)), and nothing
|
||||
of them is recorded. Clients can make up any number of paths and methods, and
|
||||
each would add labels to the metrics for as long as the server runs.
|
||||
|
||||
`/metrics` is rate limited, so that its password cannot be guessed quickly: each
|
||||
client address, resolved through `TRUSTED_PROXIES`, may make 60 requests to it a
|
||||
|
||||
Reference in New Issue
Block a user