Reports refused while report files cannot be written (closes #123)
check / check (push) Waiting to run
check / check (push) Waiting to run
A failed write of a report file used to drop its reports, and POST /api/v1/reports kept answering 200 for as long as writes failed. Now a failed write puts its reports back in the buffer, still counted against DATA_DIR_MAX_BYTES, and Append refuses every report, which the handler answers 500, until a write succeeds. The minute flush tries the held reports again, so storing resumes with no restart once writing works, and the buffer takes nothing new while writes fail. Model: opus-5-5
This commit is contained in:
+18
-10
@@ -140,6 +140,14 @@ toward `DATA_DIR_MAX_BYTES` from the next start, only if removing it fails too.
|
||||
Each file contains one JSON object per line, compressed with zstd. Files are
|
||||
created with `O_EXCL` to prevent overwrites.
|
||||
|
||||
The reports of a failed write stay in memory, and the next flush, within a
|
||||
minute, tries again to write them. From the failed write until a write succeeds,
|
||||
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it, so
|
||||
the page keeps its samples and sends them again at its next report. Reports
|
||||
already answered 200 are lost only if the server stops before a write succeeds;
|
||||
the final flush at stop tries them too, and if it fails, the process exits
|
||||
non-zero.
|
||||
|
||||
### Report limits
|
||||
|
||||
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
||||
@@ -158,16 +166,16 @@ credentials, so it is bounded instead. Both refusals below answer with the same
|
||||
`X-RateLimit-Reset` headers.
|
||||
- **Size cap.** The report files in `DATA_DIR` may total at most
|
||||
`DATA_DIR_MAX_BYTES`, counting the files already there at start. Reports
|
||||
waiting in memory count at their uncompressed size until they are written;
|
||||
those lost to a failed write stop counting, and the part of its file written
|
||||
is removed. When a report would take the total past the cap, the oldest report
|
||||
files are deleted to make room, and each deletion is logged with the file's
|
||||
name and size; a file still being written is never deleted. A report is
|
||||
refused with 507, and nothing of it is stored, only when the reports waiting
|
||||
to be written fill the cap on their own, and then no file is deleted. At
|
||||
start, report files past the cap, as after lowering it, are deleted the same
|
||||
way. So the cap is how much of the newest reports is kept: the default of 1
|
||||
GiB is small enough for any host; set it to the space you can give `DATA_DIR`.
|
||||
waiting in memory count at their uncompressed size until they are written,
|
||||
those of a failed write included; the part of its file written is removed.
|
||||
When a report would take the total past the cap, the oldest report files are
|
||||
deleted to make room, and each deletion is logged with the file's name and
|
||||
size; a file still being written is never deleted. A report is refused with
|
||||
507, and nothing of it is stored, only when the reports waiting to be written
|
||||
fill the cap on their own, and then no file is deleted. At start, report files
|
||||
past the cap, as after lowering it, are deleted the same way. So the cap is
|
||||
how much of the newest reports is kept: the default of 1 GiB is small enough
|
||||
for any host; set it to the space you can give `DATA_DIR`.
|
||||
|
||||
### CORS
|
||||
|
||||
|
||||
Reference in New Issue
Block a user