Reports refused while report files cannot be written (closes #123)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
A failed write of a report file used to drop its reports, and POST /api/v1/reports kept answering 200 for as long as writes failed. Now a failed write puts its reports back in the buffer, still counted against DATA_DIR_MAX_BYTES, and Append refuses every report, which the handler answers 500, until a write succeeds. The minute flush tries the held reports again, so storing resumes with no restart once writing works, and the buffer takes nothing new while writes fail. Tests also cover a report appended while a write that fails runs, and the cap once a failed write's reports are written. Model: opus-5-5
This commit was merged in pull request #130.
This commit is contained in:
@@ -228,6 +228,14 @@ the numbers: its file, if it was created, is removed. The file stays, counted
|
||||
toward `DATA_DIR_MAX_BYTES` from the next start, only if removing it fails too.
|
||||
Files are created with `O_EXCL` to prevent overwrites.
|
||||
|
||||
The reports of a failed write stay in memory, and the next flush, within a
|
||||
minute, tries again to write them. From the failed write until a write succeeds,
|
||||
`POST /api/v1/reports` refuses every report with 500 and keeps nothing of it, so
|
||||
the page keeps its samples and sends them again at its next report. Reports
|
||||
already answered 200 are lost only if the server stops before a write succeeds;
|
||||
the final flush at stop tries them too, and if it fails, the process exits
|
||||
non-zero.
|
||||
|
||||
#### Report limits
|
||||
|
||||
`POST /api/v1/reports` takes reports from anyone who can reach it, without
|
||||
@@ -252,16 +260,16 @@ credentials, so it is bounded instead. Each refusal below answers with the same
|
||||
`X-RateLimit-Reset` headers.
|
||||
- **Size cap.** The report files in `DATA_DIR` may total at most
|
||||
`DATA_DIR_MAX_BYTES`, counting the files already there at start. Reports
|
||||
waiting in memory count at their uncompressed size until they are written;
|
||||
those lost to a failed write stop counting, and the part of its file written
|
||||
is removed. When a report would take the total past the cap, the oldest report
|
||||
files are deleted to make room, and each deletion is logged with the file's
|
||||
name and size; a file still being written is never deleted. A report is
|
||||
refused with 507, and nothing of it is stored, only when the reports waiting
|
||||
to be written fill the cap on their own, and then no file is deleted. At
|
||||
start, report files past the cap, as after lowering it, are deleted the same
|
||||
way. So the cap is how much of the newest reports is kept: the default of 1
|
||||
GiB is small enough for any host; set it to the space you can give `DATA_DIR`.
|
||||
waiting in memory count at their uncompressed size until they are written,
|
||||
those of a failed write included; the part of its file written is removed.
|
||||
When a report would take the total past the cap, the oldest report files are
|
||||
deleted to make room, and each deletion is logged with the file's name and
|
||||
size; a file still being written is never deleted. A report is refused with
|
||||
507, and nothing of it is stored, only when the reports waiting to be written
|
||||
fill the cap on their own, and then no file is deleted. At start, report files
|
||||
past the cap, as after lowering it, are deleted the same way. So the cap is
|
||||
how much of the newest reports is kept: the default of 1 GiB is small enough
|
||||
for any host; set it to the space you can give `DATA_DIR`.
|
||||
|
||||
#### CORS
|
||||
|
||||
|
||||
Reference in New Issue
Block a user