Delete the oldest report files to stay under the size cap (closes #54)
check / check (push) Successful in 1m53s

When a report would take the report files past DATA_DIR_MAX_BYTES,
reportbuf now deletes the oldest report files until it fits, and does
the same at start when files left by an earlier run are already past
it. The buffer keeps the files it may delete in a list, oldest first; a
file joins it only once it is completely written, so a file still being
written is never deleted. A report is refused with 507 only when the
reports not yet written leave no room for it on their own, and then no
file is deleted. A file whose deletion fails keeps counting; one
already deleted by hand counts as freed.

Model: opus-5-5
This commit is contained in:
2026-10-03 13:21:42 +00:00
parent 4ce0814b14
commit ee959fc33a
5 changed files with 364 additions and 67 deletions
+10 -7
View File
@@ -80,7 +80,7 @@ Internal packages in `internal/` follow standard Go project layout:
| `BIND_ADDRESS` | empty | IP address to listen on; empty listens on every interface |
| `PORT` | `8080` | HTTP listen port |
| `DATA_DIR` | `./data/reports` | Directory for compressed reports |
| `DATA_DIR_MAX_BYTES` | `1073741824` (1 GiB) | Largest total size of the report files in `DATA_DIR`; see [Report limits](#report-limits) |
| `DATA_DIR_MAX_BYTES` | `1073741824` (1 GiB) | Most bytes of report files kept in `DATA_DIR`, oldest deleted first; see [Report limits](#report-limits) |
| `DEBUG` | `false` | Enable debug logging |
| `TRUSTED_PROXIES` | loopback + RFC1918 | Comma-separated CIDRs whose `X-Forwarded-For` / `X-Real-IP` headers are trusted for client IP resolution |
| `REPORTS_PER_MINUTE` | `60` | Reports each client address may send a minute; see [Report limits](#report-limits) |
@@ -148,11 +148,15 @@ credentials, so it is bounded instead. Both refusals below answer with the same
`X-RateLimit-Reset` headers.
- **Size cap.** The report files in `DATA_DIR` may total at most
`DATA_DIR_MAX_BYTES`, counting the files already there at start. Reports
waiting in memory count at their uncompressed size until they are written, so
a report that would take the total past the cap is refused with 507, and
nothing of it is stored. Deleting report files frees room only at the next
start, when the files are counted again. The default of 1 GiB is small enough
for any host; set it to the space you can give `DATA_DIR`.
waiting in memory count at their uncompressed size until they are written.
When a report would take the total past the cap, the oldest report files are
deleted to make room, and each deletion is logged with the file's name and
size; a file still being written is never deleted. A report is refused with
507, and nothing of it is stored, only when the reports not yet written leave
no room for it on their own, and then no file is deleted. At start, report
files past the cap, as after lowering it, are deleted the same way. So the cap
is how much of the newest reports is kept: the default of 1 GiB is small
enough for any host; set it to the space you can give `DATA_DIR`.
### CORS
@@ -170,7 +174,6 @@ starting, with an error naming `CORS_ALLOWED_ORIGINS`.
- Add integration test that POSTs a report and verifies the compressed output
- Add report decompression/query endpoint
- Add metrics (Prometheus) for buffer size, flush count, report count
- Add retention policy to prune old report files
## License