docs: document cache_max_bytes, update TODO.md (closes #51)
All checks were successful
check / check (push) Successful in 1m40s
All checks were successful
check / check (push) Successful in 1m40s
Add cache_max_bytes to config.example.yml and the README key settings list. TODO.md: move cache size management and eviction to Completed Steps, promote P1 blocked networks configuration into Next Step, and note in Status that the unbounded disk growth DoS vector is closed.
This commit is contained in:
@@ -115,6 +115,9 @@ Configured via YAML file (`--config`). Key settings:
|
||||
- `upstream_max_response_size` — max origin response size
|
||||
- `downstream_timeout` — client response timeout
|
||||
- `signing_key` — HMAC secret for URL signatures
|
||||
- `cache_max_bytes` — disk cache size limit in bytes; `0` disables the
|
||||
disk cache entirely; omitted defaults to 75% of the free space on
|
||||
the filesystem containing `<state_dir>/cache/` (minimum 500 MiB)
|
||||
|
||||
See `config.example.yml` for all options with defaults.
|
||||
|
||||
|
||||
31
TODO.md
31
TODO.md
@@ -12,18 +12,35 @@
|
||||
|
||||
pre-1.0. No git tags exist. Recent work extracted the internal/magic,
|
||||
internal/allowlist, internal/httpfetcher, and internal/signature
|
||||
packages. The gosec findings from the 2026-07-06 survey are resolved:
|
||||
the last two open findings (G124, session cookie attributes in
|
||||
internal/session) are fixed as of this change, so `make check` is green
|
||||
on main.
|
||||
packages. The gosec findings from the 2026-07-06 survey are resolved
|
||||
and `make check` is green on main. The disk cache is now size-bounded
|
||||
with LRU eviction (`cache_max_bytes`), closing the unbounded disk
|
||||
growth DoS vector.
|
||||
|
||||
# Next Step
|
||||
|
||||
P0: implement cache size management and eviction so the disk cannot
|
||||
fill up
|
||||
P1: implement blocked networks configuration to extend SSRF protection
|
||||
|
||||
# Completed Steps
|
||||
|
||||
- 2026-08-07 implement cache size management and eviction (closes
|
||||
#51): new `cache_max_bytes` config key validated by the startup
|
||||
framework (explicit values used exactly with no floor, `0` disables
|
||||
the disk cache entirely, omitted defaults to max(75% of free space
|
||||
on the filesystem containing `<state_dir>/cache/`, 500 MiB), logged
|
||||
at startup); processed variants are now tracked in the database
|
||||
(migration 002 adds `variant_content` and an LRU timestamp on
|
||||
`source_content`) so total usage is two SUMs, never a directory scan
|
||||
on the hot path; a background goroutine evicts globally
|
||||
least-recently-used entries (variants and source blobs merged) to
|
||||
the limit, woken by a periodic ticker and by write-pressure
|
||||
notifications from stores; a source blob and ALL of its
|
||||
`source_metadata` references are deleted in one transaction before
|
||||
the file is unlinked, so multi-referenced blobs are never removed
|
||||
while referenced and rows never point at deleted files; a startup
|
||||
reconciliation pass adopts untracked variant files, drops rows for
|
||||
missing files, removes unreachable source blobs, and sweeps stale
|
||||
temp files
|
||||
- 2026-08-07 validate configuration on startup, fail fast on bad
|
||||
config (closes #52): a config value that is set but unparseable or
|
||||
invalid aborts startup naming the key and value (defaults apply only
|
||||
@@ -79,8 +96,6 @@ fill up
|
||||
|
||||
# Future Steps
|
||||
|
||||
- P1: implement blocked networks configuration to extend SSRF
|
||||
protection
|
||||
- P1: rate limit global concurrent upstream fetches to prevent
|
||||
resource exhaustion
|
||||
- P1: strip EXIF and other metadata from processed images (privacy)
|
||||
|
||||
@@ -28,6 +28,13 @@ allow_http: false
|
||||
# Maximum concurrent connections per upstream host (default: 20)
|
||||
upstream_connections_per_host: 20
|
||||
|
||||
# Maximum disk cache size in bytes. Explicit values are used exactly as
|
||||
# given; 0 disables the disk cache entirely (every request fetches and
|
||||
# processes uncached). When omitted, the default is 75% of the free
|
||||
# space on the filesystem containing <state_dir>/cache/ at startup,
|
||||
# with a minimum of 500 MiB.
|
||||
# cache_max_bytes: 10737418240
|
||||
|
||||
# Sentry error reporting (optional)
|
||||
sentry_dsn: ""
|
||||
|
||||
|
||||
Reference in New Issue
Block a user