Bound concurrent image processing and upstream fetches (closes #64)
check / check (push) Successful in 12s

max_concurrent_processing (default: the number of CPUs Go uses) bounds
the images processed at once, and upstream_connections (default 64) the
fetches from all upstream hosts together, beside the per-host limit. A
request that finds either full waits up to 10 seconds, then gets 503
"server busy, try again later". The processor holds its slot from before
it reads the input until it returns; a fetch holds its connection until
the response body is closed, after its image is processed. libvips now
starts with one worker thread per image and no operation cache. Both
settings have PIXA_ variables and are in README.md and
config.example.yml.

Model: opus-5-5
This commit is contained in:
2026-09-29 01:11:50 +00:00
parent 1e6ef7f7f9
commit 6692714bd3
10 changed files with 253 additions and 49 deletions
+10 -3
View File
@@ -25,11 +25,19 @@ The disk cache is now size-bounded with LRU eviction
# Next Step
P1: rate limit global concurrent upstream fetches to prevent resource
exhaustion
P2: security: referer blacklist
# Completed Steps
- 2026-09-29 bound concurrent image processing and upstream fetches (closes
#64): `max_concurrent_processing` (default the number of CPUs pixa can use)
limits the images decoded and encoded at once, and `upstream_connections`
(default 64) the connections to all upstream hosts together, on top of
`upstream_connections_per_host`; a fetch holds its connection until its image
has been processed; a request that finds either limit reached waits up to 10
seconds for a free one, then gets 503 `server busy, try again later`; libvips
runs one worker thread per image with its operation cache off; documented in
`README.md` and `config.example.yml`.
- 2026-09-28 strip metadata from processed images (closes #82): every output is
exported with govips' `StripMetadata`, so it carries no EXIF, XMP, IPTC or ICC
profile; the image is first turned upright with `AutoRotate` (before sizes are
@@ -258,7 +266,6 @@ exhaustion
# Future Steps
- P2: security
- referer blacklist
- per-IP rate limiting on the image routes
- per-origin rate limiting
- P2: HTTP response handling