From bbc354fa994a9c7333c758dd74022443f05195c9 Mon Sep 17 00:00:00 2001 From: sneak Date: Thu, 15 Oct 2026 13:27:29 +0000 Subject: [PATCH] Give the size of libvips' temporary file in "Running under upaas" While pixa processes a large source, libvips writes its decoded pixels to a file in /tmp, outside /var/lib/pixa and cache_max_bytes, and not in the 1.6 GiB of resident memory. Measured on pixad for 8192x8192 sources, it is 192 MiB for an 8-bit RGB JPEG, 512 MiB for 16-bit RGBA, and 1 GiB for a floating-point RGBA JPEG XL. The section now says so, and that the file is memory where /tmp is a tmpfs. Model: opus-5-5 --- README.md | 10 ++++++++++ TODO.md | 5 ++++- 2 files changed, 14 insertions(+), 1 deletion(-) diff --git a/README.md b/README.md index 320b0ee..49b7828 100644 --- a/README.md +++ b/README.md @@ -137,6 +137,16 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app for pixa needs: times 1.6 GiB plus 3.1 GiB, about 42 GiB. Set `PIXA_MAX_CONCURRENT_PROCESSING` and `PIXA_UPSTREAM_CONNECTIONS` so that this fits the memory the container may use. +- **Temporary files:** while pixa processes a large source, libvips writes its + decoded pixels to a file in `/tmp` and deletes the file when the image is + done. For an 8192x8192 source the file was measured on pixad at 192 MiB for an + 8-bit RGB JPEG, 512 MiB for a 16-bit RGBA PNG or JPEG XL, and 1 GiB for a JPEG + XL of 32-bit floating-point RGBA, the largest measured: up to 1 GiB per image + processed at once, 24 GiB at the defaults on a host where pixa sees 48 CPUs. + This `/tmp` is in the container's own filesystem, not on the volume at + `/var/lib/pixa`, and `cache_max_bytes` does not count it. The 1.6 GiB above + does not include these files; where `/tmp` is a tmpfs, they are memory on top + of it. - **Health check:** the image's `HEALTHCHECK` requests `/.well-known/healthcheck.json`. upaas reads the container's health 60 seconds after a deploy and marks the deploy failed unless it is `healthy`. The probe diff --git a/TODO.md b/TODO.md index dcd1899..3a02985 100644 --- a/TODO.md +++ b/TODO.md @@ -38,7 +38,10 @@ P2: security: per-origin rate limiting `imageprocessor.New`; `PIXA_MAX_CONCURRENT_PROCESSING` still overrides it. "Running under upaas" in `README.md` gives the worst-case memory, measured, and tells the operator to set `PIXA_MAX_CONCURRENT_PROCESSING` and - `PIXA_UPSTREAM_CONNECTIONS` to fit the container's memory. + `PIXA_UPSTREAM_CONNECTIONS` to fit the container's memory. It also gives the + file libvips writes to `/tmp` for each large source, up to 1 GiB, measured, + which is outside `/var/lib/pixa` and `cache_max_bytes` and is memory where + `/tmp` is a tmpfs. - 2026-10-15 expired negative cache rows are removed on a schedule (closes #244): the row of a failed fetch, which makes pixa answer 502 for that source URL for 5 minutes, was deleted only when the same URL was requested again