Check cached file sizes at startup, not per request (closes #252)
check / check (push) Waiting to run

Lookup and LookupSource are as on next again: requests compare no
sizes and a hit reads no extra row. The reconciliation pass, at startup
and every five minutes, compares each cached source and variant file
with the size its row records and removes one that is empty or of
another size, with its rows and a warning naming the file; an empty
variant file without a row is removed rather than adopted. Source,
variant and .meta files all go through writeFileSynced (temp file,
sync, rename, directory sync), the .meta file before the variant, and
a failed directory sync removes the renamed file.

#252

Model: opus-5-5
This commit is contained in:
2026-10-15 16:50:05 +00:00
parent 6f3fea68c3
commit e97673c7f2
6 changed files with 157 additions and 235 deletions
+12 -9
View File
@@ -30,6 +30,18 @@ P2: security: per-origin rate limiting
# Completed Steps
- 2026-10-15 a cached file left empty or partial is no longer served, or fetched
again on every request (closes #252). Source and transformed image files, and
each transformed image's `.meta` file, are synced to disk before they are
renamed into place, and their directory after; the `.meta` file is written
before the image, and a failed directory sync removes the renamed file. The
reconciliation pass, at startup and every five minutes, removes a cached
source or transformed image whose file is empty, or not the size the database
records for it, with its rows and a warning naming the file, so the next
request fetches the source or converts the image again; requests compare no
sizes. Storing a source over a file of another size replaces that file. A
source that fails to decode is not removed for that alone: a damaged upstream
image would then be fetched on every request.
- 2026-10-15 a transcode stops once its request's `downstream_timeout` has
ended, unless it has begun encoding (closes #243): the image processor looked
at its context only while it waited for a processing slot, so an image was
@@ -121,15 +133,6 @@ P2: security: per-origin rate limiting
answered 500. Once the limit is used up, the fetcher now reads one more byte:
if the body has ended it is accepted, and if there is more it is refused with
`ErrResponseTooLarge`, as before.
- 2026-10-15 a cached file left empty or partial is no longer served, or fetched
again on every request (closes #252). Source and transformed image files are
synced to disk before they are renamed into place, and their directory after.
A request that finds a cached source or transformed image whose file is empty,
or not the size the database records for it, removes it with a log line naming
it, and fetches the source or converts the image again. Storing a source over
a file of another size replaces that file. A source that fails to decode is
not removed for that alone: a damaged upstream image would then be fetched on
every request.
- 2026-10-13 `fit=outside` no longer makes an output above 8192 on a side
(closes #257): it scales the image to cover the requested size, and the side
scaled past that size was held to no maximum, so `1x8192?fit=outside` of a