Check cached file sizes at startup, not per request (closes #252)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
Lookup and LookupSource are as on next again: requests compare no sizes and a hit reads no extra row. The reconciliation pass, at startup and every five minutes, compares each cached source and variant file with the size its row records and removes one that is empty or of another size, with its rows and a warning naming the file; an empty variant file without a row is removed rather than adopted. Source, variant and .meta files all go through writeFileSynced (temp file, sync, rename, directory sync), the .meta file before the variant, and a failed directory sync removes the renamed file. #252 Model: opus-5-5
This commit is contained in:
@@ -160,11 +160,13 @@ name.
|
||||
Multiple source paths may reference the same content blob; the database tracks
|
||||
references rather than using filesystem refcounting.
|
||||
|
||||
Source and transformed image files are written under a temporary name, synced to
|
||||
disk, and only then renamed into place, so a crash leaves no file under its own
|
||||
name with part of its data. A request that finds a cached source or transformed
|
||||
image whose file is empty, or not the size the database records for it, removes
|
||||
it and fetches the source or converts the image again.
|
||||
Source and transformed image files, and the `.meta` file beside each transformed
|
||||
image, are written under a temporary name, synced to disk, and only then renamed
|
||||
into place, so a crash leaves no file under its own name with part of its data.
|
||||
When pixa starts, and every five minutes after, it removes each cached source or
|
||||
transformed image whose file is empty, or not the size the database records for
|
||||
it, with a warning naming the file; the next request for it fetches the source
|
||||
or converts the image again.
|
||||
|
||||
pixa's target is 1-5k r/s, which has not been measured at that rate (see Load
|
||||
Test). Toward it, pixa keeps in memory the content types of the 10,000
|
||||
|
||||
Reference in New Issue
Block a user