Keep a variant's accounting row when its file cannot be read (closes #340) #342

Merged
clawbot merged 2 commits from issue-340-reconcile-read-errors into next 2026-10-27 01:04:23 +01:00
3 changed files with 21 additions and 1 deletions
Showing only changes of commit 33045a24b5 - Show all commits

No files matched your search

+4
View File
@@ -252,6 +252,10 @@ less than ten times as long after the last look ended as that look took. After a
look that took one minute, the next one starts 10 to 15 minutes after that look look that took one minute, the next one starts 10 to 15 minutes after that look
ended. ended.
Each look also drops the rows of variants whose files are missing. When pixa
cannot read a variant file's size for another reason, it keeps the variant's
row. It logs a warning naming the variant key and the error.
pixa's target is 1-5k r/s, which has not been measured at that rate (see Load pixa's target is 1-5k r/s, which has not been measured at that rate (see Load
Test). Toward it, a cache hit opens the variant's file and nothing else, and Test). Toward it, a cache hit opens the variant's file and nothing else, and
makes no database call. The content type comes from the file's first bytes, makes no database call. The content type comes from the file's first bytes,
+7
View File
@@ -30,6 +30,13 @@ P2: security: per-origin rate limiting
# Completed Steps # Completed Steps
- 2026-10-26 the reconciliation pass drops a variant's accounting row only when
its file is missing (closes #340). Any other error reading the file's size
keeps the row. It dropped the row on any error and logged the file as missing.
- The kept row gets the warning "kept accounting row for unreadable variant
file", with the variant key and the error.
- The pass checks for `ErrNotFound`, which `FileStore.Size` returns for a
missing file.
- 2026-10-26 on both image routes, a request is answered 499 and logged at info - 2026-10-26 on both image routes, a request is answered 499 and logged at info
level when its client goes away before its image is ready, unless it is the level when its client goes away before its image is ready, unless it is the
request making the image (closes #339). It was answered 500 and logged at request making the image (closes #339). It was answered 500 and logged at
+10 -1
View File
@@ -613,7 +613,9 @@ func (c *Cache) adoptVariantFile(
} }
// reconcileVariantRows drops accounting rows whose variant files are // reconcileVariantRows drops accounting rows whose variant files are
// missing, so the database never references deleted content. // missing, so the database never references deleted content. A file whose
// size cannot be read for another reason may still be on disk, so its row
// stays, with a warning naming its key and the error.
func (c *Cache) reconcileVariantRows(ctx context.Context) error { func (c *Cache) reconcileVariantRows(ctx context.Context) error {
var after imagerequest.VariantKey var after imagerequest.VariantKey
@@ -641,6 +643,13 @@ func (c *Cache) reconcileVariantRows(ctx context.Context) error {
continue continue
} }
if !errors.Is(err, ErrNotFound) {
c.log.Warn("kept accounting row for unreadable variant file",
"cache_key", key, "error", err)
continue
}
_, err = c.db.ExecContext(ctx, _, err = c.db.ExecContext(ctx,
`DELETE FROM variant_content WHERE cache_key = ?`, string(key)) `DELETE FROM variant_content WHERE cache_key = ?`, string(key))
if err != nil { if err != nil {