Keep a variant's accounting row when its file cannot be read (closes #340)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
The reconciliation pass dropped a variant's accounting row on any error reading its file's size, and logged every such file as missing. A file still on disk then had no row, so eviction no longer counted it against cache_max_bytes. Now only a missing file (ErrNotFound, which FileStore.Size returns for it) loses its row, with the same info line. Any other error keeps the row and logs a warning naming the variant key and the error, and the pass goes on to the next key. README.md says so under Storage. Model: opus-5-5
This commit was merged in pull request #342.
This commit is contained in:
@@ -252,6 +252,10 @@ less than ten times as long after the last look ended as that look took. After a
|
|||||||
look that took one minute, the next one starts 10 to 15 minutes after that look
|
look that took one minute, the next one starts 10 to 15 minutes after that look
|
||||||
ended.
|
ended.
|
||||||
|
|
||||||
|
Each look also drops the rows of variants whose files are missing. When pixa
|
||||||
|
cannot read a variant file's size for another reason, it keeps the variant's
|
||||||
|
row. It logs a warning naming the variant key and the error.
|
||||||
|
|
||||||
pixa's target is 1-5k r/s, which has not been measured at that rate (see Load
|
pixa's target is 1-5k r/s, which has not been measured at that rate (see Load
|
||||||
Test). Toward it, a cache hit opens the variant's file and nothing else, and
|
Test). Toward it, a cache hit opens the variant's file and nothing else, and
|
||||||
makes no database call. The content type comes from the file's first bytes,
|
makes no database call. The content type comes from the file's first bytes,
|
||||||
|
|||||||
@@ -30,6 +30,13 @@ P2: security: per-origin rate limiting
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-26 the reconciliation pass drops a variant's accounting row only when
|
||||||
|
its file is missing (closes #340). Any other error reading the file's size
|
||||||
|
keeps the row. It dropped the row on any error and logged the file as missing.
|
||||||
|
- The kept row gets the warning "kept accounting row for unreadable variant
|
||||||
|
file", with the variant key and the error.
|
||||||
|
- The pass checks for `ErrNotFound`, which `FileStore.Size` returns for a
|
||||||
|
missing file.
|
||||||
- 2026-10-26 on both image routes, a request is answered 499 and logged at info
|
- 2026-10-26 on both image routes, a request is answered 499 and logged at info
|
||||||
level when its client goes away before its image is ready, unless it is the
|
level when its client goes away before its image is ready, unless it is the
|
||||||
request making the image (closes #339). It was answered 500 and logged at
|
request making the image (closes #339). It was answered 500 and logged at
|
||||||
|
|||||||
@@ -613,7 +613,9 @@ func (c *Cache) adoptVariantFile(
|
|||||||
}
|
}
|
||||||
|
|
||||||
// reconcileVariantRows drops accounting rows whose variant files are
|
// reconcileVariantRows drops accounting rows whose variant files are
|
||||||
// missing, so the database never references deleted content.
|
// missing, so the database never references deleted content. A file whose
|
||||||
|
// size cannot be read for another reason may still be on disk, so its row
|
||||||
|
// stays, with a warning naming its key and the error.
|
||||||
func (c *Cache) reconcileVariantRows(ctx context.Context) error {
|
func (c *Cache) reconcileVariantRows(ctx context.Context) error {
|
||||||
var after imagerequest.VariantKey
|
var after imagerequest.VariantKey
|
||||||
|
|
||||||
@@ -641,6 +643,13 @@ func (c *Cache) reconcileVariantRows(ctx context.Context) error {
|
|||||||
continue
|
continue
|
||||||
}
|
}
|
||||||
|
|
||||||
|
if !errors.Is(err, ErrNotFound) {
|
||||||
|
c.log.Warn("kept accounting row for unreadable variant file",
|
||||||
|
"cache_key", key, "error", err)
|
||||||
|
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
_, err = c.db.ExecContext(ctx,
|
_, err = c.db.ExecContext(ctx,
|
||||||
`DELETE FROM variant_content WHERE cache_key = ?`, string(key))
|
`DELETE FROM variant_content WHERE cache_key = ?`, string(key))
|
||||||
if err != nil {
|
if err != nil {
|
||||||
|
|||||||
Reference in New Issue
Block a user