Check cached sources with upstream after the cache TTL (closes #69, closes #83)
check / check (push) Canceled after 0s

A cached source and its variants were served until eviction, so a
changed image at the same URL was never picked up.

Once a source was last checked 24 hours ago, the next request for one
of its images asks upstream with If-None-Match and If-Modified-Since,
from the ETag and Last-Modified it stored. A 304 renews the source and
its variant without a download or a conversion. New bytes replace the
source, and each variant made before them is converted again. A failed
check serves the cached image and puts the URL in the negative cache.
Sources and variants not checked successfully for 30 days are evicted.

A variant file's modification time is its source's check time, so a
hit still makes no database call.

Existing tests change only in call signatures. The test that asserted
two-day-old entries are served without asking upstream is replaced by
the new ones.

Model: opus-5-5
This commit is contained in:
clawbot committed 2026-11-11 00:24:06 +00:00
1 parent 9f1e9f5253
commit 8feed69143
28 files changed
+1262 -226

No files matched your search

+5 -4
View File
@@ -101,7 +101,7 @@ func storeEvictionTestSource(
}
key, err := cache.PutSource(
context.Background(), req.Source, bytes.NewReader(content),
context.Background(), req.Source, bytes.NewReader(content), SourceInfo{},
)
if err != nil {
t.Fatalf("PutSource(%s%s) failed: %v", host, path, err)
@@ -117,7 +117,7 @@ func storeEvictionTestVariant(
) {
t.Helper()
err := cache.PutVariant(t.Context(), key, bytes.NewReader(content))
err := cache.PutVariant(t.Context(), key, time.Now(), bytes.NewReader(content))
if err != nil {
t.Fatalf("PutVariant(%s) failed: %v", key, err)
}
@@ -432,13 +432,14 @@ func assertDisabledCacheWritesAreNoOps(
ctx := t.Context()
err := cache.PutVariant(
ctx, imagerequest.CacheKey(req), bytes.NewReader([]byte("data")),
ctx, imagerequest.CacheKey(req), time.Now(), bytes.NewReader([]byte("data")),
)
if err != nil {
t.Fatalf("PutVariant on disabled cache must be a no-op, got error: %v", err)
}
key, err := cache.PutSource(ctx, req.Source, bytes.NewReader([]byte("data")))
key, err := cache.PutSource(ctx, req.Source, bytes.NewReader([]byte("data")),
SourceInfo{})
if err != nil {
t.Fatalf("PutSource on disabled cache must be a no-op, got error: %v", err)
}