Check cached sources with upstream after the cache TTL (closes #69, closes #83)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
A cached source and its variants were served until eviction, so a changed image at the same URL was never picked up. Once a source was last checked 24 hours ago, the next request for one of its images asks upstream with If-None-Match and If-Modified-Since, from the ETag and Last-Modified it stored. A 304 renews the source and its variant without a download or a conversion. New bytes replace the source, and each variant made before them is converted again. A failed check serves the cached image and puts the URL in the negative cache. Sources and variants not checked successfully for 30 days are evicted. A variant file's modification time is its source's check time, so a hit still makes no database call. Existing tests change only in call signatures. The test that asserted two-day-old entries are served without asking upstream is replaced by the new ones. Model: opus-5-5
This commit is contained in:
1 parent
9f1e9f5253
commit
8feed69143
28 files changed
+1262
-226
No files matched your search
@@ -26,7 +26,7 @@ type heldFetcher struct {
|
||||
// Fetch sends on f.started, waits for f.release to be closed, and returns
|
||||
// httpfetcher.ErrUpstreamError.
|
||||
func (f heldFetcher) Fetch(
|
||||
context.Context, imagerequest.Source,
|
||||
context.Context, imagerequest.Source, string, string,
|
||||
) (*httpfetcher.FetchResult, error) {
|
||||
f.started <- struct{}{}
|
||||
|
||||
|
||||
@@ -131,7 +131,7 @@ func newMockFetcher(fs fs.FS) *mockFetcher {
|
||||
}
|
||||
|
||||
func (f *mockFetcher) Fetch(
|
||||
_ context.Context, source imagerequest.Source,
|
||||
_ context.Context, source imagerequest.Source, _, _ string,
|
||||
) (*httpfetcher.FetchResult, error) {
|
||||
path := source.Host + source.Path
|
||||
|
||||
|
||||
@@ -428,7 +428,7 @@ type failingFetcher struct {
|
||||
|
||||
// Fetch returns f's error.
|
||||
func (f failingFetcher) Fetch(
|
||||
context.Context, imagerequest.Source,
|
||||
context.Context, imagerequest.Source, string, string,
|
||||
) (*httpfetcher.FetchResult, error) {
|
||||
return nil, f.err
|
||||
}
|
||||
|
||||
@@ -36,10 +36,11 @@ type countingFetcher struct {
|
||||
// Fetch counts the fetch and passes it on.
|
||||
func (f *countingFetcher) Fetch(
|
||||
ctx context.Context, source imagerequest.Source,
|
||||
ifNoneMatch, ifModifiedSince string,
|
||||
) (*httpfetcher.FetchResult, error) {
|
||||
f.fetches.Add(1)
|
||||
|
||||
return f.Fetcher.Fetch(ctx, source)
|
||||
return f.Fetcher.Fetch(ctx, source, ifNoneMatch, ifModifiedSince)
|
||||
}
|
||||
|
||||
// newRefererRoutes returns both image routes of a Handlers whose
|
||||
|
||||
Reference in new issue
Block a user