Keep variant content types in memory for cache hits (closes #70) #157

Merged
clawbot merged 8 commits from issue-70-metacache into next 2026-09-29 12:00:11 +02:00
2 changed files with 7 additions and 3 deletions
Showing only changes of commit 77e9c9cd7f - Show all commits
+2 -1
View File
@@ -32,7 +32,8 @@ P2: security: referer blacklist
- 2026-09-29 variant content types kept in memory (closes #70):
`Cache.metaCache` holds the content types of up to 10,000 variants in an LRU
(`github.com/hashicorp/golang-lru/v2`), filled by `StoreVariant` and by
`GetVariant` after it reads a `.meta` file; for a variant it holds, `Lookup`
`GetVariant` after it reads a `.meta` file, where a type `StoreVariant` added
meanwhile is kept over the one read; for a variant it holds, `Lookup`
skips the check of the disk and `GetVariant` skips the `.meta` read, still
opening the variant file and taking the size from it; eviction removes the
entry before deleting the files, and `GetVariant` removes it when the file
+5 -2
View File
@@ -525,7 +525,10 @@ func (c *Cache) IncrementTransformCount(ctx context.Context) {
// loadVariantWithMeta is GetVariant for a variant metaCache does not
// hold: it reads the content type from the variant's .meta file and
// keeps it in metaCache.
// keeps it in metaCache, unless a StoreVariant has put one there
// meanwhile. The stored one wins, since this read may have found the
// variant file before the store wrote the .meta file, and so got
// application/octet-stream.
func (c *Cache) loadVariantWithMeta(
cacheKey VariantKey,
) (io.ReadCloser, int64, string, error) {
@@ -534,7 +537,7 @@ func (c *Cache) loadVariantWithMeta(
return nil, 0, "", err
}
c.metaCache.Add(cacheKey, contentType)
c.metaCache.ContainsOrAdd(cacheKey, contentType)
return reader, size, contentType, nil
}