Keep variant content types in memory for cache hits (closes #70)
check / check (push) Successful in 14s
check / check (push) Successful in 14s
Cache.metaCache was declared and never used, so every cache hit read and parsed the variant's .meta file. It is now an LRU (github.com/hashicorp/golang-lru/v2) of up to 10,000 variants' content types, filled by StoreVariant and by a read of a .meta file, so a hit for a variant it holds skips the .meta read. Only a type from a .meta file or a store ever enters memory, never the application/octet-stream fallback, and a stored type is never replaced by an older one from disk. The variant file itself is still opened on every hit, so nothing is served from memory alone. Model: opus-5-5
This commit was merged in pull request #157.
This commit is contained in:
@@ -39,8 +39,8 @@ const tempFilePrefix = ".tmp-"
|
||||
// to each variant file.
|
||||
const variantMetaSuffix = ".meta"
|
||||
|
||||
// fallbackContentType is recorded when a reconciled variant file has
|
||||
// no readable .meta sidecar.
|
||||
// fallbackContentType is the content type given to a variant file that
|
||||
// has no readable .meta sidecar, when it is served or reconciled.
|
||||
const fallbackContentType = "application/octet-stream"
|
||||
|
||||
// UsageBytes returns the total number of bytes of cache content
|
||||
@@ -271,7 +271,9 @@ func (c *Cache) sourceCandidates(ctx context.Context) ([]evictionCandidate, erro
|
||||
|
||||
// evictVariant removes one variant: accounting row first, then the
|
||||
// content and .meta files, so the database never references a deleted
|
||||
// file.
|
||||
// file. The metaCache entry goes before the files; a GetVariant that
|
||||
// read them just before may put it back, and the next GetVariant then
|
||||
// fails to open the file and removes it again.
|
||||
func (c *Cache) evictVariant(ctx context.Context, cacheKey VariantKey) error {
|
||||
_, err := c.db.ExecContext(ctx,
|
||||
`DELETE FROM variant_content WHERE cache_key = ?`, string(cacheKey))
|
||||
@@ -279,6 +281,8 @@ func (c *Cache) evictVariant(ctx context.Context, cacheKey VariantKey) error {
|
||||
return fmt.Errorf("failed to delete variant accounting row: %w", err)
|
||||
}
|
||||
|
||||
c.metaCache.Remove(cacheKey)
|
||||
|
||||
err = c.variants.DeleteWithMeta(cacheKey)
|
||||
if err != nil {
|
||||
return err
|
||||
|
||||
Reference in New Issue
Block a user