A new one-row table, cache_usage, holds the total cache usage. Triggers on source_content and variant_content (insert, delete, update of size_bytes) keep it current in the statement that changes a row. UsageBytes, which every eviction pass calls, reads that row instead of summing both tables.
The reconciliation pass reads the content tables 1000 rows per query, for its row checks and for a full sum, and sets the total to the sum when they differ. It does so only if change_count, which each trigger raises, has not moved since the sum began; otherwise a store during the sum could be missed and the total set wrong.
source_content.last_accessed_at now defaults to the time the row is added, as in variant_content. The source eviction candidates were ordered by a COALESCE expression, so SQLite read and sorted the whole table on every eviction batch; they now come from the column's index. The issue did not list this query, but its definition of done covers it.
README.md loses the sentence saying requests wait for eviction's queries.
Disclosures:
Judgement call: while stores keep arriving, a wrong total can stay uncorrected until a pass in which no store lands during its sum.
Stats still sums the tables, now in pages: TestStats_LogsFailedCountQueries drops both tables and expects that sum to fail, and changing an existing test needs owner approval. No request calls Stats.
A database created before this change lacks cache_usage and the triggers, since 001_schema.sql is not applied to it again (pre-1.0 rule).
Model: opus-5-5
Fixes https://git.eeqj.de/sneak/pixa/issues/227.
- A new one-row table, `cache_usage`, holds the total cache usage. Triggers on `source_content` and `variant_content` (insert, delete, update of `size_bytes`) keep it current in the statement that changes a row. `UsageBytes`, which every eviction pass calls, reads that row instead of summing both tables.
- The reconciliation pass reads the content tables 1000 rows per query, for its row checks and for a full sum, and sets the total to the sum when they differ. It does so only if `change_count`, which each trigger raises, has not moved since the sum began; otherwise a store during the sum could be missed and the total set wrong.
- `source_content.last_accessed_at` now defaults to the time the row is added, as in `variant_content`. The source eviction candidates were ordered by a `COALESCE` expression, so SQLite read and sorted the whole table on every eviction batch; they now come from the column's index. The issue did not list this query, but its definition of done covers it.
- `README.md` loses the sentence saying requests wait for eviction's queries.
Disclosures:
- Judgement call: while stores keep arriving, a wrong total can stay uncorrected until a pass in which no store lands during its sum.
- `Stats` still sums the tables, now in pages: `TestStats_LogsFailedCountQueries` drops both tables and expects that sum to fail, and changing an existing test needs owner approval. No request calls `Stats`.
- A database created before this change lacks `cache_usage` and the triggers, since `001_schema.sql` is not applied to it again (pre-1.0 rule).
Model: opus-5-5
internal/imgcache/cache_usage_internal_test.go:156 (TestReconciliationReadsAPageAtATime): the definition of done in #227 asks for a test that a request's query does not wait for a whole reconciliation read. This test checks that variantKeysAfter and sourceContentHashesAfter each return one page, and that the pass still reaches every row, but not that the pass reads in pages: no test fails if reconcileVariantRows (internal/imgcache/eviction.go:674), reconcileSourceRows (eviction.go:821) or sumSizeBytesInPages (eviction.go:899) reads or sums a whole table in one query. Acceptable: a test that runs the reconciliation pass over more than one page of rows in each content table and fails if any one of its reads, the row checks and the sum alike, covers more than one page.
internal/imgcache/cachesize.go:75: the nolint reason still says UsageBytes sums file sizes; it now reads the total kept in cache_usage. Acceptable: a reason that describes what UsageBytes now returns.
Model: opus-5-5
1. `internal/imgcache/cache_usage_internal_test.go:156` (`TestReconciliationReadsAPageAtATime`): the definition of done in https://git.eeqj.de/sneak/pixa/issues/227 asks for a test that a request's query does not wait for a whole reconciliation read. This test checks that `variantKeysAfter` and `sourceContentHashesAfter` each return one page, and that the pass still reaches every row, but not that the pass reads in pages: no test fails if `reconcileVariantRows` (`internal/imgcache/eviction.go:674`), `reconcileSourceRows` (`eviction.go:821`) or `sumSizeBytesInPages` (`eviction.go:899`) reads or sums a whole table in one query. Acceptable: a test that runs the reconciliation pass over more than one page of rows in each content table and fails if any one of its reads, the row checks and the sum alike, covers more than one page.
2. `internal/imgcache/cachesize.go:75`: the `nolint` reason still says `UsageBytes` sums file sizes; it now reads the total kept in `cache_usage`. Acceptable: a reason that describes what `UsageBytes` now returns.
Model: opus-5-5
UsageBytes now reads the new cache_usage row, which triggers on
source_content and variant_content keep up to date in the statement
that adds, removes or resizes a row. The reconciliation pass reads both
tables 1000 rows per query, sums them, and corrects the total when it
differs, unless a row changed while it summed. Source rows now get
last_accessed_at when added, so choosing source images to evict reads
that column's index instead of sorting the whole table. Stats still
sums the tables, now in pages: an existing test drops both tables and
expects that sum to fail.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Fixes #227.
cache_usage, holds the total cache usage. Triggers onsource_contentandvariant_content(insert, delete, update ofsize_bytes) keep it current in the statement that changes a row.UsageBytes, which every eviction pass calls, reads that row instead of summing both tables.change_count, which each trigger raises, has not moved since the sum began; otherwise a store during the sum could be missed and the total set wrong.source_content.last_accessed_atnow defaults to the time the row is added, as invariant_content. The source eviction candidates were ordered by aCOALESCEexpression, so SQLite read and sorted the whole table on every eviction batch; they now come from the column's index. The issue did not list this query, but its definition of done covers it.README.mdloses the sentence saying requests wait for eviction's queries.Disclosures:
Statsstill sums the tables, now in pages:TestStats_LogsFailedCountQueriesdrops both tables and expects that sum to fail, and changing an existing test needs owner approval. No request callsStats.cache_usageand the triggers, since001_schema.sqlis not applied to it again (pre-1.0 rule).Model: opus-5-5
internal/imgcache/cache_usage_internal_test.go:156(TestReconciliationReadsAPageAtATime): the definition of done in #227 asks for a test that a request's query does not wait for a whole reconciliation read. This test checks thatvariantKeysAfterandsourceContentHashesAftereach return one page, and that the pass still reaches every row, but not that the pass reads in pages: no test fails ifreconcileVariantRows(internal/imgcache/eviction.go:674),reconcileSourceRows(eviction.go:821) orsumSizeBytesInPages(eviction.go:899) reads or sums a whole table in one query. Acceptable: a test that runs the reconciliation pass over more than one page of rows in each content table and fails if any one of its reads, the row checks and the sum alike, covers more than one page.internal/imgcache/cachesize.go:75: thenolintreason still saysUsageBytessums file sizes; it now reads the total kept incache_usage. Acceptable: a reason that describes whatUsageBytesnow returns.Model: opus-5-5
0dfb2e9651to7cbcd3957f7cbcd3957fto160335724eReview passed.
Model: opus-5-5