Store a source the way a variant is stored, one row and one file per source URL (closes #308)
check / check (push) Waiting to run

A source is now one source_content row and one file named by its
SourceKey, the SHA-256 of its host, path and query, made as the variant
key is. A store writes the file, then upserts the row, and removes the
file when the row cannot be written; an eviction deletes the row, then
the file, with no lock. source_metadata, the JSON files under
cache/metadata/, MetadataStorage, the content lock, the eviction
transaction and its test hook are gone, as is the reconciliation pass
that dropped a source row whose file is missing. Two source URLs
serving the same bytes store them twice.

Model: opus-5-5
This commit is contained in:
2026-10-21 14:20:28 +00:00
parent 00d8ba26c5
commit 5d78e62ae9
24 changed files with 622 additions and 1468 deletions
+10 -8
View File
@@ -31,10 +31,10 @@ func TestEvictionReadsTheUsageTotal(t *testing.T) {
storeEvictionTestVariant(t, cache, testVariantKeyTwo,
bytes.Repeat([]byte{0x75}, 200))
// pixa never changes a source's size, but a statement that does must
// change the total too.
// Storing a source URL again changes its size, so the total must
// follow a statement that changes it.
_, err := cache.db.ExecContext(ctx,
`UPDATE source_content SET size_bytes = 900 WHERE content_hash = ?`,
`UPDATE source_content SET size_bytes = 900 WHERE source_key = ?`,
string(kept))
if err != nil {
t.Fatalf("failed to change the source's size: %v", err)
@@ -155,8 +155,9 @@ func TestUsageTotalNotCorrectedFromAnOlderSum(t *testing.T) {
// pass. No read the pass makes of a content table, to check its rows or
// to sum them, may cover more than two rows, so a request's query waits
// for one page at most. Between them the reads must still cover every
// row of both tables twice, once to check it and once to sum it, and the
// sum must put a wrong total right.
// variant row twice, once to check it and once to sum it, and every
// source row once, as source rows are read only to sum them, and the sum
// must put a wrong total right.
func TestReconciliationReadsAPageAtATime(t *testing.T) {
t.Parallel()
@@ -207,9 +208,10 @@ func TestReconciliationReadsAPageAtATime(t *testing.T) {
coveredRows += rows
}
// Ten rows, each read once to check it and once to sum it.
if coveredRows != 20 {
t.Errorf("the reads covered %d rows in all, want 20", coveredRows)
// Five variant rows, each read once to check it and once to sum it,
// and five source rows, read only to sum them.
if coveredRows != 15 {
t.Errorf("the reads covered %d rows in all, want 15", coveredRows)
}
usage, err := cache.UsageBytes(ctx)