Store a source the way a variant is stored, one row and one file per source URL (closes #308)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
A source is now one source_content row and one file named by its SourceKey, the SHA-256 of its host, path and query, made as the variant key is. A store writes the file, then upserts the row, and removes the file when the row cannot be written; an eviction deletes the row, then the file, with no lock. source_metadata, the JSON files under cache/metadata/, MetadataStorage, the content lock, the eviction transaction and its test hook are gone, as is the reconciliation pass that dropped a source row whose file is missing. Two source URLs serving the same bytes store them twice. Model: opus-5-5
This commit is contained in:
@@ -31,10 +31,10 @@ func TestEvictionReadsTheUsageTotal(t *testing.T) {
|
||||
storeEvictionTestVariant(t, cache, testVariantKeyTwo,
|
||||
bytes.Repeat([]byte{0x75}, 200))
|
||||
|
||||
// pixa never changes a source's size, but a statement that does must
|
||||
// change the total too.
|
||||
// Storing a source URL again changes its size, so the total must
|
||||
// follow a statement that changes it.
|
||||
_, err := cache.db.ExecContext(ctx,
|
||||
`UPDATE source_content SET size_bytes = 900 WHERE content_hash = ?`,
|
||||
`UPDATE source_content SET size_bytes = 900 WHERE source_key = ?`,
|
||||
string(kept))
|
||||
if err != nil {
|
||||
t.Fatalf("failed to change the source's size: %v", err)
|
||||
@@ -155,8 +155,9 @@ func TestUsageTotalNotCorrectedFromAnOlderSum(t *testing.T) {
|
||||
// pass. No read the pass makes of a content table, to check its rows or
|
||||
// to sum them, may cover more than two rows, so a request's query waits
|
||||
// for one page at most. Between them the reads must still cover every
|
||||
// row of both tables twice, once to check it and once to sum it, and the
|
||||
// sum must put a wrong total right.
|
||||
// variant row twice, once to check it and once to sum it, and every
|
||||
// source row once, as source rows are read only to sum them, and the sum
|
||||
// must put a wrong total right.
|
||||
func TestReconciliationReadsAPageAtATime(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
@@ -207,9 +208,10 @@ func TestReconciliationReadsAPageAtATime(t *testing.T) {
|
||||
coveredRows += rows
|
||||
}
|
||||
|
||||
// Ten rows, each read once to check it and once to sum it.
|
||||
if coveredRows != 20 {
|
||||
t.Errorf("the reads covered %d rows in all, want 20", coveredRows)
|
||||
// Five variant rows, each read once to check it and once to sum it,
|
||||
// and five source rows, read only to sum them.
|
||||
if coveredRows != 15 {
|
||||
t.Errorf("the reads covered %d rows in all, want 15", coveredRows)
|
||||
}
|
||||
|
||||
usage, err := cache.UsageBytes(ctx)
|
||||
|
||||
Reference in New Issue
Block a user