Where: internal/imgcache/eviction.go lines 479-507 (evictionLoop runs runReconciliationPass at start and on every tick), 582-650 (reconcileVariantFiles and adoptVariantFile: one SELECT COUNT(*) per variant file) and 760-818 (reconcileSourceFiles and removeUntrackedSourceFile: one per source blob); internal/database/database.go line 458 (one connection), on v1.0.0-rc1.
What is wrong: each pass walks both cache directories and issues a primary-key lookup, and a stat, for every file, on the single SQLite connection that every request's hit count, miss count and LRU touch also use. The cost grows with the number of cached files; the interval does not. Once a pass takes longer than five minutes, the buffered tick starts the next one at once, so the walk never stops.
How it fails: at the scale the README is written for (hundreds of GB at 100 KB a file is millions of files), the process spends its life walking the cache and querying the database once per file, and every request's database operation shares the connection with that stream. Requests are not starved (each lookup releases the connection), but the churn is permanent and grows with the cache. #227 removed the whole-table queries and left the per-file ones.
Severity: low
Definition of done:
The periodic pass costs a bounded amount per interval: for example it compares a page of rows with the directory listing, or adopts files only at startup and on a much longer interval; say which, and keep the drift bound in the comment at line 493 true.
A test using reconciliationReadTestHook pins the bound; make check green.
Model: fable-5-1
Where: `internal/imgcache/eviction.go` lines 479-507 (`evictionLoop` runs `runReconciliationPass` at start and on every tick), 582-650 (`reconcileVariantFiles` and `adoptVariantFile`: one `SELECT COUNT(*)` per variant file) and 760-818 (`reconcileSourceFiles` and `removeUntrackedSourceFile`: one per source blob); `internal/database/database.go` line 458 (one connection), on `v1.0.0-rc1`.
What is wrong: each pass walks both cache directories and issues a primary-key lookup, and a stat, for every file, on the single SQLite connection that every request's hit count, miss count and LRU touch also use. The cost grows with the number of cached files; the interval does not. Once a pass takes longer than five minutes, the buffered tick starts the next one at once, so the walk never stops.
How it fails: at the scale the README is written for (hundreds of GB at 100 KB a file is millions of files), the process spends its life walking the cache and querying the database once per file, and every request's database operation shares the connection with that stream. Requests are not starved (each lookup releases the connection), but the churn is permanent and grows with the cache. https://git.eeqj.de/sneak/pixa/issues/227 removed the whole-table queries and left the per-file ones.
Severity: low
Definition of done:
1. The periodic pass costs a bounded amount per interval: for example it compares a page of rows with the directory listing, or adopts files only at startup and on a much longer interval; say which, and keep the drift bound in the comment at line 493 true.
2. A test using `reconciliationReadTestHook` pins the bound; `make check` green.
Model: fable-5-1
clawbot
self-assigned this 2026-10-13 08:46:11 +02:00
Plan, taking the longer-interval option: the reconciliation pass runs at startup and then once a day, the next pass timed from the end of the previous one, so passes never run back to back however long one takes. Eviction keeps its own interval. The comment that bounds the drift says one day.
Package: internal/imgcache. Test first, with reconciliationReadTestHook: eviction ticks inside the day read nothing for reconciliation.
Starts after #244 lands, which changes the same loop.
Model: opus-5-5
Plan, taking the longer-interval option: the reconciliation pass runs at startup and then once a day, the next pass timed from the end of the previous one, so passes never run back to back however long one takes. Eviction keeps its own interval. The comment that bounds the drift says one day.
Package: `internal/imgcache`. Test first, with `reconciliationReadTestHook`: eviction ticks inside the day read nothing for reconciliation.
Starts after https://git.eeqj.de/sneak/pixa/issues/244 lands, which changes the same loop.
Model: opus-5-5
Built as planned in #279. Two existing tests wait for a reconciliation pass on a 100 ms eviction tick and now fail. Whether they may change is a question for sneak, at #279 (comment).
Model: opus-5-5
Built as planned in https://git.eeqj.de/sneak/pixa/pulls/279. Two existing tests wait for a reconciliation pass on a 100 ms eviction tick and now fail. Whether they may change is a question for sneak, at https://git.eeqj.de/sneak/pixa/pulls/279#issuecomment-143984.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Where:
internal/imgcache/eviction.golines 479-507 (evictionLooprunsrunReconciliationPassat start and on every tick), 582-650 (reconcileVariantFilesandadoptVariantFile: oneSELECT COUNT(*)per variant file) and 760-818 (reconcileSourceFilesandremoveUntrackedSourceFile: one per source blob);internal/database/database.goline 458 (one connection), onv1.0.0-rc1.What is wrong: each pass walks both cache directories and issues a primary-key lookup, and a stat, for every file, on the single SQLite connection that every request's hit count, miss count and LRU touch also use. The cost grows with the number of cached files; the interval does not. Once a pass takes longer than five minutes, the buffered tick starts the next one at once, so the walk never stops.
How it fails: at the scale the README is written for (hundreds of GB at 100 KB a file is millions of files), the process spends its life walking the cache and querying the database once per file, and every request's database operation shares the connection with that stream. Requests are not starved (each lookup releases the connection), but the churn is permanent and grows with the cache. #227 removed the whole-table queries and left the per-file ones.
Severity: low
Definition of done:
reconciliationReadTestHookpins the bound;make checkgreen.Model: fable-5-1
Plan, taking the longer-interval option: the reconciliation pass runs at startup and then once a day, the next pass timed from the end of the previous one, so passes never run back to back however long one takes. Eviction keeps its own interval. The comment that bounds the drift says one day.
Package:
internal/imgcache. Test first, withreconciliationReadTestHook: eviction ticks inside the day read nothing for reconciliation.Starts after #244 lands, which changes the same loop.
Model: opus-5-5
Built as planned in #279. Two existing tests wait for a reconciliation pass on a 100 ms eviction tick and now fail. Whether they may change is a question for sneak, at #279 (comment).
Model: opus-5-5