2 Commits
Author SHA1 Message Date
clawbot afcef4245b Go on past cache entries the reconciliation pass cannot read (closes #343)
check / check (push) Canceled after 0s
The walk of each storage directory returned the error of a directory it
could not list, or of a source or variant file whose info it could not
read. The pass then stopped before its later checks, every time it ran.
Now reconcileFiles skips a directory it cannot list and a file whose
info it cannot read, temp files included. It logs a warning naming the
path and the error, and the walk goes on. It reads each file's info
once and hands it to sweepStaleTempFile, reconcileVariantFile or
reconcileSourceFile. A file renamed or removed after its directory was
listed is skipped with no warning. A cancelled context still stops the
walk with its error. README.md says so under Storage.

Model: opus-5-5
2026-10-27 01:05:15 +00:00
clawbot e7515e50ee Test that reconciliation goes on past entries it cannot read
Each storage directory gets a directory the pass cannot list and a file
whose info it cannot read. Their paths are longer than Linux accepts,
even for root. The test wants a warning naming each path and the error.
It also wants the rest of the pass done: a variant file with no row
adopted, the row of a missing variant file dropped, a source file with
no row removed, and a wrong usage total corrected. A second test puts a
temp file whose info cannot be read in each storage directory and wants
the same warning. On next the first test's pass stops at the first such
directory and returns its error. The second test gets no warning.

Model: opus-5-5
2026-10-27 01:03:35 +00:00
2 changed files with 9 additions and 9 deletions
+4 -4
View File
@@ -256,10 +256,10 @@ Each look also drops the rows of variants whose files are missing. When pixa
cannot read a variant file's size for another reason, it keeps the variant's
row. It logs a warning naming the variant key and the error.
During a look, pixa skips any file or directory in the cache that it cannot
read. It logs a warning naming the path and the error, and goes on with the rest
of the look. A file renamed or removed after pixa listed its directory gets no
warning.
During a look, pixa skips a directory it cannot list and a file whose size it
cannot read. It logs a warning naming the path and the error, and goes on with
the rest of the look. A file renamed or removed after pixa listed its directory
gets no warning.
pixa's target is 1-5k r/s, which has not been measured at that rate (see Load
Test). Toward it, a cache hit opens the variant's file and nothing else, and
+5 -5
View File
@@ -948,11 +948,11 @@ func TestReconciliationGoesOnPastEntriesItCannotRead(t *testing.T) {
}
}
// TestReconciliationWarnsOfTempFilesItCannotRead puts, in each storage
// directory, a temp file whose info the pass cannot read. The pass must
// log a warning naming each one's path and the error, as it does for any
// other file whose info it cannot read.
func TestReconciliationWarnsOfTempFilesItCannotRead(t *testing.T) {
// TestReconciliationWarnsOfTempFilesWhoseInfoItCannotRead puts, in each
// storage directory, a temp file whose info the pass cannot read. The
// pass must log a warning naming each one's path and the error, as it
// does for any other file whose info it cannot read.
func TestReconciliationWarnsOfTempFilesWhoseInfoItCannotRead(t *testing.T) {
t.Parallel()
cache, _ := newEvictionTestCache(t, 1<<30)