1 Commits
Author SHA1 Message Date
clawbot 11e3acc76c Go on past cache entries the reconciliation pass cannot read (closes #343)
check / check (push) Canceled after 0s
The walk of each storage directory returned the error of an entry it
could not read. The pass then stopped before its later checks, and the
same entry stopped every later pass. Now reconcileFiles skips a
directory it cannot list and a file whose info it cannot read, temp
files included. It logs a warning naming the path and the error, and the
walk goes on. It reads each file's info once, before the temp-file
check, and hands it to sweepStaleTempFile, reconcileVariantFile or
reconcileSourceFile. A cancelled context still stops the walk with its
error. README.md says so under Storage.

Model: opus-5-5
2026-10-27 00:35:17 +00:00
3 changed files with 6 additions and 13 deletions
+1 -2
View File
@@ -258,8 +258,7 @@ row. It logs a warning naming the variant key and the error.
During a look, pixa skips any file or directory in the cache that it cannot
read. It logs a warning naming the path and the error, and goes on with the rest
of the look. A file renamed or removed after pixa listed its directory gets no
warning.
of the look.
pixa's target is 1-5k r/s, which has not been measured at that rate (see Load
Test). Toward it, a cache hit opens the variant's file and nothing else, and
+2 -3
View File
@@ -33,9 +33,8 @@ P2: security: per-origin rate limiting
- 2026-10-27 the reconciliation pass skips a directory it cannot list and a file
whose info it cannot read, and goes on (closes #343). It returned the error,
so one such entry stopped every pass before its later checks.
- It logs the warning "skipped unreadable entry in cache directory", with
the path and the error.
- A file renamed or removed after its directory was listed gets no warning.
- Each skipped entry gets the warning "skipped unreadable entry in cache
directory", with its path and the error.
- `reconcileFiles` reads each file's info once, before the temp-file check,
and hands it to `sweepStaleTempFile`, `reconcileVariantFile` or
`reconcileSourceFile`.
+3 -8
View File
@@ -517,10 +517,9 @@ func (c *Cache) reconcileAccounting(ctx context.Context) error {
// reconcileFiles walks baseDir, one of the cache's storage directories,
// sweeping stale temp files and passing each other file, with its info and
// name, to reconcileFile. It skips a directory it cannot list and a file
// whose info it cannot read, so one such entry does not stop the pass. It
// logs a warning naming the path and the error. A file renamed or removed
// after its directory was listed is skipped with no warning. Once ctx is
// cancelled it stops at the next file and returns ctx's error.
// whose info it cannot read. Each gets a warning naming its path and the
// error, and the walk goes on, so one such entry does not stop the pass.
// Once ctx is cancelled it stops at the next file and returns ctx's error.
func (c *Cache) reconcileFiles(
ctx context.Context, baseDir string,
reconcileFile func(path string, info fs.FileInfo, name string) error,
@@ -544,10 +543,6 @@ func (c *Cache) reconcileFiles(
}
info, err := entry.Info()
if errors.Is(err, fs.ErrNotExist) {
return nil
}
if err != nil {
c.log.Warn("skipped unreadable entry in cache directory",
"path", path, "error", err)