1 Commits
Author SHA1 Message Date
clawbot 155cc94fe6 Go on past cache entries the reconciliation pass cannot read (closes #343)
check / check (push) Canceled after 0s
The walk of each storage directory returned the error of an entry it
could not read. The pass then stopped before its later checks, and the
same entry stopped every later pass. Now reconcileFiles skips a
directory it cannot list and a file whose info it cannot read, temp
files included. It logs a warning naming the path and the error, and the
walk goes on. It reads each file's info once, before the temp-file
check, and hands it to sweepStaleTempFile, reconcileVariantFile or
reconcileSourceFile. A file renamed or removed after its directory was
listed is skipped with no warning. A cancelled context still stops the
walk with its error. README.md says so under Storage.

Model: opus-5-5
2026-10-27 00:44:00 +00:00
3 changed files with 13 additions and 6 deletions
+2 -1
View File
@@ -258,7 +258,8 @@ row. It logs a warning naming the variant key and the error.
During a look, pixa skips any file or directory in the cache that it cannot
read. It logs a warning naming the path and the error, and goes on with the rest
of the look.
of the look. A file renamed or removed after pixa listed its directory gets no
warning.
pixa's target is 1-5k r/s, which has not been measured at that rate (see Load
Test). Toward it, a cache hit opens the variant's file and nothing else, and
+3 -2
View File
@@ -33,8 +33,9 @@ P2: security: per-origin rate limiting
- 2026-10-27 the reconciliation pass skips a directory it cannot list and a file
whose info it cannot read, and goes on (closes #343). It returned the error,
so one such entry stopped every pass before its later checks.
- Each skipped entry gets the warning "skipped unreadable entry in cache
directory", with its path and the error.
- It logs the warning "skipped unreadable entry in cache directory", with
the path and the error.
- A file renamed or removed after its directory was listed gets no warning.
- `reconcileFiles` reads each file's info once, before the temp-file check,
and hands it to `sweepStaleTempFile`, `reconcileVariantFile` or
`reconcileSourceFile`.
+8 -3
View File
@@ -517,9 +517,10 @@ func (c *Cache) reconcileAccounting(ctx context.Context) error {
// reconcileFiles walks baseDir, one of the cache's storage directories,
// sweeping stale temp files and passing each other file, with its info and
// name, to reconcileFile. It skips a directory it cannot list and a file
// whose info it cannot read. Each gets a warning naming its path and the
// error, and the walk goes on, so one such entry does not stop the pass.
// Once ctx is cancelled it stops at the next file and returns ctx's error.
// whose info it cannot read, so one such entry does not stop the pass. It
// logs a warning naming the path and the error. A file renamed or removed
// after its directory was listed is skipped with no warning. Once ctx is
// cancelled it stops at the next file and returns ctx's error.
func (c *Cache) reconcileFiles(
ctx context.Context, baseDir string,
reconcileFile func(path string, info fs.FileInfo, name string) error,
@@ -543,6 +544,10 @@ func (c *Cache) reconcileFiles(
}
info, err := entry.Info()
if errors.Is(err, fs.ErrNotExist) {
return nil
}
if err != nil {
c.log.Warn("skipped unreadable entry in cache directory",
"path", path, "error", err)