1 Commits
Author SHA1 Message Date
clawbot 11e3acc76c Go on past cache entries the reconciliation pass cannot read (closes #343)
check / check (push) Canceled after 0s
The walk of each storage directory returned the error of an entry it
could not read. The pass then stopped before its later checks, and the
same entry stopped every later pass. Now reconcileFiles skips a
directory it cannot list and a file whose info it cannot read, temp
files included. It logs a warning naming the path and the error, and the
walk goes on. It reads each file's info once, before the temp-file
check, and hands it to sweepStaleTempFile, reconcileVariantFile or
reconcileSourceFile. A cancelled context still stops the walk with its
error. README.md says so under Storage.

Model: opus-5-5
2026-10-27 00:35:17 +00:00
3 changed files with 6 additions and 13 deletions

No files matched your search

+1 -2
View File
@@ -258,8 +258,7 @@ row. It logs a warning naming the variant key and the error.
During a look, pixa skips any file or directory in the cache that it cannot During a look, pixa skips any file or directory in the cache that it cannot
read. It logs a warning naming the path and the error, and goes on with the rest read. It logs a warning naming the path and the error, and goes on with the rest
of the look. A file renamed or removed after pixa listed its directory gets no of the look.
warning.
pixa's target is 1-5k r/s, which has not been measured at that rate (see Load pixa's target is 1-5k r/s, which has not been measured at that rate (see Load
Test). Toward it, a cache hit opens the variant's file and nothing else, and Test). Toward it, a cache hit opens the variant's file and nothing else, and
+2 -3
View File
@@ -33,9 +33,8 @@ P2: security: per-origin rate limiting
- 2026-10-27 the reconciliation pass skips a directory it cannot list and a file - 2026-10-27 the reconciliation pass skips a directory it cannot list and a file
whose info it cannot read, and goes on (closes #343). It returned the error, whose info it cannot read, and goes on (closes #343). It returned the error,
so one such entry stopped every pass before its later checks. so one such entry stopped every pass before its later checks.
- It logs the warning "skipped unreadable entry in cache directory", with - Each skipped entry gets the warning "skipped unreadable entry in cache
the path and the error. directory", with its path and the error.
- A file renamed or removed after its directory was listed gets no warning.
- `reconcileFiles` reads each file's info once, before the temp-file check, - `reconcileFiles` reads each file's info once, before the temp-file check,
and hands it to `sweepStaleTempFile`, `reconcileVariantFile` or and hands it to `sweepStaleTempFile`, `reconcileVariantFile` or
`reconcileSourceFile`. `reconcileSourceFile`.
+3 -8
View File
@@ -517,10 +517,9 @@ func (c *Cache) reconcileAccounting(ctx context.Context) error {
// reconcileFiles walks baseDir, one of the cache's storage directories, // reconcileFiles walks baseDir, one of the cache's storage directories,
// sweeping stale temp files and passing each other file, with its info and // sweeping stale temp files and passing each other file, with its info and
// name, to reconcileFile. It skips a directory it cannot list and a file // name, to reconcileFile. It skips a directory it cannot list and a file
// whose info it cannot read, so one such entry does not stop the pass. It // whose info it cannot read. Each gets a warning naming its path and the
// logs a warning naming the path and the error. A file renamed or removed // error, and the walk goes on, so one such entry does not stop the pass.
// after its directory was listed is skipped with no warning. Once ctx is // Once ctx is cancelled it stops at the next file and returns ctx's error.
// cancelled it stops at the next file and returns ctx's error.
func (c *Cache) reconcileFiles( func (c *Cache) reconcileFiles(
ctx context.Context, baseDir string, ctx context.Context, baseDir string,
reconcileFile func(path string, info fs.FileInfo, name string) error, reconcileFile func(path string, info fs.FileInfo, name string) error,
@@ -544,10 +543,6 @@ func (c *Cache) reconcileFiles(
} }
info, err := entry.Info() info, err := entry.Info()
if errors.Is(err, fs.ErrNotExist) {
return nil
}
if err != nil { if err != nil {
c.log.Warn("skipped unreadable entry in cache directory", c.log.Warn("skipped unreadable entry in cache directory",
"path", path, "error", err) "path", path, "error", err)