Say what the content phase's check does to a stale record
check / check (push) Successful in 53s

The README content phase item and the contentCandidates comment said a
file failing the check is not a duplicate. It keeps its record and does
not count as a match; with a content hash it is still reported until a
scan covering its tree updates or removes it.

Model: opus-5-5
This commit is contained in:
2026-09-23 13:51:07 +00:00
parent fff4409a24
commit 3fd8ea8990
2 changed files with 16 additions and 10 deletions
+8 -5
View File
@@ -339,11 +339,14 @@ during the hash phase:
`content` hash, has its file checked with `lstat` first. A file `content` hash, has its file checked with `lstat` first. A file
that is gone, is no longer a regular file, or has changed (a that is gone, is no longer a regular file, or has changed (a
different size, or an mtime newer than recorded) keeps its record different size, or an mtime newer than recorded) keeps its record
as it is and is not a duplicate. Any other `lstat` error is warned as it is and does not count as a match for the others. Any other
about and counted as skipped, with the same result. The files that `lstat` error is warned about and counted as skipped, with the same
pass and have no `content` hash are read only if at least two of result. If such a record has no `content` hash, it stays out of
those records pass, so a file whose only matches are stale costs no duplicate groups; if it has one, it is still reported until a scan
read; a file that already has a `content` hash is never read again. covering its own tree updates or removes it. The files that pass
and have no `content` hash are read only if at least two of those
records pass, so a file whose only matches are stale costs no read;
a file that already has a `content` hash is never read again.
They are read by a worker pool as in the hash phase, in inode order They are read by a worker pool as in the hash phase, in inode order
and once per inode, and their content hashes are committed in and once per inode, and their content hashes are committed in
batches. A failed read is warned about and counted as skipped; its batches. A failed read is warned about and counted as skipped; its
+8 -5
View File
@@ -607,11 +607,14 @@ func (s *scanState) contentPhase(ctx context.Context, workers int) error {
// their records by path. Every record contentCandidatesSQL returns has // their records by path. Every record contentCandidatesSQL returns has
// its file checked with lstat, whether or not it already has a content // its file checked with lstat, whether or not it already has a content
// hash: a file that is gone, is no longer a regular file, or has // hash: a file that is gone, is no longer a regular file, or has
// changed by the walk's rule keeps its record as it is and is not a // changed by the walk's rule keeps its record as it is and does not
// duplicate, and any other lstat error is warned about and counted as // count as a match for the others, and any other lstat error is warned
// skipped. The files of a group that pass and have no content hash are // about and counted as skipped, with the same result. If such a record
// read only if at least minGroupSize of the group's files pass, so a // has no content hash, it stays out of duplicate groups; if it has one,
// group whose other members are all stale costs no reads. Only the // it is still reported until a scan covering its own tree updates or
// removes it. The files of a group that pass and have no content hash
// are read only if at least minGroupSize of the group's files pass, so
// a group whose other members are all stale costs no reads. Only the
// records to be read are kept. // records to be read are kept.
func (s *scanState) contentCandidates( func (s *scanState) contentCandidates(
ctx context.Context, ctx context.Context,