Hash every file's head, tail and content in the hash phase (closes #132)

Every file the walk records now gets three hashes in the hash phase:
head and tail, of its first and last 64 KiB, and content. The content
hash covers the whole file under 100 MiB, and 1 MiB at each gigabyte
from 100 MiB up, so a large file is never read in full. One 100 MiB
constant replaces the 10 MiB and 50 MiB thresholds. The size census,
the records stored without hashes and the content phase are gone, so a
scan has four phases and a rescan reads only new and changed files.
report and trees no longer special-case a missing content hash. README
"Duplicate detection" and "Rationale" are rewritten.

Judgement call: info drops its two hash-count lines, which would now
always equal the record count.

Model: opus-5-5

scan.go

@@ -21,32 +21,21 @@ import (
 	"time"
 )
 
-// The duplicate ladder (see hashSignature and README "Duplicate
-// detection"). A same-size candidate below headTailMin is hashed in
-// full and compared directly; a larger one is separated first by the
-// hashes of its end windows, then by a content hash that is exact below
-// wholeFileMax and deliberately sampled at or above it. The hash phase
-// reads only the end windows of a larger file; the content phase reads
-// it for its content hash only once its size, head, and tail match
-// another file's.
-
-// headTailMin is the size threshold for the end-window gate. A file
-// smaller than this is hashed in full directly, with no separate head
-// and tail step: its head, tail, and content all carry the whole-file
-// hash. A file this size or larger is separated first by its end
-// windows.
-const headTailMin = 10 * 1024 * 1024
+// The hash phase gives every file it reads three hashes (see
+// hashSignature and README "Duplicate detection"): head and tail, of
+// its first and last headTailWindow bytes, and content. The content
+// hash covers the whole file below wholeFileMax, and only sampled
+// windows at or above it, so such a file is never read in full.
 
 // headTailWindow is the number of bytes hashed from each end of a file
-// at or above headTailMin (the head and tail rungs). Because
-// headTailMin is far larger than two windows, the head and tail windows
-// never overlap.
+// for its head and tail. In a file no larger than this, each window is
+// the whole file.
 const headTailWindow = 64 * 1024
 
-// wholeFileMax is the size boundary between the two content rungs: a
+// wholeFileMax is the size boundary between the two content hashes: a
 // file strictly smaller than this is content-hashed in full; a file
 // this size or larger is content-hashed by sampling.
-const wholeFileMax = 50 * 1024 * 1024
+const wholeFileMax = 100 * 1024 * 1024
 
 // sampleStride is the spacing between content samples for large files:
 // one window is read at each gigabyte-aligned offset (0, 1 GiB, ...).
@@ -76,13 +65,11 @@ type fileRec struct {
 }
 
 // fileMeta is the in-memory index entry for one existing database
-// record: just enough for change detection, plus whether the record
-// carries hashes. Hashes stay on disk; at tens of millions of records
-// they would dominate the scan's memory.
+// record: just enough for change detection. Hashes stay on disk; at
+// tens of millions of records they would dominate the scan's memory.
 type fileMeta struct {
-	size   int64
-	mtime  time.Time
-	hashed bool
+	size  int64
+	mtime time.Time
 }
 
 // scanUnit is what every scan phase counts, as its progress display
@@ -90,36 +77,31 @@ type fileMeta struct {
 const scanUnit = "files"
 
 // scanPhase names one phase of a scan with its place among all of them,
-// as "[2/5] walk", for the phase's progress display. The list here is
+// as "[2/4] walk", for the phase's progress display. The list here is
 // the one place the phases are counted.
 func scanPhase(name string) string {
-	return phaseLabel(
-		[]string{"load", "walk", "hash", "update", "content"}, name)
+	return phaseLabel([]string{"load", "walk", "hash", "update"}, name)
 }
 
-// runScan implements the scan subcommand: five sequential phases —
+// runScan implements the scan subcommand: four sequential phases —
 // load (which reads the existing records), walk (which stats each file
-// as it is discovered), hash, update, content — that synchronize the
-// persistent database with the filesystem state under the PATH
-// operands. Only files whose size at least one other file shares are
-// ever hashed: a size-unique file cannot be a duplicate. A file of
-// headTailMin or more gets its content hash only when its size, head,
-// and tail match another file's. Flag parsing and the
-// at-least-one-operand check are done by cobra. Before anything else,
-// the scan names the database on stderr, saying whether it is new. The
-// scan holds the lock on the database for its whole run, so a second
-// scan fails before it walks the filesystem or opens the database.
-// Errors are returned rather than exiting, so that the deferred close —
-// which takes the database out of WAL mode — always runs, and the lock
-// is released after it. When ctx is cancelled, as by the SIGINT or
-// SIGTERM that interruptContext catches, the scan keeps what it has
-// hashed (see syncScan), prints how many files its walk reached, and
-// returns errInterrupted. workers must be at least 1; the scan command
-// rejects anything less. Before the lock is taken, the scan lists the
-// filesystems that are not persistent, on which the walk records
-// nothing; unless noDefaultExcludes is set, the walk does not enter the
-// default excludes either. Once the database is open, the scan notes
-// its files, which the walk passes over.
+// as it is discovered), hash, update — that synchronize the persistent
+// database with the filesystem state under the PATH operands. Flag
+// parsing and the at-least-one-operand check are done by cobra. Before
+// anything else, the scan names the database on stderr, saying whether
+// it is new. The scan holds the lock on the database for its whole run,
+// so a second scan fails before it walks the filesystem or opens the
+// database. Errors are returned rather than exiting, so that the
+// deferred close — which takes the database out of WAL mode — always
+// runs, and the lock is released after it. When ctx is cancelled, as by
+// the SIGINT or SIGTERM that interruptContext catches, the scan keeps
+// what it has hashed (see syncScan), prints how many files its walk
+// reached, and returns errInterrupted. workers must be at least 1; the
+// scan command rejects anything less. Before the lock is taken, the
+// scan lists the filesystems that are not persistent, on which the walk
+// records nothing; unless noDefaultExcludes is set, the walk does not
+// enter the default excludes either. Once the database is open, the
+// scan notes its files, which the walk passes over.
 func runScan(ctx context.Context, roots []string, workers int,
 	oneFS, noDefaultExcludes bool,
 ) error {
@@ -358,13 +340,11 @@ type scanStats struct {
 // whatever remains after the walk and hash phases is deleted by the
 // update phase.
 type scanState struct {
-	db        *sql.DB
-	existing  map[string]fileMeta
-	sizes     []int64   // size census: every walked file, plus records outside the roots
-	toHash    []fileRec // files whose size is shared: must be read
-	sentinels []fileRec // new/changed size-unique files: recorded without hashes
-	batch     []scanRec
-	st        scanStats
+	db       *sql.DB
+	existing map[string]fileMeta
+	toHash   []fileRec // new and changed files: must be read
+	batch    []scanRec
+	st       scanStats
 }
 
 // syncScan synchronizes the database with the filesystem under roots;
@@ -391,19 +371,14 @@ func syncScan(ctx context.Context, db *sql.DB, roots []string,
 }
 
 // runPhases synchronizes the database with the filesystem under roots
-// in five sequential phases: load (read the existing records), walk
-// (enumerate and stat every file, building a complete size census),
-// hash (read only the new or changed — or previously unhashed — files
-// whose size at least one other file shares, committing results in
-// batches as they arrive), update (record the size-unique files without
-// reading them, and delete the records the scan no longer verifies),
-// and content (fill in the content hash of every record of headTailMin
-// or more whose size, head, and tail match another record's). Records
-// outside the roots are never touched, except that the content phase
-// fills in their content hash. Operands the walk cannot start from, or
-// that are not on a persistent filesystem, are dropped first, so the
-// records beneath them are treated as if those operands had not been
-// given. Below an operand, the walk follows rules.
+// in four sequential phases: load (read the existing records), walk
+// (enumerate and stat every file), hash (read only the new or changed
+// files, committing results in batches as they arrive), and update
+// (delete the records the scan no longer verifies). Records outside the
+// roots are never touched. Operands the walk cannot start from, or that
+// are not on a persistent filesystem, are dropped first, so the records
+// beneath them are treated as if those operands had not been given.
+// Below an operand, the walk follows rules.
 func (s *scanState) runPhases(ctx context.Context, roots []string,
 	workers int, oneFS bool, rules walkRules,
 ) error {
@@ -418,30 +393,22 @@ func (s *scanState) runPhases(ctx context.Context, roots []string,
 		return err
 	}
 
-	changed, unhashed := s.walkPhase(
-		startWalk(ctx, roots, oneFS, rules, workers), roots)
+	s.walkPhase(startWalk(ctx, roots, oneFS, rules, workers), roots)
 
-	// A cancelled walk stops early, so its size census covers only part
-	// of the roots, and every file it never reached would look vanished
-	// to the update phase. Stop before anything is written or deleted.
+	// A cancelled walk stops early, so every file it never reached would
+	// look vanished to the update phase. Stop before anything is written
+	// or deleted.
 	err = ctx.Err()
 	if err != nil {
 		return err
 	}
 
-	s.partition(changed, unhashed)
-
 	err = s.hashPhase(ctx, workers)
 	if err != nil {
 		return err
 	}
 
-	err = s.updatePhase(ctx)
-	if err != nil {
-		return err
-	}
-
-	return s.contentPhase(ctx, workers, rules)
+	return s.updatePhase(ctx)
 }
 
 // walkableRoots returns the operands the walk can start from: regular
@@ -483,10 +450,8 @@ func (s *scanState) walkableRoots(roots []string, rules walkRules) []string {
 }
 
 // loadIndex indexes the database records under the scan roots for
-// change detection and collects the sizes of every record outside
-// them: out-of-scope records join the size census so a scanned file
-// can be recognized as a possible duplicate of a tree scanned
-// separately into the same database.
+// change detection. It reads every record, those outside the roots
+// included, so its display counts against the number of records.
 func (s *scanState) loadIndex(ctx context.Context, roots []string) error {
 	// Indexing tens of millions of records takes real time; without a
 	// display the scan looks hung before the walk begins. Counting them
@@ -507,36 +472,24 @@ func (s *scanState) loadIndex(ctx context.Context, roots []string) error {
 	s.existing = make(map[string]fileMeta)
 
 	return loadFileMeta(ctx, s.db,
-		func(path string, size int64, mtime time.Time, hashed bool) {
+		func(path string, size int64, mtime time.Time) {
 			prog.increment()
 
 			if underAnyRoot(path, roots) {
-				s.existing[path] = fileMeta{
-					size: size, mtime: mtime, hashed: hashed,
-				}
-
-				return
+				s.existing[path] = fileMeta{size: size, mtime: mtime}
 			}
-
-			s.sizes = append(s.sizes, size)
 		})
 }
 
-// walkPhase drains the walk, appending every walked file's size to
-// the census and resolving what it can immediately: an unchanged file
-// whose record already has hashes needs nothing from the hash phase
-// (the content phase may still fill in its content hash). It returns
-// the new-or-changed files and the unchanged files whose records lack
-// hashes; both remain candidates until the census decides whether
-// their sizes are shared. A record at or below a path the walk left
-// out, because -x stopped at it, it is a default exclude, or it is not
-// on a persistent filesystem, counts as outside the roots: it is not
-// deleted, and its size joins the census. That does not hold below a
-// root at or inside that path, since that root's own walk verifies the
-// records below it; nor is a line printed for such a root.
-func (s *scanState) walkPhase(
-	events <-chan walkEvent, roots []string,
-) ([]fileRec, []fileRec) {
+// walkPhase drains the walk and queues every new or changed file for
+// the hash phase. An unchanged file needs nothing more, and its record
+// counts as verified. A record at or below a path the walk left out,
+// because -x stopped at it, it is a default exclude, or it is not on a
+// persistent filesystem, counts as outside the roots: it is not
+// deleted. That does not hold below a root at or inside that path,
+// since that root's own walk verifies the records below it; nor is a
+// line printed for such a root.
+func (s *scanState) walkPhase(events <-chan walkEvent, roots []string) {
 	// The records the last scan left under the roots estimate how many
 	// files the walk will find. A first scan has none, and no estimate.
 	estimate := int64(len(s.existing))
@@ -549,10 +502,7 @@ func (s *scanState) walkPhase(
 		estimate: true, stalledOn: "disk",
 	})
 
-	var (
-		changed, unhashed []fileRec
-		leftOut           []string
-	)
+	var leftOut []string
 
 	for ev := range events {
 		if ev.otherFSDir != "" {
@@ -582,28 +532,25 @@ func (s *scanState) walkPhase(
 			continue
 		}
 
-		s.sizes = append(s.sizes, ev.rec.size)
 		s.st.walked++
 
 		prog.increment()
 
 		old, ok := s.existing[ev.rec.path]
+		if !ok || old.size != ev.rec.size || mtimeAfter(ev.rec.mtime, old.mtime) {
+			s.toHash = append(s.toHash, ev.rec)
 
-		switch {
-		case !ok || old.size != ev.rec.size || mtimeAfter(ev.rec.mtime, old.mtime):
-			changed = append(changed, ev.rec)
-		case old.hashed:
-			delete(s.existing, ev.rec.path)
-
-			s.st.unchanged++
-		default:
-			unhashed = append(unhashed, ev.rec)
+			continue
 		}
+
+		delete(s.existing, ev.rec.path)
+
+		s.st.unchanged++
 	}
 
 	prog.finish()
 
-	for path, old := range s.existing {
+	for path := range s.existing {
 		// Kept only when a path the walk left out lies below the deepest
 		// root above the record, or is the record's own. A root at or
 		// below that path was walked on its own and did not find the
@@ -612,56 +559,8 @@ func (s *scanState) walkPhase(
 		left := deepestRoot(path, leftOut)
 		if len(left) > len(deepestRoot(path, roots)) {
 			delete(s.existing, path)
-
-			s.sizes = append(s.sizes, old.size)
 		}
 	}
-
-	return changed, unhashed
-}
-
-// partition decides each candidate's disposition now that the size
-// census is complete. A file whose size no other file shares cannot
-// be a duplicate and is never read: a new or changed one is recorded
-// without hashes, an unchanged unhashed one keeps its record. Every
-// file with a shared size queues for the hash phase.
-func (s *scanState) partition(changed, unhashed []fileRec) {
-	slices.Sort(s.sizes)
-
-	for _, rec := range changed {
-		if s.sizeShared(rec.size) {
-			s.toHash = append(s.toHash, rec)
-
-			continue
-		}
-
-		s.sentinels = append(s.sentinels, rec)
-		s.resolve(rec.path)
-	}
-
-	for _, rec := range unhashed {
-		if s.sizeShared(rec.size) {
-			s.toHash = append(s.toHash, rec)
-
-			continue
-		}
-
-		delete(s.existing, rec.path)
-
-		s.st.unchanged++
-	}
-
-	s.sizes = nil
-}
-
-// sizeShared reports whether at least two census entries have this
-// size. Every candidate's own size is in the census exactly once, so
-// a second entry means another file (or a record outside the scan
-// roots) could share its content.
-func (s *scanState) sizeShared(size int64) bool {
-	i, found := slices.BinarySearch(s.sizes, size)
-
-	return found && i+1 < len(s.sizes) && s.sizes[i+1] == size
 }
 
 // resolve counts one written record as added or updated and marks its
@@ -725,44 +624,30 @@ func isDatabaseFile(f fileRec, rules walkRules) bool {
 	})
 }
 
-// hashPhase hashes every queued file with hashSignature — the head and
-// tail of a file of headTailMin or more, the whole file below that —
-// committing completed records to the database in batches as results
-// arrive, so a long scan persists its progress as it goes (an
-// interrupted scan resumes cheaply: the next run skips everything
+// hashPhase hashes every queued file with hashSignature, one read per
+// inode run, committing completed records to the database in batches
+// as results arrive, so a long scan persists its progress as it goes
+// (an interrupted scan resumes cheaply: the next run skips everything
 // already recorded). A run that fails to hash is warned about and
-// skipped; stale records for its paths, if any, are deleted by the
-// update phase.
-func (s *scanState) hashPhase(ctx context.Context, workers int) error {
-	runs := hashRuns(s.toHash)
-	s.toHash = nil
-
-	return s.readRuns(ctx, workers, "hash", runs, hashSignature, s.recordRun)
-}
-
-// readRuns reads runs with the worker pool, one read per inode run, in
-// the order given, under the progress display of the phase named label.
-// The workers compute each run's hashes with hash, and each result goes
-// to record; a run that fails to read is warned about and counted as
-// skipped instead. The total counts actual reads, so the display shows
-// a real ETA.
+// counted as skipped; stale records for its paths, if any, are deleted
+// by the update phase. The total counts actual reads, so the display
+// shows a real ETA.
 //
 // Returning early — a failed database write, or a cancelled scan — must
 // not strand the pool: the feeder would park forever on a full jobs
 // channel and every worker on a full results channel. The deferred stop
 // is what prevents that.
-func (s *scanState) readRuns(ctx context.Context, workers int,
-	label string, runs [][]fileRec,
-	hash func(path string, size int64) (string, string, string, error),
-	record func(ctx context.Context, r hashResult) error,
-) error {
-	pool := startHashPool(ctx, runs, workers, hash)
+func (s *scanState) hashPhase(ctx context.Context, workers int) error {
+	runs := hashRuns(s.toHash)
+	s.toHash = nil
+
+	pool := startHashPool(ctx, runs, workers, hashSignature)
 	defer pool.stop()
 
 	// Between results the phase may wait on a file being read or on a
 	// batch of records being written.
 	prog := newProgress(progressParams{
-		label: scanPhase(label), unit: scanUnit, total: int64(len(runs)),
+		label: scanPhase("hash"), unit: scanUnit, total: int64(len(runs)),
 		stalledOn: "disk/db",
 	})
 	defer prog.finish()
@@ -781,12 +666,12 @@ func (s *scanState) readRuns(ctx context.Context, workers int,
 		if r.err != nil {
 			s.st.skipped += len(r.run)
 
-			prog.warnf("%s %s: %v", label, r.run[0].path, r.err)
+			prog.warnf("hash %s: %v", r.run[0].path, r.err)
 
 			continue
 		}
 
-		err := record(ctx, r)
+		err := s.recordRun(ctx, r)
 		if err != nil {
 			return err
 		}
@@ -835,10 +720,9 @@ func (s *scanState) commitFullBatch(ctx context.Context) error {
 }
 
 // updatePhase writes the scan's tail under one progress display: the
-// final partial batch of hashed records, a hash-less record for every
-// size-unique new or changed file, and deletions for every record the
-// scan did not verify (vanished files, plus paths that failed to stat
-// or hash).
+// final partial batch of hashed records, and deletions for every record
+// the scan did not verify (vanished files, plus paths that failed to
+// stat or hash).
 func (s *scanState) updatePhase(ctx context.Context) error {
 	deletes := make([]string, 0, len(s.existing))
 	for path := range s.existing {
@@ -850,7 +734,7 @@ func (s *scanState) updatePhase(ctx context.Context) error {
 
 	s.st.removed = len(deletes)
 
-	total := len(s.batch) + len(s.sentinels) + len(deletes)
+	total := len(s.batch) + len(deletes)
 	prog := newProgress(progressParams{
 		label: scanPhase("update"), unit: scanUnit, total: int64(total),
 		stalledOn: "db",
@@ -865,209 +749,9 @@ func (s *scanState) updatePhase(ctx context.Context) error {
 
 	s.batch = nil
 
-	// Sentinel records are converted in batch-sized chunks rather than
-	// materialized all at once; a first scan can have millions.
-	for chunk := range slices.Chunk(s.sentinels, updateBatchSize) {
-		recs := make([]scanRec, 0, len(chunk))
-		for _, rec := range chunk {
-			recs = append(recs, scanRec{
-				size: rec.size, mtime: rec.mtime, path: rec.path,
-			})
-		}
-
-		err = applyBatch(ctx, s.db, recs, nil, prog)
-		if err != nil {
-			return err
-		}
-	}
-
 	return applyChanges(ctx, s.db, nil, deletes, prog)
 }
 
-// contentPhase fills in the content hash of every record of headTailMin
-// or more that lacks one and whose size, head, and tail equal another
-// record's, anywhere in the database: records from this scan and
-// records stored by earlier scans, inside or outside the roots. Only
-// such a file can still be a duplicate, so no other file of headTailMin
-// or more is read beyond its end windows. The files are read with the
-// hash phase's worker pool and their records written back in batches. A
-// failed read is warned about and counted as skipped; the record keeps
-// its empty content, so it is never grouped, and a later scan tries
-// again. No file on a filesystem in rules that is not persistent is
-// read.
-func (s *scanState) contentPhase(ctx context.Context, workers int,
-	rules walkRules,
-) error {
-	toRead, recs, err := s.contentCandidates(ctx, rules)
-	if err != nil {
-		return err
-	}
-
-	err = s.readRuns(ctx, workers, "content", hashRuns(toRead),
-		hashContentOnly, func(ctx context.Context, r hashResult) error {
-			// Every path in the run keeps its record's head and tail
-			// and gains the one content hash read for the run.
-			for _, f := range r.run {
-				rec := recs[f.path]
-				rec.content = r.content
-				s.batch = append(s.batch, rec)
-			}
-
-			return s.commitFullBatch(ctx)
-		})
-	if err != nil {
-		return err
-	}
-
-	// The last partial batch is written under a display of its own, as
-	// the update phase writes the hash phase's.
-	prog := newProgress(progressParams{
-		label: scanPhase("content"), unit: scanUnit,
-		total: int64(len(s.batch)), stalledOn: "db",
-	})
-	defer prog.finish()
-
-	return applyChanges(ctx, s.db, s.batch, nil, prog)
-}
-
-// contentCandidates returns the files the content phase reads, and
-// their records by path. Every record contentCandidatesSQL returns has
-// its file checked with lstat, whether or not it already has a content
-// hash: a file that is gone, is no longer a regular file, has changed
-// by the walk's rule, or is on a filesystem in rules that is not
-// persistent keeps its record as it is and does not count as a match
-// for the others, and any other lstat error is warned about and counted
-// as skipped, with the same result. If such a record has no content
-// hash, it stays out of duplicate groups; if it has one, it is still
-// reported until a scan covering its own tree updates or removes it.
-// The files of a group that pass and have no content hash are read only
-// if at least minGroupSize of the group's files pass, so a group whose
-// other members are all stale costs no reads. Only the records to be
-// read are kept.
-func (s *scanState) contentCandidates(
-	ctx context.Context, rules walkRules,
-) ([]fileRec, map[string]scanRec, error) {
-	// The search and the checks take real time on a large database;
-	// without a display the scan looks hung before the reads begin.
-	// The display says it is searching until SQLite returns the first
-	// record. The checks start there, against a count of the records
-	// the search returns, and so do their rate and ETA. Between records
-	// they may wait on a file's lstat or on the search.
-	prog := newProgress(progressParams{
-		label: scanPhase("content"), unit: scanUnit, total: -1,
-		waitingFor: "searching the database", stalledOn: "disk/db",
-	})
-	defer prog.finish()
-
-	total, err := countContentCandidates(ctx, s.db)
-	if err != nil {
-		return nil, nil, err
-	}
-
-	var (
-		searching = true // no record has been returned yet
-		toRead    []fileRec
-		first     scanRec   // the current group's first record
-		passed    int       // the current group's files that passed the check
-		unread    []fileRec // those of them without a content hash
-	)
-
-	recs := make(map[string]scanRec)
-
-	// endGroup queues the current group's files to read if at least
-	// minGroupSize of its files passed, and drops their records if not.
-	endGroup := func() {
-		if passed >= minGroupSize {
-			toRead = append(toRead, unread...)
-		} else {
-			for _, f := range unread {
-				delete(recs, f.path)
-			}
-		}
-
-		passed, unread = 0, nil
-	}
-
-	err = loadContentCandidates(ctx, s.db, func(r scanRec, hashed bool) {
-		if searching {
-			searching = false
-
-			prog.setTotal(total)
-		}
-
-		prog.increment()
-
-		if r.size != first.size || r.head != first.head || r.tail != first.tail {
-			endGroup()
-
-			first = r
-		}
-
-		f, ok, err := unchangedFile(r, rules)
-		if err != nil {
-			s.st.skipped++
-
-			prog.warnf("content %s: %v", r.path, err)
-		}
-
-		if !ok {
-			return
-		}
-
-		passed++
-
-		if !hashed {
-			unread = append(unread, f)
-			recs[r.path] = r
-		}
-	})
-	if err != nil {
-		return nil, nil, err
-	}
-
-	// A search that returned no record still ends showing its count.
-	if searching {
-		prog.setTotal(total)
-	}
-
-	endGroup()
-
-	return toRead, recs, nil
-}
-
-// unchangedFile lstats the file r names and returns it for reading if
-// it is still the regular file r records: the same size, and an mtime
-// no newer than recorded (the walk's change rule). A file that is gone,
-// has changed, or is on a filesystem in rules that is not persistent
-// reports false; any other lstat error is returned.
-func unchangedFile(r scanRec, rules walkRules) (fileRec, bool, error) {
-	fi, err := os.Lstat(r.path)
-	if errors.Is(err, fs.ErrNotExist) {
-		return fileRec{}, false, nil
-	}
-
-	if err != nil {
-		return fileRec{}, false, err
-	}
-
-	if !fi.Mode().IsRegular() || fi.Size() != r.size ||
-		mtimeAfter(fi.ModTime(), r.mtime) {
-		return fileRec{}, false, nil
-	}
-
-	dev, ino := inodeOfInfo(fi)
-
-	// The scan reads nothing on a filesystem that is not persistent,
-	// whatever record an earlier scan left for the file.
-	if _, found := rules.nonPersistent[dev]; found {
-		return fileRec{}, false, nil
-	}
-
-	return fileRec{
-		path: r.path, size: r.size, mtime: r.mtime, dev: dev, ino: ino,
-	}, true, nil
-}
-
 // mtimeAfter reports whether mtime a is later than mtime b.
 // Not a.After(b): time.Time wraps an mtime past year 292 billion; Unix() undoes it.
 func mtimeAfter(a, b time.Time) bool {
@@ -1512,8 +1196,7 @@ func inodeOfInfo(fi fs.FileInfo) (uint64, uint64) {
 
 // hashResult carries the hashes computed for one inode run (or the
 // error that prevented computing them) from the pool's workers to the
-// phase that started the pool: head, tail, and content from
-// hashSignature, content alone from hashContentOnly.
+// hash phase.
 type hashResult struct {
 	run     []fileRec
 	head    string
@@ -1659,18 +1342,12 @@ func openRegularFile(path string) (*os.File, error) {
 const emptyHash = "e3b0c44298fc1c149afbf4c8996fb924" +
 	"27ae41e4649b934ca495991b7852b855"
 
-// hashSignature computes the hashes the hash phase records for a file
-// whose size is shared; with the file size they form its duplicate
-// signature. A file below headTailMin is hashed in full and its
-// whole-file SHA-256 is returned as head, tail, and content alike —
-// that range takes no separate end-window step. For a file at or above
-// headTailMin only the head and tail are computed, the SHA-256 of its
-// first and last headTailWindow bytes, and content is returned empty:
-// the content phase computes it with hashContentOnly once the file's
-// size, head, and tail match another file's. Two files are duplicates
-// only when all four agree; any mismatch means not a duplicate. size
-// is the value recorded when the file was statted; a zero-length file
-// has constant hashes and is never opened.
+// hashSignature computes the three hashes the hash phase records for a
+// file: head and tail (see hashEnds) and content (see hashContent).
+// With the file size they form its duplicate signature, and two files
+// are duplicates only when all four agree. size is the value recorded
+// when the file was statted; a zero-length file has constant hashes and
+// is never opened.
 func hashSignature(path string, size int64) (string, string, string, error) {
 	if size == 0 {
 		return emptyHash, emptyHash, emptyHash, nil
@@ -1683,48 +1360,25 @@ func hashSignature(path string, size int64) (string, string, string, error) {
 
 	defer func() { _ = f.Close() }()
 
-	// Below the threshold the whole file is hashed directly, with no
-	// end-window step: head and tail both carry the whole-file hash.
-	if size < int64(headTailMin) {
-		content, err := hashWhole(f, size)
-		if err != nil {
-			return "", "", "", err
-		}
-
-		return content, content, content, nil
-	}
-
 	head, tail, err := hashEnds(f, size)
 	if err != nil {
 		return "", "", "", err
 	}
 
-	return head, tail, "", nil
-}
-
-// hashContentOnly returns the content hash of the file at path, which
-// is at least headTailMin bytes: the content phase's read. head and
-// tail are returned empty, because the content phase keeps the ones its
-// records already hold.
-func hashContentOnly(path string, size int64) (string, string, string, error) {
-	f, err := openRegularFile(path)
+	content, err := hashContent(f, size)
 	if err != nil {
 		return "", "", "", err
 	}
 
-	defer func() { _ = f.Close() }()
-
-	content, err := hashContent(f, size)
-
-	return "", "", content, err
+	return head, tail, content, nil
 }
 
 // hashEnds returns the SHA-256 of the first and last headTailWindow
-// bytes of f. It is called only for files at least headTailMin, which
-// is far larger than two windows, so the windows never overlap and both
-// reads are always full.
+// bytes of f. In a file no larger than headTailWindow, each window is
+// the whole file. Reading fewer bytes than asked means the file shrank
+// between the stat and the hash, and is an error.
 func hashEnds(f *os.File, size int64) (string, string, error) {
-	buf := make([]byte, headTailWindow)
+	buf := make([]byte, min(int64(headTailWindow), size))
 
 	_, err := f.ReadAt(buf, 0)
 	if err != nil {
@@ -1734,7 +1388,7 @@ func hashEnds(f *os.File, size int64) (string, string, error) {
 	h := sha256.Sum256(buf)
 	head := hex.EncodeToString(h[:])
 
-	_, err = f.ReadAt(buf, size-int64(headTailWindow))
+	_, err = f.ReadAt(buf, size-int64(len(buf)))
 	if err != nil {
 		return "", "", err
 	}
@@ -1744,9 +1398,9 @@ func hashEnds(f *os.File, size int64) (string, string, error) {
 	return head, hex.EncodeToString(t[:]), nil
 }
 
-// hashContent returns the content-rung hash of f: the SHA-256 of the
-// whole file when it is smaller than wholeFileMax, or of sampled
-// windows when it is that size or larger.
+// hashContent returns the content hash of f: the SHA-256 of the whole
+// file when it is smaller than wholeFileMax, or of sampled windows when
+// it is that size or larger.
 func hashContent(f *os.File, size int64) (string, error) {
 	if size >= int64(wholeFileMax) {
 		return hashSamples(f, size)
@@ -1778,11 +1432,11 @@ func hashWhole(f *os.File, size int64) (string, error) {
 
 // hashSamples feeds sampleWindow bytes at each gigabyte-aligned offset
 // (0, sampleStride, 2*sampleStride, ... while inside the file), in
-// order, into one hash, each window truncated at end of file. This is
-// the probabilistic large-file rung: two files of equal size agreeing
-// on every sample are reported as duplicates without every byte being
-// read. Because size is part of the signature, files of different sizes
-// never reach this comparison, so the sample boundaries always align.
+// order, into one hash, each window truncated at end of file. Two files
+// of equal size agreeing on every sample are reported as duplicates
+// without every byte being read. Because size is part of the signature,
+// only files of one size are compared, so their sample boundaries
+// always align.
 func hashSamples(f *os.File, size int64) (string, error) {
 	h := sha256.New()
 	buf := make([]byte, sampleWindow)