Hash every file's head, tail and content in the hash phase (closes #132)
check / check (push) Canceled after 0s

Every file the walk records now gets three hashes in the hash phase:
head and tail, of its first and last 64 KiB, and content. The content
hash covers the whole file under 100 MiB, and 1 MiB at each gigabyte
from 100 MiB up, so a large file is never read in full. One 100 MiB
constant replaces the 10 MiB and 50 MiB thresholds. The size census,
the records stored without hashes and the content phase are gone, so a
scan has four phases and a rescan reads only new and changed files.
report and trees no longer special-case a missing content hash. README
"Duplicate detection" and "Rationale" are rewritten.

Judgement call: info drops its two hash-count lines, which would now
always equal the record count.

Model: opus-5-5
This commit is contained in:
2026-10-25 23:37:09 +00:00
parent f4ddb58ae1
commit 62b9cfc7ae
16 changed files with 530 additions and 1767 deletions
+2 -2
View File
@@ -85,7 +85,7 @@ func terminalWidth() int {
return width
}
// phaseLabel names phase with its place among phases, as "[2/5] walk".
// phaseLabel names phase with its place among phases, as "[2/4] walk".
func phaseLabel(phases []string, phase string) string {
return fmt.Sprintf("[%d/%d] %s",
slices.Index(phases, phase)+1, len(phases), phase)
@@ -438,7 +438,7 @@ func (p *progress) finish() {
// line formats the phase's progress as of now in at most room columns,
// as
//
// [3/5] hash: [1234/5000] 25%, 300 files/s, elapsed 4s, eta 13s
// [3/4] hash: [1234/5000] 25%, 300 files/s, elapsed 4s, eta 13s
//
// where "files" is the phase's unit, without the percent when percent
// is false, as before the library's bar, which shows its own. A phase