Compute the content hash only when head and tail match (closes #61)
check / check (push) Successful in 1m9s

A file of 10 MiB or more now gets only its head and tail in the hash
phase. A new content phase, after the update phase, finds every record
of that size without a content hash whose size, head and tail match
another record's, anywhere in the database, checks each file with
lstat, and reads a group only while at least two members remain. It
reuses the hash worker pool, now given its hash function. report and
trees leave out records without a content hash. The README, help text
and TODO entry describe the gate; the schema stays at version 1.

Model: opus-5-5
This commit is contained in:
2026-09-23 12:18:39 +00:00
parent 09a39ddf37
commit 89fc9e4595
12 changed files with 763 additions and 221 deletions
+8 -6
View File
@@ -127,12 +127,14 @@ func buildHierarchy(recs []scanRec) (*treeNode, []*treeNode) {
size: r.size, head: r.head, tail: r.tail, content: r.content,
}
// An unhashed record (its size was unique when last scanned)
// has unknown content: give it a signature no other file can
// share, so trees containing it never compare equal. Real
// heads are hex, so the NUL-prefixed form cannot collide.
if sig.head == "" {
sig.head = "unhashed\x00" + r.path
// A record without a content hash (its size was unique when
// last scanned, or it is headTailMin or more and has not yet
// matched another record on size, head, and tail) has unknown
// content: give it a signature no other file can share, so
// trees containing it never compare equal. Real hashes are
// hex, so the NUL-prefixed form cannot collide.
if sig.content == "" {
sig.content = "unhashed\x00" + r.path
}
node.files[comps[len(comps)-1]] = sig