Compute the content hash only when head and tail match (closes #61)
check / check (push) Successful in 1m9s
check / check (push) Successful in 1m9s
A file of 10 MiB or more now gets only its head and tail in the hash phase. A new content phase, after the update phase, finds every record of that size without a content hash whose size, head and tail match another record's, anywhere in the database, checks each file with lstat, and reads a group only while at least two members remain. It reuses the hash worker pool, now given its hash function. report and trees leave out records without a content hash. The README, help text and TODO entry describe the gate; the schema stays at version 1. Model: opus-5-5
This commit is contained in:
@@ -127,12 +127,14 @@ func buildHierarchy(recs []scanRec) (*treeNode, []*treeNode) {
|
||||
size: r.size, head: r.head, tail: r.tail, content: r.content,
|
||||
}
|
||||
|
||||
// An unhashed record (its size was unique when last scanned)
|
||||
// has unknown content: give it a signature no other file can
|
||||
// share, so trees containing it never compare equal. Real
|
||||
// heads are hex, so the NUL-prefixed form cannot collide.
|
||||
if sig.head == "" {
|
||||
sig.head = "unhashed\x00" + r.path
|
||||
// A record without a content hash (its size was unique when
|
||||
// last scanned, or it is headTailMin or more and has not yet
|
||||
// matched another record on size, head, and tail) has unknown
|
||||
// content: give it a signature no other file can share, so
|
||||
// trees containing it never compare equal. Real hashes are
|
||||
// hex, so the NUL-prefixed form cannot collide.
|
||||
if sig.content == "" {
|
||||
sig.content = "unhashed\x00" + r.path
|
||||
}
|
||||
|
||||
node.files[comps[len(comps)-1]] = sig
|
||||
|
||||
Reference in New Issue
Block a user