Hash files under 10 MiB in full, gate larger files on head/tail
check / check (push) Successful in 59s

Amends the issue 61 ladder per the owner's design change. A file under
10 MiB (new headTailMin) is now hashed in full and compared directly,
with no end-window step: its head, tail, and content all carry the
whole-file SHA-256, so its signature is decided by size and content
alone. A file at 10 MiB or above keeps the 64 KiB head/tail gate, then
the whole-file content hash below 50 MiB or gigabyte-spaced 1 MiB
samples at or above. hashEnds drops its now-unreachable single-window
branch, and hashWhole errors if the file shrank below its recorded
size (the only read for the sub-10-MiB range). README, TODO, tests,
and the schema-version note updated to match.

Model: opus-4-8
This commit is contained in:
2026-09-22 14:22:17 +00:00
parent b80c7e805e
commit dc30ad9f31
5 changed files with 203 additions and 107 deletions
+3 -2
View File
@@ -26,8 +26,9 @@ const databaseEnv = "SFDUPES_DATABASE"
// schemaVersion is the database schema version this build reads and
// writes, stored in PRAGMA user_version. Version 2 adds the content
// column and stores 65 KiB (rather than 1 KiB) end-window hashes, so a
// version 1 database is rejected and must be rescanned.
// column and the head/tail/content signature (replacing the version 1
// 1 KiB end windows), so a version 1 database is rejected and must be
// rescanned.
const schemaVersion = 2
// dbDirPerm is the mode for a database parent directory created by