Hash files under 10 MiB in full, gate larger files on head/tail

Amends the issue 61 ladder per the owner's design change. A file under
10 MiB (new headTailMin) is now hashed in full and compared directly,
with no end-window step: its head, tail, and content all carry the
whole-file SHA-256, so its signature is decided by size and content
alone. A file at 10 MiB or above keeps the 64 KiB head/tail gate, then
the whole-file content hash below 50 MiB or gigabyte-spaced 1 MiB
samples at or above. hashEnds drops its now-unreachable single-window
branch, and hashWhole errors if the file shrank below its recorded
size (the only read for the sub-10-MiB range). README, TODO, tests,
and the schema-version note updated to match.

Model: opus-4-8

README.md

@@ -7,12 +7,14 @@
 duplicate files — and, ultimately, entire duplicate directory trees —
 across very large filesystems without reading every byte of every file.
 Files are considered duplicates when their sizes are equal and they
-agree on a short ladder of hashes: the SHA-256 of their first 64 KiB and
-of their last 64 KiB, and then a content hash — the SHA-256 of the whole
-file when it is under 50 MiB, or of gigabyte-spaced 1 MiB samples when it
-is 50 MiB or larger. Below 50 MiB this is proof of identical content; at
-or above 50 MiB it is a strong candidate signal rather than proof,
-because the gaps between samples are never read. The intended use is
+agree on a short ladder of hashes. A file under 10 MiB is hashed in full
+and compared directly. A larger file is gated first on the SHA-256 of
+its first 64 KiB and of its last 64 KiB, and then compared on a content
+hash — the SHA-256 of the whole file when it is under 50 MiB, or of
+gigabyte-spaced 1 MiB samples when it is 50 MiB or larger. Below 50 MiB
+the content hash is proof of identical content; at or above 50 MiB it is
+a strong candidate signal rather than proof, because the gaps between
+samples are never read. The intended use is
 finding duplicate downloads and duplicated directory trees on
 multi-terabyte ZFS servers where reading every byte of every file is
 prohibitively expensive. `scan` maintains a persistent SQLite database of file
@@ -78,7 +80,8 @@ Goals, in order:
 2.  **Spend I/O in proportion to duplicate likelihood.** Only files
     whose size at least one other file shares are read at all — a
     size-unique file cannot be a duplicate. Those are compared by the
-    ladder in "Duplicate detection" below: cheap 64 KiB end windows
+    ladder in "Duplicate detection" below: a file under 10 MiB is hashed
+    in full, while a larger file is gated on cheap 64 KiB end windows
     first, then a content hash that reads the whole file below 50 MiB
     but only gigabyte-spaced 1 MiB samples at or above it, so the very
     largest files are still never read in full. Scale target: tens of
@@ -153,28 +156,31 @@ All three subcommands operate on a single SQLite database file:
     toward the filesystem.
 -   Schema (`PRAGMA user_version` is the schema version, currently 2; a
     database with any other version is a fatal error). Version 2 added
-    the `content` column and widened the end windows from 1 KiB to
-    64 KiB, so a version 1 database cannot be reused: it is rejected and
-    the tree must be rescanned from scratch.
+    the `content` column and the 64 KiB head/tail signature (replacing
+    the version 1 1 KiB end windows), so a version 1 database cannot be
+    reused: it is rejected and the tree must be rescanned from scratch.
 
     ```sql
     CREATE TABLE files (
         path    BLOB PRIMARY KEY,  -- absolute path, raw bytes
         size    INTEGER NOT NULL,  -- bytes, from lstat
         mtime   INTEGER NOT NULL,  -- Unix seconds, from lstat
-        head    TEXT NOT NULL,     -- lowercase-hex SHA-256, first 64 KiB
-        tail    TEXT NOT NULL,     -- lowercase-hex SHA-256, last 64 KiB
+        head    TEXT NOT NULL,     -- lowercase-hex SHA-256; first 64 KiB, or whole file under 10 MiB
+        tail    TEXT NOT NULL,     -- lowercase-hex SHA-256; last 64 KiB, or whole file under 10 MiB
         content TEXT NOT NULL      -- lowercase-hex SHA-256, whole file or samples
     ) WITHOUT ROWID;
     ```
 
     Paths are stored as BLOBs because Unix paths are raw bytes, not
     guaranteed UTF-8. `mtime` is used only for change detection; it is
-    not part of the duplicate key. `head`, `tail`, and `content` are
-    empty strings when the file has never been hashed because its size
-    was unique as of the last scan that covered it; such records still
-    define the file for tree reconstruction but never participate in
-    duplicate groups.
+    not part of the duplicate key. For a file under 10 MiB `head`, `tail`,
+    and `content` all hold the whole-file hash (that range is hashed in
+    full, with no end windows); for a larger file `head` and `tail` hold
+    the first- and last-64 KiB hashes and `content` the whole-file or
+    sampled hash. All three are empty strings when the file has never
+    been hashed because its size was unique as of the last scan that
+    covered it; such records still define the file for tree
+    reconstruction but never participate in duplicate groups.
 
 ### Duplicate detection
 
@@ -187,27 +193,33 @@ the database, even across separate scans.
 
 1.  **Size.** Files of different sizes are never compared. Only files
     whose size at least one other file shares are hashed at all.
-2.  **Head and tail.** The SHA-256 of the first 64 KiB (`head`) and of
-    the last 64 KiB (`tail`). When a file is 64 KiB or smaller the two
-    windows are the whole file and coincide, so `head` and `tail` are
-    equal and only one read is issued; when it is between one and two
-    windows the two windows overlap, which is harmless. These reads are
-    cheap and eliminate most same-size pairs before any bulk reading.
-3.  **Content, below 50 MiB.** The SHA-256 of the entire file. Agreement
-    here is proof of identical content (barring a SHA-256 collision).
-4.  **Content, 50 MiB and above.** A sampled SHA-256: the 1 MiB window
-    at each gigabyte-aligned offset (0, 1 GiB, 2 GiB, … while inside the
-    file, the final window truncated at end of file) is fed, in order,
-    into one hash. This is **deliberately probabilistic** — the gaps
-    between samples are never read, so two large files that agree on
-    every sample are reported as duplicates without being read in full.
-    It is the price of never reading a 150 GB file end to end. Because
-    size is already part of the signature, only equal-size files reach
-    this rung, so their sample boundaries always align.
+2.  **Under 10 MiB: whole file.** A file smaller than 10 MiB is hashed
+    in full and compared directly, with no separate end-window step —
+    small files are cheap to read to the last byte, and doing so makes
+    the comparison exact. `head`, `tail`, and `content` all hold this
+    whole-file SHA-256, so such a file's signature is decided entirely
+    by its size and its content.
+3.  **10 MiB and above: head and tail.** For a larger file, the SHA-256
+    of the first 64 KiB (`head`) and of the last 64 KiB (`tail`) are a
+    cheap gate that eliminates most same-size pairs before any bulk
+    reading. At 10 MiB and above the two windows never overlap.
+4.  **10 MiB and above, content below 50 MiB.** The SHA-256 of the
+    entire file. Agreement here is proof of identical content (barring a
+    SHA-256 collision).
+5.  **10 MiB and above, content 50 MiB and above.** A sampled SHA-256:
+    the 1 MiB window at each gigabyte-aligned offset (0, 1 GiB, 2 GiB, …
+    while inside the file, the final window truncated at end of file) is
+    fed, in order, into one hash. This is **deliberately probabilistic**
+    — the gaps between samples are never read, so two large files that
+    agree on every sample are reported as duplicates without being read
+    in full. It is the price of never reading a 150 GB file end to end.
+    Because size is already part of the signature, only equal-size files
+    reach this rung, so their sample boundaries always align.
 
-`head`, `tail`, and `content` are one column each; a file below 50 MiB
-and a file at or above it never share a size, so a `content` value is
-never ambiguous between the whole-file and sampled forms.
+`head`, `tail`, and `content` are one column each. A file below 10 MiB
+and one at or above it never share a size, and neither do a file below
+50 MiB and one at or above it, so a stored value is never ambiguous
+between the whole-file, end-window, and sampled forms.
 
 ### `scan` mode