The old README was one dense specification: long sentences, lists
chained with commas, exceptions ahead of the rules, and release
history in the user text. This rewrite leads with what the program is
and how to use it, then gives a reference for the settings, each
subcommand, the database, the output formats, the progress display
and the exit codes. Non-goals come before the reference; the
constraints, the entrypoints and the build come last. Every statement
was checked against the code on next and against the built binary on
small trees. The section names the code comments cite are kept. Five
comments in main_test.go cite Settings, and one each in scan_test.go,
trees_test.go and exclude_test.go cites what the README now says.
TODO.md records the step.
Model: fable-5-1
Every file the walk records now gets three hashes: head and tail, of
its first and last 64 KiB, and content. The content hash covers the
whole file under 100 MiB, and 1 MiB at each gigabyte from 100 MiB up,
so a large file is never read in full. One 100 MiB constant replaces
the 10 MiB and 50 MiB thresholds. The size census, the records stored
without hashes and the content phase are gone, so a rescan reads only
new and changed files. report and trees no longer special-case a
missing content hash. README "Duplicate detection" and "Rationale" are
rewritten.
Judgement call: info drops its two hash-count lines, which now always
equal the record count.
Model: opus-5-5
scan never records or reads anything on a filesystem that is not
persistent (tmpfs, proc, sysfs and the like), however it reaches it; no
flag turns this off. It notes each such filesystem's device ID as it
starts: on Linux from /proc/self/mountinfo, on macOS by lstat of each
listed mount point. Below an operand the walk also skips the default
excludes, a path list per system (Linux, macOS, and Windows, not used
yet); --no-default-excludes turns off only those. A default exclude that
is itself an operand gets no skip line. The tests stop at once, saying
why, when TMPDIR is not on a persistent filesystem.
Judgement call: an unreadable mount list fails the scan, and no flag gets round it.
Judgement call: a file on such a filesystem below an operand gets the same line as a directory.
Judgement call: such an operand that another operand's walk also reaches is named twice.
Unverified: whether macOS's mount list gives the device lstat reports, so macOS keeps lstat, and a hidden mount there can still add another filesystem's device.
Unverified: the macOS code is never built or run here; bind mounts need root to test.
Model: opus-5-5