Rewrite README.md sentence by sentence for a newcomer

The old README was a specification written as one dense document:
sentences past 30 words, lists chained with commas, exceptions ahead
of the rules, and release history in the user text. This rewrite
leads with what the program is, what it is for and how to use it,
then gives a reference for the settings, each subcommand, the
database, the output formats, the progress display and the exit
codes. Non-goals come before the reference; the constraints, the
entrypoints and the build come last. Every statement was checked
against the code on next and against what the built binary prints on
a small tree. The section names the code comments cite are kept.
TODO.md records the step.

Model: fable-5-1

README.md

@@ -20,7 +20,7 @@ proof.
 file takes too long. It has four subcommands. `scan` walks the filesystem and
 keeps its results between runs in an SQLite database, a database held in one
 file. A rescan reads only new and changed files, so `scan` can run every day
-from `cron`, the system's job scheduler. `report`, `trees` and `info` read the
+from cron, the system's job scheduler. `report`, `trees` and `info` read the
 database alone and can run at any time.
 
 ## Getting Started
@@ -67,9 +67,9 @@ The binary goes to `$(go env GOPATH)/bin`, or to `$GOBIN` when that is set.
 `go install`, `VERSION` is the version Go recorded in it. For `@latest` that is
 a release tag such as `v0.0.3`. For `@main` it is a pseudo-version, Go's version
 string for an untagged commit, which ends in the commit's hash. A binary built
-from a clone carries the `git` tag or commit it was built from. `VERSION` is
+from a clone carries the git tag or commit it was built from. `VERSION` is `dev`
-`dev` when no version is known. To see everything Go recorded in an installed
+only when the binary carries no version at all. To see everything Go recorded in
-binary:
+an installed binary:
 
 ```sh
 go version -m "$(command -v sfdupes)"
@@ -83,15 +83,14 @@ cd sfdupes
 make build
 ```
 
-Copy the binary to `/usr/local/bin` for the `cron` job below.
+Copy the binary to `/usr/local/bin` for the cron job below.
 
 `make docker` builds a Docker image tagged `sfdupes`. The build runs the tests
 and the linter first (see "Build"). Inside the image `sfdupes` runs as root,
-with the database at its default path. Each `-v` flag below makes a bind mount,
+with the database at its default path. Bind-mount `/var/lib/sfdupes` to keep the
-which shows a host directory at a path inside the container. Bind-mount
+database between runs. Mount the tree to scan at the same path inside the
-`/var/lib/sfdupes` to keep the database between runs. Mount the tree to scan at
+container as on the host; read-only is enough. The database then records the
-the same path inside the container as on the host; read-only is enough. The
+host's paths, and so do the reports.
-database then records the host's paths, and so do the reports.
 
 ```sh
 make docker
@@ -100,7 +99,7 @@ docker run --rm -v /srv:/srv:ro -v /var/lib/sfdupes:/var/lib/sfdupes \
 docker run --rm -v /var/lib/sfdupes:/var/lib/sfdupes sfdupes report > dupes.tsv
 ```
 
-### Daily scan from `cron`
+### Daily scan from cron
 
 Run `scan` as root, so that it can read every file. A path it cannot read is
 skipped with a warning and loses its database record. As the file
@@ -121,21 +120,18 @@ skipped with a warning and loses its database record. As the file
     - a warning for each path it skips;
     - a line for each path the walk leaves out on purpose (see "Rules for the
       walk");
-    - the summary line;
+    - the summary line.
-    - a warning when a report still has the database open as the scan ends (see
-      "Concurrent reads").
 - The log grows with every scan; rotate it like any other log.
-- Skipped paths do not fail a scan. It still exits 0, and `cron` sends no mail.
+- Skipped paths do not fail a scan. It still exits 0, and cron sends no mail.
 - A scan that fails exits 1, with the reason among the last lines of the log.
-  `tail` prints those lines, and `cron` mails them to root if the host can send
+  `tail` prints those lines, and cron mails them to root if the host can send
   mail.
 - A scan still running when the next one starts carries on. The new one fails at
-  once, and the lines `cron` mails include
+  once, and the lines cron mails include
   `sfdupes: another scan is running (lock held on /var/lib/sfdupes/db.sqlite.lock)`.
-- `report`, `trees` and `info` need only read access to the database. The umask
+- `report`, `trees` and `info` need only read access to the database. With the
-  is the setting that removes permissions from the files a process creates.
+  usual umask of `022`, the first scan creates it readable by every user, so an
-  Under the usual umask of `022`, the first scan creates the database readable
+  unprivileged user can run them against root's database.
-  by every user, so an unprivileged user can run them against root's database.
 
 ### Reading the reports
 
@@ -185,8 +181,7 @@ symlink, which `trees` does not see.
 
 Duplicate finders that hash every file in full do not scale to the target
 environment. That is about ten million files and 150 TB. The disks may be slow
-or busy. A ZFS storage pool, a set of disks managed as one filesystem, is busy
+or busy, as a ZFS storage pool is while it rebuilds a disk.
-while it rebuilds a disk.
 
 `sfdupes` reads a file under 100 MiB in full. It reads a larger file only at its
 two ends and at 1 MiB per gigabyte. A full read of a small file is cheap, and it
@@ -222,7 +217,7 @@ In order:
 3. **Scan incrementally, analyze offline.** The scan keeps a persistent
    database, and an unchanged file is never read again. `report`, `trees` and
    `info` work from the database alone and never touch the scanned filesystem.
-   `scan` is built to run from `cron`; the reports run at any time, against what
+   `scan` is built to run from cron; the reports run at any time, against what
    the database holds.
 4. **Keep the streams apart.** Everything on stdout is machine-readable data.
    Progress, warnings, summaries, help and usage text go to stderr.
@@ -253,7 +248,7 @@ The reference below is also the program's specification.
 - No byte-for-byte compare, and no deleting or linking of duplicates. The
   reports are advisory; acting on them is your job.
 - No persistence beyond the SQLite database; no export or import formats.
-- No daemon or filesystem watcher; scheduling rescans is `cron`'s job.
+- No daemon or filesystem watcher; scheduling rescans is cron's job.
 
 ## Reference
 
@@ -271,8 +266,8 @@ The reference below is also the program's specification.
   operand's filesystem, as the same flag does for `du` and `rsync`. By default
   the walk crosses filesystem boundaries.
 - `--no-default-excludes` (`scan`) lets the walk enter the default excludes: the
-  system directories the walk otherwise leaves out, such as `/var/tmp` on Linux
+  system directories the walk otherwise leaves out, such as `/proc` and `/tmp`
-  (see "Rules for the walk").
+  on Linux (see "Rules for the walk").
 - `--version`, `-v` prints `sfdupes VERSION` (see "Install") to stdout and exits
   0, writing nothing to stderr.
 - `--help`, `-h`, alone or after a subcommand, prints the help text to stderr
@@ -330,20 +325,16 @@ default. An operand may be a directory or a regular file.
 - Each operand is made absolute before the walk. A `.` or `..` in it is resolved
   as text, and symlinks are not followed. Records are keyed by that absolute
   path, so results do not depend on the working directory.
-- All operands belong to one scan, and one walk covers them all at once (see
+- All operands belong to one scan, and all are walked at once by the same
-  "Concurrency").
+  workers (see "Concurrency").
 - An operand that repeats another, or lies under another, is dropped before the
   walk, without a line, so every file is reached once. An operand under another
-  is kept only when the inner operand, or a directory between the two, is one of
+  is kept when the outer operand's walk would not reach it. That happens when
-  these:
+  the inner operand, or a directory between the two, is one of these:
     - a default exclude (see "Rules for the walk");
     - a directory on a filesystem that is not persistent: one the kernel makes
       up or keeps in memory (see "Rules for the walk");
     - with `-x`, a directory on another filesystem than the outer operand.
-
-    The outer operand's walk does not enter such a directory, so the inner
-    operand is walked on its own.
-
 - An operand that is a symlink, socket, FIFO or device node is not scanned, and
   neither is a directory named `.zfs`. `scan` prints a warning naming the path
   and what it is, counts it as skipped, and drops the operand. Another operand
@@ -368,10 +359,10 @@ operands:
   same-size rewrite counts as a change whenever the filesystem gives it a later
   mtime, even within the same second.
 - The scan deletes each record under a scanned operand whose file it could not
-  verify. That covers deleted files, and paths the scan could not stat or read
+  verify. That covers deleted files, and also paths whose details or contents
-  this run. To stat a file is to read its size, type and times without opening
+  the scan could not read this run. So the database holds only what the latest
-  it. So the database holds only what the latest scan over each path could
+  scan over each path could verify. A later successful scan adds such files
-  verify. A later successful scan adds such files again.
+  again.
 - Records outside the scanned operands are left alone, so disjoint trees can be
   scanned on different schedules into one database.
 - The scan keeps every record at or below a path that the walk leaves out on
@@ -393,20 +384,21 @@ phases.
 
 1. `load` reads the existing records. Each record under the operands is kept in
    memory by path, with its size and mtime, for the walk to compare against.
-2. `walk` lists the trees under all operands. Several workers share one queue of
+2. `walk` lists the trees under all operands. Several workers (see
-   directories to read. A worker is one thread of the program, and "Concurrency"
+   "Concurrency") share one queue of directories to read. Each worker reads one
-   says how many there are. Each worker reads one directory at a time, hands the
+   directory at a time, hands the subdirectories it finds back to the queue, and
-   subdirectories it finds back to the queue, and stats each regular file as it
+   stats each regular file as it finds it. To stat a file is to read its size,
-   finds it. The walk compares each file with its record. An unchanged file
+   type and times without opening it. The walk compares each file with its
-   needs nothing more. Every new or changed file is passed to the hash phase.
+   record. An unchanged file needs nothing more. Every new or changed file is
+   passed to the hash phase.
 3. `hash` reads every new or changed file and computes its three hashes. Files
-   are read in inode order. An inode is the filesystem's own record of a file.
+   are read in inode order. An inode is the filesystem's own record of a file,
-   Files with nearby inode numbers usually lie near each other on disk, so this
+   and files with nearby inode numbers usually lie near each other on disk, so
-   order keeps a spinning disk from seeking back and forth. Paths that are hard
+   this order keeps a spinning disk from seeking back and forth. Paths that are
-   links to one file are read once and share the result, so a backup tree made
+   hard links to one file are read once and share the result, so a backup tree
-   of hard links costs one read per file. Completed records are committed to the
+   made of hard links costs one read per file. Completed records are committed
-   database 10,000 at a time while hashing runs. A scan interrupted after hours
+   to the database 10,000 at a time while hashing runs. A scan interrupted after
-   therefore keeps everything hashed so far.
+   hours therefore keeps everything hashed so far.
 4. `update` commits the last partial batch of hashed records, then deletes the
    records the scan did not verify.
 
@@ -422,10 +414,8 @@ walk practical on large or busy storage pools.
   snapshots, and walking it would list every file once per snapshot.
 - The walk never records the scan's own database files. Those are the database,
   its `.lock` file, and the `-wal` and `-shm` files SQLite keeps beside it while
-  the scan runs (see "Concurrent reads"). It knows them by device ID and inode
+  the scan runs (see "Concurrent reads"). It knows them by device and inode, so
-  number. The device ID is the number by which the kernel tells filesystems
+  it passes over them however it reaches them, through a symlinked directory
-  apart, and the inode number tells the files of one filesystem apart. So the
-  walk passes over them however it reaches them, through a symlinked directory
   included. It prints no line for them and does not count them as skipped. A
   record an earlier scan left for one of them is deleted.
 - By default the walk crosses filesystem boundaries. With `-x`, it never enters
@@ -445,7 +435,8 @@ walk practical on large or busy storage pools.
 
     `devfs` is the macOS filesystem of `/dev`; the others are Linux names. As it
     starts, `scan` lists the mounted filesystems and notes the device ID of each
-    one of those types. On Linux the list comes from `/proc/self/mountinfo`,
+    one of those types. The device ID is the number by which the kernel tells
+    filesystems apart. On Linux the list comes from `/proc/self/mountinfo`,
     which gives each filesystem's own device ID. On macOS it comes from the
     system's mount list, and the device ID is read from the mount point. Where
     two filesystems share a mount point on macOS, the one mounted last counts.
@@ -505,13 +496,14 @@ walk practical on large or busy storage pools.
 
 #### Concurrency
 
-The walk phase and the hash phase each use a pool of workers, and each pool has
+The walk phase and the hash phase each use a pool of workers. A worker is one
-`--workers` of them. The walk works on several directories at once, and the hash
+thread of the program, and each pool has `--workers` of them. The walk works on
-phase on several files at once. On spinning disks both phases wait mostly on
+several directories at once, and the hash phase on several files at once. On
-seeks, the time the disk head takes to move. A `--workers` value well above the
+spinning disks both phases wait mostly on seeks, the time the disk head takes to
-CPU count can therefore help on storage pools with many disks. The main thread
+move. A `--workers` value well above the CPU count can therefore help on storage
-of the program owns the database writes and the progress counts. Each progress
+pools with many disks. The main thread of the program owns the database writes
-display draws from a thread of its own and never blocks the workers.
+and the progress counts. Each progress display draws from a thread of its own
+and never blocks the workers.
 
 #### What `scan` prints
 
@@ -520,12 +512,10 @@ display draws from a thread of its own and never blocks the workers.
 - one line naming the database (see "Database");
 - for each dropped operand, in operand order, the warning or the line described
   under "Operands". A repeated operand, or one under another, gets neither;
-- the progress display of each phase (see "Progress"). A warning for each path
+- the progress display of each phase (see "Progress"), with a warning for each
-  it skips, and a line for each path the walk leaves out on purpose, come as
+  path it skips and a line for each path the walk leaves out on purpose, as they
-  they happen;
+  happen;
-- the summary line;
+- the summary line.
-- a warning when a report still has the database open as the scan ends (see
-  "Concurrent reads").
 
 The summary line reports the files seen this run, by what happened to each, plus
 the skips:
@@ -538,8 +528,7 @@ scan: 123400 files seen (1200 added, 34 updated, 56 removed, 122166 unchanged),
 
 #### Stopping a scan
 
-`scan` stops cleanly on `SIGINT`, the signal Ctrl-C sends, or `SIGTERM`, the
+`scan` stops cleanly on `SIGINT` (Ctrl-C) or `SIGTERM`:
-signal `kill` sends by default:
 
 - The workers stop taking work. Each finishes at most the directory listing or
   the file it is reading, and the progress display is finished.
@@ -551,22 +540,20 @@ signal `kill` sends by default:
 - The line `scan: interrupted after N files` goes to stderr, `N` being the
   number of files the walk reached, and the exit code is 1.
 
-The next scan skips the records already written and brings the database up to
+The next scan skips the records already written and converges as usual.
-date as usual.
 
 After the first signal `scan` stops catching signals. A second one ends it at
 once, as any uncaught signal does: the records not yet committed are lost, and
 the database stays valid. `scan` catches `SIGTERM` even when it inherits it as
-ignored. A background job started from a script inherits `SIGINT` as ignored,
+ignored. A `SIGINT` that `scan` inherits as ignored, as a background job started
-and `scan` leaves it ignored.
+from a script does, stays ignored.
 
 ### `report` mode
 
-`report` reads every record from the database. It takes no positional arguments:
+`report` reads every record from the database and takes no positional arguments.
-nothing may follow `report` on the command line but flags. It never touches the
+It never touches the scanned filesystem: it does not stat, open or otherwise
-scanned filesystem: it does not stat, open or otherwise access any path in the
+access any path in the records. Its only I/O is the database, stdout and stderr.
-records. Its only I/O is the database, stdout and stderr. Its output is the same
+Its output is the same whether or not the scanned filesystem is still mounted.
-whether or not the scanned filesystem is still mounted.
 
 Processing:
 
@@ -638,8 +625,8 @@ Definitions:
 
 Processing:
 
-- `trees` builds the hierarchy, computes every directory's digest, and groups
+- Build the hierarchy, compute every directory's digest, and group the
-  the directories by digest. Every group with two or more directories is a
+  directories by digest. Every group with two or more directories is a
   duplicate-tree group.
 - Only the outermost duplicate trees are reported. A group is left out when its
   members' parents are distinct directories that all have the same digest: the
@@ -805,19 +792,17 @@ Only one `scan` runs against a database at a time.
   needs the database to itself. A `scan` that starts while a report is still
   reading waits for it up to the busy timeout, then fails. A `scan` that ends
   while a report has the database open prints a warning and leaves the database
-  in WAL mode until the next scan. `report` writes its rows while it reads. It
+  in WAL mode until the next scan. `report` writes its rows while it reads, so
-  is therefore still reading while a pager or a stalled pipe holds up its
+  it is still reading while a pager such as `less`, or a stalled pipe, holds up
-  output. A pager, such as `less`, shows output one screen at a time and waits
+  its output.
-  for a key press.
 
 #### What `scan` writes, and when
 
 - As it starts, before the walk: the directory, an empty database file and the
   `.lock` file, each only if missing.
-- Then, still before the walk:
+- Then, still before the walk: the schema, the table and index described under
-    - the `-wal` and `-shm` files, which stay beside the database while the scan
+  "Schema", in a new database, and the `-wal` and `-shm` files, which stay
-      runs;
+  beside the database while the scan runs.
-    - in a new database, the table and the index described under "Schema".
 - During the `load` and `walk` phases: no records. A first scan's database holds
   no records until the hash phase, however long its walk runs.
 - During the `hash` phase: records, committed 10,000 at a time.
@@ -1003,11 +988,10 @@ A fatal error is reported in one line on stderr that starts with `sfdupes:`. A
 failed write to stdout, such as a full disk, is reported the same way. Two cases
 never reach `sfdupes` as a failed write:
 
-- The reader of a stdout pipe may exit early, as `head` does in
+- When the reader of a stdout pipe exits early, as in `sfdupes report | head`,
-  `sfdupes report | head`. The next write then ends `sfdupes` with `SIGPIPE`,
+  the next write ends `sfdupes` with `SIGPIPE`, the signal for a write to a
-  the signal for a write to a closed pipe. It ends quietly and without a
+  closed pipe. It ends quietly and without a summary, as `cat` or `sort` would.
-  summary, as `cat` or `sort` would. The shell reports the signal, status 141 in
+  The shell reports the signal, status 141 in most shells, not exit 1.
-  most shells, not exit 1.
 - When stdout is closed outright, as in `sfdupes report >&-`, the Go runtime
   opens `/dev/null` in its place before `sfdupes` starts. The output is
   discarded, and the run succeeds, as with `> /dev/null`.
@@ -1017,29 +1001,23 @@ never reach `sfdupes` as a failed write:
 This repository follows the
 [Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
 standard. The executables in `script/` are the entrypoints of the development
-workflow, and each `Makefile` target does nothing but call its script. Every
+workflow, and the `Makefile` targets are thin shims that call them. Every script
-script is POSIX `sh`, finds the repository root itself, and can be run from any
+is POSIX `sh`, finds the repository root itself, and can be run from any
 directory.
 
 - `script/bootstrap` installs what building and developing need. It can be run
   again at any time; installed tools are kept. `git`, `make` and `go` are
   checked for presence only. A missing one is installed from the first of `nix`,
-  `apt`, `brew` or `apk` found on the host. Then come `node`, the runtime that
+  `apt`, `brew` or `apk` found on the host. An installed `node` is used when its
-  `prettier` needs, and `yarn`, the package manager that installs `prettier`:
+  major version is 22. Otherwise the script downloads `nvm`, checks the
-    - An installed `node` is used when its major version is 22. A `yarn` already
+  archive's SHA-256, and installs `node` 22.17.0 through it. It then activates
-      on `PATH` is then used as it is. When `yarn` is missing, `yarn` 1.22.22 is
+  `yarn` 1.22.22 under that `node` through `corepack`. With an installed `node`
-      activated through `corepack`, the `yarn` installer that ships with `node`,
+  22, a `yarn` already on `PATH` is used as it is. Otherwise `yarn` 1.22.22 is
-      or installed with `npm` when there is no `corepack`.
+  activated through `corepack`, or installed with `npm` when there is no
-    - Otherwise the script downloads `nvm`, a `node` version manager, and checks
+  `corepack`. `yarn install --frozen-lockfile` then installs the `prettier` that
-      the archive's SHA-256. It installs `node` 22.17.0 through `nvm`, and
+  `package.json` and `yarn.lock` pin. The linter is never installed; it runs in
-      activates `yarn` 1.22.22 under that `node` through `corepack`.
+  Docker (see `script/lint`). `docker` is not installed either, and testing,
-
+  linting and the image build need it. The script ends with `go mod download`.
-    `yarn install --frozen-lockfile` then installs the `prettier` that
-    `package.json` and `yarn.lock` pin. `prettier` is the Markdown formatter.
-    The linter is never installed; it runs in Docker (see `script/lint`).
-    `docker` is not installed either, and testing, linting and the image build
-    need it. The script ends with `go mod download`.
-
 - `script/setup` makes a fresh clone ready for development: `script/bootstrap`,
   then `script/install-precommit`.
 - `script/projectname` prints `sfdupes`. Scripts that need the name call it, so
@@ -1086,7 +1064,7 @@ directory.
   image as `script/docker` does. The Gitea workflow runs it on every push, and a
   successful run means every check passed. The tests and the linter run twice:
   once in `script/check`, and again as stages of the image build.
-- `script/precommit` runs from the `git` pre-commit hook. `go mod tidy` must
+- `script/precommit` runs from the git pre-commit hook. `go mod tidy` must
   change nothing, or the commit fails; then `script/check` runs.
 - `script/install-precommit` installs `.git/hooks/pre-commit`, which runs
   `script/precommit`.
@@ -1105,9 +1083,8 @@ unchanged package is not linted again.
 
 ## Build
 
-The `script/` entrypoints above hold the implementations. Each `Makefile` target
+The `script/` entrypoints above hold the implementations. The `Makefile` targets
-does nothing but call its script, except `build` and `clean`, which carry their
+are shims onto them, except `build`, which carries the compile recipe:
-own recipes:
 
 - `make` or `make build` builds the `sfdupes` binary with cgo disabled. Building
   is the default target.

TODO.md

@@ -33,8 +33,7 @@
   it. It leads with what the program is and how to use it. A reference section
   covers the settings, the database, the output formats and the exit codes;
   development and testing come last. Every statement was checked against the
-  code on `next`. Three comments in `main_test.go` cite "Settings", where their
+  code on `next` (2026-10-26, https://git.eeqj.de/sneak/sfdupes/issues/135)
-  rules now live (2026-10-26, https://git.eeqj.de/sneak/sfdupes/issues/135)
 
 - a file or directory removed while the scan runs is not an error, in the walk
   or in the hash phase. The scan prints no line for it and does not count it as

main_test.go

@@ -648,7 +648,7 @@ func TestRunUsageErrors(t *testing.T) {
 }
 
 func TestRunScanRejectsWorkersBelowOne(t *testing.T) {
-	// README §Settings: --workers below 1 is a usage error reported in
+	// README §scan mode: --workers below 1 is a usage error reported in
 	// one line on stderr, before the scan opens the database.
 	for _, workers := range []string{"0", "-1"} {
 		t.Run(workers, func(t *testing.T) {
@@ -686,8 +686,8 @@ func TestRunScanRejectsWorkersBelowOne(t *testing.T) {
 func TestRunHelp(t *testing.T) {
 	t.Parallel()
 
-	// README §Settings: help goes to stderr, exits 0, and leaves stdout
+	// README §Subcommands: help goes to stderr, exits 0, and leaves
-	// empty.
+	// stdout empty.
 	cases := [][]string{{"--help"}, {"-h"}, {cmdScan, "--help"}}
 
 	for _, args := range cases {
@@ -713,8 +713,8 @@ func TestRunHelp(t *testing.T) {
 func TestRunVersion(t *testing.T) {
 	t.Parallel()
 
-	// README §Settings: the version is one line on stdout, with nothing
+	// README §Subcommands: the version is one line on stdout, with
-	// on stderr, and exits 0.
+	// nothing on stderr, and exits 0.
 	info, _ := debug.ReadBuildInfo()
 
 	for _, arg := range []string{"--version", "-v"} {