Rewrite README.md sentence by sentence for a newcomer

The old README was a specification written as one dense document:
sentences past 30 words, lists chained with commas, exceptions ahead
of the rules, and release history in the user text. This rewrite
leads with what the program is, what it is for and how to use it,
then gives a reference for the settings, each subcommand, the
database, the output formats, the progress display and the exit
codes. Non-goals come before the reference; the constraints, the
entrypoints and the build come last. Every statement was checked
against the code on next and against what the built binary prints on
a small tree. The section names the code comments cite are kept.
TODO.md records the step.

Model: fable-5-1

README.md

@@ -17,10 +17,11 @@ never read in full. A match between two large files is a strong candidate, not
 proof.
 
 `sfdupes` is meant for multi-terabyte servers, where reading every byte of every
-file takes too long. `scan` keeps its results between runs in an SQLite
+file takes too long. It has four subcommands. `scan` walks the filesystem and
-database, a database held in one file. A rescan reads only new and changed
+keeps its results between runs in an SQLite database, a database held in one
-files, so `scan` can run from cron every day. The reports read the database
+file. A rescan reads only new and changed files, so `scan` can run every day
-alone and can run at any time.
+from cron, the system's job scheduler. `report`, `trees` and `info` read the
+database alone and can run at any time.
 
 ## Getting Started
 
@@ -33,15 +34,18 @@ export SFDUPES_DATABASE="$HOME/.local/share/sfdupes/db.sqlite"
 ./sfdupes info
 ```
 
-`scan` walks the trees you name and keeps one database record per regular file.
+`scan` walks the trees named by its operands, the paths given after `scan` on
-`report` prints the file-level duplicates. `trees` prints the duplicate
+the command line. It keeps one database record per regular file: an ordinary
-directory trees. `info` prints the database's path, size and record count.
+file, as opposed to a directory, a symlink or a device. `report` prints the
+file-level duplicates. `trees` prints the duplicate directory trees. `info`
+prints the database's path, size and record count.
 
 The database is `/var/lib/sfdupes/db.sqlite` unless `SFDUPES_DATABASE` names
 another file. The first `scan` creates it.
 
 Every command writes its data to stdout and everything else to stderr: progress,
-warnings, summaries and help text.
+warnings, summaries and help text. stdout and stderr are a program's two output
+streams. The shell shows both on the terminal, and `>` redirects stdout alone.
 
 ### Install
 
@@ -60,11 +64,12 @@ go install sneak.berlin/go/sfdupes@main
 The binary goes to `$(go env GOPATH)/bin`, or to `$GOBIN` when that is set.
 
 `sfdupes --version` prints `sfdupes VERSION` and exits. For a binary from
-`go install`, `VERSION` is the version Go recorded in it: a release tag such as
+`go install`, `VERSION` is the version Go recorded in it. For `@latest` that is
-`v0.0.3`, or for `@main` a pseudo-version ending in the commit's hash. A binary
+a release tag such as `v0.0.3`. For `@main` it is a pseudo-version, Go's version
-built from a clone carries the git tag or commit it was built from. `VERSION` is
+string for an untagged commit, which ends in the commit's hash. A binary built
-`dev` only when the binary carries no version at all. To see everything Go
+from a clone carries the git tag or commit it was built from. `VERSION` is `dev`
-recorded in an installed binary:
+only when the binary carries no version at all. To see everything Go recorded in
+an installed binary:
 
 ```sh
 go version -m "$(command -v sfdupes)"
@@ -142,8 +147,8 @@ A row is a candidate, not proof:
   changed or gone since.
 - A file of 100 MiB or more is compared only on samples of its content (see
   "Duplicate detection").
-- Two paths that are hard links to one file are listed as duplicates. They share
+- Two paths that are hard links, two names for one file, are listed as
-  their data, so removing one frees nothing.
+  duplicates. They share their data, so removing one frees nothing.
 
 Compare a pair byte for byte before removing either copy. For the row
 `/srv/a/big.iso`, `/srv/b/big-copy.iso`, `4294967296`:
@@ -176,7 +181,7 @@ symlink, which `trees` does not see.
 
 Duplicate finders that hash every file in full do not scale to the target
 environment. That is about ten million files and 150 TB. The disks may be slow
-or busy, as a ZFS pool is while it rebuilds a disk.
+or busy, as a ZFS storage pool is while it rebuilds a disk.
 
 `sfdupes` reads a file under 100 MiB in full. It reads a larger file only at its
 two ends and at 1 MiB per gigabyte. A full read of a small file is cheap, and it
@@ -212,25 +217,11 @@ In order:
 3. **Scan incrementally, analyze offline.** The scan keeps a persistent
    database, and an unchanged file is never read again. `report`, `trees` and
    `info` work from the database alone and never touch the scanned filesystem.
-   `scan` is built to run from cron; the reports run at any time against the
+   `scan` is built to run from cron; the reports run at any time, against what
-   last completed scan.
+   the database holds.
 4. **Keep the streams apart.** Everything on stdout is machine-readable data.
    Progress, warnings, summaries, help and usage text go to stderr.
 
-### Constraints
-
-- Language: Go, module `sneak.berlin/go/sfdupes`, binary `sfdupes`.
-- Dependencies beyond the standard library:
-    - `github.com/spf13/cobra` for the command line;
-    - `github.com/schollz/progressbar/v3` for the progress display;
-    - `golang.org/x/term`, to tell whether stderr is a terminal;
-    - `modernc.org/sqlite`, the SQLite driver;
-    - `golang.org/x/sys`, for the scan lock and the list of mounted filesystems.
-- `modernc.org/sqlite` is pure Go, so no build needs cgo, Go's bridge to C code.
-  The `Makefile` exports `CGO_ENABLED=0`.
-- `report` and `trees` are deterministic: the same database contents give the
-  same output, whatever order the records were inserted in.
-
 ### Subcommands
 
 - `scan` walks the filesystem and synchronizes the database: one record per
@@ -252,6 +243,13 @@ A missing or unknown subcommand prints the usage text on stderr and exits 2.
 
 The reference below is also the program's specification.
 
+## Non-goals
+
+- No byte-for-byte compare, and no deleting or linking of duplicates. The
+  reports are advisory; acting on them is your job.
+- No persistence beyond the SQLite database; no export or import formats.
+- No daemon or filesystem watcher; scheduling rescans is cron's job.
+
 ## Reference
 
 ### Settings
@@ -260,15 +258,16 @@ The reference below is also the program's specification.
   database is `/var/lib/sfdupes/db.sqlite`. There is no command-line flag for
   it. The path names the file exactly, whatever characters it holds, `?`, `#`
   and `%` included. A relative path is relative to the working directory.
-- `--workers N` (`scan`) sets the number of workers in the walk and hash phases.
+- `--workers N` (`scan`) sets how many directories the walk reads at once, and
-  The default is the number of CPUs. `N` must be at least 1; a smaller value is
+  how many files the hash phase reads at once. The default is the number of
-  a usage error, reported in one line on stderr with exit 2 before anything is
+  CPUs. `N` must be at least 1; a smaller value is a usage error, reported in
-  scanned.
+  one line on stderr with exit 2 before anything is scanned.
 - `-x`, `--one-file-system` (`scan`) keeps the walk of each operand on that
   operand's filesystem, as the same flag does for `du` and `rsync`. By default
   the walk crosses filesystem boundaries.
-- `--no-default-excludes` (`scan`) lets the walk enter the default excludes (see
+- `--no-default-excludes` (`scan`) lets the walk enter the default excludes: the
-  "Rules for the walk").
+  system directories the walk otherwise leaves out, such as `/proc` and `/tmp`
+  on Linux (see "Rules for the walk").
 - `--version`, `-v` prints `sfdupes VERSION` (see "Install") to stdout and exits
   0, writing nothing to stderr.
 - `--help`, `-h`, alone or after a subcommand, prints the help text to stderr
@@ -278,8 +277,9 @@ The reference below is also the program's specification.
 ### Duplicate detection
 
 Two files are duplicates when they have the same size and the same three hashes.
-`scan` computes the hashes and stores them in the database. `report` and `trees`
+A file's size and its three hashes together are its signature. `scan` computes
-group the files whose size and hashes all match, whichever scan recorded them.
+the hashes and stores them in the database. `report` and `trees` group the files
+with the same signature, whichever scan recorded them.
 
 Every record holds three SHA-256 hashes:
 
@@ -287,10 +287,6 @@ Every record holds three SHA-256 hashes:
 - `tail`, the hash of the file's last 64 KiB;
 - `content`, the content hash described below.
 
-In a file of 64 KiB or less, each window is the whole file, so all three hashes
-are the hash of the whole file. An empty file has fixed hashes and is never
-opened.
-
 The content hash depends on the file's size:
 
 - Under 100 MiB, it is the SHA-256 of the whole file. A match proves identical
@@ -300,10 +296,14 @@ The content hash depends on the file's size:
   samples go into one hash in order. A sample that would run past the end of the
   file stops there.
 
-The sampled hash is probabilistic by design. Bytes outside the samples and
+In a file of 64 KiB or less, each 64 KiB stretch is the whole file, so all three
-outside the first and last 64 KiB are never read. Two large files that differ
+hashes are the hash of the whole file. An empty file has fixed hashes and is
-only in such bytes are reported as duplicates. That is the price of never
+never opened.
-reading a 150 GiB file end to end.
+
+The sampled hash is not exact. Bytes outside the samples and outside the first
+and last 64 KiB are never read. Two large files that differ only in such bytes
+are reported as duplicates. That is the price of never reading a 150 GiB file
+end to end.
 
 Only files of the same size are compared, so their samples lie at the same
 offsets. A file under 100 MiB and one of 100 MiB or more never share a size, so
@@ -322,25 +322,27 @@ default. An operand may be a directory or a regular file.
 - `scan` with no operand is a usage error: the usage text goes to stderr, and
   the exit code is 2.
 - An operand that does not exist is a fatal error, exit 1.
-- Each operand is made absolute and cleaned before the walk, without resolving
+- Each operand is made absolute before the walk. A `.` or `..` in it is resolved
-  symlinks. Records are keyed by that absolute path, so results do not depend on
+  as text, and symlinks are not followed. Records are keyed by that absolute
-  the working directory.
+  path, so results do not depend on the working directory.
-- All operands belong to one scan, and every operand seeds the same pool of walk
+- All operands belong to one scan, and all are walked at once by the same
-  workers.
+  workers (see "Concurrency").
 - An operand that repeats another, or lies under another, is dropped before the
-  walk, so every file is reached once. An operand under another is kept when the
+  walk, without a line, so every file is reached once. An operand under another
-  outer operand's walk would not reach it. That happens when the inner operand,
+  is kept when the outer operand's walk would not reach it. That happens when
-  or a directory between the two, is one of these:
+  the inner operand, or a directory between the two, is one of these:
-    - a default exclude;
+    - a default exclude (see "Rules for the walk");
-    - a directory that is not on a persistent filesystem;
+    - a directory on a filesystem that is not persistent: one the kernel makes
+      up or keeps in memory (see "Rules for the walk");
     - with `-x`, a directory on another filesystem than the outer operand.
 - An operand that is a symlink, socket, FIFO or device node is not scanned, and
   neither is a directory named `.zfs`. `scan` prints a warning naming the path
   and what it is, counts it as skipped, and drops the operand. Another operand
-  beneath it is still scanned. The records beneath a dropped operand count as
+  beneath it is still scanned. The scan keeps the records beneath a dropped
-  outside the scanned operands, unless another operand covers them.
+  operand, unless another operand covers them.
-- An operand that is not on a persistent filesystem is dropped the same way,
+- An operand on a filesystem that is not persistent is dropped too. `scan`
-  with a line that is not a warning, and is not counted as skipped.
+  prints a line naming it and its filesystem type, and does not count it as
+  skipped.
 - A scan whose every operand is dropped walks nothing and exits 0.
 
 #### What a scan does to the database
@@ -356,19 +358,24 @@ operands:
   is compared to the nanosecond, as finely as the filesystem records it. A
   same-size rewrite counts as a change whenever the filesystem gives it a later
   mtime, even within the same second.
-- A record under a scanned operand whose file the scan did not verify is
+- The scan deletes each record under a scanned operand whose file it could not
-  deleted. That covers deleted files, and also paths that failed to stat or hash
+  verify. That covers deleted files, and also paths whose details or contents
-  this run. The database only ever holds signatures that the latest scan
+  the scan could not read this run. So the database holds only what the latest
-  covering them could verify. A later successful scan adds such files again.
+  scan over each path could verify. A later successful scan adds such files
+  again.
 - Records outside the scanned operands are left alone, so disjoint trees can be
   scanned on different schedules into one database.
-- A record at or below a path the walk left out on purpose counts as outside the
+- The scan keeps every record at or below a path that the walk leaves out on
-  scanned operands, so it is kept. The walk leaves out a default exclude, a path
+  purpose. Such records count as outside the scanned operands. The walk leaves
-  that is not on a persistent filesystem, and, with `-x`, a directory on another
+  out:
-  filesystem than its operand. A filesystem mounted under an operand can
+    - a default exclude;
-  therefore be scanned on its own schedule into the same database. Records below
+    - a path that is not on a persistent filesystem;
-  an operand that is itself at or inside such a path are verified by that
+    - with `-x`, a directory on another filesystem than its operand.
-  operand's own walk, so they are deleted when their files are gone.
+
+    A filesystem mounted under an operand can therefore be scanned on its own
+    schedule into the same database. An operand at or inside such a path is
+    walked on its own. Its walk verifies the records below it, so they are
+    deleted when their files are gone.
 
 #### Phases
 
@@ -377,23 +384,27 @@ phases.
 
 1. `load` reads the existing records. Each record under the operands is kept in
    memory by path, with its size and mtime, for the walk to compare against.
-2. `walk` lists the trees under all operands with the walk worker pool. Each
+2. `walk` lists the trees under all operands. Several workers (see
-   worker reads one directory at a time, hands subdirectories back to the queue,
+   "Concurrency") share one queue of directories to read. Each worker reads one
-   and stats each regular file as it finds it. The walk compares each file with
+   directory at a time, hands the subdirectories it finds back to the queue, and
-   its record. An unchanged file needs nothing more. Every new or changed file
+   stats each regular file as it finds it. To stat a file is to read its size,
-   is passed to the hash phase.
+   type and times without opening it. The walk compares each file with its
+   record. An unchanged file needs nothing more. Every new or changed file is
+   passed to the hash phase.
 3. `hash` reads every new or changed file and computes its three hashes. Files
-   are hashed in inode order, which keeps a spinning disk from seeking back and
+   are read in inode order. An inode is the filesystem's own record of a file,
-   forth. Paths that are hard links to one file are read once and share the
+   and files with nearby inode numbers usually lie near each other on disk, so
-   result, so a farm of hard-linked backups costs one read per file. Completed
+   this order keeps a spinning disk from seeking back and forth. Paths that are
-   records are committed to the database 10,000 at a time while hashing runs. A
+   hard links to one file are read once and share the result, so a backup tree
-   scan interrupted after hours therefore keeps everything hashed so far.
+   made of hard links costs one read per file. Completed records are committed
+   to the database 10,000 at a time while hashing runs. A scan interrupted after
+   hours therefore keeps everything hashed so far.
 4. `update` commits the last partial batch of hashed records, then deletes the
    records the scan did not verify.
 
 Reading a tree one directory at a time waits on the disk at every step and takes
 hours at millions of files. Reading many directories at once is what makes the
-walk practical on large or busy pools.
+walk practical on large or busy storage pools.
 
 #### Rules for the walk
 
@@ -401,8 +412,9 @@ walk practical on large or busy pools.
   passed over, and symlinks are never followed.
 - The walk never enters a directory named `.zfs`. Such a directory holds ZFS
   snapshots, and walking it would list every file once per snapshot.
-- The walk never records the scan's own database files: the database, its `-wal`
+- The walk never records the scan's own database files. Those are the database,
-  and `-shm` files and its `.lock` file. It knows them by device and inode, so
+  its `.lock` file, and the `-wal` and `-shm` files SQLite keeps beside it while
+  the scan runs (see "Concurrent reads"). It knows them by device and inode, so
   it passes over them however it reaches them, through a symlinked directory
   included. It prints no line for them and does not count them as skipped. A
   record an earlier scan left for one of them is deleted.
@@ -422,25 +434,27 @@ walk practical on large or busy pools.
     ```
 
     `devfs` is the macOS filesystem of `/dev`; the others are Linux names. As it
-    starts, `scan` lists the mounted filesystems and notes the device of each
+    starts, `scan` lists the mounted filesystems and notes the device ID of each
-    one of those types. On Linux the list comes from `/proc/self/mountinfo`,
+    one of those types. The device ID is the number by which the kernel tells
-    which gives each filesystem's own device. On macOS it comes from the
+    filesystems apart. On Linux the list comes from `/proc/self/mountinfo`,
-    system's mount list, and the device is read from the mount point. Where two
+    which gives each filesystem's own device ID. On macOS it comes from the
-    filesystems share a mount point on macOS, the one mounted last counts. On
+    system's mount list, and the device ID is read from the mount point. Where
-    any other system the list is empty. If `scan` cannot read the list, it fails
+    two filesystems share a mount point on macOS, the one mounted last counts.
-    with exit
+    On any other system the list is empty. A list that `scan` cannot read is a
-    1. No flag turns this rule off. The walk never enters a directory, and never
+    fatal error, and nothing is scanned. No flag turns this rule off.
-       records a file, on one of those devices, however it reaches it, through a
+
-       bind mount included. For each path it leaves out this way, `scan` prints
+    The walk never enters a directory on one of those devices, and never records
-       a line on stderr naming the path and its filesystem type:
+    a file on one. That holds however the walk reaches the path, through a bind
+    mount included. A bind mount shows a directory at a second path, and the
+    files under it keep their device ID. For each path it leaves out this way,
+    `scan` prints a line on stderr naming the path and its filesystem type:
 
     ```
     scan: skipped /dev/shm (tmpfs, not a persistent filesystem)
     ```
 
-    The line is not a warning, and the path is not counted as skipped. The
+    The path is not counted as skipped. The records below it are kept, as
-    records below the path are kept, as described under "What a scan does to the
+    described under "What a scan does to the database".
-    database".
 
 - Below an operand, the walk does not enter a default exclude. The default
   excludes hold the kernel's view of the running system and temporary files, so
@@ -449,8 +463,9 @@ walk practical on large or busy pools.
     - On Linux: `/proc`, `/sys`, `/dev`, `/run`, `/tmp` and `/var/tmp`.
     - On macOS: `/dev`, `/private/tmp`, `/private/var/tmp`, `/private/var/vm`,
       `/System/Volumes/VM` and `/System/Volumes/Data`. A walk of `/` reaches
-      everything in `/System/Volumes/Data` through firmlinks already; walking it
+      everything in `/System/Volumes/Data` already, through firmlinks: the links
-      as well would record each of those files twice.
+      macOS uses to show the data volume inside the system volume. Walking it as
+      well would record each of those files twice.
     - On any other system: none.
 
     `--no-default-excludes` turns the default excludes off. What is not on a
@@ -463,12 +478,11 @@ walk practical on large or busy pools.
     scan: skipped /var/tmp (default exclude)
     ```
 
-    The line is not a warning, and the directory is not counted as skipped. A
+    The directory is not counted as skipped. A default exclude that is not on a
-    default exclude that is not on a persistent filesystem, as `/proc` is, gets
+    persistent filesystem, as `/proc` is, gets the line for its filesystem
-    the line for its filesystem instead. One that is itself an operand gets no
+    instead. One that is itself an operand gets no line, because it is walked as
-    line, because it is walked as that operand. The records below a default
+    that operand. The scan keeps the records below a default exclude that the
-    exclude the walk did not enter are kept, as described under "What a scan
+    walk did not enter, as described under "What a scan does to the database".
-    does to the database".
 
 - A path that cannot be read, for lack of permission or any other error, gets a
   one-line warning on stderr and is skipped. The scan continues, and the summary
@@ -482,19 +496,22 @@ walk practical on large or busy pools.
 
 #### Concurrency
 
-The walk phase and the hash phase each use a pool of `--workers` workers. The
+The walk phase and the hash phase each use a pool of workers. A worker is one
-walk works on several directories at once, and the hash phase on several files
+thread of the program, and each pool has `--workers` of them. The walk works on
-at once. On spinning disks both phases wait mostly on seeks, so a `--workers`
+several directories at once, and the hash phase on several files at once. On
-value well above the CPU count can help on pools with many spindles. The main
+spinning disks both phases wait mostly on seeks, the time the disk head takes to
-thread of the program owns the database writes and the progress counts. Each
+move. A `--workers` value well above the CPU count can therefore help on storage
-progress display draws from a thread of its own and never blocks the workers.
+pools with many disks. The main thread of the program owns the database writes
+and the progress counts. Each progress display draws from a thread of its own
+and never blocks the workers.
 
 #### What `scan` prints
 
 `scan` writes nothing to stdout. On stderr it prints, in this order:
 
 - one line naming the database (see "Database");
-- a line for each operand it drops;
+- for each dropped operand, in operand order, the warning or the line described
+  under "Operands". A repeated operand, or one under another, gets neither;
 - the progress display of each phase (see "Progress"), with a warning for each
   path it skips and a line for each path the walk leaves out on purpose, as they
   happen;
@@ -527,8 +544,9 @@ The next scan skips the records already written and converges as usual.
 
 After the first signal `scan` stops catching signals. A second one ends it at
 once, as any uncaught signal does: the records not yet committed are lost, and
-the database stays valid. `SIGTERM` is always caught. A `SIGINT` that `scan`
+the database stays valid. `scan` catches `SIGTERM` even when it inherits it as
-inherits as ignored, as a shell's background job does, stays ignored.
+ignored. A `SIGINT` that `scan` inherits as ignored, as a background job started
+from a script does, stays ignored.
 
 ### `report` mode
 
@@ -539,7 +557,7 @@ Its output is the same whether or not the scanned filesystem is still mounted.
 
 Processing:
 
-- Records are grouped by the key `(size, head, tail, content)`.
+- Records are grouped by signature: `(size, head, tail, content)`.
 - Every group with two or more paths is a duplicate group.
 - Within a group, paths are sorted in byte order. The first is `first`, and
   every other path is a `dupe`.
@@ -557,8 +575,8 @@ smaller size.
 
 #### Report output format
 
-TSV on stdout: a header line, then one row per duplicate file. A group of N
+TSV (tab-separated values) on stdout: a header line, then one row per duplicate
-paths gives N-1 rows.
+file. A group of N paths gives N-1 rows.
 
 ```
 first	dupe	size
@@ -566,16 +584,20 @@ first	dupe	size
 /srv/a/big.iso	/srv/c/big-copy2.iso	4294967296
 ```
 
-Paths are raw bytes and may hold any byte except NUL. The path columns are
+Paths are raw bytes and may hold any byte except NUL, the zero byte. The path
-escaped, so that every row stays one line of tab-separated fields. A backslash
+columns are escaped, so that every row stays one line of tab-separated fields. A
-is written `\\`, a tab `\t`, a newline `\n` and a carriage return `\r`. Every
+backslash is written `\\`, a tab `\t`, a newline `\n` and a carriage return
-other byte is written unchanged, including bytes that are not valid UTF-8.
+`\r`. Every other byte is written unchanged, including bytes that are not valid
-Undoing the four escapes gives back the stored path. Grouping and ordering use
+UTF-8. Undoing the four escapes gives back the stored path. Grouping and
-the stored path. The warnings `scan` prints are escaped the same way, so each
+ordering use the stored path. The warnings `scan` prints are escaped the same
-warning stays one line.
+way, so each warning stays one line.
 
-The summary line on stderr gives the records read, the duplicate groups, the
+The summary line on stderr gives:
-dupe files and the reclaimable bytes, which are the sum of `size` over all rows:
+
+- the records read;
+- the duplicate groups;
+- the dupe files, which is the number of rows;
+- the reclaimable bytes: the sum of `size` over all rows.
 
 ```
 report: 14 records read, 5 duplicate groups, 8 dupe files, 10.0 KiB reclaimable
@@ -584,15 +606,15 @@ report: 14 records read, 5 duplicate groups, 8 dupe files, 10.0 KiB reclaimable
 ### `trees` mode
 
 `trees` reads the same database as `report` and takes no positional arguments.
-It reports whole duplicate directory trees: directories under which the same
+It reports whole duplicate directory trees. Two directories are duplicate trees
-relative paths hold files with the same signatures. Like `report`, it never
+when they hold the same relative paths, and each path holds a file with the same
-touches the scanned filesystem. It rebuilds the directory hierarchy from the
+signature in both. Like `report`, it never touches the scanned filesystem. It
-paths in the records, split on `/`.
+rebuilds the directory hierarchy from the paths in the records, split on `/`.
 
 Definitions:
 
-- A file's **signature** is `(size, head, tail, content)`. The mtime is not part
+- A file's **signature** is `(size, head, tail, content)`, as for `report`. The
-  of it.
+  mtime is not part of it.
 - A directory's **digest** is a SHA-256 hash computed from the bottom up. Each
   child contributes one entry. A file's entry is its name and signature; a
   subdirectory's entry is its name and digest. The entries are sorted in byte
@@ -606,10 +628,11 @@ Processing:
 - Build the hierarchy, compute every directory's digest, and group the
   directories by digest. Every group with two or more directories is a
   duplicate-tree group.
-- Report only the largest trees. A group is left out when its members' parents
+- Only the outermost duplicate trees are reported. A group is left out when its
-  are distinct directories that all share one digest: the parents' group, or an
+  members' parents are distinct directories that all have the same digest: the
-  ancestor's, already implies it. A group holding sibling directories, or
+  parents' group, or an ancestor's, already implies it. A group holding sibling
-  members whose parents differ, is always reported.
+  directories, or members whose parents have different digests, is always
+  reported.
 - Within a group, paths are sorted in byte order. The first is `first`, and
   every other path is a `dupe`.
 - Groups are ordered by total tree size, largest first, then by `first` path.
@@ -644,9 +667,12 @@ first	dupe	files	size
 The paths are escaped as described under "Report output format". The root
 directory's path is `/`.
 
-The summary line on stderr gives the records read, the duplicate-tree groups,
+The summary line on stderr gives:
-the dupe trees and the reclaimable bytes, which are the sum of `size` over all
+
-rows:
+- the records read;
+- the duplicate-tree groups;
+- the dupe trees, which is the number of rows;
+- the reclaimable bytes: the sum of `size` over all rows.
 
 ```
 trees: 14 records read, 1 duplicate tree groups, 1 dupe trees, 3.0 KiB reclaimable
@@ -674,9 +700,9 @@ size of recorded files	150000000000000 (136.4 TiB)
 - `database file size` and `-wal file size` are in bytes, then in human units.
   The `-wal` line appears only while a `-wal` file is beside the database, as
   during a scan.
-- `modified` is when the database file was last changed, in UTC. While the
+- `modified` is when the database file was last changed, in UTC. While a scan
-  database is in WAL mode (see "Database"), the newest changes may be in the
+  runs, the newest changes may be in the `-wal` file alone (see "Concurrent
-  `-wal` file alone.
+  reads").
 - `schema version` is the database's `PRAGMA user_version`. A database with
   another version than this build's is refused, as by `report`.
 - `records` is the number of records. `size of recorded files` adds up the sizes
@@ -722,17 +748,18 @@ the error for a missing database comes after it.
   a fatal error, exit 1, telling you to run `scan` first.
 - `report`, `trees` and `info` open the database read-only. They need read
   access to the database file and no write access to its directory. While the
-  database is in WAL mode, they also read the `-wal` and `-shm` files beside it,
+  database has `-wal` and `-shm` files beside it (see "Concurrent reads"), they
-  which SQLite creates with the database file's permissions.
+  read those too. SQLite creates them with the database file's permissions.
 
 #### The scan lock
 
 Only one `scan` runs against a database at a time.
 
-- For its whole run, `scan` holds an exclusive `flock(2)` lock on a lock file
+- For its whole run, `scan` holds an exclusive lock, taken with the `flock(2)`
-  beside the database. The lock file's name is the database's path, with
+  system call, on a lock file beside the database. The lock file's name is the
-  symlinks resolved, plus `.lock`: `/var/lib/sfdupes/db.sqlite.lock` by default.
+  database's path, with symlinks resolved, plus `.lock`:
-  Every path to one database therefore names the same lock file.
+  `/var/lib/sfdupes/db.sqlite.lock` by default. Every path to one database
+  therefore names the same lock file.
 - `scan` takes the lock before it walks the filesystem or opens the database.
   Just before that, it creates the database file, empty, if it is missing, so
   that the lock file is named the same way before the first scan as after. A
@@ -748,29 +775,34 @@ Only one `scan` runs against a database at a time.
 
 #### Concurrent reads
 
-- While `scan` runs, the database is in write-ahead log (WAL) mode, with a
+- While `scan` runs, the database is in write-ahead log (WAL) mode. In WAL mode
-  10-second busy timeout. In WAL mode SQLite appends changes to a `-wal` file
+  SQLite appends changes to a `-wal` file beside the database, and keeps an
-  beside the database, and readers do not block the writer. Running a report
+  index of that file in a `-shm` file. Readers do not block the writer, so
-  during a scan is therefore safe.
+  running a report during a scan is safe.
+- Every connection has a busy timeout of 10 seconds: how long it waits for a
+  database another connection has locked before it gives up.
 - The filesystem is authoritative, and the database follows it. Hashed records
   are committed 10,000 at a time while the scan runs, so a report may see a
   scan's changes partly applied. A scan that dies partway leaves a valid
   database holding every batch committed so far, and the next scan carries on
   from there.
 - When `scan` closes the database, it switches it back to rollback-journal mode,
-  so that between scans the database file alone holds everything. Each switch
+  SQLite's default mode, in which the database file alone holds everything.
+  Between scans a reader therefore needs no `-wal` or `-shm` file. Each switch
   needs the database to itself. A `scan` that starts while a report is still
   reading waits for it up to the busy timeout, then fails. A `scan` that ends
   while a report has the database open prints a warning and leaves the database
   in WAL mode until the next scan. `report` writes its rows while it reads, so
-  it is still reading while a pager or a stalled pipe holds up its output.
+  it is still reading while a pager such as `less`, or a stalled pipe, holds up
+  its output.
 
 #### What `scan` writes, and when
 
 - As it starts, before the walk: the directory, an empty database file and the
   `.lock` file, each only if missing.
-- Then, still before the walk: the schema in a new database, and the `-wal` and
+- Then, still before the walk: the schema, the table and index described under
-  `-shm` files, which stay beside the database while the scan runs.
+  "Schema", in a new database, and the `-wal` and `-shm` files, which stay
+  beside the database while the scan runs.
 - During the `load` and `walk` phases: no records. A first scan's database holds
   no records until the hash phase, however long its walk runs.
 - During the `hash` phase: records, committed 10,000 at a time.
@@ -805,7 +837,7 @@ CREATE INDEX files_signature ON files (size, head, tail, content);
   `mtime_nsec` the nanoseconds past that second. Split this way, they hold any
   mtime a filesystem can record, before 1678 or after 2262 included, which one
   64-bit count of nanoseconds cannot. They serve change detection only and are
-  not part of the duplicate key.
+  not part of the signature.
 - `head`, `tail` and `content` each hold the raw 32 bytes of a SHA-256 (see
   "Duplicate detection"). A record whose hash is not 32 bytes is an error.
 - `files_signature` lets `report` read the records in signature order without
@@ -824,9 +856,10 @@ The phases, and what each display counts:
     - `load` counts the records first, then reads them against that count. While
       it counts, the display says `counting the records`.
     - `walk` has no exact total. Its estimate is the number of records the last
-      scan left under the operands, shown with a `~`. A first scan has no
+      scan left under the operands, shown with a `~`. A scan with no records
-      estimate. The time remaining is `unknown` when there is no estimate, and
+      under its operands, such as a first scan, has no estimate. The time
-      once the walk passes the estimate.
+      remaining is `unknown` when there is no estimate, and once the walk passes
+      the estimate.
     - `hash` and `update` have exact totals. Only the files that need hashing
       count toward the hash total, so its time remaining is meaningful.
 - `report`: `[1/2] open`, `[2/2] read`.
@@ -852,7 +885,8 @@ Each display is one line holding:
 - the current rate, in the items the phase counts: files, records, directories
   or groups. It covers the items completed over the last 5 seconds, or, when
   stderr is not a terminal, since the line before. It falls to 0 while nothing
-  completes, and a burst after a wait shows as a burst;
+  completes. A burst of items after a wait shows as a high rate for the next 5
+  seconds, then the rate falls again;
 - the elapsed time, labelled `elapsed`;
 - the time remaining, labelled `remain`: a duration such as `12m9s`, or
   `unknown`. It comes from the phase's average rate since it started counting,
@@ -882,15 +916,18 @@ wait in two places names both, as `disk/db`:
 
 On a terminal:
 
-- Every display starts with a spinner, followed by its line. The display is
+- Every display starts with a spinner, a turning character, followed by its
-  drawn again several times a second, so the spinner turns and the elapsed time
+  line. The display is drawn again several times a second, so the spinner turns
-  keeps moving while a phase waits for its next item.
+  and the elapsed time keeps moving while a phase waits for its next item.
 - A phase with an exact total ends its line with its percent and a bar 10
   columns wide, at the terminal's right edge. The percent after the count is
   left out then. A phase with nothing to do shows no bar. Every phase has an
-  exact total once it has counted its work, except `scan`'s `walk`, the `open`
+  exact total once it has counted its work, except these:
-  phase of `report` and `trees`, and both phases of `info`. On a terminal 100
+    - `scan`'s `walk`;
-  columns wide:
+    - the `open` phase of `report` and `trees`;
+    - both phases of `info`.
+
+    On a terminal 100 columns wide:
 
     ```
     ⠹ [3/4] hash: [12345/98765], 92 files/s, elapsed 2m32s, remain 17m44s            12% |█         |
@@ -900,10 +937,16 @@ On a terminal:
   line too wide for the terminal first leaves out its rate, or the words of a
   stalled phase, or what the phase is waiting for. It leaves out its bar next.
   It is cut short at the edge only when the four remaining parts do not fit on
-  their own. Those are the phase and its place, the progress, the elapsed time
+  their own. Those are:
-  and the time remaining. That happens hours into a scan of millions of files,
+    - the phase and its place;
-  on a terminal narrower than about 75 columns. A terminal made narrower while a
+    - the progress;
-  phase runs is fitted at the next draw.
+    - the elapsed time;
+    - the time remaining.
+
+    That happens hours into a scan of millions of files, on a terminal narrower
+    than about 75 columns. A terminal made narrower while a phase runs is fitted
+    at the next draw.
+
 - A warning printed during a phase lands on a line of its own, never inside the
   display. So does every line written to stdout when stdout and stderr are the
   same file: one terminal, or one pipe or file that both are sent to. When they
@@ -920,17 +963,25 @@ the count it stopped at.
 ### Error handling and exit codes
 
 - `0`: success, even when files were skipped with warnings.
-- `1`: a fatal error, or a `scan` stopped by `SIGINT` or `SIGTERM`. The fatal
+- `1`: a fatal error, or a `scan` stopped by `SIGINT` or `SIGTERM`. A fatal
-  errors are:
+  error is one of these:
-    - an operand that does not exist;
+    - an operand that does not exist, or whose details `scan` cannot read;
+    - a list of mounted filesystems that `scan` cannot read;
     - another `scan` already running against the same database;
-    - a database that cannot be created, opened, read or written;
+    - a database that cannot be created, opened, read or written, its directory
+      and its lock file included;
+    - a relative database path when the working directory cannot be read;
+    - a database with another schema version, or one at version 0 that already
+      has a `files` table;
+    - a record whose `head`, `tail` or `content` is not 32 bytes;
     - a missing database for `report`, `trees` or `info`;
     - a failed write to stdout.
-- `2`: a usage error:
+- `2`: a usage error, which is one of these:
     - a missing or unknown subcommand;
+    - an unknown flag;
     - `scan` with no operand;
-    - `scan` with `--workers` below 1;
+    - `--workers` without a value, or with a value that is not a whole number or
+      is below 1;
     - `report`, `trees` or `info` with a positional argument.
 
 A fatal error is reported in one line on stderr that starts with `sfdupes:`. A
@@ -938,9 +989,9 @@ failed write to stdout, such as a full disk, is reported the same way. Two cases
 never reach `sfdupes` as a failed write:
 
 - When the reader of a stdout pipe exits early, as in `sfdupes report | head`,
-  the next write ends `sfdupes` with `SIGPIPE`. It ends quietly and without a
+  the next write ends `sfdupes` with `SIGPIPE`, the signal for a write to a
-  summary, as `cat` or `sort` would. The shell reports the signal, status 141 in
+  closed pipe. It ends quietly and without a summary, as `cat` or `sort` would.
-  most shells, not exit 1.
+  The shell reports the signal, status 141 in most shells, not exit 1.
 - When stdout is closed outright, as in `sfdupes report >&-`, the Go runtime
   opens `/dev/null` in its place before `sfdupes` starts. The output is
   discarded, and the run succeeds, as with `> /dev/null`.
@@ -955,43 +1006,52 @@ is POSIX `sh`, finds the repository root itself, and can be run from any
 directory.
 
 - `script/bootstrap` installs what building and developing need. It can be run
-  again at any time; installed tools are kept. `git`, `make` and `go` come from
+  again at any time; installed tools are kept. `git`, `make` and `go` are
-  the first of nix, apt, brew or apk found on the host, and are only checked for
+  checked for presence only. A missing one is installed from the first of `nix`,
-  presence. An installed node is used when its major version is 22. Otherwise
+  `apt`, `brew` or `apk` found on the host. An installed `node` is used when its
-  node 22.17.0 is installed through nvm, from a release archive whose SHA-256
+  major version is 22. Otherwise the script downloads `nvm`, checks the
-  the script checks, and yarn 1.22.22 is activated under it through corepack.
+  archive's SHA-256, and installs `node` 22.17.0 through it. It then activates
-  With an installed node 22, a yarn already on `PATH` is used as it is.
+  `yarn` 1.22.22 under that `node` through `corepack`. With an installed `node`
-  Otherwise yarn 1.22.22 is activated through corepack, or installed with `npm`
+  22, a `yarn` already on `PATH` is used as it is. Otherwise `yarn` 1.22.22 is
-  when there is no corepack. `yarn install --frozen-lockfile` then installs the
+  activated through `corepack`, or installed with `npm` when there is no
-  prettier that `package.json` and `yarn.lock` pin. The linter is never
+  `corepack`. `yarn install --frozen-lockfile` then installs the `prettier` that
-  installed; it runs in Docker (see `script/lint`). `docker` is not installed
+  `package.json` and `yarn.lock` pin. The linter is never installed; it runs in
-  either, and testing, linting and the image build need it. The script ends with
+  Docker (see `script/lint`). `docker` is not installed either, and testing,
-  `go mod download`.
+  linting and the image build need it. The script ends with `go mod download`.
 - `script/setup` makes a fresh clone ready for development: `script/bootstrap`,
   then `script/install-precommit`.
 - `script/projectname` prints `sfdupes`. Scripts that need the name call it, so
   they stay identical across repositories.
-- `script/test` runs the test suite under the race detector, with a 90-second
+- `script/test` runs the test suite under the race detector, Go's check for
-  timeout and coverage enabled. On failure it reruns the suite verbosely, so
+  unsynchronized access to memory from several threads. The suite runs with a
-  that the log names the failing test. It builds the `Dockerfile`'s `test` phase
+  90-second timeout and coverage enabled. On failure it reruns the suite
-  alone and writes no image. The race detector needs cgo and a C compiler, so
+  verbosely, so that the log names the failing test. It builds the
-  the phase starts from a digest-pinned Debian `golang` image, which has `gcc`.
+  `Dockerfile`'s `test` phase alone and writes no image. The race detector needs
-  The tests run as `nobody`, because several of them make a file unreadable, and
+  cgo, Go's bridge to C code, and a C compiler. The phase therefore starts from
-  root reads it anyway.
+  a Debian `golang` image, which has `gcc`. The image is pinned by its digest,
+  its content hash. The tests run as `nobody`, because several of them make a
+  file unreadable, and root reads it anyway.
 - `script/lint` runs the linter. It builds the `Dockerfile`'s `lint` phase alone
-  and writes no image. In the digest-pinned `golangci/golangci-lint` image, the
+  and writes no image. The phase starts from the `golangci/golangci-lint` image,
-  gofmt check, `golangci-lint config verify` and `golangci-lint run` run as
+  pinned by its digest, and runs three build steps:
-  build steps, so a successful build is a clean lint. The linter never runs on
+    - the `gofmt` check;
-  the host, so a working `docker` is the one thing linting needs.
+    - `golangci-lint config verify`;
+    - `golangci-lint run`.
+
+    A successful build is a clean lint. The linter never runs on the host, so a
+    working `docker` is the one thing linting needs.
+
 - `script/fmt` formats in place: the Go sources with `gofmt -s -w`, and every
-  Markdown file with prettier. Prettier uses 4-space indents and wraps prose at
+  Markdown file with `prettier`. `prettier` indents by 4 spaces and wraps prose,
-  80 columns, as `.prettierrc` sets. Both run on the host; prettier runs through
+  as `.prettierrc` sets and as the script also passes on the command line. It
-  yarn at the version `yarn.lock` pins. When the node on `PATH` is not node 22,
+  wraps at 80 columns, its default width. Both run on the host; `prettier` runs
-  the script loads the node 22.17.0 that `script/bootstrap` installed.
+  through `yarn` at the version `yarn.lock` pins. When the `node` on `PATH` is
+  not `node` 22, the script loads the `node` 22.17.0 that `script/bootstrap`
+  installed.
 - `script/fmt-check` is the read-only form of `script/fmt`: it names any
-  unformatted file and exits non-zero instead of writing. gofmt and prettier
+  unformatted file and exits non-zero instead of writing. `gofmt` and `prettier`
   both run every time, and each names itself when it fails. The `Dockerfile`'s
-  `lint` phase runs the same gofmt check.
+  `lint` phase runs the same `gofmt` check.
 - `script/check` runs `script/test`, `script/lint` and `script/fmt-check`, in
   that order, and changes nothing. The first two need `docker`; the third needs
   what `script/bootstrap` installs.
@@ -1011,15 +1071,15 @@ directory.
 
 Every `docker build` in `script/` passes a new `CACHEBUST` build argument. In
 every stage of the `Dockerfile`, `ARG CACHEBUST` comes after the steps that
-install system packages and download the Go modules, and before every step that
+install system packages and download the Go modules. Every step that lints,
-lints, tests or compiles. Every `RUN` step after it runs again on each build.
+tests or compiles comes after it. Every `RUN` step after it runs again on each
-Without that, Docker would serve the test and lint steps from its cache on an
+build. Without that, Docker would serve the test and lint steps from its cache
-unchanged tree, and the build would pass having run neither. The layers above
+on an unchanged tree, and the build would pass having run neither. The layers
-the line stay cached until the base image, their own lines, `go.mod` or `go.sum`
+above the line stay cached until the base image, their own lines, `go.mod` or
-change. Go's build cache is kept between builds, so only what changed is
+`go.sum` change. Go's build cache is kept between builds, so only what changed
-compiled again. The tests run with `-count=1`, so no test reports a stored
+is compiled again. The tests run with `-count=1`, so no test reports a stored
-result. `golangci-lint run` keeps golangci-lint's cache as well, so an unchanged
+result. `golangci-lint run` keeps the linter's own cache as well, so an
-package is not linted again.
+unchanged package is not linted again.
 
 ## Build
 
@@ -1032,7 +1092,7 @@ are shims onto them, except `build`, which carries the compile recipe:
 - `make setup` prepares a fresh clone: `bootstrap`, then the pre-commit hook.
 - `make test` runs the test suite under the race detector, in Docker (see
   `script/test`). It needs `docker`.
-- `make lint` runs the linter and the gofmt check, in Docker (see
+- `make lint` runs the linter and the `gofmt` check, in Docker (see
   `script/lint`). It needs `docker`.
 - `make fmt` formats the Go sources and the Markdown on the host, and
   `make fmt-check` checks them without writing. Both need `go` and what
@@ -1044,11 +1104,27 @@ are shims onto them, except `build`, which carries the compile recipe:
 - `make hooks` installs the pre-commit hook.
 - `make clean` removes the binary.
 
+### Constraints
+
+- The language is Go. The module is `sneak.berlin/go/sfdupes`, and the binary is
+  `sfdupes`.
+- The dependencies beyond the standard library are:
+    - `github.com/spf13/cobra`, for the command line;
+    - `github.com/schollz/progressbar/v3`, for the progress display;
+    - `golang.org/x/term`, to tell whether stderr is a terminal;
+    - `modernc.org/sqlite`, the SQLite driver;
+    - `golang.org/x/sys`, for the scan lock and the list of mounted filesystems.
+- `modernc.org/sqlite` is pure Go, so the binary builds without cgo. The
+  `Makefile` exports `CGO_ENABLED=0`. The test build is the one exception: the
+  race detector needs cgo (see `script/test`).
+- `report` and `trees` are deterministic: the same database contents give the
+  same output, whatever order the records were inserted in.
+
 ### Definition of done
 
 All of the following, run in this directory, must pass:
 
-1. `make check` passes: the tests, the linter, `gofmt` and prettier.
+1. `make check` passes: the tests, the linter, `gofmt` and `prettier`.
 2. `make docker` succeeds.
 3. The smoke test below behaves as described. It builds a throwaway tree in a
    temporary directory; never test against real data.
@@ -1108,13 +1184,6 @@ All of the following, run in this directory, must pass:
 
 Tracked in [TODO.md](TODO.md).
 
-## Non-goals
-
-- No byte-for-byte compare, and no deleting or linking of duplicates. The
-  reports are advisory; acting on them is your job.
-- No persistence beyond the SQLite database; no export or import formats.
-- No daemon or filesystem watcher; scheduling rescans is cron's job.
-
 ## License
 
 MIT. See [LICENSE](LICENSE).