Rewrite README.md sentence by sentence for a newcomer

The old README was a specification written as one dense document:
sentences past 30 words, lists chained with commas, exceptions ahead
of the rules, and release history in the user text. This rewrite
leads with what the program is, what it is for and how to use it,
then gives a reference for the settings, each subcommand, the
database, the output formats, the progress display and the exit
codes. Non-goals come before the reference; the constraints, the
entrypoints and the build come last. Every statement was checked
against the code on next and against what the built binary prints on
a small tree. The section names the code comments cite are kept, and
five comments in main_test.go now cite Settings. TODO.md records the
step.

Model: fable-5-1

README.md

@@ -18,11 +18,11 @@ a strong candidate, not proof.
 
 `sfdupes` is meant for multi-terabyte servers, where reading every byte of every
 file takes too long. It has four subcommands. `scan` walks the filesystem and
-records the files it finds in an SQLite database. SQLite keeps a whole database
+records every file in an SQLite database. SQLite keeps a whole database in one
-in one file. The database is kept between runs. A rescan reads only new and
+file. The database is kept between runs. A rescan reads only new and changed
-changed files. That makes `scan` cheap enough to run every day from `cron`, the
+files. That makes `scan` cheap enough to run every day from `cron`, the system's
-system's job scheduler. `report`, `trees` and `info` read the database alone and
+job scheduler. `report`, `trees` and `info` read the database alone and can run
-can run at any time.
+at any time.
 
 ## Getting Started
 
@@ -45,9 +45,10 @@ database's path, size and record count.
 The database is `/var/lib/sfdupes/db.sqlite` unless `SFDUPES_DATABASE` names
 another file. The first `scan` creates it.
 
-`report`, `trees` and `info` write their data to stdout. Progress, warnings and
+`report`, `trees` and `info` write their data to stdout. Everything else, such
-errors go to stderr. stdout and stderr are a program's two output streams. The
+as progress and warnings, goes to stderr. stdout and stderr are a program's two
-shell shows both on the terminal, and `>` redirects stdout alone.
+output streams. The shell shows both on the terminal, and `>` redirects stdout
+alone.
 
 ### Install
 
@@ -121,10 +122,8 @@ skipped with a warning and loses its database record. As the file
     - one line naming the database;
     - progress lines, as described under "Progress";
     - a warning for each path it skips;
-    - a line for each system or temporary directory the walk leaves out (see
+    - a line for each path the walk leaves out as a system or temporary
-      "Rules for the walk");
+      directory (see "Rules for the walk");
-    - a line for each path left out because its filesystem is one the kernel
-      makes up or keeps in memory, such as `tmpfs`;
     - the summary line;
     - a warning when a report still has the database open as the scan ends (see
       "Concurrent reads").
@@ -173,8 +172,8 @@ where the files differ. The second line prints `hard links` when both paths are
 the same file.
 
 A path holding a special character, such as a tab, is escaped in the reports.
-The character is written as a backslash and one more character, `\t` for a tab
+The character is written as a backslash and a letter, `\t` for a tab (see
-(see "Report output format"). Undo the escapes before using such a path.
+"Report output format"). Undo the escapes before using such a path.
 `printf '%b'` does that, because every backslash in an escaped path starts an
 escape. The shell drops trailing newlines from the output of `$(...)`. To keep a
 newline at the end of a path, print an `x` after the path and remove it
@@ -348,36 +347,23 @@ default. An operand may be a directory or a regular file.
   absolute path, so results do not depend on the working directory.
 - All operands belong to one scan, and one walk covers them all at once (see
   "Concurrency").
-- An operand that repeats another is dropped before the walk, without a line.
+- An operand that repeats another, or lies under another, is dropped before the
-- An operand under another operand is dropped the same way, unless one of the
+  walk, without a line. Every file is then reached once. An operand under
-  cases below applies. No file is then reached twice.
+  another is kept when the outer operand's walk would not reach it. That is the
-- An operand under another is kept, and walked on its own, when the inner
+  case when the inner operand, or a directory between the two, is one of these:
-  operand or a directory between the two is one of these:
     - a default exclude (see "Rules for the walk");
     - with `-x`, a directory on another filesystem than the outer operand.
 
-    It is also kept when a directory between the two is on a filesystem that is
+    It is also the case when a directory between the two is on a filesystem that
-    not persistent. Such a filesystem is one the kernel makes up or keeps in
+    is not persistent. Such a filesystem is one the kernel makes up or keeps in
     memory (see "Rules for the walk"). The outer operand's walk does not enter
-    such a directory, so the inner operand is walked on its own. For an inner
+    such a directory, so the inner operand is walked on its own.
-    operand that is a regular file, the check starts at its directory.
 
-- No other case keeps an operand under another. An operand reached through a
+- An operand that is neither a regular file nor a directory is not scanned, and
-  symlinked directory below the outer operand is dropped too, without a line.
+  neither is a directory named `.zfs`. `scan` prints a warning naming the path
-  The outer operand's walk does not follow the symlink, so no file under the
+  and what it is. It counts the operand as skipped and drops it. An operand
-  inner operand is scanned. The records an earlier scan left under it are
+  beneath a dropped one is scanned all the same. The scan keeps the records
-  deleted.
+  beneath a dropped operand, unless another operand covers them.
-- An operand that is neither a regular file nor a directory is not scanned.
-  `scan` prints a warning naming the path and what it is. It counts the operand
-  as skipped and drops it. An operand beneath a dropped one is scanned all the
-  same. The scan keeps the records beneath a dropped operand, unless another
-  operand covers them.
-- An operand at or inside a directory named `.zfs` is never scanned, whether
-  given alone or under another operand. The check looks at the path as given and
-  at the path with its symlinks resolved. `scan` prints a warning naming the
-  path, counts the operand as skipped and drops it. The records beneath it are
-  kept, unless it lies under another operand. Then they are deleted, because
-  that operand's walk never enters `.zfs` (see "Rules for the walk").
 - An operand on a filesystem that is not persistent is dropped too. `scan`
   prints a line naming it and its filesystem type, and does not count it as
   skipped.
@@ -451,8 +437,7 @@ walk practical on large or busy storage pools.
   or a device node, is passed over. Symlinks are never followed.
 - The walk never enters a directory named `.zfs`. Such a directory holds the
   filesystem's ZFS snapshots, which are its saved earlier states. Walking it
-  would list every file once per snapshot. `scan` never starts a walk at or
+  would list every file once per snapshot.
-  inside one either (see "Operands").
 - The walk never records the scan's own database files:
     - the database file;
     - its `.lock` file (see "The scan lock");
@@ -574,8 +559,7 @@ from a thread of its own and never blocks the workers.
 
 - one line naming the database (see "Database");
 - for each dropped operand, in operand order, the warning or the line described
-  under "Operands". An operand dropped as a repeat of another, or as lying under
+  under "Operands". A repeated operand, or one under another, gets neither;
-  another, gets neither;
 - the progress display of each phase (see "Progress"). `scan` prints each
   warning for a skipped path as it happens. It prints a line the same way for
   each path left out as a default exclude or as not on a persistent filesystem;
@@ -618,12 +602,12 @@ and `scan` leaves it ignored.
 
 ### `report` mode
 
-`report` reads every record from the database. It takes no positional arguments.
+`report` reads every record from the database. It takes no positional arguments,
-A positional argument is anything on the command line that is not a flag, such
+meaning nothing may follow `report` on the command line but flags. It never
-as the paths `scan` takes. `report` never touches the scanned filesystem. It
+touches the scanned filesystem. It neither stats nor opens any path in the
-neither stats nor opens any path in the records. It reads the database and
+records. It reads the database and writes to stdout and stderr, and nothing
-writes to stdout and stderr, and nothing else. Its output is the same whether or
+else. Its output is the same whether or not the scanned filesystem is still
-not the scanned filesystem is still mounted.
+mounted.
 
 Processing:
 
@@ -789,10 +773,9 @@ size of recorded files	150000000000000 (136.4 TiB)
 
 `info` has two phases, each with its own progress display on stderr (see
 "Progress"). `open` opens the database. `count` reads the schema version and
-counts the records in one transaction. A transaction is a group of reads or
+counts the records in one transaction, a read of the database at one moment. So
-writes that SQLite carries out as one step, against the database as it stands at
+the two agree with each other even while a scan is writing. `info` writes its
-one moment. So both describe the database at one moment, even while a scan is
+lines after the `count` display has ended.
-writing. `info` writes its lines after the `count` display has ended.
 
 ### Database
 
@@ -901,11 +884,11 @@ Only one `scan` runs against a database at a time.
 The schema is the database's layout, its table and its index.
 `PRAGMA user_version` holds the schema version, currently 1. A database with any
 other version is a fatal error. `scan` creates the table and the index and sets
-the version in one transaction (see "`info` mode"). So the three land together
+the version in one transaction, a group of writes that lands whole or not at
-or not at all. A first scan stopped while doing so leaves an empty database,
+all. A first scan stopped while doing so leaves an empty database, which the
-which the next scan sets up. A database at version 0 that already has a `files`
+next scan sets up. A database at version 0 that already has a `files` table was
-table was not made by `sfdupes`. Every subcommand refuses it, with an error
+not made by `sfdupes`. Every subcommand refuses it, with an error telling you to
-telling you to remove the file and rescan.
+remove the file and rescan.
 
 ```sql
 CREATE TABLE files (
@@ -999,8 +982,8 @@ while a disk or the database is slow. Each phase names the places it may wait
 in, and a phase that may wait in two places names both, as `disk/db`:
 
 - `walk` says `disk`, for files.
-- `hash` says `disk/db`, because it reads files and writes their records to the
+- `hash` says `disk/db`: it reads files and writes their records to the database
-  database between them.
+  between them.
 - `load`, `update` and the `read` phase of `trees` say `db`.
 - The `read` phase of `report` says `db/out`, because it writes the report to
   stdout while it reads.
@@ -1101,8 +1084,9 @@ This repository follows the
 [Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
 standard. The executables in `script/` are the entrypoints of the development
 workflow, and the `Makefile` targets call them, except `build` and `clean` (see
-"Build"). Every script runs under plain `sh`, the standard Unix shell. It finds
+"Build"). Every script runs under plain `sh`, the standard Unix shell, and uses
-the repository root itself, so it can be run from any directory.
+no `bash` feature. It finds the repository root itself, so it can be run from
+any directory.
 
 - `script/bootstrap` installs what building and developing need. It can be run
   again at any time, and installed tools are kept. `git`, `make` and `go` are
@@ -1239,10 +1223,10 @@ commands of their own:
 
     No other dependency outside the standard library is added.
 
-- The code stays pure Go, with no C code. Building on one system for another is
+- The code stays pure Go, with no C code. Cross-compilation is not a concern.
-  not a goal. `modernc.org/sqlite` is pure Go too, so the binary builds without
+  `modernc.org/sqlite` is pure Go too, so the binary builds without cgo. The
-  cgo. The `Makefile` exports `CGO_ENABLED=0`. The test build is the one
+  `Makefile` exports `CGO_ENABLED=0`. The test build is the one exception,
-  exception, because the race detector needs cgo (see `script/test`).
+  because the race detector needs cgo (see `script/test`).
 - `report` and `trees` give the same output for the same database contents,
   whatever order the records were inserted in.
 
@@ -1310,11 +1294,11 @@ All of the following, run in this directory, must pass:
 
 ## TODO
 
-Tracked in [`TODO.md`](TODO.md).
+Tracked in [TODO.md](TODO.md).
 
 ## License
 
-MIT. See [`LICENSE`](LICENSE).
+MIT. See [LICENSE](LICENSE).
 
 ## Author
 

TODO.md

@@ -33,14 +33,8 @@
   it. It leads with what the program is and how to use it. A reference section
   covers every setting and every subcommand. Development and testing come last.
   Every statement was checked against the code on `next`. Five comments in
-  `main_test.go` cite "Settings", where their rules now live. One comment each
+  `main_test.go` cite "Settings", where their rules now live (2026-10-26,
-  in `scan_test.go`, `trees_test.go` and `exclude_test.go` cites what the README
+  https://git.eeqj.de/sneak/sfdupes/issues/135)
-  now says (2026-10-26, https://git.eeqj.de/sneak/sfdupes/issues/135)
-
-- an operand at or inside a directory named `.zfs` is never scanned, whether
-  given alone or under another operand. `scan` warns about it, counts it as
-  skipped and does not walk it (2026-10-26,
-  https://git.eeqj.de/sneak/sfdupes/issues/142)
 
 - a file or directory removed while the scan runs is not an error, in the walk
   or in the hash phase. The scan prints no line for it and does not count it as

exclude_test.go

@@ -17,9 +17,7 @@ import (
 func TestDefaultExcludes(t *testing.T) {
 	t.Parallel()
 
-	// README "Rules for the walk": the Linux and macOS lists, and none
+	// README "Rules for the walk": each system's list.
-	// on any other system. The Windows list exists in the code only: the
-	// binary does not build for Windows, so the README leaves it out.
 	cases := map[string][]string{
 		goosLinux: {"/proc", "/sys", "/dev", "/run", "/tmp", "/var/tmp"},
 		goosDarwin: {

main_test.go

@@ -928,21 +928,24 @@ func TestRunScanWalksOperandUnderSymlinkOperand(t *testing.T) {
 }
 
 func TestRunScanSkipsZFSOperand(t *testing.T) {
-	zfs := filepath.Join(t.TempDir(), ".zfs")
+	path := testDBPath(t)
-	f := writeFile(t, zfs, "snapshot/hourly/f", pattern(1, 10))
-
-	// No scan records a file inside a .zfs directory, so the record
-	// beneath the operand is written straight into the database.
-	path := seedDatabase(t, []scanRec{{size: 10, path: f}})
 	t.Setenv(databaseEnv, path)
 
+	zfs := filepath.Join(t.TempDir(), ".zfs")
+	snapshot := filepath.Join(zfs, "snapshot", "hourly")
+	f := writeFile(t, snapshot, "f", pattern(1, 10))
+
+	// An operand beneath a .zfs directory is walked, because it is not
+	// itself named .zfs.
+	scanOK(t, snapshot)
+
 	assertOperandSkipped(t, path, zfs, ".zfs directory", f)
 }
 
 // assertOperandSkipped scans operand alone and checks that it is skipped
 // as kind: a warning naming it, one skip in the summary, exit 0, and the
-// record for kept, stored beneath operand before this scan, still in the
+// record for kept, which an earlier scan stored beneath operand, still
-// database at dbPath.
+// in the database at dbPath.
 func assertOperandSkipped(t *testing.T, dbPath, operand, kind,
 	kept string,
 ) {
@@ -971,121 +974,6 @@ func assertOperandSkipped(t *testing.T, dbPath, operand, kind,
 	recordByPath(t, dbRecords(t, db), kept)
 }
 
-// TestRunScanSkipsOperandInsideZFS scans a directory inside a .zfs
-// directory, first alone and then under another operand. Each time it
-// is warned about and counted as skipped, and nothing in it is walked.
-// The other operand is scanned as usual.
-func TestRunScanSkipsOperandInsideZFS(t *testing.T) {
-	path := testDBPath(t)
-	t.Setenv(databaseEnv, path)
-
-	dir := t.TempDir()
-	hourly := filepath.Join(dir, ".zfs", "snapshot", "hourly")
-	writeFile(t, hourly, "f", pattern(1, 10))
-	f := writeFile(t, dir, "f", pattern(2, 10))
-
-	warning := "walk " + hourly + ": skipping operand inside a .zfs directory\n"
-
-	for _, tc := range []struct {
-		operands []string
-		summary  string
-	}{
-		{
-			[]string{hourly},
-			"scan: 0 files seen (0 added, 0 updated, 0 removed, " +
-				"0 unchanged), 1 skipped\n",
-		},
-		{
-			[]string{dir, hourly},
-			"scan: 1 files seen (1 added, 0 updated, 0 removed, " +
-				"0 unchanged), 1 skipped\n",
-		},
-	} {
-		stderr := scanOK(t, tc.operands...)
-		if !strings.Contains(stderr, warning) ||
-			!strings.Contains(stderr, tc.summary) {
-			t.Errorf("scan %q: stderr = %q, want %q and %q", tc.operands,
-				stderr, warning, tc.summary)
-		}
-	}
-
-	db, err := openDB(path, reportParams)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	t.Cleanup(func() { _ = db.Close() })
-
-	if got := recordPaths(dbRecords(t, db)); !slices.Equal(got, []string{f}) {
-		t.Errorf("records = %q, want %q", got, []string{f})
-	}
-}
-
-// TestRunScanSkipsOperandThroughSymlinkIntoZFS scans snaps/daily, where
-// snaps is a symlink to pool/.zfs/snapshot. The operand is warned about
-// by the path given, counted as skipped, and not walked.
-func TestRunScanSkipsOperandThroughSymlinkIntoZFS(t *testing.T) {
-	t.Setenv(databaseEnv, testDBPath(t))
-
-	dir := t.TempDir()
-	writeFile(t, dir, "pool/.zfs/snapshot/daily/f", pattern(1, 10))
-
-	snaps := filepath.Join(dir, "snaps")
-
-	err := os.Symlink(filepath.Join(dir, "pool", ".zfs", "snapshot"), snaps)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	daily := filepath.Join(snaps, "daily")
-	stderr := scanOK(t, daily)
-
-	warning := "walk " + daily + ": skipping operand inside a .zfs directory\n"
-	summary := "scan: 0 files seen (0 added, 0 updated, 0 removed, " +
-		"0 unchanged), 1 skipped\n"
-
-	if !strings.Contains(stderr, warning) ||
-		!strings.Contains(stderr, summary) {
-		t.Errorf("stderr = %q, want %q and %q", stderr, warning, summary)
-	}
-}
-
-// TestRunScanWalksOperandThroughSymlink scans data/daily, where data is
-// a symlink to a directory outside any .zfs directory. The operand is
-// scanned as usual, and its file is recorded by the path given.
-func TestRunScanWalksOperandThroughSymlink(t *testing.T) {
-	path := testDBPath(t)
-	t.Setenv(databaseEnv, path)
-
-	dir := t.TempDir()
-	writeFile(t, dir, "pool/data/daily/f", pattern(1, 10))
-
-	data := filepath.Join(dir, "data")
-
-	err := os.Symlink(filepath.Join(dir, "pool", "data"), data)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	daily := filepath.Join(data, "daily")
-	stderr := scanOK(t, daily)
-
-	summary := "scan: 1 files seen (1 added, 0 updated, 0 removed, " +
-		"0 unchanged), 0 skipped\n"
-	if !strings.Contains(stderr, summary) {
-		t.Errorf("stderr = %q, want %q", stderr, summary)
-	}
-
-	db, err := openDB(path, reportParams)
-	if err != nil {
-		t.Fatal(err)
-	}
-
-	t.Cleanup(func() { _ = db.Close() })
-
-	recordByPath(t, dbRecords(t, db), filepath.Join(daily, "f"))
-}
-
 func TestRunReportSucceeds(t *testing.T) {
 	path := testDBPath(t)
 	t.Setenv(databaseEnv, path)

scan.go

@@ -383,8 +383,7 @@ func (s *scanState) runPhases(ctx context.Context, roots []string,
 ) error {
 	// Types and filesystems are checked before pruning so that an
 	// operand under a dropped one is still scanned, not dropped as lying
-	// under it, and so that an operand inside a .zfs directory gets its
+	// under it.
-	// warning even when it lies under another operand.
 	roots = pruneRoots(s.walkableRoots(roots, rules), oneFS, rules,
 		deviceOfPath)
 
@@ -412,15 +411,15 @@ func (s *scanState) runPhases(ctx context.Context, roots []string,
 }
 
 // walkableRoots returns the operands the walk can start from: regular
-// files, and directories not named .zfs, none of them inside a
+// files, and directories not named .zfs. Every other operand is warned
-// directory named .zfs. Every other operand is warned about, counted as
+// about, counted as skipped, and dropped. A dropped operand is no
-// skipped, and dropped. A dropped operand is no longer a root, so the
+// longer a root, so the records stored beneath it are treated as if it
-// records stored beneath it are treated as if it had not been given:
+// had not been given: those under no other root count as outside the
-// those under no other root count as outside the roots and are not
+// roots and are not deleted as unverified. An operand on a filesystem
-// deleted as unverified. An operand on a filesystem in rules that is not
+// in rules that is not persistent is dropped the same way, with a line
-// persistent is dropped the same way, with a line saying so that is not
+// saying so that is not a warning, and is not counted as skipped. An
-// a warning, and is not counted as skipped. An operand that fails lstat
+// operand that fails lstat here is kept, and the walk warns about it
-// here is kept, and the walk warns about it unless it no longer exists.
+// unless it no longer exists.
 func (s *scanState) walkableRoots(roots []string, rules walkRules) []string {
 	kept := make([]string, 0, len(roots))
 
@@ -875,24 +874,9 @@ func sendEvent(ctx context.Context, events chan<- walkEvent,
 
 // operandWarning returns the one-line warning for an operand the walk
 // does not start from, naming the path and what it is, or "" for one it
-// does: a regular file, or a directory not named .zfs, that is not
+// does: a regular file, or a directory not named .zfs. Symlinks are
-// inside a directory named .zfs, neither as given nor with its symlinks
+// never followed, including as operands.
-// resolved. Symlinks are never followed, including as operands.
 func operandWarning(root string, fi fs.FileInfo) string {
-	// The walk never enters a .zfs directory, so it starts from nothing
-	// inside one either, not even through a symlink. An operand whose
-	// symlinks cannot be resolved is checked as given.
-	resolved, err := filepath.EvalSymlinks(root)
-	if err != nil {
-		resolved = root
-	}
-
-	if slices.Contains(strings.Split(filepath.Dir(root), "/"), ".zfs") ||
-		slices.Contains(strings.Split(filepath.Dir(resolved), "/"), ".zfs") {
-		return fmt.Sprintf("walk %s: skipping operand inside a .zfs directory",
-			root)
-	}
-
 	var kind string
 
 	switch mode := fi.Mode(); {

scan_test.go

@@ -1064,9 +1064,8 @@ func TestScanOneFilesystemWalksNestedFileOperand(t *testing.T) {
 	}
 }
 
-// buildSmokeTree recreates the tree of the test under README
+// buildSmokeTree recreates the README smoke-test filesystem layout
-// "Definition of done" with deterministic content and returns the tree
+// with deterministic content and returns the tree root.
-// root.
 func buildSmokeTree(t *testing.T) string {
 	t.Helper()
 

trees_test.go

@@ -15,9 +15,8 @@ import (
 const treesOpenLine = "[1/4] open: opening the database, elapsed 0s, " +
 	"remain unknown"
 
-// smokeTreeRecs mirrors the tree of the test under README "Definition
+// smokeTreeRecs mirrors the README smoke-test tree layout: /d/t1 and
-// of done": /d/t1 and /d/t2 are identical, /d/t3 differs from them only
+// /d/t2 are identical, /d/t3 differs from them only by one filename.
-// by one filename.
 func smokeTreeRecs() []scanRec {
 	f1Head, f1Tail, f1Content := testHash("f1h"), testHash("f1t"), testHash("f1c")
 	f2Head, f2Tail, f2Content := testHash("f2h"), testHash("f2t"), testHash("f2c")