scan: skip OS default excludes, and record nothing on filesystems that are not persistent (closes #124)

scan never records anything on a filesystem that is not persistent
(tmpfs, proc, sysfs and the other kernel or memory-backed types). As it
starts it notes the device ID of each such mount point, from
/proc/self/mountinfo on Linux and the mounted-filesystem list on macOS,
and checks every operand, directory and file the walk meets against
them; no flag turns this off. Below an operand the walk also skips the
default excludes, a path list per system (Linux, macOS, and Windows,
which is not used yet), and --no-default-excludes turns off only those.
A default exclude that is itself an operand gets no skip line. The tests
stop at once, saying why, when TMPDIR is not on a persistent filesystem.

Judgement call: an unreadable mount list fails the scan, and no flag gets round it.
Judgement call: a file on such a filesystem below an operand gets the same line as a directory.
Judgement call: an operand on such a filesystem that another operand's walk also reaches is named twice.
Unverified: the macOS mount list is built for macOS but never run; bind mounts need root to test.

Model: opus-5-5

README.md

@@ -100,9 +100,11 @@ As a file `/etc/cron.d/sfdupes`:
   `/var/log/sfdupes.log`, holds a first line naming the database (see
   "Database"), plain progress lines naming each phase and its place among the
   five, as `[2/5] walk` (one as the phase starts, one every 5 seconds while it
-  runs, and one as it ends), a warning for each path it skips, and the summary
-  line (see "Progress" and "`scan` mode"). The log grows with every scan; rotate
-  it like any other.
+  runs, and one as it ends), a warning for each path it skips, a line for each
+  default exclude its walk does not enter and for each path it leaves out
+  because it is not on a persistent filesystem (see "Rules for the walk"), and
+  the summary line (see "Progress" and "`scan` mode"). The log grows with every
+  scan; rotate it like any other.
 - Skipped paths do not fail a scan: it still exits 0, and cron sends nothing. A
   scan that fails, or is stopped by `SIGINT` or `SIGTERM`, exits 1 with the
   reason among the last lines of the log; `tail` prints them, and cron mails
@@ -248,7 +250,7 @@ Four subcommands, all implemented:
     schema version and the counts of its records.
 
 ```
-sfdupes scan [--workers N] [-x] PATH...
+sfdupes scan [--workers N] [-x] [--no-default-excludes] PATH...
 sfdupes report > dupes.tsv
 sfdupes trees > dupetrees.tsv
 sfdupes info
@@ -257,8 +259,9 @@ sfdupes [command] --help
 ```
 
 `--workers N` sets the size of each `scan` worker pool (default: the number of
-CPUs), and `-x` (`--one-file-system`) keeps the walk of each operand on that
-operand's filesystem; both are described under "`scan` mode".
+CPUs), `-x` (`--one-file-system`) keeps the walk of each operand on that
+operand's filesystem, and `--no-default-excludes` lets the walk enter the
+default excludes, a list of paths; all three are described under "`scan` mode".
 `sfdupes --version` (or `-v`) prints one line, `sfdupes VERSION`, to stdout and
 exits 0, writing nothing to stderr. `-h` or `--help`, alone or after a
 subcommand, prints the help text to stderr and exits 0, writing nothing to
@@ -439,11 +442,12 @@ results do not depend on the working directory. All operands belong to a single
 scan and are enumerated concurrently: every operand seeds the shared walk worker
 pool. Overlapping operands are harmless — an operand that duplicates another or
 lies under another is dropped before walking, so every file is reached exactly
-once and produces one database record. With `-x`, though, an operand under
-another one is kept and walked on its own when any directory from it (from its
-directory, for a regular file) up to but not including the operand covering it
-is on a different filesystem than that operand, because that operand's walk does
-not enter such a directory and so does not reach it.
+once and produces one database record. An operand under another one is kept,
+though, and walked on its own when any directory from it (from its directory,
+for a regular file) up to but not including the operand covering it is a default
+exclude, is not on a persistent filesystem (see "Rules for the walk"), or, with
+`-x`, is on a different filesystem than that operand, because that operand's
+walk does not enter such a directory and so does not reach it.
 
 An operand that is a symlink (never followed, not even as an operand), socket,
 FIFO, or device node, or a directory named `.zfs`, is not scanned. `scan` prints
@@ -453,7 +457,10 @@ beneath it is still scanned. The records stored beneath it are not deleted: they
 are treated like any other record outside the scanned operands, including the
 content-phase exception below. If it lies under another operand, they are under
 that operand instead, and are treated like any other record there. This is not
-an error: a scan whose every operand is dropped walks nothing and exits 0.
+an error: a scan whose every operand is dropped walks nothing and exits 0. An
+operand that is not on a persistent filesystem (see "Rules for the walk") is
+dropped the same way, except that the line `scan` prints for it is not a
+warning, and it is not counted as skipped.
 
 `scan` synchronizes the database with the filesystem state under the scanned
 operands:
@@ -491,12 +498,13 @@ operands:
   a `content` hash is read for one, wherever it lies, once its size, `head`, and
   `tail` match another record's. If that file is gone or has changed since its
   record was written, the record is left as it is.
-- With `-x`, a record below a directory the walk did not enter because that
-  directory is on another filesystem than its operand counts as outside the
-  scanned operands: it is not deleted, and its size joins the size census. A
-  filesystem mounted under an operand can therefore be scanned on its own
-  schedule into the same database. Records below an operand at or inside such a
-  directory are not kept this way, since that operand's own walk verifies them.
+- A record at or below a path the walk left out, because that path is a default
+  exclude, is not on a persistent filesystem, or, with `-x`, is a directory on
+  another filesystem than its operand, counts as outside the scanned operands:
+  it is not deleted, and its size joins the size census. A filesystem mounted
+  under an operand can therefore be scanned on its own schedule into the same
+  database. Records below an operand at or inside such a path are not kept this
+  way, since that operand's own walk verifies them.
 
 `scan` runs **five sequential phases over the whole scan**. Parallelism lives
 inside the walk, hash and content phases; batched database writes begin during
@@ -572,6 +580,71 @@ Rules for the walk:
   unless they lie below an operand at or inside that directory, count as outside
   the scanned operands, as described in "`scan` mode" above: they are not
   deleted, and their sizes join the size census.
+- Never record anything that is not on a persistent filesystem. A filesystem is
+  not persistent when the kernel makes it up or keeps it in memory: one of type
+  `proc`, `sysfs`, `devtmpfs`, `devpts`, `tmpfs`, `ramfs`, `cgroup`, `cgroup2`,
+  `debugfs`, `tracefs`, `securityfs`, `pstore`, `bpf`, `configfs`, `fusectl`,
+  `mqueue`, `hugetlbfs`, `efivarfs`, `binfmt_misc`, `autofs`, `nsfs`,
+  `rpc_pipefs`, `selinuxfs` or `devfs` (the macOS filesystem of `/dev`). After
+  it names the database and before it takes the lock, `scan` lists the mounted
+  filesystems, from `/proc/self/mountinfo` on Linux and from the system's list
+  of mounted filesystems on macOS (on any other system it lists none), and notes
+  the device ID of the mount point of each one of those types, read with
+  `lstat`. Where one filesystem is mounted over another, the one mounted last
+  counts. If `scan` cannot list them, it fails (exit 1). The walk then never
+  enters a directory, and never records a file, on one of those devices, however
+  it reaches it: as an operand, below one, or through a bind mount, which keeps
+  the device ID of what it binds. No flag turns this off. For each operand,
+  directory or file it leaves out this way, `scan` prints a line on stderr
+  naming it and the type of its filesystem, as it prints a warning, so it never
+  lands inside a progress display:
+
+    ```
+    scan: skipped /dev/shm (tmpfs, not a persistent filesystem)
+    ```
+
+    That line is not a warning, and the path is not counted as skipped. Records
+    that an earlier scan left at or below it, unless they lie below an operand
+    at or inside it, count as outside the scanned operands, as described in
+    "`scan` mode" above: they are not deleted, and their sizes join the size
+    census.
+
+- Below an operand, never descend into a default exclude: a path that holds the
+  kernel's view of the running system, or temporary files, so that
+  `sfdupes scan /` walks only the files worth comparing. The default excludes
+  depend on the system, and `sfdupes scan --help` lists those of the system it
+  runs on:
+    - On Linux: `/proc`, `/sys`, `/dev`, `/run`, `/tmp` and `/var/tmp`.
+    - On macOS: `/dev`, `/private/tmp`, `/private/var/tmp`, `/private/var/vm`,
+      `/System/Volumes/VM` and `/System/Volumes/Data`. A walk of `/` reaches
+      everything in `/System/Volumes/Data` through the firmlinks, so walking it
+      as well would record each of those files twice.
+    - On Windows, relative to the root of the drive an operand lies on: the
+      page, swap and hibernation files `pagefile.sys`, `swapfile.sys` and
+      `hiberfil.sys`, and the directories `System Volume Information`,
+      `$Recycle.Bin`, `Windows\Temp` and `Users\*\AppData\Local\Temp`, where `*`
+      matches one path component, as it does in any default exclude. sfdupes
+      does not build for Windows yet, so this list is not used.
+    - On any other system there are none.
+
+    `--no-default-excludes` turns them all off; what is not on a persistent
+    filesystem stays left out. An operand at or inside a default exclude is
+    still walked, so `sfdupes scan /tmp/x` works, unless it is not on a
+    persistent filesystem. For each default exclude its walk does not enter,
+    `scan` prints a line on stderr naming it, as it prints a warning:
+
+    ```
+    scan: skipped /var/tmp (default exclude)
+    ```
+
+    That line is not a warning, and the directory is not counted as skipped. A
+    default exclude that is not on a persistent filesystem, as `/proc` is, gets
+    the line for its filesystem instead, and one that is itself an operand gets
+    no line, since it is walked as that operand. The records below a default
+    exclude the walk does not enter, unless they lie below an operand at or
+    inside it, count as outside the scanned operands, as described in "`scan`
+    mode" above: they are not deleted, and their sizes join the size census.
+
 - On any per-path error (permission denied, file vanished between passes,
   unreadable): print a one-line warning to stderr, skip the path, and continue.
   Per-file errors never abort the run; the final summary reports how many were