Compare commits

..
1 Commits
Author SHA1 Message Date
sneak 0342307a9b Rewrite README.md sentence by sentence for a newcomer
check / check (push) Canceled after 0s
Rewrites `README.md` from scratch for
#135. Every statement was checked
against the code on `next` and against what the built binary prints on
small trees under `/var/tmp`. The headings that the code comments cite
are kept.

Where the old text and the repository disagreed, the new text states
what the code does: `@latest` installs the newest tagged release, ten
million files in both "Rationale" and "Design", no Windows excludes, a
relative operand is joined to the working directory as `pwd` prints it,
and an operand reached through a symlinked directory below another
operand is dropped (#144).

Dropped from "Constraints": the permission to add `github.com/spf13/viper`
if configuration-file support is ever needed.

Model: fable-5-1
2026-10-26 21:39:00 +00:00
+35 -41
View File
@@ -106,10 +106,9 @@ docker run --rm -v /var/lib/sfdupes:/var/lib/sfdupes sfdupes report > dupes.tsv
### Daily scan from `cron` ### Daily scan from `cron`
Run `scan` as root, so that it can read every file. `scan` warns about a Run `scan` as root, so that it can read every file. A path that it cannot read
directory it cannot read, and about a new or changed file it cannot read. It is skipped with a warning and loses its database record. As the file
skips the path and deletes the records at or below it (see "Rules for the `/etc/cron.d/sfdupes`:
walk"). As the file `/etc/cron.d/sfdupes`:
``` ```
30 3 * * * root /usr/local/bin/sfdupes scan /srv 2>>/var/log/sfdupes.log || tail -n 3 /var/log/sfdupes.log 30 3 * * * root /usr/local/bin/sfdupes scan /srv 2>>/var/log/sfdupes.log || tail -n 3 /var/log/sfdupes.log
@@ -349,7 +348,10 @@ default. An operand may be a directory or a regular file.
- Each operand is made absolute before the walk. A relative operand is joined to - Each operand is made absolute before the walk. A relative operand is joined to
the working directory as `pwd` prints it, with its symlinks kept. A `.` or the working directory as `pwd` prints it, with its symlinks kept. A `.` or
`..` in it is resolved as text, and symlinks are not followed. Each record is `..` in it is resolved as text, and symlinks are not followed. Each record is
stored under that absolute path. stored under that absolute path. After a `cd` through a symlink, `scan .`
names the symlink and is skipped as a symlink operand. `scan sub` in that
directory records its files under the symlink's path. `report` can then list
each as a duplicate of the same file recorded under its real path.
- All operands belong to one scan, and one walk covers them all at once (see - All operands belong to one scan, and one walk covers them all at once (see
"Concurrency"). "Concurrency").
- An operand that repeats another is dropped before the walk, without a line. - An operand that repeats another is dropped before the walk, without a line.
@@ -374,10 +376,7 @@ default. An operand may be a directory or a regular file.
`scan` prints a warning naming the path and what it is. It counts the operand `scan` prints a warning naming the path and what it is. It counts the operand
as skipped and drops it. An operand beneath a symlink operand is scanned all as skipped and drops it. An operand beneath a symlink operand is scanned all
the same. The scan keeps the records beneath a dropped operand, unless another the same. The scan keeps the records beneath a dropped operand, unless another
operand covers them. After a `cd` through a symlink, `scan .` names the operand covers them.
symlink and is skipped as a symlink operand. `scan sub` in that directory
records its files under the symlink's path. `report` can then list each as a
duplicate of the same file recorded under its real path.
- An operand at or inside a directory named `.zfs` is never scanned, whether - An operand at or inside a directory named `.zfs` is never scanned, whether
given alone or under another operand. The check looks at the path as given and given alone or under another operand. The check looks at the path as given and
at the path with its symlinks resolved. `scan` prints a warning naming the at the path with its symlinks resolved. `scan` prints a warning naming the
@@ -405,9 +404,8 @@ operands:
- The scan deletes each record under a scanned operand whose file it could not - The scan deletes each record under a scanned operand whose file it could not
verify. That covers deleted files, and paths the scan could not stat or read verify. That covers deleted files, and paths the scan could not stat or read
this run. To stat a file is to read its details, such as its size, without this run. To stat a file is to read its details, such as its size, without
opening it. So after a complete scan, the database holds only what that scan opening it. So the database holds only what the latest complete scan over each
could verify under its operands. A later successful scan adds such files path could verify. A later successful scan adds such files again.
again.
- Records outside the scanned operands are left alone, so separate trees can be - Records outside the scanned operands are left alone, so separate trees can be
scanned on different schedules into one database. scanned on different schedules into one database.
- The scan keeps every record at or below a path that the walk leaves out for - The scan keeps every record at or below a path that the walk leaves out for
@@ -516,10 +514,11 @@ walk practical on large or busy storage pools.
The path is not counted as skipped. The records below it are kept, as The path is not counted as skipped. The records below it are kept, as
described under "What a scan does to the database". described under "What a scan does to the database".
- Below an operand, the walk does not enter a default exclude. On Linux, the - Below an operand, the walk does not enter a default exclude. The default
default excludes hold temporary files and the kernel's view of the running excludes hold temporary files and the kernel's view of the running system, so
system. They depend on the system, and `sfdupes scan --help` lists those of that `sfdupes scan /` walks only the files worth comparing. They depend on the
the system it runs on. On Linux they are: system, and `sfdupes scan --help` lists those of the system it runs on. On
Linux they are:
``` ```
/proc, /sys, /dev, /run, /tmp, /var/tmp /proc, /sys, /dev, /run, /tmp, /var/tmp
@@ -548,22 +547,17 @@ walk practical on large or busy storage pools.
scan: skipped /var/tmp (default exclude) scan: skipped /var/tmp (default exclude)
``` ```
The directory is not counted as skipped. `scan` prints the filesystem line The directory is not counted as skipped. A default exclude on a filesystem
instead for a default exclude that is not on a persistent filesystem, such that is not persistent, such as `/proc`, gets the line for its filesystem
as `/proc`. One that is itself an operand gets no line, because it is walked instead. One that is itself an operand gets no line, because it is walked as
as that operand. The scan keeps the records below a default exclude that the that operand. The scan keeps the records below a default exclude that the
walk did not enter, as described under "What a scan does to the database". walk did not enter, as described under "What a scan does to the database".
- `scan` skips each of these, with a one-line warning on stderr: - A path that cannot be read, for lack of permission or any other error, gets a
- a directory it cannot read; one-line warning on stderr and is skipped. The scan continues, and the summary
- a path it cannot stat; line says how many paths were skipped. A skipped path that has a record from
- a new or changed file it cannot read. an earlier scan loses that record, and so does every path under an unreadable
directory. The database mirrors what the latest complete scan could verify.
An unchanged file is not opened, so it keeps its record even when it has
become unreadable. The scan continues, and the summary line says how many
paths were skipped. A skipped path that has a record from an earlier scan
loses that record, and so does every path under an unreadable directory.
- A file or directory that disappears while the scan runs is not an error. The - A file or directory that disappears while the scan runs is not an error. The
scan prints no line for it and does not count it as skipped. A path gone when scan prints no line for it and does not count it as skipped. A path gone when
the walk reaches it gets no record, and so does a file gone when the hash the walk reaches it gets no record, and so does a file gone when the hash
@@ -586,8 +580,8 @@ from a thread of its own and never blocks the workers.
- one line naming the database (see "Database"); - one line naming the database (see "Database");
- for each dropped operand, in operand order, the warning or the line described - for each dropped operand, in operand order, the warning or the line described
under "Operands". `scan` prints neither for an operand that repeats another or under "Operands". An operand dropped as a repeat of another, or as lying under
lies under another; another, gets neither;
- the progress display of each phase (see "Progress"). `scan` prints each - the progress display of each phase (see "Progress"). `scan` prints each
warning for a skipped path as it happens. It prints a line the same way for warning for a skipped path as it happens. It prints a line the same way for
each path left out as a default exclude or as not on a persistent filesystem; each path left out as a default exclude or as not on a persistent filesystem;
@@ -721,9 +715,9 @@ Processing:
directories is a duplicate-tree group. directories is a duplicate-tree group.
- Only the outermost duplicate trees are reported. A group is left out when its - Only the outermost duplicate trees are reported. A group is left out when its
members' parents are distinct directories that all have the same digest. The members' parents are distinct directories that all have the same digest. The
parents' group, or an ancestor's, already implies it. A group is always parents' group, or an ancestor's, already implies it. A group holding sibling
reported when it holds sibling directories, or when its members' parents have directories, or members whose parents have different digests, is always
different digests. reported.
- Within a group, paths are sorted in byte order. The first is `first`, and - Within a group, paths are sorted in byte order. The first is `first`, and
every other path is a `dupe`. every other path is a `dupe`.
- Groups are ordered by total tree size, largest first, then by `first` path. - Groups are ordered by total tree size, largest first, then by `first` path.
@@ -1069,8 +1063,8 @@ Each phase prints its line:
- every 5 seconds while it runs, whether or not an item has completed; - every 5 seconds while it runs, whether or not an item has completed;
- once more as it ends. - once more as it ends.
When a phase stops short of its total, as an interrupted one does, it keeps A phase that stops short of its total, as an interrupted one does, keeps showing
showing the count it stopped at. the count it stopped at.
### Error handling and exit codes ### Error handling and exit codes
@@ -1134,8 +1128,8 @@ the repository root itself, so it can be run from any directory.
- An installed `node` is used when its major version is 22. A `yarn` already - An installed `node` is used when its major version is 22. A `yarn` already
on `PATH` is then used as it is. When `yarn` is missing, `yarn` 1.22.22 is on `PATH` is then used as it is. When `yarn` is missing, `yarn` 1.22.22 is
activated through `corepack`, the `yarn` installer that ships with `node`. activated through `corepack`, the `yarn` installer that ships with `node`.
When there is no `corepack` either, the script installs `yarn` with `npm`, When there is no `corepack` either, `npm`, the package manager that ships
the package manager that ships with `node`. with `node`, installs it.
- Otherwise the script installs `node` 22.17.0 through `nvm`, a `node` - Otherwise the script installs `node` 22.17.0 through `nvm`, a `node`
version manager, and activates `yarn` 1.22.22 under that `node` through version manager, and activates `yarn` 1.22.22 under that `node` through
`corepack`. When `nvm` is missing, the script first downloads it and `corepack`. When `nvm` is missing, the script first downloads it and
@@ -1166,8 +1160,8 @@ the repository root itself, so it can be run from any directory.
for common mistakes. It builds only the `lint` stage of the `Dockerfile` and for common mistakes. It builds only the `lint` stage of the `Dockerfile` and
writes no image. The stage starts from the `golangci/golangci-lint` image, writes no image. The stage starts from the `golangci/golangci-lint` image,
pinned by its digest, and runs three build steps: pinned by its digest, and runs three build steps:
- the `gofmt` check, which fails when `gofmt`, Go's own formatter, would - the `gofmt` check, which fails on a Go source that `gofmt`, Go's own
change a Go source; formatter, would change;
- `golangci-lint config verify`, which checks the linter's settings file - `golangci-lint config verify`, which checks the linter's settings file
`.golangci.yml`; `.golangci.yml`;
- `golangci-lint run`, which lints the Go sources. - `golangci-lint run`, which lints the Go sources.