Compare commits

..
1 Commits
Author SHA1 Message Date
sneak 468f0cb844 Rewrite README.md sentence by sentence for a newcomer
check / check (push) Canceled after 0s
Rewrites `README.md` from scratch for
#135. Every statement was checked
against the code on `next` and against what the built binary prints on
small trees under `/var/tmp`. The headings that the code comments cite
are kept.

Where the old text and the repository disagreed, the new text states
what the code does. #140 lists
every such point. Among them: `@latest` installs the newest tagged
release, ten million files in both "Rationale" and "Design", no Windows
excludes, a relative operand is joined to the working directory as `pwd`
prints it, and an operand reached through a symlinked directory below
another operand is dropped (#144).

Dropped from "Constraints": the permission to add `github.com/spf13/viper`
if configuration-file support is ever needed.

Model: fable-5-1
2026-10-26 22:42:11 +00:00
+41 -35
View File
@@ -106,9 +106,10 @@ docker run --rm -v /var/lib/sfdupes:/var/lib/sfdupes sfdupes report > dupes.tsv
### Daily scan from `cron`
Run `scan` as root, so that it can read every file. A path that it cannot read
is skipped with a warning and loses its database record. As the file
`/etc/cron.d/sfdupes`:
Run `scan` as root, so that it can read every file. `scan` warns about a
directory it cannot read, and about a new or changed file it cannot read. It
skips the path and deletes the records at or below it (see "Rules for the
walk"). As the file `/etc/cron.d/sfdupes`:
```
30 3 * * * root /usr/local/bin/sfdupes scan /srv 2>>/var/log/sfdupes.log || tail -n 3 /var/log/sfdupes.log
@@ -348,10 +349,7 @@ default. An operand may be a directory or a regular file.
- Each operand is made absolute before the walk. A relative operand is joined to
the working directory as `pwd` prints it, with its symlinks kept. A `.` or
`..` in it is resolved as text, and symlinks are not followed. Each record is
stored under that absolute path. After a `cd` through a symlink, `scan .`
names the symlink and is skipped as a symlink operand. `scan sub` in that
directory records its files under the symlink's path. `report` can then list
each as a duplicate of the same file recorded under its real path.
stored under that absolute path.
- All operands belong to one scan, and one walk covers them all at once (see
"Concurrency").
- An operand that repeats another is dropped before the walk, without a line.
@@ -376,7 +374,10 @@ default. An operand may be a directory or a regular file.
`scan` prints a warning naming the path and what it is. It counts the operand
as skipped and drops it. An operand beneath a symlink operand is scanned all
the same. The scan keeps the records beneath a dropped operand, unless another
operand covers them.
operand covers them. After a `cd` through a symlink, `scan .` names the
symlink and is skipped as a symlink operand. `scan sub` in that directory
records its files under the symlink's path. `report` can then list each as a
duplicate of the same file recorded under its real path.
- An operand at or inside a directory named `.zfs` is never scanned, whether
given alone or under another operand. The check looks at the path as given and
at the path with its symlinks resolved. `scan` prints a warning naming the
@@ -404,8 +405,9 @@ operands:
- The scan deletes each record under a scanned operand whose file it could not
verify. That covers deleted files, and paths the scan could not stat or read
this run. To stat a file is to read its details, such as its size, without
opening it. So the database holds only what the latest complete scan over each
path could verify. A later successful scan adds such files again.
opening it. So after a complete scan, the database holds only what that scan
could verify under its operands. A later successful scan adds such files
again.
- Records outside the scanned operands are left alone, so separate trees can be
scanned on different schedules into one database.
- The scan keeps every record at or below a path that the walk leaves out for
@@ -514,11 +516,10 @@ walk practical on large or busy storage pools.
The path is not counted as skipped. The records below it are kept, as
described under "What a scan does to the database".
- Below an operand, the walk does not enter a default exclude. The default
excludes hold temporary files and the kernel's view of the running system, so
that `sfdupes scan /` walks only the files worth comparing. They depend on the
system, and `sfdupes scan --help` lists those of the system it runs on. On
Linux they are:
- Below an operand, the walk does not enter a default exclude. On Linux, the
default excludes hold temporary files and the kernel's view of the running
system. They depend on the system, and `sfdupes scan --help` lists those of
the system it runs on. On Linux they are:
```
/proc, /sys, /dev, /run, /tmp, /var/tmp
@@ -547,17 +548,22 @@ walk practical on large or busy storage pools.
scan: skipped /var/tmp (default exclude)
```
The directory is not counted as skipped. A default exclude on a filesystem
that is not persistent, such as `/proc`, gets the line for its filesystem
instead. One that is itself an operand gets no line, because it is walked as
that operand. The scan keeps the records below a default exclude that the
The directory is not counted as skipped. `scan` prints the filesystem line
instead for a default exclude that is not on a persistent filesystem, such
as `/proc`. One that is itself an operand gets no line, because it is walked
as that operand. The scan keeps the records below a default exclude that the
walk did not enter, as described under "What a scan does to the database".
- A path that cannot be read, for lack of permission or any other error, gets a
one-line warning on stderr and is skipped. The scan continues, and the summary
line says how many paths were skipped. A skipped path that has a record from
an earlier scan loses that record, and so does every path under an unreadable
directory. The database mirrors what the latest complete scan could verify.
- `scan` skips each of these, with a one-line warning on stderr:
- a directory it cannot read;
- a path it cannot stat;
- a new or changed file it cannot read.
An unchanged file is not opened, so it keeps its record even when it has
become unreadable. The scan continues, and the summary line says how many
paths were skipped. A skipped path that has a record from an earlier scan
loses that record, and so does every path under an unreadable directory.
- A file or directory that disappears while the scan runs is not an error. The
scan prints no line for it and does not count it as skipped. A path gone when
the walk reaches it gets no record, and so does a file gone when the hash
@@ -580,8 +586,8 @@ from a thread of its own and never blocks the workers.
- one line naming the database (see "Database");
- for each dropped operand, in operand order, the warning or the line described
under "Operands". An operand dropped as a repeat of another, or as lying under
another, gets neither;
under "Operands". `scan` prints neither for an operand that repeats another or
lies under another;
- the progress display of each phase (see "Progress"). `scan` prints each
warning for a skipped path as it happens. It prints a line the same way for
each path left out as a default exclude or as not on a persistent filesystem;
@@ -715,9 +721,9 @@ Processing:
directories is a duplicate-tree group.
- Only the outermost duplicate trees are reported. A group is left out when its
members' parents are distinct directories that all have the same digest. The
parents' group, or an ancestor's, already implies it. A group holding sibling
directories, or members whose parents have different digests, is always
reported.
parents' group, or an ancestor's, already implies it. A group is always
reported when it holds sibling directories, or when its members' parents have
different digests.
- Within a group, paths are sorted in byte order. The first is `first`, and
every other path is a `dupe`.
- Groups are ordered by total tree size, largest first, then by `first` path.
@@ -1063,8 +1069,8 @@ Each phase prints its line:
- every 5 seconds while it runs, whether or not an item has completed;
- once more as it ends.
A phase that stops short of its total, as an interrupted one does, keeps showing
the count it stopped at.
When a phase stops short of its total, as an interrupted one does, it keeps
showing the count it stopped at.
### Error handling and exit codes
@@ -1128,8 +1134,8 @@ the repository root itself, so it can be run from any directory.
- An installed `node` is used when its major version is 22. A `yarn` already
on `PATH` is then used as it is. When `yarn` is missing, `yarn` 1.22.22 is
activated through `corepack`, the `yarn` installer that ships with `node`.
When there is no `corepack` either, `npm`, the package manager that ships
with `node`, installs it.
When there is no `corepack` either, the script installs `yarn` with `npm`,
the package manager that ships with `node`.
- Otherwise the script installs `node` 22.17.0 through `nvm`, a `node`
version manager, and activates `yarn` 1.22.22 under that `node` through
`corepack`. When `nvm` is missing, the script first downloads it and
@@ -1160,8 +1166,8 @@ the repository root itself, so it can be run from any directory.
for common mistakes. It builds only the `lint` stage of the `Dockerfile` and
writes no image. The stage starts from the `golangci/golangci-lint` image,
pinned by its digest, and runs three build steps:
- the `gofmt` check, which fails on a Go source that `gofmt`, Go's own
formatter, would change;
- the `gofmt` check, which fails when `gofmt`, Go's own formatter, would
change a Go source;
- `golangci-lint config verify`, which checks the linter's settings file
`.golangci.yml`;
- `golangci-lint run`, which lints the Go sources.