Compare commits

..
1 Commits
Author SHA1 Message Date
sneak 2b46d374b1 Rewrite README.md sentence by sentence for a newcomer
check / check (push) Canceled after 0s
Rewrites `README.md` from scratch for
#135. Every statement was checked
against the code on `next` and against what the built binary prints on
small trees under `/var/tmp`. The headings that the code comments cite
are kept.

Where the old text and the repository disagreed, the new text states
what the code does. #140 lists
every such point. Among them: `@latest` installs the newest tagged
release, ten million files in both "Rationale" and "Design", no Windows
excludes, a relative operand is joined to the working directory as `pwd`
prints it, and an operand reached through a symlinked directory below
another operand is dropped (#144).

Dropped from "Constraints": the permission to add `github.com/spf13/viper`
if configuration-file support is ever needed.

Model: fable-5-1
2026-10-27 00:39:18 +00:00
+30 -25
View File
@@ -38,9 +38,11 @@ export SFDUPES_DATABASE="$HOME/.local/share/sfdupes/db.sqlite"
`scan` walks the trees named by its operands, the paths given after `scan` on `scan` walks the trees named by its operands, the paths given after `scan` on
the command line. The walk is the listing of every directory under them. `scan` the command line. The walk is the listing of every directory under them. `scan`
keeps one database record per regular file. A regular file is an ordinary file, keeps one database record per regular file. A regular file is an ordinary file,
as opposed to a directory, a symlink or a device. `report` prints the file-level as opposed to a directory, a symlink or a device node. A symlink is a file that
duplicates. `trees` prints the duplicate directory trees. `info` prints the holds the path of another file and stands in for it. A device node is a file
database's path, size and record count. that stands for a piece of hardware, such as a disk or a terminal. `report`
prints the file-level duplicates. `trees` prints the duplicate directory trees.
`info` prints the database's path, size and record count.
The database is `/var/lib/sfdupes/db.sqlite` unless `SFDUPES_DATABASE` names The database is `/var/lib/sfdupes/db.sqlite` unless `SFDUPES_DATABASE` names
another file. The first `scan` creates it. another file. The first `scan` creates it.
@@ -125,8 +127,9 @@ walk"). As the file `/etc/cron.d/sfdupes`:
- a warning for each path it skips; - a warning for each path it skips;
- a line for each system or temporary directory the walk leaves out (see - a line for each system or temporary directory the walk leaves out (see
"Rules for the walk"); "Rules for the walk");
- a line for each path left out because its filesystem is one the kernel - a line for each path left out because its filesystem is one the kernel,
makes up or keeps in memory, such as `tmpfs`; the core of the operating system, makes up or keeps in memory, such as
`tmpfs`;
- the summary line; - the summary line;
- a warning when a report still has the database open as the scan ends (see - a warning when a report still has the database open as the scan ends (see
"Concurrent reads"). "Concurrent reads").
@@ -284,9 +287,9 @@ The reference below is also the program's specification.
hashes them. The default is the number of CPUs. `N` must be at least 1. A hashes them. The default is the number of CPUs. `N` must be at least 1. A
smaller value is a usage error, reported in one line on stderr with exit 2 smaller value is a usage error, reported in one line on stderr with exit 2
before anything is scanned. before anything is scanned.
- `-x`, `--one-file-system` (`scan`) keeps the walk of each operand on that - `-x`, `--one-file-system` (`scan`) keeps the walk of each operand out of
operand's filesystem, as the same flag does for `du` and `rsync`. By default directories on other filesystems, as the same flag does for `du` and `rsync`.
the walk also enters directories on other filesystems. By default the walk also enters directories on other filesystems.
- `--no-default-excludes` (`scan`) lets the walk enter the default excludes. - `--no-default-excludes` (`scan`) lets the walk enter the default excludes.
Those are the system and temporary directories the walk otherwise leaves out, Those are the system and temporary directories the walk otherwise leaves out,
such as `/var/tmp` on Linux (see "Rules for the walk"). such as `/var/tmp` on Linux (see "Rules for the walk").
@@ -353,9 +356,7 @@ default. An operand may be a directory or a regular file.
stored under that absolute path. stored under that absolute path.
- All operands belong to one scan, and one walk covers them all at once (see - All operands belong to one scan, and one walk covers them all at once (see
"Concurrency"). "Concurrency").
- The warnings and the line described below come first, even for an operand that - An operand that repeats another is dropped before the walk.
repeats or lies under another. After them, an operand that repeats another is
dropped before the walk, without a line.
- An operand under another operand is dropped the same way, unless one of the - An operand under another operand is dropped the same way, unless one of the
cases below applies. No path is then walked twice. cases below applies. No path is then walked twice.
- An operand under another is kept, and walked on its own, when the inner - An operand under another is kept, and walked on its own, when the inner
@@ -390,6 +391,9 @@ default. An operand may be a directory or a regular file.
- An operand on a filesystem that is not persistent is dropped too. `scan` - An operand on a filesystem that is not persistent is dropped too. `scan`
prints a line naming it and its filesystem type, and does not count it as prints a line naming it and its filesystem type, and does not count it as
skipped. skipped.
- `scan` prints these warnings, and the line for an operand that is not on a
persistent filesystem, even when the operand repeats another or lies under
another.
- A scan whose every operand is dropped walks nothing and exits 0. - A scan whose every operand is dropped walks nothing and exits 0.
#### What a scan does to the database #### What a scan does to the database
@@ -401,8 +405,8 @@ operands:
- A file already in the database is not read again when its size and mtime match - A file already in the database is not read again when its size and mtime match
its record. The mtime matches when it is not newer than the recorded one. This its record. The mtime matches when it is not newer than the recorded one. This
is what makes a daily rescan cheap. is what makes a daily rescan cheap.
- A file whose size differs, or whose mtime is newer, is hashed again. The mtime - `scan` hashes a file again when its size differs or its mtime is newer. The
is compared to the nanosecond, as finely as the filesystem records it. A mtime is compared to the nanosecond, as finely as the filesystem records it. A
same-size rewrite counts as a change whenever the filesystem gives it a later same-size rewrite counts as a change whenever the filesystem gives it a later
mtime, even within the same second. mtime, even within the same second.
- The scan deletes each record under a scanned operand whose file it could not - The scan deletes each record under a scanned operand whose file it could not
@@ -589,8 +593,8 @@ from a thread of its own and never blocks the workers.
- one line naming the database (see "Database"); - one line naming the database (see "Database");
- for each dropped operand, in operand order, the warning or the line described - for each dropped operand, in operand order, the warning or the line described
under "Operands". An operand dropped only because it repeats another or lies under "Operands". `scan` prints neither for an operand it drops only because
under another gets neither; it repeats another or lies under another;
- the progress display of each phase (see "Progress"). `scan` prints each - the progress display of each phase (see "Progress"). `scan` prints each
warning for a skipped path as it happens. It prints a line the same way for warning for a skipped path as it happens. It prints a line the same way for
each path left out as a default exclude or as not on a persistent filesystem; each path left out as a default exclude or as not on a persistent filesystem;
@@ -722,11 +726,11 @@ Processing:
- `trees` builds the directory tree and computes every directory's digest. It - `trees` builds the directory tree and computes every directory's digest. It
then groups the directories by digest. Every group with two or more then groups the directories by digest. Every group with two or more
directories is a duplicate-tree group. directories is a duplicate-tree group.
- Only the outermost duplicate trees are reported. A group is left out when its - `trees` leaves out a group that a group of its members' parents already
members' parents are distinct directories that all have the same digest. The implies. A group is left out when its members' parents are distinct
parents' group, or an ancestor's, already implies it. A group is always directories that all have the same digest. The parents' group, or an
reported when it holds sibling directories, or when its members' parents have ancestor's, already implies it. A group is always reported when it holds
different digests. sibling directories, or when its members' parents have different digests.
- Within a group, paths are sorted in byte order. The first is `first`, and - Within a group, paths are sorted in byte order. The first is `first`, and
every other path is a `dupe`. every other path is a `dupe`.
- Groups are ordered by total tree size, largest first, then by `first` path. - Groups are ordered by total tree size, largest first, then by `first` path.
@@ -853,10 +857,11 @@ the error for a missing database comes after it.
Only one `scan` at a time can hold the lock. Only one `scan` at a time can hold the lock.
- For its whole run, `scan` holds a lock on a lock file beside the database. The - For its whole run, `scan` holds a lock on a lock file beside the database. The
lock is taken with the `flock(2)` system call, and only one process can hold lock is taken with the `flock(2)` system call, a request a program makes to
it. The lock file's name is the database's path, with symlinks resolved, plus the kernel, and only one process can hold it. The lock file's name is the
`.lock`. By default that is `/var/lib/sfdupes/db.sqlite.lock`. A path through database's path, with symlinks resolved, plus `.lock`. By default that is
a symlink names the same lock file. `/var/lib/sfdupes/db.sqlite.lock`. A path through a symlink names the same
lock file.
- `scan` takes the lock before it walks the filesystem or opens the database. - `scan` takes the lock before it walks the filesystem or opens the database.
Just before that, it creates the database file, empty, if it is missing. So Just before that, it creates the database file, empty, if it is missing. So
the lock file is named the same way before the first scan as after. A symlink the lock file is named the same way before the first scan as after. A symlink
@@ -1169,7 +1174,7 @@ the repository root itself, so it can be run from any directory.
- `script/lint` runs the linter, `golangci-lint`, which checks the Go sources - `script/lint` runs the linter, `golangci-lint`, which checks the Go sources
for common mistakes. It builds only the `lint` stage of the `Dockerfile` and for common mistakes. It builds only the `lint` stage of the `Dockerfile` and
writes no image. The stage starts from the `golangci/golangci-lint` image, writes no image. The stage starts from the `golangci/golangci-lint` image,
pinned by its digest, and runs three build steps: pinned by its digest, and runs these checks as build steps:
- the `gofmt` check, which fails when `gofmt`, Go's own formatter, would - the `gofmt` check, which fails when `gofmt`, Go's own formatter, would
change a Go source; change a Go source;
- `golangci-lint config verify`, which checks the linter's settings file - `golangci-lint config verify`, which checks the linter's settings file