1 Commits
Author SHA1 Message Date
sneak 070674e9ee Rewrite README.md sentence by sentence for a newcomer
check / check (push) Canceled after 0s
The old README was a specification written as one dense document:
sentences past 30 words, lists chained with commas, exceptions ahead
of the rules, and release history in the user text. This rewrite
leads with what the program is, what it is for and how to use it,
then gives a reference for the settings, each subcommand, the
database, the output formats, the progress display and the exit
codes. Non-goals come before the reference; the constraints, the
entrypoints and the build come last. Every statement was checked
against the code on next and against what the built binary prints on
a small tree. The section names the code comments cite are kept, and
three comments in main_test.go now cite Settings. TODO.md records the
step.

Model: fable-5-1
2026-10-26 09:13:51 +00:00
+39 -36
View File
@@ -119,8 +119,8 @@ skipped with a warning and loses its database record. As the file
- a progress line as each phase starts, one every 5 seconds while it runs,
and one as it ends;
- a warning for each path it skips;
- a line for each path the walk leaves out on purpose (see "Rules for the
walk");
- a line for each path left out as a default exclude or as not on a
persistent filesystem (see "Rules for the walk");
- the summary line;
- a warning when a report still has the database open as the scan ends (see
"Concurrent reads").
@@ -381,9 +381,9 @@ operands:
- a path that is not on a persistent filesystem;
- with `-x`, a directory on another filesystem than its operand.
A filesystem mounted under an operand can therefore be scanned on its own
schedule into the same database. An operand at or inside such a path is
walked on its own. Its walk verifies the records below it, so they are
With `-x`, a filesystem mounted under an operand can therefore be scanned on
its own schedule into the same database. An operand at or inside such a path
is walked on its own. Its walk verifies the records below it, so they are
deleted when their files are gone.
#### Phases
@@ -422,15 +422,18 @@ walk practical on large or busy storage pools.
snapshots, and walking it would list every file once per snapshot.
- The walk never records the scan's own database files. Those are the database,
its `.lock` file, and the `-wal` and `-shm` files SQLite keeps beside it while
the scan runs (see "Concurrent reads"). It knows them by device ID and inode
number. The device ID is the number by which the kernel tells filesystems
apart, and the inode number tells the files of one filesystem apart. So the
walk passes over them however it reaches them, through a symlinked directory
included. It prints no line for them and does not count them as skipped. A
record an earlier scan left for one of them is deleted.
the scan runs (see "Concurrent reads"). The walk knows them by device ID and
inode number. The device ID is the number by which the kernel tells
filesystems apart, and the inode number tells the files of one filesystem
apart. So the walk passes over them whatever path leads to them. An operand
whose path runs through a symlinked directory is one such path: the walk
follows no symlink it finds, but it opens each operand at the path given. The
walk prints no line for the database files and does not count them as skipped.
A record an earlier scan left for one of them is deleted.
- By default the walk crosses filesystem boundaries. With `-x`, it never enters
a directory on a different filesystem than its operand. The records below such
a directory are kept, as described under "What a scan does to the database".
a directory on a different filesystem than its operand. `scan` prints no line
for that directory and does not count it as skipped. The records below it are
kept, as described under "What a scan does to the database".
- The walk never records anything on a filesystem that is not persistent. A
filesystem is not persistent when the kernel makes it up or keeps it in
memory. These are the types, as `sfdupes scan --help` lists them on Linux and
@@ -477,11 +480,11 @@ walk practical on large or busy storage pools.
well would record each of those files twice.
- On any other system: none.
An operand at or inside a default exclude is walked, so
`sfdupes scan /tmp/x` works when `/tmp/x` is on a persistent filesystem.
`--no-default-excludes` turns the default excludes off. What is not on a
persistent filesystem stays left out. An operand at or inside a default
exclude is still walked, so `sfdupes scan /tmp/x` works when `/tmp/x` is on
a persistent filesystem. For each default exclude the walk does not enter,
`scan` prints a line on stderr naming it:
persistent filesystem stays left out. For each default exclude the walk does
not enter, `scan` prints a line on stderr naming it:
```
scan: skipped /var/tmp (default exclude)
@@ -520,9 +523,9 @@ display draws from a thread of its own and never blocks the workers.
- one line naming the database (see "Database");
- for each dropped operand, in operand order, the warning or the line described
under "Operands". A repeated operand, or one under another, gets neither;
- the progress display of each phase (see "Progress"). A warning for each path
it skips, and a line for each path the walk leaves out on purpose, come as
they happen;
- the progress display of each phase (see "Progress"). `scan` prints each
warning for a skipped path as it happens. It prints a line the same way for
each path left out as a default exclude or as not on a persistent filesystem;
- the summary line;
- a warning when a report still has the database open as the scan ends (see
"Concurrent reads").
@@ -541,8 +544,8 @@ scan: 123400 files seen (1200 added, 34 updated, 56 removed, 122166 unchanged),
`scan` stops cleanly on `SIGINT`, the signal Ctrl-C sends, or `SIGTERM`, the
signal `kill` sends by default:
- The workers stop taking work. Each finishes at most the directory listing or
the file it is reading, and the progress display is finished.
- The workers stop taking work. Each finishes no more than the directory listing
or the file it is reading, and the progress display is finished.
- The records hashed but not yet committed are committed, so the next scan does
not hash them again.
- Nothing else is written or deleted. Records are deleted only after a complete
@@ -748,11 +751,11 @@ trees: database /srv/tmp/sfdupes/db.db (from SFDUPES_DATABASE)
A relative path is printed with the working directory in front of it, as
`pwd -P` prints it, with symlinks resolved. The rest of the path is printed as
set, so a `..` in it stays, and the line names the same file. After a `cd`
through a symlink, a `../` therefore counts from the directory the symlink leads
to, as it does for any other program. The path is escaped like the paths in a
report. `scan` says `new database` when no file is at that path as it starts.
`report`, `trees` and `info` print the line before they open the database, so
the error for a missing database comes after it.
through a symlink, a `..` counts from the directory the symlink leads to, as it
does for any other program. The printed path shows that directory. The path is
escaped like the paths in a report. `scan` says `new database` when no file is
at that path as it starts. `report`, `trees` and `info` print the line before
they open the database, so the error for a missing database comes after it.
#### Creation and access
@@ -1005,7 +1008,7 @@ never reach `sfdupes` as a failed write:
- The reader of a stdout pipe may exit early, as `head` does in
`sfdupes report | head`. The next write then ends `sfdupes` with `SIGPIPE`,
the signal for a write to a closed pipe. It ends quietly and without a
the signal for a write to a closed pipe. `sfdupes` ends quietly and without a
summary, as `cat` or `sort` would. The shell reports the signal, status 141 in
most shells, not exit 1.
- When stdout is closed outright, as in `sfdupes report >&-`, the Go runtime
@@ -1017,9 +1020,9 @@ never reach `sfdupes` as a failed write:
This repository follows the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard. The executables in `script/` are the entrypoints of the development
workflow, and each `Makefile` target does nothing but call its script. Every
script is POSIX `sh`, finds the repository root itself, and can be run from any
directory.
workflow, and the `Makefile` targets call them, except `build` and `clean` (see
"Build"). Every script is POSIX `sh`, finds the repository root itself, and can
be run from any directory.
- `script/bootstrap` installs what building and developing need. It can be run
again at any time; installed tools are kept. `git`, `make` and `go` are
@@ -1097,11 +1100,11 @@ install system packages and download the Go modules. Every step that lints,
tests or compiles comes after it. Every `RUN` step after it runs again on each
build. Without that, Docker would serve the test and lint steps from its cache
on an unchanged tree, and the build would pass having run neither. The layers
above the line stay cached until the base image, their own lines, `go.mod` or
`go.sum` change. Go's build cache is kept between builds, so only what changed
is compiled again. The tests run with `-count=1`, so no test reports a stored
result. `golangci-lint run` keeps the linter's own cache as well, so an
unchanged package is not linted again.
above `ARG CACHEBUST` stay cached until the base image, their own lines,
`go.mod` or `go.sum` change. Go's build cache is kept between builds, so only
what changed is compiled again. The tests run with `-count=1`, so no test
reports a stored result. `golangci-lint run` keeps the linter's own cache as
well, so an unchanged package is not linted again.
## Build