1 Commits
Author SHA1 Message Date
sneak 988bf9937f Rewrite README.md sentence by sentence for a newcomer
check / check (push) Canceled after 0s
The old README was a specification written as one dense document:
sentences past 30 words, lists chained with commas, exceptions ahead
of the rules, and release history in the user text. This rewrite
leads with what the program is, what it is for and how to use it,
then gives a reference for the settings, each subcommand, the
database, the output formats, the progress display and the exit
codes. Non-goals come before the reference; the constraints, the
entrypoints and the build come last. Every statement was checked
against the code on next and against what the built binary prints on
a small tree. The section names the code comments cite are kept, and
three comments in main_test.go now cite Settings. TODO.md records the
step.

Model: fable-5-1
2026-10-26 08:15:26 +00:00
+36 -39
View File
@@ -119,8 +119,8 @@ skipped with a warning and loses its database record. As the file
- a progress line as each phase starts, one every 5 seconds while it runs, - a progress line as each phase starts, one every 5 seconds while it runs,
and one as it ends; and one as it ends;
- a warning for each path it skips; - a warning for each path it skips;
- a line for each path left out as a default exclude or as not on a - a line for each path the walk leaves out on purpose (see "Rules for the
persistent filesystem (see "Rules for the walk"); walk");
- the summary line; - the summary line;
- a warning when a report still has the database open as the scan ends (see - a warning when a report still has the database open as the scan ends (see
"Concurrent reads"). "Concurrent reads").
@@ -381,9 +381,9 @@ operands:
- a path that is not on a persistent filesystem; - a path that is not on a persistent filesystem;
- with `-x`, a directory on another filesystem than its operand. - with `-x`, a directory on another filesystem than its operand.
With `-x`, a filesystem mounted under an operand can therefore be scanned on A filesystem mounted under an operand can therefore be scanned on its own
its own schedule into the same database. An operand at or inside such a path schedule into the same database. An operand at or inside such a path is
is walked on its own. Its walk verifies the records below it, so they are walked on its own. Its walk verifies the records below it, so they are
deleted when their files are gone. deleted when their files are gone.
#### Phases #### Phases
@@ -422,18 +422,15 @@ walk practical on large or busy storage pools.
snapshots, and walking it would list every file once per snapshot. snapshots, and walking it would list every file once per snapshot.
- The walk never records the scan's own database files. Those are the database, - The walk never records the scan's own database files. Those are the database,
its `.lock` file, and the `-wal` and `-shm` files SQLite keeps beside it while its `.lock` file, and the `-wal` and `-shm` files SQLite keeps beside it while
the scan runs (see "Concurrent reads"). The walk knows them by device ID and the scan runs (see "Concurrent reads"). It knows them by device ID and inode
inode number. The device ID is the number by which the kernel tells number. The device ID is the number by which the kernel tells filesystems
filesystems apart, and the inode number tells the files of one filesystem apart, and the inode number tells the files of one filesystem apart. So the
apart. So the walk passes over them whatever path leads to them. An operand walk passes over them however it reaches them, through a symlinked directory
whose path runs through a symlinked directory is one such path: the walk included. It prints no line for them and does not count them as skipped. A
follows no symlink it finds, but it opens each operand at the path given. The record an earlier scan left for one of them is deleted.
walk prints no line for the database files and does not count them as skipped.
A record an earlier scan left for one of them is deleted.
- By default the walk crosses filesystem boundaries. With `-x`, it never enters - By default the walk crosses filesystem boundaries. With `-x`, it never enters
a directory on a different filesystem than its operand. `scan` prints no line a directory on a different filesystem than its operand. The records below such
for that directory and does not count it as skipped. The records below it are a directory are kept, as described under "What a scan does to the database".
kept, as described under "What a scan does to the database".
- The walk never records anything on a filesystem that is not persistent. A - The walk never records anything on a filesystem that is not persistent. A
filesystem is not persistent when the kernel makes it up or keeps it in filesystem is not persistent when the kernel makes it up or keeps it in
memory. These are the types, as `sfdupes scan --help` lists them on Linux and memory. These are the types, as `sfdupes scan --help` lists them on Linux and
@@ -480,11 +477,11 @@ walk practical on large or busy storage pools.
well would record each of those files twice. well would record each of those files twice.
- On any other system: none. - On any other system: none.
An operand at or inside a default exclude is walked, so
`sfdupes scan /tmp/x` works when `/tmp/x` is on a persistent filesystem.
`--no-default-excludes` turns the default excludes off. What is not on a `--no-default-excludes` turns the default excludes off. What is not on a
persistent filesystem stays left out. For each default exclude the walk does persistent filesystem stays left out. An operand at or inside a default
not enter, `scan` prints a line on stderr naming it: exclude is still walked, so `sfdupes scan /tmp/x` works when `/tmp/x` is on
a persistent filesystem. For each default exclude the walk does not enter,
`scan` prints a line on stderr naming it:
``` ```
scan: skipped /var/tmp (default exclude) scan: skipped /var/tmp (default exclude)
@@ -523,9 +520,9 @@ display draws from a thread of its own and never blocks the workers.
- one line naming the database (see "Database"); - one line naming the database (see "Database");
- for each dropped operand, in operand order, the warning or the line described - for each dropped operand, in operand order, the warning or the line described
under "Operands". A repeated operand, or one under another, gets neither; under "Operands". A repeated operand, or one under another, gets neither;
- the progress display of each phase (see "Progress"). `scan` prints each - the progress display of each phase (see "Progress"). A warning for each path
warning for a skipped path as it happens. It prints a line the same way for it skips, and a line for each path the walk leaves out on purpose, come as
each path left out as a default exclude or as not on a persistent filesystem; they happen;
- the summary line; - the summary line;
- a warning when a report still has the database open as the scan ends (see - a warning when a report still has the database open as the scan ends (see
"Concurrent reads"). "Concurrent reads").
@@ -544,8 +541,8 @@ scan: 123400 files seen (1200 added, 34 updated, 56 removed, 122166 unchanged),
`scan` stops cleanly on `SIGINT`, the signal Ctrl-C sends, or `SIGTERM`, the `scan` stops cleanly on `SIGINT`, the signal Ctrl-C sends, or `SIGTERM`, the
signal `kill` sends by default: signal `kill` sends by default:
- The workers stop taking work. Each finishes no more than the directory listing - The workers stop taking work. Each finishes at most the directory listing or
or the file it is reading, and the progress display is finished. the file it is reading, and the progress display is finished.
- The records hashed but not yet committed are committed, so the next scan does - The records hashed but not yet committed are committed, so the next scan does
not hash them again. not hash them again.
- Nothing else is written or deleted. Records are deleted only after a complete - Nothing else is written or deleted. Records are deleted only after a complete
@@ -751,11 +748,11 @@ trees: database /srv/tmp/sfdupes/db.db (from SFDUPES_DATABASE)
A relative path is printed with the working directory in front of it, as A relative path is printed with the working directory in front of it, as
`pwd -P` prints it, with symlinks resolved. The rest of the path is printed as `pwd -P` prints it, with symlinks resolved. The rest of the path is printed as
set, so a `..` in it stays, and the line names the same file. After a `cd` set, so a `..` in it stays, and the line names the same file. After a `cd`
through a symlink, a `..` counts from the directory the symlink leads to, as it through a symlink, a `../` therefore counts from the directory the symlink leads
does for any other program. The printed path shows that directory. The path is to, as it does for any other program. The path is escaped like the paths in a
escaped like the paths in a report. `scan` says `new database` when no file is report. `scan` says `new database` when no file is at that path as it starts.
at that path as it starts. `report`, `trees` and `info` print the line before `report`, `trees` and `info` print the line before they open the database, so
they open the database, so the error for a missing database comes after it. the error for a missing database comes after it.
#### Creation and access #### Creation and access
@@ -1008,7 +1005,7 @@ never reach `sfdupes` as a failed write:
- The reader of a stdout pipe may exit early, as `head` does in - The reader of a stdout pipe may exit early, as `head` does in
`sfdupes report | head`. The next write then ends `sfdupes` with `SIGPIPE`, `sfdupes report | head`. The next write then ends `sfdupes` with `SIGPIPE`,
the signal for a write to a closed pipe. `sfdupes` ends quietly and without a the signal for a write to a closed pipe. It ends quietly and without a
summary, as `cat` or `sort` would. The shell reports the signal, status 141 in summary, as `cat` or `sort` would. The shell reports the signal, status 141 in
most shells, not exit 1. most shells, not exit 1.
- When stdout is closed outright, as in `sfdupes report >&-`, the Go runtime - When stdout is closed outright, as in `sfdupes report >&-`, the Go runtime
@@ -1020,9 +1017,9 @@ never reach `sfdupes` as a failed write:
This repository follows the This repository follows the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all) [Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard. The executables in `script/` are the entrypoints of the development standard. The executables in `script/` are the entrypoints of the development
workflow, and the `Makefile` targets call them, except `build` and `clean` (see workflow, and each `Makefile` target does nothing but call its script. Every
"Build"). Every script is POSIX `sh`, finds the repository root itself, and can script is POSIX `sh`, finds the repository root itself, and can be run from any
be run from any directory. directory.
- `script/bootstrap` installs what building and developing need. It can be run - `script/bootstrap` installs what building and developing need. It can be run
again at any time; installed tools are kept. `git`, `make` and `go` are again at any time; installed tools are kept. `git`, `make` and `go` are
@@ -1100,11 +1097,11 @@ install system packages and download the Go modules. Every step that lints,
tests or compiles comes after it. Every `RUN` step after it runs again on each tests or compiles comes after it. Every `RUN` step after it runs again on each
build. Without that, Docker would serve the test and lint steps from its cache build. Without that, Docker would serve the test and lint steps from its cache
on an unchanged tree, and the build would pass having run neither. The layers on an unchanged tree, and the build would pass having run neither. The layers
above `ARG CACHEBUST` stay cached until the base image, their own lines, above the line stay cached until the base image, their own lines, `go.mod` or
`go.mod` or `go.sum` change. Go's build cache is kept between builds, so only `go.sum` change. Go's build cache is kept between builds, so only what changed
what changed is compiled again. The tests run with `-count=1`, so no test is compiled again. The tests run with `-count=1`, so no test reports a stored
reports a stored result. `golangci-lint run` keeps the linter's own cache as result. `golangci-lint run` keeps the linter's own cache as well, so an
well, so an unchanged package is not linted again. unchanged package is not linted again.
## Build ## Build