Compare commits

..
1 Commits
Author SHA1 Message Date
sneak f6230f410c Rewrite README.md sentence by sentence for a newcomer
check / check (push) Canceled after 0s
Rewrites `README.md` from scratch for
#135. Every statement was checked
against the code on `next` and against what the built binary prints on
small trees under `/var/tmp`. The headings that the code comments cite
are kept. Comments in `main_test.go`, `scan_test.go`, `trees_test.go` and
`exclude_test.go` cite what the README now says.

Where the old text and the repository disagreed, the new text states
what the code does: `@latest` installs the newest tagged release, ten
million files, no Windows excludes, and an operand reached through a
symlinked directory below another operand is dropped
(#144).

Dropped from "Constraints": the permission to add `github.com/spf13/viper`
if configuration-file support is ever needed.

Model: fable-5-1
2026-10-26 19:49:12 +00:00
+35 -30
View File
@@ -67,11 +67,12 @@ The binary goes to `$(go env GOPATH)/bin`, or to `$GOBIN` when that is set.
`sfdupes --version` prints `sfdupes VERSION` and exits. For a binary from
`go install`, `VERSION` is the version Go recorded in it. For `@latest` that is
a release tag such as `v0.0.3`. For `@main` it is a pseudo-version, Go's version
string for an untagged commit. That string ends in the commit's hash. A binary
built from a clone carries the output of `git describe --tags --always --dirty`
in that clone, such as `v0.0.3` on a tagged commit. `VERSION` is `dev` when no
version is known. To see everything Go recorded in an installed binary:
a release tag such as `v0.0.3`. For `@main` it is the commit's tag when it has
one. Otherwise it is a pseudo-version, Go's version string for an untagged
commit. That string ends in the commit's hash. A binary built from a clone
carries the output of `git describe --tags --always --dirty` in that clone, such
as `v0.0.3` on a tagged commit. `VERSION` is `dev` when no version is known. To
see everything Go recorded in an installed binary:
```sh
go version -m "$(command -v sfdupes)"
@@ -201,9 +202,9 @@ two ends and at 1 MiB per gigabyte. A full read of a small file is cheap, and it
makes the comparison exact. Reading samples cuts a 150 GiB file to about 150 MiB
of reads, at the price of a comparison that is not exact.
The hashes live in a database that is kept between runs. A rescan reads only the
files that are new or whose size or modification time (mtime) changed. All
analysis then works from the database alone.
The hashes live in a database that is kept between runs. A rescan reads only new
files, and files whose size changed or whose modification time (mtime) is newer
than recorded. All analysis then works from the database alone.
The goal is whole duplicated trees, not only single files. A download extracted
twice, or a project copied once too often, is such a tree. It can be removed as
@@ -366,7 +367,8 @@ default. An operand may be a directory or a regular file.
symlinked directory below the outer operand is dropped too, without a line.
The outer operand's walk does not follow the symlink, so no file under the
inner operand is scanned. The records that an earlier scan left under it are
deleted.
deleted. With `-x`, a directory operand reached that way is kept when the
symlink leads to another filesystem than the outer operand.
- An operand that is neither a regular file nor a directory is not scanned.
`scan` prints a warning naming the path and what it is. It counts the operand
as skipped and drops it. An operand beneath a symlink operand is scanned all
@@ -411,10 +413,10 @@ operands:
Such records count as outside the scanned operands. So with `-x`, a
filesystem mounted under an operand can be scanned on its own schedule into
the same database. An operand may lie at or inside a default exclude, or,
with `-x`, at or inside a directory on another filesystem. Such an operand
is walked on its own. Its walk verifies the records below it, so they are
deleted when their files are gone.
the same database. An operand may itself lie at or inside a default exclude.
With `-x`, it may also lie on another filesystem than an operand above it.
Such an operand is walked on its own. Its walk verifies the records below
it, so they are deleted when their files are gone.
#### Phases
@@ -510,7 +512,7 @@ walk practical on large or busy storage pools.
described under "What a scan does to the database".
- Below an operand, the walk does not enter a default exclude. The default
excludes hold the kernel's view of the running system and temporary files, so
excludes hold temporary files and the kernel's view of the running system, so
that `sfdupes scan /` walks only the files worth comparing. They depend on the
system, and `sfdupes scan --help` lists those of the system it runs on. On
Linux they are:
@@ -609,8 +611,8 @@ signal `kill` sends by default:
- The line `scan: interrupted after N files` goes to stderr. `N` is the number
of files the walk reached. The exit code is 1.
The next scan reads none of the files already recorded and brings the database
up to date as usual.
The next scan reads none of the unchanged files already recorded, and brings the
database up to date as usual.
After the first signal `scan` stops catching signals. A second one ends it at
once, as any uncaught signal does. The records not yet written are lost, and the
@@ -901,13 +903,15 @@ Only one `scan` at a time can hold the lock.
#### Schema
The schema is the database's layout, its table and its index.
`PRAGMA user_version` holds the schema version, currently 1. A database with any
other version is a fatal error. `scan` creates the table and the index and sets
the version in one transaction (see "`info` mode"). So the three land together
or not at all. A first scan stopped while doing so leaves an empty database,
which the next scan sets up. A database at version 0 that already has a `files`
table was not made by `sfdupes`. Every subcommand refuses it, with an error
telling you to remove the file and rescan.
`PRAGMA user_version` holds the schema version, currently 1. An empty database
is at version 0, and `scan` sets it up. Any other version is a fatal error.
`report`, `trees` and `info` set nothing up, so for them version 0 is a fatal
error too. `scan` creates the table and the index and sets the version in one
transaction (see "`info` mode"). So the three land together or not at all. A
first scan stopped while doing so leaves an empty database, which the next scan
sets up. A database at version 0 that already has a `files` table was not made
by `sfdupes`. Every subcommand refuses it, with an error telling you to remove
the file and rescan.
```sql
CREATE TABLE files (
@@ -937,9 +941,9 @@ CREATE INDEX files_signature ON files (size, head, tail, content);
### Progress
Every subcommand always shows that it is working. Each phase gets a display of
its own on stderr from the moment it starts. The display names the phase and
shows its progress. The parts of its line are listed below.
Every subcommand shows that it is working: each phase gets a display of its own
on stderr from the moment it starts. The display names the phase and shows its
progress. The parts of its line are listed below.
The phases, and what each display counts:
@@ -1070,8 +1074,9 @@ the count it stopped at.
- a database that cannot be created or opened, or whose reads or writes
fail, its directory and its lock file included;
- a relative database path when the working directory cannot be read;
- a database with another schema version, or one at version 0 that already
has a `files` table;
- a database with another schema version, or one at version 0 that `scan` is
not setting up: a database read by `report`, `trees` or `info`, or one
that already has a `files` table (see "Schema");
- a record with a hash that is not 32 bytes, read by `report` or `trees`;
- a missing database for `report`, `trees` or `info`;
- a failed write to stdout.
@@ -1187,8 +1192,8 @@ the repository root itself, so it can be run from any directory.
every check passed. The tests and the linter run twice, once in `script/check`
and again as stages of the image build.
- `script/precommit` runs from the `git` pre-commit hook, a script that `git`
runs before each commit. It fails the commit when `go mod tidy` would change
anything. Then it runs `script/check`.
runs before each commit. It runs `go mod tidy`, and fails the commit when that
changes `go.mod` or `go.sum`. Then it runs `script/check`.
- `script/install-precommit` installs `.git/hooks/pre-commit`, which runs
`script/precommit`.