Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0d9442c12d |
@@ -67,12 +67,11 @@ The binary goes to `$(go env GOPATH)/bin`, or to `$GOBIN` when that is set.
|
||||
|
||||
`sfdupes --version` prints `sfdupes VERSION` and exits. For a binary from
|
||||
`go install`, `VERSION` is the version Go recorded in it. For `@latest` that is
|
||||
a release tag such as `v0.0.3`. For `@main` it is the commit's tag when it has
|
||||
one. Otherwise it is a pseudo-version, Go's version string for an untagged
|
||||
commit. That string ends in the commit's hash. A binary built from a clone
|
||||
carries the output of `git describe --tags --always --dirty` in that clone, such
|
||||
as `v0.0.3` on a tagged commit. `VERSION` is `dev` when no version is known. To
|
||||
see everything Go recorded in an installed binary:
|
||||
a release tag such as `v0.0.3`. For `@main` it is a pseudo-version, Go's version
|
||||
string for an untagged commit. That string ends in the commit's hash. A binary
|
||||
built from a clone carries the output of `git describe --tags --always --dirty`
|
||||
in that clone, such as `v0.0.3` on a tagged commit. `VERSION` is `dev` when no
|
||||
version is known. To see everything Go recorded in an installed binary:
|
||||
|
||||
```sh
|
||||
go version -m "$(command -v sfdupes)"
|
||||
@@ -202,9 +201,9 @@ two ends and at 1 MiB per gigabyte. A full read of a small file is cheap, and it
|
||||
makes the comparison exact. Reading samples cuts a 150 GiB file to about 150 MiB
|
||||
of reads, at the price of a comparison that is not exact.
|
||||
|
||||
The hashes live in a database that is kept between runs. A rescan reads only new
|
||||
files, and files whose size changed or whose modification time (mtime) is newer
|
||||
than recorded. All analysis then works from the database alone.
|
||||
The hashes live in a database that is kept between runs. A rescan reads only the
|
||||
files that are new or whose size or modification time (mtime) changed. All
|
||||
analysis then works from the database alone.
|
||||
|
||||
The goal is whole duplicated trees, not only single files. A download extracted
|
||||
twice, or a project copied once too often, is such a tree. It can be removed as
|
||||
@@ -367,8 +366,7 @@ default. An operand may be a directory or a regular file.
|
||||
symlinked directory below the outer operand is dropped too, without a line.
|
||||
The outer operand's walk does not follow the symlink, so no file under the
|
||||
inner operand is scanned. The records that an earlier scan left under it are
|
||||
deleted. With `-x`, a directory operand reached that way is kept when the
|
||||
symlink leads to another filesystem than the outer operand.
|
||||
deleted.
|
||||
- An operand that is neither a regular file nor a directory is not scanned.
|
||||
`scan` prints a warning naming the path and what it is. It counts the operand
|
||||
as skipped and drops it. An operand beneath a symlink operand is scanned all
|
||||
@@ -413,10 +411,10 @@ operands:
|
||||
|
||||
Such records count as outside the scanned operands. So with `-x`, a
|
||||
filesystem mounted under an operand can be scanned on its own schedule into
|
||||
the same database. An operand may itself lie at or inside a default exclude.
|
||||
With `-x`, it may also lie on another filesystem than an operand above it.
|
||||
Such an operand is walked on its own. Its walk verifies the records below
|
||||
it, so they are deleted when their files are gone.
|
||||
the same database. An operand may lie at or inside a default exclude, or,
|
||||
with `-x`, at or inside a directory on another filesystem. Such an operand
|
||||
is walked on its own. Its walk verifies the records below it, so they are
|
||||
deleted when their files are gone.
|
||||
|
||||
#### Phases
|
||||
|
||||
@@ -512,7 +510,7 @@ walk practical on large or busy storage pools.
|
||||
described under "What a scan does to the database".
|
||||
|
||||
- Below an operand, the walk does not enter a default exclude. The default
|
||||
excludes hold temporary files and the kernel's view of the running system, so
|
||||
excludes hold the kernel's view of the running system and temporary files, so
|
||||
that `sfdupes scan /` walks only the files worth comparing. They depend on the
|
||||
system, and `sfdupes scan --help` lists those of the system it runs on. On
|
||||
Linux they are:
|
||||
@@ -611,8 +609,8 @@ signal `kill` sends by default:
|
||||
- The line `scan: interrupted after N files` goes to stderr. `N` is the number
|
||||
of files the walk reached. The exit code is 1.
|
||||
|
||||
The next scan reads none of the unchanged files already recorded, and brings the
|
||||
database up to date as usual.
|
||||
The next scan reads none of the files already recorded and brings the database
|
||||
up to date as usual.
|
||||
|
||||
After the first signal `scan` stops catching signals. A second one ends it at
|
||||
once, as any uncaught signal does. The records not yet written are lost, and the
|
||||
@@ -903,15 +901,13 @@ Only one `scan` at a time can hold the lock.
|
||||
#### Schema
|
||||
|
||||
The schema is the database's layout, its table and its index.
|
||||
`PRAGMA user_version` holds the schema version, currently 1. An empty database
|
||||
is at version 0, and `scan` sets it up. Any other version is a fatal error.
|
||||
`report`, `trees` and `info` set nothing up, so for them version 0 is a fatal
|
||||
error too. `scan` creates the table and the index and sets the version in one
|
||||
transaction (see "`info` mode"). So the three land together or not at all. A
|
||||
first scan stopped while doing so leaves an empty database, which the next scan
|
||||
sets up. A database at version 0 that already has a `files` table was not made
|
||||
by `sfdupes`. Every subcommand refuses it, with an error telling you to remove
|
||||
the file and rescan.
|
||||
`PRAGMA user_version` holds the schema version, currently 1. A database with any
|
||||
other version is a fatal error. `scan` creates the table and the index and sets
|
||||
the version in one transaction (see "`info` mode"). So the three land together
|
||||
or not at all. A first scan stopped while doing so leaves an empty database,
|
||||
which the next scan sets up. A database at version 0 that already has a `files`
|
||||
table was not made by `sfdupes`. Every subcommand refuses it, with an error
|
||||
telling you to remove the file and rescan.
|
||||
|
||||
```sql
|
||||
CREATE TABLE files (
|
||||
@@ -941,9 +937,9 @@ CREATE INDEX files_signature ON files (size, head, tail, content);
|
||||
|
||||
### Progress
|
||||
|
||||
Every subcommand shows that it is working: each phase gets a display of its own
|
||||
on stderr from the moment it starts. The display names the phase and shows its
|
||||
progress. The parts of its line are listed below.
|
||||
Every subcommand always shows that it is working. Each phase gets a display of
|
||||
its own on stderr from the moment it starts. The display names the phase and
|
||||
shows its progress. The parts of its line are listed below.
|
||||
|
||||
The phases, and what each display counts:
|
||||
|
||||
@@ -1074,9 +1070,8 @@ the count it stopped at.
|
||||
- a database that cannot be created or opened, or whose reads or writes
|
||||
fail, its directory and its lock file included;
|
||||
- a relative database path when the working directory cannot be read;
|
||||
- a database with another schema version, or one at version 0 that `scan` is
|
||||
not setting up: a database read by `report`, `trees` or `info`, or one
|
||||
that already has a `files` table (see "Schema");
|
||||
- a database with another schema version, or one at version 0 that already
|
||||
has a `files` table;
|
||||
- a record with a hash that is not 32 bytes, read by `report` or `trees`;
|
||||
- a missing database for `report`, `trees` or `info`;
|
||||
- a failed write to stdout.
|
||||
@@ -1192,8 +1187,8 @@ the repository root itself, so it can be run from any directory.
|
||||
every check passed. The tests and the linter run twice, once in `script/check`
|
||||
and again as stages of the image build.
|
||||
- `script/precommit` runs from the `git` pre-commit hook, a script that `git`
|
||||
runs before each commit. It runs `go mod tidy`, and fails the commit when that
|
||||
changes `go.mod` or `go.sum`. Then it runs `script/check`.
|
||||
runs before each commit. It fails the commit when `go mod tidy` would change
|
||||
anything. Then it runs `script/check`.
|
||||
- `script/install-precommit` installs `.git/hooks/pre-commit`, which runs
|
||||
`script/precommit`.
|
||||
|
||||
|
||||
Reference in new issue
Block a user