Limit report and trees to given paths (closes #174) (#191)
check / check (push) Canceled after 0s

This commit was merged in pull request #191.
This commit is contained in:
clawbot committed 2026-11-11 09:37:14 +01:00
1 parent 58d27a6df3
commit d4f90f43ff
8 files changed
+412 -75

No files matched your search

+53 -18
View File
@@ -43,7 +43,9 @@ as opposed to a directory, a symlink or a device node. A symlink is a file that
holds the path of another file and stands in for it. A device node is a file
that stands for a piece of hardware, such as a disk or a terminal. `report`
prints the file-level duplicates. `trees` prints the duplicate directory trees.
`info` prints the database's path, size and record count.
Both cover the whole database, or only the paths given after them, as in
`sfdupes report /srv/photos`. `info` prints the database's path, size and record
count.
The database is `/var/lib/sfdupes/db.sqlite` unless `SFDUPES_DATABASE` names
another file. The first `scan` creates it.
@@ -263,8 +265,8 @@ In order:
```
sfdupes scan [--workers N] [-x] [--no-default-excludes] [--exclude PATH]... PATH...
sfdupes report > dupes.tsv
sfdupes trees > dupetrees.tsv
sfdupes report [PATH...] > dupes.tsv
sfdupes trees [PATH...] > dupetrees.tsv
sfdupes info
sfdupes dwim [--workers N] [-x] [--no-default-excludes] [--exclude PATH]...
sfdupes --version
@@ -691,11 +693,12 @@ and `scan` leaves it ignored.
### `report` mode
`report` reads every record from the database. It takes no positional arguments.
A positional argument is anything on the command line that is not a flag, such
as the paths `scan` takes. `report` never opens a scanned file. It checks that
each path in a group still exists, and looks at no other path. The check is
`lstat`, the system call that looks up a path without following a symlink:
`report` reads every record from the database. It takes optional `PATH`
operands. With none, it uses every record. With one or more, it uses only the
records at or below them (see "Operands of `report` and `trees`"). `report`
never opens a scanned file. It checks that each path in a group still exists,
and looks at no other path. It never looks up an operand. The check is `lstat`,
the system call that looks up a path without following a symlink:
- A path is gone when `lstat` finds that it, or a directory above it, does not
exist.
@@ -727,6 +730,27 @@ with one signature come together, and so do the records of one size. So `report`
holds only the duplicate groups of the size it is reading. Once it reaches a
smaller size, it writes them, ordered by `first` path.
#### Operands of `report` and `trees`
`report` and `trees` take the same `PATH` operands:
- A record is used when its path is an operand or lies below one. `/srv/photos`
covers `/srv/photos/x` but not `/srv/photos2/x`.
- The records used are grouped, ordered and totalled as they are without
operands. A copy outside the operands is not reported. With copies `/a/f`,
`/srv/photos/x` and `/srv/photos/y`, `sfdupes report /srv/photos` prints one
row, `/srv/photos/x` and `/srv/photos/y`.
- A relative operand is joined to the working directory as `pwd -P` prints it.
- Each operand is then cleaned as text, as `realpath -ms` prints it. Its `.`
elements and repeated or trailing slashes are dropped. Each `..` drops the
element before it.
- No operand is looked up on the filesystem. A symlink in an operand is not
followed, and an operand need not exist. The records hold real paths (see
"Operands" under "`scan` mode"), so an operand that passes through a symlink
matches no record.
- An operand that matches no record is not an error; it adds nothing to the
report.
#### Report output format
The report is TSV (tab-separated values) on stdout. It holds a header line, then
@@ -754,7 +778,8 @@ Grouping and ordering use the stored path. The warnings about skipped paths that
The summary line on stderr gives:
- the records read;
- the records read, which are all the records in the database, with or without
operands;
- the duplicate groups;
- the dupe files, which is the number of rows;
- the reclaimable bytes, the sum of `size` over all rows.
@@ -765,11 +790,12 @@ report: 14 records read, 4 duplicate groups, 7 dupe files, 10.0 KiB reclaimable
### `trees` mode
`trees` reads the same database as `report` and takes no positional arguments.
It reports whole duplicate directory trees. Two directories are duplicate trees
when they hold the same relative paths, and each path holds a file with the same
signature in both. It rebuilds the directory tree from the paths in the records,
split on `/`. Like `report`, it never opens a scanned file. It checks only that
`trees` reads the same database as `report` and takes the same `PATH` operands
(see "Operands of `report` and `trees`"). It reports whole duplicate directory
trees. Two directories are duplicate trees when they hold the same relative
paths, and each path holds a file with the same signature in both. It rebuilds
the directory tree from the paths in the records, split on `/`. Like `report`,
it never opens a scanned file and never looks up an operand. It checks only that
the directory at the top of each copy still exists, as `report` checks a path.
Definitions:
@@ -797,6 +823,11 @@ Processing:
directories that all have the same digest. The parents' group, or an
ancestor's, already implies it. A group is always reported when it holds
sibling directories, or when its members' parents have different digests.
- With operands, `trees` builds the tree from the records it uses. It reports
only directories at or below an operand. Any other directory lies above an
operand and can hold files that are not used, so its digest does not describe
it. Such a directory is never reported and never implies a group of its
children.
- A directory that is gone is dropped from its group. A group left with fewer
than two directories is not reported.
- Within a group, paths are sorted in byte order. The first is `first`, and
@@ -850,7 +881,8 @@ directory's path is `/`.
The summary line on stderr gives:
- the records read;
- the records read, which are all the records in the database, with or without
operands;
- the duplicate-tree groups;
- the dupe trees, which is the number of rows;
- the reclaimable bytes, the sum of `size` over all rows.
@@ -862,8 +894,10 @@ trees: 14 records read, 1 duplicate tree groups, 1 dupe trees, 3.0 KiB reclaimab
### `dwim` mode
`dwim`, short for "do what I mean", scans a new machine and reports its
duplicate trees in one command. It takes no positional arguments. Like `scan`,
it should run as root, so that it can read every file.
duplicate trees in one command. It takes no positional arguments. A positional
argument is anything on the command line that is not a flag, such as the paths
`scan` takes. Like `scan`, it should run as root, so that it can read every
file.
`dwim` takes these flags of `scan`, with the same meaning (see "Settings"):
@@ -1265,7 +1299,8 @@ showing the count it stopped at.
- `scan` with no operand;
- `--workers` below 1;
- an empty `--exclude` value, as in `--exclude ""`;
- `report`, `trees`, `info` or `dwim` with a positional argument.
- an empty `PATH` for `report` or `trees`, as in `sfdupes report ""`;
- `info` or `dwim` with a positional argument.
A fatal error is reported on stderr, starting with `sfdupes:`. A failed write to
stdout, such as a full disk, is reported the same way. Two cases never reach