Compare commits
1
Commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
070674e9ee |
@@ -119,8 +119,8 @@ skipped with a warning and loses its database record. As the file
|
||||
- a progress line as each phase starts, one every 5 seconds while it runs,
|
||||
and one as it ends;
|
||||
- a warning for each path it skips;
|
||||
- a line for each path the walk leaves out on purpose (see "Rules for the
|
||||
walk");
|
||||
- a line for each path left out as a default exclude or as not on a
|
||||
persistent filesystem (see "Rules for the walk");
|
||||
- the summary line;
|
||||
- a warning when a report still has the database open as the scan ends (see
|
||||
"Concurrent reads").
|
||||
@@ -381,9 +381,9 @@ operands:
|
||||
- a path that is not on a persistent filesystem;
|
||||
- with `-x`, a directory on another filesystem than its operand.
|
||||
|
||||
A filesystem mounted under an operand can therefore be scanned on its own
|
||||
schedule into the same database. An operand at or inside such a path is
|
||||
walked on its own. Its walk verifies the records below it, so they are
|
||||
With `-x`, a filesystem mounted under an operand can therefore be scanned on
|
||||
its own schedule into the same database. An operand at or inside such a path
|
||||
is walked on its own. Its walk verifies the records below it, so they are
|
||||
deleted when their files are gone.
|
||||
|
||||
#### Phases
|
||||
@@ -422,15 +422,18 @@ walk practical on large or busy storage pools.
|
||||
snapshots, and walking it would list every file once per snapshot.
|
||||
- The walk never records the scan's own database files. Those are the database,
|
||||
its `.lock` file, and the `-wal` and `-shm` files SQLite keeps beside it while
|
||||
the scan runs (see "Concurrent reads"). It knows them by device ID and inode
|
||||
number. The device ID is the number by which the kernel tells filesystems
|
||||
apart, and the inode number tells the files of one filesystem apart. So the
|
||||
walk passes over them however it reaches them, through a symlinked directory
|
||||
included. It prints no line for them and does not count them as skipped. A
|
||||
record an earlier scan left for one of them is deleted.
|
||||
the scan runs (see "Concurrent reads"). The walk knows them by device ID and
|
||||
inode number. The device ID is the number by which the kernel tells
|
||||
filesystems apart, and the inode number tells the files of one filesystem
|
||||
apart. So the walk passes over them whatever path leads to them. An operand
|
||||
whose path runs through a symlinked directory is one such path: the walk
|
||||
follows no symlink it finds, but it opens each operand at the path given. The
|
||||
walk prints no line for the database files and does not count them as skipped.
|
||||
A record an earlier scan left for one of them is deleted.
|
||||
- By default the walk crosses filesystem boundaries. With `-x`, it never enters
|
||||
a directory on a different filesystem than its operand. The records below such
|
||||
a directory are kept, as described under "What a scan does to the database".
|
||||
a directory on a different filesystem than its operand. `scan` prints no line
|
||||
for that directory and does not count it as skipped. The records below it are
|
||||
kept, as described under "What a scan does to the database".
|
||||
- The walk never records anything on a filesystem that is not persistent. A
|
||||
filesystem is not persistent when the kernel makes it up or keeps it in
|
||||
memory. These are the types, as `sfdupes scan --help` lists them on Linux and
|
||||
@@ -477,11 +480,11 @@ walk practical on large or busy storage pools.
|
||||
well would record each of those files twice.
|
||||
- On any other system: none.
|
||||
|
||||
An operand at or inside a default exclude is walked, so
|
||||
`sfdupes scan /tmp/x` works when `/tmp/x` is on a persistent filesystem.
|
||||
`--no-default-excludes` turns the default excludes off. What is not on a
|
||||
persistent filesystem stays left out. An operand at or inside a default
|
||||
exclude is still walked, so `sfdupes scan /tmp/x` works when `/tmp/x` is on
|
||||
a persistent filesystem. For each default exclude the walk does not enter,
|
||||
`scan` prints a line on stderr naming it:
|
||||
persistent filesystem stays left out. For each default exclude the walk does
|
||||
not enter, `scan` prints a line on stderr naming it:
|
||||
|
||||
```
|
||||
scan: skipped /var/tmp (default exclude)
|
||||
@@ -520,9 +523,9 @@ display draws from a thread of its own and never blocks the workers.
|
||||
- one line naming the database (see "Database");
|
||||
- for each dropped operand, in operand order, the warning or the line described
|
||||
under "Operands". A repeated operand, or one under another, gets neither;
|
||||
- the progress display of each phase (see "Progress"). A warning for each path
|
||||
it skips, and a line for each path the walk leaves out on purpose, come as
|
||||
they happen;
|
||||
- the progress display of each phase (see "Progress"). `scan` prints each
|
||||
warning for a skipped path as it happens. It prints a line the same way for
|
||||
each path left out as a default exclude or as not on a persistent filesystem;
|
||||
- the summary line;
|
||||
- a warning when a report still has the database open as the scan ends (see
|
||||
"Concurrent reads").
|
||||
@@ -541,8 +544,8 @@ scan: 123400 files seen (1200 added, 34 updated, 56 removed, 122166 unchanged),
|
||||
`scan` stops cleanly on `SIGINT`, the signal Ctrl-C sends, or `SIGTERM`, the
|
||||
signal `kill` sends by default:
|
||||
|
||||
- The workers stop taking work. Each finishes at most the directory listing or
|
||||
the file it is reading, and the progress display is finished.
|
||||
- The workers stop taking work. Each finishes no more than the directory listing
|
||||
or the file it is reading, and the progress display is finished.
|
||||
- The records hashed but not yet committed are committed, so the next scan does
|
||||
not hash them again.
|
||||
- Nothing else is written or deleted. Records are deleted only after a complete
|
||||
@@ -748,11 +751,11 @@ trees: database /srv/tmp/sfdupes/db.db (from SFDUPES_DATABASE)
|
||||
A relative path is printed with the working directory in front of it, as
|
||||
`pwd -P` prints it, with symlinks resolved. The rest of the path is printed as
|
||||
set, so a `..` in it stays, and the line names the same file. After a `cd`
|
||||
through a symlink, a `../` therefore counts from the directory the symlink leads
|
||||
to, as it does for any other program. The path is escaped like the paths in a
|
||||
report. `scan` says `new database` when no file is at that path as it starts.
|
||||
`report`, `trees` and `info` print the line before they open the database, so
|
||||
the error for a missing database comes after it.
|
||||
through a symlink, a `..` counts from the directory the symlink leads to, as it
|
||||
does for any other program. The printed path shows that directory. The path is
|
||||
escaped like the paths in a report. `scan` says `new database` when no file is
|
||||
at that path as it starts. `report`, `trees` and `info` print the line before
|
||||
they open the database, so the error for a missing database comes after it.
|
||||
|
||||
#### Creation and access
|
||||
|
||||
@@ -1005,7 +1008,7 @@ never reach `sfdupes` as a failed write:
|
||||
|
||||
- The reader of a stdout pipe may exit early, as `head` does in
|
||||
`sfdupes report | head`. The next write then ends `sfdupes` with `SIGPIPE`,
|
||||
the signal for a write to a closed pipe. It ends quietly and without a
|
||||
the signal for a write to a closed pipe. `sfdupes` ends quietly and without a
|
||||
summary, as `cat` or `sort` would. The shell reports the signal, status 141 in
|
||||
most shells, not exit 1.
|
||||
- When stdout is closed outright, as in `sfdupes report >&-`, the Go runtime
|
||||
@@ -1017,9 +1020,9 @@ never reach `sfdupes` as a failed write:
|
||||
This repository follows the
|
||||
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
|
||||
standard. The executables in `script/` are the entrypoints of the development
|
||||
workflow, and each `Makefile` target does nothing but call its script. Every
|
||||
script is POSIX `sh`, finds the repository root itself, and can be run from any
|
||||
directory.
|
||||
workflow, and the `Makefile` targets call them, except `build` and `clean` (see
|
||||
"Build"). Every script is POSIX `sh`, finds the repository root itself, and can
|
||||
be run from any directory.
|
||||
|
||||
- `script/bootstrap` installs what building and developing need. It can be run
|
||||
again at any time; installed tools are kept. `git`, `make` and `go` are
|
||||
@@ -1097,11 +1100,11 @@ install system packages and download the Go modules. Every step that lints,
|
||||
tests or compiles comes after it. Every `RUN` step after it runs again on each
|
||||
build. Without that, Docker would serve the test and lint steps from its cache
|
||||
on an unchanged tree, and the build would pass having run neither. The layers
|
||||
above the line stay cached until the base image, their own lines, `go.mod` or
|
||||
`go.sum` change. Go's build cache is kept between builds, so only what changed
|
||||
is compiled again. The tests run with `-count=1`, so no test reports a stored
|
||||
result. `golangci-lint run` keeps the linter's own cache as well, so an
|
||||
unchanged package is not linted again.
|
||||
above `ARG CACHEBUST` stay cached until the base image, their own lines,
|
||||
`go.mod` or `go.sum` change. Go's build cache is kept between builds, so only
|
||||
what changed is compiled again. The tests run with `-count=1`, so no test
|
||||
reports a stored result. `golangci-lint run` keeps the linter's own cache as
|
||||
well, so an unchanged package is not linted again.
|
||||
|
||||
## Build
|
||||
|
||||
|
||||
Reference in New Issue
Block a user