Compare commits

..
Author SHA1 Message Date
sneak bcd373da7a Add scan --exclude PATH (closes #163)
check / check (push) Successful in 2m43s
The walk neither enters nor records an --exclude path or anything
below it, and prints "scan: skipped PATH (--exclude)" for each one it
reaches. The flag repeats. The walk knows each --exclude path by
device ID and inode number, as it knows the database files, so it also
leaves one out when it reaches it by another path, such as a hard
link. The records below an excluded path are kept, as for a default
exclude. An empty value is a usage error. An --exclude path whose real
path scan cannot find, or an operand at or below one, is a fatal error
before the scan starts. walkRules names its two lists excludePaths and
defaultExcludes. printDatabasePath, split out of runScan, keeps runScan
under the lint length limit.

Model: opus-5-5
2026-11-11 05:19:08 +00:00
4 changed files with 15 additions and 75 deletions

No files matched your search

+12 -22
View File
@@ -129,8 +129,8 @@ walk"). As the file `/etc/cron.d/sfdupes`:
- one line naming the database; - one line naming the database;
- progress lines, as described under "Progress"; - progress lines, as described under "Progress";
- a warning for each path it skips; - a warning for each path it skips;
- a line for each system, temporary or cache directory the walk leaves out - a line for each system or temporary directory the walk leaves out (see
(see "Rules for the walk"); "Rules for the walk");
- a line for each path left out because its filesystem is one the kernel, - a line for each path left out because its filesystem is one the kernel,
the core of the operating system, makes up or keeps in memory, such as the core of the operating system, makes up or keeps in memory, such as
`tmpfs`; `tmpfs`;
@@ -302,9 +302,8 @@ The reference below is also the program's specification.
directories on other filesystems, as the same flag does for `du` and `rsync`. directories on other filesystems, as the same flag does for `du` and `rsync`.
By default the walk also enters directories on other filesystems. By default the walk also enters directories on other filesystems.
- `--no-default-excludes` (`scan`) lets the walk enter the default excludes. - `--no-default-excludes` (`scan`) lets the walk enter the default excludes.
Those are the system, temporary and cache directories the walk otherwise Those are the system and temporary directories the walk otherwise leaves out,
leaves out, such as `/var/tmp` and `/var/cache` on Linux (see "Rules for the such as `/var/tmp` on Linux (see "Rules for the walk").
walk").
- `--exclude PATH` (`scan`) keeps the walk out of `PATH` and everything below - `--exclude PATH` (`scan`) keeps the walk out of `PATH` and everything below
it. `PATH` may be a directory or a file. The flag may be given more than once. it. `PATH` may be a directory or a file. The flag may be given more than once.
The walk knows `PATH` by its device ID and inode number, as it knows the The walk knows `PATH` by its device ID and inode number, as it knows the
@@ -571,18 +570,13 @@ walk practical on large or busy storage pools.
it, as described under "What a scan does to the database". it, as described under "What a scan does to the database".
`--no-default-excludes` does not change this rule. `--no-default-excludes` does not change this rule.
- Below an operand, the walk does not enter a default exclude. The default - Below an operand, the walk does not enter a default exclude. On Linux, the
excludes hold the kernel's view of the running system, temporary files, and default excludes hold temporary files and the kernel's view of the running
caches. A cache here is data that programs and package managers download or system. They depend on the system, and `sfdupes scan --help` lists those of
build again on their own. The default excludes depend on the system, and the system it runs on. On Linux they are:
`sfdupes scan --help` lists those of the system it runs on. In the lists, `*`
stands for any one directory name, so `/home/*/.cache` covers each user's
cache. On Linux they are:
``` ```
/proc, /sys, /dev, /run, /tmp, /var/tmp, /var/cache /proc, /sys, /dev, /run, /tmp, /var/tmp
/root/.cache, /root/.npm, /root/.cargo/registry, /root/go/pkg/mod
/home/*/.cache, /home/*/.npm, /home/*/.cargo/registry, /home/*/go/pkg/mod
``` ```
On macOS they are: On macOS they are:
@@ -590,15 +584,11 @@ walk practical on large or busy storage pools.
``` ```
/dev, /private/tmp, /private/var/tmp, /private/var/vm /dev, /private/tmp, /private/var/tmp, /private/var/vm
/System/Volumes/VM, /System/Volumes/Data /System/Volumes/VM, /System/Volumes/Data
/private/var/folders, /Library/Caches
/Users/*/Library/Caches, /Users/*/.cache, /Users/*/.npm
/Users/*/.cargo/registry, /Users/*/go/pkg/mod
``` ```
`/private/var/folders` holds each user's temporary files and caches on A walk of `/` on macOS reaches everything in `/System/Volumes/Data` already,
macOS. A walk of `/` on macOS reaches everything in `/System/Volumes/Data` through firmlinks. Firmlinks are the links macOS uses to show the data
already, through firmlinks. Firmlinks are the links macOS uses to show the volume inside the system volume. Walking `/System/Volumes/Data` as well
data volume inside the system volume. Walking `/System/Volumes/Data` as well
would record each of those files twice. Any other system has no default would record each of those files twice. Any other system has no default
excludes. excludes.
-6
View File
@@ -29,12 +29,6 @@
Newest first. An entry ends with its date and its issue or branch, where those Newest first. An entry ends with its date and its issue or branch, where those
are known. are known.
- The default excludes cover caches as well: `/var/cache`, and each user's
`.cache`, `.npm`, `.cargo/registry` and `go/pkg/mod`, root's included. On
macOS they add `/private/var/folders`, `/Library/Caches` and each user's
`Library/Caches` and the same four. `--no-default-excludes` still turns them
all off (2026-11-11, https://git.eeqj.de/sneak/sfdupes/issues/181)
- New `scan` flag `--exclude PATH`, which may be repeated. The walk neither - New `scan` flag `--exclude PATH`, which may be repeated. The walk neither
enters nor records an excluded path, and keeps the records below it. An enters nor records an excluded path, and keeps the records below it. An
operand at or below one is a fatal error (2026-11-11, operand at or below one is a fatal error (2026-11-11,
+2 -15
View File
@@ -72,27 +72,14 @@ type walkRules struct {
func defaultExcludes(goos string) []string { func defaultExcludes(goos string) []string {
switch goos { switch goos {
case goosLinux: case goosLinux:
// The kernel's view of the running system, temporary files, and return []string{"/proc", "/sys", "/dev", "/run", "/tmp", "/var/tmp"}
// caches that programs and package managers fill again on their
// own: the system's, root's and each user's.
return []string{
"/proc", "/sys", "/dev", "/run", "/tmp", "/var/tmp", "/var/cache",
"/root/.cache", "/root/.npm", "/root/.cargo/registry",
"/root/go/pkg/mod",
"/home/*/.cache", "/home/*/.npm", "/home/*/.cargo/registry",
"/home/*/go/pkg/mod",
}
case goosDarwin: case goosDarwin:
// A walk of / reaches everything in /System/Volumes/Data through // A walk of / reaches everything in /System/Volumes/Data through
// the firmlinks, so walking it as well would record each of // the firmlinks, so walking it as well would record each of
// those files twice. /private/var/folders holds each user's // those files twice.
// temporary files and caches.
return []string{ return []string{
"/dev", "/private/tmp", "/private/var/tmp", "/private/var/vm", "/dev", "/private/tmp", "/private/var/tmp", "/private/var/vm",
"/System/Volumes/VM", "/System/Volumes/Data", "/System/Volumes/VM", "/System/Volumes/Data",
"/private/var/folders", "/Library/Caches",
"/Users/*/Library/Caches", "/Users/*/.cache", "/Users/*/.npm",
"/Users/*/.cargo/registry", "/Users/*/go/pkg/mod",
} }
case goosWindows: case goosWindows:
// Relative to the root of the drive an operand lies on. The // Relative to the root of the drive an operand lies on. The
+1 -32
View File
@@ -24,19 +24,10 @@ func TestDefaultExcludes(t *testing.T) {
// on any other system. The Windows list exists in the code only: the // on any other system. The Windows list exists in the code only: the
// binary does not build for Windows, so the README leaves it out. // binary does not build for Windows, so the README leaves it out.
cases := map[string][]string{ cases := map[string][]string{
goosLinux: { goosLinux: {"/proc", "/sys", "/dev", "/run", "/tmp", "/var/tmp"},
"/proc", "/sys", "/dev", "/run", "/tmp", "/var/tmp", "/var/cache",
"/root/.cache", "/root/.npm", "/root/.cargo/registry",
"/root/go/pkg/mod",
"/home/*/.cache", "/home/*/.npm", "/home/*/.cargo/registry",
"/home/*/go/pkg/mod",
},
goosDarwin: { goosDarwin: {
"/dev", "/private/tmp", "/private/var/tmp", "/private/var/vm", "/dev", "/private/tmp", "/private/var/tmp", "/private/var/vm",
"/System/Volumes/VM", "/System/Volumes/Data", "/System/Volumes/VM", "/System/Volumes/Data",
"/private/var/folders", "/Library/Caches",
"/Users/*/Library/Caches", "/Users/*/.cache", "/Users/*/.npm",
"/Users/*/.cargo/registry", "/Users/*/go/pkg/mod",
}, },
goosWindows: { goosWindows: {
"pagefile.sys", "swapfile.sys", "hiberfil.sys", "pagefile.sys", "swapfile.sys", "hiberfil.sys",
@@ -72,28 +63,6 @@ func TestIsDefaultExclude(t *testing.T) {
t.Errorf("isDefaultExclude(%q) = %v, want %v", path, got, want) t.Errorf("isDefaultExclude(%q) = %v, want %v", path, got, want)
} }
} }
// The Linux list matches each user's cache directory itself, and no
// .cache directory elsewhere.
linux := map[string]bool{
"/home/u/.cache": true,
"/home/u/go/pkg/mod": true,
"/root/.cache": true,
"/var/cache": true,
"/home/u/.cache/pip": false,
"/home/u/src/.cache": false,
"/srv/home/u/.cache": false,
"/home/u/.cargo/bin": false,
"/home/u/go/pkg/mod/cache": false,
}
for path, want := range linux {
got := isDefaultExclude(path, defaultExcludes(goosLinux))
if got != want {
t.Errorf("isDefaultExclude(%q, Linux list) = %v, want %v", path,
got, want)
}
}
} }
func TestReadMountInfo(t *testing.T) { func TestReadMountInfo(t *testing.T) {