scan: skip OS-appropriate default excludes (/proc, /sys, /tmp, /run, /var/tmp on Linux) (closes #124)
check / check (push) Canceled after 0s

Below an operand, the walk no longer enters the default excludes: on
Linux /proc, /sys, /dev, /run, /tmp, /var/tmp and every mount point of a
kernel or memory-backed filesystem, read once from /proc/self/mountinfo;
on macOS /dev, /private/tmp, /private/var/tmp, /private/var/vm,
/System/Volumes/VM and /System/Volumes/Data. The lists live in the new
exclude.go. An operand at or inside one is still walked, also under
another operand, as with -x. Each skipped directory gets one stderr line
through the warning path, is not counted as skipped, and its records
count as outside the scanned operands. --no-default-excludes turns them
off; scan --help lists them.

Judgement call: an unreadable /proc/self/mountinfo fails the scan.
Judgement call: where filesystems are mounted over one another, the last decides.

Model: opus-5-5
This commit is contained in:
2026-10-16 23:28:07 +00:00
parent 68f6bde5ee
commit 5af53914e8
9 changed files with 693 additions and 120 deletions
+16 -8
View File
@@ -12,7 +12,7 @@
//
// Usage:
//
// sfdupes scan [--workers N] [-x] PATH...
// sfdupes scan [--workers N] [-x] [--no-default-excludes] PATH...
// sfdupes report > dupes.tsv
// sfdupes trees > dupetrees.tsv
// sfdupes info
@@ -182,17 +182,22 @@ func newRootCommand(stdout, stderr io.Writer) *cobra.Command {
return root
}
// newScanCommand builds the scan subcommand, with its --workers and -x
// flags.
// newScanCommand builds the scan subcommand, with its --workers, -x and
// --no-default-excludes flags. Its help lists the default excludes of
// the system it runs on.
func newScanCommand() *cobra.Command {
var (
scanWorkers int
scanOneFS bool
scanWorkers int
scanOneFS bool
scanNoDefaultExcludes bool
)
short := "Walk trees and synchronize the scan database"
scanCmd := &cobra.Command{
Use: cmdScan + " [--workers N] [-x] PATH...",
Short: "Walk trees and synchronize the scan database",
Use: cmdScan + " [--workers N] [-x] [--no-default-excludes] PATH...",
Short: short,
Long: short + ".\n\n" + defaultExcludesHelp(runtime.GOOS),
Args: cobra.MinimumNArgs(1),
PreRunE: func(cmd *cobra.Command, _ []string) error {
return checkScanWorkers(cmd, scanWorkers)
@@ -201,13 +206,16 @@ func newScanCommand() *cobra.Command {
ctx, stop := interruptContext(ctx)
defer stop()
return runScan(ctx, args, scanWorkers, scanOneFS)
return runScan(ctx, args, scanWorkers, scanOneFS,
scanNoDefaultExcludes)
}),
}
scanCmd.Flags().IntVar(&scanWorkers, "workers", runtime.NumCPU(),
"concurrent workers for the walk, hash, and content phases")
scanCmd.Flags().BoolVarP(&scanOneFS, "one-file-system", "x", false,
"do not cross filesystem boundaries")
scanCmd.Flags().BoolVar(&scanNoDefaultExcludes, "no-default-excludes",
false, "enter the default excludes")
return scanCmd
}