Compare commits

..
Author SHA1 Message Date
clawbot 56301dac74 dwim: take the scan flags (closes #162)
check / check (push) Canceled after 0s
dwim now takes --workers, -x, --no-default-excludes and --exclude with
the meaning they have for scan. One scanOptions struct, one function
that adds the flags and one PreRunE check serve both commands, and
runScan takes the struct instead of four separate values. The dwim help
lists the default excludes, as the scan help does.

Model: opus-5-5
2026-11-11 05:44:05 +00:00
6 changed files with 25 additions and 182 deletions

No files matched your search

+6 -7
View File
@@ -22,8 +22,9 @@ records the files it finds in an SQLite database. SQLite keeps a whole database
in one file. The database is kept between runs. A rescan reads only new and
changed files. That makes `scan` cheap enough to run every day from `cron`, the
system's job scheduler. `report`, `trees` and `info` work from the database and
can run at any time. `dwim` scans `/` and then prints the `trees` report, for a
first look at a new machine with one command.
can run at any time. `report` and `trees` leave out a copy that was deleted
after the scan. `dwim` scans `/` and then prints the `trees` report, for a first
look at a new machine with one command.
## Getting Started
@@ -315,11 +316,9 @@ The reference below is also the program's specification.
- `--version`, `-v` prints `sfdupes VERSION` (see "Install") to stdout and
exits 0. It writes nothing to stderr.
- `--help`, `-h`, alone or after a subcommand, prints the help text to stderr
and exits 0. It writes nothing to stdout. The help of `sfdupes` and of `scan`
names the database. The help of `report`, `trees`, `info` and `dwim` says what
each prints. `sfdupes scan --help` and `sfdupes dwim --help` also list the
default excludes of the system they run on, and the filesystem types the walk
never records anything on.
and exits 0. It writes nothing to stdout. `sfdupes scan --help` and
`sfdupes dwim --help` also list the default excludes of the system they run
on, and the filesystem types the walk never records anything on.
### Duplicate detection
-5
View File
@@ -33,11 +33,6 @@ are known.
`--exclude`, with the same meaning. Both commands register and check them with
the same code (2026-11-11, https://git.eeqj.de/sneak/sfdupes/issues/162)
- The help of `sfdupes` and of `scan` names the database, its default path and
`SFDUPES_DATABASE`. The help of `report` and `trees` lists the columns, the
order of the rows and the escapes. The help of `info` lists its lines. The
help of `dwim` says that it scans `/`, should run as root and keeps the scan
in the database (2026-11-11, https://git.eeqj.de/sneak/sfdupes/issues/177)
- New `scan` flag `--exclude PATH`, which may be repeated. The walk neither
enters nor records an excluded path, and keeps the records below it. An
operand at or below one is a fatal error (2026-11-11,
+11 -82
View File
@@ -32,69 +32,6 @@ const (
cmdDwim = "dwim"
)
// databaseHelp is the paragraph of the root and scan help that names
// the database (README "Location").
const databaseHelp = "scan and dwim store what they find in a database " +
"file. report, trees\nand info read it. The file is $" + databaseEnv +
" when that is set and\nnot empty, and otherwise " +
defaultDatabasePath + ", where creating it\nneeds root.\n"
// escapesHelp is the paragraph of the report and trees help on how
// paths are escaped (README "Report output format").
const escapesHelp = `Four characters in a path are escaped, so each row stays one line:
\\ a backslash
\t a tab
\n a newline
\r a carriage return
`
// reportHelp, treesHelp, infoHelp and dwimHelp follow the one-line
// summary in the help of their subcommands.
const (
reportHelp = `The report is tab-separated text on stdout: a header line, then one
row per duplicate file. The columns are:
first the path that sorts first, byte by byte, in its group of copies
dupe another copy of that file
size the file's size in bytes
Rows are ordered by size, largest first, then by path.
` + escapesHelp
treesHelp = `The report is tab-separated text on stdout: a header line, then one
row per duplicate directory tree. The columns are:
first the path that sorts first, byte by byte, in its group of copies
dupe another copy of that tree
files the number of regular files in one copy
size their total size in bytes
disk_size the bytes the dupe copy takes on disk
Rows are ordered by size, largest first, then by path.
` + escapesHelp
infoHelp = `The details go to stdout, one per line. Each line holds a name and its
value, separated by a tab:
database the path, and where it came from
database file size in bytes, then rounded, as 2147483648 (2.0 GiB)
-wal file size in bytes, then rounded; only if a -wal file exists
modified when the database file last changed, in UTC
schema version the version of the database's layout
records the number of records
size of recorded files their total size, in bytes, then rounded
`
dwimHelp = `dwim runs what "sfdupes scan /" runs with the same flags, then what
"sfdupes trees" runs. Run it as root, so that the scan can read every
file. The scan is stored in the database, as scan stores it, so
"sfdupes trees" prints the report again later.
`
)
// errNoSubcommand is returned by the root command when it is invoked
// without a subcommand. That is a usage error, and the usage text
// cobra prints for it is the whole message.
@@ -255,11 +192,6 @@ func newRootCommand(stdout, stderr io.Writer) *cobra.Command {
}),
}
root.Long = root.Short + ".\n\n" + databaseHelp
reportCmd.Long = reportCmd.Short + ".\n\n" + reportHelp
treesCmd.Long = treesCmd.Short + ".\n\n" + treesHelp
infoCmd.Long = infoCmd.Short + ".\n\n" + infoHelp
root.AddCommand(newScanCommand(), reportCmd, treesCmd, infoCmd,
newDwimCommand(stdout))
@@ -306,9 +238,9 @@ func (o *scanOptions) check(cmd *cobra.Command, _ []string) error {
return rejectEmptyExcludeValue(cmd, o.excludePaths)
}
// newScanCommand builds the scan subcommand. Its help names the
// database, and lists the default excludes of the system it runs on and
// the filesystem types it never records anything on.
// newScanCommand builds the scan subcommand. Its help lists the default
// excludes of the system it runs on, and the filesystem types it never
// records anything on.
func newScanCommand() *cobra.Command {
var opts scanOptions
@@ -320,10 +252,9 @@ func newScanCommand() *cobra.Command {
// append its generic "[flags]" to it.
DisableFlagsInUseLine: true,
Short: short,
Long: short + ".\n\n" + databaseHelp + "\n" +
defaultExcludesHelp(runtime.GOOS),
Args: cobra.MinimumNArgs(1),
PreRunE: opts.check,
Long: short + ".\n\n" + defaultExcludesHelp(runtime.GOOS),
Args: cobra.MinimumNArgs(1),
PreRunE: opts.check,
RunE: runE(func(ctx context.Context, args []string) error {
ctx, stop := interruptContext(ctx)
defer stop()
@@ -337,9 +268,8 @@ func newScanCommand() *cobra.Command {
}
// newDwimCommand builds the dwim subcommand, which takes the scan flags
// with the meaning they have for scan. Its help says what dwim runs,
// then lists the default excludes and the filesystem types the scan
// never records anything on.
// with the meaning they have for scan. Its help lists what the scan
// help lists.
func newDwimCommand(stdout io.Writer) *cobra.Command {
var opts scanOptions
@@ -349,10 +279,9 @@ func newDwimCommand(stdout io.Writer) *cobra.Command {
Use: cmdDwim + " " + scanFlagsUsage,
DisableFlagsInUseLine: true,
Short: short,
Long: short + ".\n\n" + dwimHelp + "\n" +
defaultExcludesHelp(runtime.GOOS),
Args: cobra.NoArgs,
PreRunE: opts.check,
Long: short + ".\n\n" + defaultExcludesHelp(runtime.GOOS),
Args: cobra.NoArgs,
PreRunE: opts.check,
RunE: runE(func(ctx context.Context, _ []string) error {
return runDwim(ctx, "/", opts, stdout)
}),
+1 -71
View File
@@ -22,9 +22,6 @@ import (
// error prints and a runtime failure must not.
const usageMarker = "Usage:"
// helpFlag asks for the help text.
const helpFlag = "--help"
// workersFlag is the scan and dwim flag that sets the worker count.
const workersFlag = "--workers"
@@ -699,7 +696,7 @@ func TestRunHelp(t *testing.T) {
// README §Settings: help goes to stderr, exits 0, and leaves stdout
// empty.
cases := [][]string{{helpFlag}, {"-h"}, {cmdScan, helpFlag}}
cases := [][]string{{"--help"}, {"-h"}, {cmdScan, "--help"}}
for _, args := range cases {
var stdout, stderr bytes.Buffer
@@ -760,73 +757,6 @@ func TestRunScanUsageLine(t *testing.T) {
}
}
func TestRunHelpDescribesDatabaseAndOutput(t *testing.T) {
t.Parallel()
// README §Settings: the help names the database, and says what
// report, trees, info and dwim print.
database := []string{
"scan and dwim store", databaseEnv, defaultDatabasePath, "needs root",
}
escapes := []string{`\\`, `\t`, `\n`, `\r`}
// The subcommand whose help is printed; none for the root.
cases := []struct {
subcommand []string
want []string
}{
{nil, database},
{[]string{cmdScan}, database},
{
[]string{cmdReport},
append([]string{
"tab-separated", "\n first ", "\n dupe ", "\n size ",
"largest first",
}, escapes...),
},
{
[]string{cmdTrees},
append([]string{
"tab-separated", "\n first ", "\n dupe ", "\n files ",
"\n size ", "\n disk_size ", "largest first",
}, escapes...),
},
{
[]string{cmdInfo},
[]string{
"\n database ", "\n database file size ",
"\n -wal file size ", "\n modified ",
"\n schema version ", "\n records ",
"\n size of recorded files ",
},
},
{
[]string{cmdDwim},
[]string{
"sfdupes scan /", "as root", "sfdupes trees", "database",
},
},
}
for _, tc := range cases {
var stdout, stderr bytes.Buffer
args := slices.Concat(tc.subcommand, []string{helpFlag})
code := run(args, &stdout, &stderr)
if code != exitOK {
t.Fatalf("run(%v) = %d, want %d", args, code, exitOK)
}
for _, want := range tc.want {
if !strings.Contains(stderr.String(), want) {
t.Errorf("run(%v) help lacks %q:\n%s",
args, want, stderr.String())
}
}
}
}
func TestRunVersion(t *testing.T) {
t.Parallel()
-9
View File
@@ -5,7 +5,6 @@ import (
"errors"
"os"
"path/filepath"
"runtime"
"strings"
"testing"
)
@@ -199,12 +198,4 @@ func TestRunDwimHelpListsScanFlags(t *testing.T) {
t.Errorf("dwim -h stderr = %q, want %s PATH", stderr.String(),
excludeFlag)
}
// README "Settings": the help lists the default excludes and the
// filesystem types the walk never records anything on.
help := strings.TrimSpace(defaultExcludesHelp(runtime.GOOS))
if !strings.Contains(stderr.String(), help) {
t.Errorf("dwim -h stderr = %q, want it to hold %q", stderr.String(),
help)
}
}
+7 -8
View File
@@ -99,14 +99,13 @@ func scanPhase(name string) string {
// the SIGINT or SIGTERM that interruptContext catches, the scan keeps
// what it has hashed (see syncScan), prints how many files its walk
// reached, and returns errInterrupted. opts.workers must be at least 1;
// the scan and dwim commands reject anything less. Before the lock is
// taken, the scan lists the filesystems that are not persistent, on
// which the walk records nothing; unless opts.noDefaultExcludes is set,
// the walk does not enter the default excludes either. The walk neither
// enters nor records the paths in opts.excludePaths, the --exclude
// values; an operand at or below one is a fatal error. Once the
// database is open, the scan notes its files, which the walk passes
// over.
// the scan command rejects anything less. Before the lock is taken, the
// scan lists the filesystems that are not persistent, on which the walk
// records nothing; unless opts.noDefaultExcludes is set, the walk does
// not enter the default excludes either. The walk neither enters nor
// records the paths in opts.excludePaths, the --exclude values; an
// operand at or below one is a fatal error. Once the database is open,
// the scan notes its files, which the walk passes over.
func runScan(ctx context.Context, roots []string, opts scanOptions) error {
dbPath, source, err := databasePath()
if err != nil {