1 Commits
Author SHA1 Message Date
sneak 8e39e1b293 Resolve scan operands to their real paths (closes #144)
check / check (push) Canceled after 0s
scan now resolves every operand with its symlinks followed before the
walk, and uses that real path for pruning, walking, records and
warnings. A relative operand is joined to the working directory as the
kernel reports it, so a ".." after a symlink counts from its target, as
with realpath. A symlink operand is scanned at its target instead of
skipped. An operand given through a symlink below another operand, to a
directory outside it, is walked on its own instead of dropped with its
records deleted. An operand that cannot be resolved is a fatal error
naming the operand as given. The walk itself still follows no symlink.

Model: opus-5-5
2026-11-06 18:06:05 +00:00
5 changed files with 113 additions and 26 deletions

No files matched your search

+6 -6
View File
@@ -352,9 +352,10 @@ default. An operand may be a directory or a regular file.
target does not exist. `scan` then stops before it walks anything, and exits target does not exist. `scan` then stops before it walks anything, and exits
with code 1. with code 1.
- Each operand is resolved to its real path before the walk, as `realpath` - Each operand is resolved to its real path before the walk, as `realpath`
prints it. The path is made absolute, and every symlink in it is followed. prints it. A relative operand is joined to the working directory as `pwd -P`
From then on the operand is that real path. Its files are recorded under it, prints it. Every symlink in the path is then followed. A `..` after a symlink
and warnings name it. counts from the directory the symlink leads to. From then on the operand is
that real path. Its files are recorded under it, and warnings name it.
- An operand that is a symlink is scanned at its target's real path. After a - An operand that is a symlink is scanned at its target's real path. After a
`cd` through a symlink, `scan .` records its files under the real path of the `cd` through a symlink, `scan .` records its files under the real path of the
working directory. working directory.
@@ -376,9 +377,8 @@ default. An operand may be a directory or a regular file.
such a directory, so the inner operand is walked on its own. such a directory, so the inner operand is walked on its own.
- No other case keeps an operand under another. - No other case keeps an operand under another.
- An operand given through a symlinked directory below another operand, as in - With `/srv/link` a symlink to a directory outside `/srv`,
`scan /srv /srv/link/sub`, is not under it. It resolves to the real path of `scan /srv /srv/link/sub` walks `sub` on its own, at its real path.
`sub`, which lies outside `/srv`, so it is walked on its own.
- An operand that is neither a regular file nor a directory is not scanned. Such - An operand that is neither a regular file nor a directory is not scanned. Such
an operand is, for example, a symlink to a device node. `scan` prints a an operand is, for example, a symlink to a device node. `scan` prints a
warning naming the real path and what it is. It counts the operand as skipped warning naming the real path and what it is. It counts the operand as skipped
+4 -4
View File
@@ -30,10 +30,10 @@ Newest first. An entry ends with its date and its issue or branch, where those
are known. are known.
- `scan` resolves each operand to its real path, symlinks followed, before the - `scan` resolves each operand to its real path, symlinks followed, before the
walk. A symlink operand is scanned at its target instead of skipped. An walk. A symlink operand is scanned at its target instead of skipped. With
operand given through a symlinked directory below another operand is walked on `/srv/link` a symlink to a directory outside `/srv`, `scan /srv /srv/link/sub`
its own instead of dropped with its records deleted (2026-11-06, walks `sub` on its own instead of dropping it and deleting its records
https://git.eeqj.de/sneak/sfdupes/issues/144) (2026-11-06, https://git.eeqj.de/sneak/sfdupes/issues/144)
- `README.md` and a comment in `scan.go` no longer say ZFS counts a new file's - `README.md` and a comment in `scan.go` no longer say ZFS counts a new file's
blocks only seconds after the write. The count can land in under a second. blocks only seconds after the write. The count can land in under a second.
+74 -2
View File
@@ -14,6 +14,8 @@ import (
"strconv" "strconv"
"strings" "strings"
"testing" "testing"
"golang.org/x/sys/unix"
) )
// usageMarker is the first line of cobra's usage text, which a usage // usageMarker is the first line of cobra's usage text, which a usage
@@ -1024,8 +1026,9 @@ func TestRunScanDropsSymlinkOperandInsideOperand(t *testing.T) {
} }
// TestRunScanDanglingSymlinkOperandIsFatal scans a valid directory and // TestRunScanDanglingSymlinkOperandIsFatal scans a valid directory and
// a symlink whose target does not exist. The scan fails with exit 1 // a symlink whose target does not exist. The scan fails with exit 1 and
// before it walks anything, so no database is created. // an error naming the symlink as given, before it walks anything, so no
// database is created.
func TestRunScanDanglingSymlinkOperandIsFatal(t *testing.T) { func TestRunScanDanglingSymlinkOperandIsFatal(t *testing.T) {
path := testDBPath(t) path := testDBPath(t)
t.Setenv(databaseEnv, path) t.Setenv(databaseEnv, path)
@@ -1048,6 +1051,11 @@ func TestRunScanDanglingSymlinkOperandIsFatal(t *testing.T) {
assertFatalOutput(t, stderr.String(), stdout()) assertFatalOutput(t, stderr.String(), stdout())
want := "sfdupes: resolve " + dangling + ": "
if !strings.Contains(stderr.String(), want) {
t.Errorf("stderr = %q, want %q", stderr.String(), want)
}
_, err := os.Stat(path) _, err := os.Stat(path)
if !errors.Is(err, fs.ErrNotExist) { if !errors.Is(err, fs.ErrNotExist) {
t.Errorf("database: stat error %v, want it missing", err) t.Errorf("database: stat error %v, want it missing", err)
@@ -1083,6 +1091,70 @@ func TestRunScanFromSymlinkedWorkingDirectoryRecordsRealPath(t *testing.T) {
} }
} }
// TestRunScanFollowsSymlinkBeforeDotDot scans a path with a ".." after
// home/proj, a symlink to data/proj: home/proj/.. from dir, and .. after
// a cd through home/proj. As with realpath, the ".." counts from
// data/proj, so data is scanned and home is not.
func TestRunScanFollowsSymlinkBeforeDotDot(t *testing.T) {
for _, tc := range []struct{ wd, operand string }{
{"", "home/proj/.."},
{"home/proj", ".."},
} {
t.Run(tc.operand, func(t *testing.T) {
path := testDBPath(t)
t.Setenv(databaseEnv, path)
dir := realTempDir(t)
f := writeFile(t, dir, "data/proj/f", pattern(1, 10))
g := writeFile(t, dir, "data/g", pattern(2, 10))
writeFile(t, dir, "home/h", pattern(3, 10))
symlink(t, filepath.Join(dir, "data", "proj"),
filepath.Join(dir, "home", "proj"))
// t.Chdir also sets PWD to the path given, as a shell does
// after cd.
t.Chdir(filepath.Join(dir, tc.wd))
scanOK(t, tc.operand)
want := []string{f, g}
slices.Sort(want)
if got := storedPaths(t, path); !slices.Equal(got, want) {
t.Errorf("records = %q, want %q", got, want)
}
})
}
}
// TestRunScanSkipsSymlinkToFIFOOperand scans link, a symlink to a FIFO.
// The scan warns about the FIFO by its real path, counts it as skipped,
// and exits 0.
func TestRunScanSkipsSymlinkToFIFOOperand(t *testing.T) {
t.Setenv(databaseEnv, testDBPath(t))
dir := realTempDir(t)
fifo := filepath.Join(dir, "fifo")
err := unix.Mkfifo(fifo, 0o600)
if err != nil {
t.Fatal(err)
}
link := filepath.Join(dir, "link")
symlink(t, fifo, link)
stderr := scanOK(t, link)
warning := "walk " + fifo + ": skipping FIFO operand\n"
if !strings.Contains(stderr, warning) {
t.Errorf("stderr = %q, want %q", stderr, warning)
}
assertSummary(t, stderr, "scan: 0 files seen (0 added, 0 updated, "+
"0 removed, 0 unchanged), 1 skipped\n")
}
func TestRunScanSkipsZFSOperand(t *testing.T) { func TestRunScanSkipsZFSOperand(t *testing.T) {
zfs := filepath.Join(t.TempDir(), ".zfs") zfs := filepath.Join(t.TempDir(), ".zfs")
f := writeFile(t, zfs, "snapshot/hourly/f", pattern(1, 10)) f := writeFile(t, zfs, "snapshot/hourly/f", pattern(1, 10))
+27 -12
View File
@@ -18,6 +18,8 @@ import (
"sync" "sync"
"syscall" "syscall"
"time" "time"
"golang.org/x/sys/unix"
) )
// The hash phase gives every file it reads three hashes (see // The hash phase gives every file it reads three hashes (see
@@ -219,14 +221,27 @@ func resolveRoots(roots []string) ([]string, error) {
resolved := make([]string, 0, len(roots)) resolved := make([]string, 0, len(roots))
for _, root := range roots { for _, root := range roots {
a, err := filepath.Abs(root) a := root
if err != nil { if !filepath.IsAbs(a) {
return nil, fmt.Errorf("resolve %s: %w", root, err) // The working directory as the kernel reports it, with no
// symlink in it. os.Getwd may return PWD instead, which can
// name it through a symlink.
wd, err := unix.Getwd()
if err != nil {
return nil, fmt.Errorf("resolve %s: %w", root, err)
}
// Not filepath.Abs or filepath.Join: they clean the path,
// which drops each ".." together with the directory name
// before it. EvalSymlinks follows a symlink before it
// applies the ".." after it, as realpath does. The working
// directory ends in "/" only when it is the root.
a = strings.TrimSuffix(wd, "/") + "/" + root
} }
r, err := filepath.EvalSymlinks(a) r, err := filepath.EvalSymlinks(a)
if err != nil { if err != nil {
return nil, err return nil, fmt.Errorf("resolve %s: %w", root, err)
} }
resolved = append(resolved, r) resolved = append(resolved, r)
@@ -420,14 +435,14 @@ func (s *scanState) runPhases(ctx context.Context, roots []string,
// files, and directories not named .zfs, none of them inside a // files, and directories not named .zfs, none of them inside a
// directory named .zfs. The operands are real paths from resolveRoots, // directory named .zfs. The operands are real paths from resolveRoots,
// so these checks apply to what a symlink operand points at. Every // so these checks apply to what a symlink operand points at. Every
// other operand is warned about, counted as // other operand is warned about, counted as skipped, and dropped. A
// skipped, and dropped. A dropped operand is no longer a root, so the // dropped operand is no longer a root, so the records stored beneath it
// records stored beneath it are treated as if it had not been given: // are treated as if it had not been given: those under no other root
// those under no other root count as outside the roots and are not // count as outside the roots and are not deleted as unverified. An
// deleted as unverified. An operand on a filesystem in rules that is not // operand on a filesystem in rules that is not persistent is dropped
// persistent is dropped the same way, with a line saying so that is not // the same way, with a line saying so that is not a warning, and is not
// a warning, and is not counted as skipped. An operand that fails lstat // counted as skipped. An operand that fails lstat here is kept, and the
// here is kept, and the walk warns about it unless it no longer exists. // walk warns about it unless it no longer exists.
func (s *scanState) walkableRoots(roots []string, rules walkRules) []string { func (s *scanState) walkableRoots(roots []string, rules walkRules) []string {
kept := make([]string, 0, len(roots)) kept := make([]string, 0, len(roots))
+2 -2
View File
@@ -2314,8 +2314,8 @@ func TestUnderRoot(t *testing.T) {
// the scan resolves to the same real path. The walk passes over the // the scan resolves to the same real path. The walk passes over the
// database file, its -wal and -shm files and its .lock file with no // database file, its -wal and -shm files and its .lock file with no
// line and no count, and the record an earlier scan left for the .lock // line and no count, and the record an earlier scan left for the .lock
// file is deleted. A rescan of the unchanged tree // file is deleted. A rescan of the unchanged tree updates nothing, and
// updates nothing, and report lists none of the database's files. // report lists none of the database's files.
func TestRunScanPassesOverDatabase(t *testing.T) { func TestRunScanPassesOverDatabase(t *testing.T) {
for _, viaSymlink := range []bool{false, true} { for _, viaSymlink := range []bool{false, true} {
t.Run(fmt.Sprintf("via symlink %v", viaSymlink), func(t *testing.T) { t.Run(fmt.Sprintf("via symlink %v", viaSymlink), func(t *testing.T) {