An operand reached through a symlink below another operand is skipped without a line #144

Closed
opened 2026-10-26 13:50:45 +01:00 by clawbot · 4 comments
Collaborator

sfdupes scan /srv /srv/link/sub, with /srv/link a symlink to a directory, skips /srv/link/sub without a line. The walk of /srv does not follow the symlink, and the second operand is dropped as covered by the first. The records an earlier scan left for files under /srv/link/sub are then deleted.

Question: what should scan do with such an operand?

  • Recommended: walk it on its own, so its files are scanned once and keep their records. The README already says an operand the outer walk does not reach is walked on its own.
  • Or: skip it with a warning, as for an operand that is itself a symlink, and keep its records.

Model: opus-5-5

`sfdupes scan /srv /srv/link/sub`, with `/srv/link` a symlink to a directory, skips `/srv/link/sub` without a line. The walk of `/srv` does not follow the symlink, and the second operand is dropped as covered by the first. The records an earlier scan left for files under `/srv/link/sub` are then deleted. Question: what should `scan` do with such an operand? - Recommended: walk it on its own, so its files are scanned once and keep their records. The README already says an operand the outer walk does not reach is walked on its own. - Or: skip it with a warning, as for an operand that is itself a symlink, and keep its records. Model: opus-5-5
sneak was assigned by clawbot 2026-10-26 13:50:45 +01:00
Author
Collaborator

Proposal, not yet a ruling: sneak asked in chat whether every path argument should be resolved to its real path, with symlinks followed, before the scan begins. If he says yes:

  • sfdupes scan /srv /srv/link/sub resolves the second argument to the directory the symlink points at; that directory is not under /srv, so it is scanned on its own and its files keep their records.
  • An argument that is itself a symlink is followed and scanned, instead of skipped with a warning as today.
  • Two arguments that resolve to the same directory, or one inside another, are scanned once.
  • Files are recorded and shown under their resolved paths, not the paths typed.
  • The .zfs rule (#142 (comment)) applies to the resolved path: a symlink into a .zfs directory is still never entered.
  • During the walk itself, symlinks are still not followed.

Nothing is built until he answers.

Model: opus-5-5

Proposal, not yet a ruling: sneak asked in chat whether every path argument should be resolved to its real path, with symlinks followed, before the scan begins. If he says yes: - `sfdupes scan /srv /srv/link/sub` resolves the second argument to the directory the symlink points at; that directory is not under `/srv`, so it is scanned on its own and its files keep their records. - An argument that is itself a symlink is followed and scanned, instead of skipped with a warning as today. - Two arguments that resolve to the same directory, or one inside another, are scanned once. - Files are recorded and shown under their resolved paths, not the paths typed. - The `.zfs` rule (https://git.eeqj.de/sneak/sfdupes/issues/142#issuecomment-155846) applies to the resolved path: a symlink into a `.zfs` directory is still never entered. - During the walk itself, symlinks are still not followed. Nothing is built until he answers. Model: opus-5-5
sneak was unassigned by clawbot 2026-10-26 16:47:36 +01:00
clawbot self-assigned this 2026-10-26 16:47:36 +01:00
clawbot removed their assignment 2026-10-26 16:48:34 +01:00
sneak was assigned by clawbot 2026-10-26 16:48:34 +01:00
Author
Collaborator

Ruled by sneak in chat on 6 November: yes. Every path argument is resolved to its real path, symlinks followed, before the scan starts, as proposed in #144 (comment). This is now clawbot's, for a worker once a worker account resets.

Model: opus-5-5

Ruled by sneak in chat on 6 November: yes. Every path argument is resolved to its real path, symlinks followed, before the scan starts, as proposed in https://git.eeqj.de/sneak/sfdupes/issues/144#issuecomment-156122. This is now `clawbot`'s, for a worker once a worker account resets. Model: opus-5-5
sneak was unassigned by clawbot 2026-11-06 18:21:15 +01:00
clawbot self-assigned this 2026-11-06 18:21:15 +01:00
Author
Collaborator

For the worker: plan, from sneak's ruling in #144 (comment) on the proposal in #144 (comment).

Branch off current next; PR into next, title ending (closes #144). Everything stays in package main, in the existing files named below; no new files.

Change (scan.go):

  • resolveRoots makes each operand absolute as now, then resolves it with filepath.EvalSymlinks. The resolved path is the operand from then on: it is pruned, walked, recorded and printed under that path.
  • An operand that cannot be resolved is the existing fatal error before anything is walked, exit 1. That covers a missing path, a dangling symlink, a symlink loop and an unreadable component. Decision taken: a dangling symlink names nothing, so it counts as an operand that does not exist.
  • An operand that is a symlink to a directory or regular file is scanned at the target's real path. One whose target is a device node, socket or FIFO gets the existing warning, naming the resolved path.
  • Repeats and nesting are judged on resolved paths, so two operands that resolve to the same directory, or one inside the other, are walked once. pruneRoots and walkReaches need no change for this.
  • The .zfs check in operandWarning looks at the resolved path, which is now the operand itself; drop the separate EvalSymlinks call there. A symlink into a .zfs directory is still never entered (#142 (comment)).
  • Keep the symlink case in operandWarning: seedRoot still needs it for an operand that turns into a symlink after it was resolved.
  • The walk is unchanged: it follows no symlink it finds.
  • Update the doc comments of resolveRoots, operandWarning and walkableRoots to match.
  • No code for records an earlier scan stored under a path through a symlink: the repo is a prototype with no installed base.

README (README.md), then make fmt:

  • "Operands": the bullet on making operands absolute (resolved now, symlinks followed); the bullets on repeats and nesting (judged on resolved paths); the "No other case" bullet about an operand reached through a symlinked directory (it now resolves to its real path and is walked on its own unless another operand's walk reaches that path); the symlink-operand sentences, including the cd through a symlink example; the .zfs bullet (checked on the resolved path).
  • "Rules for the walk", the database-files paragraph: each operand is opened at its resolved path, not the path given.
  • Every other sentence that describes a symlink operand. Leave "Database" alone: the database path is resolved separately.

Tests (main_test.go, scan_test.go):

  • Rewrite TestRunScanSkipsSymlinkOperand, TestRunScanWalksOperandUnderSymlinkOperand, TestRunScanWalksOperandThroughSymlink and TestRunScanSkipsOperandThroughSymlinkIntoZFS for the new behaviour, and fix any other test the change breaks. Keep TestWalkFileAndSymlinkOperands: it covers the operand that turns into a symlink after it was resolved.
  • New: the issue's case. scan D D/link/sub with D/link a symlink to a directory outside D records sub's file once, under its real path. A record left there by an earlier scan of the real path survives.
  • New: a symlink operand pointing inside another operand is dropped as covered, and its files are recorded once.
  • New: a dangling symlink operand fails with exit 1 and walks nothing.
  • New: scan . after a cd through a symlink records under the real path.
  • Compare against filepath.EvalSymlinks(t.TempDir()), so the tests also hold where the temp directory runs through a symlink.

Done when the above is in, make check passes on a fresh clone, and the PR body says what changed for a user in plain words.

Model: opus-5-5

For the worker: plan, from sneak's ruling in https://git.eeqj.de/sneak/sfdupes/issues/144#issuecomment-169642 on the proposal in https://git.eeqj.de/sneak/sfdupes/issues/144#issuecomment-156122. Branch off current `next`; PR into `next`, title ending ` (closes #144)`. Everything stays in package `main`, in the existing files named below; no new files. Change (`scan.go`): - `resolveRoots` makes each operand absolute as now, then resolves it with `filepath.EvalSymlinks`. The resolved path is the operand from then on: it is pruned, walked, recorded and printed under that path. - An operand that cannot be resolved is the existing fatal error before anything is walked, exit 1. That covers a missing path, a dangling symlink, a symlink loop and an unreadable component. Decision taken: a dangling symlink names nothing, so it counts as an operand that does not exist. - An operand that is a symlink to a directory or regular file is scanned at the target's real path. One whose target is a device node, socket or FIFO gets the existing warning, naming the resolved path. - Repeats and nesting are judged on resolved paths, so two operands that resolve to the same directory, or one inside the other, are walked once. `pruneRoots` and `walkReaches` need no change for this. - The `.zfs` check in `operandWarning` looks at the resolved path, which is now the operand itself; drop the separate `EvalSymlinks` call there. A symlink into a `.zfs` directory is still never entered (https://git.eeqj.de/sneak/sfdupes/issues/142#issuecomment-155846). - Keep the `symlink` case in `operandWarning`: `seedRoot` still needs it for an operand that turns into a symlink after it was resolved. - The walk is unchanged: it follows no symlink it finds. - Update the doc comments of `resolveRoots`, `operandWarning` and `walkableRoots` to match. - No code for records an earlier scan stored under a path through a symlink: the repo is a prototype with no installed base. README (`README.md`), then `make fmt`: - "Operands": the bullet on making operands absolute (resolved now, symlinks followed); the bullets on repeats and nesting (judged on resolved paths); the "No other case" bullet about an operand reached through a symlinked directory (it now resolves to its real path and is walked on its own unless another operand's walk reaches that path); the symlink-operand sentences, including the `cd` through a symlink example; the `.zfs` bullet (checked on the resolved path). - "Rules for the walk", the database-files paragraph: each operand is opened at its resolved path, not the path given. - Every other sentence that describes a symlink operand. Leave "Database" alone: the database path is resolved separately. Tests (`main_test.go`, `scan_test.go`): - Rewrite `TestRunScanSkipsSymlinkOperand`, `TestRunScanWalksOperandUnderSymlinkOperand`, `TestRunScanWalksOperandThroughSymlink` and `TestRunScanSkipsOperandThroughSymlinkIntoZFS` for the new behaviour, and fix any other test the change breaks. Keep `TestWalkFileAndSymlinkOperands`: it covers the operand that turns into a symlink after it was resolved. - New: the issue's case. `scan D D/link/sub` with `D/link` a symlink to a directory outside `D` records `sub`'s file once, under its real path. A record left there by an earlier scan of the real path survives. - New: a symlink operand pointing inside another operand is dropped as covered, and its files are recorded once. - New: a dangling symlink operand fails with exit 1 and walks nothing. - New: `scan .` after a `cd` through a symlink records under the real path. - Compare against `filepath.EvalSymlinks(t.TempDir())`, so the tests also hold where the temp directory runs through a symlink. Done when the above is in, `make check` passes on a fresh clone, and the PR body says what changed for a user in plain words. Model: opus-5-5
Author
Collaborator

Built in #155: scan resolves every path argument to its real path, symlinks followed, before the walk. The case in this issue now scans sub on its own and keeps its records. A dangling symlink argument is a fatal error.

Model: opus-5-5

Built in https://git.eeqj.de/sneak/sfdupes/pulls/155: `scan` resolves every path argument to its real path, symlinks followed, before the walk. The case in this issue now scans `sub` on its own and keeps its records. A dangling symlink argument is a fatal error. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/sfdupes#144