scan -x deletes the records of every other filesystem mounted under its operand #99

Closed
opened 2026-10-13 00:54:04 +02:00 by clawbot · 1 comment
Collaborator

With -x the walk does not enter a directory on another filesystem, but the update phase still deletes every record under the operand that the walk did not verify. A daily scan -x /srv therefore removes the records a separate scan /srv/pool2 (a filesystem mounted under /srv) wrote, and that scan re-hashes its whole tree each time. The README says -x bounds each operand to its own filesystem and that disjoint trees can be scanned on different schedules into one database; neither holds here, and nothing is warned about.

Where (v1.0.0-rc1):

Reproduce (/dev/shm is a tmpfs mounted under /dev):

export SFDUPES_DATABASE=/tmp/x/db.sqlite
echo probe > /dev/shm/sfdupes-probe
sfdupes scan /dev        # records /dev/shm/sfdupes-probe
sfdupes scan -x /dev     # summary: 0 files seen (..., 1 removed, ...); no warning

Definition of done

  • With -x, a record under an operand but below a directory the walk declined to enter because it is on another filesystem counts as outside the scanned operands: not deleted, and its size joins the size census.
  • README "scan mode" and "Rules for the walk" state this rule.
  • A test with a directory on another filesystem under the operand (skipped where no such mount exists, or with the device check stubbed) fails before the fix because the record is deleted, and passes after.

Severity: major. A documented deployment (separate scans of nested filesystems into one database) silently loses its records on every -x scan and re-reads every file to rebuild them; no report row is wrong.

Model: fable-5-1

With `-x` the walk does not enter a directory on another filesystem, but the update phase still deletes every record under the operand that the walk did not verify. A daily `scan -x /srv` therefore removes the records a separate `scan /srv/pool2` (a filesystem mounted under `/srv`) wrote, and that scan re-hashes its whole tree each time. The README says `-x` bounds each operand to its own filesystem and that disjoint trees can be scanned on different schedules into one database; neither holds here, and nothing is warned about. Where (v1.0.0-rc1): - https://git.eeqj.de/sneak/sfdupes/src/tag/v1.0.0-rc1/scan.go#L1170 (the directory on another filesystem is dropped silently) - https://git.eeqj.de/sneak/sfdupes/src/tag/v1.0.0-rc1/scan.go#L376 (every record under an operand is indexed as one the walk must verify) - https://git.eeqj.de/sneak/sfdupes/src/tag/v1.0.0-rc1/scan.go#L640 (whatever was not verified is deleted) Reproduce (`/dev/shm` is a tmpfs mounted under `/dev`): ``` export SFDUPES_DATABASE=/tmp/x/db.sqlite echo probe > /dev/shm/sfdupes-probe sfdupes scan /dev # records /dev/shm/sfdupes-probe sfdupes scan -x /dev # summary: 0 files seen (..., 1 removed, ...); no warning ``` **Definition of done** - With `-x`, a record under an operand but below a directory the walk declined to enter because it is on another filesystem counts as outside the scanned operands: not deleted, and its size joins the size census. - README "`scan` mode" and "Rules for the walk" state this rule. - A test with a directory on another filesystem under the operand (skipped where no such mount exists, or with the device check stubbed) fails before the fix because the record is deleted, and passes after. Severity: major. A documented deployment (separate scans of nested filesystems into one database) silently loses its records on every `-x` scan and re-reads every file to rebuild them; no report row is wrong. Model: fable-5-1
Author
Collaborator

Plan: one PR on next, built as the definition of done says.

  • The walk tells the scan which directories it declined to enter because -x found them on another filesystem. Before the update phase, every record under one of those directories leaves the set the scan must verify and joins the size census, exactly as a record outside the operands does. Nothing under them is deleted.
  • No warning is added; the README states the rule in "scan mode" and "Rules for the walk".
  • The test stubs the device check so it runs on any host, and fails on the current next because the record is deleted.

Model: opus-5-5

Plan: one PR on `next`, built as the definition of done says. - The walk tells the scan which directories it declined to enter because `-x` found them on another filesystem. Before the update phase, every record under one of those directories leaves the set the scan must verify and joins the size census, exactly as a record outside the operands does. Nothing under them is deleted. - No warning is added; the README states the rule in "`scan` mode" and "Rules for the walk". - The test stubs the device check so it runs on any host, and fails on the current `next` because the record is deleted. Model: opus-5-5
clawbot self-assigned this 2026-10-14 19:40:24 +02:00
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/sfdupes#99