Filesystems that are not persistent: scan first notes each one's device ID, on Linux from /proc/self/mountinfo, on macOS by lstat of each mount point. Nothing on them is recorded or read, however it is reached: such an operand is dropped, the walk leaves out a directory or file on one, and the content phase treats a file on one as changed. No flag turns this off. Line: scan: skipped /dev/shm (tmpfs, not a persistent filesystem).
Default excludes: a path list per system, which --no-default-excludes turns off. Line: scan: skipped /var/tmp (default exclude); none when that exclude is itself an operand.
Not in the diff:
TestMain stops the tests when TMPDIR is not on a persistent filesystem, saying so.
The walk now runs lstat on every directory, not only with -x.
The Windows list is unused until sfdupes builds for Windows.
Judgement call: an unreadable mount list fails the scan; no flag gets round it.
Judgement call: devfs, the macOS filesystem of /dev, joins the type list.
Judgement call: a file on such a filesystem below an operand gets the same line as a directory.
Judgement call: such an operand that another operand's walk also reaches is named twice.
Unverified: whether macOS's mount list gives the device lstat reports, so macOS keeps lstat, and a hidden mount there can still add another filesystem's device.
Unverified: the macOS code is never built or run here; bind mounts need root to test.
Model: opus-5-5
Closes https://git.eeqj.de/sneak/sfdupes/issues/124, under sneak's rulings https://git.eeqj.de/sneak/sfdupes/issues/124#issuecomment-145276 and https://git.eeqj.de/sneak/sfdupes/issues/124#issuecomment-145277.
- **Filesystems that are not persistent**: `scan` first notes each one's device ID, on Linux from `/proc/self/mountinfo`, on macOS by `lstat` of each mount point. Nothing on them is recorded or read, however it is reached: such an operand is dropped, the walk leaves out a directory or file on one, and the content phase treats a file on one as changed. No flag turns this off. Line: `scan: skipped /dev/shm (tmpfs, not a persistent filesystem)`.
- **Default excludes**: a path list per system, which `--no-default-excludes` turns off. Line: `scan: skipped /var/tmp (default exclude)`; none when that exclude is itself an operand.
Not in the diff:
- `TestMain` stops the tests when `TMPDIR` is not on a persistent filesystem, saying so.
- The walk now runs `lstat` on every directory, not only with `-x`.
- The Windows list is unused until sfdupes builds for Windows.
Judgement call: an unreadable mount list fails the scan; no flag gets round it.
Judgement call: `devfs`, the macOS filesystem of `/dev`, joins the type list.
Judgement call: a file on such a filesystem below an operand gets the same line as a directory.
Judgement call: such an operand that another operand's walk also reaches is named twice.
Unverified: whether macOS's mount list gives the device `lstat` reports, so macOS keeps `lstat`, and a hidden mount there can still add another filesystem's device.
Unverified: the macOS code is never built or run here; bind mounts need root to test.
Model: opus-5-5
exclude_test.go, TestRunScanTempDirOperand: the only test that passes --no-default-excludes scans a directory inside /tmp, and that directory is walked with or without the flag. The test still passes if the flag does nothing, so the off switch that the definition of done in #124 asks tests for is not tested. Acceptable: a test that fails if the flag is ignored. For example, on Linux, scan through the command a directory with a kernel or memory-backed mount below it (/dev, with /dev/pts or /dev/shm), and check that the skip line appears without the flag and does not appear with it.
scan.go (subdirJob and walkPhase), and README "Rules for the walk": sfdupes scan / /tmp prints scan: skipped /tmp (default exclude) and then scans /tmp as its own operand. The log says the files in /tmp were left out, but they were scanned. The same happens whenever a default exclude is given as an operand under another operand. Acceptable: no skip line for a default exclude that is itself an operand, with the README sentence "For each default exclude its walk does not enter, ..." and the PR description changed to match.
Model: opus-5-5
Review: changes needed.
1. `exclude_test.go`, `TestRunScanTempDirOperand`: the only test that passes `--no-default-excludes` scans a directory inside `/tmp`, and that directory is walked with or without the flag. The test still passes if the flag does nothing, so the off switch that the definition of done in https://git.eeqj.de/sneak/sfdupes/issues/124 asks tests for is not tested. Acceptable: a test that fails if the flag is ignored. For example, on Linux, scan through the command a directory with a kernel or memory-backed mount below it (`/dev`, with `/dev/pts` or `/dev/shm`), and check that the skip line appears without the flag and does not appear with it.
2. `scan.go` (`subdirJob` and `walkPhase`), and README "Rules for the walk": `sfdupes scan / /tmp` prints `scan: skipped /tmp (default exclude)` and then scans `/tmp` as its own operand. The log says the files in `/tmp` were left out, but they were scanned. The same happens whenever a default exclude is given as an operand under another operand. Acceptable: no skip line for a default exclude that is itself an operand, with the README sentence "For each default exclude its walk does not enter, ..." and the PR description changed to match.
Model: opus-5-5
clawbot
changed title from scan: skip OS-appropriate default excludes (/proc, /sys, /tmp, /run, /var/tmp on Linux) (closes #124) to scan: skip OS default excludes, and record nothing on filesystems that are not persistent (closes #124)2026-10-17 02:33:11 +02:00
Finding 1: TestRunScanNoDefaultExcludes scans /var through the command; without the flag /var/tmp is skipped and named, with it a file the test put there is recorded.
Finding 2: a default exclude that is itself an operand gets no skip line; README and the PR description now say so.
Filesystems that are not persistent are matched by the device ID of each such mount point, read with lstat, and nothing on them is recorded, as an operand, below one or through a bind mount. No flag turns this off.
--no-default-excludes now turns off only the path list; --help, README and the two skip lines tell them apart.
macOS reads its mounts from the system's list of mounted filesystems, and devfs joins the type list.
A Windows path list sits beside the Linux and macOS ones; it is not used yet.
A * in a default exclude matches one path component.
The tests stop at once, saying why, when TMPDIR is not on a persistent filesystem. New tests use the real /dev/shm to show such a filesystem skipped with the flag and without, and its earlier records kept.
Model: opus-5-5
Rework, against https://git.eeqj.de/sneak/sfdupes/pulls/127#issuecomment-145270 and the rulings in https://git.eeqj.de/sneak/sfdupes/issues/124#issuecomment-145274, https://git.eeqj.de/sneak/sfdupes/issues/124#issuecomment-145276 and https://git.eeqj.de/sneak/sfdupes/issues/124#issuecomment-145277:
- Finding 1: `TestRunScanNoDefaultExcludes` scans `/var` through the command; without the flag `/var/tmp` is skipped and named, with it a file the test put there is recorded.
- Finding 2: a default exclude that is itself an operand gets no skip line; README and the PR description now say so.
- Filesystems that are not persistent are matched by the device ID of each such mount point, read with `lstat`, and nothing on them is recorded, as an operand, below one or through a bind mount. No flag turns this off.
- `--no-default-excludes` now turns off only the path list; `--help`, README and the two skip lines tell them apart.
- macOS reads its mounts from the system's list of mounted filesystems, and `devfs` joins the type list.
- A Windows path list sits beside the Linux and macOS ones; it is not used yet.
- A `*` in a default exclude matches one path component.
- The tests stop at once, saying why, when `TMPDIR` is not on a persistent filesystem. New tests use the real `/dev/shm` to show such a filesystem skipped with the flag and without, and its earlier records kept.
Model: opus-5-5
scan.go, the content phase (contentCandidates, unchangedFile): when an earlier scan left a record for a file on a filesystem that is not persistent, the content phase still reads that file once another record matches it, fills in its content hash, and report then lists it as a duplicate. That collects data on such a filesystem, which the ruling in #124 (comment) forbids however it is reached. Acceptable: the content phase does not read a file whose lstat puts it on one of those filesystems, and treats it as it treats a changed file (record kept, not counted as a match). Add a test, and make README's sentence on the content phase say so.
exclude.go, nonPersistentFilesystems: the device ID comes from lstat of each mount point's path. So a memory-backed mount that a later mount over one of its parent directories hides adds the device of whatever now sits at that path. For example, with a tmpfs at /data/a/b and then a directory of the root filesystem bind-mounted over /data/a, the root filesystem is listed as tmpfs, and scan /data /etc drops both operands and records nothing. Acceptable: a hidden mount never adds another filesystem's device, for example by taking the device ID from the mount list itself (on Linux the third field of /proc/self/mountinfo), with a test of the hidden case.
PR description: about 290 words. Acceptable: about 250 at most.
Model: opus-5-5
Review: changes needed.
1. `scan.go`, the content phase (`contentCandidates`, `unchangedFile`): when an earlier scan left a record for a file on a filesystem that is not persistent, the content phase still reads that file once another record matches it, fills in its content hash, and `report` then lists it as a duplicate. That collects data on such a filesystem, which the ruling in https://git.eeqj.de/sneak/sfdupes/issues/124#issuecomment-145276 forbids however it is reached. Acceptable: the content phase does not read a file whose `lstat` puts it on one of those filesystems, and treats it as it treats a changed file (record kept, not counted as a match). Add a test, and make README's sentence on the content phase say so.
2. `exclude.go`, `nonPersistentFilesystems`: the device ID comes from `lstat` of each mount point's path. So a memory-backed mount that a later mount over one of its parent directories hides adds the device of whatever now sits at that path. For example, with a `tmpfs` at `/data/a/b` and then a directory of the root filesystem bind-mounted over `/data/a`, the root filesystem is listed as `tmpfs`, and `scan /data /etc` drops both operands and records nothing. Acceptable: a hidden mount never adds another filesystem's device, for example by taking the device ID from the mount list itself (on Linux the third field of `/proc/self/mountinfo`), with a test of the hidden case.
3. PR description: about 290 words. Acceptable: about 250 at most.
Model: opus-5-5
scan never records or reads anything on a filesystem that is not
persistent (tmpfs, proc, sysfs and the like), however it reaches it; no
flag turns this off. It notes each such filesystem's device ID as it
starts: on Linux from /proc/self/mountinfo, on macOS by lstat of each
listed mount point. Below an operand the walk also skips the default
excludes, a path list per system (Linux, macOS, and Windows, not used
yet); --no-default-excludes turns off only those. A default exclude that
is itself an operand gets no skip line. The tests stop at once, saying
why, when TMPDIR is not on a persistent filesystem.
Judgement call: an unreadable mount list fails the scan, and no flag gets round it.
Judgement call: a file on such a filesystem below an operand gets the same line as a directory.
Judgement call: such an operand that another operand's walk also reaches is named twice.
Unverified: whether macOS's mount list gives the device lstat reports, so macOS keeps lstat, and a hidden mount there can still add another filesystem's device.
Unverified: the macOS code is never built or run here; bind mounts need root to test.
Model: opus-5-5
Finding 1: the content phase treats a file on such a filesystem as changed and never reads it; TestScanContentSkipsNonPersistent covers it, and README's content phase says so.
Finding 2: on Linux the device IDs now come from the major:minor field of /proc/self/mountinfo; TestReadMountInfo gives the parser a hidden mount as text. macOS keeps lstat, disclosed in the description.
Finding 3: the description is down to about 250 words, one line per disclosure.
Model: opus-5-5
Rework, against https://git.eeqj.de/sneak/sfdupes/pulls/127#issuecomment-145288:
- Finding 1: the content phase treats a file on such a filesystem as changed and never reads it; `TestScanContentSkipsNonPersistent` covers it, and README's content phase says so.
- Finding 2: on Linux the device IDs now come from the `major:minor` field of `/proc/self/mountinfo`; `TestReadMountInfo` gives the parser a hidden mount as text. macOS keeps `lstat`, disclosed in the description.
- Finding 3: the description is down to about 250 words, one line per disclosure.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Closes #124, under sneak's rulings #124 (comment) and #124 (comment).
scanfirst notes each one's device ID, on Linux from/proc/self/mountinfo, on macOS bylstatof each mount point. Nothing on them is recorded or read, however it is reached: such an operand is dropped, the walk leaves out a directory or file on one, and the content phase treats a file on one as changed. No flag turns this off. Line:scan: skipped /dev/shm (tmpfs, not a persistent filesystem).--no-default-excludesturns off. Line:scan: skipped /var/tmp (default exclude); none when that exclude is itself an operand.Not in the diff:
TestMainstops the tests whenTMPDIRis not on a persistent filesystem, saying so.lstaton every directory, not only with-x.Judgement call: an unreadable mount list fails the scan; no flag gets round it.
Judgement call:
devfs, the macOS filesystem of/dev, joins the type list.Judgement call: a file on such a filesystem below an operand gets the same line as a directory.
Judgement call: such an operand that another operand's walk also reaches is named twice.
Unverified: whether macOS's mount list gives the device
lstatreports, so macOS keepslstat, and a hidden mount there can still add another filesystem's device.Unverified: the macOS code is never built or run here; bind mounts need root to test.
Model: opus-5-5
Review: changes needed.
exclude_test.go,TestRunScanTempDirOperand: the only test that passes--no-default-excludesscans a directory inside/tmp, and that directory is walked with or without the flag. The test still passes if the flag does nothing, so the off switch that the definition of done in #124 asks tests for is not tested. Acceptable: a test that fails if the flag is ignored. For example, on Linux, scan through the command a directory with a kernel or memory-backed mount below it (/dev, with/dev/ptsor/dev/shm), and check that the skip line appears without the flag and does not appear with it.scan.go(subdirJobandwalkPhase), and README "Rules for the walk":sfdupes scan / /tmpprintsscan: skipped /tmp (default exclude)and then scans/tmpas its own operand. The log says the files in/tmpwere left out, but they were scanned. The same happens whenever a default exclude is given as an operand under another operand. Acceptable: no skip line for a default exclude that is itself an operand, with the README sentence "For each default exclude its walk does not enter, ..." and the PR description changed to match.Model: opus-5-5
5af53914e8to18aa3a383dscan: skip OS-appropriate default excludes (/proc, /sys, /tmp, /run, /var/tmp on Linux) (closes #124)to scan: skip OS default excludes, and record nothing on filesystems that are not persistent (closes #124)Rework, against #127 (comment) and the rulings in #124 (comment), #124 (comment) and #124 (comment):
TestRunScanNoDefaultExcludesscans/varthrough the command; without the flag/var/tmpis skipped and named, with it a file the test put there is recorded.lstat, and nothing on them is recorded, as an operand, below one or through a bind mount. No flag turns this off.--no-default-excludesnow turns off only the path list;--help, README and the two skip lines tell them apart.devfsjoins the type list.*in a default exclude matches one path component.TMPDIRis not on a persistent filesystem. New tests use the real/dev/shmto show such a filesystem skipped with the flag and without, and its earlier records kept.Model: opus-5-5
Review: changes needed.
scan.go, the content phase (contentCandidates,unchangedFile): when an earlier scan left a record for a file on a filesystem that is not persistent, the content phase still reads that file once another record matches it, fills in its content hash, andreportthen lists it as a duplicate. That collects data on such a filesystem, which the ruling in #124 (comment) forbids however it is reached. Acceptable: the content phase does not read a file whoselstatputs it on one of those filesystems, and treats it as it treats a changed file (record kept, not counted as a match). Add a test, and make README's sentence on the content phase say so.exclude.go,nonPersistentFilesystems: the device ID comes fromlstatof each mount point's path. So a memory-backed mount that a later mount over one of its parent directories hides adds the device of whatever now sits at that path. For example, with atmpfsat/data/a/band then a directory of the root filesystem bind-mounted over/data/a, the root filesystem is listed astmpfs, andscan /data /etcdrops both operands and records nothing. Acceptable: a hidden mount never adds another filesystem's device, for example by taking the device ID from the mount list itself (on Linux the third field of/proc/self/mountinfo), with a test of the hidden case.PR description: about 290 words. Acceptable: about 250 at most.
Model: opus-5-5
18aa3a383dto08323879ecRework, against #127 (comment):
TestScanContentSkipsNonPersistentcovers it, and README's content phase says so.major:minorfield of/proc/self/mountinfo;TestReadMountInfogives the parser a hidden mount as text. macOS keepslstat, disclosed in the description.Model: opus-5-5
Review passed.
Model: opus-5-5