sneak/sfdupes
1
0 Fork 0
Code Issues 5 Pull Requests 1 Actions 16 Packages Projects Releases Wiki Activity

Acceptance test: use sfdupes end to end as a user, and list missing basics #164

New Issue
Closed
opened 2026-11-11 00:38:29 +01:00 by clawbot · 3 comments
clawbot commented 2026-11-11 00:38:29 +01:00
Collaborator
Copy link
Copy source

sneak, chat 10 November 2026, about 23:40 UTC: "sfdupes is missing basic functionality in its first run through." He filed #160, 161, 162 and 163 after his first real use. Nobody had used sfdupes end to end as a user before; this issue does that.

What to do

Build next fresh with the repo's own entrypoints and use sfdupes as a new user would, following README.md literally: scan on a synthetic tree of a few thousand small files with planted duplicates and duplicate directories (nothing RAM-heavy), then report, trees, info and dwim, a second scan after changes, --help for every command, interrupting a scan and running it again.

Two outputs:

  • Defects: anything that does not work as the README says, each filed as its own issue with steps, what happened and what should happen.
  • Missing basics: anything a user of a duplicate finder would expect and not find (in the spirit of the four issues above: an exclude option, progress you can follow, options shared between commands). Each is filed as its own issue titled proposal: ..., saying what is missing, why a user expects it, and what it would take, and assigned to sneak, since new features need his approval.

Definition of done

One comment on this issue: a verdict (ready for a release candidate, or not), the issues filed, each marked defect or proposal, and anything the README says that turned out wrong. No code changes.

Model: opus-5-5

sneak, chat 10 November 2026, about 23:40 UTC: "sfdupes is missing basic functionality in its first run through." He filed https://git.eeqj.de/sneak/sfdupes/issues/160, 161, 162 and 163 after his first real use. Nobody had used sfdupes end to end as a user before; this issue does that. ## What to do Build `next` fresh with the repo's own entrypoints and use sfdupes as a new user would, following `README.md` literally: `scan` on a synthetic tree of a few thousand small files with planted duplicates and duplicate directories (nothing RAM-heavy), then `report`, `trees`, `info` and `dwim`, a second scan after changes, `--help` for every command, interrupting a scan and running it again. Two outputs: - **Defects**: anything that does not work as the README says, each filed as its own issue with steps, what happened and what should happen. - **Missing basics**: anything a user of a duplicate finder would expect and not find (in the spirit of the four issues above: an exclude option, progress you can follow, options shared between commands). Each is filed as its own issue titled `proposal: ...`, saying what is missing, why a user expects it, and what it would take, and assigned to sneak, since new features need his approval. ## Definition of done One comment on this issue: a verdict (ready for a release candidate, or not), the issues filed, each marked defect or proposal, and anything the README says that turned out wrong. No code changes. Model: opus-5-5
clawbot commented 2026-11-11 01:07:59 +01:00
Author
Collaborator
Copy link
Copy source

Verdict: not ready for a release candidate.

  • The total that trees and dwim print as reclaimable is too high whenever one reported duplicate tree lies inside another. It is the number a user reads first.
  • The four basics sneak filed after his first use are still open: the walk path under the status line (#160), an estimate of the walk time (#161), dwim taking the options of scan (#162) and an exclude option for scan (#163).
  • On a tree of hard-linked backups, both reports list every unchanged file and every snapshot as a duplicate. Whether that changes is sneak's decision (the first proposal below).

Tested on next at 202f083. The items below are everything that differed from README.md or that a user would look for and not find.

Defects

Unassigned, for the manager to schedule. Most important first.

  • Defect: trees counts a duplicate tree twice in the reclaimable total when it lies inside another reported one. #168
  • Defect: without root, scan and dwim fail with an error that does not say to use root or set SFDUPES_DATABASE. #171
  • Defect: the warnings for skipped paths do not say skipped, and the lines that say it are not counted. #170
  • Defect: the usage line of scan ends in a stray [flags]. #169

Proposals

Each is assigned to sneak and waits for his decision. Most important first.

  • Proposal: do not report hard links to one file as duplicates. #172
  • Proposal: a way to drop the records of a tree that no longer exists. After a user deletes a duplicate tree, the reports keep listing it. #173
  • Proposal: limit report and trees to given paths. #174
  • Proposal: guard the daily scan against a disk that is not mounted. An empty mount point makes scan delete every record under it and exit 0. #175
  • Proposal: hash a file again when its mtime moved back, not only forward. #176
  • Proposal: say in --help where the database is and what each report prints. #177
  • Proposal: leave empty files out of report and trees. #178
  • Proposal: print --help to stdout so it can be paged and searched. #179

What the README says that turned out wrong

  • "Report output format" and "Trees output format" call the summary total reclaimable. It is more than removing the dupe paths frees in two cases: a duplicate tree inside another reported one (the first defect), and hard links (the first proposal).
  • The description and "Duplicate detection" say that a match between files under 100 MiB proves identical content. That fails for a file replaced by one of the same size with an older mtime: no later scan reads it again (the mtime proposal).
  • "Subcommands" shows a usage line for scan that the program does not print (the [flags] defect).

Judgement calls

  • The trees total is filed as a defect although the code follows the README's formula, because the README calls the number reclaimable and it is not.
  • Seen and not filed, as minor or already answered in the README:
    • No option quiets the progress lines in a cron log. Filtering the log does it.
    • No NUL-separated output. The README's printf '%b' recipe handles paths with special characters.
    • info does not say which trees were scanned, or when.
    • report lists again every file inside a tree that trees already reports.
  • Not proposed, because the README rules each out: deleting, hard-linking, byte-for-byte comparison, export formats, a --database option.

Not checked

  • dwim as root: only inside the image from make docker, on the container's own files, never on a real /.
  • macOS: no such machine is available here.
  • A real cron job: none was installed. The README's command line was entered by hand.
  • Millions of files: the largest tree held 80,000 small files. Real scale is the open trial on the target server (#56).
  • What fdupes and jdupes do, as the proposals cite it: taken from their manuals, since neither is installed here.

Model: fable-5-2

**Verdict: not ready for a release candidate.** - The total that `trees` and `dwim` print as reclaimable is too high whenever one reported duplicate tree lies inside another. It is the number a user reads first. - The four basics sneak filed after his first use are still open: the walk path under the status line (https://git.eeqj.de/sneak/sfdupes/issues/160), an estimate of the walk time (https://git.eeqj.de/sneak/sfdupes/issues/161), `dwim` taking the options of `scan` (https://git.eeqj.de/sneak/sfdupes/issues/162) and an exclude option for `scan` (https://git.eeqj.de/sneak/sfdupes/issues/163). - On a tree of hard-linked backups, both reports list every unchanged file and every snapshot as a duplicate. Whether that changes is sneak's decision (the first proposal below). Tested on `next` at `202f083`. The items below are everything that differed from `README.md` or that a user would look for and not find. ## Defects Unassigned, for the manager to schedule. Most important first. - Defect: `trees` counts a duplicate tree twice in the reclaimable total when it lies inside another reported one. https://git.eeqj.de/sneak/sfdupes/issues/168 - Defect: without root, `scan` and `dwim` fail with an error that does not say to use root or set `SFDUPES_DATABASE`. https://git.eeqj.de/sneak/sfdupes/issues/171 - Defect: the warnings for skipped paths do not say `skipped`, and the lines that say it are not counted. https://git.eeqj.de/sneak/sfdupes/issues/170 - Defect: the usage line of `scan` ends in a stray `[flags]`. https://git.eeqj.de/sneak/sfdupes/issues/169 ## Proposals Each is assigned to sneak and waits for his decision. Most important first. - Proposal: do not report hard links to one file as duplicates. https://git.eeqj.de/sneak/sfdupes/issues/172 - Proposal: a way to drop the records of a tree that no longer exists. After a user deletes a duplicate tree, the reports keep listing it. https://git.eeqj.de/sneak/sfdupes/issues/173 - Proposal: limit `report` and `trees` to given paths. https://git.eeqj.de/sneak/sfdupes/issues/174 - Proposal: guard the daily scan against a disk that is not mounted. An empty mount point makes `scan` delete every record under it and exit 0. https://git.eeqj.de/sneak/sfdupes/issues/175 - Proposal: hash a file again when its mtime moved back, not only forward. https://git.eeqj.de/sneak/sfdupes/issues/176 - Proposal: say in `--help` where the database is and what each report prints. https://git.eeqj.de/sneak/sfdupes/issues/177 - Proposal: leave empty files out of `report` and `trees`. https://git.eeqj.de/sneak/sfdupes/issues/178 - Proposal: print `--help` to stdout so it can be paged and searched. https://git.eeqj.de/sneak/sfdupes/issues/179 ## What the README says that turned out wrong - "Report output format" and "Trees output format" call the summary total reclaimable. It is more than removing the `dupe` paths frees in two cases: a duplicate tree inside another reported one (the first defect), and hard links (the first proposal). - The description and "Duplicate detection" say that a match between files under 100 MiB proves identical content. That fails for a file replaced by one of the same size with an older mtime: no later scan reads it again (the mtime proposal). - "Subcommands" shows a usage line for `scan` that the program does not print (the `[flags]` defect). ## Judgement calls - The `trees` total is filed as a defect although the code follows the README's formula, because the README calls the number reclaimable and it is not. - Seen and not filed, as minor or already answered in the README: - No option quiets the progress lines in a `cron` log. Filtering the log does it. - No NUL-separated output. The README's `printf '%b'` recipe handles paths with special characters. - `info` does not say which trees were scanned, or when. - `report` lists again every file inside a tree that `trees` already reports. - Not proposed, because the README rules each out: deleting, hard-linking, byte-for-byte comparison, export formats, a `--database` option. ## Not checked - `dwim` as root: only inside the image from `make docker`, on the container's own files, never on a real `/`. - macOS: no such machine is available here. - A real `cron` job: none was installed. The README's command line was entered by hand. - Millions of files: the largest tree held 80,000 small files. Real scale is the open trial on the target server (https://git.eeqj.de/sneak/sfdupes/issues/56). - What `fdupes` and `jdupes` do, as the proposals cite it: taken from their manuals, since neither is installed here. Model: fable-5-2
clawbot added the needs-decision label 2026-11-11 14:10:30 +01:00
clawbot commented 2026-11-11 14:10:30 +01:00
Author
Collaborator
Copy link
Copy source

No PR: this issue is an acceptance test with no code change, and its definition of done (the verdict comment above, with the defects and proposals filed) is already met; it can be closed.

Model: opus-5-5

No PR: this issue is an acceptance test with no code change, and its definition of done (the verdict comment above, with the defects and proposals filed) is already met; it can be closed. Model: opus-5-5
clawbot commented 2026-11-11 14:10:56 +01:00
Author
Collaborator
Copy link
Copy source

Done: the test ran and its verdict and issues are above; every defect and every approved proposal it raised is fixed on next. A fresh hands-on test follows once the remaining work merges.

Model: opus-5-5

Done: the test ran and its verdict and issues are above; every defect and every approved proposal it raised is fixed on `next`. A fresh hands-on test follows once the remaining work merges. Model: opus-5-5
Sign in to join this conversation.
No Branch/Tag Specified
Labels
Clear labels
critical
Extremely important: data loss, a security hole, production down, or basic use impossible
 merge-ready
Review passed; ready to merge
 needs-checks
CI checks failing or not yet green
 needs-decision
needs-decision
Waits on a decision only sneak can make
 needs-decision
Waits on a decision before work continues
 needs-rebase
Branch conflicts with main; rebase required
 needs-review
PR awaiting independent adversarial review
 needs-rework
Review found blocking problems; author must rework
 post-1.0
postponed by sneak until after 1.0
No labels needs-decision
Milestone
No items
No Milestone
Projects
Clear projects
No projects
Assignees
Clear assignees
clawbot sneak (Jeffrey Paul)
No Assignees
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/sfdupes#164