trees: the reclaimable total counts a duplicate tree twice when it lies inside another reported one #168

New Issue

2026-11-11T01:02:55+01:00

clawbot commented

2026-11-11 01:02:55 +01:00

Defect, found in the acceptance test (#164) on next at 202f083.

Steps

make build
d=$(mktemp -d /var/tmp/sfdupes.XXXXXX)
export SFDUPES_DATABASE="$(mktemp -d /var/tmp/sfdupes.XXXXXX)/db.sqlite"
mkdir -p "$d/A/x" "$d/A/y" "$d/B/x" "$d/B/y" "$d/C/x"
dd if=/dev/urandom of="$d/A/x/f" bs=1000 count=1 status=none
dd if=/dev/urandom of="$d/A/y/g" bs=500 count=1 status=none
cp "$d/A/x/f" "$d/B/x/f"
cp "$d/A/y/g" "$d/B/y/g"
cp "$d/A/x/f" "$d/C/x/f"
./sfdupes scan "$d"
./sfdupes trees

What happened

first	dupe	files	size	disk_size
$d/A	$d/B	2	1500	1024
$d/A/x	$d/B/x	1	1000	512
$d/A/x	$d/C/x	1	1000	512
trees: 5 records read, 2 duplicate tree groups, 3 dupe trees, 3.4 KiB reclaimable

The total is 3500 bytes. Removing every dupe path (B, B/x, C/x) frees 2500 bytes: B/x lies inside B, and its 1000 bytes are counted in both rows.

On the acceptance tree of about 4000 files the total read 7.5 MiB, where removing every dupe path frees 5.7 MiB.

Why it matters

The total is the number a user reads to decide whether a cleanup is worth doing. It is too high whenever a subtree has a copy outside two otherwise identical parents, which is common in backup trees.

What should happen

The total is what removing every dupe path in the report would free, by the recorded sizes: each recorded file at or below a dupe path is counted once.
The rows stay as they are.
README.md ("Trees output format") defines the total as "the sum of size over all rows". The code follows that sentence, so the sentence changes with the code.

Model: fable-5-2

Defect, found in the acceptance test (https://git.eeqj.de/sneak/sfdupes/issues/164) on `next` at `202f083`. ## Steps ```sh make build d=$(mktemp -d /var/tmp/sfdupes.XXXXXX) export SFDUPES_DATABASE="$(mktemp -d /var/tmp/sfdupes.XXXXXX)/db.sqlite" mkdir -p "$d/A/x" "$d/A/y" "$d/B/x" "$d/B/y" "$d/C/x" dd if=/dev/urandom of="$d/A/x/f" bs=1000 count=1 status=none dd if=/dev/urandom of="$d/A/y/g" bs=500 count=1 status=none cp "$d/A/x/f" "$d/B/x/f" cp "$d/A/y/g" "$d/B/y/g" cp "$d/A/x/f" "$d/C/x/f" ./sfdupes scan "$d" ./sfdupes trees ``` ## What happened ``` first dupe files size disk_size $d/A $d/B 2 1500 1024 $d/A/x $d/B/x 1 1000 512 $d/A/x $d/C/x 1 1000 512 trees: 5 records read, 2 duplicate tree groups, 3 dupe trees, 3.4 KiB reclaimable ``` The total is 3500 bytes. Removing every `dupe` path (`B`, `B/x`, `C/x`) frees 2500 bytes: `B/x` lies inside `B`, and its 1000 bytes are counted in both rows. On the acceptance tree of about 4000 files the total read 7.5 MiB, where removing every `dupe` path frees 5.7 MiB. ## Why it matters The total is the number a user reads to decide whether a cleanup is worth doing. It is too high whenever a subtree has a copy outside two otherwise identical parents, which is common in backup trees. ## What should happen - The total is what removing every `dupe` path in the report would free, by the recorded sizes: each recorded file at or below a `dupe` path is counted once. - The rows stay as they are. - `README.md` ("Trees output format") defines the total as "the sum of `size` over all rows". The code follows that sentence, so the sentence changes with the code. Model: fable-5-2

clawbot referenced this issue

2026-11-11 01:07:59 +01:00

Acceptance test: use sfdupes end to end as a user, and list missing basics #164

sneak commented

2026-11-11 03:25:39 +01:00

yes, this is a bug, fix it

clawbot commented

2026-11-11 03:39:35 +01:00

sneak (11 November, about 02:25 UTC): a bug; fix it. The reclaimable total counts each duplicate tree once, never again when it lies inside another reported tree; a test covers a nested pair.

Model: opus-5-5

sneak (11 November, about 02:25 UTC): a bug; fix it. The reclaimable total counts each duplicate tree once, never again when it lies inside another reported tree; a test covers a nested pair. Model: opus-5-5

clawbot referenced this issue from a commit

2026-11-11 04:55:17 +01:00

trees: count a nested dupe tree once in the reclaimable total (closes #168)

clawbot referenced a pull request that will close this issue

2026-11-11 04:55:33 +01:00

trees: count a nested dupe tree once in the reclaimable total (closes #168) #188

clawbot commented

2026-11-11 04:55:43 +01:00

Fixed in #188: the reclaimable total skips a dupe tree that lies inside another dupe tree. The rows are unchanged, and the README defines the total to match. The steps above now report 2.4 KiB (2500 bytes).

Model: opus-5-5

Fixed in https://git.eeqj.de/sneak/sfdupes/pulls/188: the reclaimable total skips a dupe tree that lies inside another dupe tree. The rows are unchanged, and the README defines the total to match. The steps above now report 2.4 KiB (2500 bytes). Model: opus-5-5

clawbot referenced this issue from a commit

2026-11-11 06:45:53 +01:00

trees: count a nested dupe tree once in the reclaimable total (closes #168)

clawbot referenced this issue from a commit

2026-11-11 07:55:39 +01:00

trees: count a nested dupe tree once in the reclaimable total (closes #168)

clawbot referenced this issue from a commit

2026-11-11 08:55:29 +01:00

trees: count a nested dupe tree once in the reclaimable total (closes #168)

clawbot referenced this issue from a commit

2026-11-11 09:24:27 +01:00

trees: count a nested dupe tree once in the reclaimable total (closes #168)