report writes paths raw (report.go:79, fmt.Fprintf(out, "%s\t%s\t%d\n", ...)) and trees does the same (trees.go:59). The database layer deliberately supports paths containing tabs and newlines — db_test.go:138-141 round-trips /a/tab\tnew\nline with the comment "Paths may contain tabs and newlines" — and README §Database states paths are stored as raw bytes because Unix paths are not guaranteed UTF-8.
So a single file whose name contains a tab or a newline silently corrupts the machine-readable output that is this tool's entire product. README §Report output format and §Trees output format say nothing about escaping.
Second, related defect in the same output path: trees.go:96-101 splits paths on /, which yields a leading empty component, and because the node's parent is the synthetic super-root its path is set to the empty string rather than /. suppressed (trees.go:221-223) never suppresses a group whose parent is the super-root, so such a group is always reported — printing an empty first or dupe field. Unreachable with a single scanned root today, but it is the same class of malformed-TSV bug and the node path is simply wrong on inspection.
Definition of done
README specifies the escaping rule for the path columns of both reports. Recommended: C-style escaping of \t, \n, \r and \\, applied to path columns only, and applied to the per-path warnings on stderr as well so the two agree.
report and trees implement it.
The top-level node's path is normalized to / so no empty field can ever be emitted.
Tests: a record whose path contains a tab and a newline produces an exact, asserted output line from both report and trees; a hierarchy built from /f asserts the root node's path is /.
Determinism is unaffected — sorting still happens on the raw byte path, not the escaped form, so group ordering does not change.
make check green.
`report` writes paths raw (`report.go:79`, `fmt.Fprintf(out, "%s\t%s\t%d\n", ...)`) and `trees` does the same (`trees.go:59`). The database layer deliberately supports paths containing tabs and newlines — `db_test.go:138-141` round-trips `/a/tab\tnew\nline` with the comment "Paths may contain tabs and newlines" — and README §Database states paths are stored as raw bytes because Unix paths are not guaranteed UTF-8.
So a single file whose name contains a tab or a newline silently corrupts the machine-readable output that is this tool's entire product. README §Report output format and §Trees output format say nothing about escaping.
Second, related defect in the same output path: `trees.go:96-101` splits paths on `/`, which yields a leading empty component, and because the node's parent is the synthetic super-root its `path` is set to the empty string rather than `/`. `suppressed` (`trees.go:221-223`) never suppresses a group whose parent is the super-root, so such a group is always reported — printing an empty `first` or `dupe` field. Unreachable with a single scanned root today, but it is the same class of malformed-TSV bug and the node path is simply wrong on inspection.
## Definition of done
1. README specifies the escaping rule for the path columns of both reports. Recommended: C-style escaping of `\t`, `\n`, `\r` and `\\`, applied to path columns only, and applied to the per-path warnings on stderr as well so the two agree.
2. `report` and `trees` implement it.
3. The top-level node's path is normalized to `/` so no empty field can ever be emitted.
4. Tests: a record whose path contains a tab and a newline produces an exact, asserted output line from both `report` and `trees`; a hierarchy built from `/f` asserts the root node's path is `/`.
5. Determinism is unaffected — sorting still happens on the raw byte path, not the escaped form, so group ordering does not change.
6. `make check` green.
clawbot
added this to the 1.0.0 milestone 2026-08-09 03:43:59 +02:00
Plan (implementer's brief; the definition of done above stands, these are the points it leaves open).
Escaping: take the recommended rule. In the path columns of report and trees, write \ as \\, tab as \t, newline as \n, carriage return as \r; every other byte is written unchanged (paths are raw bytes, not UTF-8). One small function in report.go, used by both commands.
Warnings on stderr: escape the same way so a warning is always one line. Do it at the one place warnings are written (warnf in progress.go) rather than at each call site, unless that would escape something that must stay raw; say which you chose in the PR. #13 is editing progress.go at the same time; keep your change there small.
Sorting keeps using the raw path; only the written field is escaped.
trees: the node for / gets the path /, so no row can have an empty field. Children of / must then not become //x.
README.md: state the rule once, under the report and trees output formats, and note that the warnings use it too.
Tests per the definition of done: exact asserted lines for a path holding a tab and a newline from both commands, and the / root path.
#30 changes the same write loops in parallel; whichever lands second rebases.
Model: opus-5-5
Plan (implementer's brief; the definition of done above stands, these are the points it leaves open).
- Escaping: take the recommended rule. In the path columns of `report` and `trees`, write `\` as `\\`, tab as `\t`, newline as `\n`, carriage return as `\r`; every other byte is written unchanged (paths are raw bytes, not UTF-8). One small function in `report.go`, used by both commands.
- Warnings on stderr: escape the same way so a warning is always one line. Do it at the one place warnings are written (`warnf` in `progress.go`) rather than at each call site, unless that would escape something that must stay raw; say which you chose in the PR. https://git.eeqj.de/sneak/sfdupes/issues/13 is editing `progress.go` at the same time; keep your change there small.
- Sorting keeps using the raw path; only the written field is escaped.
- `trees`: the node for `/` gets the path `/`, so no row can have an empty field. Children of `/` must then not become `//x`.
- `README.md`: state the rule once, under the report and trees output formats, and note that the warnings use it too.
- Tests per the definition of done: exact asserted lines for a path holding a tab and a newline from both commands, and the `/` root path.
https://git.eeqj.de/sneak/sfdupes/issues/30 changes the same write loops in parallel; whichever lands second rebases.
Model: opus-5-5
Built in #70: report and trees escape backslash, tab, newline and carriage return in their path columns, warnings are escaped the same way, and the root directory's path in trees is /. The PR body has the details.
Model: opus-5-5
Built in https://git.eeqj.de/sneak/sfdupes/pulls/70: `report` and `trees` escape backslash, tab, newline and carriage return in their path columns, warnings are escaped the same way, and the root directory's path in `trees` is `/`. The PR body has the details.
Model: opus-5-5
Labelled critical: a file whose name contains a tab or a newline (which any user who can write to the scanned tree can create) breaks or forges rows in the duplicates report, so an operator or script that deletes the listed duplicates can delete files that are not duplicates.
Model: opus-5-5
Labelled critical: a file whose name contains a tab or a newline (which any user who can write to the scanned tree can create) breaks or forges rows in the duplicates report, so an operator or script that deletes the listed duplicates can delete files that are not duplicates.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
reportwrites paths raw (report.go:79,fmt.Fprintf(out, "%s\t%s\t%d\n", ...)) andtreesdoes the same (trees.go:59). The database layer deliberately supports paths containing tabs and newlines —db_test.go:138-141round-trips/a/tab\tnew\nlinewith the comment "Paths may contain tabs and newlines" — and README §Database states paths are stored as raw bytes because Unix paths are not guaranteed UTF-8.So a single file whose name contains a tab or a newline silently corrupts the machine-readable output that is this tool's entire product. README §Report output format and §Trees output format say nothing about escaping.
Second, related defect in the same output path:
trees.go:96-101splits paths on/, which yields a leading empty component, and because the node's parent is the synthetic super-root itspathis set to the empty string rather than/.suppressed(trees.go:221-223) never suppresses a group whose parent is the super-root, so such a group is always reported — printing an emptyfirstordupefield. Unreachable with a single scanned root today, but it is the same class of malformed-TSV bug and the node path is simply wrong on inspection.Definition of done
\t,\n,\rand\\, applied to path columns only, and applied to the per-path warnings on stderr as well so the two agree.reportandtreesimplement it./so no empty field can ever be emitted.reportandtrees; a hierarchy built from/fasserts the root node's path is/.make checkgreen.Plan (implementer's brief; the definition of done above stands, these are the points it leaves open).
reportandtrees, write\as\\, tab as\t, newline as\n, carriage return as\r; every other byte is written unchanged (paths are raw bytes, not UTF-8). One small function inreport.go, used by both commands.warnfinprogress.go) rather than at each call site, unless that would escape something that must stay raw; say which you chose in the PR. #13 is editingprogress.goat the same time; keep your change there small.trees: the node for/gets the path/, so no row can have an empty field. Children of/must then not become//x.README.md: state the rule once, under the report and trees output formats, and note that the warnings use it too./root path.#30 changes the same write loops in parallel; whichever lands second rebases.
Model: opus-5-5
Built in #70:
reportandtreesescape backslash, tab, newline and carriage return in their path columns, warnings are escaped the same way, and the root directory's path intreesis/. The PR body has the details.Model: opus-5-5
Labelled critical: a file whose name contains a tab or a newline (which any user who can write to the scanned tree can create) breaks or forges rows in the duplicates report, so an operator or script that deletes the listed duplicates can delete files that are not duplicates.
Model: opus-5-5
clawbot referenced this issue2026-10-03 15:32:40 +02:00