TSV output is unescaped: paths with tabs or newlines corrupt the report #7

New Issue

clawbot · 2026-08-09T03:43:59+02:00

clawbot commented

2026-08-09 03:43:59 +02:00

report writes paths raw (report.go:79, fmt.Fprintf(out, "%s\t%s\t%d\n", ...)) and trees does the same (trees.go:59). The database layer deliberately supports paths containing tabs and newlines — db_test.go:138-141 round-trips /a/tab\tnew\nline with the comment "Paths may contain tabs and newlines" — and README §Database states paths are stored as raw bytes because Unix paths are not guaranteed UTF-8.

So a single file whose name contains a tab or a newline silently corrupts the machine-readable output that is this tool's entire product. README §Report output format and §Trees output format say nothing about escaping.

Second, related defect in the same output path: trees.go:96-101 splits paths on /, which yields a leading empty component, and because the node's parent is the synthetic super-root its path is set to the empty string rather than /. suppressed (trees.go:221-223) never suppresses a group whose parent is the super-root, so such a group is always reported — printing an empty first or dupe field. Unreachable with a single scanned root today, but it is the same class of malformed-TSV bug and the node path is simply wrong on inspection.

Definition of done

README specifies the escaping rule for the path columns of both reports. Recommended: C-style escaping of \t, \n, \r and \\, applied to path columns only, and applied to the per-path warnings on stderr as well so the two agree.
report and trees implement it.
The top-level node's path is normalized to / so no empty field can ever be emitted.
Tests: a record whose path contains a tab and a newline produces an exact, asserted output line from both report and trees; a hierarchy built from /f asserts the root node's path is /.
Determinism is unaffected — sorting still happens on the raw byte path, not the escaped form, so group ordering does not change.
make check green.

`report` writes paths raw (`report.go:79`, `fmt.Fprintf(out, "%s\t%s\t%d\n", ...)`) and `trees` does the same (`trees.go:59`). The database layer deliberately supports paths containing tabs and newlines — `db_test.go:138-141` round-trips `/a/tab\tnew\nline` with the comment "Paths may contain tabs and newlines" — and README §Database states paths are stored as raw bytes because Unix paths are not guaranteed UTF-8. So a single file whose name contains a tab or a newline silently corrupts the machine-readable output that is this tool's entire product. README §Report output format and §Trees output format say nothing about escaping. Second, related defect in the same output path: `trees.go:96-101` splits paths on `/`, which yields a leading empty component, and because the node's parent is the synthetic super-root its `path` is set to the empty string rather than `/`. `suppressed` (`trees.go:221-223`) never suppresses a group whose parent is the super-root, so such a group is always reported — printing an empty `first` or `dupe` field. Unreachable with a single scanned root today, but it is the same class of malformed-TSV bug and the node path is simply wrong on inspection. ## Definition of done 1. README specifies the escaping rule for the path columns of both reports. Recommended: C-style escaping of `\t`, `\n`, `\r` and `\\`, applied to path columns only, and applied to the per-path warnings on stderr as well so the two agree. 2. `report` and `trees` implement it. 3. The top-level node's path is normalized to `/` so no empty field can ever be emitted. 4. Tests: a record whose path contains a tab and a newline produces an exact, asserted output line from both `report` and `trees`; a hierarchy built from `/f` asserts the root node's path is `/`. 5. Determinism is unaffected — sorting still happens on the raw byte path, not the escaped form, so group ordering does not change. 6. `make check` green.

clawbot added this to the 1.0.0 milestone 2026-08-09 03:43:59 +02:00

clawbot referenced this issue

2026-08-09 03:47:54 +02:00

Land golangci-lint v2.12.2 and the canonical .golangci.yml on main #3

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sneak/sfdupes#7