Add CollapseWhitespace to squeeze runs of whitespace into single spaces #8

clawbot · 2026-09-05T05:26:30+02:00

clawbot commented

2026-09-05 05:26:30 +02:00

Adds CollapseWhitespace(s string) string, which replaces every run of
whitespace with a single space and removes whitespace at the start and end.

This belongs here because text that comes from a form field, a scraped page or
a pasted document is full of stray tabs, newlines and double spaces, and
flattening it is a step before almost any comparison, display or storage.

Things to know:

It is built on strings.Fields, which splits on whitespace as Unicode
defines it, so a non-breaking space or a line separator counts too. A regular
expression using \s would not have caught those, since Go's regular
expressions treat \s as ASCII only.
The result is always a single line. If a caller needs paragraph breaks kept,
this is the wrong function.
The test file defines the non-breaking space as a code point rather than
pasting the character, so that the interesting test case is visible to
someone reading the source.
The code is in a new file, whitespace.go, so that the ten proposal branches
do not all conflict in the same place.
make test on this branch reports one failure, TestNowUnixMicro. That test
already fails on master and is unrelated to this change.

Model: opus-5

Adds `CollapseWhitespace(s string) string`, which replaces every run of whitespace with a single space and removes whitespace at the start and end. This belongs here because text that comes from a form field, a scraped page or a pasted document is full of stray tabs, newlines and double spaces, and flattening it is a step before almost any comparison, display or storage. Things to know: - It is built on `strings.Fields`, which splits on whitespace as Unicode defines it, so a non-breaking space or a line separator counts too. A regular expression using `\s` would not have caught those, since Go's regular expressions treat `\s` as ASCII only. - The result is always a single line. If a caller needs paragraph breaks kept, this is the wrong function. - The test file defines the non-breaking space as a code point rather than pasting the character, so that the interesting test case is visible to someone reading the source. - The code is in a new file, `whitespace.go`, so that the ten proposal branches do not all conflict in the same place. - `make test` on this branch reports one failure, `TestNowUnixMicro`. That test already fails on `master` and is unrelated to this change. Model: opus-5

clawbot self-assigned this 2026-09-05 05:26:30 +02:00

clawbot added 1 commit 2026-09-05 05:26:30 +02:00

add CollapseWhitespace for flattening runs of whitespace d52588d635

CollapseWhitespace turns every run of whitespace into a single space and
trims the ends, so a block of text becomes one tidy line. Comes with a doc
comment and table-driven tests. (closes #7)

Model: opus-5

sneak merged commit c656ce964e into master

2026-09-05 05:45:12 +02:00

sneak referenced this issue from a commit

2026-09-05 05:45:14 +02:00

Merge pull request 'Add CollapseWhitespace to squeeze runs of whitespace into single spaces' (#8) from clawbot/util:proposal-collapse-whitespace into master

Sign in to join this conversation.

1 Participants

Notifications

Due Date

No due date set.

Dependencies

No dependencies set.

Reference: sneak/util#8

Allow edits from maintainers