quoteIfNeeded leaves invalid UTF-8 bare, so raw invalid bytes reach the terminal #24

Closed
opened 2026-08-10 15:36:50 +02:00 by clawbot · 2 comments
Collaborator

Found reviewing #21
(#21 (comment)). Pre-existing
for values; that PR extended quoteIfNeeded to keys, which is how it surfaced.

quoteIfNeeded quotes on delimiter ambiguity, and unicode.IsPrint treats
U+FFFD as printable, so a string containing an invalid UTF-8 byte stays bare.
Console output is bad\xffkey=v where slog.NewTextHandler emits
"bad\xffkey"=v.

The consequence is not parse ambiguity — bad\xffkey=v still splits correctly —
but that a raw invalid byte reaches the terminal and any UTF-8-strict consumer.

Definition of done

  • Invalid UTF-8 is quoted rather than emitted raw, matching
    slog.NewTextHandler.
  • Applied consistently to keys and values: quoteIfNeeded serves both, so a
    fix to one changes the other. A differential against slog.NewTextHandler
    covering both is the evidence.
  • Valid non-ASCII stays unquoted where stdlib leaves it unquoted — do not fix
    this by quoting everything non-ASCII.

Implementation requirements

  • Do not change WithGroup("") behaviour; its divergence from
    slog.NewJSONHandler is deliberate and was confirmed twice.
  • Landing commit title must end with (closes #24).
Found reviewing https://git.eeqj.de/sneak/simplelog/pulls/21 (https://git.eeqj.de/sneak/simplelog/pulls/21#issuecomment-54029). Pre-existing for values; that PR extended `quoteIfNeeded` to keys, which is how it surfaced. `quoteIfNeeded` quotes on delimiter ambiguity, and `unicode.IsPrint` treats U+FFFD as printable, so a string containing an invalid UTF-8 byte stays bare. Console output is `bad\xffkey=v` where `slog.NewTextHandler` emits `"bad\xffkey"=v`. The consequence is not parse ambiguity — `bad\xffkey=v` still splits correctly — but that a raw invalid byte reaches the terminal and any UTF-8-strict consumer. ## Definition of done - Invalid UTF-8 is quoted rather than emitted raw, matching `slog.NewTextHandler`. - Applied consistently to keys **and** values: `quoteIfNeeded` serves both, so a fix to one changes the other. A differential against `slog.NewTextHandler` covering both is the evidence. - Valid non-ASCII stays unquoted where stdlib leaves it unquoted — do not fix this by quoting everything non-ASCII. ## Implementation requirements - Do not change `WithGroup("")` behaviour; its divergence from `slog.NewJSONHandler` is deliberate and was confirmed twice. - Landing commit title must end with ` (closes #24)`.
Owner

work this into 21

work this into 21
sneak closed this issue 2026-08-10 15:39:04 +02:00
Author
Collaborator

Being done in #21 as ordered above; review found it still missing (#21 (comment)). The tests go in that PR's first commit and the fix in its second, whose subject becomes Emit slog attributes from every handler (closes #19, closes #24). Invalid UTF-8 in console keys and values is quoted as slog.NewTextHandler does, and valid non-ASCII stays bare. The DEL character stays quoted, as the PR already does, even where the stdlib leaves it bare: a raw control byte on the terminal is what this issue exists to stop. That difference is disclosed in one line.

Model: opus-5-5

Being done in https://git.eeqj.de/sneak/simplelog/pulls/21 as ordered above; review found it still missing (https://git.eeqj.de/sneak/simplelog/pulls/21#issuecomment-103467). The tests go in that PR's first commit and the fix in its second, whose subject becomes `Emit slog attributes from every handler (closes #19, closes #24)`. Invalid UTF-8 in console keys and values is quoted as `slog.NewTextHandler` does, and valid non-ASCII stays bare. The DEL character stays quoted, as the PR already does, even where the stdlib leaves it bare: a raw control byte on the terminal is what this issue exists to stop. That difference is disclosed in one line. Model: opus-5-5
Sign in to join this conversation.
2 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/simplelog#24