TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmit in internal/logfield now checks logfield.EncodedBytes against every Unicode code point (surrogates aside) for both slog handlers. It replaces the dense range plus a sample, as planned on #172.
Below U+1000 each code point is checked on its own, since that range holds the quote, the backslash and the control characters next to code points written in fewer bytes than their charge, which in a sum would cover one charged too little. Each is measured in a value of it alone and again in one the text handler quotes, which writes U+007F as \x7f only when quoting.
From U+1000 up, code points are logged 4,096 to a value. For each batch, the bytes the handler writes are measured as a line carrying the batch twice, less a line carrying it once, so quoting and the rest of the line cancel out, and the batch's summed charge must cover that. A failing batch is measured again one code point at a time, so the failure names how many are undercharged and the first.
The batch charges are worked out once, shared by both handlers, because under -race -cover charging every code point takes longer than logging them.
Judgement call: from U+1000 up a sum can miss the JSON handler alone writing one code point in more bytes than its charge, when it writes others in the same batch in fewer; the text handler sets the charge there.
Judgement call: a failing run is slow, because each failing batch is measured again per code point to count them.
Model: opus-5-5
`TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmit` in `internal/logfield` now checks `logfield.EncodedBytes` against every Unicode code point (surrogates aside) for both slog handlers. It replaces the dense range plus a sample, as planned on https://git.eeqj.de/sneak/webhooker/issues/172.
Below U+1000 each code point is checked on its own, since that range holds the quote, the backslash and the control characters next to code points written in fewer bytes than their charge, which in a sum would cover one charged too little. Each is measured in a value of it alone and again in one the text handler quotes, which writes U+007F as `\x7f` only when quoting.
From U+1000 up, code points are logged 4,096 to a value. For each batch, the bytes the handler writes are measured as a line carrying the batch twice, less a line carrying it once, so quoting and the rest of the line cancel out, and the batch's summed charge must cover that. A failing batch is measured again one code point at a time, so the failure names how many are undercharged and the first.
The batch charges are worked out once, shared by both handlers, because under `-race -cover` charging every code point takes longer than logging them.
- Judgement call: from U+1000 up a sum can miss the JSON handler alone writing one code point in more bytes than its charge, when it writes others in the same batch in fewer; the text handler sets the charge there.
- Judgement call: a failing run is slow, because each failing batch is measured again per code point to count them.
Model: opus-5-5
internal/logfield/logfield_test.go, TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmit: a code point charged too little passes whenever other code points in its batch are charged more than the handler writes for them, and in U+0000–U+0FFF that holds for both handlers (the text handler writes most C0 controls as \x01 or \a, under their 6-byte charge; the JSON handler writes C1 controls and unassigned code points raw). So the test no longer fails when the quote, backslash, newline, carriage return or tab is charged one byte, or a C0 control 5 bytes against the JSON handler's 6: the code points the line ceiling exists for, all caught by the test this replaces. Looking one code point at a time only when a batch's total comes close to its charge does not fix it. Acceptable: charging any single code point less than either handler writes for it fails the test and names that code point, within the 3-second budget. The cheap way: check U+0000–U+0FFF one code point at a time for both handlers, and keep the batch sums from U+1000 up, where the text handler writes every code point in exactly its charge; reword "the price of fitting all of Unicode into the suite" to say what the sums still cannot see.
internal/logfield/logfield_test.go, the batchRunes comment: "272 lines per handler" is wrong; each batch is logged on two lines (once, then twice over), so it is 272 batches and 544 lines. Acceptable: the comment matches the code.
Judgement call: the slow failing run disclosed in the PR body is acceptable.
Judgement call: after 1, a change to the JSON handler alone that writes one code point from U+1000 up in more bytes than its charge can still hide in a sum; the text handler sets the charge there, so that limit is accepted once the doc comment states it.
Model: opus-5-5
**FAIL: needs-rework**
1. `internal/logfield/logfield_test.go`, `TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmit`: a code point charged too little passes whenever other code points in its batch are charged more than the handler writes for them, and in U+0000–U+0FFF that holds for both handlers (the text handler writes most C0 controls as `\x01` or `\a`, under their 6-byte charge; the JSON handler writes C1 controls and unassigned code points raw). So the test no longer fails when the quote, backslash, newline, carriage return or tab is charged one byte, or a C0 control 5 bytes against the JSON handler's 6: the code points the line ceiling exists for, all caught by the test this replaces. Looking one code point at a time only when a batch's total comes close to its charge does not fix it. Acceptable: charging any single code point less than either handler writes for it fails the test and names that code point, within the 3-second budget. The cheap way: check U+0000–U+0FFF one code point at a time for both handlers, and keep the batch sums from U+1000 up, where the text handler writes every code point in exactly its charge; reword "the price of fitting all of Unicode into the suite" to say what the sums still cannot see.
2. `internal/logfield/logfield_test.go`, the `batchRunes` comment: "272 lines per handler" is wrong; each batch is logged on two lines (once, then twice over), so it is 272 batches and 544 lines. Acceptable: the comment matches the code.
- Judgement call: the slow failing run disclosed in the PR body is acceptable.
- Judgement call: after 1, a change to the JSON handler alone that writes one code point from U+1000 up in more bytes than its charge can still hide in a sum; the text handler sets the charge there, so that limit is accepted once the doc comment states it.
Model: opus-5-5
Every code point below U+1000 is now checked on its own for both handlers, and the batch sums start at U+1000. The test's doc comment now says what the sums still miss: the JSON handler alone writing one code point from U+1000 up in more bytes than its charge, when it writes others in the same batch in fewer.
The batchRunes comment now gives the counts the code has: 271 batches, each logged on two lines, so 542 lines per handler.
Charging the quote, newline, carriage return or tab one byte each fails the test on both handlers and names that code point; charging the backslash one byte, or U+0001 five bytes, fails it on the JSON handler and names it (the text handler writes those in no more).
Under -race the test takes about 1.2 seconds run alone; inside a full make test (with -cover) on the loaded shared host its package took 3.3 seconds, against 3.5 for the batch-only version measured straight after.
Deviation: the review's 272 batches and 544 lines were the counts before finding 1; the comment gives 271 and 542, since U+0000 to U+0FFF is no longer batched.
Model: opus-5-5
Rework for the review above.
1. Every code point below U+1000 is now checked on its own for both handlers, and the batch sums start at U+1000. The test's doc comment now says what the sums still miss: the JSON handler alone writing one code point from U+1000 up in more bytes than its charge, when it writes others in the same batch in fewer.
2. The `batchRunes` comment now gives the counts the code has: 271 batches, each logged on two lines, so 542 lines per handler.
- Charging the quote, newline, carriage return or tab one byte each fails the test on both handlers and names that code point; charging the backslash one byte, or U+0001 five bytes, fails it on the JSON handler and names it (the text handler writes those in no more).
- Under `-race` the test takes about 1.2 seconds run alone; inside a full `make test` (with `-cover`) on the loaded shared host its package took 3.3 seconds, against 3.5 for the batch-only version measured straight after.
- Deviation: the review's 272 batches and 544 lines were the counts before finding 1; the comment gives 271 and 542, since U+0000 to U+0FFF is no longer batched.
Model: opus-5-5
internal/logfield/logfield_test.go, TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmit, the check below U+1000: each code point is measured in a value holding only that code point. The text handler leaves a value of only U+007F (DEL) unquoted and writes it raw in one byte, but in any value it quotes (one with a space, = or a control character, say) it writes DEL as \x7f, four bytes. So charging DEL one to three bytes passes the test, and nothing else in the suite catches it. Acceptable: each code point below U+1000 is also measured in a value the text handler quotes (for example with a space at the front of both values), so that charging U+007F under four bytes fails and names it, and the doc comment says so.
Judgement call: the batchRunes comment's 542 lines count only the batches from U+1000 up, the range its sentence is about; accepted.
Model: opus-5-5
**FAIL: needs-rework**
1. `internal/logfield/logfield_test.go`, `TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmit`, the check below U+1000: each code point is measured in a value holding only that code point. The text handler leaves a value of only U+007F (DEL) unquoted and writes it raw in one byte, but in any value it quotes (one with a space, `=` or a control character, say) it writes DEL as `\x7f`, four bytes. So charging DEL one to three bytes passes the test, and nothing else in the suite catches it. Acceptable: each code point below U+1000 is also measured in a value the text handler quotes (for example with a space at the front of both values), so that charging U+007F under four bytes fails and names it, and the doc comment says so.
- Judgement call: the `batchRunes` comment's 542 lines count only the batches from U+1000 up, the range its sentence is about; accepted.
Model: opus-5-5
TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmit now covers all of
Unicode, surrogates aside, in place of a dense range plus a sample.
Below U+1000 each code point is checked on its own against both
handlers, alone in a value and again in one the text handler quotes,
which writes U+007F there as \x7f. From U+1000 up code points are
logged 4,096 to a value, and each batch's summed charge must cover the
bytes the real handler writes for it, measured as a line carrying the
batch twice less one carrying it once. A failing batch is then
measured one code point at a time, so the failure names how many are
undercharged and the first of them.
Model: opus-5-5
Each code point below U+1000 is now measured both in a value of it alone and in one starting with a space, which the text handler quotes. The test's doc comment says so and why: that handler writes U+007F as one raw byte in a bare value but as \x7f, four bytes, in a quoted one.
Charging U+007F three bytes fails the test on the text handler and names U+007F.
Under -race the test takes about 1.6 seconds run alone; inside a full make test (with -cover) on the loaded shared host its package took 3.8 seconds, against 3.4 for the version before this change measured straight after.
Model: opus-5-5
Rework for the review above.
1. Each code point below U+1000 is now measured both in a value of it alone and in one starting with a space, which the text handler quotes. The test's doc comment says so and why: that handler writes U+007F as one raw byte in a bare value but as `\x7f`, four bytes, in a quoted one.
- Charging U+007F three bytes fails the test on the text handler and names U+007F.
- Under `-race` the test takes about 1.6 seconds run alone; inside a full `make test` (with `-cover`) on the loaded shared host its package took 3.8 seconds, against 3.4 for the version before this change measured straight after.
Model: opus-5-5
Review passed: each code point below U+1000 is now also measured in a quoted value, so charging U+007F under four bytes fails the test and names it, and the doc comment is true of the code.
Model: opus-5-5
Review passed: each code point below U+1000 is now also measured in a quoted value, so charging U+007F under four bytes fails the test and names it, and the doc comment is true of the code.
Model: opus-5-5
clawbot
merged commit e67fffb05d into next2026-10-02 17:53:11 +02:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmitininternal/logfieldnow checkslogfield.EncodedBytesagainst every Unicode code point (surrogates aside) for both slog handlers. It replaces the dense range plus a sample, as planned on #172.Below U+1000 each code point is checked on its own, since that range holds the quote, the backslash and the control characters next to code points written in fewer bytes than their charge, which in a sum would cover one charged too little. Each is measured in a value of it alone and again in one the text handler quotes, which writes U+007F as
\x7fonly when quoting.From U+1000 up, code points are logged 4,096 to a value. For each batch, the bytes the handler writes are measured as a line carrying the batch twice, less a line carrying it once, so quoting and the rest of the line cancel out, and the batch's summed charge must cover that. A failing batch is measured again one code point at a time, so the failure names how many are undercharged and the first.
The batch charges are worked out once, shared by both handlers, because under
-race -covercharging every code point takes longer than logging them.Model: opus-5-5
FAIL: needs-rework
internal/logfield/logfield_test.go,TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmit: a code point charged too little passes whenever other code points in its batch are charged more than the handler writes for them, and in U+0000–U+0FFF that holds for both handlers (the text handler writes most C0 controls as\x01or\a, under their 6-byte charge; the JSON handler writes C1 controls and unassigned code points raw). So the test no longer fails when the quote, backslash, newline, carriage return or tab is charged one byte, or a C0 control 5 bytes against the JSON handler's 6: the code points the line ceiling exists for, all caught by the test this replaces. Looking one code point at a time only when a batch's total comes close to its charge does not fix it. Acceptable: charging any single code point less than either handler writes for it fails the test and names that code point, within the 3-second budget. The cheap way: check U+0000–U+0FFF one code point at a time for both handlers, and keep the batch sums from U+1000 up, where the text handler writes every code point in exactly its charge; reword "the price of fitting all of Unicode into the suite" to say what the sums still cannot see.internal/logfield/logfield_test.go, thebatchRunescomment: "272 lines per handler" is wrong; each batch is logged on two lines (once, then twice over), so it is 272 batches and 544 lines. Acceptable: the comment matches the code.Model: opus-5-5
ba6ce94860to81ade15175Rework for the review above.
batchRunescomment now gives the counts the code has: 271 batches, each logged on two lines, so 542 lines per handler.-racethe test takes about 1.2 seconds run alone; inside a fullmake test(with-cover) on the loaded shared host its package took 3.3 seconds, against 3.5 for the batch-only version measured straight after.Model: opus-5-5
FAIL: needs-rework
internal/logfield/logfield_test.go,TestEncodedBytes_ChargesAtLeastWhatTheHandlersEmit, the check below U+1000: each code point is measured in a value holding only that code point. The text handler leaves a value of only U+007F (DEL) unquoted and writes it raw in one byte, but in any value it quotes (one with a space,=or a control character, say) it writes DEL as\x7f, four bytes. So charging DEL one to three bytes passes the test, and nothing else in the suite catches it. Acceptable: each code point below U+1000 is also measured in a value the text handler quotes (for example with a space at the front of both values), so that charging U+007F under four bytes fails and names it, and the doc comment says so.batchRunescomment's 542 lines count only the batches from U+1000 up, the range its sentence is about; accepted.Model: opus-5-5
81ade15175to0b3ce46768Rework for the review above.
\x7f, four bytes, in a quoted one.-racethe test takes about 1.6 seconds run alone; inside a fullmake test(with-cover) on the loaded shared host its package took 3.8 seconds, against 3.4 for the version before this change measured straight after.Model: opus-5-5
Review passed: each code point below U+1000 is now also measured in a quoted value, so charging U+007F under four bytes fails the test and names it, and the doc comment is true of the code.
Model: opus-5-5