2 Commits
Author SHA1 Message Date
clawbot 8721d89f4f Log the client address next to the peer address (closes #270)
check / check (push) Waiting to run
The access log, the rate-limit rejection lines, the CSRF warning and
the receiver's "webhook request received" line now carry clientIP,
the address the rate limiters key on, next to remoteIP, the
connecting peer. Logging works it out once per request from the same
code the rate limiters use and stores it on the request context for
the other lines. The CSRF and receiver lines name the peer as
remoteIP instead of remote_addr. The README documents the field and
that it is only as trustworthy as TRUSTED_PROXIES.

Model: opus-5-5
2026-10-02 14:37:44 +00:00
clawbot debe588bba Seed the retention tests 50 rows per insert, not 500 (closes #198)
check / check (push) Waiting to run
Three retention tests made internal/database the slowest test package, mostly by seeding thousands of rows 500 per insert: the SQLite driver finds each parameter's value by scanning the statement's arguments from the first until it reaches that parameter's, so binding grows with the square of the parameter count. They now seed the same rows 50 per insert, about three times faster; no test case or assertion changes. The package drops from 13 to 22s to about 7s. What keeps make test above the 20s target is now mostly the cold -race compile of the tree, which moves with host load. The 90s per-package timeout stays; script/test's header records the new figures.

Model: opus-5-5
2026-10-02 16:27:00 +02:00
4 changed files with 34 additions and 18 deletions
+5 -4
View File
@@ -2495,10 +2495,11 @@ client-chosen text in the path, in the query, and in each of
including cases built from the characters the handlers escape, and including cases built from the characters the handlers escape, and
against a 5xx that keeps its concrete path while all three header fields against a 5xx that keeps its concrete path while all three header fields
are also at their budget and an `X-Forwarded-For` sent from a trusted are also at their budget and an `X-Forwarded-For` sent from a trusted
proxy ends in an IPv6 client address at its longest. Every case runs proxy ends in an IPv6 client address at its longest followed by an 8 KB
through both handlers `internal/logger` can select — the JSON one and zone, where `clientIP` must name the address without the zone. Every
the text one it installs on a tty — since the two do not escape alike case runs through both handlers `internal/logger` can select — the JSON
and the ceiling is quoted unqualified. one and the text one it installs on a tty — since the two do not escape
alike and the ceiling is quoted unqualified.
Multiply that ceiling by the request rate to size log storage. Note Multiply that ceiling by the request rate to size log storage. Note
that the rate is not bounded by the limits above on every route: that the rate is not bounded by the limits above on every route:
+11 -4
View File
@@ -102,6 +102,13 @@ func TestWebhookDBManager_TotalsSurviveReopen(t *testing.T) {
// seedExpiredEvents stores count events created at the given time, // seedExpiredEvents stores count events created at the given time,
// each with a delivered delivery to one target and a failed delivery // each with a delivered delivery to one target and a failed delivery
// to the other, and one attempt for each delivery. // to the other, and one attempt for each delivery.
//
// It and seedBareEvents insert 50 rows per statement, not more. The
// SQLite driver looks up each parameter's value by scanning the
// statement's arguments from the first until it reaches that
// parameter's, so the time to bind a statement grows with the square of
// its parameter count: at 500 rows, several thousand parameters, the
// seeding took most of these tests' time under -race.
func seedExpiredEvents( func seedExpiredEvents(
t *testing.T, t *testing.T,
db *gorm.DB, db *gorm.DB,
@@ -138,8 +145,8 @@ func seedExpiredEvents(
) )
} }
require.NoError(t, db.CreateInBatches(events, 500).Error) require.NoError(t, db.CreateInBatches(events, 50).Error)
require.NoError(t, db.CreateInBatches(deliveries, 500).Error) require.NoError(t, db.CreateInBatches(deliveries, 50).Error)
results := make([]database.DeliveryResult, len(deliveries)) results := make([]database.DeliveryResult, len(deliveries))
for i := range deliveries { for i := range deliveries {
@@ -148,7 +155,7 @@ func seedExpiredEvents(
} }
} }
require.NoError(t, db.CreateInBatches(results, 500).Error) require.NoError(t, db.CreateInBatches(results, 50).Error)
} }
// seedBareEvents stores count events created at the given time, with // seedBareEvents stores count events created at the given time, with
@@ -172,7 +179,7 @@ func seedBareEvents(
events[i].CreatedAt = createdAt events[i].CreatedAt = createdAt
} }
require.NoError(t, db.CreateInBatches(events, 500).Error) require.NoError(t, db.CreateInBatches(events, 50).Error)
} }
// TestRetentionReaper_PrunesMoreThanOneBatch verifies that a prune // TestRetentionReaper_PrunesMoreThanOneBatch verifies that a prune
+13 -9
View File
@@ -64,10 +64,11 @@ const (
// lines into the returned buffer, so the access log can be asserted // lines into the returned buffer, so the access log can be asserted
// on directly. // on directly.
// //
// It trusts 192.0.2.1, the peer httptest sends every request from, as // It trusts 192.0.2.1, the peer address httptest.NewRequestWithContext
// a proxy, the way a deployment trusts its reverse proxy: a request // gives a request, as a proxy, the way a deployment trusts its reverse
// carrying X-Forwarded-For is logged with the client that header names // proxy: a request built that way and carrying X-Forwarded-For is
// as clientIP, and one without it with the peer. // logged with the client that header names as clientIP, and one
// without it with the peer.
func capturingMiddleware(t *testing.T) (*middleware.Middleware, *bytes.Buffer) { func capturingMiddleware(t *testing.T) (*middleware.Middleware, *bytes.Buffer) {
t.Helper() t.Helper()
@@ -432,14 +433,17 @@ func lineSizeCases() map[string]sizeCase {
} }
// From a trusted proxy, clientIP is read out of X-Forwarded-For, // From a trusted proxy, clientIP is read out of X-Forwarded-For,
// which the client writes. Only the address the header ends in may // which the client writes. What bounds the field is that only one
// reach the line, and an IPv6 address with all eight groups at four // address from the header is written, and it is written parsed, with
// digits is that address at its longest. It goes on the 5xx line // no zone. An IPv6 address with all eight groups at four digits is
// with all three header fields at their budget. // the longest such address; here it carries an 8 KB zone, which must
// not reach the line. It goes on the 5xx line with all three header
// fields at their budget.
const longestIPv6 = "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff" const longestIPv6 = "ffff:ffff:ffff:ffff:ffff:ffff:ffff:ffff"
forwarded := oversizedHeaders(oversizedValue("h")) forwarded := oversizedHeaders(oversizedValue("h"))
forwarded[headerXFF] = oversizedValue("h") + ", " + longestIPv6 forwarded[headerXFF] = oversizedValue("h") + ", " +
longestIPv6 + "%" + oversizedValue("h")
cases["oversized X-Forwarded-For from a trusted proxy "+ cases["oversized X-Forwarded-For from a trusted proxy "+
"with a 5xx concrete url"] = sizeCase{ "with a 5xx concrete url"] = sizeCase{
+5 -1
View File
@@ -27,7 +27,11 @@
# Those figures predate tests hashing the admin password at 1 MB instead of # Those figures predate tests hashing the admin password at 1 MB instead of
# 64 MB (https://git.eeqj.de/sneak/webhooker/pulls/404). After that change, in # 64 MB (https://git.eeqj.de/sneak/webhooker/pulls/404). After that change, in
# a cache-defeated build at host load 44-109 (2026-10-02), internal/handlers # a cache-defeated build at host load 44-109 (2026-10-02), internal/handlers
# took 8.5s and the slowest package was internal/database at 15.8s. # took 8.5s and the slowest package was internal/database at 15.8s. Once its
# retention tests seeded 50 rows per insert instead of 500
# (https://git.eeqj.de/sneak/webhooker/issues/198), internal/database took
# 7.3s and the slowest package was internal/handlers at 8.1s to 10.0s, at host
# load 25-48 (2026-10-02).
# #
# -p 4 -parallel 8 keep the run under 2 GB of memory: at most four test # -p 4 -parallel 8 keep the run under 2 GB of memory: at most four test
# binaries build or run at once, each with at most eight parallel tests. Under # binaries build or run at once, each with at most eight parallel tests. Under