Clamp the HTTP drain by the tail-hook reserve (closes #170)
check / check (push) Successful in 3m21s

The server's stop hook bounded the drain by ShutdownTimeout alone, so
once the archive sweeper or retention reaper had spent part of the fx
stop budget, a request held open could use up the reserve and fx
skipped every hook after the server, the database close included.
The drain now gets the shorter of ShutdownTimeout and what is left
less TailHookReserve, the clamp the Sentry flush already has.

The headroom test also sweeps the time earlier hooks spent; a new
test holds a request open against a stop context with only the
reserve left. The README and the reserve's comment say how the
reserve is derived and what a slow sweeper now costs.

Model: opus-5-5
This commit is contained in:
2026-10-02 16:05:09 +00:00
parent bf3df0312b
commit 6aa9907c8e
6 changed files with 180 additions and 36 deletions
+90
View File
@@ -1,6 +1,9 @@
package server_test
import (
"context"
"net/http"
"net/http/httptest"
"testing"
"time"
@@ -8,6 +11,93 @@ import (
"sneak.berlin/go/webhooker/internal/server"
)
// TestDrainBudget covers the clamp that keeps the HTTP drain from
// spending the tail hooks' share of the fx stop budget when the hooks
// before the server have already used part of it.
func TestDrainBudget(t *testing.T) {
t.Parallel()
tests := []struct {
name string
remaining time.Duration
want time.Duration
}{
{
name: "only the reserve is left",
remaining: server.TailHookReserve,
want: 0,
},
{
name: "earlier hooks spent part of the budget",
remaining: server.TailHookReserve + time.Second,
want: time.Second,
},
{
name: "capped at the nominal timeout",
remaining: time.Hour,
want: server.ShutdownTimeout,
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
require.Equal(t, tt.want, server.DrainBudget(tt.remaining))
})
}
}
// TestCleanShutdown_LeavesTailHookReserve stops the server with a
// request still in flight, after the hooks before it have spent all
// of the stop budget but TailHookReserve. The drain must give up at
// once rather than wait for the request: what is left belongs to the
// hooks after the server, the database close among them. A drain
// bounded only by ShutdownTimeout waits until the stop context
// expires, and fx then skips those hooks.
func TestCleanShutdown_LeavesTailHookReserve(t *testing.T) {
t.Parallel()
entered := make(chan struct{})
release := make(chan struct{})
ts := httptest.NewServer(http.HandlerFunc(
func(http.ResponseWriter, *http.Request) {
close(entered)
<-release
},
))
// Cleanups run last first: the request is released before Close,
// which waits for it.
t.Cleanup(ts.Close)
t.Cleanup(func() { close(release) })
req, err := http.NewRequestWithContext(
t.Context(), http.MethodGet, ts.URL, nil,
)
require.NoError(t, err)
go func() {
resp, err := ts.Client().Do(req)
if err == nil {
_ = resp.Body.Close()
}
}()
<-entered
stopCtx, cancel := context.WithTimeout(
t.Context(), server.TailHookReserve,
)
defer cancel()
server.CleanShutdownForTest(stopCtx, ts.Config)
require.NoError(
t, stopCtx.Err(), "the drain spent the tail hooks' reserve",
)
}
// TestSentryFlushBudget covers the clamp that keeps the Sentry flush
// from spending the tail hooks' share of the fx stop budget.
// sentry.Flush ignores the stop context, so without the clamp a