fix(server): shut down through fx so buffered reports flush (closes #22)
check / check (push) Failing after 1s

The server ran os.Exit at the end of its own goroutine, racing fx's
teardown and sometimes killing the process before reportbuf's OnStop
flushed — silently losing a full flush window of telemetry on every
restart, at exit 0. Shutdown now goes through fx.Shutdowner, so every
OnStop runs in order.

The http.Server is built synchronously in OnStart before the serving
goroutine, so shutdown can no longer race or nil-deref it. A listen
failure exits non-zero via fx.ExitCode(1). reportbuf's OnStop is guarded
by sync.Once. writeTimeout now exceeds the chi per-request budget so that
budget is reachable. Dead startupTime, exitCode, and cancelFunc fields
are gone. A new test asserts a buffered report reaches disk after the
lifecycle stops.

Model: opus-4-8
This commit is contained in:
2026-09-21 13:10:22 +00:00
parent f3895789d2
commit 924d4b7177
5 changed files with 154 additions and 100 deletions
+30 -11
View File
@@ -5,20 +5,31 @@ import (
"fmt"
"net/http"
"time"
"go.uber.org/fx"
)
const (
readTimeout = 10 * time.Second
readHeaderTimeout = 5 * time.Second
writeTimeout = 10 * time.Second
idleTimeout = 60 * time.Second
maxHeaderBytes = 1 << 20 // 1 MiB
// requestTimeout (routes.go) is the single per-request
// processing budget, enforced by chi's middleware.Timeout.
// writeTimeout must exceed that budget so a handler can write
// its 503 when the chi timeout fires; if it were shorter the
// server would abort the write first and the chi budget would
// be unreachable dead configuration.
writeTimeout = requestTimeout + 5*time.Second
)
func (s *Server) serveUntilShutdown() {
// newHTTPServer constructs the http.Server. It performs no I/O
// and does not start listening.
func (s *Server) newHTTPServer() *http.Server {
listenAddr := fmt.Sprintf(":%d", s.params.Config.Port)
s.httpServer = &http.Server{
return &http.Server{
Addr: listenAddr,
Handler: s,
MaxHeaderBytes: maxHeaderBytes,
@@ -27,21 +38,29 @@ func (s *Server) serveUntilShutdown() {
WriteTimeout: writeTimeout,
IdleTimeout: idleTimeout,
}
}
s.SetupRoutes()
// listenAndServe runs the listener until the server is shut
// down. A genuine listen failure (not the expected
// ErrServerClosed from a clean shutdown) requests process
// shutdown through fx with a non-zero exit code, so the failure
// is visible to any supervisor.
func (s *Server) listenAndServe() {
s.log.Info("http begin listen",
"listenaddr", listenAddr,
"listenaddr", s.httpServer.Addr,
"version", s.params.Globals.Version,
"buildarch", s.params.Globals.Buildarch,
)
err := s.httpServer.ListenAndServe()
if err != nil && !errors.Is(err, http.ErrServerClosed) {
s.log.Error("listen error", "error", err)
if err == nil || errors.Is(err, http.ErrServerClosed) {
return
}
if s.cancelFunc != nil {
s.cancelFunc()
}
s.log.Error("listen error", "error", err)
shutdownErr := s.shutdowner.Shutdown(fx.ExitCode(1))
if shutdownErr != nil {
s.log.Error("request shutdown failed", "error", shutdownErr)
}
}