sneak
c0c13ec80c
All checks were successful
check / check (push) Successful in 3m16s
fx defaults the stop timeout to 15s and the Dockerfile sets no STOPSIGNAL or grace override, so Docker's 10s default SIGKILLs the process five seconds before the bound can fire. Everything gated on it — including the "shutdown timed out, goroutines still running" error log that tells an operator a component is wedged — was unreachable in the image this repo produces. Set fx.StopTimeout to 5s: inside the grace with headroom for signal delivery and process exit, and equal to the HTTP server's own drain budget so the first hook can spend its whole budget without the sequence bound truncating it. The option set moves into newApp() so a test can read (*fx.App).StopTimeout() back and pin it against drift; dropping the option makes that test report fx's 15s default. Also fix a latent coin flip in the shared stop-hook waiter. It selected on the drained channel against ctx.Done() with no preamble, and select picks uniformly among ready cases, so a component that drained against an already-expired context reported a timeout about half the time. Not reachable through fx, which re-checks ctx.Err() before each hook, but the helper is shared and a direct caller can reach it. waitDone now settles the drained case in a non-blocking preamble first; the test drives it over 1000 passes, so a restored coin flip cannot pass by luck. README records the timeout and its relationship to the container stop grace: that lowering the grace below it puts SIGKILL back in front of the bound, and that an expired stop context makes fx skip its remaining hooks, so a wedge in the first-stopped component means the database close never runs. Adds the missing internal/lifecycle/ entry to the Package Layout tree.
81 lines
2.0 KiB
Go
81 lines
2.0 KiB
Go
// Package lifecycle holds helpers shared by the components that
|
|
// register fx start and stop hooks.
|
|
package lifecycle
|
|
|
|
import (
|
|
"context"
|
|
"fmt"
|
|
"log/slog"
|
|
"sync"
|
|
)
|
|
|
|
// WaitForShutdown waits for wg to drain, bounded by ctx.
|
|
//
|
|
// fx hands OnStop a context carrying the application's stop
|
|
// timeout. A bare wg.Wait() discards that deadline, so a single
|
|
// goroutine that never observes cancellation — a delivery target
|
|
// that never returns, a SQLite operation blocked on a lock —
|
|
// hangs the process forever instead of letting it exit when the
|
|
// timeout expires, which is exactly when a clean shutdown matters
|
|
// most.
|
|
//
|
|
// On timeout it logs at error naming component and returns an
|
|
// error: the goroutines are still running, and reporting success
|
|
// would hide an unclean shutdown from the operator. The waiting
|
|
// goroutine outlives this call and exits when (if) wg drains; it
|
|
// holds nothing but the channel it closes.
|
|
func WaitForShutdown(
|
|
ctx context.Context,
|
|
log *slog.Logger,
|
|
component string,
|
|
wg *sync.WaitGroup,
|
|
) error {
|
|
done := make(chan struct{})
|
|
|
|
go func() {
|
|
defer close(done)
|
|
|
|
wg.Wait()
|
|
}()
|
|
|
|
return waitDone(ctx, log, component, done)
|
|
}
|
|
|
|
// waitDone waits for done to close, bounded by ctx.
|
|
//
|
|
// The non-blocking preamble is load-bearing. When the component has
|
|
// already drained and ctx has already expired, both cases of the
|
|
// bounded select are ready and Go picks between them uniformly at
|
|
// random, so a clean shutdown would be reported as a timeout about
|
|
// half the time. Draining wins: the goroutines are gone, and there
|
|
// is nothing left for the operator to act on.
|
|
func waitDone(
|
|
ctx context.Context,
|
|
log *slog.Logger,
|
|
component string,
|
|
done <-chan struct{},
|
|
) error {
|
|
select {
|
|
case <-done:
|
|
return nil
|
|
default:
|
|
}
|
|
|
|
select {
|
|
case <-done:
|
|
return nil
|
|
case <-ctx.Done():
|
|
log.Error(
|
|
"shutdown timed out, goroutines still running",
|
|
"component", component,
|
|
"error", ctx.Err(),
|
|
)
|
|
|
|
return fmt.Errorf(
|
|
"%s: shutdown timed out, "+
|
|
"goroutines still running: %w",
|
|
component, ctx.Err(),
|
|
)
|
|
}
|
|
}
|