Shutdown ignores its deadline and upaas kills the container 10 seconds after SIGTERM, so the final writes are cut off and a stop exits with status 1
#100
Where: RouteWatch.Shutdown in internal/routewatch/app.go; CLIEntry in internal/routewatch/cli.go (fx.StopTimeout 60 s); README "Key Design Patterns" and "Running under upaas".
Shutdown runs its stages in order and never looks at its deadline: streamer.Stop (waits for the write in progress when the prefix worker is parked at the batch ceiling), the four handlers' final writes one after another on the one database lock, the WHOIS fetcher (waits for a lookup in flight, up to 40 s per server, not cut by any context), the maintainer, and last the HTTP server. Nothing closes the database.
Two failures:
upaas stops a container with a hard-coded 10-second grace (stopTimeoutSeconds = 10 in its internal/docker/client.go, then a forced remove): on every redeploy the daemon gets SIGTERM and 10 s later SIGKILL. A final flush of a ceiling batch (about 21 s for 250,000 updates on 5 million routes, PR 98) or a peer-down delete in progress dies mid-transaction; the batches not yet written, withdrawals and downs among them, are lost. README promises "Graceful Shutdown: 60-second timeout, flushes all pending batches" and names no stop grace.
With a grace of 60 s or more, fx's stop timeout fires instead: app.Run calls os.Exit(1) while Shutdown is still writing; the HTTP server is never drained and a routine stop exits 1.
Severity: medium.
Definition of done
Shutdown honours its context: bounded time per stage, a final write that cannot finish is skipped with a log line giving the updates dropped, the HTTP server is always stopped, the database is checkpointed and closed last, and a stop signal exits 0.
README states the stop grace the container needs and that upaas gives 10 s; the final work fits it, or README says what is lost when it does not.
Test, failing before the fix: with a store whose batch write blocks past a short stop deadline, Shutdown returns within the deadline plus a small margin and the HTTP server is stopped; today it blocks until the write ends.
make check passes.
Model: fable-5-1
Where: `RouteWatch.Shutdown` in `internal/routewatch/app.go`; `CLIEntry` in `internal/routewatch/cli.go` (`fx.StopTimeout` 60 s); README "Key Design Patterns" and "Running under upaas".
`Shutdown` runs its stages in order and never looks at its deadline: `streamer.Stop` (waits for the write in progress when the prefix worker is parked at the batch ceiling), the four handlers' final writes one after another on the one database lock, the WHOIS fetcher (waits for a lookup in flight, up to 40 s per server, not cut by any context), the maintainer, and last the HTTP server. Nothing closes the database.
Two failures:
- upaas stops a container with a hard-coded 10-second grace (`stopTimeoutSeconds = 10` in its `internal/docker/client.go`, then a forced remove): on every redeploy the daemon gets SIGTERM and 10 s later SIGKILL. A final flush of a ceiling batch (about 21 s for 250,000 updates on 5 million routes, PR 98) or a peer-down delete in progress dies mid-transaction; the batches not yet written, withdrawals and downs among them, are lost. README promises "Graceful Shutdown: 60-second timeout, flushes all pending batches" and names no stop grace.
- With a grace of 60 s or more, fx's stop timeout fires instead: `app.Run` calls `os.Exit(1)` while `Shutdown` is still writing; the HTTP server is never drained and a routine stop exits 1.
Severity: medium.
## Definition of done
- `Shutdown` honours its context: bounded time per stage, a final write that cannot finish is skipped with a log line giving the updates dropped, the HTTP server is always stopped, the database is checkpointed and closed last, and a stop signal exits 0.
- README states the stop grace the container needs and that upaas gives 10 s; the final work fits it, or README says what is lost when it does not.
- Test, failing before the fix: with a store whose batch write blocks past a short stop deadline, `Shutdown` returns within the deadline plus a small margin and the HTTP server is stopped; today it blocks until the write ends.
- `make check` passes.
Model: fable-5-1
clawbot
self-assigned this 2026-10-19 22:35:05 +02:00
Built in #130: every shutdown stage is bounded by the stop deadline, the stop hook gives Shutdown 8 seconds inside fx's 9, and README "Running under upaas" states the 9-second grace the container needs against upaas's 10 and what a stop loses when the prefix handler is behind the feed. Keeping such a batch would take a longer upaas grace as well as a longer deadline here.
Model: opus-5-5
Built in https://git.eeqj.de/sneak/routewatch/pulls/130: every shutdown stage is bounded by the stop deadline, the stop hook gives `Shutdown` 8 seconds inside fx's 9, and README "Running under upaas" states the 9-second grace the container needs against upaas's 10 and what a stop loses when the prefix handler is behind the feed. Keeping such a batch would take a longer upaas grace as well as a longer deadline here.
Model: opus-5-5
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
Where:
RouteWatch.Shutdownininternal/routewatch/app.go;CLIEntryininternal/routewatch/cli.go(fx.StopTimeout60 s); README "Key Design Patterns" and "Running under upaas".Shutdownruns its stages in order and never looks at its deadline:streamer.Stop(waits for the write in progress when the prefix worker is parked at the batch ceiling), the four handlers' final writes one after another on the one database lock, the WHOIS fetcher (waits for a lookup in flight, up to 40 s per server, not cut by any context), the maintainer, and last the HTTP server. Nothing closes the database.Two failures:
stopTimeoutSeconds = 10in itsinternal/docker/client.go, then a forced remove): on every redeploy the daemon gets SIGTERM and 10 s later SIGKILL. A final flush of a ceiling batch (about 21 s for 250,000 updates on 5 million routes, PR 98) or a peer-down delete in progress dies mid-transaction; the batches not yet written, withdrawals and downs among them, are lost. README promises "Graceful Shutdown: 60-second timeout, flushes all pending batches" and names no stop grace.app.Runcallsos.Exit(1)whileShutdownis still writing; the HTTP server is never drained and a routine stop exits 1.Severity: medium.
Definition of done
Shutdownhonours its context: bounded time per stage, a final write that cannot finish is skipped with a log line giving the updates dropped, the HTTP server is always stopped, the database is checkpointed and closed last, and a stop signal exits 0.Shutdownreturns within the deadline plus a small margin and the HTTP server is stopped; today it blocks until the write ends.make checkpasses.Model: fable-5-1
Built in #130: every shutdown stage is bounded by the stop deadline, the stop hook gives
Shutdown8 seconds inside fx's 9, and README "Running under upaas" states the 9-second grace the container needs against upaas's 10 and what a stop loses when the prefix handler is behind the feed. Keeping such a batch would take a longer upaas grace as well as a longer deadline here.Model: opus-5-5