Bound each shutdown stage by the stop deadline (closes #100)
check / check (push) Canceled after 0s
check / check (push) Canceled after 0s
Shutdown passes its context to every stage: the streamer, the four handlers, the WHOIS fetcher, the database maintainer and the HTTP server each stop waiting when it ends. A handler then skips its last write and logs how many updates it did not write, and the HTTP server closes the connections still open. The database is checkpointed and closed last, and left open once the deadline has passed, since a write may still hold it. The stop hook gives Shutdown 8 seconds and fx 9, inside the 10 seconds upaas and docker stop allow, so a stop signal exits 0. Model: opus-5-5
This commit is contained in:
@@ -191,7 +191,11 @@ routewatch/
|
||||
batches for performance
|
||||
- **Backpressure**: Probabilistic message dropping when queues exceed 50%
|
||||
capacity
|
||||
- **Graceful Shutdown**: 60-second timeout, flushes all pending batches
|
||||
- **Graceful Shutdown**: on SIGTERM or SIGINT the streamer stops, the handlers
|
||||
write the batches they hold, the HTTP server stops and the database is
|
||||
checkpointed and closed, all within 8 seconds, and the daemon exits with
|
||||
status 0. What is not written by then is lost, and the handler logs how many
|
||||
updates (see "Running under upaas").
|
||||
- **Reconnection**: Any end of the stream other than shutdown, including RIS
|
||||
Live ending the response normally, marks the feed disconnected and reconnects
|
||||
with exponential backoff (5s-320s). The backoff resets after a stream that
|
||||
@@ -345,6 +349,19 @@ What the [upaas](https://git.eeqj.de/sneak/upaas) app needs:
|
||||
`/.well-known/healthcheck.json` on the container port. upaas reads the
|
||||
container's health 60 seconds after a deploy and fails the deploy unless it is
|
||||
`healthy`.
|
||||
- Stop grace: the container needs 9 seconds between SIGTERM and SIGKILL. upaas
|
||||
gives 10 (`stopTimeoutSeconds` in its `internal/docker/client.go`), as
|
||||
`docker stop` does by default. The daemon allows 8 seconds for the handlers to
|
||||
write what they hold and for the requests in progress to finish, then stops
|
||||
the HTTP server, closes the database and exits with status 0. A longer grace
|
||||
changes nothing. What the handlers have not written after 8 seconds is lost,
|
||||
and each logs how many updates: when the prefix handler has fallen behind the
|
||||
feed, its batch holds up to 250,000 updates, which take about 20 seconds to
|
||||
write, so a stop then loses them, withdrawals and peer downs included. A route
|
||||
withdrawn in them stays in the live route table until its peer announces or
|
||||
withdraws the prefix again or its session goes down. When the 8 seconds run
|
||||
out the database is not closed, and SQLite rolls back a write the exit cut off
|
||||
when the database next opens.
|
||||
|
||||
## Development
|
||||
|
||||
|
||||
Reference in New Issue
Block a user