On SIGTERM, fx runs the OnStop hooks in reverse order. stop in internal/ircserver/server.go closes the listener and every connection's socket and returns at once; it does not wait for the connections' serve goroutines, whose cleanup (internal/ircserver/conn.go) is what runs BroadcastQuit and deletes the session. The database's OnStop then closes the connection and the process exits, so the teardown races the exit and loses. The HTTP server's OnStop (internal/server/server.go) does nothing either: its cleanShutdown runs from a goroutine nobody waits for, and srv.httpServer is written in serveUntilShutdown and read in cleanShutdown with no synchronisation.
Seen on eceb5f2: an IRC listener user shutirc, registered and on #shut; SIGTERM; the process exited within 0.2 s; afterwards sessions still held shutirc, with its client row and its #shut membership. After a restart, NICK shutirc from the same user got 433 Nickname is already in use, and the stale row stays until the idle cleanup removes it, by default up to 30 days (SESSION_IDLE_TIMEOUT); the channel peers never got a QUIT. README.md says closing an IRC listener connection ends the session as QUIT does, and that nicks are released when a session is destroyed. HTTP API sessions are meant to survive a restart; IRC listener ones are not.
Fix: ircserver.stop waits, bounded by a few seconds, for every connection's cleanup to finish before it returns, so the sessions are deleted before the database closes; the HTTP server's OnStop performs httpServer.Shutdown itself and waits, instead of leaving it to the signal goroutine. A sweep at startup cannot stand in for this, because the clients table does not record which transport a client came from.
Definition of done
After SIGTERM with IRC listener connections open, the database holds none of their sessions, clients or memberships, and their channel peers received a QUIT.
A test in internal/ircserver: a registered connection, Stop(), then the session is gone from the database.
One reviewed PR against next.
Model: fable-5-1
On SIGTERM, fx runs the OnStop hooks in reverse order. `stop` in `internal/ircserver/server.go` closes the listener and every connection's socket and returns at once; it does not wait for the connections' `serve` goroutines, whose `cleanup` (`internal/ircserver/conn.go`) is what runs `BroadcastQuit` and deletes the session. The database's OnStop then closes the connection and the process exits, so the teardown races the exit and loses. The HTTP server's OnStop (`internal/server/server.go`) does nothing either: its `cleanShutdown` runs from a goroutine nobody waits for, and `srv.httpServer` is written in `serveUntilShutdown` and read in `cleanShutdown` with no synchronisation.
Seen on `eceb5f2`: an IRC listener user `shutirc`, registered and on `#shut`; SIGTERM; the process exited within 0.2 s; afterwards `sessions` still held `shutirc`, with its client row and its `#shut` membership. After a restart, `NICK shutirc` from the same user got `433 Nickname is already in use`, and the stale row stays until the idle cleanup removes it, by default up to 30 days (`SESSION_IDLE_TIMEOUT`); the channel peers never got a QUIT. `README.md` says closing an IRC listener connection ends the session as QUIT does, and that nicks are released when a session is destroyed. HTTP API sessions are meant to survive a restart; IRC listener ones are not.
Fix: `ircserver.stop` waits, bounded by a few seconds, for every connection's cleanup to finish before it returns, so the sessions are deleted before the database closes; the HTTP server's OnStop performs `httpServer.Shutdown` itself and waits, instead of leaving it to the signal goroutine. A sweep at startup cannot stand in for this, because the `clients` table does not record which transport a client came from.
## Definition of done
1. After SIGTERM with IRC listener connections open, the database holds none of their sessions, clients or memberships, and their channel peers received a QUIT.
2. A test in `internal/ircserver`: a registered connection, `Stop()`, then the session is gone from the database.
3. One reviewed PR against `next`.
Model: fable-5-1
clawbot
self-assigned this 2026-10-20 17:15:26 +02:00
Blocking a user prevents them from interacting with repositories, such as opening or commenting on pull requests or issues. Learn more about blocking a user.
On SIGTERM, fx runs the OnStop hooks in reverse order.
stopininternal/ircserver/server.gocloses the listener and every connection's socket and returns at once; it does not wait for the connections'servegoroutines, whosecleanup(internal/ircserver/conn.go) is what runsBroadcastQuitand deletes the session. The database's OnStop then closes the connection and the process exits, so the teardown races the exit and loses. The HTTP server's OnStop (internal/server/server.go) does nothing either: itscleanShutdownruns from a goroutine nobody waits for, andsrv.httpServeris written inserveUntilShutdownand read incleanShutdownwith no synchronisation.Seen on
eceb5f2: an IRC listener usershutirc, registered and on#shut; SIGTERM; the process exited within 0.2 s; afterwardssessionsstill heldshutirc, with its client row and its#shutmembership. After a restart,NICK shutircfrom the same user got433 Nickname is already in use, and the stale row stays until the idle cleanup removes it, by default up to 30 days (SESSION_IDLE_TIMEOUT); the channel peers never got a QUIT.README.mdsays closing an IRC listener connection ends the session as QUIT does, and that nicks are released when a session is destroyed. HTTP API sessions are meant to survive a restart; IRC listener ones are not.Fix:
ircserver.stopwaits, bounded by a few seconds, for every connection's cleanup to finish before it returns, so the sessions are deleted before the database closes; the HTTP server's OnStop performshttpServer.Shutdownitself and waits, instead of leaving it to the signal goroutine. A sweep at startup cannot stand in for this, because theclientstable does not record which transport a client came from.Definition of done
internal/ircserver: a registered connection,Stop(), then the session is gone from the database.next.Model: fable-5-1