Use one recovery routine at start and on every sweep tick (closes #542)
check / check (push) Waiting to run

Start-up recovery and the 60-second sweep were two parallel sets of
functions doing the same job. recoverDeliveries now does it, run once at
start and then on every tick from one goroutine. A retrying delivery is
scheduled after what is left of its backoff, now if nothing is left, so
backoffElapsed is gone. The run at start still takes every pending
delivery and opens a webhook database whose file is missing; a tick
still takes only pending deliveries untouched for 15 minutes, at most
500, and skips a missing file. A retrying delivery is taken before its
event is read. The README describes one routine.

Model: opus-5-5
This commit was merged in pull request #553.
This commit is contained in:
2026-10-15 21:57:05 +02:00
parent 0cdda43e6c
commit ded39aa158
16 changed files with 705 additions and 594 deletions
+9 -9
View File
@@ -6,10 +6,10 @@ import "sync"
//
// A delivery is owned from the moment a task for it is handed to a
// channel or to a retry timer until the engine has no further plan for
// it in memory. Restart recovery and both arms of the periodic sweep
// re-dispatch only deliveries the set does not hold, which is what
// makes them exact rather than a guess about how long a row has sat at
// pending.
// it in memory. The recovery routine, at start and on every tick of the
// sweep, re-dispatches only deliveries the set does not hold, which is
// what makes it exact rather than a guess about how long a row has sat
// at pending.
//
// This replaces reasoning from timestamps. A delivery's row says
// pending from creation until its outcome is written, which covers
@@ -25,8 +25,8 @@ import "sync"
// In-memory state is sufficient because a data directory admits one
// process: internal/datadir takes an flock on it at startup and a
// second instance refuses to run. Deliveries owned by a process that
// died are not in any successor's set, and restart recovery is what
// picks those up.
// died are not in any successor's set, and the recovery routine's run
// at start is what picks those up.
//
// References are counted rather than held as a plain set because
// ownership outlives the worker that took it. A target that schedules
@@ -61,9 +61,9 @@ func (s *inflightSet) retain(deliveryID string) {
// retainIdle takes the first reference on a delivery, and reports
// whether it got it. It fails when the engine already owns the
// delivery, which is what makes two claimants — restart recovery and
// the sweep run concurrently, or two sweep arms — mutually exclusive
// rather than merely atomic.
// delivery, which is what makes two claimants — Notify and the
// recovery routine, say — mutually exclusive rather than merely
// atomic.
func (s *inflightSet) retainIdle(deliveryID string) bool {
s.mu.Lock()
defer s.mu.Unlock()