sneak
cb5fbb4db7
check / check (push) Waiting to run
The receiver, resubmit and replay handlers now read and check the request and make one call. Engine.StoreEvent stores an event with a pending delivery for each active target and the totals in one transaction, then queues them; Engine.CreateReplay does the same for one replay delivery. The handlers take *delivery.Engine, so the Notifier interface and its bridge are gone, and the engine's notify is no longer exported. Archives and CircuitBreakers stay because the handler tests fake them. The tests that read a queued task moved to the engine's tests; the handler tests use an engine that is never started. Model: opus-5-5
111 lines
3.6 KiB
Go
111 lines
3.6 KiB
Go
package delivery
|
|
|
|
import "sync"
|
|
|
|
// inflightSet records which deliveries the engine currently owns.
|
|
//
|
|
// A delivery is owned from the moment a task for it is handed to a
|
|
// channel or to a retry timer until the engine has no further plan for
|
|
// it in memory. The recovery routine, at start and on every tick of the
|
|
// sweep, re-dispatches only deliveries the set does not hold, which is
|
|
// what makes it exact rather than a guess about how long a row has sat
|
|
// at pending.
|
|
//
|
|
// This replaces reasoning from timestamps. A delivery's row says
|
|
// pending from creation until its outcome is written, which covers
|
|
// four different situations — never dispatched, waiting in a channel,
|
|
// being attempted right now, and genuinely stranded — and no column
|
|
// distinguishes them. Only the engine knows which, and it knows
|
|
// exactly. `deliveryChannelSize` is 10000 against 10 workers, so a
|
|
// perfectly healthy delivery can wait far longer than any age bound
|
|
// worth setting before its attempt even begins; an age bound alone
|
|
// re-sends it. See
|
|
// https://git.eeqj.de/sneak/webhooker/issues/256.
|
|
//
|
|
// In-memory state is sufficient because a data directory admits one
|
|
// process: internal/datadir takes an flock on it at startup and a
|
|
// second instance refuses to run. Deliveries owned by a process that
|
|
// died are not in any successor's set, and the recovery routine's run
|
|
// at start is what picks those up.
|
|
//
|
|
// References are counted rather than held as a plain set because
|
|
// ownership outlives the worker that took it. A target that schedules
|
|
// a retry from inside Deliver adds a reference while the worker still
|
|
// holds one, so the delivery stays owned across the gap between the
|
|
// worker returning and the timer firing — the window in which a sweep
|
|
// would otherwise find the row at retrying and send it again.
|
|
//
|
|
// The zero value is ready to use, and the Engine holds one by value.
|
|
// That is deliberate: an engine built by a constructor that forgot to
|
|
// initialise this would not refuse to re-dispatch anything, and the
|
|
// symptom would be duplicate deliveries rather than a failure anybody
|
|
// notices.
|
|
type inflightSet struct {
|
|
mu sync.Mutex
|
|
ids map[string]int
|
|
}
|
|
|
|
// retain adds a reference to a delivery the caller already knows the
|
|
// engine owns, so that ownership survives the current holder letting
|
|
// go. It cannot fail.
|
|
func (s *inflightSet) retain(deliveryID string) {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
|
|
if s.ids == nil {
|
|
s.ids = make(map[string]int)
|
|
}
|
|
|
|
s.ids[deliveryID]++
|
|
}
|
|
|
|
// retainIdle takes the first reference on a delivery, and reports
|
|
// whether it got it. It fails when the engine already owns the
|
|
// delivery, which is what makes two claimants — notify and the
|
|
// recovery routine, say — mutually exclusive rather than merely
|
|
// atomic.
|
|
func (s *inflightSet) retainIdle(deliveryID string) bool {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
|
|
if s.ids[deliveryID] > 0 {
|
|
return false
|
|
}
|
|
|
|
if s.ids == nil {
|
|
s.ids = make(map[string]int)
|
|
}
|
|
|
|
s.ids[deliveryID] = 1
|
|
|
|
return true
|
|
}
|
|
|
|
// release drops one reference. The delivery becomes eligible for
|
|
// re-dispatch again once the last one goes.
|
|
func (s *inflightSet) release(deliveryID string) {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
|
|
n := s.ids[deliveryID] - 1
|
|
if n <= 0 {
|
|
delete(s.ids, deliveryID)
|
|
|
|
return
|
|
}
|
|
|
|
s.ids[deliveryID] = n
|
|
}
|
|
|
|
// held reports how many deliveries the engine currently owns. It
|
|
// exists so a test can assert that ownership is released rather than
|
|
// leaked: a reference that is never dropped hides its delivery from
|
|
// every sweep for the life of the process, which is the one way this
|
|
// mechanism can fail silently.
|
|
func (s *inflightSet) held() int {
|
|
s.mu.Lock()
|
|
defer s.mu.Unlock()
|
|
|
|
return len(s.ids)
|
|
}
|