feat: webhooker 1.0 MVP — entity rename, core engine, delivery, management UI #16
48
README.md
48
README.md
@@ -291,20 +291,23 @@ events should be forwarded.
|
|||||||
| `id` | UUID | Primary key |
|
| `id` | UUID | Primary key |
|
||||||
| `webhook_id` | UUID | Foreign key → Webhook |
|
| `webhook_id` | UUID | Foreign key → Webhook |
|
||||||
| `name` | string | Human-readable name |
|
| `name` | string | Human-readable name |
|
||||||
| `type` | TargetType | One of: `http`, `retry`, `database`, `log` |
|
| `type` | TargetType | One of: `http`, `database`, `log` |
|
||||||
| `active` | boolean | Whether deliveries are enabled (default: true) |
|
| `active` | boolean | Whether deliveries are enabled (default: true) |
|
||||||
| `config` | JSON text | Type-specific configuration |
|
| `config` | JSON text | Type-specific configuration |
|
||||||
| `max_retries` | integer | Maximum retry attempts (for retry targets) |
|
| `max_retries` | integer | Maximum retry attempts for HTTP targets (0 = fire-and-forget, >0 = retries with backoff) |
|
||||||
| `max_queue_size` | integer | Maximum queued deliveries (for retry targets) |
|
| `max_queue_size` | integer | Maximum queued deliveries (for HTTP targets with retries) |
|
||||||
|
|
||||||
**Relations:** Belongs to Webhook. Has many Deliveries.
|
**Relations:** Belongs to Webhook. Has many Deliveries.
|
||||||
|
|
||||||
**Target types:**
|
**Target types:**
|
||||||
|
|
||||||
- **`http`** — Forward the event as an HTTP POST to a configured URL.
|
- **`http`** — Forward the event as an HTTP POST to a configured URL.
|
||||||
Fire-and-forget: a single attempt with no retries.
|
Behavior depends on `max_retries`: when `max_retries` is 0 (the
|
||||||
- **`retry`** — Forward the event via HTTP POST with automatic retry on
|
default), the target operates in fire-and-forget mode — a single
|
||||||
failure. Uses exponential backoff up to `max_retries` attempts.
|
attempt with no retries and no circuit breaker. When `max_retries` is
|
||||||
|
greater than 0, failed deliveries are retried with exponential backoff
|
||||||
|
up to `max_retries` attempts, protected by a per-target circuit
|
||||||
|
breaker.
|
||||||
- **`database`** — Confirm the event is stored in the webhook's
|
- **`database`** — Confirm the event is stored in the webhook's
|
||||||
per-webhook database (no external delivery). Since events are always
|
per-webhook database (no external delivery). Since events are always
|
||||||
written to the per-webhook DB on ingestion, this target marks delivery
|
written to the per-webhook DB on ingestion, this target marks delivery
|
||||||
@@ -495,10 +498,12 @@ External Service
|
|||||||
┌── bounded worker pool (N workers) ──┐
|
┌── bounded worker pool (N workers) ──┐
|
||||||
▼ ▼ ▼
|
▼ ▼ ▼
|
||||||
┌────────────┐ ┌────────────┐ ┌────────────┐
|
┌────────────┐ ┌────────────┐ ┌────────────┐
|
||||||
│ HTTP Target│ │Retry Target│ │ Log Target │
|
│ HTTP Target│ │ HTTP Target│ │ Log Target │
|
||||||
│ (1 attempt)│ │ (backoff + │ │ (stdout) │
|
│(max_retries│ │(max_retries│ │ (stdout) │
|
||||||
└────────────┘ │ circuit │ └────────────┘
|
│ == 0) │ │ > 0, │ └────────────┘
|
||||||
│ breaker) │
|
│ fire+forget│ │ backoff + │
|
||||||
|
└────────────┘ │ circuit │
|
||||||
|
│ breaker) │
|
||||||
└────────────┘
|
└────────────┘
|
||||||
```
|
```
|
||||||
|
|
||||||
@@ -553,9 +558,9 @@ This means:
|
|||||||
durable fallback that ensures no retry is permanently lost, even under
|
durable fallback that ensures no retry is permanently lost, even under
|
||||||
extreme backpressure.
|
extreme backpressure.
|
||||||
|
|
||||||
### Circuit Breaker (Retry Targets)
|
### Circuit Breaker (HTTP Targets with Retries)
|
||||||
|
|
||||||
Retry targets are protected by a **per-target circuit breaker** that
|
HTTP targets with `max_retries` > 0 are protected by a **per-target circuit breaker** that
|
||||||
prevents hammering a down target with repeated failed delivery attempts.
|
prevents hammering a down target with repeated failed delivery attempts.
|
||||||
The circuit breaker is in-memory only and resets on restart (which is
|
The circuit breaker is in-memory only and resets on restart (which is
|
||||||
fine — startup recovery rescans the database anyway).
|
fine — startup recovery rescans the database anyway).
|
||||||
@@ -594,9 +599,10 @@ fine — startup recovery rescans the database anyway).
|
|||||||
- **Failure threshold:** 5 consecutive failures before opening
|
- **Failure threshold:** 5 consecutive failures before opening
|
||||||
- **Cooldown:** 30 seconds in open state before probing
|
- **Cooldown:** 30 seconds in open state before probing
|
||||||
|
|
||||||
**Scope:** Circuit breakers only apply to **retry** target types. HTTP
|
**Scope:** Circuit breakers only apply to **HTTP targets with
|
||||||
targets (fire-and-forget), database targets (local operations), and log
|
`max_retries` > 0**. Fire-and-forget HTTP targets (`max_retries` == 0),
|
||||||
targets (stdout) do not use circuit breakers.
|
database targets (local operations), and log targets (stdout) do not use
|
||||||
|
circuit breakers.
|
||||||
|
|
||||||
When a circuit is open and a new delivery arrives, the engine marks the
|
When a circuit is open and a new delivery arrives, the engine marks the
|
||||||
delivery as `retrying` and schedules a retry timer for after the
|
delivery as `retrying` and schedules a retry timer for after the
|
||||||
@@ -704,7 +710,7 @@ webhooker/
|
|||||||
│ │ └── globals.go # Build-time variables (appname, version, arch)
|
│ │ └── globals.go # Build-time variables (appname, version, arch)
|
||||||
│ ├── delivery/
|
│ ├── delivery/
|
||||||
│ │ ├── engine.go # Event-driven delivery engine (channel + timer based)
|
│ │ ├── engine.go # Event-driven delivery engine (channel + timer based)
|
||||||
│ │ └── circuit_breaker.go # Per-target circuit breaker for retry targets
|
│ │ └── circuit_breaker.go # Per-target circuit breaker for HTTP targets with retries
|
||||||
│ ├── handlers/
|
│ ├── handlers/
|
||||||
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
|
│ │ ├── handlers.go # Base handler struct, JSON helpers, template rendering
|
||||||
│ │ ├── auth.go # Login, logout handlers
|
│ │ ├── auth.go # Login, logout handlers
|
||||||
@@ -838,8 +844,8 @@ linted, tested, and compiled.
|
|||||||
### Completed: Core Webhook Engine (Phase 2 of MVP)
|
### Completed: Core Webhook Engine (Phase 2 of MVP)
|
||||||
- [x] Implement webhook reception and event storage at `/webhook/{uuid}`
|
- [x] Implement webhook reception and event storage at `/webhook/{uuid}`
|
||||||
- [x] Build event processing and target delivery engine
|
- [x] Build event processing and target delivery engine
|
||||||
- [x] Implement HTTP target type (fire-and-forget POST)
|
- [x] Implement HTTP target type (fire-and-forget with max_retries=0,
|
||||||
- [x] Implement retry target type (exponential backoff)
|
retries with exponential backoff when max_retries>0)
|
||||||
- [x] Implement database target type (store events in per-webhook DB)
|
- [x] Implement database target type (store events in per-webhook DB)
|
||||||
- [x] Implement log target type (console output)
|
- [x] Implement log target type (console output)
|
||||||
- [x] Webhook management pages (list, create, edit, delete)
|
- [x] Webhook management pages (list, create, edit, delete)
|
||||||
@@ -861,9 +867,9 @@ linted, tested, and compiled.
|
|||||||
(events are already in the per-webhook DB)
|
(events are already in the per-webhook DB)
|
||||||
- [x] Parallel fan-out: all targets for an event are delivered via
|
- [x] Parallel fan-out: all targets for an event are delivered via
|
||||||
the bounded worker pool (no goroutine-per-target)
|
the bounded worker pool (no goroutine-per-target)
|
||||||
- [x] Circuit breaker for retry targets: tracks consecutive failures
|
- [x] Circuit breaker for HTTP targets with retries: tracks consecutive
|
||||||
per target, opens after 5 failures (30s cooldown), half-open
|
failures per target, opens after 5 failures (30s cooldown),
|
||||||
probe to test recovery
|
half-open probe to test recovery
|
||||||
|
|
||||||
### Remaining: Core Features
|
### Remaining: Core Features
|
||||||
- [ ] Per-webhook rate limiting in the receiver handler
|
- [ ] Per-webhook rate limiting in the receiver handler
|
||||||
|
|||||||
@@ -92,6 +92,16 @@ func (d *Database) migrate() error {
|
|||||||
}
|
}
|
||||||
d.log.Info("database migrations completed")
|
d.log.Info("database migrations completed")
|
||||||
|
|
||||||
|
// Data migration: merge "retry" target type into "http".
|
||||||
|
// Previously there were two separate HTTP-based target types: "http"
|
||||||
|
// (fire-and-forget) and "retry" (with retries). Now "http" handles
|
||||||
|
// both: max_retries=0 means fire-and-forget, max_retries>0 enables
|
||||||
|
// retries with exponential backoff and circuit breaker.
|
||||||
|
if err := d.db.Exec("UPDATE targets SET type = 'http' WHERE type = 'retry'").Error; err != nil {
|
||||||
|
d.log.Error("failed to migrate retry targets to http", "error", err)
|
||||||
|
return err
|
||||||
|
}
|
||||||
|
|
||||||
// Check if admin user exists
|
// Check if admin user exists
|
||||||
var userCount int64
|
var userCount int64
|
||||||
if err := d.db.Model(&User{}).Count(&userCount).Error; err != nil {
|
if err := d.db.Model(&User{}).Count(&userCount).Error; err != nil {
|
||||||
|
|||||||
@@ -5,7 +5,6 @@ type TargetType string
|
|||||||
|
|
||||||
const (
|
const (
|
||||||
TargetTypeHTTP TargetType = "http"
|
TargetTypeHTTP TargetType = "http"
|
||||||
TargetTypeRetry TargetType = "retry"
|
|
||||||
TargetTypeDatabase TargetType = "database"
|
TargetTypeDatabase TargetType = "database"
|
||||||
TargetTypeLog TargetType = "log"
|
TargetTypeLog TargetType = "log"
|
||||||
)
|
)
|
||||||
@@ -22,7 +21,7 @@ type Target struct {
|
|||||||
// Configuration fields (JSON stored based on type)
|
// Configuration fields (JSON stored based on type)
|
||||||
Config string `gorm:"type:text" json:"config"` // JSON configuration
|
Config string `gorm:"type:text" json:"config"` // JSON configuration
|
||||||
|
|
||||||
// For retry targets
|
// For HTTP targets (max_retries=0 means fire-and-forget, >0 enables retries with backoff)
|
||||||
MaxRetries int `json:"max_retries,omitempty"`
|
MaxRetries int `json:"max_retries,omitempty"`
|
||||||
MaxQueueSize int `json:"max_queue_size,omitempty"`
|
MaxQueueSize int `json:"max_queue_size,omitempty"`
|
||||||
|
|
||||||
|
|||||||
@@ -133,7 +133,8 @@ type Engine struct {
|
|||||||
workers int
|
workers int
|
||||||
|
|
||||||
// circuitBreakers stores a *CircuitBreaker per target ID. Only used
|
// circuitBreakers stores a *CircuitBreaker per target ID. Only used
|
||||||
// for retry targets — HTTP, database, and log targets do not need
|
// for HTTP targets with MaxRetries > 0 — fire-and-forget HTTP targets
|
||||||
|
// (MaxRetries == 0), database targets, and log targets do not need
|
||||||
// circuit breakers because they either fire once or are local ops.
|
// circuit breakers because they either fire once or are local ops.
|
||||||
circuitBreakers sync.Map
|
circuitBreakers sync.Map
|
||||||
}
|
}
|
||||||
@@ -829,9 +830,7 @@ func (e *Engine) sweepWebhookRetries(ctx context.Context, webhookID string) {
|
|||||||
func (e *Engine) processDelivery(ctx context.Context, webhookDB *gorm.DB, d *database.Delivery, task *DeliveryTask) {
|
func (e *Engine) processDelivery(ctx context.Context, webhookDB *gorm.DB, d *database.Delivery, task *DeliveryTask) {
|
||||||
switch d.Target.Type {
|
switch d.Target.Type {
|
||||||
case database.TargetTypeHTTP:
|
case database.TargetTypeHTTP:
|
||||||
e.deliverHTTP(ctx, webhookDB, d)
|
e.deliverHTTP(ctx, webhookDB, d, task)
|
||||||
case database.TargetTypeRetry:
|
|
||||||
e.deliverRetry(ctx, webhookDB, d, task)
|
|
||||||
case database.TargetTypeDatabase:
|
case database.TargetTypeDatabase:
|
||||||
e.deliverDatabase(webhookDB, d)
|
e.deliverDatabase(webhookDB, d)
|
||||||
case database.TargetTypeLog:
|
case database.TargetTypeLog:
|
||||||
@@ -845,47 +844,43 @@ func (e *Engine) processDelivery(ctx context.Context, webhookDB *gorm.DB, d *dat
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func (e *Engine) deliverHTTP(_ context.Context, webhookDB *gorm.DB, d *database.Delivery) {
|
func (e *Engine) deliverHTTP(_ context.Context, webhookDB *gorm.DB, d *database.Delivery, task *DeliveryTask) {
|
||||||
cfg, err := e.parseHTTPConfig(d.Target.Config)
|
cfg, err := e.parseHTTPConfig(d.Target.Config)
|
||||||
if err != nil {
|
if err != nil {
|
||||||
e.log.Error("invalid HTTP target config",
|
e.log.Error("invalid HTTP target config",
|
||||||
"target_id", d.TargetID,
|
"target_id", d.TargetID,
|
||||||
"error", err,
|
"error", err,
|
||||||
)
|
)
|
||||||
e.recordResult(webhookDB, d, 1, false, 0, "", err.Error(), 0)
|
|
||||||
e.updateDeliveryStatus(webhookDB, d, database.DeliveryStatusFailed)
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
statusCode, respBody, duration, err := e.doHTTPRequest(cfg, &d.Event)
|
|
||||||
|
|
||||||
success := err == nil && statusCode >= 200 && statusCode < 300
|
|
||||||
errMsg := ""
|
|
||||||
if err != nil {
|
|
||||||
errMsg = err.Error()
|
|
||||||
}
|
|
||||||
|
|
||||||
e.recordResult(webhookDB, d, 1, success, statusCode, respBody, errMsg, duration)
|
|
||||||
|
|
||||||
if success {
|
|
||||||
e.updateDeliveryStatus(webhookDB, d, database.DeliveryStatusDelivered)
|
|
||||||
} else {
|
|
||||||
e.updateDeliveryStatus(webhookDB, d, database.DeliveryStatusFailed)
|
|
||||||
}
|
|
||||||
}
|
|
||||||
|
|
||||||
func (e *Engine) deliverRetry(_ context.Context, webhookDB *gorm.DB, d *database.Delivery, task *DeliveryTask) {
|
|
||||||
cfg, err := e.parseHTTPConfig(d.Target.Config)
|
|
||||||
if err != nil {
|
|
||||||
e.log.Error("invalid retry target config",
|
|
||||||
"target_id", d.TargetID,
|
|
||||||
"error", err,
|
|
||||||
)
|
|
||||||
e.recordResult(webhookDB, d, task.AttemptNum, false, 0, "", err.Error(), 0)
|
e.recordResult(webhookDB, d, task.AttemptNum, false, 0, "", err.Error(), 0)
|
||||||
e.updateDeliveryStatus(webhookDB, d, database.DeliveryStatusFailed)
|
e.updateDeliveryStatus(webhookDB, d, database.DeliveryStatusFailed)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
|
maxRetries := d.Target.MaxRetries
|
||||||
|
|
||||||
|
// Fire-and-forget mode: max_retries == 0 means attempt once with no
|
||||||
|
// circuit breaker and no retry scheduling.
|
||||||
|
if maxRetries == 0 {
|
||||||
|
statusCode, respBody, duration, reqErr := e.doHTTPRequest(cfg, &d.Event)
|
||||||
|
|
||||||
|
success := reqErr == nil && statusCode >= 200 && statusCode < 300
|
||||||
|
errMsg := ""
|
||||||
|
if reqErr != nil {
|
||||||
|
errMsg = reqErr.Error()
|
||||||
|
}
|
||||||
|
|
||||||
|
e.recordResult(webhookDB, d, 1, success, statusCode, respBody, errMsg, duration)
|
||||||
|
|
||||||
|
if success {
|
||||||
|
e.updateDeliveryStatus(webhookDB, d, database.DeliveryStatusDelivered)
|
||||||
|
} else {
|
||||||
|
e.updateDeliveryStatus(webhookDB, d, database.DeliveryStatusFailed)
|
||||||
|
}
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
// Retry mode: max_retries > 0 — use circuit breaker and exponential backoff.
|
||||||
|
|
||||||
// Check the circuit breaker for this target before attempting delivery.
|
// Check the circuit breaker for this target before attempting delivery.
|
||||||
cb := e.getCircuitBreaker(task.TargetID)
|
cb := e.getCircuitBreaker(task.TargetID)
|
||||||
if !cb.Allow() {
|
if !cb.Allow() {
|
||||||
@@ -910,12 +905,12 @@ func (e *Engine) deliverRetry(_ context.Context, webhookDB *gorm.DB, d *database
|
|||||||
|
|
||||||
// Attempt delivery immediately — backoff is handled by the timer
|
// Attempt delivery immediately — backoff is handled by the timer
|
||||||
// that triggered this call, not by polling.
|
// that triggered this call, not by polling.
|
||||||
statusCode, respBody, duration, err := e.doHTTPRequest(cfg, &d.Event)
|
statusCode, respBody, duration, reqErr := e.doHTTPRequest(cfg, &d.Event)
|
||||||
|
|
||||||
success := err == nil && statusCode >= 200 && statusCode < 300
|
success := reqErr == nil && statusCode >= 200 && statusCode < 300
|
||||||
errMsg := ""
|
errMsg := ""
|
||||||
if err != nil {
|
if reqErr != nil {
|
||||||
errMsg = err.Error()
|
errMsg = reqErr.Error()
|
||||||
}
|
}
|
||||||
|
|
||||||
e.recordResult(webhookDB, d, attemptNum, success, statusCode, respBody, errMsg, duration)
|
e.recordResult(webhookDB, d, attemptNum, success, statusCode, respBody, errMsg, duration)
|
||||||
@@ -929,11 +924,6 @@ func (e *Engine) deliverRetry(_ context.Context, webhookDB *gorm.DB, d *database
|
|||||||
// Delivery failed — record failure in circuit breaker
|
// Delivery failed — record failure in circuit breaker
|
||||||
cb.RecordFailure()
|
cb.RecordFailure()
|
||||||
|
|
||||||
maxRetries := d.Target.MaxRetries
|
|
||||||
if maxRetries <= 0 {
|
|
||||||
maxRetries = 5 // default
|
|
||||||
}
|
|
||||||
|
|
||||||
if attemptNum >= maxRetries {
|
if attemptNum >= maxRetries {
|
||||||
e.updateDeliveryStatus(webhookDB, d, database.DeliveryStatusFailed)
|
e.updateDeliveryStatus(webhookDB, d, database.DeliveryStatusFailed)
|
||||||
} else {
|
} else {
|
||||||
|
|||||||
@@ -141,13 +141,26 @@ func TestDeliverHTTP_Success(t *testing.T) {
|
|||||||
defer ts.Close()
|
defer ts.Close()
|
||||||
|
|
||||||
e := testEngine(t, 1)
|
e := testEngine(t, 1)
|
||||||
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
event := seedEvent(t, db, `{"hello":"world"}`)
|
event := seedEvent(t, db, `{"hello":"world"}`)
|
||||||
delivery := seedDelivery(t, db, event.ID, uuid.New().String(), database.DeliveryStatusPending)
|
delivery := seedDelivery(t, db, event.ID, targetID, database.DeliveryStatusPending)
|
||||||
|
|
||||||
|
task := &DeliveryTask{
|
||||||
|
DeliveryID: delivery.ID,
|
||||||
|
EventID: event.ID,
|
||||||
|
WebhookID: event.WebhookID,
|
||||||
|
TargetID: targetID,
|
||||||
|
TargetName: "test-http",
|
||||||
|
TargetType: database.TargetTypeHTTP,
|
||||||
|
TargetConfig: newHTTPTargetConfig(ts.URL),
|
||||||
|
MaxRetries: 0,
|
||||||
|
AttemptNum: 1,
|
||||||
|
}
|
||||||
|
|
||||||
d := &database.Delivery{
|
d := &database.Delivery{
|
||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
TargetID: delivery.TargetID,
|
TargetID: targetID,
|
||||||
Status: database.DeliveryStatusPending,
|
Status: database.DeliveryStatusPending,
|
||||||
Event: event,
|
Event: event,
|
||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
@@ -158,7 +171,7 @@ func TestDeliverHTTP_Success(t *testing.T) {
|
|||||||
}
|
}
|
||||||
d.ID = delivery.ID
|
d.ID = delivery.ID
|
||||||
|
|
||||||
e.deliverHTTP(context.TODO(), db, d)
|
e.deliverHTTP(context.TODO(), db, d, task)
|
||||||
|
|
||||||
assert.True(t, received.Load(), "HTTP target should have received request")
|
assert.True(t, received.Load(), "HTTP target should have received request")
|
||||||
|
|
||||||
@@ -185,13 +198,26 @@ func TestDeliverHTTP_Failure(t *testing.T) {
|
|||||||
defer ts.Close()
|
defer ts.Close()
|
||||||
|
|
||||||
e := testEngine(t, 1)
|
e := testEngine(t, 1)
|
||||||
|
targetID := uuid.New().String()
|
||||||
|
|
||||||
event := seedEvent(t, db, `{"test":true}`)
|
event := seedEvent(t, db, `{"test":true}`)
|
||||||
delivery := seedDelivery(t, db, event.ID, uuid.New().String(), database.DeliveryStatusPending)
|
delivery := seedDelivery(t, db, event.ID, targetID, database.DeliveryStatusPending)
|
||||||
|
|
||||||
|
task := &DeliveryTask{
|
||||||
|
DeliveryID: delivery.ID,
|
||||||
|
EventID: event.ID,
|
||||||
|
WebhookID: event.WebhookID,
|
||||||
|
TargetID: targetID,
|
||||||
|
TargetName: "test-http-fail",
|
||||||
|
TargetType: database.TargetTypeHTTP,
|
||||||
|
TargetConfig: newHTTPTargetConfig(ts.URL),
|
||||||
|
MaxRetries: 0,
|
||||||
|
AttemptNum: 1,
|
||||||
|
}
|
||||||
|
|
||||||
d := &database.Delivery{
|
d := &database.Delivery{
|
||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
TargetID: delivery.TargetID,
|
TargetID: targetID,
|
||||||
Status: database.DeliveryStatusPending,
|
Status: database.DeliveryStatusPending,
|
||||||
Event: event,
|
Event: event,
|
||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
@@ -202,7 +228,7 @@ func TestDeliverHTTP_Failure(t *testing.T) {
|
|||||||
}
|
}
|
||||||
d.ID = delivery.ID
|
d.ID = delivery.ID
|
||||||
|
|
||||||
e.deliverHTTP(context.TODO(), db, d)
|
e.deliverHTTP(context.TODO(), db, d, task)
|
||||||
|
|
||||||
// HTTP (fire-and-forget) marks as failed on non-2xx
|
// HTTP (fire-and-forget) marks as failed on non-2xx
|
||||||
var updated database.Delivery
|
var updated database.Delivery
|
||||||
@@ -280,7 +306,7 @@ func TestDeliverLog_ImmediateSuccess(t *testing.T) {
|
|||||||
assert.True(t, result.Success)
|
assert.True(t, result.Success)
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestDeliverRetry_Success(t *testing.T) {
|
func TestDeliverHTTP_WithRetries_Success(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
db := testWebhookDB(t)
|
db := testWebhookDB(t)
|
||||||
|
|
||||||
@@ -300,8 +326,8 @@ func TestDeliverRetry_Success(t *testing.T) {
|
|||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
WebhookID: event.WebhookID,
|
WebhookID: event.WebhookID,
|
||||||
TargetID: targetID,
|
TargetID: targetID,
|
||||||
TargetName: "test-retry",
|
TargetName: "test-http-retry",
|
||||||
TargetType: database.TargetTypeRetry,
|
TargetType: database.TargetTypeHTTP,
|
||||||
TargetConfig: newHTTPTargetConfig(ts.URL),
|
TargetConfig: newHTTPTargetConfig(ts.URL),
|
||||||
MaxRetries: 5,
|
MaxRetries: 5,
|
||||||
AttemptNum: 1,
|
AttemptNum: 1,
|
||||||
@@ -313,8 +339,8 @@ func TestDeliverRetry_Success(t *testing.T) {
|
|||||||
Status: database.DeliveryStatusPending,
|
Status: database.DeliveryStatusPending,
|
||||||
Event: event,
|
Event: event,
|
||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: "test-retry",
|
Name: "test-http-retry",
|
||||||
Type: database.TargetTypeRetry,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: newHTTPTargetConfig(ts.URL),
|
Config: newHTTPTargetConfig(ts.URL),
|
||||||
MaxRetries: 5,
|
MaxRetries: 5,
|
||||||
},
|
},
|
||||||
@@ -322,7 +348,7 @@ func TestDeliverRetry_Success(t *testing.T) {
|
|||||||
d.ID = delivery.ID
|
d.ID = delivery.ID
|
||||||
d.Target.ID = targetID
|
d.Target.ID = targetID
|
||||||
|
|
||||||
e.deliverRetry(context.TODO(), db, d, task)
|
e.deliverHTTP(context.TODO(), db, d, task)
|
||||||
|
|
||||||
var updated database.Delivery
|
var updated database.Delivery
|
||||||
require.NoError(t, db.First(&updated, "id = ?", delivery.ID).Error)
|
require.NoError(t, db.First(&updated, "id = ?", delivery.ID).Error)
|
||||||
@@ -333,7 +359,7 @@ func TestDeliverRetry_Success(t *testing.T) {
|
|||||||
assert.Equal(t, CircuitClosed, cb.State())
|
assert.Equal(t, CircuitClosed, cb.State())
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestDeliverRetry_MaxRetriesExhausted(t *testing.T) {
|
func TestDeliverHTTP_MaxRetriesExhausted(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
db := testWebhookDB(t)
|
db := testWebhookDB(t)
|
||||||
|
|
||||||
@@ -354,8 +380,8 @@ func TestDeliverRetry_MaxRetriesExhausted(t *testing.T) {
|
|||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
WebhookID: event.WebhookID,
|
WebhookID: event.WebhookID,
|
||||||
TargetID: targetID,
|
TargetID: targetID,
|
||||||
TargetName: "test-retry-exhaust",
|
TargetName: "test-http-exhaust",
|
||||||
TargetType: database.TargetTypeRetry,
|
TargetType: database.TargetTypeHTTP,
|
||||||
TargetConfig: newHTTPTargetConfig(ts.URL),
|
TargetConfig: newHTTPTargetConfig(ts.URL),
|
||||||
MaxRetries: maxRetries,
|
MaxRetries: maxRetries,
|
||||||
AttemptNum: maxRetries, // final attempt
|
AttemptNum: maxRetries, // final attempt
|
||||||
@@ -367,8 +393,8 @@ func TestDeliverRetry_MaxRetriesExhausted(t *testing.T) {
|
|||||||
Status: database.DeliveryStatusRetrying,
|
Status: database.DeliveryStatusRetrying,
|
||||||
Event: event,
|
Event: event,
|
||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: "test-retry-exhaust",
|
Name: "test-http-exhaust",
|
||||||
Type: database.TargetTypeRetry,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: newHTTPTargetConfig(ts.URL),
|
Config: newHTTPTargetConfig(ts.URL),
|
||||||
MaxRetries: maxRetries,
|
MaxRetries: maxRetries,
|
||||||
},
|
},
|
||||||
@@ -376,7 +402,7 @@ func TestDeliverRetry_MaxRetriesExhausted(t *testing.T) {
|
|||||||
d.ID = delivery.ID
|
d.ID = delivery.ID
|
||||||
d.Target.ID = targetID
|
d.Target.ID = targetID
|
||||||
|
|
||||||
e.deliverRetry(context.TODO(), db, d, task)
|
e.deliverHTTP(context.TODO(), db, d, task)
|
||||||
|
|
||||||
// After max retries exhausted, delivery should be failed
|
// After max retries exhausted, delivery should be failed
|
||||||
var updated database.Delivery
|
var updated database.Delivery
|
||||||
@@ -385,7 +411,7 @@ func TestDeliverRetry_MaxRetriesExhausted(t *testing.T) {
|
|||||||
"delivery should be failed after max retries exhausted")
|
"delivery should be failed after max retries exhausted")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestDeliverRetry_SchedulesRetryOnFailure(t *testing.T) {
|
func TestDeliverHTTP_SchedulesRetryOnFailure(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
db := testWebhookDB(t)
|
db := testWebhookDB(t)
|
||||||
|
|
||||||
@@ -405,8 +431,8 @@ func TestDeliverRetry_SchedulesRetryOnFailure(t *testing.T) {
|
|||||||
EventID: event.ID,
|
EventID: event.ID,
|
||||||
WebhookID: event.WebhookID,
|
WebhookID: event.WebhookID,
|
||||||
TargetID: targetID,
|
TargetID: targetID,
|
||||||
TargetName: "test-retry-schedule",
|
TargetName: "test-http-schedule",
|
||||||
TargetType: database.TargetTypeRetry,
|
TargetType: database.TargetTypeHTTP,
|
||||||
TargetConfig: newHTTPTargetConfig(ts.URL),
|
TargetConfig: newHTTPTargetConfig(ts.URL),
|
||||||
MaxRetries: 5,
|
MaxRetries: 5,
|
||||||
AttemptNum: 1,
|
AttemptNum: 1,
|
||||||
@@ -418,8 +444,8 @@ func TestDeliverRetry_SchedulesRetryOnFailure(t *testing.T) {
|
|||||||
Status: database.DeliveryStatusPending,
|
Status: database.DeliveryStatusPending,
|
||||||
Event: event,
|
Event: event,
|
||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: "test-retry-schedule",
|
Name: "test-http-schedule",
|
||||||
Type: database.TargetTypeRetry,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: newHTTPTargetConfig(ts.URL),
|
Config: newHTTPTargetConfig(ts.URL),
|
||||||
MaxRetries: 5,
|
MaxRetries: 5,
|
||||||
},
|
},
|
||||||
@@ -427,7 +453,7 @@ func TestDeliverRetry_SchedulesRetryOnFailure(t *testing.T) {
|
|||||||
d.ID = delivery.ID
|
d.ID = delivery.ID
|
||||||
d.Target.ID = targetID
|
d.Target.ID = targetID
|
||||||
|
|
||||||
e.deliverRetry(context.TODO(), db, d, task)
|
e.deliverHTTP(context.TODO(), db, d, task)
|
||||||
|
|
||||||
// Delivery should be in retrying status (not failed — retries remain)
|
// Delivery should be in retrying status (not failed — retries remain)
|
||||||
var updated database.Delivery
|
var updated database.Delivery
|
||||||
@@ -591,6 +617,21 @@ func TestWorkerPool_BoundedConcurrency(t *testing.T) {
|
|||||||
tasks[i].ID = delivery.ID
|
tasks[i].ID = delivery.ID
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// Build DeliveryTask structs for each delivery (needed by deliverHTTP)
|
||||||
|
deliveryTasks := make([]DeliveryTask, numTasks)
|
||||||
|
for i := 0; i < numTasks; i++ {
|
||||||
|
deliveryTasks[i] = DeliveryTask{
|
||||||
|
DeliveryID: tasks[i].ID,
|
||||||
|
EventID: tasks[i].EventID,
|
||||||
|
TargetID: tasks[i].TargetID,
|
||||||
|
TargetName: tasks[i].Target.Name,
|
||||||
|
TargetType: tasks[i].Target.Type,
|
||||||
|
TargetConfig: tasks[i].Target.Config,
|
||||||
|
MaxRetries: 0,
|
||||||
|
AttemptNum: 1,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Process all tasks through a bounded pool of goroutines to simulate
|
// Process all tasks through a bounded pool of goroutines to simulate
|
||||||
// the engine's worker pool behavior
|
// the engine's worker pool behavior
|
||||||
var wg sync.WaitGroup
|
var wg sync.WaitGroup
|
||||||
@@ -606,7 +647,7 @@ func TestWorkerPool_BoundedConcurrency(t *testing.T) {
|
|||||||
go func() {
|
go func() {
|
||||||
defer wg.Done()
|
defer wg.Done()
|
||||||
for idx := range taskCh {
|
for idx := range taskCh {
|
||||||
e.deliverHTTP(context.TODO(), db, &tasks[idx])
|
e.deliverHTTP(context.TODO(), db, &tasks[idx], &deliveryTasks[idx])
|
||||||
}
|
}
|
||||||
}()
|
}()
|
||||||
}
|
}
|
||||||
@@ -629,7 +670,7 @@ func TestWorkerPool_BoundedConcurrency(t *testing.T) {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestDeliverRetry_CircuitBreakerBlocks(t *testing.T) {
|
func TestDeliverHTTP_CircuitBreakerBlocks(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
db := testWebhookDB(t)
|
db := testWebhookDB(t)
|
||||||
e := testEngine(t, 1)
|
e := testEngine(t, 1)
|
||||||
@@ -651,7 +692,7 @@ func TestDeliverRetry_CircuitBreakerBlocks(t *testing.T) {
|
|||||||
WebhookID: event.WebhookID,
|
WebhookID: event.WebhookID,
|
||||||
TargetID: targetID,
|
TargetID: targetID,
|
||||||
TargetName: "test-cb-block",
|
TargetName: "test-cb-block",
|
||||||
TargetType: database.TargetTypeRetry,
|
TargetType: database.TargetTypeHTTP,
|
||||||
TargetConfig: newHTTPTargetConfig("http://will-not-be-called.invalid"),
|
TargetConfig: newHTTPTargetConfig("http://will-not-be-called.invalid"),
|
||||||
MaxRetries: 5,
|
MaxRetries: 5,
|
||||||
AttemptNum: 1,
|
AttemptNum: 1,
|
||||||
@@ -664,7 +705,7 @@ func TestDeliverRetry_CircuitBreakerBlocks(t *testing.T) {
|
|||||||
Event: event,
|
Event: event,
|
||||||
Target: database.Target{
|
Target: database.Target{
|
||||||
Name: "test-cb-block",
|
Name: "test-cb-block",
|
||||||
Type: database.TargetTypeRetry,
|
Type: database.TargetTypeHTTP,
|
||||||
Config: newHTTPTargetConfig("http://will-not-be-called.invalid"),
|
Config: newHTTPTargetConfig("http://will-not-be-called.invalid"),
|
||||||
MaxRetries: 5,
|
MaxRetries: 5,
|
||||||
},
|
},
|
||||||
@@ -672,7 +713,7 @@ func TestDeliverRetry_CircuitBreakerBlocks(t *testing.T) {
|
|||||||
d.ID = delivery.ID
|
d.ID = delivery.ID
|
||||||
d.Target.ID = targetID
|
d.Target.ID = targetID
|
||||||
|
|
||||||
e.deliverRetry(context.TODO(), db, d, task)
|
e.deliverHTTP(context.TODO(), db, d, task)
|
||||||
|
|
||||||
// Delivery should be retrying (circuit open, no attempt made)
|
// Delivery should be retrying (circuit open, no attempt made)
|
||||||
var updated database.Delivery
|
var updated database.Delivery
|
||||||
|
|||||||
@@ -519,16 +519,16 @@ func (h *Handlers) HandleTargetCreate() http.HandlerFunc {
|
|||||||
|
|
||||||
// Validate target type
|
// Validate target type
|
||||||
switch targetType {
|
switch targetType {
|
||||||
case database.TargetTypeHTTP, database.TargetTypeRetry, database.TargetTypeDatabase, database.TargetTypeLog:
|
case database.TargetTypeHTTP, database.TargetTypeDatabase, database.TargetTypeLog:
|
||||||
// valid
|
// valid
|
||||||
default:
|
default:
|
||||||
http.Error(w, "Invalid target type", http.StatusBadRequest)
|
http.Error(w, "Invalid target type", http.StatusBadRequest)
|
||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// Build config JSON for HTTP-based targets
|
// Build config JSON for HTTP targets
|
||||||
var configJSON string
|
var configJSON string
|
||||||
if targetType == database.TargetTypeHTTP || targetType == database.TargetTypeRetry {
|
if targetType == database.TargetTypeHTTP {
|
||||||
if url == "" {
|
if url == "" {
|
||||||
http.Error(w, "URL is required for HTTP targets", http.StatusBadRequest)
|
http.Error(w, "URL is required for HTTP targets", http.StatusBadRequest)
|
||||||
return
|
return
|
||||||
@@ -544,9 +544,9 @@ func (h *Handlers) HandleTargetCreate() http.HandlerFunc {
|
|||||||
configJSON = string(configBytes)
|
configJSON = string(configBytes)
|
||||||
}
|
}
|
||||||
|
|
||||||
maxRetries := 5
|
maxRetries := 0 // default: fire-and-forget (no retries)
|
||||||
if maxRetriesStr != "" {
|
if maxRetriesStr != "" {
|
||||||
if v, err := strconv.Atoi(maxRetriesStr); err == nil && v > 0 {
|
if v, err := strconv.Atoi(maxRetriesStr); err == nil && v >= 0 {
|
||||||
maxRetries = v
|
maxRetries = v
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -92,17 +92,16 @@
|
|||||||
<input type="text" name="name" placeholder="Target name" required class="input text-sm flex-1">
|
<input type="text" name="name" placeholder="Target name" required class="input text-sm flex-1">
|
||||||
<select name="type" x-model="targetType" class="input text-sm w-32">
|
<select name="type" x-model="targetType" class="input text-sm w-32">
|
||||||
<option value="http">HTTP</option>
|
<option value="http">HTTP</option>
|
||||||
<option value="retry">Retry</option>
|
|
||||||
<option value="database">Database</option>
|
<option value="database">Database</option>
|
||||||
<option value="log">Log</option>
|
<option value="log">Log</option>
|
||||||
</select>
|
</select>
|
||||||
</div>
|
</div>
|
||||||
<div x-show="targetType === 'http' || targetType === 'retry'">
|
<div x-show="targetType === 'http'">
|
||||||
<input type="url" name="url" placeholder="https://example.com/webhook" class="input text-sm">
|
<input type="url" name="url" placeholder="https://example.com/webhook" class="input text-sm">
|
||||||
</div>
|
</div>
|
||||||
<div x-show="targetType === 'retry'" class="flex gap-2 items-center">
|
<div x-show="targetType === 'http'" class="flex gap-2 items-center">
|
||||||
<label class="text-sm text-gray-700">Max retries:</label>
|
<label class="text-sm text-gray-700">Max retries (0 = fire-and-forget):</label>
|
||||||
<input type="number" name="max_retries" value="5" min="1" max="20" class="input text-sm w-24">
|
<input type="number" name="max_retries" value="0" min="0" max="20" class="input text-sm w-24">
|
||||||
</div>
|
</div>
|
||||||
<button type="submit" class="btn-primary text-sm">Add Target</button>
|
<button type="submit" class="btn-primary text-sm">Add Target</button>
|
||||||
</form>
|
</form>
|
||||||
|
|||||||
Reference in New Issue
Block a user