The JSON log wrote a Go duration as a bare count of nanoseconds, so
the watcher starting line showed dnsInterval 120000000000 for 2m and a
delivery retry showed retryIn 1015437050. Each duration logged is now
passed through its String() form: dnsInterval and tlsInterval when the
watcher starts, retryIn on a delivery retry, and latency on a
succeeded port check. A test checks that retryIn is logged as the
text of the wait the retry actually took.
The request log's latency_ms is left as it is: its key names its
unit.
Model: opus-5-5
## Summary
Notifications were fire-and-forget: if Slack, Mattermost, or ntfy was temporarily down, changes were silently lost. This adds automatic retry with exponential backoff and jitter to all notification endpoints.
## Changes
### New file: `internal/notify/retry.go`
- `RetryConfig` struct with configurable max retries, base delay, max delay
- `backoff()` computes delay as `BaseDelay * 2^attempt`, capped at `MaxDelay`, with ±25% jitter
- `deliverWithRetry()` wraps any send function with the retry loop
- Defaults: 3 retries (4 total attempts), 1s base delay, 10s max delay
- Context-aware: respects cancellation during retry sleep
- Injectable `sleepFn` for test determinism
### Modified: `internal/notify/notify.go`
- Added `retryConfig` and `sleepFn` fields to `Service`
- Updated `dispatchNtfy`, `dispatchSlack`, `dispatchMattermost` to wrap sends in `deliverWithRetry`
- Structured logging: warns on each retry, logs error only after all retries exhausted, logs info on success after retry
### Modified: `internal/notify/export_test.go`
- Added test helpers: `SetRetryConfig`, `SetSleepFunc`, `DeliverWithRetry`, `BackoffDuration`
### New file: `internal/notify/retry_test.go`
- Backoff calculation tests (exponential increase, max cap with jitter)
- `deliverWithRetry` unit tests: first-attempt success, transient failure recovery, exhausted retries, context cancellation
- Integration tests via `SendNotification`: transient failure retries, all-endpoints retry independently, permanent failure exhausts retries
## Verification
- `make fmt` ✅
- `make check` (format + lint + tests + build) ✅
- `docker build .` ✅
- All existing tests continue to pass unchanged
- No DNS client mocking — notification tests use `httptest` servers
closes #62
Co-authored-by: clawbot <clawbot@noreply.git.eeqj.de>
Reviewed-on: #87
Co-authored-by: clawbot <clawbot@noreply.example.org>
Co-committed-by: clawbot <clawbot@noreply.example.org>
clawbot
