sneak
dfd559417e
All checks were successful
check / check (push) Successful in 3m10s
/metrics carried only the inbound HTTP surface, so a destination failing for an hour, a growing retry backlog and a stuck-open circuit breaker were all invisible: the receive side stays healthy in each case because it is. New internal/metrics registers, on the existing default registry that the go-http-metrics recorder and the promhttp handler already share: - webhooker_events_received_total - webhooker_delivery_attempts_total - webhooker_deliveries_succeeded_total - webhooker_deliveries_failed_total - webhooker_delivery_retries_total - webhooker_delivery_duration_seconds - webhooker_deliveries_pending / _retrying - webhooker_circuit_breakers_open The route mounting is untouched. Every delivery metric carries one label, target_type, whose domain is the four target-type constants; anything outside it collapses to "unknown" so no series can be minted from a UUID. Target ids, event ids and entrypoint ids are deliberately not labels. An attempt is counted, and its duration observed, only where one was actually dispatched — the target's own result path, which is also where the DeliveryResult is written. A delivery an open circuit breaker refuses sends nothing and records no result row; counting it would climb the attempts counter with no traffic behind it and pull the duration quantiles down for as long as the breaker stayed open, moving the metric the wrong way during the outage it exists to reveal. The log and database targets now time their own work, so their result rows carry a real duration too. The outcome counters move after the status row is written rather than before, so a transition the database rejected is never reported as an outcome that happened. The queue-depth gauges are counted out of the per-webhook databases by a 30s sampler rather than tracked as deltas, which would need seeding at startup and would drift on any transition that failed to persist. They publish an "unknown" series from registration: deliveries queued against a target that has since been deleted resolve to the empty type and are folded there, because a backlog behind a deleted target is precisely the one nobody is watching. The open-breaker gauge is recounted from the target's breaker registry on every state change. The orphaned-retry terminal path takes the target type as an argument rather than attaching the loaded target to the delivery. That path loads the delivery without its target relation on purpose: a populated Delivery.Target makes GORM's SaveBeforeAssociations upsert the whole target row on the status UPDATE, writing the plaintext target config — the credential, for a slack target — into the per-webhook events database. A test asserts that path leaves the targets table empty.
66 lines
2.5 KiB
Modula-2
66 lines
2.5 KiB
Modula-2
module sneak.berlin/go/webhooker
|
|
|
|
go 1.26.1
|
|
|
|
require (
|
|
github.com/99designs/basicauth-go v0.0.0-20230316000542-bf6f9cbbf0f8
|
|
github.com/getsentry/sentry-go v0.25.0
|
|
github.com/go-chi/chi v1.5.5
|
|
github.com/go-chi/cors v1.2.1
|
|
github.com/go-chi/httprate v0.15.0
|
|
github.com/google/uuid v1.6.0
|
|
github.com/gorilla/csrf v1.7.3
|
|
github.com/gorilla/sessions v1.4.0
|
|
github.com/joho/godotenv v1.5.1
|
|
github.com/prometheus/client_golang v1.18.0
|
|
github.com/prometheus/client_model v0.5.0
|
|
github.com/slok/go-http-metrics v0.11.0
|
|
github.com/stretchr/testify v1.8.4
|
|
go.uber.org/fx v1.20.1
|
|
golang.org/x/crypto v0.38.0
|
|
gopkg.in/yaml.v3 v3.0.1
|
|
gorm.io/driver/sqlite v1.5.4
|
|
gorm.io/gorm v1.25.5
|
|
modernc.org/sqlite v1.28.0
|
|
)
|
|
|
|
require (
|
|
github.com/beorn7/perks v1.0.1 // indirect
|
|
github.com/cespare/xxhash/v2 v2.2.0 // indirect
|
|
github.com/davecgh/go-spew v1.1.2-0.20180830191138-d8f796af33cc // indirect
|
|
github.com/dustin/go-humanize v1.0.1 // indirect
|
|
github.com/gorilla/securecookie v1.1.2 // indirect
|
|
github.com/jinzhu/inflection v1.0.0 // indirect
|
|
github.com/jinzhu/now v1.1.5 // indirect
|
|
github.com/kballard/go-shellquote v0.0.0-20180428030007-95032a82bc51 // indirect
|
|
github.com/klauspost/cpuid/v2 v2.2.10 // indirect
|
|
github.com/kr/text v0.2.0 // indirect
|
|
github.com/mattn/go-isatty v0.0.20 // indirect
|
|
github.com/mattn/go-sqlite3 v1.14.17 // indirect
|
|
github.com/matttproud/golang_protobuf_extensions/v2 v2.0.0 // indirect
|
|
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
|
|
github.com/prometheus/common v0.45.0 // indirect
|
|
github.com/prometheus/procfs v0.12.0 // indirect
|
|
github.com/remyoudompheng/bigfft v0.0.0-20230129092748-24d4a6f8daec // indirect
|
|
github.com/zeebo/xxh3 v1.0.2 // indirect
|
|
go.uber.org/atomic v1.9.0 // indirect
|
|
go.uber.org/dig v1.17.0 // indirect
|
|
go.uber.org/multierr v1.9.0 // indirect
|
|
go.uber.org/zap v1.23.0 // indirect
|
|
golang.org/x/mod v0.17.0 // indirect
|
|
golang.org/x/sync v0.14.0 // indirect
|
|
golang.org/x/sys v0.33.0 // indirect
|
|
golang.org/x/text v0.25.0 // indirect
|
|
golang.org/x/tools v0.21.1-0.20240508182429-e35e4ccd0d2d // indirect
|
|
google.golang.org/protobuf v1.31.0 // indirect
|
|
lukechampine.com/uint128 v1.2.0 // indirect
|
|
modernc.org/cc/v3 v3.40.0 // indirect
|
|
modernc.org/ccgo/v3 v3.16.13 // indirect
|
|
modernc.org/libc v1.29.0 // indirect
|
|
modernc.org/mathutil v1.6.0 // indirect
|
|
modernc.org/memory v1.7.2 // indirect
|
|
modernc.org/opt v0.1.3 // indirect
|
|
modernc.org/strutil v1.1.3 // indirect
|
|
modernc.org/token v1.0.1 // indirect
|
|
)
|