Compare commits
2
Commits
1b617847eb
...
dbd3343251
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
dbd3343251 | ||
|
|
c9510a986c |
@@ -121,12 +121,22 @@ notification endpoint set, changes show only on the dashboard; see
|
|||||||
failed on it, and answers differently is reported on the check where it
|
failed on it, and answers differently is reported on the check where it
|
||||||
answers. If a pair agrees again and later disagrees, the alert is sent
|
answers. If a pair agrees again and later disagrees, the alert is sent
|
||||||
again.
|
again.
|
||||||
|
- **CNAME address change**: For a name whose nameservers answer with a CNAME
|
||||||
|
and no address, the addresses at the end of its CNAME chain differ from
|
||||||
|
those of the previous check, including when there are none now. Nothing is
|
||||||
|
sent when the previous check saved none, or when the previous addresses
|
||||||
|
were kept because the chain could not be followed or none of the name's
|
||||||
|
nameservers answered.
|
||||||
|
|
||||||
### TCP Port Monitoring
|
### TCP Port Monitoring
|
||||||
|
|
||||||
- For every configured domain and hostname, constructs a deduplicated list of
|
- For every configured domain and hostname, constructs a deduplicated list of
|
||||||
all IPv4 and IPv6 addresses resolved via A, AAAA, and CNAME chain resolution
|
the IPv4 and IPv6 addresses in the A and AAAA records its authoritative
|
||||||
across all authoritative nameservers.
|
nameservers returned. When they returned a CNAME and no address, the CNAME
|
||||||
|
chain is followed and the addresses at its end are used, and a change in those
|
||||||
|
is notified as a CNAME address change. When the chain cannot be followed, or
|
||||||
|
none of the name's nameservers answered, the addresses the last check found at
|
||||||
|
its end are used.
|
||||||
- Checks TCP connectivity on ports **80** and **443** for each IP address.
|
- Checks TCP connectivity on ports **80** and **443** for each IP address.
|
||||||
- Every **1 hour**, re-checks all ports.
|
- Every **1 hour**, re-checks all ports.
|
||||||
- Any change in port availability triggers a notification:
|
- Any change in port availability triggers a notification:
|
||||||
@@ -173,6 +183,8 @@ includes:
|
|||||||
- **DNS NS changes**: Which domain, which nameservers were added/removed.
|
- **DNS NS changes**: Which domain, which nameservers were added/removed.
|
||||||
- **NS address changes**: Which domain, which nameserver, its old and new
|
- **NS address changes**: Which domain, which nameserver, its old and new
|
||||||
addresses.
|
addresses.
|
||||||
|
- **CNAME address changes**: Which hostname, the old and new addresses at the
|
||||||
|
end of its CNAME chain.
|
||||||
- **NS query failures**: Which nameserver failed, error type (timeout, SERVFAIL,
|
- **NS query failures**: Which nameserver failed, error type (timeout, SERVFAIL,
|
||||||
REFUSED, network error), which hostname/domain affected.
|
REFUSED, network error), which hostname/domain affected.
|
||||||
- **NS recoveries**: Which nameserver recovered, which hostname/domain.
|
- **NS recoveries**: Which nameserver recovered, which hostname/domain.
|
||||||
@@ -389,8 +401,11 @@ This approach ensures:
|
|||||||
servers.
|
servers.
|
||||||
- Visibility into the full delegation chain.
|
- Visibility into the full delegation chain.
|
||||||
|
|
||||||
For hostname monitoring, the resolver follows CNAME chains (with a depth limit
|
A watched name's records are stored as its nameservers return them, CNAME
|
||||||
to prevent loops) before collecting terminal A/AAAA records.
|
included. When they return a CNAME and no address, the CNAME chain is followed
|
||||||
|
(with a depth limit to prevent loops) to the A and AAAA records at its end, and
|
||||||
|
the port and TLS checks use those addresses. Nameservers' addresses are found
|
||||||
|
the same way.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
@@ -478,6 +493,12 @@ nameservers, has status `error`, empty `records`, and the reason in `error`.
|
|||||||
resolves to. A state file without it loads, and the next check fills it in
|
resolves to. A state file without it loads, and the next check fills it in
|
||||||
without a notification.
|
without a notification.
|
||||||
|
|
||||||
|
`cnameAddresses` lists the sorted addresses at the end of a hostname's CNAME
|
||||||
|
chain, found when its nameservers answered with a CNAME and no address, and kept
|
||||||
|
from the previous check when the chain cannot be followed or none of the name's
|
||||||
|
nameservers answered. It is left out otherwise. A state file without it loads,
|
||||||
|
and the next check fills it in without a notification.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Entrypoints
|
## Entrypoints
|
||||||
@@ -617,7 +638,9 @@ docker run -d \
|
|||||||
completes.
|
completes.
|
||||||
- Port and TLS checks always use freshly resolved IP addresses from the DNS
|
- Port and TLS checks always use freshly resolved IP addresses from the DNS
|
||||||
phase that immediately precedes them — never stale IPs from a previous
|
phase that immediately precedes them — never stale IPs from a previous
|
||||||
cycle.
|
cycle, with one exception: when a name's CNAME chain cannot be followed,
|
||||||
|
or none of the name's nameservers answered, the addresses the previous
|
||||||
|
cycle found at the end of the chain are used.
|
||||||
4. **On change detection**: Send notifications to all configured endpoints,
|
4. **On change detection**: Send notifications to all configured endpoints,
|
||||||
update in-memory state, persist to disk.
|
update in-memory state, persist to disk.
|
||||||
5. **Shutdown**: The watcher stops checking and saves the final state to disk,
|
5. **Shutdown**: The watcher stops checking and saves the final state to disk,
|
||||||
|
|||||||
@@ -19,6 +19,10 @@ trial run of the finished image: https://git.eeqj.de/sneak/dnswatcher/issues/149
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-10-01: a watched name whose nameservers answer with a CNAME and no
|
||||||
|
address gets port and TLS checks at the end of its CNAME chain (closes #203).
|
||||||
|
- 2026-10-01: a certificate within the expiry warning period is warned about on
|
||||||
|
every TLS check, where some checks used to skip it at random (closes #204).
|
||||||
- 2026-10-01: a domain's NS set is its delegation from the parent zone's
|
- 2026-10-01: a domain's NS set is its delegation from the parent zone's
|
||||||
servers, not whichever of its own servers answered first (closes #200).
|
servers, not whichever of its own servers answered first (closes #200).
|
||||||
- 2026-10-01: README has Getting Started, Rationale and TODO sections, and its
|
- 2026-10-01: README has Getting Started, Rationale and TODO sections, and its
|
||||||
|
|||||||
@@ -53,8 +53,12 @@ type NameserverRecordState struct {
|
|||||||
}
|
}
|
||||||
|
|
||||||
// HostnameState holds per-nameserver monitoring state for a hostname.
|
// HostnameState holds per-nameserver monitoring state for a hostname.
|
||||||
|
// CNAMEAddresses holds the sorted addresses at the end of the name's
|
||||||
|
// CNAME chain, found when its nameservers answered with a CNAME and no
|
||||||
|
// address; it is empty otherwise.
|
||||||
type HostnameState struct {
|
type HostnameState struct {
|
||||||
RecordsByNameserver map[string]*NameserverRecordState `json:"recordsByNameserver"`
|
RecordsByNameserver map[string]*NameserverRecordState `json:"recordsByNameserver"`
|
||||||
|
CNAMEAddresses []string `json:"cnameAddresses,omitempty"`
|
||||||
LastChecked time.Time `json:"lastChecked"`
|
LastChecked time.Time `json:"lastChecked"`
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,199 @@
|
|||||||
|
package watcher_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"context"
|
||||||
|
"log/slog"
|
||||||
|
"slices"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"sneak.berlin/go/dnswatcher/internal/livednstest"
|
||||||
|
"sneak.berlin/go/dnswatcher/internal/resolver"
|
||||||
|
"sneak.berlin/go/dnswatcher/internal/state"
|
||||||
|
"sneak.berlin/go/dnswatcher/internal/watcher"
|
||||||
|
)
|
||||||
|
|
||||||
|
// cnameHost is a CNAME into another zone: its nameservers answer with
|
||||||
|
// the CNAME and no address.
|
||||||
|
const cnameHost = "www.python.org"
|
||||||
|
|
||||||
|
// TestCNAMEIntoAnotherZonePortAndTLSChecks checks cnameHost against
|
||||||
|
// live DNS. Its port and TLS checks must use the addresses at the end
|
||||||
|
// of its CNAME chain.
|
||||||
|
func TestCNAMEIntoAnotherZonePortAndTLSChecks(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
cfg := defaultTestConfig(t)
|
||||||
|
cfg.Hostnames = []string{cnameHost}
|
||||||
|
|
||||||
|
deps := runChecks(t, cfg, nil, nil)
|
||||||
|
|
||||||
|
snap := deps.state.GetSnapshot()
|
||||||
|
hs := snap.Hostnames[cnameHost]
|
||||||
|
|
||||||
|
if len(hs.CNAMEAddresses) == 0 {
|
||||||
|
t.Fatalf(
|
||||||
|
"%s: no addresses saved from following its CNAME; if it "+
|
||||||
|
"is no longer a CNAME into another zone, this test "+
|
||||||
|
"needs another name",
|
||||||
|
cnameHost,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, ip := range hs.CNAMEAddresses {
|
||||||
|
ps, ok := snap.Ports[ip+":443"]
|
||||||
|
if !ok || !slices.Contains(ps.Hostnames, cnameHost) {
|
||||||
|
t.Errorf("no port state for %s at %s:443", cnameHost, ip)
|
||||||
|
}
|
||||||
|
|
||||||
|
certKey := ip + ":443:" + cnameHost
|
||||||
|
if _, ok := snap.Certificates[certKey]; !ok {
|
||||||
|
t.Errorf("no certificate state %s", certKey)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestCNAMEThatCannotBeFollowedKeepsPrevious gives a name under
|
||||||
|
// .invalid, whose lookup fails, answers with a CNAME and no address.
|
||||||
|
// The addresses the previous check saved from following its CNAME are
|
||||||
|
// kept.
|
||||||
|
func TestCNAMEThatCannotBeFollowedKeepsPrevious(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
const name = "www.example.invalid"
|
||||||
|
|
||||||
|
w := watcher.NewForTest(
|
||||||
|
nil, nil, resolver.NewFromLogger(slog.Default()), nil, nil, nil,
|
||||||
|
)
|
||||||
|
|
||||||
|
current := hostnameState(map[string]map[string][]string{
|
||||||
|
nsA: {"CNAME": {"target.example.invalid."}},
|
||||||
|
})
|
||||||
|
prev := &state.HostnameState{CNAMEAddresses: []string{oldIP}}
|
||||||
|
|
||||||
|
// The result is the same whether or not live DNS answers, so the
|
||||||
|
// lookup is not retried.
|
||||||
|
_ = livednstest.Run(func(ctx context.Context) error {
|
||||||
|
w.ResolveCNAMEAddresses(ctx, name, current, prev)
|
||||||
|
|
||||||
|
return nil
|
||||||
|
})
|
||||||
|
|
||||||
|
if !slices.Equal(current.CNAMEAddresses, prev.CNAMEAddresses) {
|
||||||
|
t.Errorf(
|
||||||
|
"saved %v, want %v",
|
||||||
|
current.CNAMEAddresses, prev.CNAMEAddresses,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestCNAMEWhoseNameserversAllFailedKeepsPrevious checks a name none of
|
||||||
|
// whose nameservers answered. The addresses the previous check saved
|
||||||
|
// from following its CNAME are kept, and nothing is looked up: the
|
||||||
|
// watcher has no resolver.
|
||||||
|
func TestCNAMEWhoseNameserversAllFailedKeepsPrevious(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
w := watcher.NewForTest(nil, nil, nil, nil, nil, nil)
|
||||||
|
|
||||||
|
current := saved(map[string]*state.NameserverRecordState{
|
||||||
|
nsA: failed(), nsB: failed(),
|
||||||
|
})
|
||||||
|
prev := cnameState(oldIP)
|
||||||
|
|
||||||
|
w.ResolveCNAMEAddresses(t.Context(), host, current, prev)
|
||||||
|
|
||||||
|
if !slices.Equal(current.CNAMEAddresses, prev.CNAMEAddresses) {
|
||||||
|
t.Errorf(
|
||||||
|
"saved %v, want %v",
|
||||||
|
current.CNAMEAddresses, prev.CNAMEAddresses,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// cnameState builds the state a check leaves behind for a name whose
|
||||||
|
// nameserver answered with a CNAME and no address, when following the
|
||||||
|
// CNAME found these addresses.
|
||||||
|
func cnameState(addresses ...string) *state.HostnameState {
|
||||||
|
hs := hostnameState(map[string]map[string][]string{
|
||||||
|
nsA: {"CNAME": {"target.example.org."}},
|
||||||
|
})
|
||||||
|
hs.CNAMEAddresses = addresses
|
||||||
|
|
||||||
|
return hs
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCNAMEAddressChangeAlerts(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
// Each case is the state saved by the previous check and by the
|
||||||
|
// current one. The name's records are the same in both.
|
||||||
|
tests := []struct {
|
||||||
|
name string
|
||||||
|
prev, current *state.HostnameState
|
||||||
|
want int
|
||||||
|
}{
|
||||||
|
{
|
||||||
|
"same addresses",
|
||||||
|
cnameState(ip1, ip2), cnameState(ip1, ip2), 0,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"same addresses in another order",
|
||||||
|
cnameState(ip2, ip1), cnameState(ip1, ip2), 0,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"address replaced",
|
||||||
|
cnameState(ip1), cnameState(ip2), 1,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"address added",
|
||||||
|
cnameState(ip1), cnameState(ip1, ip2), 1,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"no address at the end of the chain now",
|
||||||
|
cnameState(ip1), cnameState(), 1,
|
||||||
|
},
|
||||||
|
{
|
||||||
|
"state file from before addresses were saved",
|
||||||
|
cnameState(), cnameState(ip1), 0,
|
||||||
|
},
|
||||||
|
}
|
||||||
|
|
||||||
|
for _, tt := range tests {
|
||||||
|
t.Run(tt.name, func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
notifier := &mockNotifier{}
|
||||||
|
w := watcher.NewForTest(nil, nil, nil, nil, nil, notifier)
|
||||||
|
|
||||||
|
w.DetectHostnameChanges(t.Context(), host, tt.prev, tt.current)
|
||||||
|
|
||||||
|
got := len(notifier.getNotifications())
|
||||||
|
if got != tt.want {
|
||||||
|
t.Errorf("sent %d notifications, want %d", got, tt.want)
|
||||||
|
}
|
||||||
|
})
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
func TestCNAMEAddressChangeAlertNamesHostnameAndAddresses(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
notifier := &mockNotifier{}
|
||||||
|
w := watcher.NewForTest(nil, nil, nil, nil, nil, notifier)
|
||||||
|
|
||||||
|
w.DetectHostnameChanges(
|
||||||
|
t.Context(), host, cnameState(ip1), cnameState(ip2, ip3),
|
||||||
|
)
|
||||||
|
|
||||||
|
want := notification{
|
||||||
|
Title: "CNAME Address Change: " + host,
|
||||||
|
Message: "Hostname: " + host +
|
||||||
|
"\nOld: " + ip1 + "\nNew: " + ip2 + ", " + ip3,
|
||||||
|
Priority: "warning",
|
||||||
|
}
|
||||||
|
|
||||||
|
got := notifier.getNotifications()
|
||||||
|
if len(got) != 1 || got[0] != want {
|
||||||
|
t.Errorf("sent %v, want %v", got, want)
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -20,15 +20,14 @@ func NewForTest(
|
|||||||
n Notifier,
|
n Notifier,
|
||||||
) *Watcher {
|
) *Watcher {
|
||||||
return &Watcher{
|
return &Watcher{
|
||||||
log: slog.Default(),
|
log: slog.Default(),
|
||||||
config: cfg,
|
config: cfg,
|
||||||
state: st,
|
state: st,
|
||||||
resolver: res,
|
resolver: res,
|
||||||
portCheck: pc,
|
portCheck: pc,
|
||||||
tlsCheck: tc,
|
tlsCheck: tc,
|
||||||
notify: n,
|
notify: n,
|
||||||
firstRun: true,
|
firstRun: true,
|
||||||
expiryNotified: make(map[string]time.Time),
|
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
@@ -58,6 +57,15 @@ func (w *Watcher) ResolveNameserverAddresses(
|
|||||||
return w.resolveNameserverAddresses(ctx, nameservers, prev)
|
return w.resolveNameserverAddresses(ctx, nameservers, prev)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ResolveCNAMEAddresses exports resolveCNAMEAddresses for testing.
|
||||||
|
func (w *Watcher) ResolveCNAMEAddresses(
|
||||||
|
ctx context.Context,
|
||||||
|
hostname string,
|
||||||
|
current, prev *state.HostnameState,
|
||||||
|
) {
|
||||||
|
w.resolveCNAMEAddresses(ctx, hostname, current, prev)
|
||||||
|
}
|
||||||
|
|
||||||
// DetectNSAddressChanges exports detectNSAddressChanges for testing.
|
// DetectNSAddressChanges exports detectNSAddressChanges for testing.
|
||||||
func (w *Watcher) DetectNSAddressChanges(
|
func (w *Watcher) DetectNSAddressChanges(
|
||||||
ctx context.Context,
|
ctx context.Context,
|
||||||
@@ -72,6 +80,11 @@ func (w *Watcher) CheckAllPorts(ctx context.Context) {
|
|||||||
w.checkAllPorts(ctx)
|
w.checkAllPorts(ctx)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// RunTLSChecks exports runTLSChecks for testing.
|
||||||
|
func (w *Watcher) RunTLSChecks(ctx context.Context) {
|
||||||
|
w.runTLSChecks(ctx)
|
||||||
|
}
|
||||||
|
|
||||||
// BuildHostnameState exports buildHostnameState for testing.
|
// BuildHostnameState exports buildHostnameState for testing.
|
||||||
func BuildHostnameState(
|
func BuildHostnameState(
|
||||||
results map[string]*resolver.NameserverResponse,
|
results map[string]*resolver.NameserverResponse,
|
||||||
|
|||||||
+124
-60
@@ -7,7 +7,6 @@ import (
|
|||||||
"slices"
|
"slices"
|
||||||
"sort"
|
"sort"
|
||||||
"strings"
|
"strings"
|
||||||
"sync"
|
|
||||||
"time"
|
"time"
|
||||||
|
|
||||||
"go.uber.org/fx"
|
"go.uber.org/fx"
|
||||||
@@ -49,18 +48,16 @@ type Params struct {
|
|||||||
|
|
||||||
// Watcher orchestrates all monitoring checks on a schedule.
|
// Watcher orchestrates all monitoring checks on a schedule.
|
||||||
type Watcher struct {
|
type Watcher struct {
|
||||||
log *slog.Logger
|
log *slog.Logger
|
||||||
config *config.Config
|
config *config.Config
|
||||||
state *state.State
|
state *state.State
|
||||||
resolver DNSResolver
|
resolver DNSResolver
|
||||||
portCheck PortChecker
|
portCheck PortChecker
|
||||||
tlsCheck TLSChecker
|
tlsCheck TLSChecker
|
||||||
notify Notifier
|
notify Notifier
|
||||||
cancel context.CancelFunc
|
cancel context.CancelFunc
|
||||||
done chan struct{} // closed when Run returns
|
done chan struct{} // closed when Run returns
|
||||||
firstRun bool
|
firstRun bool
|
||||||
expiryNotifiedMu sync.Mutex
|
|
||||||
expiryNotified map[string]time.Time
|
|
||||||
}
|
}
|
||||||
|
|
||||||
// New creates a new Watcher instance wired into the fx lifecycle.
|
// New creates a new Watcher instance wired into the fx lifecycle.
|
||||||
@@ -69,15 +66,14 @@ func New(
|
|||||||
params Params,
|
params Params,
|
||||||
) (*Watcher, error) {
|
) (*Watcher, error) {
|
||||||
w := &Watcher{
|
w := &Watcher{
|
||||||
log: params.Logger.Get(),
|
log: params.Logger.Get(),
|
||||||
config: params.Config,
|
config: params.Config,
|
||||||
state: params.State,
|
state: params.State,
|
||||||
resolver: params.Resolver,
|
resolver: params.Resolver,
|
||||||
portCheck: params.PortCheck,
|
portCheck: params.PortCheck,
|
||||||
tlsCheck: params.TLSCheck,
|
tlsCheck: params.TLSCheck,
|
||||||
notify: params.Notify,
|
notify: params.Notify,
|
||||||
firstRun: true,
|
firstRun: true,
|
||||||
expiryNotified: make(map[string]time.Time),
|
|
||||||
}
|
}
|
||||||
|
|
||||||
lifecycle.Append(fx.Hook{
|
lifecycle.Append(fx.Hook{
|
||||||
@@ -256,28 +252,9 @@ func (w *Watcher) checkDomain(
|
|||||||
LastChecked: now,
|
LastChecked: now,
|
||||||
})
|
})
|
||||||
|
|
||||||
// Also look up A/AAAA records for the apex domain so that
|
// The apex domain's records are also checked as a hostname's, so
|
||||||
// port and TLS checks (which read HostnameState) can find
|
// that the port and TLS checks find its addresses.
|
||||||
// the domain's IP addresses.
|
w.checkHostname(ctx, domain)
|
||||||
results, err := w.resolver.LookupAllRecords(ctx, domain)
|
|
||||||
if err != nil {
|
|
||||||
w.log.Error(
|
|
||||||
"failed to lookup records for domain",
|
|
||||||
"domain", domain,
|
|
||||||
"error", err,
|
|
||||||
)
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
newState := buildHostnameState(results, now)
|
|
||||||
|
|
||||||
prevHS, hasPrevHS := w.state.GetHostnameState(domain)
|
|
||||||
if hasPrevHS && !w.firstRun {
|
|
||||||
w.detectHostnameChanges(ctx, domain, prevHS, newState)
|
|
||||||
}
|
|
||||||
|
|
||||||
w.state.SetHostnameState(domain, newState)
|
|
||||||
}
|
}
|
||||||
|
|
||||||
func (w *Watcher) detectNSChanges(
|
func (w *Watcher) detectNSChanges(
|
||||||
@@ -405,6 +382,9 @@ func (w *Watcher) checkHostname(
|
|||||||
newState := buildHostnameState(results, time.Now().UTC())
|
newState := buildHostnameState(results, time.Now().UTC())
|
||||||
|
|
||||||
prev, hasPrev := w.state.GetHostnameState(hostname)
|
prev, hasPrev := w.state.GetHostnameState(hostname)
|
||||||
|
|
||||||
|
w.resolveCNAMEAddresses(ctx, hostname, newState, prev)
|
||||||
|
|
||||||
if hasPrev && !w.firstRun {
|
if hasPrev && !w.firstRun {
|
||||||
w.detectHostnameChanges(ctx, hostname, prev, newState)
|
w.detectHostnameChanges(ctx, hostname, prev, newState)
|
||||||
}
|
}
|
||||||
@@ -412,6 +392,68 @@ func (w *Watcher) checkHostname(
|
|||||||
w.state.SetHostnameState(hostname, newState)
|
w.state.SetHostnameState(hostname, newState)
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// resolveCNAMEAddresses saves in current the addresses at the end of
|
||||||
|
// hostname's CNAME chain, when the nameservers' answers in current hold
|
||||||
|
// a CNAME and no address. ResolveIPAddresses looks the name up again
|
||||||
|
// and follows the chain. The addresses saved in prev, which may be nil,
|
||||||
|
// are kept when none of the name's nameservers answered, and when the
|
||||||
|
// chain cannot be followed, as when no nameserver of a zone in it
|
||||||
|
// answers.
|
||||||
|
func (w *Watcher) resolveCNAMEAddresses(
|
||||||
|
ctx context.Context,
|
||||||
|
hostname string,
|
||||||
|
current, prev *state.HostnameState,
|
||||||
|
) {
|
||||||
|
var prevAddresses []string
|
||||||
|
if prev != nil {
|
||||||
|
prevAddresses = prev.CNAMEAddresses
|
||||||
|
}
|
||||||
|
|
||||||
|
answered := false
|
||||||
|
hasCNAME := false
|
||||||
|
|
||||||
|
for _, nsState := range current.RecordsByNameserver {
|
||||||
|
if nsState.Status != statusOK {
|
||||||
|
continue
|
||||||
|
}
|
||||||
|
|
||||||
|
answered = true
|
||||||
|
|
||||||
|
if len(nsState.Records["A"]) > 0 || len(nsState.Records["AAAA"]) > 0 {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if len(nsState.Records["CNAME"]) > 0 {
|
||||||
|
hasCNAME = true
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
if !answered {
|
||||||
|
current.CNAMEAddresses = prevAddresses
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
if !hasCNAME {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
ips, err := w.resolver.ResolveIPAddresses(ctx, hostname)
|
||||||
|
if err != nil {
|
||||||
|
w.log.Error(
|
||||||
|
"failed to follow CNAME",
|
||||||
|
"hostname", hostname,
|
||||||
|
"error", err,
|
||||||
|
)
|
||||||
|
|
||||||
|
current.CNAMEAddresses = prevAddresses
|
||||||
|
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
current.CNAMEAddresses = ips
|
||||||
|
}
|
||||||
|
|
||||||
// buildHostnameState saves each nameserver's response. A nameserver
|
// buildHostnameState saves each nameserver's response. A nameserver
|
||||||
// that answered, even with NXDOMAIN or no records, is saved as ok; one
|
// that answered, even with NXDOMAIN or no records, is saved as ok; one
|
||||||
// that timed out or failed is saved as error with the reason, and its
|
// that timed out or failed is saved as error with the reason, and its
|
||||||
@@ -455,6 +497,37 @@ func (w *Watcher) detectHostnameChanges(
|
|||||||
w.detectNSDisappearances(ctx, hostname, prev, current)
|
w.detectNSDisappearances(ctx, hostname, prev, current)
|
||||||
w.detectNSFailures(ctx, hostname, prev, current)
|
w.detectNSFailures(ctx, hostname, prev, current)
|
||||||
w.detectInconsistencies(ctx, hostname, prev, current)
|
w.detectInconsistencies(ctx, hostname, prev, current)
|
||||||
|
w.detectCNAMEAddressChanges(ctx, hostname, prev, current)
|
||||||
|
}
|
||||||
|
|
||||||
|
// detectCNAMEAddressChanges notifies when the addresses at the end of
|
||||||
|
// hostname's CNAME chain differ from those the previous check saved,
|
||||||
|
// including when there are none now. When the previous check saved
|
||||||
|
// none, as in a state file from before they were saved, nothing is
|
||||||
|
// compared.
|
||||||
|
func (w *Watcher) detectCNAMEAddressChanges(
|
||||||
|
ctx context.Context,
|
||||||
|
hostname string,
|
||||||
|
prev, current *state.HostnameState,
|
||||||
|
) {
|
||||||
|
old, cur := prev.CNAMEAddresses, current.CNAMEAddresses
|
||||||
|
if len(old) == 0 || sliceEqual(old, cur) {
|
||||||
|
return
|
||||||
|
}
|
||||||
|
|
||||||
|
msg := fmt.Sprintf(
|
||||||
|
"Hostname: %s\nOld: %s\nNew: %s",
|
||||||
|
hostname,
|
||||||
|
strings.Join(old, ", "),
|
||||||
|
strings.Join(cur, ", "),
|
||||||
|
)
|
||||||
|
|
||||||
|
w.notify.SendNotification(
|
||||||
|
ctx,
|
||||||
|
"CNAME Address Change: "+hostname,
|
||||||
|
msg,
|
||||||
|
"warning",
|
||||||
|
)
|
||||||
}
|
}
|
||||||
|
|
||||||
// detectRecordChanges compares each nameserver's records with those of
|
// detectRecordChanges compares each nameserver's records with those of
|
||||||
@@ -751,6 +824,9 @@ func (w *Watcher) noNameserverAnswered(name string) bool {
|
|||||||
return true
|
return true
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// collectIPs returns the addresses saved for hostname: those in its
|
||||||
|
// nameservers' A and AAAA records, and those at the end of its CNAME
|
||||||
|
// chain.
|
||||||
func (w *Watcher) collectIPs(hostname string) []string {
|
func (w *Watcher) collectIPs(hostname string) []string {
|
||||||
hs, ok := w.state.GetHostnameState(hostname)
|
hs, ok := w.state.GetHostnameState(hostname)
|
||||||
if !ok {
|
if !ok {
|
||||||
@@ -769,6 +845,10 @@ func (w *Watcher) collectIPs(hostname string) []string {
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
for _, ip := range hs.CNAMEAddresses {
|
||||||
|
ipSet[ip] = true
|
||||||
|
}
|
||||||
|
|
||||||
result := make([]string, 0, len(ipSet))
|
result := make([]string, 0, len(ipSet))
|
||||||
for ip := range ipSet {
|
for ip := range ipSet {
|
||||||
result = append(result, ip)
|
result = append(result, ip)
|
||||||
@@ -1028,22 +1108,6 @@ func (w *Watcher) checkTLSExpiry(
|
|||||||
return
|
return
|
||||||
}
|
}
|
||||||
|
|
||||||
// Deduplicate expiry warnings: don't re-notify for the same
|
|
||||||
// hostname within the TLS check interval.
|
|
||||||
dedupKey := fmt.Sprintf("expiry:%s:%s", hostname, ip)
|
|
||||||
|
|
||||||
w.expiryNotifiedMu.Lock()
|
|
||||||
|
|
||||||
lastNotified, seen := w.expiryNotified[dedupKey]
|
|
||||||
if seen && time.Since(lastNotified) < w.config.TLSInterval {
|
|
||||||
w.expiryNotifiedMu.Unlock()
|
|
||||||
|
|
||||||
return
|
|
||||||
}
|
|
||||||
|
|
||||||
w.expiryNotified[dedupKey] = time.Now()
|
|
||||||
w.expiryNotifiedMu.Unlock()
|
|
||||||
|
|
||||||
msg := fmt.Sprintf(
|
msg := fmt.Sprintf(
|
||||||
"Host: %s\nIP: %s\nCN: %s\n"+
|
"Host: %s\nIP: %s\nCN: %s\n"+
|
||||||
"Expires: %s (%.0f days)",
|
"Expires: %s (%.0f days)",
|
||||||
|
|||||||
@@ -315,7 +315,8 @@ func lookupNameservers(t *testing.T, domain string) []string {
|
|||||||
return nameservers
|
return nameservers
|
||||||
}
|
}
|
||||||
|
|
||||||
// addresses returns the A and AAAA values saved for a hostname.
|
// addresses returns the A and AAAA values saved for a hostname, and the
|
||||||
|
// addresses saved at the end of its CNAME chain.
|
||||||
func addresses(hs *state.HostnameState) []string {
|
func addresses(hs *state.HostnameState) []string {
|
||||||
var ips []string
|
var ips []string
|
||||||
|
|
||||||
@@ -324,7 +325,7 @@ func addresses(hs *state.HostnameState) []string {
|
|||||||
ips = append(ips, nsState.Records["AAAA"]...)
|
ips = append(ips, nsState.Records["AAAA"]...)
|
||||||
}
|
}
|
||||||
|
|
||||||
return ips
|
return append(ips, hs.CNAMEAddresses...)
|
||||||
}
|
}
|
||||||
|
|
||||||
// assertNotified checks that a notification with this title and
|
// assertNotified checks that a notification with this title and
|
||||||
@@ -615,33 +616,54 @@ func TestTLSExpiryWarning(t *testing.T) {
|
|||||||
assertNotified(t, deps, "TLS Expiry Warning: "+testHost, "warning")
|
assertNotified(t, deps, "TLS Expiry Warning: "+testHost, "warning")
|
||||||
}
|
}
|
||||||
|
|
||||||
func TestTLSExpiryWarningDedup(t *testing.T) {
|
// TestTLSExpiryWarningEachCheck runs the TLS checks three times in a
|
||||||
|
// row on hostname and port state built here, for a certificate that
|
||||||
|
// expires within the warning period. Each check warns once, whether the
|
||||||
|
// TLS interval is a nanosecond, shorter than the time between two
|
||||||
|
// checks, or a day, longer than it.
|
||||||
|
func TestTLSExpiryWarningEachCheck(t *testing.T) {
|
||||||
t.Parallel()
|
t.Parallel()
|
||||||
|
|
||||||
cfg := defaultTestConfig(t)
|
title := "TLS Expiry Warning: " + host
|
||||||
cfg.Hostnames = []string{testHost}
|
|
||||||
cfg.TLSInterval = 24 * time.Hour
|
|
||||||
|
|
||||||
title := "TLS Expiry Warning: " + testHost
|
for _, interval := range []time.Duration{time.Nanosecond, 24 * time.Hour} {
|
||||||
|
t.Run(interval.String(), func(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
// The second check comes within the TLS interval of the first,
|
cfg := defaultTestConfig(t)
|
||||||
// so it must not warn again.
|
cfg.Hostnames = []string{host}
|
||||||
var warnings int
|
cfg.TLSInterval = interval
|
||||||
|
|
||||||
deps := runChecks(t, cfg, expiresInThreeDays, func(deps *testDeps) {
|
// The TLS checks read the saved hostname and port state and
|
||||||
warnings = countNotifications(deps, title)
|
// look nothing up, so the watcher has no resolver.
|
||||||
})
|
deps := newTestDeps(t, cfg)
|
||||||
|
w := watcher.NewForTest(
|
||||||
|
cfg, deps.state, nil,
|
||||||
|
deps.portChecker, deps.tlsChecker, deps.notifier,
|
||||||
|
)
|
||||||
|
|
||||||
if warnings == 0 {
|
expiresInThreeDays(deps)
|
||||||
t.Fatal("expected expiry warnings from the first check")
|
deps.state.SetHostnameState(host, saved(
|
||||||
}
|
map[string]*state.NameserverRecordState{
|
||||||
|
nsA: answered(map[string][]string{"A": {ip1}}),
|
||||||
|
},
|
||||||
|
))
|
||||||
|
deps.state.SetPortState(ip1+":443", &state.PortState{
|
||||||
|
Open: true, Hostnames: []string{host},
|
||||||
|
})
|
||||||
|
|
||||||
got := countNotifications(deps, title)
|
for check := 1; check <= 3; check++ {
|
||||||
if got != warnings {
|
w.RunTLSChecks(t.Context())
|
||||||
t.Errorf(
|
|
||||||
"expected %d expiry warnings (dedup), got %d",
|
got := countNotifications(deps, title)
|
||||||
warnings, got,
|
if got != check {
|
||||||
)
|
t.Fatalf(
|
||||||
|
"after check %d: %d expiry warnings, want %d",
|
||||||
|
check, got, check,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
})
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
|||||||
Reference in New Issue
Block a user