Compare commits

..
2 Commits
Author SHA1 Message Date
clawbot 623e7b2c69 resolver: a nameserver is asked at each of its addresses in turn (closes #285)
check / check (push) Canceled after 0s
A nameserver's queries went only to the first IPv4 address its name
resolves to, so one unreachable address made it fail whenever that
address came first in the answer. Each record type is now asked at the
addresses one at a time until one gives a usable reply, and the next
type is asked first at the address that gave it. The README's DNS
Resolution Strategy says which addresses are asked.

Model: opus-5-5
2026-10-15 10:17:14 +00:00
clawbot b79726acc6 watcher: a hostname whose zone's nameservers all stop answering gets NS Failure (closes #284)
check / check (push) Canceled after 0s
When none of the nameservers of a hostname's zone answered, the walk to
them failed and the check was only logged. The walk now returns a
NoAnswerError naming the nameservers of the zone it reached through a
referral when none of them answers. When those are the nameservers the
previous check saved for the name, each is saved as failed with the
reason, so it gets an NS Failure and later an NS Recovery. A domain with
no delegation of its own, whose NS lookup is that same walk, gets the
same. A walk that fails before that zone, as when the host loses its
network, is still only logged.

Model: opus-5-5
2026-10-14 00:27:29 +02:00
10 changed files with 458 additions and 57 deletions
+30 -16
View File
@@ -104,9 +104,11 @@ notification endpoint set, changes show only on the dashboard; see
NS change notification above. A domain listed in NS change notification above. A domain listed in
`DNSWATCHER_SKIP_RECORD_NOTIFICATIONS` gets no record change or inconsistency `DNSWATCHER_SKIP_RECORD_NOTIFICATIONS` gets no record change or inconsistency
notification. A domain with no delegation of its own has these records asked notification. A domain with no delegation of its own has these records asked
at the servers of the zone it is in, as a hostname has. A nameserver the at the servers of the zone it is in, as a hostname has. When none of those
domain's records were asked at on the previous check but not on this one gets servers answers, the lookup of its NS record set fails, and they are saved as
a hostname's NS change notification (see below), unless it was in the domain's failed and notified as a hostname's are (see below). A nameserver the domain's
records were asked at on the previous check but not on this one gets a
hostname's NS change notification (see below), unless it was in the domain's
delegation on the previous check, as when that delegation is withdrawn. A delegation on the previous check, as when that delegation is withdrawn. A
domain that does not exist has none: they are not asked for, and those saved domain that does not exist has none: they are not asked for, and those saved
by an earlier check are removed without a notification. by an earlier check are removed without a notification.
@@ -121,22 +123,29 @@ notification endpoint set, changes show only on the dashboard; see
trace moves from a name to its parent only when the servers asked answer that trace moves from a name to its parent only when the servers asked answer that
the name has no delegation of its own, or does not exist. When they do not the name has no delegation of its own, or does not exist. When they do not
answer, the check fails and the hostname's records from the previous check are answer, the check fails and the hostname's records from the previous check are
kept. kept, unless they are the nameservers the previous check saved for the
hostname. Each of those is then saved as failed, with the reason, as a
domain's nameservers are when none of them answers, so it gets an NS query
failure notification, and an NS recovery notification when it answers again
(see below). A trace that gets no answer before it reaches them, as when the
host running dnswatcher loses its network, fails the check without a
notification.
- Queries **each** authoritative nameserver independently for **all** record - Queries **each** authoritative nameserver independently for **all** record
types: A, AAAA, CNAME, MX, TXT, SRV, CAA, NS. types: A, AAAA, CNAME, MX, TXT, SRV, CAA, NS.
- Each record type is a query of its own. When a nameserver answers some types - Each record type is a query of its own. When a nameserver answers some types
but the query for another gets no usable reply (no reply after two tries, an but the query for another gets no usable reply at any of its addresses (no
error reply such as SERVFAIL, a referral, or a reply too large for UDP whose reply after two tries, an error reply such as SERVFAIL, a referral, or a reply
retry over TCP fails), the failure is logged with the reason, and the type is too large for UDP whose retry over TCP fails), the failure is logged with the
listed in the nameserver's `failedTypes` and keeps the records saved for the reason, and the type is listed in the nameserver's `failedTypes` and keeps the
nameserver by the previous check. On that check those records are not compared records saved for the nameserver by the previous check. On that check those
with the other nameservers', so no record change or inconsistency is reported records are not compared with the other nameservers', so no record change or
for the type; on the next check they are compared with the nameserver's answer inconsistency is reported for the type; on the next check they are compared
as usual. When the previous check did not know the type's records either, with the nameserver's answer as usual. When the previous check did not know
because the nameserver was new or failing then or the type was already listed the type's records either, because the nameserver was new or failing then or
in `unknownTypes`, the type is also listed in `unknownTypes` and left out of the type was already listed in `unknownTypes`, the type is also listed in
every comparison until it answers. A nameserver none of whose queries got a `unknownTypes` and left out of every comparison until it answers. A nameserver
usable reply has failed (see NS query failure below). none of whose queries got a usable reply has failed (see NS query failure
below).
- Stores results **per nameserver**. The state for a hostname is not a merged - Stores results **per nameserver**. The state for a hostname is not a merged
view — it is a map from nameserver to record set. view — it is a map from nameserver to record set.
- DNS names inside record values (CNAME, MX, SRV and NS targets) are stored in - DNS names inside record values (CNAME, MX, SRV and NS targets) are stored in
@@ -541,6 +550,11 @@ name's nameservers and in the lookup of a nameserver's own address. Such a
lookup can need others in turn; lookups go at most three deep, one inside lookup can need others in turn; lookups go at most three deep, one inside
another, so delegations that point at each other still end. another, so delegations that point at each other still end.
In step 4 each record type is asked at the IPv4 addresses the nameserver's name
resolves to, one at a time, until one gives a usable reply; the next type is
asked first at that address. A type's query fails only when none of them gives
one, and the nameserver has failed only when that is so for every type.
This approach ensures: This approach ensures:
- Independence from any upstream resolver's cache or filtering. - Independence from any upstream resolver's cache or filtering.
+4
View File
@@ -19,6 +19,10 @@ milestone: https://git.eeqj.de/sneak/dnswatcher/milestone/7
# Completed Steps # Completed Steps
- 2026-10-15: a nameserver is asked at each of its IPv4 addresses in turn, so it
has failed only when none of them gives a usable reply (closes #285).
- 2026-10-13: a hostname or a domain whose zone's nameservers all stop answering
gets an NS Failure for each, then an NS Recovery (closes #284).
- 2026-10-13: an IPv6 address is written in brackets before its port, as in - 2026-10-13: an IPv6 address is written in brackets before its port, as in
`[2001:db8::1]:443`, in keys and Port Change notifications (closes #270). `[2001:db8::1]:443`, in keys and Port Change notifications (closes #270).
- 2026-10-13: `TODO.md` names only open issues as next and future steps, and its - 2026-10-13: `TODO.md` names only open issues as next and future steps, and its
+17
View File
@@ -2,6 +2,23 @@ package resolver
import "errors" import "errors"
// NoAnswerError is returned when the walk to a name's nameservers
// reached a zone through a referral and none of that zone's
// nameservers, Nameservers as the referral names them, answered. Err
// says why they did not.
type NoAnswerError struct {
Nameservers []string
Err error
}
func (e *NoAnswerError) Error() string {
return e.Err.Error()
}
func (e *NoAnswerError) Unwrap() error {
return e.Err
}
// Sentinel errors returned by the resolver. // Sentinel errors returned by the resolver.
var ( var (
// ErrNoNameservers is returned when no authoritative NS // ErrNoNameservers is returned when no authoritative NS
+14 -1
View File
@@ -31,9 +31,11 @@ func NewWithQueryTimeout(log *slog.Logger, timeout time.Duration) *Resolver {
func (r *Resolver) FollowDelegation( func (r *Resolver) FollowDelegation(
ctx context.Context, ctx context.Context,
domain string, domain string,
zone string,
nameservers []string,
servers []string, servers []string,
) ([]string, error) { ) ([]string, error) {
return r.followDelegation(ctx, domain, servers) return r.followDelegation(ctx, domain, zone, nameservers, servers)
} }
// FindAuthoritativeNameserversFrom exports findAuthoritativeNameservers // FindAuthoritativeNameserversFrom exports findAuthoritativeNameservers
@@ -94,6 +96,17 @@ func (r *Resolver) QueryServers(
return r.queryServers(ctx, servers, zone, name, qtype) return r.queryServers(ctx, servers, zone, name, qtype)
} }
// QueryTypes exports queryTypes for testing.
func (r *Resolver) QueryTypes(
ctx context.Context,
nsHostname string,
nsIPs []string,
hostname string,
qtypes []uint16,
) (*NameserverResponse, error) {
return r.queryTypes(ctx, nsHostname, nsIPs, hostname, qtypes)
}
// QueryEachNS exports queryEachNS for testing. // QueryEachNS exports queryEachNS for testing.
func (r *Resolver) QueryEachNS( func (r *Resolver) QueryEachNS(
ctx context.Context, ctx context.Context,
+70 -32
View File
@@ -204,20 +204,24 @@ func glueIPs(nsNames []string, glue map[string][]net.IP) []string {
return ips return ips
} }
// followDelegation follows referrals from servers, the root servers, to // followDelegation follows referrals to domain from servers, the
// domain and returns the NS set of domain's delegation. When the servers // addresses of the servers of zone, and returns the NS set of domain's
// of the zone domain is in answer that domain does not exist, the error // delegation. nameservers are the names a referral gave the servers of
// is ErrNXDomain. When they answer that it has no delegation of its own, // zone; the walk to a name's nameservers starts at the root servers, the
// servers of zone ".", which no referral names. When the servers of the
// zone domain is in answer that domain does not exist, the error is
// ErrNXDomain. When they answer that it has no delegation of its own,
// because it is not the zone's apex, the set is empty and there is no // because it is not the zone's apex, the set is empty and there is no
// error. Any other error means that no such answer came. // error. Any other error means that no such answer came; it is a
// *NoAnswerError when the servers that did not answer are those of a
// zone a referral named.
func (r *Resolver) followDelegation( func (r *Resolver) followDelegation(
ctx context.Context, ctx context.Context,
domain string, domain string,
zone string,
nameservers []string,
servers []string, servers []string,
) ([]string, error) { ) ([]string, error) {
// servers are the root servers, the servers of zone ".".
zone := "."
var withoutAddresses []string var withoutAddresses []string
for range maxDelegation { for range maxDelegation {
@@ -228,6 +232,12 @@ func (r *Resolver) followDelegation(
resp, err := r.queryZone( resp, err := r.queryZone(
ctx, servers, withoutAddresses, zone, domain, dns.TypeNS, 0, ctx, servers, withoutAddresses, zone, domain, dns.TypeNS, 0,
) )
// A query that ctx cut short says nothing about the servers.
if err != nil && len(nameservers) > 0 && checkCtx(ctx) == nil {
return nil, &NoAnswerError{Nameservers: nameservers, Err: err}
}
if err != nil { if err != nil {
return nil, err return nil, err
} }
@@ -256,6 +266,7 @@ func (r *Resolver) followDelegation(
return r.resolveNSIterative(ctx, domain) return r.resolveNSIterative(ctx, domain)
} }
nameservers = authNS
servers, withoutAddresses = referralNameservers(resp) servers, withoutAddresses = referralNameservers(resp)
zone = referralZone(resp) zone = referralZone(resp)
} }
@@ -626,7 +637,8 @@ func (r *Resolver) resolveARecord(
// them. When the servers asked answer that the name has no delegation // them. When the servers asked answer that the name has no delegation
// of its own, or does not exist, it tries each parent name in turn, so // of its own, or does not exist, it tries each parent name in turn, so
// it returns the nameservers of the zone the name is in. When they do // it returns the nameservers of the zone the name is in. When they do
// not answer, it returns the error and tries no parent name. // not answer, it returns the error and tries no parent name; when a
// referral named them, the error is a *NoAnswerError naming them.
func (r *Resolver) FindAuthoritativeNameservers( func (r *Resolver) FindAuthoritativeNameservers(
ctx context.Context, ctx context.Context,
domain string, domain string,
@@ -655,7 +667,7 @@ func (r *Resolver) findAuthoritativeNameservers(
candidate := strings.Join(labels[i:], ".") + "." candidate := strings.Join(labels[i:], ".") + "."
nsNames, err := r.followDelegation(ctx, candidate, servers) nsNames, err := r.followDelegation(ctx, candidate, ".", nil, servers)
if err != nil && !errors.Is(err, ErrNXDomain) { if err != nil && !errors.Is(err, ErrNXDomain) {
return nil, err return nil, err
} }
@@ -696,8 +708,9 @@ func (r *Resolver) QueryNameserver(
return r.queryNameserver(ctx, nsHostname, hostname, recordTypes()) return r.queryNameserver(ctx, nsHostname, hostname, recordTypes())
} }
// queryNameserver queries a specific nameserver for the record types // queryNameserver queries a specific nameserver, at the IPv4 addresses
// in qtypes and builds a NameserverResponse. // its name resolves to, for the record types in qtypes and builds a
// NameserverResponse.
func (r *Resolver) queryNameserver( func (r *Resolver) queryNameserver(
ctx context.Context, ctx context.Context,
nsHostname string, nsHostname string,
@@ -715,7 +728,7 @@ func (r *Resolver) queryNameserver(
hostname = dns.Fqdn(hostname) hostname = dns.Fqdn(hostname)
return r.queryTypes(ctx, nsHostname, nsIPs[0], hostname, qtypes) return r.queryTypes(ctx, nsHostname, nsIPs, hostname, qtypes)
} }
// QueryNameserverIP queries a nameserver by its IP address directly, // QueryNameserverIP queries a nameserver by its IP address directly,
@@ -732,13 +745,18 @@ func (r *Resolver) QueryNameserverIP(
hostname = dns.Fqdn(hostname) hostname = dns.Fqdn(hostname)
return r.queryTypes(ctx, nsHostname, nsIP, hostname, recordTypes()) return r.queryTypes(
ctx, nsHostname, []string{nsIP}, hostname, recordTypes(),
)
} }
// queryTypes asks the nameserver nsHostname, at its addresses nsIPs,
// about hostname's records of each type in qtypes and builds a
// NameserverResponse.
func (r *Resolver) queryTypes( func (r *Resolver) queryTypes(
ctx context.Context, ctx context.Context,
nsHostname string, nsHostname string,
nsIP string, nsIPs []string,
hostname string, hostname string,
qtypes []uint16, qtypes []uint16,
) (*NameserverResponse, error) { ) (*NameserverResponse, error) {
@@ -748,7 +766,7 @@ func (r *Resolver) queryTypes(
Status: StatusOK, Status: StatusOK,
} }
state := r.queryEachType(ctx, nsIP, hostname, qtypes, resp) state := r.queryEachType(ctx, nsIPs, hostname, qtypes, resp)
classifyResponse(resp, state) classifyResponse(resp, state)
return resp, nil return resp, nil
@@ -766,28 +784,37 @@ type queryState struct {
answered bool answered bool
} }
// queryEachType asks the nameserver at nsIP about hostname once for each // queryEachType asks the nameserver at its addresses nsIPs about hostname
// record type in qtypes, and lists in resp.FailedTypes the types whose // once for each record type in qtypes, and lists in resp.FailedTypes the
// query got no usable reply, logging each with the reason unless ctx was // types whose query got no usable reply at any of them, logging each with
// cancelled: shutdown cancels it, and a query it cut short did not fail. // the reason unless ctx was cancelled: shutdown cancels it, and a query
// it cut short did not fail.
func (r *Resolver) queryEachType( func (r *Resolver) queryEachType(
ctx context.Context, ctx context.Context,
nsIP string, nsIPs []string,
hostname string, hostname string,
qtypes []uint16, qtypes []uint16,
resp *NameserverResponse, resp *NameserverResponse,
) queryState { ) queryState {
var state queryState var state queryState
// Reordered below; the caller's list is left as it was.
nsIPs = slices.Clone(nsIPs)
for _, qtype := range qtypes { for _, qtype := range qtypes {
if checkCtx(ctx) != nil { if checkCtx(ctx) != nil {
break break
} }
err := r.querySingleType(ctx, nsIP, hostname, qtype, resp, &state) answeredAt, err := r.querySingleType(
ctx, nsIPs, hostname, qtype, resp, &state,
)
if err == nil { if err == nil {
state.answered = true state.answered = true
// The next type is asked first at the address that answered.
nsIPs[0], nsIPs[answeredAt] = nsIPs[answeredAt], nsIPs[0]
continue continue
} }
@@ -821,20 +848,29 @@ func (r *Resolver) queryEachType(
return state return state
} }
// querySingleType asks the nameserver at nsIP about hostname's records // querySingleType asks the nameserver about hostname's records of type
// of type qtype. It returns nil when the nameserver answered: with // qtype at its addresses nsIPs, one at a time, until one answers: with
// records, with none, or with NXDOMAIN; otherwise it returns why not. // records, with none, or with NXDOMAIN. It returns the index in nsIPs of
// the address that answered; when none did, it returns why the last one
// did not.
func (r *Resolver) querySingleType( func (r *Resolver) querySingleType(
ctx context.Context, ctx context.Context,
nsIP string, nsIPs []string,
hostname string, hostname string,
qtype uint16, qtype uint16,
resp *NameserverResponse, resp *NameserverResponse,
state *queryState, state *queryState,
) error { ) (int, error) {
msg, err := r.queryDNS(ctx, nsIP, hostname, qtype) var err error
if err != nil {
for i, nsIP := range nsIPs {
var msg *dns.Msg
msg, err = r.queryDNS(ctx, nsIP, hostname, qtype)
switch { switch {
case err == nil:
err = readReply(msg, resp, state)
case isTimeout(err): case isTimeout(err):
state.gotTimeout = true state.gotTimeout = true
case errors.Is(err, ErrRefused): case errors.Is(err, ErrRefused):
@@ -843,10 +879,12 @@ func (r *Resolver) querySingleType(
state.netErr = err state.netErr = err
} }
return err if err == nil {
return i, nil
}
} }
return readReply(msg, resp, state) return 0, err
} }
// readReply adds to resp the records in msg, a nameserver's reply to a // readReply adds to resp the records in msg, a nameserver's reply to a
@@ -1067,7 +1105,7 @@ func (r *Resolver) LookupNS(
} }
return r.followDelegation( return r.followDelegation(
ctx, dns.Fqdn(strings.ToLower(domain)), rootServerList(), ctx, dns.Fqdn(strings.ToLower(domain)), ".", nil, rootServerList(),
) )
} }
+81 -2
View File
@@ -121,7 +121,9 @@ func TestFindAuthoritativeNameservers_DelegatedSubdomain(
// delegation of its own, so the lookup returns that walk's error, about // delegation of its own, so the lookup returns that walk's error, about
// www.google.com, and tries no parent name: trying google.com and com // www.google.com, and tries no parent name: trying google.com and com
// would end in ErrNoNameservers, or in the error of a walk for one of // would end in ErrNoNameservers, or in the error of a walk for one of
// them. // them. The servers that did not answer are those the walk started at,
// which no referral named, so the error is no NoAnswerError, as when
// the host running dnswatcher has lost its network.
func TestFindAuthoritativeNameservers_NoAnswer(t *testing.T) { func TestFindAuthoritativeNameservers_NoAnswer(t *testing.T) {
t.Parallel() t.Parallel()
@@ -134,6 +136,9 @@ func TestFindAuthoritativeNameservers_NoAnswer(t *testing.T) {
require.NotErrorIs(t, err, resolver.ErrNoNameservers) require.NotErrorIs(t, err, resolver.ErrNoNameservers)
assert.Contains(t, err.Error(), "query www.google.com. @"+noAnswerAddress) assert.Contains(t, err.Error(), "query www.google.com. @"+noAnswerAddress)
assert.Empty(t, nameservers) assert.Empty(t, nameservers)
var noAnswer *resolver.NoAnswerError
assert.NotErrorAs(t, err, &noAnswer)
} }
func TestFindAuthoritativeNameservers_ReturnsSorted( func TestFindAuthoritativeNameservers_ReturnsSorted(
@@ -355,6 +360,58 @@ func TestQueryNameserver_ZoneDelegatedWithoutAddresses(t *testing.T) {
assert.Equal(t, resolver.StatusOK, resp.Status) assert.Equal(t, resolver.StatusOK, resp.Status)
} }
// TestQueryTypes_FirstAddressNeverAnswers asks a google.com nameserver
// about google.com's A and AAAA records at a list of addresses that puts
// 192.0.2.1, where nothing answers, before the nameserver's own. The
// nameserver answers at its own address, so it is ok, with its records.
// Each attempt has time for one round of tries at 192.0.2.1, two tries
// of two seconds, and not for two, so AAAA must be asked first at the
// address that answered A.
func TestQueryTypes_FirstAddressNeverAnswers(t *testing.T) {
t.Parallel()
r := newTestResolver(t)
ns := findOneNSForDomain(t, r, "google.com")
nsIPs := append(
[]string{noAnswerAddress}, liveResolveNSIPs(t, r, []string{ns}, 1)...,
)
var resp *resolver.NameserverResponse
livednstest.Retry(
t,
"QueryTypes("+ns+" at "+strings.Join(nsIPs, ", ")+", google.com)",
func(ctx context.Context) error {
ctx, cancel := context.WithTimeout(ctx, 6*time.Second)
defer cancel()
var err error
resp, err = r.QueryTypes(
ctx, ns, nsIPs, "google.com.",
[]uint16{dns.TypeA, dns.TypeAAAA},
)
if err != nil {
return err
}
if len(resp.FailedTypes) > 0 {
return fmt.Errorf(
"%w: %s gave no usable reply for %s",
livednstest.ErrNoAnswer, ns,
strings.Join(resp.FailedTypes, ", "),
)
}
return nil
},
)
assert.Equal(t, resolver.StatusOK, resp.Status)
assert.NotEmpty(t, resp.Records["A"])
assert.NotEmpty(t, resp.Records["AAAA"])
}
func TestQueryNameserver_AAAA(t *testing.T) { func TestQueryNameserver_AAAA(t *testing.T) {
t.Parallel() t.Parallel()
@@ -954,12 +1011,34 @@ func TestFollowDelegation_NoAnswer(t *testing.T) {
r := resolver.NewWithQueryTimeout(slog.Default(), 100*time.Millisecond) r := resolver.NewWithQueryTimeout(slog.Default(), 100*time.Millisecond)
nameservers, err := r.FollowDelegation( nameservers, err := r.FollowDelegation(
t.Context(), "google.com.", []string{noAnswerAddress}, t.Context(), "google.com.", ".", nil, []string{noAnswerAddress},
) )
require.Error(t, err) require.Error(t, err)
assert.Empty(t, nameservers) assert.Empty(t, nameservers)
} }
// TestFollowDelegation_NoAnswerAfterReferral starts the walk for
// www.example.net at the servers of example.net as a referral names
// them, ns1.example.net and ns2.example.net, at 192.0.2.1, where nothing
// answers. The error is a NoAnswerError naming them, which the watcher
// compares with the nameservers it saved for the name.
func TestFollowDelegation_NoAnswerAfterReferral(t *testing.T) {
t.Parallel()
r := resolver.NewWithQueryTimeout(slog.Default(), 100*time.Millisecond)
nameservers := []string{"ns1.example.net.", "ns2.example.net."}
_, err := r.FollowDelegation(
t.Context(), "www.example.net.", "example.net.", nameservers,
[]string{noAnswerAddress},
)
var noAnswer *resolver.NoAnswerError
require.ErrorAs(t, err, &noAnswer)
assert.Equal(t, nameservers, noAnswer.Nameservers)
assert.Contains(t, err.Error(), "query www.example.net. @"+noAnswerAddress)
}
// TestResolveNSIterative_NoDelegationOfItsOwn walks to the nameservers // TestResolveNSIterative_NoDelegationOfItsOwn walks to the nameservers
// of www.google.com as the fallback walk does. As in // of www.google.com as the fallback walk does. As in
// TestLookupNS_NoDelegationOfItsOwn, the set is empty, with no error. // TestLookupNS_NoDelegationOfItsOwn, the set is empty, with no error.
+28
View File
@@ -122,6 +122,34 @@ func (w *Watcher) RunTLSChecks(ctx context.Context) {
w.runTLSChecks(ctx) w.runTLSChecks(ctx)
} }
// CheckHostnameResult exports checkHostnameResult for testing.
func (w *Watcher) CheckHostnameResult(
ctx context.Context,
hostname string,
results map[string]*resolver.NameserverResponse,
err error,
) bool {
return w.checkHostnameResult(ctx, hostname, results, err)
}
// CheckDomainResult exports checkDomainResult for testing.
func (w *Watcher) CheckDomainResult(
ctx context.Context,
domain string,
nameservers []string,
err error,
) {
w.checkDomainResult(ctx, domain, nameservers, err)
}
// FailedNameservers exports failedNameservers for testing.
func FailedNameservers(
err error,
prev *state.HostnameState,
) map[string]*resolver.NameserverResponse {
return failedNameservers(err, prev)
}
// BuildHostnameState exports buildHostnameState for testing. // BuildHostnameState exports buildHostnameState for testing.
func BuildHostnameState( func BuildHostnameState(
results map[string]*resolver.NameserverResponse, results map[string]*resolver.NameserverResponse,
+4 -1
View File
@@ -20,7 +20,10 @@ type DNSResolver interface {
) ([]string, error) ) ([]string, error)
// LookupAllRecords queries all record types for a hostname, // LookupAllRecords queries all record types for a hostname,
// returning each nameserver's response keyed by nameserver. // returning each nameserver's response keyed by nameserver. When
// the walk to the nameservers reached a zone through a referral and
// none of that zone's nameservers answered, the error is a
// *resolver.NoAnswerError naming them.
LookupAllRecords( LookupAllRecords(
ctx context.Context, ctx context.Context,
hostname string, hostname string,
+133
View File
@@ -2,6 +2,7 @@ package watcher_test
import ( import (
"context" "context"
"errors"
"fmt" "fmt"
"log/slog" "log/slog"
"strings" "strings"
@@ -14,6 +15,10 @@ import (
"sneak.berlin/go/dnswatcher/internal/watcher" "sneak.berlin/go/dnswatcher/internal/watcher"
) )
// errNoReply is why a walk that got no reply from a zone's servers
// failed.
var errNoReply = errors.New("all servers failed: i/o timeout")
// answered is what a check saves for a nameserver that answered with // answered is what a check saves for a nameserver that answered with
// these records. // these records.
func answered(records map[string][]string) *state.NameserverRecordState { func answered(records map[string][]string) *state.NameserverRecordState {
@@ -183,6 +188,134 @@ func TestNSFailureAlertNamesHostnameNameserverAndReason(t *testing.T) {
} }
} }
// TestZoneNameserversAllStopAnswering runs three checks of host from
// the point where its lookup has returned. On the first, nsA and nsB,
// the nameservers of its zone, answer. On the second, the lookup fails
// with the error the walk to them returns when neither answers (see
// the resolver's TestFollowDelegation_NoAnswerAfterReferral), given
// here because no live zone has nameservers that never answer. For a
// hostname it is the lookup of its records that fails, and for a domain
// with no delegation of its own the lookup of its delegation. Both are
// saved as failed, with the reason, and each gets an NS Failure. On the
// third, both answer, and each gets an NS Recovery. A domain's records
// are checked as a hostname's, so its first and third checks are too.
func TestZoneNameserversAllStopAnswering(t *testing.T) {
t.Parallel()
records := map[string][]string{"A": {ip1}}
bothAnswer := map[string]*resolver.NameserverResponse{
nsA: response(records), nsB: response(records),
}
noAnswer := &resolver.NoAnswerError{
Nameservers: []string{nsA, nsB}, Err: errNoReply,
}
tests := []struct {
name string
isDomain bool
}{
{"hostname", false},
{"domain with no delegation of its own", true},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
cfg := defaultTestConfig(t)
if tt.isDomain {
cfg.Domains = []string{host}
} else {
cfg.Hostnames = []string{host}
}
// The watcher is given each lookup's result, so it has no
// resolver.
deps := newTestDeps(t, cfg)
w := watcher.NewForTest(
cfg, deps.state, nil,
deps.portChecker, deps.tlsChecker, deps.notifier,
)
w.SetFirstRun(false)
w.CheckHostnameResult(t.Context(), host, bothAnswer, nil)
if tt.isDomain {
w.CheckDomainResult(t.Context(), host, nil, noAnswer)
} else {
w.CheckHostnameResult(t.Context(), host, nil, noAnswer)
}
current, _ := deps.state.GetHostnameState(host)
for _, ns := range []string{nsA, nsB} {
got := current.RecordsByNameserver[ns]
if got.Status != failed().Status || got.Error != errNoReply.Error() {
t.Errorf(
"%s saved with status %q, error %q; want %q, %q",
ns, got.Status, got.Error, failed().Status, errNoReply,
)
}
}
if n := countNotifications(deps, "NS Failure: "+host); n != 2 {
t.Errorf("sent %d NS Failure notifications, want 2", n)
}
w.CheckHostnameResult(t.Context(), host, bothAnswer, nil)
if n := countNotifications(deps, "NS Recovery: "+host); n != 2 {
t.Errorf("sent %d NS Recovery notifications, want 2", n)
}
})
}
}
// TestFailedNameserversOnlyForThoseOfThePreviousCheck gives
// FailedNameservers lookup errors after a check in which nsA and nsB
// answered. Only an error that names both makes them failed.
func TestFailedNameserversOnlyForThoseOfThePreviousCheck(t *testing.T) {
t.Parallel()
records := map[string][]string{"A": {ip1}}
bothAnswered := saved(map[string]*state.NameserverRecordState{
nsA: answered(records), nsB: answered(records),
})
noAnswer := func(nameservers ...string) error {
return &resolver.NoAnswerError{Nameservers: nameservers, Err: errNoReply}
}
tests := []struct {
name string
err error
prev *state.HostnameState
wantFailed bool
}{
{"both did not answer", noAnswer(nsA, nsB), bothAnswered, true},
{
"the servers of a parent zone did not answer",
noAnswer(nsC), bothAnswered, false,
},
{
"the zone's delegation lists only nsA now",
noAnswer(nsA), bothAnswered, false,
},
{"no root server answered", errNoReply, bothAnswered, false},
{"no previous check", noAnswer(nsA, nsB), nil, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
got := watcher.FailedNameservers(tt.err, tt.prev)
if (got != nil) != tt.wantFailed {
t.Errorf("returned %v, want failed: %v", got, tt.wantFailed)
}
})
}
}
// TestNameserverThatNeverAnswers asks a nameserver address where // TestNameserverThatNeverAnswers asks a nameserver address where
// nothing answers, 192.0.2.1, and checks what the watcher saves for it. // nothing answers, 192.0.2.1, and checks what the watcher saves for it.
// The deadline outlasts the resolver's first two-second try, as in the // The deadline outlasts the resolver's first two-second try, as in the
+77 -5
View File
@@ -291,12 +291,25 @@ func (w *Watcher) logFailedLookup(
w.log.Error(msg, args...) w.log.Error(msg, args...)
} }
// checkDomain looks up domain's delegation and checks domain with it
// (see checkDomainResult).
func (w *Watcher) checkDomain( func (w *Watcher) checkDomain(
ctx context.Context, ctx context.Context,
domain string, domain string,
) { ) {
nameservers, err := w.resolver.LookupNS(ctx, domain) nameservers, err := w.resolver.LookupNS(ctx, domain)
w.checkDomainResult(ctx, domain, nameservers, err)
}
// checkDomainResult finishes a check of domain from nameservers and err,
// what the lookup of its delegation returned.
func (w *Watcher) checkDomainResult(
ctx context.Context,
domain string,
nameservers []string,
err error,
) {
// A domain that does not exist has no nameservers. // A domain that does not exist has no nameservers.
nxdomain := errors.Is(err, resolver.ErrNXDomain) nxdomain := errors.Is(err, resolver.ErrNXDomain)
if nxdomain { if nxdomain {
@@ -311,6 +324,12 @@ func (w *Watcher) checkDomain(
"error", err, "error", err,
) )
// For a domain with no delegation of its own, the servers that
// did not answer can be those of the zone it is in, at which
// its records are asked. They are then saved as failed, as a
// hostname's are.
w.checkHostnameResult(ctx, domain, nil, err)
return return
} }
@@ -464,8 +483,8 @@ func (w *Watcher) detectNSAddressChanges(
} }
} }
// checkHostname checks hostname's records and reports whether it saved // checkHostname looks up hostname's records and checks them (see
// them; it does not when their lookup fails. // checkHostnameResult). It reports whether it saved them.
func (w *Watcher) checkHostname( func (w *Watcher) checkHostname(
ctx context.Context, ctx context.Context,
hostname string, hostname string,
@@ -478,12 +497,31 @@ func (w *Watcher) checkHostname(
"hostname", hostname, "hostname", hostname,
"error", err, "error", err,
) )
return false
} }
return w.checkHostnameResult(ctx, hostname, results, err)
}
// checkHostnameResult finishes a check of hostname from results and err,
// what the lookup of its records returned, and reports whether it saved
// them. When the lookup failed, it saves nothing, unless the nameservers
// that did not answer are those of the previous check (see
// failedNameservers).
func (w *Watcher) checkHostnameResult(
ctx context.Context,
hostname string,
results map[string]*resolver.NameserverResponse,
err error,
) bool {
prev, _ := w.state.GetHostnameState(hostname) prev, _ := w.state.GetHostnameState(hostname)
if err != nil {
results = failedNameservers(err, prev)
if results == nil {
return false
}
}
w.updateHostnameState( w.updateHostnameState(
ctx, hostname, buildHostnameState(results, prev, time.Now().UTC()), ctx, hostname, buildHostnameState(results, prev, time.Now().UTC()),
) )
@@ -491,6 +529,40 @@ func (w *Watcher) checkHostname(
return true return true
} }
// failedNameservers returns a failed response for each nameserver in
// prev, a name's state from the previous check, with err as the reason,
// when err says that the walk to the name's nameservers reached their
// zone and none of them answered. For any other error, as when the walk
// got no answer before it reached their zone, or when prev is nil, it
// returns nil.
func failedNameservers(
err error,
prev *state.HostnameState,
) map[string]*resolver.NameserverResponse {
var noAnswer *resolver.NoAnswerError
if !errors.As(err, &noAnswer) || prev == nil {
return nil
}
saved := slices.Sorted(maps.Keys(prev.RecordsByNameserver))
if !slices.Equal(noAnswer.Nameservers, saved) {
return nil
}
results := make(map[string]*resolver.NameserverResponse)
for _, ns := range saved {
results[ns] = &resolver.NameserverResponse{
Nameserver: ns,
Records: make(map[string][]string),
Status: resolver.StatusError,
Error: err.Error(),
}
}
return results
}
// updateHostnameState finishes a check of hostname from newState, built // updateHostnameState finishes a check of hostname from newState, built
// from its nameservers' answers: it follows the CNAME in them, notifies // from its nameservers' answers: it follows the CNAME in them, notifies
// what changed since the previous check, and saves newState. // what changed since the previous check, and saves newState.
@@ -790,7 +862,7 @@ func (w *Watcher) detectRecordChanges(
// saved delegation, which detectNSChanges notifies when they are removed // saved delegation, which detectNSChanges notifies when they are removed
// from it. checkDomain saves a new delegation only after this runs, and // from it. checkDomain saves a new delegation only after this runs, and
// only together with the records, so the saved delegation and the // only together with the records, so the saved delegation and the
// records in prev come from the same check. // nameservers in prev come from the same check.
func (w *Watcher) detectNSDisappearances( func (w *Watcher) detectNSDisappearances(
ctx context.Context, ctx context.Context,
hostname string, hostname string,