3 Commit
Autore SHA1 Messaggio Data
sneak c0647eef26 resolver, watcher: a record type whose query fails keeps its previous records (closes #231)
check / check (push) Canceled after 0s
The resolver lists in FailedTypes each record type whose query to a
nameserver got no usable reply (no reply, a code other than NOERROR or
NXDOMAIN, a referral, or a truncated reply whose TCP retry failed) and
logs it unless shutdown cut it short. A nameserver that answered no
type has failed.
The watcher saves such a type in failedTypes with the previous check's
records, leaves it out of the comparison with other nameservers on that
check, and compares it with the next answer. When the previous check
did not know its records either, it is also in unknownTypes and not
compared until it answers. A nameserver whose A, AAAA or CNAME query
failed is no answer when following a CNAME or resolving addresses.

Model: opus-5-5
2026-10-02 07:53:15 +00:00
clawbot 250f3dd687 dashboard and status API show why a check failed (closes #225)
check / check (push) Canceled after 0s
/api/v1/status now gives `error` for each nameserver entry and
certificate entry whose status is `error`, copied from the state, which
already kept it. The dashboard shows that reason in place of the records
for a failed nameserver, which used to show the same `-` as one that
answered with no records, and across the CN, issuer and expiry cells
for a failed certificate, wrapped at a width of 20rem so the long TLS
error does not narrow the Endpoint column. The dashboard stylesheet is a
trimmed build, so the new markup uses only classes the page already had.
README Web Dashboard and HTTP API say so.

Model: opus-5-5
2026-10-02 09:14:28 +02:00
clawbot c07976a73a resolver: store a name's CNAME once per nameserver (closes #220)
check / check (push) Canceled after 0s
For a name with a CNAME, a nameserver answers a query of any type with
that CNAME, and the records of every answer were added, so the CNAME
was stored once for each of the eight record types asked for.
collectAnswerRecords now adds each value once per record type.

A state file saved before this holds the repeated values. Load keeps
each record value once, so the first check after upgrading sees no
record change and notifies nothing for them.

Model: opus-5-5
2026-10-02 09:09:51 +02:00
21 ha cambiato i file con 1293 aggiunte e 63 eliminazioni
+38 -10
Vedi File
@@ -80,7 +80,8 @@ notification endpoint set, changes show only on the dashboard; see
different addresses than on the previous check. A nameserver added or different addresses than on the previous check. A nameserver added or
removed gets only the NS change notification. When the lookup of a removed gets only the NS change notification. When the lookup of a
nameserver's addresses fails or finds none, its previous addresses are nameserver's addresses fails or finds none, its previous addresses are
kept and nothing is sent. kept and nothing is sent. The lookup fails when no nameserver it asks
answers every one of its queries, for A, AAAA and CNAME.
### DNS Hostname Monitoring (Subdomains) ### DNS Hostname Monitoring (Subdomains)
@@ -91,6 +92,19 @@ notification endpoint set, changes show only on the dashboard; see
its last two labels (a name under `co.uk`, or in a delegated subdomain). its last two labels (a name under `co.uk`, or in a delegated subdomain).
- Queries **each** authoritative nameserver independently for **all** record - Queries **each** authoritative nameserver independently for **all** record
types: A, AAAA, CNAME, MX, TXT, SRV, CAA, NS. types: A, AAAA, CNAME, MX, TXT, SRV, CAA, NS.
- Each record type is a query of its own. When a nameserver answers some types
but the query for another gets no usable reply (no reply after two tries, an
error reply such as SERVFAIL, a referral, or a reply too large for UDP whose
retry over TCP fails), the failure is logged with the reason, and the type is
listed in the nameserver's `failedTypes` and keeps the records saved for the
nameserver by the previous check. On that check those records are not compared
with the other nameservers', so no record change or inconsistency is reported
for the type; on the next check they are compared with the nameserver's answer
as usual. When the previous check did not know the type's records either,
because the nameserver was new or failing then or the type was already listed
in `unknownTypes`, the type is also listed in `unknownTypes` and left out of
every comparison until it answers. A nameserver none of whose queries got a
usable reply has failed (see NS query failure below).
- Stores results **per nameserver**. The state for a hostname is not a merged - Stores results **per nameserver**. The state for a hostname is not a merged
view — it is a map from nameserver to record set. view — it is a map from nameserver to record set.
- DNS names inside record values (CNAME, MX, SRV and NS targets) are stored in - DNS names inside record values (CNAME, MX, SRV and NS targets) are stored in
@@ -119,8 +133,9 @@ notification endpoint set, changes show only on the dashboard; see
they keep disagreeing, including after a restart. A nameserver that was they keep disagreeing, including after a restart. A nameserver that was
not in the previous check (newly added, or back after dropping out), or not in the previous check (newly added, or back after dropping out), or
failed on it, and answers differently is reported on the check where it failed on it, and answers differently is reported on the check where it
answers. If a pair agrees again and later disagrees, the alert is sent answers. So is a pair that differs in a record type whose query to either
again. nameserver failed on the previous check. If a pair agrees again and later
disagrees, the alert is sent again.
- **CNAME address change**: The addresses at the end of a name's CNAME chain - **CNAME address change**: The addresses at the end of a name's CNAME chain
differ from those of the previous check. They are found when its differ from those of the previous check. They are found when its
nameservers answer with a CNAME and no address; a name that answers with nameservers answer with a CNAME and no address; a name that answers with
@@ -230,9 +245,11 @@ dnswatcher includes an unauthenticated, read-only web dashboard at the root URL
- **Summary counts** for monitored domains, hostnames, ports, and certificates. - **Summary counts** for monitored domains, hostnames, ports, and certificates.
- **Domains** with their discovered nameservers. - **Domains** with their discovered nameservers.
- **Hostnames** with per-nameserver DNS records and status. - **Hostnames** with per-nameserver DNS records and status. For a nameserver
whose query failed, the reason is shown in place of the records.
- **Ports** with open/closed state and associated hostnames. - **Ports** with open/closed state and associated hostnames.
- **TLS certificates** with CN, issuer, expiry, and status. - **TLS certificates** with CN, issuer, expiry, and status. For a failed check,
the reason is shown in place of CN, issuer and expiry.
- **Recent alerts** (last 100 notifications sent since the process started), - **Recent alerts** (last 100 notifications sent since the process started),
displayed in reverse chronological order. displayed in reverse chronological order.
@@ -259,6 +276,10 @@ dnswatcher exposes a lightweight HTTP API for operational visibility:
| `GET /api/v1/status` | Current monitoring state | | `GET /api/v1/status` | Current monitoring state |
| `GET /metrics` | Prometheus metrics, see below | | `GET /metrics` | Prometheus metrics, see below |
In `/api/v1/status`, each nameserver entry and certificate entry whose `status`
is `error` also has `error`, the reason, as in the state file (see State File
Format).
`/metrics` is served only when `DNSWATCHER_METRICS_USERNAME` is set, behind `/metrics` is served only when `DNSWATCHER_METRICS_USERNAME` is set, behind
Basic Auth. It has the Prometheus Go client's default metrics only (Go runtime, Basic Auth. It has the Prometheus Go client's default metrics only (Go runtime,
process, and counts of `/metrics` requests); dnswatcher records no metrics of process, and counts of `/metrics` requests); dnswatcher records no metrics of
@@ -519,7 +540,7 @@ reachability:
| Status | Meaning | | Status | Meaning |
| ------- | -------------------------------------------------------- | | ------- | -------------------------------------------------------- |
| `ok` | Query succeeded, records are current | | `ok` | Query succeeded, records are current except as below |
| `error` | Query failed (timeout, SERVFAIL, REFUSED, network error) | | `error` | Query failed (timeout, SERVFAIL, REFUSED, network error) |
A nameserver that answers NXDOMAIN or with no records has status `ok` and empty A nameserver that answers NXDOMAIN or with no records has status `ok` and empty
@@ -528,6 +549,13 @@ nameservers, has status `error`, empty `records`, and the reason in `error`. A
certificate entry whose TLS connection or handshake failed likewise has status certificate entry whose TLS connection or handshake failed likewise has status
`error`, the reason in `error`, and the certificate fields left empty or zero. `error`, the reason in `error`, and the certificate fields left empty or zero.
A nameserver with status `ok` whose query for one record type failed lists that
type in `failedTypes` and holds its records from the previous check, which may
not be current. When the previous check did not know the type's records either,
because the nameserver was new or failing then or the type was already listed in
`unknownTypes`, the type is also listed in `unknownTypes`, and `records` holds
nothing for it. Both lists are left out when empty.
`nameserverAddresses` lists, by nameserver, the sorted addresses its name `nameserverAddresses` lists, by nameserver, the sorted addresses its name
resolves to. A state file without it loads, and the next check fills it in resolves to. A state file without it loads, and the next check fills it in
without a notification. without a notification.
@@ -535,10 +563,10 @@ without a notification.
`cnameAddresses` lists the sorted addresses at the end of the chain of every `cnameAddresses` lists the sorted addresses at the end of the chain of every
CNAME target a hostname's nameservers gave, found when they answered with a CNAME target a hostname's nameservers gave, found when they answered with a
CNAME and no address; it is empty when they answered with an address. When a CNAME and no address; it is empty when they answered with an address. When a
chain cannot be followed, or none of the name's nameservers answered, the chain cannot be followed, or none of the name's nameservers answered its queries
previous check's list is kept, or `null` when no earlier check saved one. A for A, AAAA and CNAME, the previous check's list is kept, or `null` when no
state file without it loads, and the first check after that saves it without a earlier check saved one. A state file without it loads, and the first check
notification. after that saves it without a notification.
A port entry in the older format, with one `hostname` instead of the `hostnames` A port entry in the older format, with one `hostname` instead of the `hostnames`
list, loads as a list of that one name. list, loads as a list of that one name.
+6
Vedi File
@@ -19,6 +19,12 @@ trial run of the finished image: https://git.eeqj.de/sneak/dnswatcher/issues/149
# Completed Steps # Completed Steps
- 2026-10-02: a record type whose query to a nameserver fails keeps its previous
records and alerts nothing; the other types are still saved (closes #231).
- 2026-10-02: the dashboard and `/api/v1/status` show why a nameserver query or
a certificate check failed, which only the state file showed (closes #225).
- 2026-10-02: a name's CNAME is stored once per nameserver, not once per record
type asked for; a state file with repeats loads each value once (closes #220).
- 2026-10-02: a DNS lookup that shutdown cuts short logs no error; one that - 2026-10-02: a DNS lookup that shutdown cuts short logs no error; one that
fails otherwise, or runs out of time, still does (closes #229). fails otherwise, or runs out of time, still does (closes #229).
- 2026-10-02: Record Change and Inconsistency notifications list only the record - 2026-10-02: Record Change and Inconsistency notifications list only the record
+37
Vedi File
@@ -1,6 +1,7 @@
package handlers_test package handlers_test
import ( import (
"strings"
"testing" "testing"
"time" "time"
@@ -78,3 +79,39 @@ func TestFormatRecords(t *testing.T) {
t.Errorf("unexpected format: %q", got) t.Errorf("unexpected format: %q", got)
} }
} }
// dashboardRow returns the table row of page that contains name.
func dashboardRow(t *testing.T, page string, name string) string {
t.Helper()
for row := range strings.SplitSeq(page, "<tr") {
if strings.Contains(row, name) {
return row
}
}
t.Fatalf("dashboard has no row containing %q", name)
return ""
}
// TestDashboardShowsFailureReasons checks that the dashboard shows the
// reason in the row of a failed nameserver and of a failed certificate,
// and not in the row of a nameserver that answered.
func TestDashboardShowsFailureReasons(t *testing.T) {
t.Parallel()
page := get(t, newHandlersWithFailures(t).HandleDashboard())
if !strings.Contains(dashboardRow(t, page, failedNS), nsFailureReason) {
t.Errorf("row of %s does not show %q", failedNS, nsFailureReason)
}
if strings.Contains(dashboardRow(t, page, answeringNS), nsFailureReason) {
t.Errorf("row of %s shows %q", answeringNS, nsFailureReason)
}
if !strings.Contains(dashboardRow(t, page, certKey), certFailedReason) {
t.Errorf("row of %s does not show %q", certKey, certFailedReason)
}
}
+4
Vedi File
@@ -18,6 +18,7 @@ type statusDomainInfo struct {
type statusHostnameNSInfo struct { type statusHostnameNSInfo struct {
Records map[string][]string `json:"records"` Records map[string][]string `json:"records"`
Status string `json:"status"` Status string `json:"status"`
Error string `json:"error,omitempty"`
LastChecked time.Time `json:"lastChecked"` LastChecked time.Time `json:"lastChecked"`
} }
@@ -41,6 +42,7 @@ type statusCertificateInfo struct {
NotAfter time.Time `json:"notAfter"` NotAfter time.Time `json:"notAfter"`
SubjectAlternativeNames []string `json:"subjectAlternativeNames"` SubjectAlternativeNames []string `json:"subjectAlternativeNames"`
Status string `json:"status"` Status string `json:"status"`
Error string `json:"error,omitempty"`
LastChecked time.Time `json:"lastChecked"` LastChecked time.Time `json:"lastChecked"`
} }
@@ -144,6 +146,7 @@ func buildHostnames(
info.Nameservers[ns] = &statusHostnameNSInfo{ info.Nameservers[ns] = &statusHostnameNSInfo{
Records: recs, Records: recs,
Status: nsState.Status, Status: nsState.Status,
Error: nsState.Error,
LastChecked: nsState.LastChecked, LastChecked: nsState.LastChecked,
} }
} }
@@ -183,6 +186,7 @@ func buildCertificates(
NotAfter: cs.NotAfter, NotAfter: cs.NotAfter,
SubjectAlternativeNames: sans, SubjectAlternativeNames: sans,
Status: cs.Status, Status: cs.Status,
Error: cs.Error,
LastChecked: cs.LastChecked, LastChecked: cs.LastChecked,
} }
} }
+158
Vedi File
@@ -0,0 +1,158 @@
package handlers_test
import (
"encoding/json"
"net/http"
"net/http/httptest"
"testing"
"time"
"go.uber.org/fx/fxtest"
"sneak.berlin/go/dnswatcher/internal/config"
"sneak.berlin/go/dnswatcher/internal/globals"
"sneak.berlin/go/dnswatcher/internal/handlers"
"sneak.berlin/go/dnswatcher/internal/logger"
"sneak.berlin/go/dnswatcher/internal/notify"
"sneak.berlin/go/dnswatcher/internal/state"
)
// The state the handler tests serve: www.example.com has one nameserver
// that answered and one whose query failed, and its certificate check
// failed.
const (
testHostname = "www.example.com"
answeringNS = "ns1.example.com."
failedNS = "ns2.example.com."
nsFailureReason = "server returned a referral"
certKey = "192.0.2.1:443:www.example.com"
certFailedReason = "x509: certificate has expired or is not yet valid"
)
// newHandlersWithFailures builds real Handlers whose state holds the
// entries described above.
func newHandlersWithFailures(t *testing.T) *handlers.Handlers {
t.Helper()
glob, err := globals.New(nil)
if err != nil {
t.Fatalf("globals.New: %v", err)
}
log, err := logger.New(nil, logger.Params{Globals: glob})
if err != nil {
t.Fatalf("logger.New: %v", err)
}
notifier, err := notify.New(fxtest.NewLifecycle(t), notify.Params{
Logger: log,
Config: &config.Config{},
})
if err != nil {
t.Fatalf("notify.New: %v", err)
}
st, err := state.New(fxtest.NewLifecycle(t), state.Params{
Logger: log,
Config: &config.Config{DataDir: t.TempDir()},
})
if err != nil {
t.Fatalf("state.New: %v", err)
}
now := time.Now()
st.SetHostnameState(testHostname, &state.HostnameState{
RecordsByNameserver: map[string]*state.NameserverRecordState{
answeringNS: {
Records: map[string][]string{"A": {"192.0.2.1"}},
Status: "ok",
LastChecked: now,
},
failedNS: {
Records: map[string][]string{},
Status: "error",
Error: nsFailureReason,
LastChecked: now,
},
},
LastChecked: now,
})
st.SetCertificateState(certKey, &state.CertificateState{
Status: "error",
Error: certFailedReason,
LastChecked: now,
})
hnd, err := handlers.New(nil, handlers.Params{
Logger: log,
Globals: glob,
State: st,
Notify: notifier,
})
if err != nil {
t.Fatalf("handlers.New: %v", err)
}
return hnd
}
// get serves one GET request to handler and returns the response body.
func get(t *testing.T, handler http.HandlerFunc) string {
t.Helper()
rec := httptest.NewRecorder()
req := httptest.NewRequestWithContext(
t.Context(), http.MethodGet, "/", nil,
)
handler(rec, req)
if rec.Code != http.StatusOK {
t.Fatalf("status = %d, want 200", rec.Code)
}
return rec.Body.String()
}
// TestStatusGivesFailureReasons checks that /api/v1/status gives the
// reason for a failed nameserver entry and a failed certificate entry,
// and no error for a nameserver that answered.
func TestStatusGivesFailureReasons(t *testing.T) {
t.Parallel()
body := get(t, newHandlersWithFailures(t).HandleStatus())
var resp struct {
Hostnames map[string]struct {
Nameservers map[string]map[string]any `json:"nameservers"`
} `json:"hostnames"`
Certificates map[string]map[string]any `json:"certificates"`
}
err := json.Unmarshal([]byte(body), &resp)
if err != nil {
t.Fatalf("decoding response: %v", err)
}
nameservers := resp.Hostnames[testHostname].Nameservers
got := nameservers[failedNS]["error"]
if got != nsFailureReason {
t.Errorf("failed nameserver error = %v, want %q",
got, nsFailureReason)
}
_, has := nameservers[answeringNS]["error"]
if has {
t.Errorf("answering nameserver has an error field: %v",
nameservers[answeringNS])
}
got = resp.Certificates[certKey]["error"]
if got != certFailedReason {
t.Errorf("failed certificate error = %v, want %q",
got, certFailedReason)
}
}
@@ -146,7 +146,11 @@
<td <td
class="py-2 px-3 text-slate-400 break-all max-w-xs" class="py-2 px-3 text-slate-400 break-all max-w-xs"
> >
{{ if $nsr.Error }}
<span class="text-red-400">{{ $nsr.Error }}</span>
{{ else }}
{{ formatRecords $nsr.Records }} {{ formatRecords $nsr.Records }}
{{ end }}
</td> </td>
<td class="py-2 px-3 text-slate-500 whitespace-nowrap"> <td class="py-2 px-3 text-slate-500 whitespace-nowrap">
{{ relTime $nsr.LastChecked }} {{ relTime $nsr.LastChecked }}
@@ -258,6 +262,11 @@
> >
{{ end }} {{ end }}
</td> </td>
{{ if $cs.Error }}
<td colspan="3" class="py-2 px-3 text-red-400 break-all">
<div class="max-w-xs">{{ $cs.Error }}</div>
</td>
{{ else }}
<td class="py-2 px-3 text-slate-200"> <td class="py-2 px-3 text-slate-200">
{{ $cs.CommonName }} {{ $cs.CommonName }}
</td> </td>
@@ -285,6 +294,7 @@
{{ end }} {{ end }}
{{ end }} {{ end }}
</td> </td>
{{ end }}
<td class="py-2 px-3 text-slate-500 whitespace-nowrap"> <td class="py-2 px-3 text-slate-500 whitespace-nowrap">
{{ relTime $cs.LastChecked }} {{ relTime $cs.LastChecked }}
</td> </td>
+6
Vedi File
@@ -22,6 +22,12 @@ var (
"reply is an error or a referral that leads no closer", "reply is an error or a referral that leads no closer",
) )
// ErrTruncated is the reason given for a reply too large for UDP
// whose retry over TCP failed.
ErrTruncated = errors.New(
"reply truncated and its retry over TCP failed",
)
// ErrIntercepted is returned when every root server refused a // ErrIntercepted is returned when every root server refused a
// query. Root servers refuse no query, so the refusals came from // query. Root servers refuse no query, so the refusals came from
// something on the network answering in their place. // something on the network answering in their place.
+18
Vedi File
@@ -2,15 +2,33 @@ package resolver
import ( import (
"context" "context"
"log/slog"
"time"
"github.com/miekg/dns" "github.com/miekg/dns"
) )
// NewWithFailingTCP returns a Resolver whose TCP client gives up before
// it can connect, so the retry over TCP of every truncated reply fails.
func NewWithFailingTCP(log *slog.Logger) *Resolver {
r := NewFromLogger(log)
r.tcp = &tcpClient{timeout: time.Nanosecond}
return r
}
// ExtractRecordValue exports extractRecordValue for testing. // ExtractRecordValue exports extractRecordValue for testing.
func ExtractRecordValue(rr dns.RR) string { func ExtractRecordValue(rr dns.RR) string {
return extractRecordValue(rr) return extractRecordValue(rr)
} }
// CollectAnswerRecords exports collectAnswerRecords for testing.
func CollectAnswerRecords(msg *dns.Msg, resp *NameserverResponse) {
var state queryState
collectAnswerRecords(msg, resp, &state)
}
// UsableReply exports usableReply for testing. // UsableReply exports usableReply for testing.
func UsableReply(resp *dns.Msg, zone string, name string) bool { func UsableReply(resp *dns.Msg, zone string, name string) bool {
return usableReply(resp, zone, name) return usableReply(resp, zone, name)
+97 -19
Vedi File
@@ -89,6 +89,9 @@ func (r *Resolver) tryExchange(
return resp, err return resp, err
} }
// retryTCP returns the reply to msg over TCP when resp, its reply over
// UDP, is truncated. When that fails it returns resp, still truncated,
// which holds only the records that fit.
func (r *Resolver) retryTCP( func (r *Resolver) retryTCP(
ctx context.Context, ctx context.Context,
msg *dns.Msg, msg *dns.Msg,
@@ -632,14 +635,19 @@ func (r *Resolver) queryTypes(
type queryState struct { type queryState struct {
gotNXDomain bool gotNXDomain bool
gotSERVFAIL bool errorReply string // code of an error reply, such as SERVFAIL
gotRefused bool gotRefused bool
gotTimeout bool gotTimeout bool
gotReferral bool gotReferral bool
netErr error netErr error
hasRecords bool hasRecords bool
answered bool
} }
// queryEachType asks the nameserver at nsIP about hostname once for each
// record type in qtypes, and lists in resp.FailedTypes the types whose
// query got no usable reply, logging each with the reason unless ctx was
// cancelled: shutdown cancels it, and a query it cut short did not fail.
func (r *Resolver) queryEachType( func (r *Resolver) queryEachType(
ctx context.Context, ctx context.Context,
nsIP string, nsIP string,
@@ -654,7 +662,34 @@ func (r *Resolver) queryEachType(
break break
} }
r.querySingleType(ctx, nsIP, hostname, qtype, resp, &state) err := r.querySingleType(ctx, nsIP, hostname, qtype, resp, &state)
if err == nil {
state.answered = true
continue
}
rtype := dns.TypeToString[qtype]
resp.FailedTypes = append(resp.FailedTypes, rtype)
if errors.Is(ctx.Err(), context.Canceled) {
continue
}
r.log.Warn(
"record type query failed",
"hostname", hostname,
"nameserver", resp.Nameserver,
"type", rtype,
"error", err,
)
}
// The reply about another type can carry the name's CNAME. When the
// query for CNAME itself failed, that is left out too, so Records
// holds nothing for a failed type.
for _, rtype := range resp.FailedTypes {
delete(resp.Records, rtype)
} }
for k := range resp.Records { for k := range resp.Records {
@@ -664,6 +699,9 @@ func (r *Resolver) queryEachType(
return state return state
} }
// querySingleType asks the nameserver at nsIP about hostname's records
// of type qtype. It returns nil when the nameserver answered: with
// records, with none, or with NXDOMAIN; otherwise it returns why not.
func (r *Resolver) querySingleType( func (r *Resolver) querySingleType(
ctx context.Context, ctx context.Context,
nsIP string, nsIP string,
@@ -671,7 +709,7 @@ func (r *Resolver) querySingleType(
qtype uint16, qtype uint16,
resp *NameserverResponse, resp *NameserverResponse,
state *queryState, state *queryState,
) { ) error {
msg, err := r.queryDNS(ctx, nsIP, hostname, qtype) msg, err := r.queryDNS(ctx, nsIP, hostname, qtype)
if err != nil { if err != nil {
switch { switch {
@@ -683,19 +721,36 @@ func (r *Resolver) querySingleType(
state.netErr = err state.netErr = err
} }
return return err
} }
return readReply(msg, resp, state)
}
// readReply adds to resp the records in msg, a nameserver's reply to a
// query about one record type. It returns nil when the nameserver
// answered: with records, with none, or with NXDOMAIN; otherwise it
// returns why not.
func readReply(
msg *dns.Msg,
resp *NameserverResponse,
state *queryState,
) error {
if msg.Rcode == dns.RcodeNameError { if msg.Rcode == dns.RcodeNameError {
state.gotNXDomain = true state.gotNXDomain = true
return return nil
} }
if msg.Rcode == dns.RcodeServerFailure { // A reply with any other code but NOERROR, such as SERVFAIL, NOTIMP or
state.gotSERVFAIL = true // FORMERR, is an error reply and says nothing about the name's
// records. usableReply takes the same codes as no usable reply.
if msg.Rcode != dns.RcodeSuccess {
state.errorReply = dns.RcodeToString[msg.Rcode]
return return fmt.Errorf(
"server returned %s: %w", state.errorReply, ErrUnusableReply,
)
} }
// A reply with no answer that lists other nameservers, from a server // A reply with no answer that lists other nameservers, from a server
@@ -708,12 +763,26 @@ func (r *Resolver) querySingleType(
len(extractNSSet(msg.Ns)) > 0 { len(extractNSSet(msg.Ns)) > 0 {
state.gotReferral = true state.gotReferral = true
return return fmt.Errorf("server returned a referral: %w", ErrUnusableReply)
}
// A reply still truncated is one whose TCP retry failed, and holds
// only the records that fit.
if msg.Truncated {
state.netErr = ErrTruncated
return ErrTruncated
} }
collectAnswerRecords(msg, resp, state) collectAnswerRecords(msg, resp, state)
return nil
} }
// collectAnswerRecords adds the records in msg's answer to resp, each
// value once per record type. For a name with a CNAME, a nameserver
// answers a query of any type with that CNAME, so the same value comes
// in the answer to every type asked for.
func collectAnswerRecords( func collectAnswerRecords(
msg *dns.Msg, msg *dns.Msg,
resp *NameserverResponse, resp *NameserverResponse,
@@ -726,9 +795,12 @@ func collectAnswerRecords(
} }
typeName := dns.TypeToString[rr.Header().Rrtype] typeName := dns.TypeToString[rr.Header().Rrtype]
if !slices.Contains(resp.Records[typeName], val) {
resp.Records[typeName] = append( resp.Records[typeName] = append(
resp.Records[typeName], val, resp.Records[typeName], val,
) )
}
state.hasRecords = true state.hasRecords = true
} }
} }
@@ -743,23 +815,26 @@ func isTimeout(err error) bool {
return false return false
} }
// classifyResponse sets the nameserver's status. One that answered no
// record type has failed, and Error says why; one that answered some has
// the status of those answers.
func classifyResponse(resp *NameserverResponse, state queryState) { func classifyResponse(resp *NameserverResponse, state queryState) {
switch { switch {
case state.gotNXDomain && !state.hasRecords: case state.gotNXDomain && !state.hasRecords:
resp.Status = StatusNXDomain resp.Status = StatusNXDomain
case state.gotTimeout && !state.hasRecords: case state.gotTimeout && !state.answered:
resp.Status = StatusTimeout resp.Status = StatusTimeout
resp.Error = "all queries timed out" resp.Error = "all queries timed out"
case state.gotSERVFAIL && !state.hasRecords: case state.errorReply != "" && !state.answered:
resp.Status = StatusError resp.Status = StatusError
resp.Error = "server returned SERVFAIL" resp.Error = "server returned " + state.errorReply
case state.gotRefused && !state.hasRecords: case state.gotRefused && !state.answered:
resp.Status = StatusError resp.Status = StatusError
resp.Error = "server returned REFUSED" resp.Error = "server returned REFUSED"
case state.netErr != nil && !state.hasRecords: case state.netErr != nil && !state.answered:
resp.Status = StatusError resp.Status = StatusError
resp.Error = "network error: " + state.netErr.Error() resp.Error = "network error: " + state.netErr.Error()
case state.gotReferral && !state.hasRecords: case state.gotReferral && !state.answered:
resp.Status = StatusError resp.Status = StatusError
resp.Error = "server returned a referral" resp.Error = "server returned a referral"
case !state.hasRecords && !state.gotNXDomain: case !state.hasRecords && !state.gotNXDomain:
@@ -920,9 +995,11 @@ func (r *Resolver) resolveIPWithCNAME(
} }
// collectIPs returns the addresses in the nameservers' answers and the // collectIPs returns the addresses in the nameservers' answers and the
// first CNAME target among them. It returns ErrNoNameserverAnswered when // first CNAME target among them. A nameserver whose query for one of the
// every nameserver timed out, failed or returned a referral: that is not // types failed gave only part of the addresses, and is left out. It
// a name with no addresses. // returns ErrNoNameserverAnswered when every nameserver timed out,
// failed, returned a referral or was left out: that is not a name with
// no addresses.
func collectIPs( func collectIPs(
results map[string]*NameserverResponse, results map[string]*NameserverResponse,
) ([]string, string, error) { ) ([]string, string, error) {
@@ -935,7 +1012,8 @@ func collectIPs(
answered := false answered := false
for _, resp := range results { for _, resp := range results {
if resp.Status == StatusTimeout || resp.Status == StatusError { if resp.Status == StatusTimeout || resp.Status == StatusError ||
len(resp.FailedTypes) > 0 {
continue continue
} }
@@ -0,0 +1,116 @@
package resolver
import (
"syscall"
"testing"
"github.com/miekg/dns"
"github.com/stretchr/testify/assert"
)
// TestClassifyResponse sets a nameserver's status from the results of
// its queries, built here. One that answered some record types, even
// with no records, has not failed when its query for another type got
// no usable reply, whatever the reason; one whose every query got none
// has.
func TestClassifyResponse(t *testing.T) {
t.Parallel()
tests := []struct {
name string
results queryState
wantStatus string
wantError string
}{
{
"some types answered with no records, another timed out",
queryState{answered: true, gotTimeout: true},
StatusNoData, "",
},
{
"some types answered with no records, another got SERVFAIL",
queryState{answered: true, errorReply: "SERVFAIL"},
StatusNoData, "",
},
{
"some types answered with no records, another was refused",
queryState{answered: true, gotRefused: true},
StatusNoData, "",
},
{
"some types answered with no records, another got a network error",
queryState{answered: true, netErr: syscall.ECONNREFUSED},
StatusNoData, "",
},
{
"some types answered with no records, another's reply was " +
"truncated and its retry over TCP failed",
queryState{answered: true, netErr: ErrTruncated},
StatusNoData, "",
},
{
"some types answered with no records, another got a referral",
queryState{answered: true, gotReferral: true},
StatusNoData, "",
},
{
"every query timed out",
queryState{gotTimeout: true},
StatusTimeout, "all queries timed out",
},
{
"every query got NOTIMP",
queryState{errorReply: "NOTIMP"},
StatusError, "server returned NOTIMP",
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
resp := &NameserverResponse{Status: StatusOK}
classifyResponse(resp, tt.results)
assert.Equal(t, tt.wantStatus, resp.Status)
assert.Equal(t, tt.wantError, resp.Error)
})
}
}
// TestReadReply checks which replies to a query about one record type,
// built here, are an answer: one with the code NOERROR or NXDOMAIN. A
// reply with any other code is not, and the type's query has failed.
func TestReadReply(t *testing.T) {
t.Parallel()
tests := []struct {
rcode int
answered bool
}{
{dns.RcodeSuccess, true},
{dns.RcodeNameError, true},
{dns.RcodeServerFailure, false},
{dns.RcodeNotImplemented, false},
{dns.RcodeFormatError, false},
}
for _, tt := range tests {
t.Run(dns.RcodeToString[tt.rcode], func(t *testing.T) {
t.Parallel()
msg := new(dns.Msg)
msg.Authoritative = true
msg.Rcode = tt.rcode
resp := &NameserverResponse{Records: map[string][]string{}}
err := readReply(msg, resp, &queryState{})
if tt.answered {
assert.NoError(t, err)
} else {
assert.ErrorIs(t, err, ErrUnusableReply)
}
})
}
}
+51
Vedi File
@@ -43,6 +43,25 @@ func TestCollectIPs_FailedIsNoAnswer(t *testing.T) {
assert.Empty(t, ips) assert.Empty(t, ips)
} }
// TestCollectIPs_FailedTypeIsNoAnswer checks that a nameserver whose
// query for one of the types failed is no answer: its addresses are
// only part of them.
func TestCollectIPs_FailedTypeIsNoAnswer(t *testing.T) {
t.Parallel()
ips, _, err := resolver.CollectIPs(
map[string]*resolver.NameserverResponse{
nsExample1: {
Records: map[string][]string{"A": {"192.0.2.1"}},
FailedTypes: []string{"AAAA"},
Status: resolver.StatusOK,
},
},
)
require.ErrorIs(t, err, resolver.ErrNoNameserverAnswered)
assert.Empty(t, ips)
}
const ( const (
// exampleCom is the zone most cases of TestUsableReply and // exampleCom is the zone most cases of TestUsableReply and
// TestNSSetFrom are about, and wwwExampleCom a name in it. // TestNSSetFrom are about, and wwwExampleCom a name in it.
@@ -238,6 +257,38 @@ func TestExtractRecordValue_LetterCase(t *testing.T) {
} }
} }
// TestCollectAnswerRecords_CNAMEOnce collects the answers a nameserver
// gives for a name with a CNAME, one for each record type a check asks
// for. Each answer holds the CNAME, which must be stored once.
func TestCollectAnswerRecords_CNAMEOnce(t *testing.T) {
t.Parallel()
cname := &dns.CNAME{
Hdr: dns.RR_Header{
Name: "git.eeqj.de.", Rrtype: dns.TypeCNAME, Class: dns.ClassINET,
},
Target: "fsn1app1.datavi.be.",
}
resp := &resolver.NameserverResponse{Records: map[string][]string{}}
for _, qtype := range []uint16{
dns.TypeA, dns.TypeAAAA, dns.TypeCNAME, dns.TypeMX,
dns.TypeTXT, dns.TypeSRV, dns.TypeCAA, dns.TypeNS,
} {
msg := new(dns.Msg)
msg.SetQuestion("git.eeqj.de.", qtype)
msg.Answer = []dns.RR{cname}
resolver.CollectAnswerRecords(msg, resp)
}
assert.Equal(t,
map[string][]string{"CNAME": {"fsn1app1.datavi.be."}},
resp.Records,
)
}
// TestShuffled shuffles the root servers with many seeds. Every order // TestShuffled shuffles the root servers with many seeds. Every order
// must hold each root server once, so each is tried before a // must hold each root server once, so each is tried before a
// resolution fails; each root server must come first for some seed, so // resolution fails; each root server must come first for some seed, so
+4
Vedi File
@@ -31,9 +31,13 @@ type Params struct {
} }
// NameserverResponse holds one nameserver's response for a query. // NameserverResponse holds one nameserver's response for a query.
// FailedTypes lists the record types whose query got no usable reply,
// and Records holds nothing for them: their records are not known. When
// no record type got one, Status and Error say the nameserver failed.
type NameserverResponse struct { type NameserverResponse struct {
Nameserver string Nameserver string
Records map[string][]string Records map[string][]string
FailedTypes []string
Status string Status string
Error string Error string
} }
+48
Vedi File
@@ -1,6 +1,7 @@
package resolver_test package resolver_test
import ( import (
"bytes"
"context" "context"
"fmt" "fmt"
"log/slog" "log/slog"
@@ -245,6 +246,30 @@ func TestQueryNameserver_TXT(t *testing.T) {
) )
} }
// TestQueryNameserver_TruncatedReplyWhoseTCPRetryFails asks a google.com
// nameserver about google.com with a resolver whose retries over TCP
// fail. google.com's TXT records do not fit in a reply over UDP, so TXT
// is reported as failed, holding none of the records that fit, and
// logged with the reason, while the nameserver, which answered the other
// types, is ok.
func TestQueryNameserver_TruncatedReplyWhoseTCPRetryFails(t *testing.T) {
t.Parallel()
ns := findOneNSForDomain(t, newTestResolver(t), "google.com")
var logs bytes.Buffer
r := resolver.NewWithFailingTCP(slog.New(slog.NewTextHandler(&logs, nil)))
resp := liveQueryNameserver(t, r, ns, "google.com")
assert.Equal(t, resolver.StatusOK, resp.Status)
assert.Contains(t, resp.FailedTypes, "TXT")
assert.NotContains(t, resp.Records, "TXT")
assert.Contains(t, logs.String(),
"hostname=google.com. nameserver="+ns+" type=TXT error=",
)
}
func TestQueryNameserver_NXDomain(t *testing.T) { func TestQueryNameserver_NXDomain(t *testing.T) {
t.Parallel() t.Parallel()
@@ -865,6 +890,29 @@ func TestQueryNameserverIP_Timeout(t *testing.T) {
assert.NotEmpty(t, resp.Error) assert.NotEmpty(t, resp.Error)
} }
// TestQueryNameserverIP_CancelledLogsNothing cancels the context while
// a query to 192.0.2.1, where nothing answers, is waiting for a reply,
// as shutdown does. The query was cut short, not failed, so nothing is
// logged.
func TestQueryNameserverIP_CancelledLogsNothing(t *testing.T) {
t.Parallel()
var logs bytes.Buffer
r := resolver.NewFromLogger(slog.New(slog.NewTextHandler(&logs, nil)))
ctx, cancel := context.WithCancel(context.Background())
t.Cleanup(cancel)
time.AfterFunc(100*time.Millisecond, cancel)
_, err := r.QueryNameserverIP(
ctx, "unreachable.test.", "192.0.2.1", "example.com",
)
require.NoError(t, err)
assert.Empty(t, logs.String())
}
// TestCollectIPs_NoNameserverAnswered takes the response of a // TestCollectIPs_NoNameserverAnswered takes the response of a
// nameserver at 192.0.2.1, where nothing answers, as // nameserver at 192.0.2.1, where nothing answers, as
// TestQueryNameserverIP_Timeout does. Addresses collected from // TestQueryNameserverIP_Timeout does. Addresses collected from
+21
Vedi File
@@ -8,6 +8,7 @@ import (
"log/slog" "log/slog"
"os" "os"
"path/filepath" "path/filepath"
"slices"
"sync" "sync"
"time" "time"
@@ -45,8 +46,15 @@ type DomainState struct {
} }
// NameserverRecordState holds one NS's response for a hostname. // NameserverRecordState holds one NS's response for a hostname.
// FailedTypes lists the record types whose query to the nameserver
// failed on this check: Records holds for them the records saved by the
// previous check, which are kept. UnknownTypes lists those of them whose
// records the previous check did not know either, as when the
// nameserver was new or failing then: Records holds nothing for them.
type NameserverRecordState struct { type NameserverRecordState struct {
Records map[string][]string `json:"records"` Records map[string][]string `json:"records"`
FailedTypes []string `json:"failedTypes,omitempty"`
UnknownTypes []string `json:"unknownTypes,omitempty"`
Status string `json:"status"` Status string `json:"status"`
Error string `json:"error,omitempty"` Error string `json:"error,omitempty"`
LastChecked time.Time `json:"lastChecked"` LastChecked time.Time `json:"lastChecked"`
@@ -201,6 +209,19 @@ func (s *State) Load() error {
return fmt.Errorf("parsing state file: %w", err) return fmt.Errorf("parsing state file: %w", err)
} }
// A state file saved before each record value was stored once can
// hold a hostname's CNAME once for every record type asked for.
// Each value is kept once, so the first check does not see a
// record change.
for _, hs := range snapshot.Hostnames {
for _, ns := range hs.RecordsByNameserver {
for recordType, values := range ns.Records {
slices.Sort(values)
ns.Records[recordType] = slices.Compact(values)
}
}
}
s.snapshot = &snapshot s.snapshot = &snapshot
s.log.Info("loaded state from disk", "path", path) s.log.Info("loaded state from disk", "path", path)
+108
Vedi File
@@ -236,6 +236,61 @@ func TestSaveLoadRoundTrip_CNAMEAddresses(t *testing.T) {
} }
} }
// TestSaveLoadRoundTrip_FailedTypes checks that a nameserver's
// failedTypes and unknownTypes survive a save and load. Without
// unknownTypes, a type whose records were not known would load as one
// with no records.
func TestSaveLoadRoundTrip_FailedTypes(t *testing.T) {
t.Parallel()
dir := t.TempDir()
s := state.NewForTestWithDataDir(dir)
failed := []string{"TXT", "CAA"}
unknown := []string{"CAA"}
s.SetHostnameState(testHostname, &state.HostnameState{
RecordsByNameserver: map[string]*state.NameserverRecordState{
testNS1: {
Records: map[string][]string{"TXT": {"v=spf1 -all"}},
FailedTypes: failed,
UnknownTypes: unknown,
Status: "ok",
},
},
})
err := s.Save()
if err != nil {
t.Fatalf("Save() error: %v", err)
}
loaded := state.NewForTestWithDataDir(dir)
err = loaded.Load()
if err != nil {
t.Fatalf("Load() error: %v", err)
}
hs, ok := loaded.GetHostnameState(testHostname)
if !ok {
t.Fatal("missing hostname " + testHostname)
}
ns1 := hs.RecordsByNameserver[testNS1]
if ns1 == nil {
t.Fatal("missing nameserver " + testNS1)
}
if !reflect.DeepEqual(ns1.FailedTypes, failed) {
t.Errorf("failedTypes: got %#v", ns1.FailedTypes)
}
if !reflect.DeepEqual(ns1.UnknownTypes, unknown) {
t.Errorf("unknownTypes: got %#v", ns1.UnknownTypes)
}
}
// TestLoadStateFromBeforeCNAMEAddresses loads a state file written // TestLoadStateFromBeforeCNAMEAddresses loads a state file written
// before the addresses at the end of a hostname's CNAME chain were // before the addresses at the end of a hostname's CNAME chain were
// saved. They load as not known (nil), not as none. // saved. They load as not known (nil), not as none.
@@ -277,6 +332,59 @@ func TestLoadStateFromBeforeCNAMEAddresses(t *testing.T) {
} }
} }
// TestLoadStateWithRepeatedValues loads a state file saved when a
// hostname's CNAME was stored once for every record type asked for.
// Each value must load once, and every different value must load.
func TestLoadStateWithRepeatedValues(t *testing.T) {
t.Parallel()
dir := t.TempDir()
data := []byte(`{
"version": 1,
"hostnames": {
"www.example.com": {
"recordsByNameserver": {
"ns1.example.com.": {
"records": {
"A": ["192.0.2.2", "192.0.2.1", "192.0.2.2", "192.0.2.1"],
"CNAME": ["a.example.net.", "a.example.net.", "a.example.net."]
},
"status": "ok"
}
}
}
}
}`)
err := os.WriteFile(filepath.Join(dir, "state.json"), data, 0o600)
if err != nil {
t.Fatalf("writing state file: %v", err)
}
s := state.NewForTestWithDataDir(dir)
err = s.Load()
if err != nil {
t.Fatalf("Load() error: %v", err)
}
hs, ok := s.GetHostnameState(testHostname)
if !ok {
t.Fatal("missing hostname " + testHostname)
}
want := map[string][]string{
"A": {"192.0.2.1", "192.0.2.2"},
"CNAME": {"a.example.net."},
}
got := hs.RecordsByNameserver[testNS1].Records
if !reflect.DeepEqual(got, want) {
t.Errorf("records: got %v, want %v", got, want)
}
}
// TestSaveLoadRoundTrip_Hostnames verifies hostname data survives a save/load cycle. // TestSaveLoadRoundTrip_Hostnames verifies hostname data survives a save/load cycle.
func TestSaveLoadRoundTrip_Hostnames(t *testing.T) { func TestSaveLoadRoundTrip_Hostnames(t *testing.T) {
t.Parallel() t.Parallel()
+36
Vedi File
@@ -202,6 +202,42 @@ func TestCNAMEWhoseNameserversAllFailedKeepsPrevious(t *testing.T) {
} }
} }
// TestCNAMEWhoseAddressQueryFailedKeepsPrevious checks a name whose
// nameserver answered, but whose query for A, AAAA or CNAME failed with
// nothing kept for it. That is not an answer with no address: the
// addresses the previous check saved from following its CNAME are kept,
// and nothing is looked up, the watcher having no resolver.
func TestCNAMEWhoseAddressQueryFailedKeepsPrevious(t *testing.T) {
t.Parallel()
for _, rtype := range []string{"A", "AAAA", "CNAME"} {
t.Run(rtype, func(t *testing.T) {
t.Parallel()
w := watcher.NewForTest(nil, nil, nil, nil, nil, nil)
current := saved(map[string]*state.NameserverRecordState{
nsA: {
Records: map[string][]string{},
FailedTypes: []string{rtype},
UnknownTypes: []string{rtype},
Status: "ok",
},
})
prev := cnameState(oldIP)
w.ResolveCNAMEAddresses(t.Context(), host, current, prev)
if !slices.Equal(current.CNAMEAddresses, prev.CNAMEAddresses) {
t.Errorf(
"saved %v, want %v",
current.CNAMEAddresses, prev.CNAMEAddresses,
)
}
})
}
}
// cnameTo builds the records of a nameserver that answered with a CNAME // cnameTo builds the records of a nameserver that answered with a CNAME
// to target and no address. // to target and no address.
func cnameTo(target string) map[string][]string { func cnameTo(target string) map[string][]string {
+2 -1
Vedi File
@@ -109,7 +109,8 @@ func (w *Watcher) RunTLSChecks(ctx context.Context) {
// BuildHostnameState exports buildHostnameState for testing. // BuildHostnameState exports buildHostnameState for testing.
func BuildHostnameState( func BuildHostnameState(
results map[string]*resolver.NameserverResponse, results map[string]*resolver.NameserverResponse,
prev *state.HostnameState,
now time.Time, now time.Time,
) *state.HostnameState { ) *state.HostnameState {
return buildHostnameState(results, now) return buildHostnameState(results, prev, now)
} }
+364
Vedi File
@@ -0,0 +1,364 @@
package watcher_test
import (
"maps"
"slices"
"testing"
"time"
"sneak.berlin/go/dnswatcher/internal/resolver"
"sneak.berlin/go/dnswatcher/internal/state"
"sneak.berlin/go/dnswatcher/internal/watcher"
)
const (
// txt is the record type whose query fails in these tests.
txt = "TXT"
spf1 = "v=spf1 -all"
spf2 = "v=spf1 include:example.net -all"
)
// response is a nameserver's response with these records, whose queries
// for failedTypes failed.
func response(
records map[string][]string,
failedTypes ...string,
) *resolver.NameserverResponse {
return &resolver.NameserverResponse{
Records: records,
FailedTypes: failedTypes,
Status: resolver.StatusOK,
}
}
// savedChecks saves the state of each check in turn from the
// nameservers' responses, each from the state the check before saved.
func savedChecks(
checks ...map[string]*resolver.NameserverResponse,
) []*state.HostnameState {
states := make([]*state.HostnameState, 0, len(checks))
var prev *state.HostnameState
for _, results := range checks {
prev = watcher.BuildHostnameState(results, prev, time.Now())
states = append(states, prev)
}
return states
}
// TestFailedTypeKeepsPreviousRecords saves a check in which nsA's query
// for TXT failed, after previous checks of several kinds. TXT is always
// saved in FailedTypes, and in UnknownTypes when there was nothing to
// keep.
func TestFailedTypeKeepsPreviousRecords(t *testing.T) {
t.Parallel()
aOnly := map[string][]string{"A": {ip1}}
withTXT := map[string][]string{"A": {ip1}, txt: {spf1}}
txtKept := &state.NameserverRecordState{
Records: withTXT, FailedTypes: []string{txt}, Status: "ok",
}
txtNotKnown := &state.NameserverRecordState{
Records: aOnly,
FailedTypes: []string{txt},
UnknownTypes: []string{txt},
Status: "ok",
}
tests := []struct {
name string
prev *state.HostnameState
wantRecords map[string][]string
wantUnknown []string
}{
{
"previous TXT records are kept",
saved(map[string]*state.NameserverRecordState{nsA: answered(withTXT)}),
withTXT, nil,
},
{
"previous check had no TXT records",
saved(map[string]*state.NameserverRecordState{nsA: answered(aOnly)}),
aOnly, nil,
},
{
"TXT failed on the previous check, which kept its records",
saved(map[string]*state.NameserverRecordState{nsA: txtKept}),
withTXT, nil,
},
{"first check", nil, aOnly, []string{txt}},
{
"nameserver new on this check",
saved(map[string]*state.NameserverRecordState{nsB: answered(withTXT)}),
aOnly, []string{txt},
},
{
"nameserver failed on the previous check",
saved(map[string]*state.NameserverRecordState{nsA: failed()}),
aOnly, []string{txt},
},
{
"TXT failed on the previous check with nothing to keep",
saved(map[string]*state.NameserverRecordState{nsA: txtNotKnown}),
aOnly, []string{txt},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
hs := watcher.BuildHostnameState(
map[string]*resolver.NameserverResponse{
nsA: response(map[string][]string{"A": {ip1}}, txt),
},
tt.prev, time.Now(),
)
got := hs.RecordsByNameserver[nsA]
if got.Status != "ok" ||
!maps.EqualFunc(got.Records, tt.wantRecords, slices.Equal) ||
!slices.Equal(got.FailedTypes, []string{txt}) ||
!slices.Equal(got.UnknownTypes, tt.wantUnknown) {
t.Errorf(
"saved status %q, records %v, failed types %v, "+
"unknown types %v; want ok, %v, [%s], %v",
got.Status, got.Records, got.FailedTypes,
got.UnknownTypes, tt.wantRecords, txt, tt.wantUnknown,
)
}
})
}
}
// TestFailedTypeAlerts saves the checks of each case in turn from the
// nameservers' responses, the first being the state loaded at startup,
// and counts the alerts sent. nsB's TXT query fails on one check, and
// nothing changes.
func TestFailedTypeAlerts(t *testing.T) {
t.Parallel()
records := map[string][]string{"A": {ip1}, txt: {spf1}}
aOnly := map[string][]string{"A": {ip1}}
bothAnswer := map[string]*resolver.NameserverResponse{
nsA: response(records), nsB: response(records),
}
bTXTFails := map[string]*resolver.NameserverResponse{
nsA: response(records), nsB: response(aOnly, txt),
}
onlyA := map[string]*resolver.NameserverResponse{
nsA: response(records),
}
bFails := map[string]*resolver.NameserverResponse{
nsA: response(records),
nsB: {
Records: map[string][]string{},
Status: resolver.StatusTimeout,
Error: "all queries timed out",
},
}
tests := []struct {
name string
checks []map[string]*resolver.NameserverResponse
want alertCounts
}{
{
"type failing at one nameserver alerts nothing, nor its next answer",
[]map[string]*resolver.NameserverResponse{
bothAnswer, bTXTFails, bothAnswer,
},
alertCounts{},
},
{
"type failing on the first check alerts nothing on the next",
[]map[string]*resolver.NameserverResponse{bTXTFails, bothAnswer},
alertCounts{},
},
{
"type failing at a nameserver new on that check alerts nothing",
[]map[string]*resolver.NameserverResponse{
onlyA, bTXTFails, bothAnswer,
},
alertCounts{},
},
{
"type failing at a recovering nameserver alerts the recovery",
[]map[string]*resolver.NameserverResponse{
bFails, bTXTFails, bothAnswer,
},
alertCounts{recoveries: 1},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
states := savedChecks(tt.checks...)
got := countAlerts(t, states[0], states[1:])
if got != tt.want {
t.Errorf("sent %+v, want %+v", got, tt.want)
}
})
}
}
// TestFailedTypeComparedOnceItAnswers saves the checks of each case in
// turn as TestFailedTypeAlerts does. nsB's TXT query fails on one check,
// and the TXT record changes: the change is sent as a Record Change for
// each nameserver on the check where it answers it, and an Inconsistency
// only when nsB still answers the old record.
func TestFailedTypeComparedOnceItAnswers(t *testing.T) {
t.Parallel()
records := map[string][]string{"A": {ip1}, txt: {spf1}}
changed := map[string][]string{"A": {ip1}, txt: {spf2}}
aOnly := map[string][]string{"A": {ip1}}
bothAnswer := map[string]*resolver.NameserverResponse{
nsA: response(records), nsB: response(records),
}
bTXTFails := map[string]*resolver.NameserverResponse{
nsA: response(records), nsB: response(aOnly, txt),
}
bothChange := map[string]*resolver.NameserverResponse{
nsA: response(changed), nsB: response(changed),
}
aChangesBTXTFails := map[string]*resolver.NameserverResponse{
nsA: response(changed), nsB: response(aOnly, txt),
}
bStillOld := map[string]*resolver.NameserverResponse{
nsA: response(changed), nsB: response(records),
}
tests := []struct {
name string
checks []map[string]*resolver.NameserverResponse
want alertCounts
}{
{
"change made while the type failed",
[]map[string]*resolver.NameserverResponse{
bothAnswer, bTXTFails, bothChange,
},
alertCounts{recordChanges: 2},
},
{
"change seen at one nameserver while the other's type failed",
[]map[string]*resolver.NameserverResponse{
bothAnswer, aChangesBTXTFails, bothChange,
},
alertCounts{recordChanges: 2},
},
{
"old record answered after the type failed",
[]map[string]*resolver.NameserverResponse{
bothAnswer, aChangesBTXTFails, bStillOld,
},
alertCounts{recordChanges: 1, inconsistencies: 1},
},
{
"change after the type failed on the first check and answered",
[]map[string]*resolver.NameserverResponse{
bTXTFails, bothAnswer, bothChange,
},
alertCounts{recordChanges: 2},
},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
states := savedChecks(tt.checks...)
got := countAlerts(t, states[0], states[1:])
if got != tt.want {
t.Errorf("sent %+v, want %+v", got, tt.want)
}
})
}
}
// TestFailedTypeLeftOutOfMessages checks that a Record Change and an
// Inconsistency name only the record types they compared. nsB's TXT
// records are not known on the first check, and on the second either
// answered or still not known; nsB's A record changes, so both alerts
// are sent and name the A record alone.
func TestFailedTypeLeftOutOfMessages(t *testing.T) {
t.Parallel()
withTXT := map[string][]string{"A": {ip1}, txt: {spf1}}
txtNotKnown := func(address string) *state.NameserverRecordState {
return &state.NameserverRecordState{
Records: map[string][]string{"A": {address}},
FailedTypes: []string{txt},
UnknownTypes: []string{txt},
Status: "ok",
}
}
before := saved(map[string]*state.NameserverRecordState{
nsA: answered(withTXT), nsB: txtNotKnown(ip1),
})
tests := []struct {
name string
after *state.HostnameState
}{
{
"TXT answers",
saved(map[string]*state.NameserverRecordState{
nsA: answered(withTXT),
nsB: answered(map[string][]string{"A": {ip2}, txt: {spf1}}),
}),
},
{
"TXT still not known",
saved(map[string]*state.NameserverRecordState{
nsA: answered(withTXT), nsB: txtNotKnown(ip2),
}),
},
}
want := map[string]string{
"Record Change: " + host: "Hostname: " + host +
"\nNameserver: " + nsB + "\nType: A\nOld: " + ip1 + "\nNew: " + ip2,
"Inconsistency: " + host: "Hostname: " + host +
"\nType: A\n" + nsA + ": " + ip1 + "\n" + nsB + ": " + ip2,
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
t.Parallel()
// The hostname change detection uses only the notifier.
notifier := &mockNotifier{}
w := watcher.NewForTest(nil, nil, nil, nil, nil, notifier)
w.DetectHostnameChanges(t.Context(), host, before, tt.after)
notifications := notifier.getNotifications()
if len(notifications) != len(want) {
t.Fatalf(
"sent %d notifications, want %d: %v",
len(notifications), len(want), notifications,
)
}
for _, n := range notifications {
if n.Message != want[n.Title] {
t.Errorf(
"%s message:\n%s\nwant:\n%s",
n.Title, n.Message, want[n.Title],
)
}
}
})
}
}
+55
Vedi File
@@ -183,3 +183,58 @@ func TestInconsistencyAlert(t *testing.T) {
}) })
} }
} }
// TestFirstCheckAfterRepeatedValuesLoaded saves a state file holding a
// hostname's CNAME once for every record type asked for, as checks did
// before each value was stored once, and two addresses each repeated,
// and loads it. A check that then finds each value once at each
// nameserver must notify nothing.
func TestFirstCheckAfterRepeatedValuesLoaded(t *testing.T) {
t.Parallel()
const (
cnameType = "CNAME"
cname = "c.example.net."
)
cfg := defaultTestConfig(t)
repeated := map[string][]string{
"A": {ip2, ip1, ip2, ip1},
cnameType: {cname, cname, cname, cname, cname, cname, cname, cname},
}
once := map[string][]string{"A": {ip1, ip2}, cnameType: {cname}}
saved := newTestDeps(t, cfg).state
saved.SetHostnameState(host, hostnameState(map[string]map[string][]string{
nsA: repeated, nsB: repeated,
}))
err := saved.Save()
if err != nil {
t.Fatalf("saving the state file: %v", err)
}
deps := newTestDeps(t, cfg)
err = deps.state.Load()
if err != nil {
t.Fatalf("loading the state file: %v", err)
}
prev, ok := deps.state.GetHostnameState(host)
if !ok {
t.Fatal("the state file has no state for " + host)
}
current := hostnameState(map[string]map[string][]string{
nsA: once, nsB: once,
})
// The hostname change detection uses only the notifier.
w := watcher.NewForTest(nil, nil, nil, nil, nil, deps.notifier)
w.DetectHostnameChanges(t.Context(), host, prev, current)
if got := deps.notifier.getNotifications(); len(got) != 0 {
t.Errorf("sent %v, want no notification", got)
}
}
+3 -3
Vedi File
@@ -201,7 +201,7 @@ func TestNameserverThatNeverAnswers(t *testing.T) {
} }
hs := watcher.BuildHostnameState( hs := watcher.BuildHostnameState(
map[string]*resolver.NameserverResponse{nsA: resp}, time.Now(), map[string]*resolver.NameserverResponse{nsA: resp}, nil, time.Now(),
) )
got := hs.RecordsByNameserver[nsA] got := hs.RecordsByNameserver[nsA]
@@ -256,7 +256,7 @@ func TestNameserverThatAnswersNXDOMAIN(t *testing.T) {
} }
hs := watcher.BuildHostnameState( hs := watcher.BuildHostnameState(
map[string]*resolver.NameserverResponse{ns: resp}, time.Now(), map[string]*resolver.NameserverResponse{ns: resp}, nil, time.Now(),
) )
got := hs.RecordsByNameserver[ns] got := hs.RecordsByNameserver[ns]
@@ -320,7 +320,7 @@ func TestNameserverThatRefuses(t *testing.T) {
} }
hs := watcher.BuildHostnameState( hs := watcher.BuildHostnameState(
map[string]*resolver.NameserverResponse{ns: resp}, time.Now(), map[string]*resolver.NameserverResponse{ns: resp}, nil, time.Now(),
) )
got := hs.RecordsByNameserver[ns] got := hs.RecordsByNameserver[ns]
+100 -19
Vedi File
@@ -5,6 +5,7 @@ import (
"errors" "errors"
"fmt" "fmt"
"log/slog" "log/slog"
"maps"
"slices" "slices"
"sort" "sort"
"strings" "strings"
@@ -400,8 +401,10 @@ func (w *Watcher) checkHostname(
return return
} }
prev, _ := w.state.GetHostnameState(hostname)
w.updateHostnameState( w.updateHostnameState(
ctx, hostname, buildHostnameState(results, time.Now().UTC()), ctx, hostname, buildHostnameState(results, prev, time.Now().UTC()),
) )
} }
@@ -431,8 +434,9 @@ func (w *Watcher) updateHostnameState(
// the addresses found for all of them are saved, so nameservers that // the addresses found for all of them are saved, so nameservers that
// disagree on the target do not change the result from check to check. // disagree on the target do not change the result from check to check.
// The addresses saved in prev, which may be nil, are kept when none of // The addresses saved in prev, which may be nil, are kept when none of
// the name's nameservers answered, and when a target cannot be // the name's nameservers answered its queries for A, AAAA and CNAME,
// followed, as when no nameserver of a zone in its chain answers. // and when a target cannot be followed, as when no nameserver of a zone
// in its chain answers.
func (w *Watcher) resolveCNAMEAddresses( func (w *Watcher) resolveCNAMEAddresses(
ctx context.Context, ctx context.Context,
hostname string, hostname string,
@@ -450,7 +454,10 @@ func (w *Watcher) resolveCNAMEAddresses(
targets := make(map[string]bool) targets := make(map[string]bool)
for _, nsState := range current.RecordsByNameserver { for _, nsState := range current.RecordsByNameserver {
if nsState.Status != statusOK { if nsState.Status != statusOK ||
slices.Contains(nsState.FailedTypes, "A") ||
slices.Contains(nsState.FailedTypes, "AAAA") ||
slices.Contains(nsState.FailedTypes, "CNAME") {
continue continue
} }
@@ -496,11 +503,13 @@ func (w *Watcher) resolveCNAMEAddresses(
} }
// buildHostnameState saves each nameserver's response. A nameserver // buildHostnameState saves each nameserver's response. A nameserver
// that answered, even with NXDOMAIN or no records, is saved as ok; one // that answered, even with NXDOMAIN or no records, is saved as ok, with
// that timed out or failed is saved as error with the reason, and its // the record types whose query failed; one that timed out or failed is
// empty record set is not an answer. // saved as error with the reason, and its empty record set is not an
// answer. prev is the hostname's state from the previous check, or nil.
func buildHostnameState( func buildHostnameState(
results map[string]*resolver.NameserverResponse, results map[string]*resolver.NameserverResponse,
prev *state.HostnameState,
now time.Time, now time.Time,
) *state.HostnameState { ) *state.HostnameState {
hs := &state.HostnameState{ hs := &state.HostnameState{
@@ -512,7 +521,7 @@ func buildHostnameState(
for ns, resp := range results { for ns, resp := range results {
nsState := &state.NameserverRecordState{ nsState := &state.NameserverRecordState{
Records: resp.Records, Records: maps.Clone(resp.Records),
Status: statusOK, Status: statusOK,
LastChecked: now, LastChecked: now,
} }
@@ -521,6 +530,15 @@ func buildHostnameState(
resp.Status == resolver.StatusError { resp.Status == resolver.StatusError {
nsState.Status = statusError nsState.Status = statusError
nsState.Error = resp.Error nsState.Error = resp.Error
} else {
nsState.FailedTypes = resp.FailedTypes
var prevNS *state.NameserverRecordState
if prev != nil {
prevNS = prev.RecordsByNameserver[ns]
}
keepFailedTypes(nsState, prevNS)
} }
hs.RecordsByNameserver[ns] = nsState hs.RecordsByNameserver[ns] = nsState
@@ -529,6 +547,27 @@ func buildHostnameState(
return hs return hs
} }
// keepFailedTypes copies into nsState, for each record type in its
// FailedTypes, the records prevNS, the nameserver's state from the
// previous check, holds for that type, which may be none. When prevNS
// does not know them either, because the nameserver was new or failing
// then or the type was in its UnknownTypes, the type goes in
// nsState.UnknownTypes instead.
func keepFailedTypes(nsState, prevNS *state.NameserverRecordState) {
for _, rtype := range nsState.FailedTypes {
if prevNS == nil || prevNS.Status != statusOK ||
slices.Contains(prevNS.UnknownTypes, rtype) {
nsState.UnknownTypes = append(nsState.UnknownTypes, rtype)
continue
}
if records, ok := prevNS.Records[rtype]; ok {
nsState.Records[rtype] = records
}
}
}
func (w *Watcher) detectHostnameChanges( func (w *Watcher) detectHostnameChanges(
ctx context.Context, ctx context.Context,
hostname string, hostname string,
@@ -573,7 +612,10 @@ func (w *Watcher) detectCNAMEAddressChanges(
// detectRecordChanges compares each nameserver's records with those of // detectRecordChanges compares each nameserver's records with those of
// the previous check. Only answers are compared: a nameserver that // the previous check. Only answers are compared: a nameserver that
// failed on either check has no records to compare. // failed on either check has no records to compare. The records kept
// for a record type whose query failed are compared too, but not those
// of a type in UnknownTypes on either check, which the message leaves
// out as well.
func (w *Watcher) detectRecordChanges( func (w *Watcher) detectRecordChanges(
ctx context.Context, ctx context.Context,
hostname string, hostname string,
@@ -585,7 +627,11 @@ func (w *Watcher) detectRecordChanges(
continue continue
} }
if recordsEqual(prevNS.Records, cur.Records) { unknown := slices.Concat(prevNS.UnknownTypes, cur.UnknownTypes)
oldRecords := withoutTypes(prevNS.Records, unknown)
newRecords := withoutTypes(cur.Records, unknown)
if recordsEqual(oldRecords, newRecords) {
continue continue
} }
@@ -593,8 +639,8 @@ func (w *Watcher) detectRecordChanges(
"Hostname: %s\nNameserver: %s\n%s", "Hostname: %s\nNameserver: %s\n%s",
hostname, ns, hostname, ns,
recordDifferences( recordDifferences(
"Old", prevNS.Records, "Old", oldRecords,
"New", cur.Records, "New", newRecords,
), ),
) )
@@ -681,13 +727,19 @@ func (w *Watcher) detectInconsistencies(
) { ) {
for _, pair := range newlyDisagreeingPairs(prev, current) { for _, pair := range newlyDisagreeingPairs(prev, current) {
ns1, ns2 := pair[0], pair[1] ns1, ns2 := pair[0], pair[1]
state1 := current.RecordsByNameserver[ns1]
state2 := current.RecordsByNameserver[ns2]
// The record types left out of the comparison are left out of
// the message too.
failed := slices.Concat(state1.FailedTypes, state2.FailedTypes)
msg := fmt.Sprintf( msg := fmt.Sprintf(
"Hostname: %s\n%s", "Hostname: %s\n%s",
hostname, hostname,
recordDifferences( recordDifferences(
ns1, current.RecordsByNameserver[ns1].Records, ns1, withoutTypes(state1.Records, failed),
ns2, current.RecordsByNameserver[ns2].Records, ns2, withoutTypes(state2.Records, failed),
), ),
) )
@@ -705,7 +757,9 @@ func (w *Watcher) detectInconsistencies(
// except pairs where both nameservers answered in prev and already // except pairs where both nameservers answered in prev and already
// differed there. A nameserver missing from prev, or that failed there, // differed there. A nameserver missing from prev, or that failed there,
// is paired with every nameserver it differs from. A nameserver that // is paired with every nameserver it differs from. A nameserver that
// failed in current has no records to compare and is in no pair. // failed in current has no records to compare and is in no pair. In
// both checks, a record type whose query failed at either nameserver is
// not compared.
func newlyDisagreeingPairs( func newlyDisagreeingPairs(
prev, current *state.HostnameState, prev, current *state.HostnameState,
) [][2]string { ) [][2]string {
@@ -722,9 +776,9 @@ func newlyDisagreeingPairs(
for i, ns1 := range nameservers { for i, ns1 := range nameservers {
for _, ns2 := range nameservers[i+1:] { for _, ns2 := range nameservers[i+1:] {
if recordsEqual( if nameserversAgree(
current.RecordsByNameserver[ns1].Records, current.RecordsByNameserver[ns1],
current.RecordsByNameserver[ns2].Records, current.RecordsByNameserver[ns2],
) { ) {
continue continue
} }
@@ -734,7 +788,7 @@ func newlyDisagreeingPairs(
if ok1 && ok2 && if ok1 && ok2 &&
prev1.Status == statusOK && prev2.Status == statusOK && prev1.Status == statusOK && prev2.Status == statusOK &&
!recordsEqual(prev1.Records, prev2.Records) { !nameserversAgree(prev1, prev2) {
continue continue
} }
@@ -1221,6 +1275,33 @@ func toSet(items []string) map[string]bool {
return set return set
} }
// nameserversAgree reports whether two nameservers' states from the same
// check hold the same records, leaving out the record types either lists
// in FailedTypes: the records held for those are kept from an earlier
// check, or not known.
func nameserversAgree(a, b *state.NameserverRecordState) bool {
failed := slices.Concat(a.FailedTypes, b.FailedTypes)
return recordsEqual(
withoutTypes(a.Records, failed), withoutTypes(b.Records, failed),
)
}
// withoutTypes returns a copy of records without the record types in
// types.
func withoutTypes(
records map[string][]string,
types []string,
) map[string][]string {
records = maps.Clone(records)
for _, rtype := range types {
delete(records, rtype)
}
return records
}
func recordsEqual( func recordsEqual(
a, b map[string][]string, a, b map[string][]string,
) bool { ) bool {