notify: a failed Mattermost delivery's error names Mattermost (closes #227)

Mattermost is sent by the Slack sender, which wrapped every HTTP error
status in ErrSlackFailed, so a Mattermost endpoint answering 503 was
logged as "slack notification failed". The sender now takes the error
to wrap: the Slack endpoint passes ErrSlackFailed and the Mattermost
endpoint passes ErrMattermostFailed, which was defined but unused.

A new delivery test sets both endpoints to a stand-in server answering
503 and checks the error logged for each names its own endpoint.

Model: opus-5-5

README.md

@@ -121,14 +121,26 @@ notification endpoint set, changes show only on the dashboard; see
       failed on it, and answers differently is reported on the check where it
       answers. If a pair agrees again and later disagrees, the alert is sent
       again.
+    - **CNAME address change**: The addresses at the end of a name's CNAME chain
+      differ from those of the previous check. They are found when its
+      nameservers answer with a CNAME and no address; a name that answers with
+      an address has none. A change from or to no addresses is sent too, as when
+      a name moves between A records and a CNAME. Nothing is sent when the
+      previous addresses were kept because a chain could not be followed or none
+      of the name's nameservers answered. The first check after loading a state
+      file without `cnameAddresses` sends nothing: it saves the addresses it
+      finds for the next check to compare.
 
 ### TCP Port Monitoring
 
 - For every configured domain and hostname, constructs a deduplicated list of
   the IPv4 and IPv6 addresses in the A and AAAA records its authoritative
-  nameservers returned. A CNAME is not followed: a name whose CNAME points into
+  nameservers returned. When they returned a CNAME and no address, the CNAME
-  another zone usually has no addresses here, so its ports and certificate are
+  chain is followed and the addresses at its end are used, and a change in those
-  not checked.
+  is notified as a CNAME address change. When the nameservers gave different
+  CNAME targets, each is followed and the addresses of all are used. When a
+  chain cannot be followed, or none of the name's nameservers answered, the
+  addresses the last check found at its end are used.
 - Checks TCP connectivity on ports **80** and **443** for each IP address.
 - Every **1 hour** by default, re-checks all ports.
 - Any change in port availability triggers a notification:
@@ -176,6 +188,8 @@ includes:
 - **DNS NS changes**: Which domain, which nameservers were added/removed.
 - **NS address changes**: Which domain, which nameserver, its old and new
   addresses.
+- **CNAME address changes**: Which hostname, the old and new addresses at the
+  end of its CNAME chain.
 - **NS query failures**: Which nameserver failed, error type (timeout, SERVFAIL,
   REFUSED, network error), which hostname/domain affected.
 - **NS recoveries**: Which nameserver recovered, which hostname/domain.
@@ -420,9 +434,11 @@ This approach ensures:
 - Ability to detect split-horizon or inconsistent responses across authoritative
   servers.
 
-CNAME chains are followed (with a depth limit to prevent loops) only to find the
+A watched name's records are stored as its nameservers return them, CNAME
-addresses of nameservers. A watched name's records are stored as its nameservers
+included. When they return a CNAME and no address, the chain of every CNAME
-return them, CNAME included, without following it.
+target they gave is followed (with a depth limit to prevent loops) to the A and
+AAAA records at its end, and the port and TLS checks use those addresses.
+Nameservers' addresses are also found by following CNAME chains.
 
 Sending a notification or a Sentry report is the one use of the system's
 resolver: the HTTP client looks up the webhook's or Sentry's host name with it.
@@ -469,6 +485,7 @@ merged view, to enable inconsistency detection.
                     "lastChecked": "2026-02-19T12:00:00Z"
                 }
             },
+            "cnameAddresses": [],
             "lastChecked": "2026-02-19T12:00:00Z"
         }
     },
@@ -515,6 +532,14 @@ certificate entry whose TLS connection or handshake failed likewise has status
 resolves to. A state file without it loads, and the next check fills it in
 without a notification.
 
+`cnameAddresses` lists the sorted addresses at the end of the chain of every
+CNAME target a hostname's nameservers gave, found when they answered with a
+CNAME and no address; it is empty when they answered with an address. When a
+chain cannot be followed, or none of the name's nameservers answered, the
+previous check's list is kept, or `null` when no earlier check saved one. A
+state file without it loads, and the first check after that saves it without a
+notification.
+
 A port entry in the older format, with one `hostname` instead of the `hostnames`
 list, loads as a list of that one name.
 
@@ -672,7 +697,9 @@ docker run -d \
     - Port and TLS checks use the IP addresses found by the DNS phase that
       immediately precedes them. When that phase cannot find a name's
       nameservers at all, the addresses an earlier check saved for the name are
-      used.
+      used. When it cannot follow a name's CNAME chain, or none of the name's
+      nameservers answered, the addresses an earlier check found at the end of
+      the chain are used.
 4. **On change detection**: Send notifications to all configured endpoints,
    update in-memory state, persist to disk.
 5. **Shutdown**: The watcher stops checking and saves the final state to disk,

TODO.md

@@ -19,6 +19,12 @@ trial run of the finished image: https://git.eeqj.de/sneak/dnswatcher/issues/149
 
 # Completed Steps
 
+- 2026-10-02: a Mattermost webhook that answers an HTTP error is logged as
+  `mattermost notification failed`, not as a Slack failure (closes #227).
+- 2026-10-02: durations in the log are written as text such as `2m0s`, not as a
+  bare count of nanoseconds (closes #228).
+- 2026-10-02: a watched name whose nameservers answer with a CNAME and no
+  address gets port and TLS checks at the end of its CNAME chain (closes #203).
 - 2026-10-02: a resolver test that reads one record type from a nameserver's
   answer asks again when that type is missing from it (closes #218).
 - 2026-10-02: a plain `docker build .` of a clone stamps its tag or short

internal/notify/delivery_test.go

@@ -6,9 +6,11 @@ import (
 	"encoding/json"
 	"errors"
 	"io"
+	"maps"
 	"net/http"
 	"net/http/httptest"
 	"net/url"
+	"strings"
 	"sync"
 	"testing"
 	"time"
@@ -413,7 +415,8 @@ func sendSlackInfo(
 	svc *notify.Service, target *url.URL,
 ) error {
 	return svc.SendSlack(
-		context.Background(), target, "t", "m", prioInfo,
+		context.Background(), target, notify.ErrSlackFailed,
+		"t", "m", prioInfo,
 	)
 }
 
@@ -506,6 +509,7 @@ func TestSendSlackPayloadFields(t *testing.T) {
 	err := svc.SendSlack(
 		context.Background(),
 		webhookURL,
+		notify.ErrSlackFailed,
 		"Alert Title",
 		"Alert body text",
 		"warning",
@@ -608,7 +612,8 @@ func TestSendSlackAllColors(t *testing.T) {
 
 			err := svc.SendSlack(
 				context.Background(),
-				webhookURL, "t", "m", tc.priority,
+				webhookURL, notify.ErrSlackFailed,
+				"t", "m", tc.priority,
 			)
 			if err != nil {
 				t.Fatalf("SendSlack error: %v", err)
@@ -659,7 +664,8 @@ func TestSendSlackNetworkError(t *testing.T) {
 	)
 
 	err := svc.SendSlack(
-		context.Background(), webhookURL, "t", "m", "info",
+		context.Background(), webhookURL, notify.ErrSlackFailed,
+		"t", "m", "info",
 	)
 	if err == nil {
 		t.Fatal("expected error for network failure")
@@ -1028,6 +1034,66 @@ func TestSendNotificationMattermostError(t *testing.T) {
 	)
 }
 
+// TestSendNotificationErrorNamesEndpoint verifies that, with both
+// Slack and Mattermost set, a failed delivery's logged error names
+// the endpoint that failed. Both are sent by the Slack sender.
+func TestSendNotificationErrorNamesEndpoint(t *testing.T) {
+	t.Parallel()
+
+	srv := httptest.NewServer(
+		http.HandlerFunc(
+			func(w http.ResponseWriter, _ *http.Request) {
+				w.WriteHeader(http.StatusServiceUnavailable)
+			}),
+	)
+	defer srv.Close()
+
+	target, _ := url.Parse(srv.URL)
+
+	svc, logs := newLoggingService(http.DefaultTransport)
+	svc.SetSlackWebhookURL(target)
+	svc.SetMattermostWebhookURL(target)
+	svc.SetSleepFunc(instantSleep)
+	svc.SetRetryConfig(notify.RetryConfig{
+		MaxRetries: 1,
+		BaseDelay:  time.Millisecond,
+		MaxDelay:   time.Millisecond,
+	})
+
+	svc.SendNotification(
+		context.Background(), "t", "m", prioError,
+	)
+
+	waitForCondition(t, func() bool {
+		return svc.OutstandingDeliveries() == 0
+	})
+
+	got := map[string]string{}
+
+	for line := range strings.Lines(logs.String()) {
+		var record struct {
+			Msg      string `json:"msg"`
+			Endpoint string `json:"endpoint"`
+			Error    string `json:"error"`
+		}
+
+		_ = json.Unmarshal([]byte(line), &record)
+
+		if record.Msg == "failed to send notification after retries" {
+			got[record.Endpoint] = record.Error
+		}
+	}
+
+	want := map[string]string{
+		"slack":      "slack notification failed: status 503",
+		"mattermost": "mattermost notification failed: status 503",
+	}
+
+	if !maps.Equal(got, want) {
+		t.Errorf("logged errors = %v, want %v", got, want)
+	}
+}
+
 // ── SlackPayload JSON marshaling ──────────────────────────
 
 func TestSlackPayloadJSON(t *testing.T) {

internal/notify/export_test.go

@@ -85,10 +85,11 @@ func (svc *Service) SendNtfy(
 func (svc *Service) SendSlack(
 	ctx context.Context,
 	webhookURL *url.URL,
+	failed error,
 	title, message, priority string,
 ) error {
 	return svc.sendSlack(
-		ctx, webhookURL, title, message, priority,
+		ctx, webhookURL, failed, title, message, priority,
 	)
 }
 

internal/notify/notify.go

@@ -277,7 +277,8 @@ func (svc *Service) dispatchSlack(
 
 	svc.dispatch(ctx, "slack", func(c context.Context) error {
 		return svc.sendSlack(
-			c, svc.slackWebhookURL, title, message, priority,
+			c, svc.slackWebhookURL, ErrSlackFailed,
+			title, message, priority,
 		)
 	})
 }
@@ -294,7 +295,7 @@ func (svc *Service) dispatchMattermost(
 		ctx, "mattermost",
 		func(c context.Context) error {
 			return svc.sendSlack(
-				c, svc.mattermostWebhookURL,
+				c, svc.mattermostWebhookURL, ErrMattermostFailed,
 				title, message, priority,
 			)
 		},
@@ -370,9 +371,14 @@ type SlackAttachment struct {
 	Text  string `json:"text"`
 }
 
+// sendSlack posts to a Slack or Mattermost incoming webhook, which
+// take the same payload. An HTTP error status is returned wrapped in
+// failed, ErrSlackFailed or ErrMattermostFailed, so the error names
+// the endpoint.
 func (svc *Service) sendSlack(
 	ctx context.Context,
 	webhookURL *url.URL,
+	failed error,
 	title, message, priority string,
 ) error {
 	ctx, cancel := context.WithTimeout(
@@ -420,7 +426,7 @@ func (svc *Service) sendSlack(
 	if resp.StatusCode >= httpStatusClientError {
 		return fmt.Errorf(
 			"%w: status %d",
-			ErrSlackFailed, resp.StatusCode,
+			failed, resp.StatusCode,
 		)
 	}
 

internal/notify/retry.go

@@ -115,7 +115,9 @@ func (svc *Service) deliverWithRetry(
 			"endpoint", endpoint,
 			"attempt", attempt+1,
 			"maxAttempts", cfg.MaxRetries+1,
-			"retryIn", delay,
+			// As text: the JSON log writes a time.Duration as
+			// bare nanoseconds.
+			"retryIn", delay.String(),
 			"error", lastErr,
 		)
 

internal/notify/retry_test.go

@@ -2,6 +2,7 @@ package notify_test
 
 import (
 	"context"
+	"encoding/json"
 	"errors"
 	"net/http"
 	"net/http/httptest"
@@ -189,6 +190,50 @@ func TestDeliverWithRetryExhaustsAttempts(t *testing.T) {
 	}
 }
 
+// TestDeliverWithRetryLogsRetryInAsText checks that the wait
+// before a retry is logged as text such as "1.02s", not as a
+// count of nanoseconds.
+func TestDeliverWithRetryLogsRetryInAsText(t *testing.T) {
+	t.Parallel()
+
+	svc, logs := newLoggingService(http.DefaultTransport)
+	svc.SetRetryConfig(notify.RetryConfig{
+		MaxRetries: 1,
+		BaseDelay:  time.Second,
+		MaxDelay:   time.Second,
+	})
+
+	var waited time.Duration
+
+	svc.SetSleepFunc(func(d time.Duration) <-chan time.Time {
+		waited = d
+
+		return instantSleep(d)
+	})
+
+	_ = svc.DeliverWithRetry(
+		context.Background(), "test",
+		func(_ context.Context) error {
+			return errFail
+		},
+	)
+
+	// With one retry, only the first failure is logged.
+	var record map[string]any
+
+	err := json.Unmarshal([]byte(logs.String()), &record)
+	if err != nil {
+		t.Fatalf("log is not one JSON record: %v\n%s", err, logs)
+	}
+
+	if record["retryIn"] != waited.String() {
+		t.Errorf(
+			"retryIn logged as %v, want %q",
+			record["retryIn"], waited.String(),
+		)
+	}
+}
+
 func TestDeliverWithRetryRespectsContextCancellation(
 	t *testing.T,
 ) {

internal/portcheck/portcheck.go

@@ -193,7 +193,9 @@ func (c *Checker) checkConnection(
 	c.log.Debug(
 		"port check succeeded",
 		"target", target,
-		"latency", latency,
+		// As text: the JSON log writes a time.Duration as bare
+		// nanoseconds.
+		"latency", latency.String(),
 	)
 
 	return &PortResult{

internal/state/state.go

@@ -53,8 +53,13 @@ type NameserverRecordState struct {
 }
 
 // HostnameState holds per-nameserver monitoring state for a hostname.
+// CNAMEAddresses holds the sorted addresses at the end of the name's
+// CNAME chain, found when its nameservers answered with a CNAME and no
+// address; it is empty otherwise. It is nil when they are not known: a
+// state file written before it existed loads with it nil.
 type HostnameState struct {
 	RecordsByNameserver map[string]*NameserverRecordState `json:"recordsByNameserver"`
+	CNAMEAddresses      []string                          `json:"cnameAddresses"`
 	LastChecked         time.Time                         `json:"lastChecked"`
 }
 

internal/state/state_test.go

@@ -188,6 +188,95 @@ func TestLoadStateFromBeforeNameserverAddresses(t *testing.T) {
 	}
 }
 
+// TestSaveLoadRoundTrip_CNAMEAddresses checks that no addresses at the
+// end of a hostname's CNAME chain load as an empty list, and addresses
+// that are not known load as nil: the watcher tells the two apart.
+func TestSaveLoadRoundTrip_CNAMEAddresses(t *testing.T) {
+	t.Parallel()
+
+	dir := t.TempDir()
+	s := state.NewForTestWithDataDir(dir)
+
+	want := map[string][]string{
+		"cname.example.com":     {testIP},
+		"none.example.com":      {},
+		"not-known.example.com": nil,
+	}
+
+	for name, addresses := range want {
+		s.SetHostnameState(name, &state.HostnameState{
+			CNAMEAddresses: addresses,
+		})
+	}
+
+	err := s.Save()
+	if err != nil {
+		t.Fatalf("Save() error: %v", err)
+	}
+
+	loaded := state.NewForTestWithDataDir(dir)
+
+	err = loaded.Load()
+	if err != nil {
+		t.Fatalf("Load() error: %v", err)
+	}
+
+	for name, addresses := range want {
+		hs, ok := loaded.GetHostnameState(name)
+		if !ok {
+			t.Fatalf("missing hostname %s", name)
+		}
+
+		if !reflect.DeepEqual(hs.CNAMEAddresses, addresses) {
+			t.Errorf(
+				"%s: loaded %#v, want %#v",
+				name, hs.CNAMEAddresses, addresses,
+			)
+		}
+	}
+}
+
+// TestLoadStateFromBeforeCNAMEAddresses loads a state file written
+// before the addresses at the end of a hostname's CNAME chain were
+// saved. They load as not known (nil), not as none.
+func TestLoadStateFromBeforeCNAMEAddresses(t *testing.T) {
+	t.Parallel()
+
+	dir := t.TempDir()
+
+	data := []byte(`{
+		"version": 1,
+		"lastUpdated": "2026-02-19T12:00:00Z",
+		"hostnames": {
+			"www.example.com": {
+				"recordsByNameserver": {},
+				"lastChecked": "2026-02-19T12:00:00Z"
+			}
+		}
+	}`)
+
+	err := os.WriteFile(filepath.Join(dir, "state.json"), data, 0o600)
+	if err != nil {
+		t.Fatalf("writing state file: %v", err)
+	}
+
+	s := state.NewForTestWithDataDir(dir)
+
+	err = s.Load()
+	if err != nil {
+		t.Fatalf("Load() error: %v", err)
+	}
+
+	hs, ok := s.GetHostnameState(testHostname)
+	if !ok {
+		t.Fatal("missing hostname " + testHostname)
+	}
+
+	if hs.CNAMEAddresses != nil {
+		t.Errorf("CNAME addresses: got %#v, want nil", hs.CNAMEAddresses)
+	}
+}
+
 // TestSaveLoadRoundTrip_Hostnames verifies hostname data survives a save/load cycle.
 func TestSaveLoadRoundTrip_Hostnames(t *testing.T) {
 	t.Parallel()

internal/watcher/cname_test.go

@@ -0,0 +1,336 @@
+package watcher_test
+
+import (
+	"context"
+	"log/slog"
+	"slices"
+	"testing"
+
+	"sneak.berlin/go/dnswatcher/internal/livednstest"
+	"sneak.berlin/go/dnswatcher/internal/resolver"
+	"sneak.berlin/go/dnswatcher/internal/state"
+	"sneak.berlin/go/dnswatcher/internal/watcher"
+)
+
+// TestCNAMEIntoAnotherZonePortAndTLSChecks runs the port and TLS
+// checks on hostname state built here: the name's nameserver answered
+// with a CNAME into another zone, and following it found ip1. Both
+// checks must use ip1. They look nothing up, so the watcher has no
+// resolver.
+func TestCNAMEIntoAnotherZonePortAndTLSChecks(t *testing.T) {
+	t.Parallel()
+
+	cfg := defaultTestConfig(t)
+	cfg.Hostnames = []string{host}
+
+	deps := newTestDeps(t, cfg)
+	w := watcher.NewForTest(
+		cfg, deps.state, nil,
+		deps.portChecker, deps.tlsChecker, deps.notifier,
+	)
+
+	deps.state.SetHostnameState(host, cnameState(ip1))
+
+	w.CheckAllPorts(t.Context())
+	w.RunTLSChecks(t.Context())
+
+	snap := deps.state.GetSnapshot()
+
+	ps, ok := snap.Ports[ip1+":443"]
+	if !ok || !slices.Contains(ps.Hostnames, host) {
+		t.Errorf("no port state for %s at %s:443", host, ip1)
+	}
+
+	certKey := ip1 + ":443:" + host
+	if _, ok := snap.Certificates[certKey]; !ok {
+		t.Errorf("no certificate state %s", certKey)
+	}
+}
+
+// TestCNAMEThatCannotBeFollowedKeepsPrevious runs a check of a name, not
+// the watcher's first, from the point where its records have been looked
+// up: they hold a CNAME to a target under .invalid, whose lookup fails.
+// The previous check found the same records, and oldIP at the end of the
+// CNAME. The check must keep oldIP and send nothing.
+func TestCNAMEThatCannotBeFollowedKeepsPrevious(t *testing.T) {
+	t.Parallel()
+
+	w, deps := newTestWatcher(t, defaultTestConfig(t))
+	w.SetFirstRun(false)
+
+	records := map[string]map[string][]string{
+		nsA: cnameTo("target.example.invalid."),
+	}
+
+	prev := hostnameState(records)
+	prev.CNAMEAddresses = []string{oldIP}
+	deps.state.SetHostnameState(host, prev)
+
+	// The result is the same whether or not live DNS answers, so the
+	// lookup is not retried.
+	_ = livednstest.Run(func(ctx context.Context) error {
+		w.UpdateHostnameState(ctx, host, hostnameState(records))
+
+		return nil
+	})
+
+	hs, _ := deps.state.GetHostnameState(host)
+	if !slices.Equal(hs.CNAMEAddresses, prev.CNAMEAddresses) {
+		t.Errorf(
+			"saved %v, want %v",
+			hs.CNAMEAddresses, prev.CNAMEAddresses,
+		)
+	}
+
+	notifications := deps.notifier.getNotifications()
+	if len(notifications) != 0 {
+		t.Errorf("sent %v, want no notifications", notifications)
+	}
+}
+
+// followLive follows in live DNS the CNAMEs in a name's records, built
+// from records, and returns the addresses saved for the name. The
+// previous check saved oldIP, which is kept when a target cannot be
+// followed; that is retried. The tests point CNAMEs only at names in
+// zones with two nameservers, to keep queries few (see the top of
+// watcher_test.go).
+func followLive(
+	t *testing.T,
+	records map[string]map[string][]string,
+) []string {
+	t.Helper()
+
+	w := watcher.NewForTest(
+		nil, nil, resolver.NewFromLogger(slog.Default()), nil, nil, nil,
+	)
+	prev := cnameState(oldIP)
+
+	var current *state.HostnameState
+
+	livednstest.Retry(t, "following CNAMEs", func(ctx context.Context) error {
+		current = hostnameState(records)
+
+		w.ResolveCNAMEAddresses(ctx, host, current, prev)
+
+		if slices.Equal(current.CNAMEAddresses, prev.CNAMEAddresses) {
+			return livednstest.ErrNoAnswer
+		}
+
+		return nil
+	})
+
+	return current.CNAMEAddresses
+}
+
+// TestCNAMEAddressesOfEveryTarget gives a name's two nameservers
+// different CNAME targets, as when a secondary still serves an old one.
+// The addresses at the end of both are saved, whichever answer is read
+// first: one.one.one.one has 1.1.1.1, and dns.adguard-dns.com has
+// 94.140.14.14.
+func TestCNAMEAddressesOfEveryTarget(t *testing.T) {
+	t.Parallel()
+
+	found := followLive(t, map[string]map[string][]string{
+		nsA: cnameTo("one.one.one.one."),
+		nsB: cnameTo("dns.adguard-dns.com."),
+	})
+
+	for _, ip := range []string{"1.1.1.1", "94.140.14.14"} {
+		if !slices.Contains(found, ip) {
+			t.Errorf("saved %v, want %s among them", found, ip)
+		}
+	}
+}
+
+// TestCNAMEChainEndingInNoAddressSavesEmptyList follows a CNAME to a
+// name live DNS answers with NXDOMAIN. An empty list is saved, not nil,
+// which would mean the addresses are not known.
+func TestCNAMEChainEndingInNoAddressSavesEmptyList(t *testing.T) {
+	t.Parallel()
+
+	found := followLive(t, map[string]map[string][]string{
+		nsA: cnameTo("this-surely-does-not-exist-xyz.example.org."),
+	})
+
+	if found == nil || len(found) != 0 {
+		t.Errorf("saved %#v, want an empty list", found)
+	}
+}
+
+// TestCNAMEBesideAnAddressNotFollowed gives one nameserver of a name an
+// address and another a CNAME. The CNAME is not followed: an empty list
+// is saved, not nil, and nothing is looked up, the watcher having no
+// resolver.
+func TestCNAMEBesideAnAddressNotFollowed(t *testing.T) {
+	t.Parallel()
+
+	w := watcher.NewForTest(nil, nil, nil, nil, nil, nil)
+
+	current := hostnameState(map[string]map[string][]string{
+		nsA: {"A": {ip1}},
+		nsB: cnameTo("target.example.org."),
+	})
+
+	w.ResolveCNAMEAddresses(t.Context(), host, current, nil)
+
+	if current.CNAMEAddresses == nil || len(current.CNAMEAddresses) != 0 {
+		t.Errorf("saved %#v, want an empty list", current.CNAMEAddresses)
+	}
+}
+
+// TestCNAMEWhoseNameserversAllFailedKeepsPrevious checks a name none of
+// whose nameservers answered. The addresses the previous check saved
+// from following its CNAME are kept, and nothing is looked up: the
+// watcher has no resolver.
+func TestCNAMEWhoseNameserversAllFailedKeepsPrevious(t *testing.T) {
+	t.Parallel()
+
+	w := watcher.NewForTest(nil, nil, nil, nil, nil, nil)
+
+	current := saved(map[string]*state.NameserverRecordState{
+		nsA: failed(), nsB: failed(),
+	})
+	prev := cnameState(oldIP)
+
+	w.ResolveCNAMEAddresses(t.Context(), host, current, prev)
+
+	if !slices.Equal(current.CNAMEAddresses, prev.CNAMEAddresses) {
+		t.Errorf(
+			"saved %v, want %v",
+			current.CNAMEAddresses, prev.CNAMEAddresses,
+		)
+	}
+}
+
+// cnameTo builds the records of a nameserver that answered with a CNAME
+// to target and no address.
+func cnameTo(target string) map[string][]string {
+	return map[string][]string{"CNAME": {target}}
+}
+
+// cnameState builds the state a check leaves behind for a name whose
+// nameserver answered with a CNAME and no address, when following the
+// CNAME found these addresses, which may be none.
+func cnameState(addresses ...string) *state.HostnameState {
+	hs := hostnameState(map[string]map[string][]string{
+		nsA: cnameTo("target.example.org."),
+	})
+
+	hs.CNAMEAddresses = append([]string{}, addresses...)
+
+	return hs
+}
+
+func TestCNAMEAddressChangeAlerts(t *testing.T) {
+	t.Parallel()
+
+	// A state file written before the addresses were saved loads with
+	// them nil.
+	olderStateFile := cnameState()
+	olderStateFile.CNAMEAddresses = nil
+
+	// Each case is the state saved by the previous check and by the
+	// current one. The name's records are the same in both.
+	tests := []struct {
+		name          string
+		prev, current *state.HostnameState
+		want          int
+	}{
+		{
+			"same addresses",
+			cnameState(ip1, ip2), cnameState(ip1, ip2), 0,
+		},
+		{
+			"same addresses in another order",
+			cnameState(ip2, ip1), cnameState(ip1, ip2), 0,
+		},
+		{
+			"address replaced",
+			cnameState(ip1), cnameState(ip2), 1,
+		},
+		{
+			"address added",
+			cnameState(ip1), cnameState(ip1, ip2), 1,
+		},
+		{
+			"no address at the end of the chain now",
+			cnameState(ip1), cnameState(), 1,
+		},
+		{
+			"addresses at the end of the chain again",
+			cnameState(), cnameState(ip1), 1,
+		},
+		{
+			"state file from before addresses were saved",
+			olderStateFile, cnameState(ip1), 0,
+		},
+	}
+
+	for _, tt := range tests {
+		t.Run(tt.name, func(t *testing.T) {
+			t.Parallel()
+
+			notifier := &mockNotifier{}
+			w := watcher.NewForTest(nil, nil, nil, nil, nil, notifier)
+
+			w.DetectHostnameChanges(t.Context(), host, tt.prev, tt.current)
+
+			got := len(notifier.getNotifications())
+			if got != tt.want {
+				t.Errorf("sent %d notifications, want %d", got, tt.want)
+			}
+		})
+	}
+}
+
+func TestCNAMEAddressChangeAlertNamesHostnameAndAddresses(t *testing.T) {
+	t.Parallel()
+
+	notifier := &mockNotifier{}
+	w := watcher.NewForTest(nil, nil, nil, nil, nil, notifier)
+
+	w.DetectHostnameChanges(
+		t.Context(), host, cnameState(ip1), cnameState(ip2, ip3),
+	)
+
+	want := notification{
+		Title: "CNAME Address Change: " + host,
+		Message: "Hostname: " + host +
+			"\nOld: " + ip1 + "\nNew: " + ip2 + ", " + ip3,
+		Priority: "warning",
+	}
+
+	got := notifier.getNotifications()
+	if len(got) != 1 || got[0] != want {
+		t.Errorf("sent %v, want %v", got, want)
+	}
+}
+
+// TestNameMovedFromARecordsToCNAMEAlerts checks a name that answers
+// with an A record and then with a CNAME whose chain ends in ip2. The
+// second check is notified as a CNAME address change from no addresses,
+// beside the record change. Nothing is looked up: the watcher has no
+// resolver.
+func TestNameMovedFromARecordsToCNAMEAlerts(t *testing.T) {
+	t.Parallel()
+
+	notifier := &mockNotifier{}
+	w := watcher.NewForTest(nil, nil, nil, nil, nil, notifier)
+
+	prev := hostnameState(map[string]map[string][]string{
+		nsA: {"A": {ip1}},
+	})
+	w.ResolveCNAMEAddresses(t.Context(), host, prev, nil)
+
+	w.DetectHostnameChanges(t.Context(), host, prev, cnameState(ip2))
+
+	title := "CNAME Address Change: " + host
+	message := "Hostname: " + host + "\nOld: \nNew: " + ip2
+
+	got := notifier.getNotifications()
+	if !slices.ContainsFunc(got, func(n notification) bool {
+		return n.Title == title && n.Message == message
+	}) {
+		t.Errorf("sent %v, want %q with %q among them", got, title, message)
+	}
+}

internal/watcher/export_test.go

@@ -38,6 +38,21 @@ func NewlyDisagreeingPairs(
 	return newlyDisagreeingPairs(prev, current)
 }
 
+// SetFirstRun sets whether the watcher is on its first check, in which
+// nothing is compared with the previous check. NewForTest's watcher is.
+func (w *Watcher) SetFirstRun(firstRun bool) {
+	w.firstRun = firstRun
+}
+
+// UpdateHostnameState exports updateHostnameState for testing.
+func (w *Watcher) UpdateHostnameState(
+	ctx context.Context,
+	hostname string,
+	newState *state.HostnameState,
+) {
+	w.updateHostnameState(ctx, hostname, newState)
+}
+
 // DetectHostnameChanges exports detectHostnameChanges for testing.
 func (w *Watcher) DetectHostnameChanges(
 	ctx context.Context,
@@ -57,6 +72,15 @@ func (w *Watcher) ResolveNameserverAddresses(
 	return w.resolveNameserverAddresses(ctx, nameservers, prev)
 }
 
+// ResolveCNAMEAddresses exports resolveCNAMEAddresses for testing.
+func (w *Watcher) ResolveCNAMEAddresses(
+	ctx context.Context,
+	hostname string,
+	current, prev *state.HostnameState,
+) {
+	w.resolveCNAMEAddresses(ctx, hostname, current, prev)
+}
+
 // DetectNSAddressChanges exports detectNSAddressChanges for testing.
 func (w *Watcher) DetectNSAddressChanges(
 	ctx context.Context,

internal/watcher/watcher.go

@@ -123,8 +123,10 @@ func (w *Watcher) Run(ctx context.Context) {
 		"watcher starting",
 		"domains", len(w.config.Domains),
 		"hostnames", len(w.config.Hostnames),
-		"dnsInterval", w.config.DNSInterval,
+		// As text: the JSON log writes a time.Duration as bare
-		"tlsInterval", w.config.TLSInterval,
+		// nanoseconds.
+		"dnsInterval", w.config.DNSInterval.String(),
+		"tlsInterval", w.config.TLSInterval.String(),
 	)
 
 	w.RunOnce(ctx)
@@ -252,28 +254,9 @@ func (w *Watcher) checkDomain(
 		LastChecked:         now,
 	})
 
-	// Also look up A/AAAA records for the apex domain so that
+	// The apex domain's records are also checked as a hostname's, so
-	// port and TLS checks (which read HostnameState) can find
+	// that the port and TLS checks find its addresses.
-	// the domain's IP addresses.
+	w.checkHostname(ctx, domain)
-	results, err := w.resolver.LookupAllRecords(ctx, domain)
-	if err != nil {
-		w.log.Error(
-			"failed to lookup records for domain",
-			"domain", domain,
-			"error", err,
-		)
-
-		return
-	}
-
-	newState := buildHostnameState(results, now)
-
-	prevHS, hasPrevHS := w.state.GetHostnameState(domain)
-	if hasPrevHS && !w.firstRun {
-		w.detectHostnameChanges(ctx, domain, prevHS, newState)
-	}
-
-	w.state.SetHostnameState(domain, newState)
 }
 
 func (w *Watcher) detectNSChanges(
@@ -398,9 +381,23 @@ func (w *Watcher) checkHostname(
 		return
 	}
 
-	newState := buildHostnameState(results, time.Now().UTC())
+	w.updateHostnameState(
+		ctx, hostname, buildHostnameState(results, time.Now().UTC()),
+	)
+}
 
+// updateHostnameState finishes a check of hostname from newState, built
+// from its nameservers' answers: it follows the CNAME in them, notifies
+// what changed since the previous check, and saves newState.
+func (w *Watcher) updateHostnameState(
+	ctx context.Context,
+	hostname string,
+	newState *state.HostnameState,
+) {
 	prev, hasPrev := w.state.GetHostnameState(hostname)
+
+	w.resolveCNAMEAddresses(ctx, hostname, newState, prev)
+
 	if hasPrev && !w.firstRun {
 		w.detectHostnameChanges(ctx, hostname, prev, newState)
 	}
@@ -408,6 +405,76 @@ func (w *Watcher) checkHostname(
 	w.state.SetHostnameState(hostname, newState)
 }
 
+// resolveCNAMEAddresses saves in current the addresses at the end of
+// hostname's CNAME chain, when the nameservers' answers in current hold
+// a CNAME and no address, and an empty list otherwise. Every CNAME
+// target the nameservers gave is followed with ResolveIPAddresses and
+// the addresses found for all of them are saved, so nameservers that
+// disagree on the target do not change the result from check to check.
+// The addresses saved in prev, which may be nil, are kept when none of
+// the name's nameservers answered, and when a target cannot be
+// followed, as when no nameserver of a zone in its chain answers.
+func (w *Watcher) resolveCNAMEAddresses(
+	ctx context.Context,
+	hostname string,
+	current, prev *state.HostnameState,
+) {
+	var prevAddresses []string
+	if prev != nil {
+		prevAddresses = prev.CNAMEAddresses
+	}
+
+	// Empty, not nil: nil means the addresses are not known.
+	current.CNAMEAddresses = []string{}
+
+	answered := false
+	targets := make(map[string]bool)
+
+	for _, nsState := range current.RecordsByNameserver {
+		if nsState.Status != statusOK {
+			continue
+		}
+
+		answered = true
+
+		if len(nsState.Records["A"]) > 0 || len(nsState.Records["AAAA"]) > 0 {
+			return
+		}
+
+		for _, target := range nsState.Records["CNAME"] {
+			targets[target] = true
+		}
+	}
+
+	if !answered {
+		current.CNAMEAddresses = prevAddresses
+
+		return
+	}
+
+	for target := range targets {
+		ips, err := w.resolver.ResolveIPAddresses(ctx, target)
+		if err != nil {
+			w.log.Error(
+				"failed to follow CNAME",
+				"hostname", hostname,
+				"target", target,
+				"error", err,
+			)
+
+			current.CNAMEAddresses = prevAddresses
+
+			return
+		}
+
+		current.CNAMEAddresses = append(current.CNAMEAddresses, ips...)
+	}
+
+	// Still the empty list when every chain ends in no address.
+	slices.Sort(current.CNAMEAddresses)
+	current.CNAMEAddresses = slices.Compact(current.CNAMEAddresses)
+}
+
 // buildHostnameState saves each nameserver's response. A nameserver
 // that answered, even with NXDOMAIN or no records, is saved as ok; one
 // that timed out or failed is saved as error with the reason, and its
@@ -451,6 +518,37 @@ func (w *Watcher) detectHostnameChanges(
 	w.detectNSDisappearances(ctx, hostname, prev, current)
 	w.detectNSFailures(ctx, hostname, prev, current)
 	w.detectInconsistencies(ctx, hostname, prev, current)
+	w.detectCNAMEAddressChanges(ctx, hostname, prev, current)
+}
+
+// detectCNAMEAddressChanges notifies when the addresses at the end of
+// hostname's CNAME chain differ from those the previous check saved,
+// including a change from or to none. When the previous addresses are
+// not known (nil), as on the first check after loading a state file
+// written before they were saved, nothing is compared.
+func (w *Watcher) detectCNAMEAddressChanges(
+	ctx context.Context,
+	hostname string,
+	prev, current *state.HostnameState,
+) {
+	old, cur := prev.CNAMEAddresses, current.CNAMEAddresses
+	if old == nil || sliceEqual(old, cur) {
+		return
+	}
+
+	msg := fmt.Sprintf(
+		"Hostname: %s\nOld: %s\nNew: %s",
+		hostname,
+		strings.Join(old, ", "),
+		strings.Join(cur, ", "),
+	)
+
+	w.notify.SendNotification(
+		ctx,
+		"CNAME Address Change: "+hostname,
+		msg,
+		"warning",
+	)
 }
 
 // detectRecordChanges compares each nameserver's records with those of
@@ -747,6 +845,9 @@ func (w *Watcher) noNameserverAnswered(name string) bool {
 	return true
 }
 
+// collectIPs returns the addresses saved for hostname: those in its
+// nameservers' A and AAAA records, and those at the end of its CNAME
+// chain.
 func (w *Watcher) collectIPs(hostname string) []string {
 	hs, ok := w.state.GetHostnameState(hostname)
 	if !ok {
@@ -765,6 +866,10 @@ func (w *Watcher) collectIPs(hostname string) []string {
 		}
 	}
 
+	for _, ip := range hs.CNAMEAddresses {
+		ipSet[ip] = true
+	}
+
 	result := make([]string, 0, len(ipSet))
 	for ip := range ipSet {
 		result = append(result, ip)

internal/watcher/watcher_test.go

@@ -312,7 +312,8 @@ func lookupNameservers(t *testing.T, name string) []string {
 	return nameservers
 }
 
-// addresses returns the A and AAAA values saved for a hostname.
+// addresses returns the A and AAAA values saved for a hostname, and the
+// addresses saved at the end of its CNAME chain.
 func addresses(hs *state.HostnameState) []string {
 	var ips []string
 
@@ -321,7 +322,7 @@ func addresses(hs *state.HostnameState) []string {
 		ips = append(ips, nsState.Records["AAAA"]...)
 	}
 
-	return ips
+	return append(ips, hs.CNAMEAddresses...)
 }
 
 // assertNotified checks that a notification with this title and
@@ -371,6 +372,14 @@ func TestFirstRunBaseline(t *testing.T) {
 
 	assertNoNotifications(t, deps)
 	assertStatePopulated(t, deps)
+
+	// testHost answers with an address, so the check saves an empty list
+	// of CNAME addresses for it; nil would mean the check did not look
+	// at whether to follow a CNAME.
+	hs, _ := deps.state.GetHostnameState(testHost)
+	if hs.CNAMEAddresses == nil || len(hs.CNAMEAddresses) != 0 {
+		t.Errorf("saved CNAME addresses %#v, want []", hs.CNAMEAddresses)
+	}
 }
 
 func assertNoNotifications(