Trial run of the finished image against real targets before tagging 1.0 #149

Closed
opened 2026-09-21 09:18:32 +02:00 by clawbot · 2 comments
Collaborator

Before 1.0 is tagged, someone has to run the finished thing and read what it does. The last such run was in March, before the dashboard, notification retry, shutdown drain and the nameserver failure work existed.

This is a test-and-report task, not a code change. It runs after #104 and #105 are on next.

What to do

Build the image from current next and run it for at least 30 minutes against 10 to 20 real targets (a mix of apex domains and hostnames, including at least one that does not exist and one with a certificate close to expiry if one can be found), with notifications pointed at a local listener you start yourself so every delivery can be read. Use short intervals so several cycles complete. Restart it once in the middle.

Read the whole log and every notification. Check the dashboard and /api/v1/status against what the log says happened.

Definition of done

  • One comment on this issue: what was run (one line), then only the defects found. Each defect is either fixed by a linked PR or filed as its own issue with a definition of done, full URL given.
  • If nothing is wrong, the comment is one line saying so.
  • DNS is never mocked; the run uses live DNS.

Model: fable-5-1

Before 1.0 is tagged, someone has to run the finished thing and read what it does. The last such run was in March, before the dashboard, notification retry, shutdown drain and the nameserver failure work existed. This is a test-and-report task, not a code change. It runs after https://git.eeqj.de/sneak/dnswatcher/issues/104 and https://git.eeqj.de/sneak/dnswatcher/issues/105 are on `next`. ## What to do Build the image from current `next` and run it for at least 30 minutes against 10 to 20 real targets (a mix of apex domains and hostnames, including at least one that does not exist and one with a certificate close to expiry if one can be found), with notifications pointed at a local listener you start yourself so every delivery can be read. Use short intervals so several cycles complete. Restart it once in the middle. Read the whole log and every notification. Check the dashboard and `/api/v1/status` against what the log says happened. ## Definition of done - One comment on this issue: what was run (one line), then only the defects found. Each defect is either fixed by a linked PR or filed as its own issue with a definition of done, full URL given. - If nothing is wrong, the comment is one line saying so. - DNS is never mocked; the run uses live DNS. Model: fable-5-1
clawbot added this to the 1.0 milestone 2026-09-21 09:18:32 +02:00
Author
Collaborator

Plan: the run described above, on an image built from the current next, with two additions: one hostname whose CNAME points into another zone, and a changed target list across the restart. Each defect found becomes its own issue on the 1.0 milestone; nothing is fixed during the run.

Not covered: #209 (port and TLS checks follow a CNAME into another zone) is still in rework and not on next.

Model: opus-5-5

Plan: the run described above, on an image built from the current `next`, with two additions: one hostname whose CNAME points into another zone, and a changed target list across the restart. Each defect found becomes its own issue on the 1.0 milestone; nothing is fixed during the run. Not covered: https://git.eeqj.de/sneak/dnswatcher/pulls/209 (port and TLS checks follow a CNAME into another zone) is still in rework and not on `next`. Model: opus-5-5
Author
Collaborator

Ran the image built from next at 3182fc9 for about 40 minutes against 16 live targets (7 apex domains and 9 hostnames, among them a domain and a hostname that do not exist, hostnames whose CNAME points into another zone and certificates within 35 days of expiry), with Slack, Mattermost and ntfy pointed at a local listener, restarted once midway with two targets swapped for two others.

  • #219 Record Change and Inconsistency notifications print whole record sets in Go map syntax, not the record type that changed.
  • #220 A hostname's CNAME is stored eight times, once for each record type asked for.
  • #221 Nameservers that a referral names without addresses are not looked up, so pool.ntp.org cannot be watched.
  • #222 A failed nameserver lookup falls back to a parent zone's nameservers: a domain that does not exist shows the .com servers.
  • #223 Targets removed from DNSWATCHER_TARGETS stay on the dashboard, in /api/v1/status and in the startup counts.
  • #224 Apex domains are counted, listed and notified as hostnames.
  • #225 The dashboard and /api/v1/status never say why a nameserver or certificate check failed.
  • #226 The dashboard lists each nameserver's records in a random order of record type.
  • #227 A failed Mattermost delivery is logged as "slack notification failed".
  • #228 Durations in the log are bare nanosecond counts.
  • #229 A DNS lookup cut short by shutdown is logged as an error.
  • #230 The startup notification says all notification channels work before any delivery is known.

Model: opus-5-5

Ran the image built from `next` at `3182fc9` for about 40 minutes against 16 live targets (7 apex domains and 9 hostnames, among them a domain and a hostname that do not exist, hostnames whose CNAME points into another zone and certificates within 35 days of expiry), with Slack, Mattermost and ntfy pointed at a local listener, restarted once midway with two targets swapped for two others. - https://git.eeqj.de/sneak/dnswatcher/issues/219 Record Change and Inconsistency notifications print whole record sets in Go map syntax, not the record type that changed. - https://git.eeqj.de/sneak/dnswatcher/issues/220 A hostname's CNAME is stored eight times, once for each record type asked for. - https://git.eeqj.de/sneak/dnswatcher/issues/221 Nameservers that a referral names without addresses are not looked up, so `pool.ntp.org` cannot be watched. - https://git.eeqj.de/sneak/dnswatcher/issues/222 A failed nameserver lookup falls back to a parent zone's nameservers: a domain that does not exist shows the `.com` servers. - https://git.eeqj.de/sneak/dnswatcher/issues/223 Targets removed from `DNSWATCHER_TARGETS` stay on the dashboard, in `/api/v1/status` and in the startup counts. - https://git.eeqj.de/sneak/dnswatcher/issues/224 Apex domains are counted, listed and notified as hostnames. - https://git.eeqj.de/sneak/dnswatcher/issues/225 The dashboard and `/api/v1/status` never say why a nameserver or certificate check failed. - https://git.eeqj.de/sneak/dnswatcher/issues/226 The dashboard lists each nameserver's records in a random order of record type. - https://git.eeqj.de/sneak/dnswatcher/issues/227 A failed Mattermost delivery is logged as "slack notification failed". - https://git.eeqj.de/sneak/dnswatcher/issues/228 Durations in the log are bare nanosecond counts. - https://git.eeqj.de/sneak/dnswatcher/issues/229 A DNS lookup cut short by shutdown is logged as an error. - https://git.eeqj.de/sneak/dnswatcher/issues/230 The startup notification says all notification channels work before any delivery is known. Model: opus-5-5
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: sneak/dnswatcher#149