1 Commits
Author SHA1 Message Date
sneak d8c1325aa5 watcher: keep port state when no nameserver of a name answered (closes #193)
check / check (push) Successful in 1m47s
The port check removed the saved port state of every address no
configured name resolves to. A name whose nameservers all timed out
or failed is saved with no records, so its addresses looked gone and
lost their port state; when the nameservers answered again it was
recorded afresh, and a port that opened or closed meanwhile was not
notified.

An entry is now kept when one of the names saved on it is configured
and none of its nameservers answered on its last check, and such a
name stays on the entry when the port is checked again for another
name. A name whose nameservers answer with no addresses still loses
it, and so does a name no longer configured.

Model: opus-5-5
2026-10-01 22:16:12 +00:00
23 changed files with 429 additions and 772 deletions
+1 -4
View File
@@ -1,9 +1,6 @@
.git/ .git/
bin/ bin/
node_modules/ *.md
# No .md may be excluded: Dockerfile.fmt checks every document with
# prettier, and an exclusion here would drop a file from that check while
# prettier still reports every file it was handed clean.
LICENSE LICENSE
.editorconfig .editorconfig
.gitignore .gitignore
-1
View File
@@ -1,5 +1,4 @@
bin/ bin/
node_modules/
vendor/ vendor/
data/ data/
.env .env
-5
View File
@@ -1,5 +0,0 @@
bin/
data/
node_modules/
.claude/
static/css/tailwind.min.css
-4
View File
@@ -1,4 +0,0 @@
{
"tabWidth": 4,
"proseWrap": "always"
}
+2 -4
View File
@@ -1,9 +1,7 @@
# Lint stage - fast feedback on lint issues, before the build starts. # Lint stage - fast feedback on lint issues, before the build starts.
# The linter is invoked directly rather than through `make lint`: that # The linter is invoked directly rather than through `make lint`: that
# target shells out to `docker build -f Dockerfile.lint`, and there is # target shells out to `docker build -f Dockerfile.lint`, and there is
# no docker daemon inside a docker build. For the same reason this stage # no docker daemon inside a docker build.
# runs only the Go half of `make fmt-check`; script/cibuild runs the
# markdown half after this build.
# script/cibuild and script/docker name this stage in --no-cache-filter. # script/cibuild and script/docker name this stage in --no-cache-filter.
# golangci/golangci-lint:v2.12.2 (Debian-based), 2026-08-10 # golangci/golangci-lint:v2.12.2 (Debian-based), 2026-08-10
FROM golangci/golangci-lint:v2.12.2@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 AS lint FROM golangci/golangci-lint:v2.12.2@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 AS lint
@@ -14,7 +12,7 @@ RUN go mod download
COPY . . COPY . .
RUN script/fmt-check-go RUN make fmt-check
RUN golangci-lint run --config .golangci.yml ./... RUN golangci-lint run --config .golangci.yml ./...
# Build stage # Build stage
-55
View File
@@ -1,55 +0,0 @@
# prettier over the markdown, in a container, so it is never installed
# on the host. script/fmt-check-markdown builds the fmt-check stage;
# script/fmt builds fmt-out and takes the formatted files back.
# node:22-bookworm-slim, 2026-09-05
FROM node:22-bookworm-slim@sha256:83f487e0a63425e5b4d146fb5e5be574bcbe1b7b843d3ebafdd95eaf7767a7e5 AS nodedeps
# prettier lives outside /src so that a `COPY . .` of the repo cannot
# overwrite it, and so that node_modules never appears in the tree
# prettier is about to walk.
WORKDIR /tools
# package.json pins the version and yarn.lock pins the bytes:
# --frozen-lockfile installs exactly the lockfile's resolution and fails
# if package.json disagrees with it, so the tool cannot float between
# runs. yarn is the one in the image above.
COPY package.json yarn.lock ./
RUN yarn install --frozen-lockfile --non-interactive --no-progress
ENV PATH="/tools/node_modules/.bin:${PATH}"
WORKDIR /src
# Read-only markdown check. Must match $stage in
# script/fmt-check-markdown.
FROM nodedeps AS fmt-check
COPY . .
# --config, not discovery: a .prettierrc that failed to arrive would
# otherwise leave prettier on its defaults, where proseWrap is "preserve"
# and every wrap this check exists to enforce passes. Missing the file is
# a hard error instead. --no-editorconfig for the same reason in reverse:
# .editorconfig is not in the build context, so honouring it here and on
# a developer's machine would be two different answers.
RUN prettier --config .prettierrc --no-editorconfig --check "**/*.md"
# Write path. Not a check: script/fmt builds this and takes the files.
FROM nodedeps AS fmt
COPY . .
RUN prettier --config .prettierrc --no-editorconfig --write "**/*.md"
# Only the markdown leaves, with its paths intact, so that the export
# below cannot put anything else back over the caller's working tree.
RUN mkdir -p /out && cd /src && \
find . -name '*.md' -type f -exec cp --parents '{}' /out/ ';'
# Export target: `docker build --target fmt-out --output type=local`
# writes /out's tree into a directory on the client, which is how
# script/fmt gets formatted markdown back without a bind mount.
# Must match $stage in script/fmt.
FROM scratch AS fmt-out
COPY --from=fmt /out/ /
+260 -260
View File
@@ -1,20 +1,16 @@
# dnswatcher # dnswatcher
dnswatcher is an MIT-licensed, pre-1.0 Go daemon by dnswatcher is an MIT-licensed, pre-1.0 Go daemon by [@sneak](https://sneak.berlin) that monitors DNS records, TCP port availability, and TLS certificates, delivering real-time change notifications via Slack, Mattermost, and ntfy webhooks.
[@sneak](https://sneak.berlin) that monitors DNS records, TCP port availability,
and TLS certificates, delivering real-time change notifications via Slack,
Mattermost, and ntfy webhooks.
> ⚠️ Pre-1.0 software. APIs, configuration, and behavior may change without > ⚠️ Pre-1.0 software. APIs, configuration, and behavior may change without notice.
> notice.
dnswatcher watches configured DNS domains and hostnames for changes, monitors dnswatcher watches configured DNS domains and hostnames for changes, monitors TCP
TCP port availability, tracks TLS certificate expiry, and delivers real-time port availability, tracks TLS certificate expiry, and delivers real-time
notifications via Slack, Mattermost, and/or ntfy webhooks. notifications via Slack, Mattermost, and/or ntfy webhooks.
It performs all DNS resolution itself via iterative (non-recursive) queries, It performs all DNS resolution itself via iterative (non-recursive) queries,
tracing from root nameservers to authoritative servers directly—never relying on tracing from root nameservers to authoritative servers directly—never relying
upstream recursive resolvers. on upstream recursive resolvers.
State is persisted to a local JSON file so that monitoring survives restarts State is persisted to a local JSON file so that monitoring survives restarts
without requiring an external database. without requiring an external database.
@@ -23,20 +19,21 @@ without requiring an external database.
## No DNS mocking. Ever. ## No DNS mocking. Ever.
**DNS is never mocked in this project — not in tests, not anywhere else.** No **DNS is never mocked in this project — not in tests, not anywhere else.**
mock resolvers, no fake DNS servers, no stubbed lookups. No mock resolvers, no fake DNS servers, no stubbed lookups.
dnswatcher's entire purpose is correct behavior against the real DNS. Tests dnswatcher's entire purpose is correct behavior against the real DNS.
exercise real iterative resolution against live nameservers by design; a test Tests exercise real iterative resolution against live nameservers by
suite that passes against a mock proves nothing about the one thing this program design; a test suite that passes against a mock proves nothing about the
exists to do. one thing this program exists to do.
When live tests are flaky, that is a robustness problem, and it gets fixed with When live tests are flaky, that is a robustness problem, and it gets
robustness: retries with backoff, querying multiple independent nameservers, fixed with robustness: retries with backoff, querying multiple
longer timeouts — or explicit opt-in gating decided by the project owner. Never independent nameservers, longer timeouts — or explicit opt-in gating
with mocks. decided by the project owner. Never with mocks.
Contributions that introduce mocked, faked, or stubbed DNS will be rejected. Contributions that introduce mocked, faked, or stubbed DNS will be
rejected.
--- ---
@@ -49,95 +46,98 @@ Contributions that introduce mocked, faked, or stubbed DNS will be rejected.
- Every **1 hour**, performs a full iterative trace from root servers to - Every **1 hour**, performs a full iterative trace from root servers to
discover all authoritative nameservers (NS records) for each domain. discover all authoritative nameservers (NS records) for each domain.
- Queries **every** discovered authoritative nameserver independently. - Queries **every** discovered authoritative nameserver independently.
- Stores the NS record set as observed by the delegation chain, and the IPv4 and - Stores the NS record set as observed by the delegation chain, and the
IPv6 addresses each nameserver's name resolves to. IPv4 and IPv6 addresses each nameserver's name resolves to.
- Any change triggers a notification: - Any change triggers a notification:
- NS added to or removed from the delegation. - NS added to or removed from the delegation.
- NS address change: a nameserver that stays in the delegation resolves to - NS address change: a nameserver that stays in the delegation
different addresses than on the previous check. A nameserver added or resolves to different addresses than on the previous check. A
removed gets only the NS change notification. When the lookup of a nameserver added or removed gets only the NS change notification.
nameserver's addresses fails or finds none, its previous addresses are When the lookup of a nameserver's addresses fails or finds none,
kept and nothing is sent. its previous addresses are kept and nothing is sent.
### DNS Hostname Monitoring (Subdomains) ### DNS Hostname Monitoring (Subdomains)
- Accepts a list of DNS hostnames (subdomains, distinguished from apex domains - Accepts a list of DNS hostnames (subdomains, distinguished from apex
via the Public Suffix List). domains via the Public Suffix List).
- Every **1 hour**, performs a full iterative trace to discover the - Every **1 hour**, performs a full iterative trace to discover the
authoritative nameservers of the zone the hostname is in, which is not always authoritative nameservers of the zone the hostname is in, which is not
its last two labels (a name under `co.uk`, or in a delegated subdomain). always its last two labels (a name under `co.uk`, or in a delegated
- Queries **each** authoritative nameserver independently for **all** record subdomain).
types: A, AAAA, CNAME, MX, TXT, SRV, CAA, NS. - Queries **each** authoritative nameserver independently for **all**
- Stores results **per nameserver**. The state for a hostname is not a merged record types: A, AAAA, CNAME, MX, TXT, SRV, CAA, NS.
view — it is a map from nameserver to record set. - Stores results **per nameserver**. The state for a hostname is not a
- DNS names inside record values (CNAME, MX, SRV and NS targets) are stored in merged view — it is a map from nameserver to record set.
lower case, because names are case-insensitive and nameservers may answer in - DNS names inside record values (CNAME, MX, SRV and NS targets) are
any letter case. TXT and CAA values keep their letter case; they are not stored in lower case, because names are case-insensitive and
lower-cased. nameservers may answer in any letter case. TXT and CAA values keep
- Any observable change in any nameserver's response triggers a notification. their letter case; they are not lower-cased.
This includes: - Any observable change in any nameserver's response triggers a
- **Record change**: A nameserver returns different records than it did on notification. This includes:
the previous check (additions, removals, value changes). - **Record change**: A nameserver returns different records than it
- **NS query failure**: A nameserver that previously responded becomes did on the previous check (additions, removals, value changes).
unreachable (timeout, SERVFAIL, REFUSED, network error). This is distinct - **NS query failure**: A nameserver that previously responded
from "responded with no records": a nameserver that answers NXDOMAIN or becomes unreachable (timeout, SERVFAIL, REFUSED, network error).
with no records has responded. The alert is sent once, on the check where This is distinct from "responded with no records": a nameserver
it starts failing. A failing nameserver gives no records, so it is not that answers NXDOMAIN or with no records has responded. The alert
reported as a record change or compared for inconsistency. A nameserver is sent once, on the check where it starts failing. A failing
that is already failing on the first check that sees it is recorded nameserver gives no records, so it is not reported as a record
silently. change or compared for inconsistency. A nameserver that is already
- **NS recovery**: A previously-unreachable nameserver starts responding failing on the first check that sees it is recorded silently.
again. Its records are not compared with those from before it failed, so a - **NS recovery**: A previously-unreachable nameserver starts
change made while it was failing is not reported as a record change. responding again. Its records are not compared with those from
- **Inconsistency detected**: Two nameservers return different record sets before it failed, so a change made while it was failing is not
for the same hostname and did not already differ on the previous check. reported as a record change.
Every pair of nameservers is compared. The alert is sent once for each - **Inconsistency detected**: Two nameservers return different record
such pair, on the check where they start to disagree, and not again while sets for the same hostname and did not already differ on the previous
they keep disagreeing, including after a restart. A nameserver that was check. Every pair of nameservers is compared. The alert is sent once
not in the previous check (newly added, or back after dropping out), or for each such pair, on the check where they start to disagree, and not
failed on it, and answers differently is reported on the check where it again while they keep disagreeing, including after a restart. A
answers. If a pair agrees again and later disagrees, the alert is sent nameserver that was not in the previous check (newly added, or back
again. after dropping out), or failed on it, and answers differently is
reported on the check where it answers. If a pair agrees again and
later disagrees, the alert is sent again.
### TCP Port Monitoring ### TCP Port Monitoring
- For every configured domain and hostname, constructs a deduplicated list of - For every configured domain and hostname, constructs a deduplicated list
all IPv4 and IPv6 addresses resolved via A, AAAA, and CNAME chain resolution of all IPv4 and IPv6 addresses resolved via A, AAAA, and CNAME chain
across all authoritative nameservers. resolution across all authoritative nameservers.
- Checks TCP connectivity on ports **80** and **443** for each IP address. - Checks TCP connectivity on ports **80** and **443** for each IP address.
- Every **1 hour**, re-checks all ports. - Every **1 hour**, re-checks all ports.
- Any change in port availability triggers a notification: - Any change in port availability triggers a notification:
- Port transitioned from open to closed (or vice versa). - Port transitioned from open to closed (or vice versa).
- New IP appeared (from DNS change) and its port state was recorded. - New IP appeared (from DNS change) and its port state was recorded.
- IP disappeared (from DNS change) — noted in the DNS change notification; - IP disappeared (from DNS change) — noted in the DNS change
port state for that IP is removed. When none of a name's nameservers notification; port state for that IP is removed. When none of a name's
answered, its addresses are not known, so the port state saved for them is nameservers answered, its addresses are not known, so the port state saved
kept. for them is kept.
### TLS Certificate Monitoring ### TLS Certificate Monitoring
- Every **12 hours**, for each IP address listening on port 443, connects via - Every **12 hours**, for each IP address listening on port 443, connects
TLS using the correct SNI hostname. via TLS using the correct SNI hostname.
- Records the certificate's Subject CN, SANs, issuer, and expiry date. - Records the certificate's Subject CN, SANs, issuer, and expiry date.
- Any change triggers a notification: - Any change triggers a notification:
- Certificate is expiring within **7 days** (warning, repeated each check - Certificate is expiring within **7 days** (warning, repeated each
until renewed or expired). check until renewed or expired).
- Certificate CN, issuer, or SANs changed (replacement detected, reports old - Certificate CN, issuer, or SANs changed (replacement detected,
and new values). reports old and new values).
- TLS connection failure to a previously-reachable IP:443 (handshake error, - TLS connection failure to a previously-reachable IP:443 (handshake
timeout, connection refused after previously succeeding). error, timeout, connection refused after previously succeeding).
- TLS recovery: a previously-failing IP:443 now completes a handshake again. - TLS recovery: a previously-failing IP:443 now completes a
handshake again.
### Notifications ### Notifications
**Every observable state change produces a notification.** dnswatcher is **Every observable state change produces a notification.** dnswatcher is
designed as a real-time change feed — degradations, failures, recoveries, and designed as a real-time change feed — degradations, failures, recoveries,
routine changes are all reported equally. and routine changes are all reported equally.
Supported notification backends: Supported notification backends:
| Backend | Configuration | Payload Format | | Backend | Configuration | Payload Format |
| -------------- | ------------------------------------------ | ---------------------------- | |----------------|--------------------------|------------------------------|
| **Slack** | Incoming Webhook URL | Attachments with color | | **Slack** | Incoming Webhook URL | Attachments with color |
| **Mattermost** | Incoming Webhook URL | Slack-compatible attachments | | **Mattermost** | Incoming Webhook URL | Slack-compatible attachments |
| **ntfy** | Topic URL (e.g. `https://ntfy.sh/mytopic`) | Title + body + priority | | **ntfy** | Topic URL (e.g. `https://ntfy.sh/mytopic`) | Title + body + priority |
@@ -145,29 +145,29 @@ Supported notification backends:
All configured endpoints receive every notification. Notification content All configured endpoints receive every notification. Notification content
includes: includes:
- **DNS record changes**: Which hostname, which nameserver, what record type, - **DNS record changes**: Which hostname, which nameserver, what record
old values, new values. type, old values, new values.
- **DNS NS changes**: Which domain, which nameservers were added/removed. - **DNS NS changes**: Which domain, which nameservers were added/removed.
- **NS address changes**: Which domain, which nameserver, its old and new - **NS address changes**: Which domain, which nameserver, its old and
addresses. new addresses.
- **NS query failures**: Which nameserver failed, error type (timeout, SERVFAIL, - **NS query failures**: Which nameserver failed, error type (timeout,
REFUSED, network error), which hostname/domain affected. SERVFAIL, REFUSED, network error), which hostname/domain affected.
- **NS recoveries**: Which nameserver recovered, which hostname/domain. - **NS recoveries**: Which nameserver recovered, which hostname/domain.
- **NS inconsistencies**: Which nameservers disagree, what each one returned, - **NS inconsistencies**: Which nameservers disagree, what each one
which hostname affected. returned, which hostname affected.
- **Port changes**: Which IP:port, old state, new state, all associated - **Port changes**: Which IP:port, old state, new state, all associated
hostnames. hostnames.
- **TLS expiry warnings**: Which certificate, days remaining, CN, issuer, - **TLS expiry warnings**: Which certificate, days remaining, CN,
associated hostname and IP. issuer, associated hostname and IP.
- **TLS certificate changes**: Old and new CN/issuer/SANs, associated hostname - **TLS certificate changes**: Old and new CN/issuer/SANs, associated
and IP. hostname and IP.
- **TLS connection failures/recoveries**: Which IP:port, error details, - **TLS connection failures/recoveries**: Which IP:port, error details,
associated hostname. associated hostname.
### State Management ### State Management
- All monitoring state is kept in memory and persisted to a JSON file on disk - All monitoring state is kept in memory and persisted to a JSON file on
(`DATA_DIR/state.json`). disk (`DATA_DIR/state.json`).
- State is loaded on startup to resume monitoring without triggering - State is loaded on startup to resume monitoring without triggering
false-positive change notifications. false-positive change notifications.
- State is written atomically (write to temp file, then rename) to prevent - State is written atomically (write to temp file, then rename) to prevent
@@ -175,33 +175,35 @@ includes:
### Web Dashboard ### Web Dashboard
dnswatcher includes an unauthenticated, read-only web dashboard at the root URL dnswatcher includes an unauthenticated, read-only web dashboard at the
(`/`). It displays: root URL (`/`). It displays:
- **Summary counts** for monitored domains, hostnames, ports, and certificates. - **Summary counts** for monitored domains, hostnames, ports, and
certificates.
- **Domains** with their discovered nameservers. - **Domains** with their discovered nameservers.
- **Hostnames** with per-nameserver DNS records and status. - **Hostnames** with per-nameserver DNS records and status.
- **Ports** with open/closed state and associated hostnames. - **Ports** with open/closed state and associated hostnames.
- **TLS certificates** with CN, issuer, expiry, and status. - **TLS certificates** with CN, issuer, expiry, and status.
- **Recent alerts** (last 100 notifications sent since the process started), - **Recent alerts** (last 100 notifications sent since the process
displayed in reverse chronological order. started), displayed in reverse chronological order.
Every data point shows its age (e.g. "5m ago") so you can tell at a glance how Every data point shows its age (e.g. "5m ago") so you can tell at a
fresh the information is. The page auto-refreshes every 30 seconds. glance how fresh the information is. The page auto-refreshes every 30
seconds.
The dashboard intentionally does not expose any configuration details such as The dashboard intentionally does not expose any configuration details
webhook URLs, notification endpoints, or API tokens. such as webhook URLs, notification endpoints, or API tokens.
All assets (CSS) are embedded in the binary and served from the application All assets (CSS) are embedded in the binary and served from the
itself. The dashboard makes zero external HTTP requests — no CDN dependencies or application itself. The dashboard makes zero external HTTP requests —
third-party resources are loaded at runtime. no CDN dependencies or third-party resources are loaded at runtime.
### HTTP API ### HTTP API
dnswatcher exposes a lightweight HTTP API for operational visibility: dnswatcher exposes a lightweight HTTP API for operational visibility:
| Endpoint | Description | | Endpoint | Description |
| ------------------------------ | ----------------------------- | |---------------------------------------|--------------------------------|
| `GET /` | Web dashboard (HTML) | | `GET /` | Web dashboard (HTML) |
| `GET /s/...` | Static assets (embedded CSS) | | `GET /s/...` | Static assets (embedded CSS) |
| `GET /.well-known/healthcheck` | Health check (JSON) | | `GET /.well-known/healthcheck` | Health check (JSON) |
@@ -216,7 +218,7 @@ constants in `internal/server/server.go`, not configurable via environment
variables. variables.
| Timeout | Value | Purpose | | Timeout | Value | Purpose |
| ------------------- | ----- | --------------------------------------------- | |---------------------|-------|-----------------------------------------------|
| `ReadHeaderTimeout` | 10s | Bounds the request header read (slowloris) | | `ReadHeaderTimeout` | 10s | Bounds the request header read (slowloris) |
| `ReadTimeout` | 15s | Bounds the whole request read, headers + body | | `ReadTimeout` | 15s | Bounds the whole request read, headers + body |
| `WriteTimeout` | 75s | Bounds handler execution plus response flush | | `WriteTimeout` | 75s | Bounds handler execution plus response flush |
@@ -224,20 +226,20 @@ variables.
These are distinct from the 60s per-request handler budget applied by These are distinct from the 60s per-request handler budget applied by
`chimw.Timeout` in `internal/server/routes.go`, which cancels the request `chimw.Timeout` in `internal/server/routes.go`, which cancels the request
context but does not touch the socket. `WriteTimeout` is deliberately larger context but does not touch the socket. `WriteTimeout` is deliberately
than that budget: the write deadline is armed once request headers are read, so larger than that budget: the write deadline is armed once request headers
a smaller value would sever the connection before a handler using its full are read, so a smaller value would sever the connection before a handler
budget could respond. `IdleTimeout` exceeds common Prometheus scrape intervals using its full budget could respond. `IdleTimeout` exceeds common
so the scraper reuses its connection. Prometheus scrape intervals so the scraper reuses its connection.
### Security Headers ### Security Headers
Every response — the dashboard, the static assets under `/s/...`, the Every response — the dashboard, the static assets under `/s/...`, the
healthchecks, the JSON API, and `/metrics` — carries the following headers, set healthchecks, the JSON API, and `/metrics` — carries the following
by a global middleware: headers, set by a global middleware:
| Header | Value | | Header | Value |
| --------------------------- | ------------------------------------- | |-----------------------------|---------------------------------------|
| `Strict-Transport-Security` | `max-age=31536000; includeSubDomains` | | `Strict-Transport-Security` | `max-age=31536000; includeSubDomains` |
| `Content-Security-Policy` | see below | | `Content-Security-Policy` | see below |
| `X-Frame-Options` | `DENY` | | `X-Frame-Options` | `DENY` |
@@ -254,21 +256,21 @@ form-action 'none'; frame-ancestors 'none'
``` ```
The dashboard ships no JavaScript (the 30-second refresh is a The dashboard ships no JavaScript (the 30-second refresh is a
`<meta http-equiv="refresh">`), no inline styles, no inline event handlers, and `<meta http-equiv="refresh">`), no inline styles, no inline event
no images; its only subresource is the embedded stylesheet at handlers, and no images; its only subresource is the embedded stylesheet
`/s/css/tailwind.min.css`, which `style-src 'self'` permits. The policy at `/s/css/tailwind.min.css`, which `style-src 'self'` permits. The
therefore needs neither `unsafe-inline` nor `unsafe-eval`. policy therefore needs neither `unsafe-inline` nor `unsafe-eval`.
`frame-ancestors 'none'` is the primary anti-framing control, with `frame-ancestors 'none'` is the primary anti-framing control, with
`X-Frame-Options: DENY` retained as the legacy fallback. `X-Frame-Options: DENY` retained as the legacy fallback.
HSTS is emitted unconditionally, including over plain HTTP. dnswatcher is HSTS is emitted unconditionally, including over plain HTTP. dnswatcher is
expected to run behind a TLS-terminating reverse proxy, and the browser must expected to run behind a TLS-terminating reverse proxy, and the browser
still be told to enforce HTTPS end to end, so the header is never gated on must still be told to enforce HTTPS end to end, so the header is never
whether the request itself arrived over TLS. gated on whether the request itself arrived over TLS.
`Referrer-Policy: no-referrer` is stricter than the `Referrer-Policy: no-referrer` is stricter than the
`strict-origin-when-cross-origin` baseline: the dashboard has no cross-origin `strict-origin-when-cross-origin` baseline: the dashboard has no
navigation needs, and its URL may name internal hosts. cross-origin navigation needs, and its URL may name internal hosts.
--- ---
@@ -301,23 +303,23 @@ internal/
### Design Principles ### Design Principles
- **No recursive resolvers**: All DNS resolution is performed iteratively, - **No recursive resolvers**: All DNS resolution is performed iteratively,
tracing from root nameservers through the delegation chain to authoritative tracing from root nameservers through the delegation chain to
servers. authoritative servers.
- **No external database**: State is persisted as a single JSON file. - **No external database**: State is persisted as a single JSON file.
- **Dependency injection**: All components are wired via - **Dependency injection**: All components are wired via
[uber/fx](https://github.com/uber-go/fx). [uber/fx](https://github.com/uber-go/fx).
- **Structured logging**: All logs use `log/slog` with JSON output in production - **Structured logging**: All logs use `log/slog` with JSON output in
(TTY detection for development). production (TTY detection for development).
- **Graceful shutdown**: All background goroutines respect context cancellation - **Graceful shutdown**: All background goroutines respect context
and the fx lifecycle. In-flight notification deliveries are drained on cancellation and the fx lifecycle. In-flight notification deliveries
shutdown, bounded by the shutdown timeout. are drained on shutdown, bounded by the shutdown timeout.
--- ---
## Configuration ## Configuration
Configuration is loaded via [Viper](https://github.com/spf13/viper) with the Configuration is loaded via [Viper](https://github.com/spf13/viper) with
following precedence (highest to lowest): the following precedence (highest to lowest):
1. Environment variables (prefixed with `DNSWATCHER_`) 1. Environment variables (prefixed with `DNSWATCHER_`)
2. `.env` file (loaded via godotenv) 2. `.env` file (loaded via godotenv)
@@ -328,7 +330,7 @@ following precedence (highest to lowest):
### Environment Variables ### Environment Variables
| Variable | Description | Default | | Variable | Description | Default |
| ----------------------------------- | ----------------------------------------------------------------------------------------------------------- | --------------------- | |---------------------------------|--------------------------------------------|-------------|
| `PORT` | HTTP listen port | `8080` | | `PORT` | HTTP listen port | `8080` |
| `DNSWATCHER_DEBUG` | Enable debug logging | `false` | | `DNSWATCHER_DEBUG` | Enable debug logging | `false` |
| `DNSWATCHER_DATA_DIR` | Directory for state file | `/var/lib/dnswatcher` | | `DNSWATCHER_DATA_DIR` | Directory for state file | `/var/lib/dnswatcher` |
@@ -348,24 +350,25 @@ following precedence (highest to lowest):
**`DNSWATCHER_TARGETS` is required.** dnswatcher will refuse to start if no **`DNSWATCHER_TARGETS` is required.** dnswatcher will refuse to start if no
monitoring targets are configured. A monitoring daemon with nothing to monitor monitoring targets are configured. A monitoring daemon with nothing to monitor
is a misconfiguration, so dnswatcher fails fast with a clear error message is a misconfiguration, so dnswatcher fails fast with a clear error message
rather than running silently. Set `DNSWATCHER_TARGETS` to a comma-separated list rather than running silently. Set `DNSWATCHER_TARGETS` to a comma-separated
of DNS names before starting. list of DNS names before starting.
**`/metrics` is rate limited.** Each client address may send it 30 requests a **`/metrics` is rate limited.** Each client address may send it 30 requests a
minute, failed logins included; beyond that it answers `429 Too Many Requests` minute, failed logins included; beyond that it answers `429 Too Many Requests`
without checking the password. A Prometheus server scraping every 15 seconds without checking the password. A Prometheus server scraping every 15 seconds
sends 4 a minute. IPv6 addresses in one /64 count as one client. When the sends 4 a minute. IPv6 addresses in one /64 count as one client. When the
request comes from a private or loopback address, such as a reverse proxy's, the request comes from a private or loopback address, such as a reverse proxy's,
client address is taken from the `X-Real-IP` header the proxy sets, or else from the client address is taken from the `X-Real-IP` header the proxy sets, or else
`X-Forwarded-For`, as the last address in it that is not private or loopback. A from `X-Forwarded-For`, as the last address in it that is not private or
proxy that sets neither makes all its clients share one allowance. loopback. A proxy that sets neither makes all its clients share one allowance.
**`DNSWATCHER_DNS_INTERVAL` and `DNSWATCHER_TLS_INTERVAL`** take a positive **`DNSWATCHER_DNS_INTERVAL` and `DNSWATCHER_TLS_INTERVAL`** take a positive
duration: a number followed by a unit such as `s`, `m` or `h`, for example duration: a number followed by a unit such as `s`, `m` or `h`, for example
`90s`, `30m`, `1h` or `1h30m`. There is no unit for days; write `24h`. An unset `90s`, `30m`, `1h` or `1h30m`. There is no unit for days; write `24h`. An
or empty variable (`DNSWATCHER_DNS_INTERVAL=`) means the default. If either is unset or empty variable (`DNSWATCHER_DNS_INTERVAL=`) means the default. If
set to anything else, including a bare number or a zero or negative duration, either is set to anything else, including a bare number or a zero or negative
dnswatcher refuses to start with an error naming the variable and the value. duration, dnswatcher refuses to start with an error naming the variable and
the value.
**`DNSWATCHER_SENTRY_DSN` reports crashes in HTTP requests to Sentry.** When it **`DNSWATCHER_SENTRY_DSN` reports crashes in HTTP requests to Sentry.** When it
is set, a panic in an HTTP request handler is sent to Sentry, and the request is set, a panic in an HTTP request handler is sent to Sentry, and the request
@@ -391,33 +394,33 @@ DNSWATCHER_SEND_TEST_NOTIFICATION=true
## DNS Resolution Strategy ## DNS Resolution Strategy
dnswatcher never uses the system's configured recursive resolver. Instead, it dnswatcher never uses the system's configured recursive resolver. Instead,
performs full iterative resolution: it performs full iterative resolution:
1. **Root servers**: Starts from the IANA root nameserver list (hardcoded, with 1. **Root servers**: Starts from the IANA root nameserver list (hardcoded,
periodic refresh). with periodic refresh).
2. **TLD delegation**: Queries root servers for the TLD NS records. 2. **TLD delegation**: Queries root servers for the TLD NS records.
3. **Domain delegation**: Queries TLD nameservers for the domain's NS records. 3. **Domain delegation**: Queries TLD nameservers for the domain's NS
4. **Authoritative query**: Queries all discovered authoritative nameservers records.
directly for the requested records. 4. **Authoritative query**: Queries all discovered authoritative
nameservers directly for the requested records.
This approach ensures: This approach ensures:
- Independence from any upstream resolver's cache or filtering. - Independence from any upstream resolver's cache or filtering.
- Ability to detect split-horizon or inconsistent responses across authoritative - Ability to detect split-horizon or inconsistent responses across
servers. authoritative servers.
- Visibility into the full delegation chain. - Visibility into the full delegation chain.
For hostname monitoring, the resolver follows CNAME chains (with a depth limit For hostname monitoring, the resolver follows CNAME chains (with a
to prevent loops) before collecting terminal A/AAAA records. depth limit to prevent loops) before collecting terminal A/AAAA records.
--- ---
## State File Format ## State File Format
The state file (`DATA_DIR/state.json`) contains the complete monitoring The state file (`DATA_DIR/state.json`) contains the complete monitoring
snapshot. Hostname records are stored **per authoritative nameserver**, not as a snapshot. Hostname records are stored **per authoritative nameserver**,
merged view, to enable inconsistency detection. not as a merged view, to enable inconsistency detection.
```json ```json
{ {
@@ -481,17 +484,17 @@ merged view, to enable inconsistency detection.
} }
``` ```
The `status` field for each per-nameserver entry and certificate entry tracks The `status` field for each per-nameserver entry and certificate entry
reachability: tracks reachability:
| Status | Meaning | | Status | Meaning |
| ------- | -------------------------------------------------------- | |-------------|------------------------------------------------------------|
| `ok` | Query succeeded, records are current | | `ok` | Query succeeded, records are current |
| `error` | Query failed (timeout, SERVFAIL, REFUSED, network error) | | `error` | Query failed (timeout, SERVFAIL, REFUSED, network error) |
A nameserver that answers NXDOMAIN or with no records has status `ok` and empty A nameserver that answers NXDOMAIN or with no records has status `ok` and
`records`. A nameserver whose query failed, or that only referred it to other empty `records`. A nameserver whose query failed has status `error`, empty
nameservers, has status `error`, empty `records`, and the reason in `error`. `records`, and the reason in `error`.
`nameserverAddresses` lists, by nameserver, the sorted addresses its name `nameserverAddresses` lists, by nameserver, the sorted addresses its name
resolves to. A state file without it loads, and the next check fills it in resolves to. A state file without it loads, and the next check fills it in
@@ -504,38 +507,31 @@ without a notification.
This repository adheres to the This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all) [Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call them. We development workflow, and the Makefile targets are thin shims that call
provide: them. We provide:
- `script/bootstrap` — install all dependencies (go, `go mod download`). It does - `script/bootstrap` — install all dependencies (go, `go mod download`).
not install golangci-lint or prettier: both run in Docker, see `script/lint` It does not install golangci-lint: see `script/lint` below.
and `script/fmt` below. - `script/setup` — make a fresh clone ready for development: bootstrap
- `script/setup` — make a fresh clone ready for development: bootstrap plus the plus the git pre-commit hook
git pre-commit hook - `script/projectname` — print the project name (used for the Docker
- `script/projectname` — print the project name (used for the Docker image tag) image tag)
- `script/test` — run the test suite (race detector, coverage). Caching is - `script/test` — run the test suite (race detector, coverage). Caching
waived for testing, exactly as it is for linting: `-count=1` forces every is waived for testing, exactly as it is for linting: `-count=1`
invocation to execute, because the suite queries live DNS and a cached pass forces every invocation to execute, because the suite queries live
queries nothing. Failures are rerun with `-v` automatically, and the build DNS and a cached pass queries nothing. Failures are rerun with `-v`
fails even if that rerun passes. automatically, and the build fails even if that rerun passes.
- `script/lint` — run golangci-lint, always inside Docker: it builds - `script/lint` — run golangci-lint, always inside Docker: it builds
`Dockerfile.lint`, which COPYs the repo into the digest-pinned `golangci-lint` `Dockerfile.lint`, which COPYs the repo into the digest-pinned
image and lints as a build step, so a successful build is a clean lint. The `golangci-lint` image and lints as a build step, so a successful
linter is never installed or run on the host, and Docker is the only build is a clean lint. The linter is never installed or run on the
prerequisite. Caching is waived for linting: the lint stage is forced to host, and Docker is the only prerequisite. Caching is waived for
execute on every run with `--no-cache-filter`, because a cached build lints linting: the lint stage is forced to execute on every run with
nothing. `--no-cache-filter`, because a cached build lints nothing.
- `script/fmt` — format all code (gofmt -s, goimports) and all Markdown - `script/fmt` — format all code (gofmt -s, goimports). goimports runs
(prettier). goimports runs with `go run` at a pinned commit, never from your with `go run` at a pinned commit, never from your `PATH`.
`PATH`. prettier runs inside Docker, built from `Dockerfile.fmt` on a - `script/fmt-check` — check formatting (read-only) with the same tools,
digest-pinned node image, at the version pinned by `package.json` and failing on any file `script/fmt` would change
`yarn.lock`; it is never installed on the host.
- `script/fmt-check` — check formatting (read-only) with the same tools, failing
on any file `script/fmt` would change. It runs the two scripts below.
- `script/fmt-check-go` — the gofmt and goimports half, on the host. The
`Dockerfile` lint stage runs it.
- `script/fmt-check-markdown` — the prettier half, inside Docker, forced to
execute on every run with `--no-cache-filter`
- `script/check` — run test, lint, and fmt-check - `script/check` — run test, lint, and fmt-check
- `script/docker` — build the Docker image tagged via `script/projectname`, with - `script/docker` — build the Docker image tagged via `script/projectname`, with
`--no-cache-filter=lint,builder` so the lint stage and the builder stage, `--no-cache-filter=lint,builder` so the lint stage and the builder stage,
@@ -544,9 +540,9 @@ provide:
- `script/cibuild` — CI entrypoint: `docker build` with - `script/cibuild` — CI entrypoint: `docker build` with
`--no-cache-filter=lint,builder`, so the lint stage and the builder stage, `--no-cache-filter=lint,builder`, so the lint stage and the builder stage,
which runs the tests, run on every invocation, because a cached build lints which runs the tests, run on every invocation, because a cached build lints
nothing and queries no DNS; then `script/fmt-check-markdown` nothing and queries no DNS
- `script/precommit` — run by the git pre-commit hook; `go mod tidy` guard, then - `script/precommit` — run by the git pre-commit hook; `go mod tidy`
`script/check` guard, then `script/check`
- `script/install-precommit` — install the git pre-commit hook - `script/install-precommit` — install the git pre-commit hook
## Building ## Building
@@ -555,21 +551,21 @@ provide:
make build # Build binary to bin/dnswatcher make build # Build binary to bin/dnswatcher
make test # Run tests with race detector make test # Run tests with race detector
make lint # Run golangci-lint in Docker (requires docker) make lint # Run golangci-lint in Docker (requires docker)
make fmt # Format code and Markdown (requires docker) make fmt # Format code
make check # Run all checks (test, lint, fmt-check) make check # Run all checks (test, lint, fmt-check)
make clean # Remove build artifacts make clean # Remove build artifacts
``` ```
### Build-Time Variables ### Build-Time Variables
`make build` sets the version with `-ldflags "-X main.Version=..."`, taking it `make build` sets the version with `-ldflags "-X main.Version=..."`, taking
from `git describe --tags --always --dirty`, or from `VERSION` when given on the it from `git describe --tags --always --dirty`, or from `VERSION` when given
command line (`make build VERSION=1.2.3`). The version appears in the startup on the command line (`make build VERSION=1.2.3`). The version appears in the
log and in the health check response. startup log and in the health check response.
The Docker image has no `.git`, so the `Dockerfile` takes the version as The Docker image has no `.git`, so the `Dockerfile` takes the version as
`--build-arg VERSION`. `make docker` passes it; a plain `docker build` passes `--build-arg VERSION`. `make docker` passes it; a plain `docker build`
none, and that image reports `dev`. passes none, and that image reports `dev`.
--- ---
@@ -590,80 +586,84 @@ docker run -d \
## Running under upaas ## Running under upaas
[upaas](https://git.eeqj.de/sneak/upaas) builds the image from this repository's [upaas](https://git.eeqj.de/sneak/upaas) builds the image from this
`Dockerfile` and runs it. The app needs: repository's `Dockerfile` and runs it. The app needs:
- **Branch:** `prod`. `prod` is cut from `main`, and merging a `main` to `prod` - **Branch:** `prod`. `prod` is cut from `main`, and merging a `main` to
pull request is a deploy. `prod` pull request is a deploy.
- **Volume:** one host directory mounted at `/var/lib/dnswatcher`, where the - **Volume:** one host directory mounted at `/var/lib/dnswatcher`, where
state file lives. the state file lives.
- **Network and port:** the dashboard is unauthenticated and shows every watched - **Network and port:** the dashboard is unauthenticated and shows every
name and recent alert, and upaas publishes every mapped port on all interfaces watched name and recent alert, and upaas publishes every mapped port on
of the host ([upaas issue 113](https://git.eeqj.de/sneak/upaas/issues/113)). all interfaces of the host
Add a port mapping to container port `8080` only if the dashboard should be ([upaas issue 113](https://git.eeqj.de/sneak/upaas/issues/113)). Add a
public. Otherwise add none: set the app's Docker network in upaas to your port mapping to container port `8080` only if the dashboard should be
reverse proxy's Docker network, and the proxy reaches the app at `upaas-` public. Otherwise add none: set the app's Docker network in upaas to
followed by the app name, port `8080`. your reverse proxy's Docker network, and the proxy reaches the app at
- **Required environment:** `DNSWATCHER_TARGETS`, a comma-separated list of the `upaas-` followed by the app name, port `8080`.
domains and hostnames to watch. dnswatcher refuses to start without it. - **Required environment:** `DNSWATCHER_TARGETS`, a comma-separated list
of the domains and hostnames to watch. dnswatcher refuses to start
without it.
- **Recommended environment:** at least one notification endpoint - **Recommended environment:** at least one notification endpoint
(`DNSWATCHER_SLACK_WEBHOOK`, `DNSWATCHER_MATTERMOST_WEBHOOK`, (`DNSWATCHER_SLACK_WEBHOOK`, `DNSWATCHER_MATTERMOST_WEBHOOK`,
`DNSWATCHER_NTFY_TOPIC`); without one, changes show only on the dashboard. `DNSWATCHER_NTFY_TOPIC`); without one, changes show only on the
`DNSWATCHER_METRICS_USERNAME` and `DNSWATCHER_METRICS_PASSWORD` serve dashboard. `DNSWATCHER_METRICS_USERNAME` and
`/metrics` behind basic auth. `DNSWATCHER_METRICS_PASSWORD` serve `/metrics` behind basic auth.
- **Leave unset:** `DNSWATCHER_DATA_DIR`, which the image sets to - **Leave unset:** `DNSWATCHER_DATA_DIR`, which the image sets to
`/var/lib/dnswatcher`, and `PORT`, which defaults to `8080`. Every setting `/var/lib/dnswatcher`, and `PORT`, which defaults to `8080`. Every
comes from the environment; the image holds no config file. setting comes from the environment; the image holds no config file.
- **Health check:** the image's own, which requests `/.well-known/healthcheck` - **Health check:** the image's own, which requests
every 10 seconds. upaas reads the container's health 60 seconds after a deploy `/.well-known/healthcheck` every 10 seconds. upaas reads the
and marks the deploy failed unless it is `healthy`. container's health 60 seconds after a deploy and marks the deploy
failed unless it is `healthy`.
--- ---
## Monitoring Lifecycle ## Monitoring Lifecycle
1. **Startup**: Check that the data directory can be written, and exit with an 1. **Startup**: Check that the data directory can be written, and exit
error naming it if not. Load state from disk. If no state file exists, start with an error naming it if not. Load state from disk. If no state
with empty state (first check will establish baseline without triggering file exists, start with empty state (first check will establish
change notifications). baseline without triggering change notifications).
2. **Initial check**: Immediately perform all DNS, port, and TLS checks on 2. **Initial check**: Immediately perform all DNS, port, and TLS checks
startup. on startup.
3. **Periodic checks** (DNS always runs first): 3. **Periodic checks** (DNS always runs first):
- DNS checks: every `DNSWATCHER_DNS_INTERVAL` (default 1h). Also re-run - DNS checks: every `DNSWATCHER_DNS_INTERVAL` (default 1h). Also
before every TLS check cycle to ensure fresh IPs. re-run before every TLS check cycle to ensure fresh IPs.
- Port checks: every `DNSWATCHER_DNS_INTERVAL`, after DNS completes. - Port checks: every `DNSWATCHER_DNS_INTERVAL`, after DNS completes.
- TLS checks: every `DNSWATCHER_TLS_INTERVAL` (default 12h), after DNS - TLS checks: every `DNSWATCHER_TLS_INTERVAL` (default 12h), after
completes. DNS completes.
- Port and TLS checks always use freshly resolved IP addresses from the DNS - Port and TLS checks always use freshly resolved IP addresses from
phase that immediately precedes them — never stale IPs from a previous the DNS phase that immediately precedes them — never stale IPs
cycle. from a previous cycle.
4. **On change detection**: Send notifications to all configured endpoints, 4. **On change detection**: Send notifications to all configured
update in-memory state, persist to disk. endpoints, update in-memory state, persist to disk.
5. **Shutdown**: The watcher stops checking and saves the final state to disk, 5. **Shutdown**: The watcher stops checking and saves the final state
and shutdown waits for that save before it goes on. Then it waits for to disk, and shutdown waits for that save before it goes on. Then it
in-flight notification deliveries to complete. Both waits share the fx waits for in-flight notification deliveries to complete. Both waits
shutdown timeout (15s by default): deliveries still retrying against an share the fx shutdown timeout (15s by default): deliveries still
unreachable endpoint when that expires are abandoned, and the number retrying against an unreachable endpoint when that expires are
abandoned is logged at warn level rather than dropped silently. Notifications abandoned, and the number abandoned is logged at warn level rather
generated after shutdown has begun are refused and logged, so a late burst than dropped silently. Notifications generated after shutdown has
cannot extend the shutdown. A DNS lookup, port check or TLS check that begun are refused and logged, so a late burst cannot extend the
shutdown cuts short saves nothing and sends no notification. shutdown. A DNS lookup, port check or TLS check that shutdown cuts
short saves nothing and sends no notification.
--- ---
## Planned Future Features (Post-1.0) ## Planned Future Features (Post-1.0)
- **DNSSEC validation**: Validate the DNSSEC chain of trust during iterative - **DNSSEC validation**: Validate the DNSSEC chain of trust during
resolution and report DNSSEC failures as notifications. iterative resolution and report DNSSEC failures as notifications.
--- ---
## Project Structure ## Project Structure
Follows the conventions defined in `REPO_POLICIES.md`, adapted from the Follows the conventions defined in `REPO_POLICIES.md`, adapted from the
[upaas](https://git.eeqj.de/sneak/upaas) project template. Uses uber/fx for [upaas](https://git.eeqj.de/sneak/upaas) project template. Uses uber/fx
dependency injection, go-chi for HTTP routing, slog for logging, and Viper for for dependency injection, go-chi for HTTP routing, slog for logging, and
configuration. Viper for configuration.
--- ---
+27 -26
View File
@@ -2,43 +2,44 @@
## DNS Resolution Tests ## DNS Resolution Tests
DNS is never mocked in this project, not in tests and not anywhere else; see the DNS is never mocked in this project, not in tests and not anywhere
README section "No DNS mocking. Ever." Every test that looks something up in DNS else; see the README section "No DNS mocking. Ever." Every test that
**MUST** query live DNS servers, never a stand-in. Logic that works on record looks something up in DNS **MUST** query live DNS servers, never a
data, such as comparing or formatting records, may be tested on that data stand-in. Logic that works on record data, such as comparing or
directly with no lookup. formatting records, may be tested on that data directly with no
lookup.
### Rationale ### Rationale
The resolver performs iterative resolution from root nameservers through the The resolver performs iterative resolution from root nameservers through
full delegation chain. Mocked responses cannot faithfully represent the variety the full delegation chain. Mocked responses cannot faithfully represent
of real-world DNS behavior (truncation, referrals, glue records, DNSSEC, varied the variety of real-world DNS behavior (truncation, referrals, glue
response times, EDNS, etc.). Testing against real servers ensures the resolver records, DNSSEC, varied response times, EDNS, etc.). Testing against
works correctly in production. real servers ensures the resolver works correctly in production.
### Constraints ### Constraints
- Tests hit real DNS infrastructure and require network access - Tests hit real DNS infrastructure and require network access
- Test duration depends on network conditions; timeout tuning keeps the suite - Test duration depends on network conditions; timeout tuning keeps
within the 60-second target the suite within the 60-second target
- Query timeout is calibrated to 3× maximum antipodal RTT (~300ms) plus - Query timeout is calibrated to 3× maximum antipodal RTT (~300ms)
processing margin plus processing margin
- Root server fan-out is limited to reduce parallel query load - Root server fan-out is limited to reduce parallel query load
- Live lookups that expect an answer go through `internal/livednstest`, which - Live lookups that expect an answer go through `internal/livednstest`,
limits how many run at once in a test binary and retries a lookup that got which limits how many run at once in a test binary and retries a
none lookup that got none
- Flaky failures from transient network issues are acceptable and should be - Flaky failures from transient network issues are acceptable and
investigated as potential resolver bugs, not papered over with mocks or skip should be investigated as potential resolver bugs, not papered over
flags with mocks or skip flags
### What NOT to do ### What NOT to do
- **Do not mock, fake or stub DNS** anywhere: no stand-in `DNSClient`, no - **Do not mock, fake or stub DNS** anywhere: no stand-in `DNSClient`,
stand-in for the watcher's `DNSResolver`, no fake DNS server, no canned no stand-in for the watcher's `DNSResolver`, no fake DNS server, no
responses canned responses
- **Do not add `-short` flags** to skip slow tests - **Do not add `-short` flags** to skip slow tests
- **Do not increase `-timeout`** to hide hanging queries - **Do not increase `-timeout`** to hide hanging queries
- **Do not remove `-count=1` from `script/test`** — Go's test cache replays a - **Do not remove `-count=1` from `script/test`** — Go's test cache
previous run's output without querying anything, so a cached pass is not replays a previous run's output without querying anything, so a
evidence that live resolution works cached pass is not evidence that live resolution works
- **Do not modify linter configuration** to suppress findings - **Do not modify linter configuration** to suppress findings
+11 -12
View File
@@ -1,12 +1,12 @@
# Workflow # Workflow
- branch (from `next`) * branch (from `next`)
- do the work in Next Step * do the work in Next Step
- move Next Step to the top of Completed Steps * move Next Step to the top of Completed Steps
- move the top item of Future Steps into Next Step * move the top item of Future Steps into Next Step
- commit (`TODO.md` changes in the same commit as the work) * commit (`TODO.md` changes in the same commit as the work)
- push * push
- open a PR against `next` * open a PR against `next`
# Status # Status
@@ -15,16 +15,13 @@ on the 1.0 milestone: https://git.eeqj.de/sneak/dnswatcher/milestone/7
# Next Step # Next Step
trial run of the finished image: https://git.eeqj.de/sneak/dnswatcher/issues/149 trial run of the finished image:
https://git.eeqj.de/sneak/dnswatcher/issues/149
# Completed Steps # Completed Steps
- 2026-10-01: when none of a configured name's nameservers answered, the port - 2026-10-01: when none of a configured name's nameservers answered, the port
state saved for its addresses is kept, not removed (closes #193). state saved for its addresses is kept, not removed (closes #193).
- 2026-10-01: `ResolveIPAddresses` returns an error, not no addresses, when no
nameserver of the name's zone answered (closes #190).
- 2026-10-01: `make fmt` and `make fmt-check` cover Markdown with prettier, run
in Docker at the version pinned by `yarn.lock` (closes #119).
- 2026-10-01: `make fmt-check` fails on a file `goimports` would change; both - 2026-10-01: `make fmt-check` fails on a file `goimports` would change; both
format scripts run `goimports` at its pinned commit, not from `PATH` (#119). format scripts run `goimports` at its pinned commit, not from `PATH` (#119).
- 2026-10-01: a hostname is queried at the servers of the zone it is in, found - 2026-10-01: a hostname is queried at the servers of the zone it is in, found
@@ -119,6 +116,8 @@ trial run of the finished image: https://git.eeqj.de/sneak/dnswatcher/issues/149
- 1.0 readiness: run it with a real config and read the logs: - 1.0 readiness: run it with a real config and read the logs:
https://git.eeqj.de/sneak/dnswatcher/issues/66 https://git.eeqj.de/sneak/dnswatcher/issues/66
- Markdown formatting with prettier:
https://git.eeqj.de/sneak/dnswatcher/issues/119
- README accuracy sweep: https://git.eeqj.de/sneak/dnswatcher/issues/108 - README accuracy sweep: https://git.eeqj.de/sneak/dnswatcher/issues/108
- README sections required by policy: - README sections required by policy:
https://git.eeqj.de/sneak/dnswatcher/issues/173 https://git.eeqj.de/sneak/dnswatcher/issues/173
-5
View File
@@ -10,11 +10,6 @@ var (
"no authoritative nameservers found", "no authoritative nameservers found",
) )
// ErrNoNameserverAnswered is returned when every nameserver
// asked about a name timed out, failed or returned a referral,
// so whether the name has addresses is unknown.
ErrNoNameserverAnswered = errors.New("no nameserver answered")
// ErrCNAMEDepthExceeded is returned when a CNAME chain // ErrCNAMEDepthExceeded is returned when a CNAME chain
// exceeds MaxCNAMEDepth. // exceeds MaxCNAMEDepth.
ErrCNAMEDepthExceeded = errors.New( ErrCNAMEDepthExceeded = errors.New(
-7
View File
@@ -11,13 +11,6 @@ func ExtractRecordValue(rr dns.RR) string {
return extractRecordValue(rr) return extractRecordValue(rr)
} }
// CollectIPs exports collectIPs for testing.
func CollectIPs(
results map[string]*NameserverResponse,
) ([]string, string, error) {
return collectIPs(results)
}
// QueryEachNS exports queryEachNS for testing. // QueryEachNS exports queryEachNS for testing.
func (r *Resolver) QueryEachNS( func (r *Resolver) QueryEachNS(
ctx context.Context, ctx context.Context,
+4 -41
View File
@@ -516,7 +516,6 @@ type queryState struct {
gotSERVFAIL bool gotSERVFAIL bool
gotRefused bool gotRefused bool
gotTimeout bool gotTimeout bool
gotReferral bool
netErr error netErr error
hasRecords bool hasRecords bool
} }
@@ -579,18 +578,6 @@ func (r *Resolver) querySingleType(
return return
} }
// A reply with no answer that lists other nameservers, from a server
// that does not hold the name's zone, is a referral and says nothing
// about the name's records. A parent zone's servers send one when
// every server of the name's own zone failed and
// FindAuthoritativeNameservers moved on to the parent name.
if !msg.Authoritative && len(msg.Answer) == 0 &&
len(extractNSSet(msg.Ns)) > 0 {
state.gotReferral = true
return
}
collectAnswerRecords(msg, resp, state) collectAnswerRecords(msg, resp, state)
} }
@@ -639,9 +626,6 @@ func classifyResponse(resp *NameserverResponse, state queryState) {
case state.netErr != nil && !state.hasRecords: case state.netErr != nil && !state.hasRecords:
resp.Status = StatusError resp.Status = StatusError
resp.Error = "network error: " + state.netErr.Error() resp.Error = "network error: " + state.netErr.Error()
case state.gotReferral && !state.hasRecords:
resp.Status = StatusError
resp.Error = "server returned a referral"
case !state.hasRecords && !state.gotNXDomain: case !state.hasRecords && !state.gotNXDomain:
resp.Status = StatusNoData resp.Status = StatusNoData
} }
@@ -750,9 +734,7 @@ func (r *Resolver) LookupAllRecords(
} }
// ResolveIPAddresses resolves a hostname to all IPv4 and IPv6 // ResolveIPAddresses resolves a hostname to all IPv4 and IPv6
// addresses, following CNAME chains up to MaxCNAMEDepth. When no // addresses, following CNAME chains up to MaxCNAMEDepth.
// nameserver of the name's zone answered, it returns an error rather
// than no addresses.
func (r *Resolver) ResolveIPAddresses( func (r *Resolver) ResolveIPAddresses(
ctx context.Context, ctx context.Context,
hostname string, hostname string,
@@ -778,10 +760,7 @@ func (r *Resolver) resolveIPWithCNAME(
return nil, err return nil, err
} }
ips, cnameTarget, err := collectIPs(results) ips, cnameTarget := collectIPs(results)
if err != nil {
return nil, fmt.Errorf("resolving %s: %w", hostname, err)
}
if len(ips) == 0 && cnameTarget != "" { if len(ips) == 0 && cnameTarget != "" {
return r.resolveIPWithCNAME(ctx, cnameTarget, depth+1) return r.resolveIPWithCNAME(ctx, cnameTarget, depth+1)
@@ -792,28 +771,16 @@ func (r *Resolver) resolveIPWithCNAME(
return ips, nil return ips, nil
} }
// collectIPs returns the addresses in the nameservers' answers and the
// first CNAME target among them. It returns ErrNoNameserverAnswered when
// every nameserver timed out, failed or returned a referral: that is not
// a name with no addresses.
func collectIPs( func collectIPs(
results map[string]*NameserverResponse, results map[string]*NameserverResponse,
) ([]string, string, error) { ) ([]string, string) {
seen := make(map[string]bool) seen := make(map[string]bool)
var ips []string var ips []string
var cnameTarget string var cnameTarget string
answered := false
for _, resp := range results { for _, resp := range results {
if resp.Status == StatusTimeout || resp.Status == StatusError {
continue
}
answered = true
if resp.Status == StatusNXDomain { if resp.Status == StatusNXDomain {
continue continue
} }
@@ -837,9 +804,5 @@ func collectIPs(
} }
} }
if !answered { return ips, cnameTarget
return nil, "", ErrNoNameserverAnswered
}
return ips, cnameTarget, nil
} }
-32
View File
@@ -5,42 +5,10 @@ import (
"github.com/miekg/dns" "github.com/miekg/dns"
"github.com/stretchr/testify/assert" "github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
"sneak.berlin/go/dnswatcher/internal/resolver" "sneak.berlin/go/dnswatcher/internal/resolver"
) )
// TestCollectIPs_OneAnswerIsEnough checks that one nameserver answering
// NXDOMAIN says the name has no addresses, though the other timed out.
func TestCollectIPs_OneAnswerIsEnough(t *testing.T) {
t.Parallel()
ips, _, err := resolver.CollectIPs(
map[string]*resolver.NameserverResponse{
"ns1.example.": {Status: resolver.StatusTimeout},
"ns2.example.": {Status: resolver.StatusNXDomain},
},
)
require.NoError(t, err)
assert.Empty(t, ips)
}
// TestCollectIPs_FailedIsNoAnswer checks that nameservers that all have
// status error, from a refusal, a server failure, a network error or a
// referral, are no answer rather than a name with no addresses.
func TestCollectIPs_FailedIsNoAnswer(t *testing.T) {
t.Parallel()
ips, _, err := resolver.CollectIPs(
map[string]*resolver.NameserverResponse{
"ns1.example.": {Status: resolver.StatusError},
"ns2.example.": {Status: resolver.StatusError},
},
)
require.ErrorIs(t, err, resolver.ErrNoNameserverAnswered)
assert.Empty(t, ips)
}
func TestExtractRecordValue_LetterCase(t *testing.T) { func TestExtractRecordValue_LetterCase(t *testing.T) {
t.Parallel() t.Parallel()
-76
View File
@@ -633,82 +633,6 @@ func TestQueryNameserverIP_Timeout(t *testing.T) {
assert.NotEmpty(t, resp.Error) assert.NotEmpty(t, resp.Error)
} }
// TestCollectIPs_NoNameserverAnswered takes the response of a
// nameserver at 192.0.2.1, where nothing answers, as
// TestQueryNameserverIP_Timeout does. Addresses collected from
// nameservers that all failed to answer are an error, not none.
func TestCollectIPs_NoNameserverAnswered(t *testing.T) {
t.Parallel()
r := newTestResolver(t)
// The deadline outlasts the first try, as in
// TestQueryNameserverIP_Timeout.
ctx, cancel := context.WithTimeout(
context.Background(), 3*time.Second,
)
t.Cleanup(cancel)
resp, err := r.QueryNameserverIP(
ctx, "unreachable.test.", "192.0.2.1",
"example.com",
)
require.NoError(t, err)
ips, _, err := resolver.CollectIPs(
map[string]*resolver.NameserverResponse{resp.Nameserver: resp},
)
require.ErrorIs(t, err, resolver.ErrNoNameserverAnswered)
assert.Empty(t, ips)
}
// TestCollectIPs_ReferralIsNoAnswer asks a root server about
// example.com, which the root zone does not hold, so it only refers the
// query to the com servers. That reply is no answer, as is a parent
// zone's when every server of the name's own zone failed.
func TestCollectIPs_ReferralIsNoAnswer(t *testing.T) {
t.Parallel()
r := newTestResolver(t)
var resp *resolver.NameserverResponse
livednstest.Retry(
t,
"QueryNameserverIP(a.root-servers.net, example.com)",
func(ctx context.Context) error {
var err error
resp, err = r.QueryNameserverIP(
ctx, "a.root-servers.net.", "198.41.0.4",
"example.com",
)
if err != nil {
return err
}
// A timeout or a network error is no reply at all.
if resp.Status == resolver.StatusTimeout ||
strings.HasPrefix(resp.Error, "network error") {
return fmt.Errorf(
"%w: %s", livednstest.ErrNoAnswer, resp.Error,
)
}
return nil
},
)
assert.Equal(t, resolver.StatusError, resp.Status)
assert.Equal(t, "server returned a referral", resp.Error)
ips, _, err := resolver.CollectIPs(
map[string]*resolver.NameserverResponse{resp.Nameserver: resp},
)
require.ErrorIs(t, err, resolver.ErrNoNameserverAnswered)
assert.Empty(t, ips)
}
func TestResolveIPAddresses_ContextCanceled(t *testing.T) { func TestResolveIPAddresses_ContextCanceled(t *testing.T) {
t.Parallel() t.Parallel()
+4 -3
View File
@@ -322,9 +322,10 @@ func (w *Watcher) detectNSChanges(
} }
// resolveNameserverAddresses returns the sorted addresses each // resolveNameserverAddresses returns the sorted addresses each
// nameserver's name resolves to. A nameserver whose lookup fails, as it // nameserver's name resolves to. A nameserver whose lookup fails or
// does when no nameserver of the name's zone answers, or finds no // finds no address keeps its addresses from prev: the resolver finds no
// address keeps its addresses from prev and is not an address change. // address, without an error, when every server it asks times out, and
// that is not an address change.
func (w *Watcher) resolveNameserverAddresses( func (w *Watcher) resolveNameserverAddresses(
ctx context.Context, ctx context.Context,
nameservers []string, nameservers []string,
-9
View File
@@ -1,9 +0,0 @@
{
"name": "dnswatcher-tooling",
"version": "0.0.0",
"private": true,
"description": "Pins the prettier that script/fmt and script/fmt-check run against this repo's markdown. Not a JavaScript project; nothing here is imported, published, or shipped.",
"devDependencies": {
"prettier": "3.9.6"
}
}
+6 -8
View File
@@ -3,12 +3,11 @@
# this repo. Idempotent: every install is guarded by a check so already # this repo. Idempotent: every install is guarded by a check so already
# installed tools are skipped. Base tooling comes from nix, apt, brew, # installed tools are skipped. Base tooling comes from nix, apt, brew,
# or apk (detected in that order); assumes nothing is present. # or apk (detected in that order); assumes nothing is present.
# goimports is not installed here: script/fmt and script/fmt-check-go # goimports is not installed here: script/fmt and script/fmt-check run
# run it with `go run` at a pinned commit. # it with `go run` at a pinned commit.
# The linter is NOT installed here: golangci-lint runs via docker only # The linter is NOT installed here: golangci-lint runs via docker only
# (script/lint), pinned by image digest, so its only prerequisite is a # (script/lint), pinned by image digest, so its only prerequisite is a
# working docker. Nor is prettier: script/fmt and # working docker.
# script/fmt-check-markdown run it in a container from Dockerfile.fmt.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
@@ -67,12 +66,11 @@ main() {
if missing make; then pkg_install gnumake make make make; fi if missing make; then pkg_install gnumake make make make; fi
if missing go; then pkg_install go golang go go; fi if missing go; then pkg_install go golang go go; fi
# Linting and the markdown formatter run via docker only. Warn, # Linting runs via docker only (script/lint). Warn, don't fail:
# don't fail: building and testing work without it. # everything except `make lint` works without it.
if missing docker; then if missing docker; then
echo "bootstrap: WARNING: docker not found; install it to" \ echo "bootstrap: WARNING: docker not found; install it to" \
"run make lint, make fmt, make fmt-check, make check" \ "run make lint and make docker." >&2
"and make docker." >&2
fi fi
go mod download go mod download
+4 -9
View File
@@ -1,23 +1,18 @@
#!/bin/sh #!/bin/sh
# script/cibuild: run the CI build. The Dockerfile's lint stage runs # script/cibuild: run the CI build. The Dockerfile's lint stage runs
# the Go half of make fmt-check and golangci-lint; its builder stage # make fmt-check and golangci-lint; its builder stage runs make test
# runs make test and make build. The markdown half of make fmt-check # and make build.
# runs after that build, as its own build of Dockerfile.fmt, because
# there is no docker inside a docker build.
# #
# --no-cache-filter=lint,builder runs both stages on every invocation; # --no-cache-filter=lint,builder runs both stages on every invocation;
# otherwise an unchanged tree is served from the layer cache and passes # otherwise an unchanged tree is served from the layer cache and passes
# without linting or querying live DNS. script/fmt-check-markdown busts # without linting or querying live DNS.
# its own cache the same way.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
ROOT="$(cd "$SCRIPT_DIR/.." && pwd -P)"
main() { main() {
cd "$ROOT" cd "$ROOT"
docker build --no-cache-filter=lint,builder . docker build --no-cache-filter=lint,builder .
"$SCRIPT_DIR/fmt-check-markdown"
} }
main "$@" main "$@"
+2 -48
View File
@@ -1,65 +1,19 @@
#!/bin/sh #!/bin/sh
# script/fmt: format all files (writes). Go with gofmt and goimports on # script/fmt: format all files (writes).
# the host, markdown with the prettier pinned by Dockerfile.fmt.
# #
# goimports runs with `go run` at a pinned commit, never from PATH, so # goimports runs with `go run` at a pinned commit, never from PATH, so
# every machine formats with the same version and nothing installs it. # every machine formats with the same version and nothing installs it.
#
# The markdown pass is a `docker build --output type=local` rather than a
# `docker run -v`, so it needs no bind mount and behaves the same against
# a remote daemon; the formatted documents come back out of the build and
# are copied over the tree here.
#
# Unlike script/fmt-check-markdown this does not bust the cache: it is
# not a gate, and any edit to a document changes the COPY layer above the
# prettier step, so a cached result is a result over this exact tree.
set -eu set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# goimports v0.42.0, 2026-08-07. Must match script/fmt-check-go. # goimports v0.42.0, 2026-08-07. Must match script/fmt-check.
GOIMPORTS_REF="golang.org/x/tools/cmd/goimports@009367f5c17a8d4c45a961a3a509277190a9a6f0" GOIMPORTS_REF="golang.org/x/tools/cmd/goimports@009367f5c17a8d4c45a961a3a509277190a9a6f0"
# Must match the export stage name in Dockerfile.fmt.
stage=fmt-out
die() {
echo "script/fmt: $*" >&2
exit 1
}
main() { main() {
cd "$ROOT" cd "$ROOT"
gofmt -s -w . gofmt -s -w .
go run "$GOIMPORTS_REF" -w . go run "$GOIMPORTS_REF" -w .
tmp="$(mktemp -d "${TMPDIR:-/tmp}/dnswatcher-fmt.XXXXXX")"
trap 'rm -rf "$tmp"' EXIT INT TERM
docker build \
--target "$stage" \
--output "type=local,dest=$tmp/out" \
-f Dockerfile.fmt .
# An empty export means prettier was handed nothing, which must not
# read as "already formatted".
(cd "$tmp/out" && find . -type f -name '*.md') |
sed 's|^\./||' | LC_ALL=C sort >"$tmp/files"
[ -s "$tmp/files" ] ||
die "the formatting build produced no markdown; the build" \
"context reached prettier empty"
# Copied only where the bytes differ, so an already-formatted tree
# keeps its timestamps and says nothing.
while IFS= read -r f; do
[ -n "$f" ] || continue
if [ -f "$f" ] && cmp -s "$tmp/out/$f" "$f"; then
continue
fi
cp "$tmp/out/$f" "$f"
echo "prettier: reformatted $f"
done <"$tmp/files"
} }
main "$@" main "$@"
+18 -5
View File
@@ -1,14 +1,27 @@
#!/bin/sh #!/bin/sh
# script/fmt-check: check formatting (read-only). Same tools and scope # script/fmt-check: check formatting (read-only). Same tools and scope
# as script/fmt, but fails instead of writing: the Go on the host, the # as script/fmt, but fails instead of writing.
# markdown with prettier in a container.
set -eu set -eu
SCRIPT_DIR="$(cd "$(dirname "$0")" && pwd -P)" ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# goimports v0.42.0, 2026-08-07. Must match script/fmt.
GOIMPORTS_REF="golang.org/x/tools/cmd/goimports@009367f5c17a8d4c45a961a3a509277190a9a6f0"
main() { main() {
"$SCRIPT_DIR/fmt-check-go" cd "$ROOT"
"$SCRIPT_DIR/fmt-check-markdown" files="$(gofmt -s -l .)"
if [ -n "$files" ]; then
echo "gofmt: files not formatted:" >&2
echo "$files" >&2
exit 1
fi
files="$(go run "$GOIMPORTS_REF" -l .)"
if [ -n "$files" ]; then
echo "goimports: files not formatted:" >&2
echo "$files" >&2
exit 1
fi
} }
main "$@" main "$@"
-31
View File
@@ -1,31 +0,0 @@
#!/bin/sh
# script/fmt-check-go: fail unless every Go source is formatted the way
# script/fmt would leave it, and name the files that are not. Read-only.
#
# Its own script because the Dockerfile's lint stage runs this half
# alone: there is no docker inside a docker build to run the markdown
# half in.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# goimports v0.42.0, 2026-08-07. Must match script/fmt.
GOIMPORTS_REF="golang.org/x/tools/cmd/goimports@009367f5c17a8d4c45a961a3a509277190a9a6f0"
main() {
cd "$ROOT"
files="$(gofmt -s -l .)"
if [ -n "$files" ]; then
echo "gofmt: files not formatted:" >&2
echo "$files" >&2
exit 1
fi
files="$(go run "$GOIMPORTS_REF" -l .)"
if [ -n "$files" ]; then
echo "goimports: files not formatted:" >&2
echo "$files" >&2
exit 1
fi
}
main "$@"
-29
View File
@@ -1,29 +0,0 @@
#!/bin/sh
# script/fmt-check-markdown: fail unless every .md is formatted the way
# script/fmt would leave it. Read-only.
#
# prettier is never installed on the host: it runs in a container built
# from Dockerfile.fmt, pinned by package.json and yarn.lock.
# --no-cache-filter is here for the reason script/lint gives: a cached
# build checks nothing.
#
# Its own script because script/cibuild runs this half alone, after the
# Dockerfile's lint stage has checked the Go.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
# Must match the markdown check stage name in Dockerfile.fmt.
stage=fmt-check
main() {
cd "$ROOT"
docker build \
--progress=plain \
--no-cache-filter="$stage" \
--target "$stage" \
-f Dockerfile.fmt \
.
}
main "$@"
-8
View File
@@ -1,8 +0,0 @@
# THIS IS AN AUTOGENERATED FILE. DO NOT EDIT THIS FILE DIRECTLY.
# yarn lockfile v1
prettier@3.9.6:
version "3.9.6"
resolved "https://registry.yarnpkg.com/prettier/-/prettier-3.9.6.tgz#b3ea5146515d40fc53f18aa63f74dfab1e10dbf6"
integrity sha512-OpN0zzVdiaiAhxpuuj5efpIS4sY9j7bY6uR5mnj5yPzGkdkjNKSJeUThPb60Jw29QuAZgA4o+/iB49kFiaBX6g==