sneak/dnswatcher
1
0
Fork 0
Code Issues 4 Pull Requests 1 Actions 5 Packages Projects Releases Wiki Activity

fmt: format and check Markdown with prettier in a container (closes #119)
check / check (push) Failing after 2m11s
Details

Browse Source 
make fmt and make fmt-check now cover every Markdown file with prettier
(4-space tabs, proseWrap always), as template-app-go does: prettier is
pinned by package.json and yarn.lock and runs in a docker build on a
digest-pinned node image, never on the host. The check is forced to run
with --no-cache-filter, as script/lint is.

script/fmt-check is split into a Go half and a Markdown half because the
Dockerfile lint stage cannot run docker: that stage now runs the Go half
and script/cibuild runs the Markdown half after the build. *.md leaves
.dockerignore so documents reach the build context.

README.md, TESTING.md and TODO.md are reformatted by make fmt; apart from
the README Entrypoints entries, its make fmt line under Building and the
TODO entry, that diff is mechanical.

Model: opus-5-5
This commit was merged in pull request #196.
This commit is contained in:
clawbot
2026-10-02 00:19:30 +02:00
parent b5814b2451
commit 4c2932d6d6
17 changed files with 602 additions and 417 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+346 -346
View File
@@ -1,16 +1,20 @@
# dnswatcher
dnswatcher is an MIT-licensed, pre-1.0 Go daemon by [@sneak](https://sneak.berlin) that monitors DNS records, TCP port availability, and TLS certificates, delivering real-time change notifications via Slack, Mattermost, and ntfy webhooks.
dnswatcher is an MIT-licensed, pre-1.0 Go daemon by
[@sneak](https://sneak.berlin) that monitors DNS records, TCP port availability,
and TLS certificates, delivering real-time change notifications via Slack,
Mattermost, and ntfy webhooks.
> ⚠️ Pre-1.0 software. APIs, configuration, and behavior may change without notice.
> ⚠️ Pre-1.0 software. APIs, configuration, and behavior may change without
> notice.
dnswatcher watches configured DNS domains and hostnames for changes, monitors TCP
port availability, tracks TLS certificate expiry, and delivers real-time
dnswatcher watches configured DNS domains and hostnames for changes, monitors
TCP port availability, tracks TLS certificate expiry, and delivers real-time
notifications via Slack, Mattermost, and/or ntfy webhooks.
It performs all DNS resolution itself via iterative (non-recursive) queries,
tracing from root nameservers to authoritative servers directly—never relying
on upstream recursive resolvers.
tracing from root nameservers to authoritative servers directly—never relying on
upstream recursive resolvers.
State is persisted to a local JSON file so that monitoring survives restarts
without requiring an external database.
@@ -19,21 +23,20 @@ without requiring an external database.
## No DNS mocking. Ever.
**DNS is never mocked in this project — not in tests, not anywhere else.**
No mock resolvers, no fake DNS servers, no stubbed lookups.
**DNS is never mocked in this project — not in tests, not anywhere else.** No
mock resolvers, no fake DNS servers, no stubbed lookups.
dnswatcher's entire purpose is correct behavior against the real DNS.
Tests exercise real iterative resolution against live nameservers by
design; a test suite that passes against a mock proves nothing about the
one thing this program exists to do.
dnswatcher's entire purpose is correct behavior against the real DNS. Tests
exercise real iterative resolution against live nameservers by design; a test
suite that passes against a mock proves nothing about the one thing this program
exists to do.
When live tests are flaky, that is a robustness problem, and it gets
fixed with robustness: retries with backoff, querying multiple
independent nameservers, longer timeouts — or explicit opt-in gating
decided by the project owner. Never with mocks.
When live tests are flaky, that is a robustness problem, and it gets fixed with
robustness: retries with backoff, querying multiple independent nameservers,
longer timeouts — or explicit opt-in gating decided by the project owner. Never
with mocks.
Contributions that introduce mocked, faked, or stubbed DNS will be
rejected.
Contributions that introduce mocked, faked, or stubbed DNS will be rejected.
---
@@ -46,126 +49,123 @@ rejected.
- Every **1 hour**, performs a full iterative trace from root servers to
discover all authoritative nameservers (NS records) for each domain.
- Queries **every** discovered authoritative nameserver independently.
- Stores the NS record set as observed by the delegation chain, and the
IPv4 and IPv6 addresses each nameserver's name resolves to.
- Stores the NS record set as observed by the delegation chain, and the IPv4 and
IPv6 addresses each nameserver's name resolves to.
- Any change triggers a notification:
- NS added to or removed from the delegation.
- NS address change: a nameserver that stays in the delegation
resolves to different addresses than on the previous check. A
nameserver added or removed gets only the NS change notification.
When the lookup of a nameserver's addresses fails or finds none,
its previous addresses are kept and nothing is sent.
- NS added to or removed from the delegation.
- NS address change: a nameserver that stays in the delegation resolves to
different addresses than on the previous check. A nameserver added or
removed gets only the NS change notification. When the lookup of a
nameserver's addresses fails or finds none, its previous addresses are
kept and nothing is sent.
### DNS Hostname Monitoring (Subdomains)
- Accepts a list of DNS hostnames (subdomains, distinguished from apex
domains via the Public Suffix List).
- Accepts a list of DNS hostnames (subdomains, distinguished from apex domains
via the Public Suffix List).
- Every **1 hour**, performs a full iterative trace to discover the
authoritative nameservers of the zone the hostname is in, which is not
always its last two labels (a name under `co.uk`, or in a delegated
subdomain).
- Queries **each** authoritative nameserver independently for **all**
record types: A, AAAA, CNAME, MX, TXT, SRV, CAA, NS.
- Stores results **per nameserver**. The state for a hostname is not a
merged view — it is a map from nameserver to record set.
- DNS names inside record values (CNAME, MX, SRV and NS targets) are
stored in lower case, because names are case-insensitive and
nameservers may answer in any letter case. TXT and CAA values keep
their letter case; they are not lower-cased.
- Any observable change in any nameserver's response triggers a
notification. This includes:
- **Record change**: A nameserver returns different records than it
did on the previous check (additions, removals, value changes).
- **NS query failure**: A nameserver that previously responded
becomes unreachable (timeout, SERVFAIL, REFUSED, network error).
This is distinct from "responded with no records": a nameserver
that answers NXDOMAIN or with no records has responded. The alert
is sent once, on the check where it starts failing. A failing
nameserver gives no records, so it is not reported as a record
change or compared for inconsistency. A nameserver that is already
failing on the first check that sees it is recorded silently.
- **NS recovery**: A previously-unreachable nameserver starts
responding again. Its records are not compared with those from
before it failed, so a change made while it was failing is not
reported as a record change.
- **Inconsistency detected**: Two nameservers return different record
sets for the same hostname and did not already differ on the previous
check. Every pair of nameservers is compared. The alert is sent once
for each such pair, on the check where they start to disagree, and not
again while they keep disagreeing, including after a restart. A
nameserver that was not in the previous check (newly added, or back
after dropping out), or failed on it, and answers differently is
reported on the check where it answers. If a pair agrees again and
later disagrees, the alert is sent again.
authoritative nameservers of the zone the hostname is in, which is not always
its last two labels (a name under `co.uk`, or in a delegated subdomain).
- Queries **each** authoritative nameserver independently for **all** record
types: A, AAAA, CNAME, MX, TXT, SRV, CAA, NS.
- Stores results **per nameserver**. The state for a hostname is not a merged
view — it is a map from nameserver to record set.
- DNS names inside record values (CNAME, MX, SRV and NS targets) are stored in
lower case, because names are case-insensitive and nameservers may answer in
any letter case. TXT and CAA values keep their letter case; they are not
lower-cased.
- Any observable change in any nameserver's response triggers a notification.
This includes:
- **Record change**: A nameserver returns different records than it did on
the previous check (additions, removals, value changes).
- **NS query failure**: A nameserver that previously responded becomes
unreachable (timeout, SERVFAIL, REFUSED, network error). This is distinct
from "responded with no records": a nameserver that answers NXDOMAIN or
with no records has responded. The alert is sent once, on the check where
it starts failing. A failing nameserver gives no records, so it is not
reported as a record change or compared for inconsistency. A nameserver
that is already failing on the first check that sees it is recorded
silently.
- **NS recovery**: A previously-unreachable nameserver starts responding
again. Its records are not compared with those from before it failed, so a
change made while it was failing is not reported as a record change.
- **Inconsistency detected**: Two nameservers return different record sets
for the same hostname and did not already differ on the previous check.
Every pair of nameservers is compared. The alert is sent once for each
such pair, on the check where they start to disagree, and not again while
they keep disagreeing, including after a restart. A nameserver that was
not in the previous check (newly added, or back after dropping out), or
failed on it, and answers differently is reported on the check where it
answers. If a pair agrees again and later disagrees, the alert is sent
again.
### TCP Port Monitoring
- For every configured domain and hostname, constructs a deduplicated list
of all IPv4 and IPv6 addresses resolved via A, AAAA, and CNAME chain
resolution across all authoritative nameservers.
- For every configured domain and hostname, constructs a deduplicated list of
all IPv4 and IPv6 addresses resolved via A, AAAA, and CNAME chain resolution
across all authoritative nameservers.
- Checks TCP connectivity on ports **80** and **443** for each IP address.
- Every **1 hour**, re-checks all ports.
- Any change in port availability triggers a notification:
- Port transitioned from open to closed (or vice versa).
- New IP appeared (from DNS change) and its port state was recorded.
- IP disappeared (from DNS change) — noted in the DNS change
notification; port state for that IP is removed.
- Port transitioned from open to closed (or vice versa).
- New IP appeared (from DNS change) and its port state was recorded.
- IP disappeared (from DNS change) — noted in the DNS change notification;
port state for that IP is removed.
### TLS Certificate Monitoring
- Every **12 hours**, for each IP address listening on port 443, connects
via TLS using the correct SNI hostname.
- Every **12 hours**, for each IP address listening on port 443, connects via
TLS using the correct SNI hostname.
- Records the certificate's Subject CN, SANs, issuer, and expiry date.
- Any change triggers a notification:
- Certificate is expiring within **7 days** (warning, repeated each
check until renewed or expired).
- Certificate CN, issuer, or SANs changed (replacement detected,
reports old and new values).
- TLS connection failure to a previously-reachable IP:443 (handshake
error, timeout, connection refused after previously succeeding).
- TLS recovery: a previously-failing IP:443 now completes a
handshake again.
- Certificate is expiring within **7 days** (warning, repeated each check
until renewed or expired).
- Certificate CN, issuer, or SANs changed (replacement detected, reports old
and new values).
- TLS connection failure to a previously-reachable IP:443 (handshake error,
timeout, connection refused after previously succeeding).
- TLS recovery: a previously-failing IP:443 now completes a handshake again.
### Notifications
**Every observable state change produces a notification.** dnswatcher is
designed as a real-time change feed — degradations, failures, recoveries,
and routine changes are all reported equally.
designed as a real-time change feed — degradations, failures, recoveries, and
routine changes are all reported equally.
Supported notification backends:
| Backend | Configuration | Payload Format |
|----------------|--------------------------|------------------------------|
| **Slack** | Incoming Webhook URL | Attachments with color |
| **Mattermost** | Incoming Webhook URL | Slack-compatible attachments |
| **ntfy** | Topic URL (e.g. `https://ntfy.sh/mytopic`) | Title + body + priority |
| Backend | Configuration | Payload Format |
| -------------- | ------------------------------------------ | ---------------------------- |
| **Slack** | Incoming Webhook URL | Attachments with color |
| **Mattermost** | Incoming Webhook URL | Slack-compatible attachments |
| **ntfy** | Topic URL (e.g. `https://ntfy.sh/mytopic`) | Title + body + priority |
All configured endpoints receive every notification. Notification content
includes:
- **DNS record changes**: Which hostname, which nameserver, what record
type, old values, new values.
- **DNS record changes**: Which hostname, which nameserver, what record type,
old values, new values.
- **DNS NS changes**: Which domain, which nameservers were added/removed.
- **NS address changes**: Which domain, which nameserver, its old and
new addresses.
- **NS query failures**: Which nameserver failed, error type (timeout,
SERVFAIL, REFUSED, network error), which hostname/domain affected.
- **NS address changes**: Which domain, which nameserver, its old and new
addresses.
- **NS query failures**: Which nameserver failed, error type (timeout, SERVFAIL,
REFUSED, network error), which hostname/domain affected.
- **NS recoveries**: Which nameserver recovered, which hostname/domain.
- **NS inconsistencies**: Which nameservers disagree, what each one
returned, which hostname affected.
- **NS inconsistencies**: Which nameservers disagree, what each one returned,
which hostname affected.
- **Port changes**: Which IP:port, old state, new state, all associated
hostnames.
- **TLS expiry warnings**: Which certificate, days remaining, CN,
issuer, associated hostname and IP.
- **TLS certificate changes**: Old and new CN/issuer/SANs, associated
hostname and IP.
- **TLS expiry warnings**: Which certificate, days remaining, CN, issuer,
associated hostname and IP.
- **TLS certificate changes**: Old and new CN/issuer/SANs, associated hostname
and IP.
- **TLS connection failures/recoveries**: Which IP:port, error details,
associated hostname.
### State Management
- All monitoring state is kept in memory and persisted to a JSON file on
disk (`DATA_DIR/state.json`).
- All monitoring state is kept in memory and persisted to a JSON file on disk
(`DATA_DIR/state.json`).
- State is loaded on startup to resume monitoring without triggering
false-positive change notifications.
- State is written atomically (write to temp file, then rename) to prevent
@@ -173,41 +173,39 @@ includes:
### Web Dashboard
dnswatcher includes an unauthenticated, read-only web dashboard at the
root URL (`/`). It displays:
dnswatcher includes an unauthenticated, read-only web dashboard at the root URL
(`/`). It displays:
- **Summary counts** for monitored domains, hostnames, ports, and
certificates.
- **Summary counts** for monitored domains, hostnames, ports, and certificates.
- **Domains** with their discovered nameservers.
- **Hostnames** with per-nameserver DNS records and status.
- **Ports** with open/closed state and associated hostnames.
- **TLS certificates** with CN, issuer, expiry, and status.
- **Recent alerts** (last 100 notifications sent since the process
started), displayed in reverse chronological order.
- **Recent alerts** (last 100 notifications sent since the process started),
displayed in reverse chronological order.
Every data point shows its age (e.g. "5m ago") so you can tell at a
glance how fresh the information is. The page auto-refreshes every 30
seconds.
Every data point shows its age (e.g. "5m ago") so you can tell at a glance how
fresh the information is. The page auto-refreshes every 30 seconds.
The dashboard intentionally does not expose any configuration details
such as webhook URLs, notification endpoints, or API tokens.
The dashboard intentionally does not expose any configuration details such as
webhook URLs, notification endpoints, or API tokens.
All assets (CSS) are embedded in the binary and served from the
application itself. The dashboard makes zero external HTTP requests —
no CDN dependencies or third-party resources are loaded at runtime.
All assets (CSS) are embedded in the binary and served from the application
itself. The dashboard makes zero external HTTP requests — no CDN dependencies or
third-party resources are loaded at runtime.
### HTTP API
dnswatcher exposes a lightweight HTTP API for operational visibility:
| Endpoint | Description |
|---------------------------------------|--------------------------------|
| `GET /` | Web dashboard (HTML) |
| `GET /s/...` | Static assets (embedded CSS) |
| `GET /.well-known/healthcheck` | Health check (JSON) |
| `GET /health` | Health check (JSON, legacy) |
| `GET /api/v1/status` | Current monitoring state |
| `GET /metrics` | Prometheus metrics (optional) |
| Endpoint | Description |
| ------------------------------ | ----------------------------- |
| `GET /` | Web dashboard (HTML) |
| `GET /s/...` | Static assets (embedded CSS) |
| `GET /.well-known/healthcheck` | Health check (JSON) |
| `GET /health` | Health check (JSON, legacy) |
| `GET /api/v1/status` | Current monitoring state |
| `GET /metrics` | Prometheus metrics (optional) |
#### Server timeouts
@@ -216,7 +214,7 @@ constants in `internal/server/server.go`, not configurable via environment
variables.
| Timeout | Value | Purpose |
|---------------------|-------|-----------------------------------------------|
| ------------------- | ----- | --------------------------------------------- |
| `ReadHeaderTimeout` | 10s | Bounds the request header read (slowloris) |
| `ReadTimeout` | 15s | Bounds the whole request read, headers + body |
| `WriteTimeout` | 75s | Bounds handler execution plus response flush |
@@ -224,20 +222,20 @@ variables.
These are distinct from the 60s per-request handler budget applied by
`chimw.Timeout` in `internal/server/routes.go`, which cancels the request
context but does not touch the socket. `WriteTimeout` is deliberately
larger than that budget: the write deadline is armed once request headers
are read, so a smaller value would sever the connection before a handler
using its full budget could respond. `IdleTimeout` exceeds common
Prometheus scrape intervals so the scraper reuses its connection.
context but does not touch the socket. `WriteTimeout` is deliberately larger
than that budget: the write deadline is armed once request headers are read, so
a smaller value would sever the connection before a handler using its full
budget could respond. `IdleTimeout` exceeds common Prometheus scrape intervals
so the scraper reuses its connection.
### Security Headers
Every response — the dashboard, the static assets under `/s/...`, the
healthchecks, the JSON API, and `/metrics` — carries the following
headers, set by a global middleware:
healthchecks, the JSON API, and `/metrics` — carries the following headers, set
by a global middleware:
| Header | Value |
|-----------------------------|---------------------------------------|
| --------------------------- | ------------------------------------- |
| `Strict-Transport-Security` | `max-age=31536000; includeSubDomains` |
| `Content-Security-Policy` | see below |
| `X-Frame-Options` | `DENY` |
@@ -254,21 +252,21 @@ form-action 'none'; frame-ancestors 'none'
```
The dashboard ships no JavaScript (the 30-second refresh is a
`<meta http-equiv="refresh">`), no inline styles, no inline event
handlers, and no images; its only subresource is the embedded stylesheet
at `/s/css/tailwind.min.css`, which `style-src 'self'` permits. The
policy therefore needs neither `unsafe-inline` nor `unsafe-eval`.
`<meta http-equiv="refresh">`), no inline styles, no inline event handlers, and
no images; its only subresource is the embedded stylesheet at
`/s/css/tailwind.min.css`, which `style-src 'self'` permits. The policy
therefore needs neither `unsafe-inline` nor `unsafe-eval`.
`frame-ancestors 'none'` is the primary anti-framing control, with
`X-Frame-Options: DENY` retained as the legacy fallback.
HSTS is emitted unconditionally, including over plain HTTP. dnswatcher is
expected to run behind a TLS-terminating reverse proxy, and the browser
must still be told to enforce HTTPS end to end, so the header is never
gated on whether the request itself arrived over TLS.
expected to run behind a TLS-terminating reverse proxy, and the browser must
still be told to enforce HTTPS end to end, so the header is never gated on
whether the request itself arrived over TLS.
`Referrer-Policy: no-referrer` is stricter than the
`strict-origin-when-cross-origin` baseline: the dashboard has no
cross-origin navigation needs, and its URL may name internal hosts.
`strict-origin-when-cross-origin` baseline: the dashboard has no cross-origin
navigation needs, and its URL may name internal hosts.
---
@@ -301,23 +299,23 @@ internal/
### Design Principles
- **No recursive resolvers**: All DNS resolution is performed iteratively,
tracing from root nameservers through the delegation chain to
authoritative servers.
tracing from root nameservers through the delegation chain to authoritative
servers.
- **No external database**: State is persisted as a single JSON file.
- **Dependency injection**: All components are wired via
[uber/fx](https://github.com/uber-go/fx).
- **Structured logging**: All logs use `log/slog` with JSON output in
production (TTY detection for development).
- **Graceful shutdown**: All background goroutines respect context
cancellation and the fx lifecycle. In-flight notification deliveries
are drained on shutdown, bounded by the shutdown timeout.
- **Structured logging**: All logs use `log/slog` with JSON output in production
(TTY detection for development).
- **Graceful shutdown**: All background goroutines respect context cancellation
and the fx lifecycle. In-flight notification deliveries are drained on
shutdown, bounded by the shutdown timeout.
---
## Configuration
Configuration is loaded via [Viper](https://github.com/spf13/viper) with
the following precedence (highest to lowest):
Configuration is loaded via [Viper](https://github.com/spf13/viper) with the
following precedence (highest to lowest):
1. Environment variables (prefixed with `DNSWATCHER_`)
2. `.env` file (loaded via godotenv)
@@ -327,46 +325,45 @@ the following precedence (highest to lowest):
### Environment Variables
| Variable | Description | Default |
|---------------------------------|--------------------------------------------|-------------|
| `PORT` | HTTP listen port | `8080` |
| `DNSWATCHER_DEBUG` | Enable debug logging | `false` |
| `DNSWATCHER_DATA_DIR` | Directory for state file | `/var/lib/dnswatcher` |
| `DNSWATCHER_TARGETS` | Comma-separated DNS names (auto-classified via PSL) | `""` |
| `DNSWATCHER_SLACK_WEBHOOK` | Slack incoming webhook URL | `""` |
| `DNSWATCHER_MATTERMOST_WEBHOOK` | Mattermost incoming webhook URL | `""` |
| `DNSWATCHER_NTFY_TOPIC` | ntfy topic URL | `""` |
| `DNSWATCHER_DNS_INTERVAL` | DNS check interval, a positive duration such as `30m`; empty means the default, anything else stops startup | `1h` |
| `DNSWATCHER_TLS_INTERVAL` | TLS check interval, a positive duration such as `6h`; empty means the default, anything else stops startup | `12h` |
| `DNSWATCHER_TLS_EXPIRY_WARNING` | Days before expiry to warn | `7` |
| `DNSWATCHER_SENTRY_DSN` | Sentry DSN for error reporting | `""` |
| `DNSWATCHER_MAINTENANCE_MODE` | Enable maintenance mode | `false` |
| `DNSWATCHER_METRICS_USERNAME` | Basic auth username for /metrics | `""` |
| `DNSWATCHER_METRICS_PASSWORD` | Basic auth password for /metrics | `""` |
| `DNSWATCHER_SEND_TEST_NOTIFICATION` | Send a test notification after first scan completes | `false` |
| Variable | Description | Default |
| ----------------------------------- | ----------------------------------------------------------------------------------------------------------- | --------------------- |
| `PORT` | HTTP listen port | `8080` |
| `DNSWATCHER_DEBUG` | Enable debug logging | `false` |
| `DNSWATCHER_DATA_DIR` | Directory for state file | `/var/lib/dnswatcher` |
| `DNSWATCHER_TARGETS` | Comma-separated DNS names (auto-classified via PSL) | `""` |
| `DNSWATCHER_SLACK_WEBHOOK` | Slack incoming webhook URL | `""` |
| `DNSWATCHER_MATTERMOST_WEBHOOK` | Mattermost incoming webhook URL | `""` |
| `DNSWATCHER_NTFY_TOPIC` | ntfy topic URL | `""` |
| `DNSWATCHER_DNS_INTERVAL` | DNS check interval, a positive duration such as `30m`; empty means the default, anything else stops startup | `1h` |
| `DNSWATCHER_TLS_INTERVAL` | TLS check interval, a positive duration such as `6h`; empty means the default, anything else stops startup | `12h` |
| `DNSWATCHER_TLS_EXPIRY_WARNING` | Days before expiry to warn | `7` |
| `DNSWATCHER_SENTRY_DSN` | Sentry DSN for error reporting | `""` |
| `DNSWATCHER_MAINTENANCE_MODE` | Enable maintenance mode | `false` |
| `DNSWATCHER_METRICS_USERNAME` | Basic auth username for /metrics | `""` |
| `DNSWATCHER_METRICS_PASSWORD` | Basic auth password for /metrics | `""` |
| `DNSWATCHER_SEND_TEST_NOTIFICATION` | Send a test notification after first scan completes | `false` |
**`DNSWATCHER_TARGETS` is required.** dnswatcher will refuse to start if no
monitoring targets are configured. A monitoring daemon with nothing to monitor
is a misconfiguration, so dnswatcher fails fast with a clear error message
rather than running silently. Set `DNSWATCHER_TARGETS` to a comma-separated
list of DNS names before starting.
rather than running silently. Set `DNSWATCHER_TARGETS` to a comma-separated list
of DNS names before starting.
**`/metrics` is rate limited.** Each client address may send it 30 requests a
minute, failed logins included; beyond that it answers `429 Too Many Requests`
without checking the password. A Prometheus server scraping every 15 seconds
sends 4 a minute. IPv6 addresses in one /64 count as one client. When the
request comes from a private or loopback address, such as a reverse proxy's,
the client address is taken from the `X-Real-IP` header the proxy sets, or else
from `X-Forwarded-For`, as the last address in it that is not private or
loopback. A proxy that sets neither makes all its clients share one allowance.
request comes from a private or loopback address, such as a reverse proxy's, the
client address is taken from the `X-Real-IP` header the proxy sets, or else from
`X-Forwarded-For`, as the last address in it that is not private or loopback. A
proxy that sets neither makes all its clients share one allowance.
**`DNSWATCHER_DNS_INTERVAL` and `DNSWATCHER_TLS_INTERVAL`** take a positive
duration: a number followed by a unit such as `s`, `m` or `h`, for example
`90s`, `30m`, `1h` or `1h30m`. There is no unit for days; write `24h`. An
unset or empty variable (`DNSWATCHER_DNS_INTERVAL=`) means the default. If
either is set to anything else, including a bare number or a zero or negative
duration, dnswatcher refuses to start with an error naming the variable and
the value.
`90s`, `30m`, `1h` or `1h30m`. There is no unit for days; write `24h`. An unset
or empty variable (`DNSWATCHER_DNS_INTERVAL=`) means the default. If either is
set to anything else, including a bare number or a zero or negative duration,
dnswatcher refuses to start with an error naming the variable and the value.
**`DNSWATCHER_SENTRY_DSN` reports crashes in HTTP requests to Sentry.** When it
is set, a panic in an HTTP request handler is sent to Sentry, and the request
@@ -392,107 +389,107 @@ DNSWATCHER_SEND_TEST_NOTIFICATION=true
## DNS Resolution Strategy
dnswatcher never uses the system's configured recursive resolver. Instead,
it performs full iterative resolution:
dnswatcher never uses the system's configured recursive resolver. Instead, it
performs full iterative resolution:
1. **Root servers**: Starts from the IANA root nameserver list (hardcoded,
with periodic refresh).
1. **Root servers**: Starts from the IANA root nameserver list (hardcoded, with
periodic refresh).
2. **TLD delegation**: Queries root servers for the TLD NS records.
3. **Domain delegation**: Queries TLD nameservers for the domain's NS
records.
4. **Authoritative query**: Queries all discovered authoritative
nameservers directly for the requested records.
3. **Domain delegation**: Queries TLD nameservers for the domain's NS records.
4. **Authoritative query**: Queries all discovered authoritative nameservers
directly for the requested records.
This approach ensures:
- Independence from any upstream resolver's cache or filtering.
- Ability to detect split-horizon or inconsistent responses across
authoritative servers.
- Ability to detect split-horizon or inconsistent responses across authoritative
servers.
- Visibility into the full delegation chain.
For hostname monitoring, the resolver follows CNAME chains (with a
depth limit to prevent loops) before collecting terminal A/AAAA records.
For hostname monitoring, the resolver follows CNAME chains (with a depth limit
to prevent loops) before collecting terminal A/AAAA records.
---
## State File Format
The state file (`DATA_DIR/state.json`) contains the complete monitoring
snapshot. Hostname records are stored **per authoritative nameserver**,
not as a merged view, to enable inconsistency detection.
snapshot. Hostname records are stored **per authoritative nameserver**, not as a
merged view, to enable inconsistency detection.
```json
{
"version": 1,
"lastUpdated": "2026-02-19T12:00:00Z",
"domains": {
"example.com": {
"nameservers": ["ns1.example.com.", "ns2.example.com."],
"nameserverAddresses": {
"ns1.example.com.": ["192.0.2.53", "2001:db8::53"],
"ns2.example.com.": ["198.51.100.53"]
},
"lastChecked": "2026-02-19T12:00:00Z"
}
},
"hostnames": {
"www.example.com": {
"recordsByNameserver": {
"ns1.example.com.": {
"records": {
"A": ["93.184.216.34"],
"AAAA": ["2606:2800:220:1:248:1893:25c8:1946"]
},
"status": "ok",
"lastChecked": "2026-02-19T12:00:00Z"
},
"ns2.example.com.": {
"records": {
"A": ["93.184.216.34"],
"AAAA": ["2606:2800:220:1:248:1893:25c8:1946"]
},
"status": "ok",
"lastChecked": "2026-02-19T12:00:00Z"
"version": 1,
"lastUpdated": "2026-02-19T12:00:00Z",
"domains": {
"example.com": {
"nameservers": ["ns1.example.com.", "ns2.example.com."],
"nameserverAddresses": {
"ns1.example.com.": ["192.0.2.53", "2001:db8::53"],
"ns2.example.com.": ["198.51.100.53"]
},
"lastChecked": "2026-02-19T12:00:00Z"
}
},
"lastChecked": "2026-02-19T12:00:00Z"
}
},
"ports": {
"93.184.216.34:80": {
"open": true,
"hostnames": ["www.example.com"],
"lastChecked": "2026-02-19T12:00:00Z"
},
"93.184.216.34:443": {
"open": true,
"hostnames": ["www.example.com"],
"lastChecked": "2026-02-19T12:00:00Z"
"hostnames": {
"www.example.com": {
"recordsByNameserver": {
"ns1.example.com.": {
"records": {
"A": ["93.184.216.34"],
"AAAA": ["2606:2800:220:1:248:1893:25c8:1946"]
},
"status": "ok",
"lastChecked": "2026-02-19T12:00:00Z"
},
"ns2.example.com.": {
"records": {
"A": ["93.184.216.34"],
"AAAA": ["2606:2800:220:1:248:1893:25c8:1946"]
},
"status": "ok",
"lastChecked": "2026-02-19T12:00:00Z"
}
},
"lastChecked": "2026-02-19T12:00:00Z"
}
},
"ports": {
"93.184.216.34:80": {
"open": true,
"hostnames": ["www.example.com"],
"lastChecked": "2026-02-19T12:00:00Z"
},
"93.184.216.34:443": {
"open": true,
"hostnames": ["www.example.com"],
"lastChecked": "2026-02-19T12:00:00Z"
}
},
"certificates": {
"93.184.216.34:443:www.example.com": {
"commonName": "www.example.com",
"issuer": "DigiCert TLS RSA SHA256 2020 CA1",
"notAfter": "2027-01-15T23:59:59Z",
"subjectAlternativeNames": ["www.example.com"],
"status": "ok",
"lastChecked": "2026-02-19T06:00:00Z"
}
}
},
"certificates": {
"93.184.216.34:443:www.example.com": {
"commonName": "www.example.com",
"issuer": "DigiCert TLS RSA SHA256 2020 CA1",
"notAfter": "2027-01-15T23:59:59Z",
"subjectAlternativeNames": ["www.example.com"],
"status": "ok",
"lastChecked": "2026-02-19T06:00:00Z"
}
}
}
```
The `status` field for each per-nameserver entry and certificate entry
tracks reachability:
The `status` field for each per-nameserver entry and certificate entry tracks
reachability:
| Status | Meaning |
|-------------|------------------------------------------------------------|
| `ok` | Query succeeded, records are current |
| `error` | Query failed (timeout, SERVFAIL, REFUSED, network error) |
| Status | Meaning |
| ------- | -------------------------------------------------------- |
| `ok` | Query succeeded, records are current |
| `error` | Query failed (timeout, SERVFAIL, REFUSED, network error) |
A nameserver that answers NXDOMAIN or with no records has status `ok` and
empty `records`. A nameserver whose query failed has status `error`, empty
`records`, and the reason in `error`.
A nameserver that answers NXDOMAIN or with no records has status `ok` and empty
`records`. A nameserver whose query failed has status `error`, empty `records`,
and the reason in `error`.
`nameserverAddresses` lists, by nameserver, the sorted addresses its name
resolves to. A state file without it loads, and the next check fills it in
@@ -505,31 +502,38 @@ without a notification.
This repository adheres to the
[Scripts to Rule Them All](https://github.com/github/scripts-to-rule-them-all)
standard: normalized scripts in `script/` are the entrypoints for the
development workflow, and the Makefile targets are thin shims that call
them. We provide:
development workflow, and the Makefile targets are thin shims that call them. We
provide:
- `script/bootstrap` — install all dependencies (go, `go mod download`).
It does not install golangci-lint: see `script/lint` below.
- `script/setup` — make a fresh clone ready for development: bootstrap
plus the git pre-commit hook
- `script/projectname` — print the project name (used for the Docker
image tag)
- `script/test` — run the test suite (race detector, coverage). Caching
is waived for testing, exactly as it is for linting: `-count=1`
forces every invocation to execute, because the suite queries live
DNS and a cached pass queries nothing. Failures are rerun with `-v`
automatically, and the build fails even if that rerun passes.
- `script/bootstrap` — install all dependencies (go, `go mod download`). It does
not install golangci-lint or prettier: both run in Docker, see `script/lint`
and `script/fmt` below.
- `script/setup` — make a fresh clone ready for development: bootstrap plus the
git pre-commit hook
- `script/projectname` — print the project name (used for the Docker image tag)
- `script/test` — run the test suite (race detector, coverage). Caching is
waived for testing, exactly as it is for linting: `-count=1` forces every
invocation to execute, because the suite queries live DNS and a cached pass
queries nothing. Failures are rerun with `-v` automatically, and the build
fails even if that rerun passes.
- `script/lint` — run golangci-lint, always inside Docker: it builds
`Dockerfile.lint`, which COPYs the repo into the digest-pinned
`golangci-lint` image and lints as a build step, so a successful
build is a clean lint. The linter is never installed or run on the
host, and Docker is the only prerequisite. Caching is waived for
linting: the lint stage is forced to execute on every run with
`--no-cache-filter`, because a cached build lints nothing.
- `script/fmt` — format all code (gofmt -s, goimports). goimports runs
with `go run` at a pinned commit, never from your `PATH`.
- `script/fmt-check` — check formatting (read-only) with the same tools,
failing on any file `script/fmt` would change
`Dockerfile.lint`, which COPYs the repo into the digest-pinned `golangci-lint`
image and lints as a build step, so a successful build is a clean lint. The
linter is never installed or run on the host, and Docker is the only
prerequisite. Caching is waived for linting: the lint stage is forced to
execute on every run with `--no-cache-filter`, because a cached build lints
nothing.
- `script/fmt` — format all code (gofmt -s, goimports) and all Markdown
(prettier). goimports runs with `go run` at a pinned commit, never from your
`PATH`. prettier runs inside Docker, built from `Dockerfile.fmt` on a
digest-pinned node image, at the version pinned by `package.json` and
`yarn.lock`; it is never installed on the host.
- `script/fmt-check` — check formatting (read-only) with the same tools, failing
on any file `script/fmt` would change. It runs the two scripts below.
- `script/fmt-check-go` — the gofmt and goimports half, on the host. The
`Dockerfile` lint stage runs it.
- `script/fmt-check-markdown` — the prettier half, inside Docker, forced to
execute on every run with `--no-cache-filter`
- `script/check` — run test, lint, and fmt-check
- `script/docker` — build the Docker image tagged via `script/projectname`, with
`--no-cache-filter=lint,builder` so the lint stage and the builder stage,
@@ -538,9 +542,9 @@ them. We provide:
- `script/cibuild` — CI entrypoint: `docker build` with
`--no-cache-filter=lint,builder`, so the lint stage and the builder stage,
which runs the tests, run on every invocation, because a cached build lints
nothing and queries no DNS
- `script/precommit` — run by the git pre-commit hook; `go mod tidy`
guard, then `script/check`
nothing and queries no DNS; then `script/fmt-check-markdown`
- `script/precommit` — run by the git pre-commit hook; `go mod tidy` guard, then
`script/check`
- `script/install-precommit` — install the git pre-commit hook
## Building
@@ -549,21 +553,21 @@ them. We provide:
make build # Build binary to bin/dnswatcher
make test # Run tests with race detector
make lint # Run golangci-lint in Docker (requires docker)
make fmt # Format code
make fmt # Format code and Markdown (requires docker)
make check # Run all checks (test, lint, fmt-check)
make clean # Remove build artifacts
```
### Build-Time Variables
`make build` sets the version with `-ldflags "-X main.Version=..."`, taking
it from `git describe --tags --always --dirty`, or from `VERSION` when given
on the command line (`make build VERSION=1.2.3`). The version appears in the
startup log and in the health check response.
`make build` sets the version with `-ldflags "-X main.Version=..."`, taking it
from `git describe --tags --always --dirty`, or from `VERSION` when given on the
command line (`make build VERSION=1.2.3`). The version appears in the startup
log and in the health check response.
The Docker image has no `.git`, so the `Dockerfile` takes the version as
`--build-arg VERSION`. `make docker` passes it; a plain `docker build`
passes none, and that image reports `dev`.
`--build-arg VERSION`. `make docker` passes it; a plain `docker build` passes
none, and that image reports `dev`.
---
@@ -584,84 +588,80 @@ docker run -d \
## Running under upaas
[upaas](https://git.eeqj.de/sneak/upaas) builds the image from this
repository's `Dockerfile` and runs it. The app needs:
[upaas](https://git.eeqj.de/sneak/upaas) builds the image from this repository's
`Dockerfile` and runs it. The app needs:
- **Branch:** `prod`. `prod` is cut from `main`, and merging a `main` to
`prod` pull request is a deploy.
- **Volume:** one host directory mounted at `/var/lib/dnswatcher`, where
the state file lives.
- **Network and port:** the dashboard is unauthenticated and shows every
watched name and recent alert, and upaas publishes every mapped port on
all interfaces of the host
([upaas issue 113](https://git.eeqj.de/sneak/upaas/issues/113)). Add a
port mapping to container port `8080` only if the dashboard should be
public. Otherwise add none: set the app's Docker network in upaas to
your reverse proxy's Docker network, and the proxy reaches the app at
`upaas-` followed by the app name, port `8080`.
- **Required environment:** `DNSWATCHER_TARGETS`, a comma-separated list
of the domains and hostnames to watch. dnswatcher refuses to start
without it.
- **Branch:** `prod`. `prod` is cut from `main`, and merging a `main` to `prod`
pull request is a deploy.
- **Volume:** one host directory mounted at `/var/lib/dnswatcher`, where the
state file lives.
- **Network and port:** the dashboard is unauthenticated and shows every watched
name and recent alert, and upaas publishes every mapped port on all interfaces
of the host ([upaas issue 113](https://git.eeqj.de/sneak/upaas/issues/113)).
Add a port mapping to container port `8080` only if the dashboard should be
public. Otherwise add none: set the app's Docker network in upaas to your
reverse proxy's Docker network, and the proxy reaches the app at `upaas-`
followed by the app name, port `8080`.
- **Required environment:** `DNSWATCHER_TARGETS`, a comma-separated list of the
domains and hostnames to watch. dnswatcher refuses to start without it.
- **Recommended environment:** at least one notification endpoint
(`DNSWATCHER_SLACK_WEBHOOK`, `DNSWATCHER_MATTERMOST_WEBHOOK`,
`DNSWATCHER_NTFY_TOPIC`); without one, changes show only on the
dashboard. `DNSWATCHER_METRICS_USERNAME` and
`DNSWATCHER_METRICS_PASSWORD` serve `/metrics` behind basic auth.
`DNSWATCHER_NTFY_TOPIC`); without one, changes show only on the dashboard.
`DNSWATCHER_METRICS_USERNAME` and `DNSWATCHER_METRICS_PASSWORD` serve
`/metrics` behind basic auth.
- **Leave unset:** `DNSWATCHER_DATA_DIR`, which the image sets to
`/var/lib/dnswatcher`, and `PORT`, which defaults to `8080`. Every
setting comes from the environment; the image holds no config file.
- **Health check:** the image's own, which requests
`/.well-known/healthcheck` every 10 seconds. upaas reads the
container's health 60 seconds after a deploy and marks the deploy
failed unless it is `healthy`.
`/var/lib/dnswatcher`, and `PORT`, which defaults to `8080`. Every setting
comes from the environment; the image holds no config file.
- **Health check:** the image's own, which requests `/.well-known/healthcheck`
every 10 seconds. upaas reads the container's health 60 seconds after a deploy
and marks the deploy failed unless it is `healthy`.
---
## Monitoring Lifecycle
1. **Startup**: Check that the data directory can be written, and exit
with an error naming it if not. Load state from disk. If no state
file exists, start with empty state (first check will establish
baseline without triggering change notifications).
2. **Initial check**: Immediately perform all DNS, port, and TLS checks
on startup.
1. **Startup**: Check that the data directory can be written, and exit with an
error naming it if not. Load state from disk. If no state file exists, start
with empty state (first check will establish baseline without triggering
change notifications).
2. **Initial check**: Immediately perform all DNS, port, and TLS checks on
startup.
3. **Periodic checks** (DNS always runs first):
- DNS checks: every `DNSWATCHER_DNS_INTERVAL` (default 1h). Also
re-run before every TLS check cycle to ensure fresh IPs.
- Port checks: every `DNSWATCHER_DNS_INTERVAL`, after DNS completes.
- TLS checks: every `DNSWATCHER_TLS_INTERVAL` (default 12h), after
DNS completes.
- Port and TLS checks always use freshly resolved IP addresses from
the DNS phase that immediately precedes them — never stale IPs
from a previous cycle.
4. **On change detection**: Send notifications to all configured
endpoints, update in-memory state, persist to disk.
5. **Shutdown**: The watcher stops checking and saves the final state
to disk, and shutdown waits for that save before it goes on. Then it
waits for in-flight notification deliveries to complete. Both waits
share the fx shutdown timeout (15s by default): deliveries still
retrying against an unreachable endpoint when that expires are
abandoned, and the number abandoned is logged at warn level rather
than dropped silently. Notifications generated after shutdown has
begun are refused and logged, so a late burst cannot extend the
shutdown. A DNS lookup, port check or TLS check that shutdown cuts
short saves nothing and sends no notification.
- DNS checks: every `DNSWATCHER_DNS_INTERVAL` (default 1h). Also re-run
before every TLS check cycle to ensure fresh IPs.
- Port checks: every `DNSWATCHER_DNS_INTERVAL`, after DNS completes.
- TLS checks: every `DNSWATCHER_TLS_INTERVAL` (default 12h), after DNS
completes.
- Port and TLS checks always use freshly resolved IP addresses from the DNS
phase that immediately precedes them — never stale IPs from a previous
cycle.
4. **On change detection**: Send notifications to all configured endpoints,
update in-memory state, persist to disk.
5. **Shutdown**: The watcher stops checking and saves the final state to disk,
and shutdown waits for that save before it goes on. Then it waits for
in-flight notification deliveries to complete. Both waits share the fx
shutdown timeout (15s by default): deliveries still retrying against an
unreachable endpoint when that expires are abandoned, and the number
abandoned is logged at warn level rather than dropped silently. Notifications
generated after shutdown has begun are refused and logged, so a late burst
cannot extend the shutdown. A DNS lookup, port check or TLS check that
shutdown cuts short saves nothing and sends no notification.
---
## Planned Future Features (Post-1.0)
- **DNSSEC validation**: Validate the DNSSEC chain of trust during
iterative resolution and report DNSSEC failures as notifications.
- **DNSSEC validation**: Validate the DNSSEC chain of trust during iterative
resolution and report DNSSEC failures as notifications.
---
## Project Structure
Follows the conventions defined in `REPO_POLICIES.md`, adapted from the
[upaas](https://git.eeqj.de/sneak/upaas) project template. Uses uber/fx
for dependency injection, go-chi for HTTP routing, slog for logging, and
Viper for configuration.
[upaas](https://git.eeqj.de/sneak/upaas) project template. Uses uber/fx for
dependency injection, go-chi for HTTP routing, slog for logging, and Viper for
configuration.
---