ResolveIPAddresses, which a domain check runs for each nameserver, asked
every nameserver of the name's zone for all eight record types and read
only A, AAAA and CNAME. It now asks for those three, so a domain check
of example.com sends about a third fewer queries.
The per-attempt deadline in livednstest goes from 8 to 18 seconds. It
was sized for one lookup, while a watcher check sends over a hundred
queries in a row and on the CI runner ran past 8 seconds. Three attempts
still end under the 60-second cap.
A nameserver that answers none of the three queries now counts as not
answering even if it would have answered another type; the watcher keeps
the previous addresses either way.
Model: opus-5-5
Every resolution walked the root servers in a fixed order, so
a.root-servers.net got every first query and its timeouts were paid on
every lookup. Each list of servers the resolver walks is now walked in a
random order from rand.Shuffle, chosen anew each time; a server that does
not reply, refuses, or gives an error reply or a referral that leads no
closer is still passed over for the next. When a referral names a zone's
nameservers without their addresses, all of them are now looked up, not
only the first that resolves, so the zone is not given up because the
first nameserver whose address was found gave no usable reply. No test
fails if the walk stops shuffling: which server a live query reached is
not observable.
Model: opus-5-5
Checked every README claim against the code on next and fixed the ones
that were wrong or missing: what /metrics serves and when, what
DNSWATCHER_MAINTENANCE_MODE does, CORS on the public routes,
notification retries and the in-memory alert history, the certificate
error field and old port entries in the state file, and the Design
tree's missing files. Also corrected: CNAMEs are not followed for
watched names, the root server list is never refreshed, the NS set is
the delegation from the domain's parent zone, notification contents,
and the system resolver being used for webhooks. Code problems found
are filed separately.
Model: opus-5-5
ClassifyTargets kept a name every time it appeared in DNSWATCHER_TARGETS,
so example.com,Example.com. put example.com in the domain list twice and
every check looked it up and checked its certificates twice, sending two
expiry warnings per TLS check (port checks were already grouped by address
and port). It now skips a name it has already kept, comparing after the
lower-casing and trailing-dot removal it already did; the list keeps the
order of first appearance.
Model: opus-5-5
An expiry warning was skipped when the last one for that hostname and
address was sent less than DNSWATCHER_TLS_INTERVAL ago. Each TLS check
runs after a DNS pass of varying length, so two checks can be less than
the interval apart, and a certificate about to expire was warned about on
every check or every other check, at random. TLS checks already start
once per interval, so the in-memory record of when each warning was sent
is removed and every check warns, as the README says.
The test that expected the second check to stay silent is replaced by one
that runs TLS checks on state built in the test, with no DNS.
Model: opus-5-5
A domain's NS set was taken from whichever of its own servers answered
first, so when they disagree (during a move between DNS providers, or
with a stale secondary) the set could change between checks and send an
NS change notification with nothing changed. The walk now stops at the
referral to the domain from its parent zone's servers and returns that
delegation, which those servers all hold alike; the domain's own
servers are no longer asked for it. The NS records in an answer are
still used where no such referral comes first, as from a server that
holds both the parent zone and the domain. Hostnames get their zone's
servers the same way.
Model: opus-5-5
REPO_POLICIES.md requires Getting Started, Rationale, Design and TODO
sections in the README, and it had none of them. Getting Started clones
the repository, builds the image and runs it watching example.com and
www.example.com; DNSWATCHER_TARGETS is the only setting it requires.
Rationale is drawn from what the README already says. The Architecture
section moves below Entrypoints and is renamed Design, its text
unchanged, so the required sections come in policy order. TODO points
to TODO.md and the 1.0 milestone instead of copying the list.
Model: opus-5-5
When the resolver walks from the root servers towards a name, a server
that answered SERVFAIL, or referred the query back to its own zone, up
or sideways, ended the step, so finding a zone's servers gave up on the
zone though its other servers would answer. Such a reply is now passed
over for the zone's next server, as a timeout or a refusal already was.
To tell a referral that leads closer to the name from one that does
not, each walk keeps the zone of the servers it is asking. Other error
replies, such as FORMERR, are passed over too. The walk that finds a
nameserver's address shares the same server loop, so it changes too.
Model: opus-5-5
The port check removed the saved port state of every address no
configured name resolves to. A name whose nameservers all timed out
or failed is saved with no records, so its addresses looked gone and
lost their port state; when the nameservers answered again it was
recorded afresh, and a port that opened or closed meanwhile was not
notified.
An entry is now kept when one of the names saved on it is configured
and none of its nameservers answered on its last check, and such a
name stays on the entry when the port is checked again for another
name. A name whose nameservers answer with no addresses still loses
it, and so does a name no longer configured.
Model: opus-5-5
ResolveIPAddresses now returns an error, not no addresses, when no
nameserver of the name's zone answered. A nameserver with status
timeout or error is not an answer; one answer, even NXDOMAIN, is
enough for an empty result without an error.
When every server of a zone fails, FindAuthoritativeNameservers moves
on to the parent name, whose servers only refer the query onward. Such
a referral now has status error, so it is no answer either, and a
hostname's saved records show it as error. The only caller, the
nameserver address lookup, already keeps the previous addresses on an
error; its comment no longer says the resolver hides this case.
Model: opus-5-5
make fmt and make fmt-check now cover every Markdown file with prettier
(4-space tabs, proseWrap always), as template-app-go does: prettier is
pinned by package.json and yarn.lock and runs in a docker build on a
digest-pinned node image, never on the host. The check is forced to run
with --no-cache-filter, as script/lint is.
script/fmt-check is split into a Go half and a Markdown half because the
Dockerfile lint stage cannot run docker: that stage now runs the Go half
and script/cibuild runs the Markdown half after the build. *.md leaves
.dockerignore so documents reach the build context.
README.md, TESTING.md and TODO.md are reformatted by make fmt; apart from
the README Entrypoints entries, its make fmt line under Building and the
TODO entry, that diff is mechanical.
Model: opus-5-5
script/fmt-check now runs goimports in list mode and fails naming any
file it would change, and checks gofmt with -s, as script/fmt applies
it. Both scripts run goimports with `go run` at the commit that
script/bootstrap used to install, so a goimports on PATH is never used
and bootstrap no longer installs it. The pin is written in both
scripts; change them together. The first run on a machine, and every
Dockerfile lint stage run, downloads and builds goimports.
The Markdown half of the issue (prettier) is not done here: it needs
node in the lint image or a separate build, a decision for the owner.
Model: opus-5-5
A hostname's nameservers came from its last two labels, so a name under
co.uk was asked at the co.uk servers and a name in a delegated subdomain
at the parent's servers; both only refer onward. The hostname now goes
through FindAuthoritativeNameservers, which follows delegations for the
name and walks up its labels until it finds the zone it is in.
followDelegation now stops at an authoritative reply: that server holds
the zone, so its reply is not a referral. Without this, a CNAME answer
that also lists the zone's NS records in its authority section, as many
servers send, was followed as a referral until the delegation limit.
Model: opus-5-5
Each domain check now looks up the addresses every nameserver's name
resolves to, with the resolver's ResolveIPAddresses, and saves them
sorted in the domain's state. A nameserver that stays in the
delegation and resolves to different addresses sends one NS Address
Change notification naming the domain, the nameserver and the old and
new addresses. Added or removed nameservers get only the NS change
notification. A failed or empty lookup keeps the previous addresses,
because the resolver returns no address without an error when every
server it asks times out. State files without the field load, and the
next check fills it in silently. Watcher tests that run domain checks
use example.com, which has two nameservers, to stay within the
per-attempt limit.
Model: opus-5-5
The final save at shutdown came from the state's own stop hook, while
the watcher's stop hook only cancelled its run loop, so a check under
way could change state after that save or be cut off at exit. Run now
saves state as it returns, and the watcher's stop hook waits for Run,
bounded by the shutdown deadline. The state's own save stays; Save
holds the state lock for the whole write, so the two cannot overlap.
The start hook derives the watcher's context with WithoutCancel, so
the linter needs no exception. A new test stops a watcher built by New
and reads the change back from the state file, with no DNS.
Model: opus-5-5
DNSWATCHER_SENTRY_DSN was read but never used. This ports the Sentry
integration from gohttpserver with sentry-go v0.49.0. The server's start
hook calls sentry.Init when the DSN is set; a DSN Sentry cannot parse
fails the hook, so startup stops with the parse error. sentryhttp, with
Repanic, reports handler panics and passes them on to chi's Recoverer.
Shutdown sends queued reports once the HTTP server has stopped. The
client uses the older transport (DisableTelemetryBuffer): with the
default one, Flush can return before sending a report made just before
it. Client reports are off, so only panics are sent. The DSN is checked
at server start, not in config, so the config test is unchanged.
sentry-go raises several golang.org/x modules and moves go-spew and
go-difflib to untagged commits.
Model: opus-5-5
When shutdown cancels a check that is under way, the rest of the check
still runs with the cancelled context. The resolver already drops a
lookup the context cut short, but a cancelled connection attempt was
saved as a closed port or a failed certificate check and notified as
Port Change or TLS Failure. The watcher now drops a port or TLS check
result when its context was cancelled, the same way. The test runs a
check with the context already cancelled, using the real resolver and
the real port and TLS checkers; no query is sent and no connection is
made.
Model: opus-5-5
realIP took the first X-Forwarded-For entry, which the client itself
can write, so behind a proxy that appends to the header a client chose
the address dnswatcher logs and the /metrics rate limit counts. It now
walks the entries from the right past trusted proxies, using the
existing trusted-proxy check, and takes the first that is not one; the
leftmost when all are. All X-Forwarded-For header lines are read as one
list, since a proxy may add its own line instead of appending to the
client's. An empty entry where the client address belongs falls back
to the peer address, as an empty first entry did before. X-Real-IP is
unchanged.
Model: opus-5-5
LookupAllRecords now returns each nameserver's response, so the
watcher saves its status: ok when it answered, NXDOMAIN and no records
included, and error with the reason when it timed out, answered
SERVFAIL or REFUSED, or could not be reached. A nameserver that starts
failing sends NS Failure and one that answers again sends NS Recovery.
A failing nameserver is left out of the record change and
inconsistency comparisons. The resolver used to report REFUSED and
network errors as an answer with no records; they are now errors. A
lookup cut short by its context now returns an error instead of a
failure of the nameserver it was querying.
Model: opus-5-5
DNSWATCHER_DNS_INTERVAL and DNSWATCHER_TLS_INTERVAL were parsed with
time.ParseDuration and silently replaced by the default when that failed,
so a value like 5 or 1d gave hourly checks with no hint why, and zero or
negative values were accepted. Both now go through parseInterval, which
returns an error naming the variable and the value, and startup stops the
same way it does for invalid targets. An unset or empty variable still gets
its default from setupViper. The three tests that pinned the old fallback
are replaced. The README says what a valid value looks like.
Model: opus-5-5
/metrics is behind a password, and REPO_POLICIES.md requires rate
limiting on password logins. Each client address may now send it 30
requests a minute, counted by httprate before Basic Auth, so failed
logins use up the allowance and a request over it gets 429 without
the password being checked. The address is the one the existing
trusted-proxy logic in internal/middleware works out, with IPv6
addresses grouped by /64; an IPv4 address a proxy reports in
IPv6-mapped form counts as the plain IPv4 address. A Prometheus
server scraping every 15 seconds sends 4 requests a minute.
Model: opus-5-5
The Dockerfile builder stage takes ARG VERSION (default `dev`) and passes
it to `make build` on the command line, which overrides the Makefile's
`git describe` default. script/docker computes the version from
`git describe` on the host and passes it as --build-arg VERSION, because
.dockerignore leaves .git out of the build context and `git describe`
inside the build only ever produced `dev`. A build that passes no
argument, such as script/cibuild, still reports `dev`.
`logger.Identify`, which logs `starting` with the version, was never
called; `main` now calls it first, so the version is in the startup log.
Model: opus-5-5
Two shutdown tests hold a delivery inside the test server's handler and
release it from a timer. The deferred timer stop ran before the server
was closed, so a drain that returned early left the handler blocked and
the server's close waited on it until the package timed out. Each test
now defers a release, guarded so the timer and the defer can both call
it, ahead of closing the server. The watchdog comment no longer names a
30-second timeout the test script does not use.
Model: opus-5-5
The script wrote the hook to .git/hooks, which fails when .git is a
file rather than a directory, as in a clone made with
--separate-git-dir. It now asks git for the repository's own git
directory with `git rev-parse --git-common-dir`, creates its hooks
directory if missing, and writes the hook there. In an ordinary clone
that is .git/hooks, so nothing moves. Before writing anything it stops
with an error when its top directory is not the top of the checkout
git finds, so a copy inside another repository cannot replace that
repository's hook. git's core.hooksPath setting is not followed; where
it is in force, git does not run the installed hook, as before.
Model: opus-5-5
Next Step and Future Steps list the open issues by full URL, in the order
of the review on #144; issues
the review does not name sit next to the entries they relate to, and
#144 itself comes last. Left
out: #146, which this change
closes; #59, DNSSEC, ruled
post-1.0; issues assigned to sneak, which wait on the owner. Shipped work,
the dropped domains and hostnames endpoints and the line about mocked
resolver tests are gone. Every Completed Steps entry is at most two lines;
none was dropped. Workflow branches from `next` and opens PRs against it.
Wrapped by hand at 80 columns: `make fmt` does not format Markdown yet
(#119). The old Next Step is now
#173.
Model: opus-5-5
The runtime image no longer sets USER. Its new entrypoint,
deploy/docker-entrypoint.sh, runs as root: it creates the data
directory if needed, gives it and everything in it to the dnswatcher
user (uid 10001) with mode 700 on the directory, then runs dnswatcher
as that user with su-exec. A bind-mounted host directory, whether
empty and root-owned or holding a state file left by another uid, no
longer has to be chowned first, and the README's upaas section now says
only which path to mount. The startup check that the data directory is
writable stays.
Model: opus-5-5
The CORS wildcard was global, so it also covered the Basic-Auth
protected /metrics, which REPO_POLICIES.md forbids, and it allowed
POST, PUT and DELETE, which no route serves, plus the Authorization
and X-CSRF-Token headers. CORS now sits on a router holding only the
public routes and allows GET and OPTIONS with the Accept and
Content-Type headers. /metrics gets no CORS at all.
Both are mounted routers rather than a Group: chi answers OPTIONS on
a Group's route with 405 before its middleware runs, and any method
/metrics does not register would otherwise fall through to the public
router. So every method on /metrics now meets Basic Auth first, and
/metrics/ is served like /metrics.
Model: opus-5-5
state.NewForTest, state.NewForTestWithDataDir and watcher.NewForTest
were compiled into and exported by the production packages
internal/state and internal/watcher. NewForTestWithDataDir and
watcher.NewForTest now live in their package's export_test.go. Tests in
other packages cannot see those files, so the watcher and middleware
tests build their State with state.New and a temporary data directory;
the watcher tests no longer try to save to /state.json.
state.NewForTest, whose State saved to /, is deleted: the state tests
that used it pass t.TempDir() to NewForTestWithDataDir, and the test of
the helper itself is gone.
Model: opus-5-5
drainSlack stood for three things: the deadline given to a drain that
should finish early, the watchdog on a drain that should time out, and
the wait for a delivery to reach the test server. It is now three
constants, each commented with what it bounds and why it is 2s; no
value changed. The idle-drain failure printed that deadline instead of
idleDrainBound, the bound it checks. The cancelled-context test now
also requires the drain to return within idleDrainBound and to log its
debug line, so a drain that logs nothing no longer passes;
newLoggingService records debug level for this.
Model: opus-5-5
The live-DNS retry and concurrency limit is only for tests, but nothing
stopped program code from importing it and compiling it into the
binary. Its directory name now ends in test, and its import path is on
the test-support deny list in .golangci.yml, so make lint fails when
program code imports it. Every import and mention is updated to the new
name.
Model: opus-5-5
The org .golangci.yml now uses gomodguard_v2 in place of the
deprecated gomodguard, which made every lint run print a deprecation
warning. The file is copied unchanged from sneak/prompts. It also turns
on depguard with the org test-support rule, which rejects
net/http/httptest except in test files and in files under a directory
whose name ends in test. This repo's previous copy had no deny entries
of its own, so there were none to carry forward.
Model: opus-5-5
The watcher tests used a stand-in resolver and the resolver timeout test
a stand-in DNS client, against the rule that DNS is never mocked.
Watcher tests that look something up in DNS now run the real resolver
against live servers, each attempt on a new watcher. A DNS change is
tested by saving values live DNS never returns (names under .invalid,
192.0.2.1) in the state a check starts from, or by marking a real
nameserver failed. The timeout test queries 192.0.2.1, where nothing
answers. The live-DNS retry and concurrency limit moved from the
resolver tests to internal/livedns, so both packages share them.
NewFromLoggerWithClient had no other use and is gone. TESTING.md now
states the README's rule.
Model: opus-5-5
detectInconsistencies alerted for neighbouring pairs of nameservers whose
records differed, on every DNS check, for as long as they differed. It now
also takes the previous hostname state, compares every pair of nameservers,
and alerts for a pair that differs unless both were in that state and
already differed there, so a nameserver new on a check that answers
differently is reported once. The state loaded at startup is the previous
state for the first check, so a disagreement saved before a restart is not
reported again. The choice of pairs is tested on record data, and the alert
through the hostname change detection with the notifier stand-in and no
resolver. The README describes the new behaviour.
Model: opus-5-5
Nameservers may answer with DNS names in any letter case. For eeqj.de,
y.ns.joker.com answers in upper case while its peers answer in lower
case, so the inconsistency check fired on every cycle. extractRecordValue
now lower-cases CNAME, MX, SRV and NS targets, so the inconsistency check
and the record-change check both compare names regardless of case. A,
AAAA, TXT and CAA values are formatted as before. State saved before this
change can hold upper-case names, which report a one-time record change
on the first check after upgrading.
Model: opus-5-5
script/cibuild and script/docker were plain docker build. On an
unchanged tree the lint stage and the builder stage, which runs make
test, came from the layer cache, so the build passed without linting
or querying live DNS. Both scripts now pass
--no-cache-filter=lint,builder so those stages run on every build, as
script/lint already does for its own lint stage. Dependency downloads
inside those stages re-run each build. Each of the two stages in the
Dockerfile now notes that the scripts name it. README and TODO.md
updated to match.
Model: opus-4-8 (implementation); opus-5-5 (rework)
The timeout tests called newHTTPServer directly, so a Run that built
its http.Server inline would drop every timeout with the suite still
green. TestRunWiresSocketTimeouts wires a Server as cmd/dnswatcher
does, minus the watcher and resolver so no live DNS is touched, drives
Run with an unbindable port so it stores its http.Server and returns
without listening, and checks that server carries all four timeouts
and both required relationships. The addr/handler test, which could
not fail, is dropped.
The ReadTimeout note now says what net/http does: a request whose
headers arrive after ReadTimeout but within ReadHeaderTimeout gets a
read deadline that has already passed, so reading its body fails at
once.
Model: opus-4-8 (implementation); opus-5-5 (rework)
The runtime image runs as uid 10001, which owns /var/lib/dnswatcher. The
working directory is /, so config loading finds no .env or dnswatcher
config file there; the binary lives in /usr/local/bin. A Docker
HEALTHCHECK probes /.well-known/healthcheck every 10 seconds with busybox
wget, well inside the 60 seconds upaas waits.
Startup now fails with an error naming the data directory when it cannot
be written, instead of running with every save failing. The check creates
the directory if needed and writes and removes the temp file Save uses;
tests cover the create and the write failing.
README gains "Running under upaas": the prod branch, host directory
setup, network and port, environment and health check.
Model: opus-5-5
Adds a SecurityHeaders middleware and registers it globally, right after the request ID middleware, so every route gets the headers, including static files, /metrics and error responses.
It sets Strict-Transport-Security (one year, includeSubDomains), a Content-Security-Policy with default-src 'self', no scripts and frame-ancestors 'none', X-Frame-Options DENY, X-Content-Type-Options nosniff, Referrer-Policy no-referrer and a Permissions-Policy that turns every listed feature off.
HSTS is sent on every response, not only over TLS: the service runs behind a TLS-terminating proxy and REPO_POLICIES.md requires the application to send it. Referrer-Policy is stricter than the policy baseline because dashboard URLs can name internal hosts.
model: claude-opus-4-8 (implementation); claude-fable-5 (commit message)
Tests for the three packages that had none, written from outside each package, each able to fail on a plausible break:
- globals: values set are read back through New, and New returns an independent copy. One sequential test function with a disclosed paralleltest suppression, because it changes shared package variables.
- healthcheck: Check returns status "ok", the documented JSON fields, an RFC3339Nano timestamp, the maintenance flag from config in both states, and version and appname from globals.
- logger: New gives a usable *slog.Logger, debug output is off by default and EnableDebugLogging turns it on.
No production code changed. The terminal output format is not asserted.
Model: opus-4-8
go.mod listed golang.org/x/sync v0.19.0 twice: once in the direct
require block and once as // indirect. The module is a genuine direct
dependency (internal/portcheck/portcheck.go imports
golang.org/x/sync/errgroup), so the indirect entry was redundant and
stale. script/bootstrap ends with go mod download, which dropped that
line as a side effect and left every fresh checkout with a dirty
go.mod. Running go mod tidy removes the line for good; no dependency
and no version changed, and go.sum is unaffected.
Model: opus-4-8
clawbot