2 Commits
Author SHA1 Message Date
clawbot 62dec447e3 test: bound watcher live DNS and serialise packages after rebase
check / check (push) Successful in 1m37s
Rebasing this branch onto next surfaced two failures that the branch
did not have in isolation. Both come from the change this PR makes:
the watcher package now queries live DNS, and it is the second package
to do so.

Burst fan-out in the watcher package. All 13 watcher tests are
parallel and each runs a full iterative resolution, so they hit the
same root servers in the same instant and get rate-limited. This is
exactly the pathology internal/resolver/livedns_test.go was written to
prevent (9cb2c2b); that gate is package-scoped and cannot reach into
this package's test binary, so liveWatcherGate is its counterpart
here, acquired in newTestWatcher and released when the test ends.

Cross-package oversubscription. Go runs package binaries in parallel,
so with both live-DNS packages in flight their gates sum rather than
hold. The excess is rate-limited and the resolver's 8s per-attempt
deadlines expire, failing ResolveIPAddresses tests this branch never
touched. script/test now passes -p 1 so each gate is authoritative
while its package runs. Serialising is also net faster here, because
the retries it removes cost more than the lost parallelism: resolver
16-22s (was 30-36s under contention), watcher 12-13s (was 27-37s),
whole suite 39-46s against the 60s cap and the 90s -timeout backstop.

TestQueryNameserverIP_UnreachableServer is dropped rather than fixed.
It asserted that a query to an RFC 5737 documentation address comes
back non-OK with no records, which does not hold in the build
environment: that network transparently intercepts all UDP/53 traffic
regardless of destination and answers it locally, so the query returns
StatusOK with 9 real records for example.com. Verified directly with
dig @192.0.2.1 inside the build network. The mock DNSClient that used
to force this classification is what this PR removes, and a live
substitute would only be testing the sandbox's network behaviour, so
the coverage gap is recorded in a comment where the test was.

Verified: three consecutive `docker build .` runs green, after three
consecutive failures without these changes.
2026-09-03 17:01:08 +00:00
sneak 11b9b5527d Remove DNS mocking from tests; use live DNS everywhere
DNS is never mocked in this repository: tests exercise live DNS,
and robustness comes from handling real-world DNS behavior with
tolerant assertions and sensible timeouts, not from mocks.

watcher: drop mockResolver and wire the real iterative resolver
into the tests, querying stable public names (example.com,
www.example.com). Change detection is exercised by seeding the
state store with a synthetic previous observation that live DNS
cannot match (reserved .invalid nameserver names and RFC 5737
documentation addresses); DNS stays live in every run. The port
checker, TLS checker, and notifier remain test doubles since they
are not DNS, keeping notification and state assertions
deterministic against whatever addresses live DNS returns.

resolver: drop the timeoutClient fake DNSClient and the
NewFromLoggerWithClient mock constructor. The timeout test is
replaced by a live query against an RFC 5737 documentation
address where no nameserver can exist, asserting a classified
non-OK response with no records.

TESTING.md: extend the live-DNS policy to every package and
remove the carve-out that permitted DNS mocks in packages that
consume the resolver.

TODO.md: update stale references to hermetic mocked-DNS work to
reflect the no-mocking policy and the current state of
feature/resolver.

Intentionally dropped coverage: the exact StatusTimeout
classification (previously forced by the fake client) is no
longer asserted, because a genuinely unreachable server may fail
fast instead of timing out depending on the network path; the
live test tolerantly accepts any failure classification.
2026-09-03 16:43:25 +00:00
7 changed files with 500 additions and 565 deletions
+22 -4
View File
@@ -2,8 +2,10 @@
## DNS Resolution Tests
All resolver tests **MUST** use live queries against real DNS servers.
No mocking of the DNS client layer is permitted.
All tests that involve DNS resolution — in every package, including
consumers of the resolver such as the watcher — **MUST** use live
queries against real DNS servers. No mocking, faking, or stubbing of
DNS at any layer is permitted.
### Rationale
@@ -12,6 +14,8 @@ the full delegation chain. Mocked responses cannot faithfully represent
the variety of real-world DNS behavior (truncation, referrals, glue
records, DNSSEC, varied response times, EDNS, etc.). Testing against
real servers ensures the resolver works correctly in production.
Robustness comes from handling real-world DNS behavior with tolerant
assertions and sensible timeouts, not from mocks.
### Constraints
@@ -24,14 +28,28 @@ real servers ensures the resolver works correctly in production.
- Flaky failures from transient network issues are acceptable and
should be investigated as potential resolver bugs, not papered over
with mocks or skip flags
- Watcher change-detection tests seed a synthetic *previous state*
and compare it against fresh live lookups; the DNS side is never
faked
- Live query concurrency is bounded per package (`liveGate` in
`internal/resolver`, `liveWatcherGate` in `internal/watcher`) so
parallel tests do not burst at the root servers
- Those gates are package-scoped and therefore per test binary, so
`script/test` also passes `-p 1`: with both live-DNS packages
running at once the gates sum instead of holding, and the resolver's
per-attempt deadlines start expiring
### What NOT to do
- **Do not mock `DNSClient`** for resolver tests (the mock constructor
exists for unit-testing other packages that consume the resolver)
- **Do not mock `DNSClient`**, the watcher's `DNSResolver` interface,
or any other DNS abstraction — in any package, for any reason
- **Do not add `-short` flags** to skip slow tests
- **Do not increase `-timeout`** to hide hanging queries
- **Do not remove `-count=1` from `script/test`** — Go's test cache
replays a previous run's output without querying anything, so a
cached pass is not evidence that live resolution works
- **Do not remove `-p 1` from `script/test`** — running the live-DNS
packages in parallel oversubscribes live DNS past what their gates
bound, which surfaces as unrelated resolver tests failing on
expired deadlines
- **Do not modify linter configuration** to suppress findings
+17 -7
View File
@@ -14,7 +14,10 @@ pre-1.0. No git tags. Core resolver work in flight on feature/resolver
(dirty: internal/resolver/resolver_test.go). Local checkout has diverged
from origin: origin/main is 8 commits ahead (watcher orchestrator,
unified TARGETS) and origin/feature/resolver already contains the full
iterative resolver implementation with hermetic mocked tests.
iterative resolver implementation. DNS mocking is banned in this repo
(see `TESTING.md`): all tests use live DNS only. The hermetic mocked
tests previously noted on `feature/resolver` are gone from its current
tip, which carries a live-DNS suite against `*.dns.sneak.cloud`.
# Next Step
@@ -92,6 +95,10 @@ Rationale, Design, TODO, License, Author) if any are still missing.
`make check` into `script/lint`. `golangci-lint config verify` is
deliberately omitted: it fetches its schema over an unpinned live
HTTPS call
- 2026-08-07: DNS mocking removed from the entire test suite; watcher
tests now drive the real iterative resolver against live DNS and
`TESTING.md` bans DNS mocks in every package (`remove-dns-mocking`
branch)
- 2026-08-07: golangci-lint bumped to v2.12.2 (commit-pinned installs
in `Dockerfile` and `script/bootstrap`); `.golangci.yml` set to the
org-standard v2-schema config used across the org's repos
@@ -103,7 +110,8 @@ Rationale, Design, TODO, License, Author) if any are still missing.
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
Makefile shims, README Entrypoints section
- 2026-02-20: iterative DNS resolver implemented; tests made hermetic
with mocked DNS (origin/feature/resolver, unmerged)
with mocked DNS (origin/feature/resolver, unmerged; superseded — DNS
mocking is banned, see `TESTING.md`)
- 2026-02-20: CI actions and go install refs pinned to commit SHAs;
Gitea Actions workflow for make check (origin/ci/make-check, unmerged)
- 2026-02-20: watcher monitoring orchestrator merged to main (#8)
@@ -128,8 +136,9 @@ Branch reconciliation:
- Sync local checkout with origin: local main is 8 commits behind
origin/main; local feature/resolver has diverged from
origin/feature/resolver, which already implements the resolver
- Merge in-flight branches to main once green: feature/resolver,
ci/make-check, feature/portcheck-implementation,
- Merge in-flight branches to main once green: feature/resolver
(confirm its tests remain live-DNS — DNS mocking is banned, see
`TESTING.md`), ci/make-check, feature/portcheck-implementation,
feature/tlscheck-implementation
Resolver (plan from untracked TODO.md; largely implemented on
@@ -213,6 +222,7 @@ Infrastructure notes (from untracked TODO.md):
- Module path sneak.berlin/go/dnswatcher differs from the git.eeqj.de
remote intentionally; do not "fix" it
- Dependencies: github.com/miekg/dns, golang.org/x/net/publicsuffix
- Resolver tests originally used live DNS against *.dns.sneak.cloud
(required records documented in the test file header); origin now has
mocked hermetic tests, keep them hermetic
- Resolver tests originally used live DNS against `*.dns.sneak.cloud`
(required records documented in the test file header); `main` now
tests against live public DNS. DNS mocking is banned (see
`TESTING.md`); never reintroduce hermetic mocked DNS tests
+2 -2
View File
@@ -7,8 +7,8 @@ import (
"github.com/miekg/dns"
)
// DNSClient abstracts DNS wire-protocol exchanges so the resolver
// can be tested without hitting real nameservers.
// DNSClient abstracts DNS wire-protocol exchanges over a single
// transport, letting the resolver switch between UDP and TCP.
type DNSClient interface {
ExchangeContext(
ctx context.Context,
-13
View File
@@ -67,17 +67,4 @@ func NewFromLogger(log *slog.Logger) *Resolver {
}
}
// NewFromLoggerWithClient creates a Resolver with a custom DNS
// client, useful for testing with mock DNS responses.
func NewFromLoggerWithClient(
log *slog.Logger,
client DNSClient,
) *Resolver {
return &Resolver{
log: log,
client: client,
tcp: client,
}
}
// Method implementations are in iterative.go.
+9 -54
View File
@@ -8,9 +8,7 @@ import (
"sort"
"strings"
"testing"
"time"
"github.com/miekg/dns"
"github.com/stretchr/testify/assert"
"github.com/stretchr/testify/require"
@@ -519,58 +517,15 @@ func TestQueryAllNameservers_ContextCanceled(t *testing.T) {
assert.Error(t, err)
}
// ----------------------------------------------------------------
// Timeout tests
// ----------------------------------------------------------------
func TestQueryNameserverIP_Timeout(t *testing.T) {
t.Parallel()
log := slog.New(slog.NewTextHandler(
os.Stderr,
&slog.HandlerOptions{Level: slog.LevelDebug},
))
r := resolver.NewFromLoggerWithClient(
log, &timeoutClient{},
)
ctx, cancel := context.WithTimeout(
context.Background(), 10*time.Second,
)
t.Cleanup(cancel)
// Query any IP — the client always returns a timeout error.
resp, err := r.QueryNameserverIP(
ctx, "unreachable.test.", "192.0.2.1",
"example.com",
)
require.NoError(t, err)
assert.Equal(t, resolver.StatusTimeout, resp.Status)
assert.NotEmpty(t, resp.Error)
}
// timeoutClient simulates DNS timeout errors for testing.
type timeoutClient struct{}
func (c *timeoutClient) ExchangeContext(
_ context.Context,
_ *dns.Msg,
_ string,
) (*dns.Msg, time.Duration, error) {
return nil, 0, &net.OpError{
Op: "read",
Net: "udp",
Err: &timeoutError{},
}
}
type timeoutError struct{}
func (e *timeoutError) Error() string { return "i/o timeout" }
func (e *timeoutError) Timeout() bool { return true }
func (e *timeoutError) Temporary() bool { return true }
// The resolver's transport-failure classification (StatusTimeout /
// StatusError for a nameserver that does not answer) is deliberately
// not covered here. Forcing it required the mock DNSClient this
// change removes, and a live substitute is not available: the build
// environment transparently intercepts all UDP/53 traffic and answers
// it locally, so a query to a black-holed address such as an RFC 5737
// documentation address comes back StatusOK with real records. See
// TESTING.md; restoring this coverage needs a mechanism that is
// neither a mock nor dependent on the sandbox's network behaviour.
func TestResolveIPAddresses_ContextCanceled(t *testing.T) {
t.Parallel()
File diff suppressed because it is too large Load Diff
+13 -2
View File
@@ -19,15 +19,26 @@
#
# -timeout 90s is a deliberate backstop above the 60s hard cap on
# suite duration. Do not lower it.
#
# -p 1 runs one test package at a time, and is load-bearing. Live DNS
# is a resource outside the process: the concurrency gates that keep
# this suite from bursting at the root servers
# (internal/resolver/livedns_test.go, internal/watcher/watcher_test.go)
# are package-scoped, so each one only bounds its own test binary. Go
# runs package binaries in parallel by default, so with both live-DNS
# packages in flight at once their gates sum instead of holding, the
# root and TLD servers rate-limit the excess, and the resolver
# package's per-attempt deadlines expire. Serialising packages is what
# makes each gate authoritative while its package runs.
set -eu
ROOT="$(cd "$(dirname "$0")/.." && pwd -P)"
main() {
cd "$ROOT"
go test -count=1 -race -timeout 90s -cover ./... || {
go test -count=1 -p 1 -race -timeout 90s -cover ./... || {
echo "--- Rerunning with -v for details ---" >&2
go test -count=1 -race -timeout 90s -v ./... || true
go test -count=1 -p 1 -race -timeout 90s -v ./... || true
exit 1
}
}