next (#136)

Long-lived integration branch. One commit per work unit lands here; this PR accumulates them until it is merged to `main`. ## Landed units - **Run all linting in Docker via `Dockerfile.lint` + `script/lint`** — #134 golangci-lint is no longer installed or run on the host. New root `Dockerfile.lint` COPYs the repo into the digest-pinned `golangci/golangci-lint:v2.12.2` image and lints as a build step, so a successful build IS a clean lint; `script/lint` is reduced to a thin wrapper that builds it. This also works where the docker daemon is remote and bind mounts are impossible. **Pinned digest and how it was verified.** `golangci/golangci-lint:v2.12.2@sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240`, exactly as quoted in the issue. It resolves, and it is genuinely v2.12.2: ``` $ docker buildx imagetools inspect golangci/golangci-lint:v2.12.2 Name: docker.io/golangci/golangci-lint:v2.12.2 MediaType: application/vnd.oci.image.index.v1+json Digest: sha256:5cceeef04e53efe1470638d4b4b4f5ceefd574955ab3941b2d9a68a8c9ad5240 $ docker run --rm golangci/golangci-lint@sha256:5cceeef04e...ad5240 golangci-lint --version golangci-lint has version 2.12.2 built with go1.26.2 from c0d3ddc9 on 2026-05-06T11:07:58Z ``` The tag's index digest is the quoted digest, and the binary inside reports commit `c0d3ddc9`, matching the org's canonical pin `c0d3ddc9cf3faa61a4e378e879ece580256d76e5`. **Forcing the linter to actually run.** `Dockerfile.lint` is split into a `deps` stage (base image + `go mod download`) and a `lint` stage (source copy + linter run). `script/lint` runs: ``` docker build --progress=plain --no-cache-filter=lint --target lint -f Dockerfile.lint . ``` Caching is explicitly waived for linting, and a cached build lints nothing, so the `lint` stage is invalidated on every invocation. The invalidation is scoped: the `deps` stage stays cached and no global cache wipe is performed. `--progress=plain` keeps the linter's own output visible. **`golangci-lint config verify`: deliberately NOT included.** It fetches its JSON schema over a live, unpinned HTTPS call, which would make linting network-dependent and defeat hash-pinning. Omitted for that reason, and the reason is recorded in a comment at the top of `Dockerfile.lint`. **`script/bootstrap`** no longer installs golangci-lint (and its pinned ref is gone); it warns non-fatally when `docker` is absent instead. The `goimports` install stays, because `script/fmt` and `script/fmt-check` still run on the host. Header comment updated accordingly. **Root `Dockerfile`** — required consequence, not scope creep. Its builder stage ran `make check`, which now calls `script/lint`, which shells out to `docker build`; there is no docker daemon inside a docker build, so `script/cibuild` and `script/docker` would have broken. It gains its own lint stage on the same pinned image (linter invoked directly, with a comment explaining why not `make lint`), with the builder stage depending on it via `COPY --from=lint /src/go.sum /dev/null` and running `make fmt-check`, `make test`, `make build`. The now-unneeded golangci-lint install is gone from the builder stage. **README** `Entrypoints` and `Building` sections now describe linting as a docker-only operation. `TODO.md` updated in the same commit. ### Verification All runs via `make` / `script/` entrypoints only. Two consecutive `make lint` runs on an unchanged tree, both executing the linter: ``` # run 1 #10 [lint 2/2] RUN golangci-lint run --config .golangci.yml ./... #10 10.98 0 issues. #10 DONE 12.0s # run 2, tree untouched #10 [lint 2/2] RUN golangci-lint run --config .golangci.yml ./... #10 14.63 0 issues. ``` A third run shows the cache scoping is working as intended — `deps` served from cache, `lint` re-executed: ``` #6 [deps 2/4] WORKDIR /src #6 CACHED #7 [deps 3/4] COPY go.mod go.sum ./ #7 CACHED #8 [deps 4/4] RUN go mod download #8 CACHED #9 [lint 1/2] COPY . . #10 [lint 2/2] RUN golangci-lint run --config .golangci.yml ./... ``` **Negative control.** A deliberate violation (an unused function containing an ineffectual assignment) was added to `internal/config/config.go`: ``` #10 11.26 internal/config/config.go:29:2: ineffectual assignment to x (ineffassign) #10 11.26 internal/config/config.go:28:6: func negativeControlUnused is unused (unused) #10 11.26 2 issues: #10 ERROR: process "/bin/sh -c golangci-lint run --config .golangci.yml ./..." did not complete successfully: exit code: 1 ERROR: failed to build: failed to solve: process "/bin/sh -c golangci-lint run --config .golangci.yml ./..." did not complete successfully: exit code: 1 make: *** [Makefile:26: lint] Error 1 ``` `make lint` exited non-zero naming both findings and their exact lines. After reverting the file, `make lint` was clean again (`0 issues.`). **`make check`** green end to end (test, lint, fmt-check), exit 0. **`script/cibuild`** green, confirming the `Dockerfile` restructure does not recurse: the lint stage ran (`#16 12.56 0 issues.`), then `#22 [builder 8/9] RUN make test` with `PASS` lines, then `#23 [builder 9/9] RUN make build`. ### Notes for the owner This supersedes two PRs you still have queued for merge, both of which tune host linting that no longer exists after this change: #128 (isolates the host golangci-lint cache and lock) and #131 (always installs the pinned lint tools in `script/bootstrap`). Neither was merged or incorporated here. Made moot by this change: #121 and #130. ### Review outcome Reviewed at #136 (comment) — **PASS**. Three non-blocking comment-accuracy findings were recorded there for the next touch of those files; the reviewer independently confirmed the lint gate is live by negative control from a warm cache. - **Live DNS tests made robust instead of gated; test caps moved to the org-wide 60s/20s/90s values** — #93 The resolver's live-DNS tests failed nondeterministically, a different subset each run. Fixed by engineering the nondeterminism out, not by routing around the network. **Nothing is mocked, faked, stubbed, recorded or replayed; there is no `-short` flag, no build tag, no skip, and no environment-tolerance for restricted egress.** Production resolver behaviour is unchanged. ### Root causes, all test-side 1. **Burst fan-out at one root server.** Every test in `internal/resolver` calls `t.Parallel()` and the build hosts have many cores (48 here), so all ~35 iterative resolutions started within milliseconds of each other, and because `queryServers` walks `rootServerList()` in fixed order they all aimed their first query at `198.41.0.4`. Root servers rate-limit that, which fits the reported symptom of a different arbitrary subset failing each run. 2. **No retry anywhere.** One dropped UDP packet in a delegation chain failed a test outright. 3. **Unanimity assertions.** `TestQueryAllNameservers_AllReturnOK` and `_NXDomainFromAllNS` required *every* nameserver of a domain to answer — four independent chances to fail per run, with no tolerance for one being slow. ### What was built New `internal/resolver/livedns_test.go` holds all the live-DNS machinery, so `resolver_test.go` itself takes only call-site edits: - **Bounded live concurrency.** A package-wide semaphore (`liveConcurrency = 6`) caps how many live resolutions are in flight at once. Tests keep `t.Parallel()`; only their network work is throttled. This is the direct fix for cause 1, and the 60s budget is what makes it affordable. - **Retry with exponential backoff.** Three attempts per live operation, 8s deadline each, 500ms base backoff doubling. The retry predicate is deliberately **transport-level** — "did a nameserver answer at all" — and never the assertion the test is making, so a resolver that answers *incorrectly* still fails on the first attempt rather than being retried into a false green. - **Quorum instead of unanimity, tolerating SILENCE ONLY.** A strict majority of the discovered nameservers must answer as expected, and every individual result must additionally fall inside a closed **allowlist** of statuses the test explicitly sanctions: `ok`/`timeout`/`error` for the all-OK test, `nxdomain`/`timeout`/`error` for the NXDOMAIN test. A nameserver that stays silent is tolerated; one that answers **wrongly** is not, at any count. The allowlist is the load-bearing part — see the rework note below for why a blocklist was not enough. New `internal/resolver/livedns_harness_test.go` tests that machinery directly — quorum arithmetic, status counting, the allowlist, the gate's concurrency bound, per-attempt deadlines, and recovery from a transient failure. It performs no DNS resolution of any kind, so it neither mocks DNS nor depends on it. ### Rework after review — the quorum could not fail on a wrong answer The review at #136 (comment) returned **FAIL** on `9cb2c2b`, correctly. Fixed in `87bce43`. **The defect.** The claim above was, as first written, false for `resolver.StatusNoData`. Each test banned exactly one wrong status — `_AllReturnOK` banned only `nxdomain`, `_NXDomainFromAllNS` banned only `ok` — and `nodata` is neither. It is a **wrong answer, not silence**: `answeredCount` counted it as answered, so it did not even trigger a retry, and with a quorum of 3-of-4 a single wrong nameserver slid through undetected. The pre-change unanimity assertions would have caught it. That is robustness work quietly becoming assertion-loosening, which is exactly what this repo cannot afford. **The fix.** Tolerance is now an allowlist, not a blocklist of one status. New `unsanctionedStatuses()` returns every per-nameserver result whose status the caller did not explicitly sanction, and each test asserts that list is empty in addition to its quorum. A blocklist bans the one wrong answer its author thought of and silently admits everything else, including any status added to the resolver later; an allowlist fails on anything nobody sanctioned. `answeredCount` was reframed the same way — it now counts the closed set `ok`/`nxdomain`/`nodata`, so an unfamiliar status is treated as silence and can only ever cause a retry and then a loud failure, never a quiet pass. **Evidence — the reviewer's exact probe, re-run.** `queryEachNS` in `internal/resolver/iterative.go` was patched to force one of `google.com`'s four nameservers to return `StatusNoData` with empty records. `make test` now goes **red**, naming the offending nameserver and status: ``` exit=2 --- FAIL: TestQueryAllNameservers_AllReturnOK (1.12s) Error: Should be empty, but was [ns1.google.com.=nodata] Messages: every nameserver must answer OK or not answer at all: ns1.google.com.=nodata ns2.google.com.=ok ns3.google.com.=ok ns4.google.com.=ok --- FAIL: TestQueryAllNameservers_NXDomainFromAllNS (1.34s) Error: Should be empty, but was [ns1.google.com.=nodata] Messages: every nameserver must report NXDOMAIN or not answer at all: ns1.google.com.=nodata ns2.google.com.=nxdomain ns3.google.com.=nxdomain ns4.google.com.=nxdomain FAIL sneak.berlin/go/dnswatcher/internal/resolver 1.704s ``` That is the same input that returned `exit=0` with both tests **passing** under the old assertions. Probe reverted, tree clean, suite green again: ``` exit=0 ok sneak.berlin/go/dnswatcher/internal/resolver 4.185s coverage: 77.1% of statements (zero `(cached)` lines) ``` Two harness tests lock the regression in without any probe: three OK plus one `nodata` (quorum satisfied, no NXDOMAIN present — the exact input that used to pass) is reported as unsanctioned, and an unknown status is neither counted as answered nor tolerated. Also fixed from the review: the per-attempt deadline assertion in `livedns_harness_test.go` had no lower bound, so it passed for a deadline far shorter than intended. It now asserts the remaining time exceeds `liveAttemptTimeout/2` as well. Deliberately **not** done in this rework, per the review and the owner: no `-count=1` in `script/test` (the test-cache issue is real but pre-existing and repo-wide, filed separately); the remaining non-DNS mocks stay for #97; `queryServers` root-ordering stays untouched under #138. ### The `-timeout` backstop value: 90s Per the ruling at sneak/prompts#41 (comment) the cap is org-wide with two tiers: **60s hard cap for CI green, 20s target, and anything between the two must be filed as an improvement bug.** The backstop is **`90s`**, matching sneak/prompts#42 and preserving the 1.5x backstop-to-cap ratio the old 20s/30s pair had. It must strictly exceed the 60s cap or the cap is unreachable — the old `-timeout 30s` would have killed a 60s-capped suite at half its allowance. Applied to `script/test`; nothing else in the repo carried the old `30s`. Worst case for one live operation is 3 attempts x 8s plus ~1.5s of backoff, about 26s — comfortably inside the 90s backstop even if several operations exhaust their attempts at once. ### `REPO_POLICIES.md` is re-vendored, not hand-edited The file is org-canonical, so it was **copied byte-for-byte** from `prompts/REPO_POLICIES.md` on `sneak/prompts` branch `org-wide-60s-test-cap` (commit `52b5192`) rather than reworded to approximately the same thing. Verified: ``` $ cmp prompts/REPO_POLICIES.md dnswatcher/REPO_POLICIES.md && echo identical identical $ sha256sum REPO_POLICIES.md bcf11c312a1bee18a0e937eb412b51914411c1ab23308b8362409f3f88379ff7 ``` **What that byte-identity does and does not certify.** The source branch `org-wide-60s-test-cap` is an **unmerged proposal** — sneak/prompts#42 — not `prompts` `main`. So, precisely: - The **60s hard cap and 20s improvement-bug tier ARE the owner's ruling** (sneak/prompts#41 (comment)). - The **`90s` backstop is our own proposed number and is NOT ratified** (sneak/prompts#41 (comment)). - The vendored text is therefore **the proposed canonical text, pending** sneak/prompts#42. If that PR lands with different numbers, this file must be re-vendored to match; it should not be hand-edited here either way. **Known mismatch with this repo's actual state, recorded not papered over.** Re-vendoring also picked up the paragraph at `REPO_POLICIES.md:266-271` mandating that canonical golangci-lint be installed commit-pinned via `go install ...@c0d3ddc9...`. This repo does **not** comply with that mechanism: `cc86473` in this same PR made linting Docker-only, and `script/bootstrap` now installs golangci-lint nowhere. The **version and commit match** (`v2.12.2` / `c0d3ddc9`); the **installation mechanism does not**. The vendored file is org-canonical and must not be edited downstream, so this is being raised upstream for the org text to accommodate Docker-only linting rather than patched here. `TESTING.md`'s stale "within the 30-second target" follows to 60. That edit is deliberately a single line so it merges cleanly when #97 lands. ### Verification All runs through `make` / `script/` entrypoints only; lint runs in Docker. **Ten consecutive `make check` runs, all green, none served from cache.** Go's test cache will happily report `ok pkg (cached)` without executing anything, which proves nothing about nondeterminism, so every run was forced to actually execute and each log was checked for zero `(cached)` lines: ``` check#1 exit=0 wall=32s resolver=2.895s fails=0 cached=0 check#2 exit=0 wall=27s resolver=2.872s fails=0 cached=0 check#3 exit=0 wall=47s resolver=2.729s fails=0 cached=0 check#4 exit=0 wall=36s resolver=2.885s fails=0 cached=0 check#5 exit=0 wall=32s resolver=2.899s fails=0 cached=0 check#6 exit=0 (harness tests added) fails=0 cached=0 check#7 exit=0 wall=41s resolver=2.891s fails=0 cached=0 check#8 exit=0 wall=36s resolver=2.871s fails=0 cached=0 check#9 exit=0 wall=26s resolver=2.846s fails=0 cached=0 check#10 exit=0 wall=46s resolver=2.833s fails=0 cached=0 ``` After the rework commit `87bce43`, `make check` green again end to end, zero `(cached)` test lines, Docker lint stage demonstrably executed rather than served from cache: ``` exit=0 cached=0 #8 [deps 4/4] RUN go mod download #8 CACHED #10 [lint 2/2] RUN golangci-lint run --config .golangci.yml ./... #10 30.72 0 issues. ``` The Docker lint stage was confirmed to execute rather than cache on each run: ``` #8 [deps 4/4] RUN go mod download #8 CACHED #9 [lint 1/2] COPY . . #9 DONE 0.6s #10 [lint 2/2] RUN golangci-lint run --config .golangci.yml ./... #10 DONE 39.7s ``` **`make test` wall time: 3.6-4.1s** across three timed uncached runs (`4093ms`, `3649ms`, `3729ms`). `internal/resolver` went from 2.0s to ~2.9s — the concurrency gate's cost. That is inside the 20s target, so no improvement bug is owed under the new two-tier rule. **Honest note on what these runs do and do not prove.** Live DNS was healthy throughout: **no live-DNS retry fired even once**, and no flake was observed either before or after the change (six pre-change baseline runs were also clean). So these runs demonstrate the change is not itself flaky and does not slow the suite; they do **not** demonstrate recovery from a real DNS failure, because no real DNS failure occurred. The original flakiness is *not reproduced* rather than *shown fixed*. The retry path is instead proven by `TestRetryLiveRecoversFromTransientFailure`, the only source of the single `retrying in 500ms` line in each log: ``` livedns_harness_test.go:90: transient: attempt 1 of 3 failed (no answer from live DNS), retrying in 500ms ``` ### Observation, not acted on The single most effective remaining lever against root-server rate limiting would be to stop `queryServers` always trying `rootServerList()` in the same order, so that load spreads across all thirteen roots instead of concentrating on `a.root-servers.net`. That is **production** code and this issue scopes the work as test-side, so it was left alone rather than changed quietly. It is now tracked for the owner's decision at #138. ### Interaction with #97 `TESTING.md` and `internal/resolver/resolver_test.go` auto-merge — that PR touches `resolver_test.go` only at the import block and the final timeout-test section, while this change touches the body of the file and adds two new files, and it leaves the mock-`DNSClient` timeout test at the tail of `resolver_test.go` entirely alone since removing it is that PR's job. `TODO.md` does conflict; that PR is already labelled `needs-rebase`, so this adds nothing material to its rebase. - **Go's test cache disabled, so every `make test` actually queries live DNS** — #139 `script/test` did not pass `-count=1`, so on an unchanged tree Go served the whole suite from cache: exit 0 in ~0.2s, every package marked `(cached)`, and not one DNS query made. This repo's suite exists to exercise live resolution on every run (`TESTING.md`), so that green asserted nothing — and it is exactly the green used as evidence that a flakiness fix works, since "run it a few times" stops being runs after the first. It had already misled two agents, each of whom forced uncached runs by hand. `-count=1` now disables caching on every invocation. **The conditional verbose rerun was missing and is added here.** `REPO_POLICIES.md` mandates it; the primary run had been unconditionally `-v`, which is the failure mode the policy exists to prevent (unreadable CI and `docker build` logs on success). Tests now run quiet, and only a failure triggers the `-v` rerun. Two properties matter and both are covered: the rerun carries `-count=1` too, so it cannot replay a cached copy of the failure it is meant to diagnose; and its exit status is discarded in favour of a forced `1`, so a flake that passes the second time cannot turn the build green — the first failure already proved the suite broken. **`-timeout 90s` untouched.** It is a deliberate backstop that must strictly exceed the 60s hard cap. **No special-casing for the Docker build**, which reaches the same script via `RUN make test`: a fresh container's test cache is empty, so `-count=1` is a no-op there, and carving out an exception would only create a second code path that could drift. ### Verification **Proven from a warm cache, not a cold one.** The suite was run first to populate the cache, and the pre-change state confirmed: ``` ok sneak.berlin/go/dnswatcher/internal/config (cached) coverage: 92.6% of statements ok sneak.berlin/go/dnswatcher/internal/resolver (cached) coverage: 77.1% of statements ...8 of 8 packages (cached)... real 0m0.203s ``` With the change applied to that same warm cache, three back-to-back runs on an unchanged tree, **zero `(cached)` markers** in all three: ``` # run 1 # run 2 ok .../internal/config 1.045s ok .../internal/config 1.040s ok .../internal/handlers 1.029s ok .../internal/handlers 1.021s ok .../internal/notify 1.148s ok .../internal/notify 1.252s ok .../internal/portcheck 1.026s ok .../internal/portcheck 1.025s ok .../internal/resolver 3.005s ok .../internal/resolver 2.846s ok .../internal/state 1.078s ok .../internal/state 1.097s ok .../internal/tlscheck 1.067s ok .../internal/tlscheck 1.079s ok .../internal/watcher 1.566s ok .../internal/watcher 1.544s real 0m4.174s real 0m4.015s # run 3: grep -c '(cached)' => 0 real 0m4.519s ``` **Measured uncached wall time: 4.0-4.5s** (was ~0.2s served from cache). Inside the 20s target, so no improvement bug is owed under the two-tier rule at sneak/prompts#41 (comment). **`-count=1` composes with `-race` and `-cover`**: both still present in the primary run, and the per-package coverage percentages above are identical to the pre-change values. **The failure path was exercised, not assumed.** A purpose-built flaky test that fails on its first run and passes every run after (marker file kept outside the module, so the tree stays byte-identical and a cached result would be served if caching were on) was run through the script: ``` exit code: 1 --- FAIL: TestFlaky (0.00s) FAIL flakeproof 0.013s --- Rerunning with -v for details --- --- PASS: TestFlaky (0.00s) ok flakeproof 1.014s ``` Quiet failure, verbose rerun that genuinely re-executed (it passed, so it did not replay the cached `FAIL`), and exit `1` regardless of the rerun passing. Scratch module removed afterwards. **`make check` green**, exit 0, with the Docker lint stage demonstrably executed rather than served from cache: ``` #8 [deps 4/4] RUN go mod download #8 CACHED #10 [lint 2/2] RUN golangci-lint run --config .golangci.yml ./... #10 21.36 0 issues. #10 DONE 26.2s ``` `README.md` and `TESTING.md` record why the cache is waived, and `TODO.md` is updated in the same commit. ### Question for the owner, not filed as a defect The Docker lint run emits `The linter 'gomodguard' is deprecated (since v2.12.0) due to: new major version. Replaced by gomodguard_v2.` It is pre-existing and out of this issue's scope. It is not filed as an issue here because `.golangci.yml` tracks the org-canonical config, so switching to `gomodguard_v2` looks like an upstream `sneak/prompts` decision rather than a per-repo fix. Say the word and it gets filed in whichever place you consider canonical. - **MIT `LICENSE` added; README states the licence** — #102 The repo had no licence file at all, so publicly readable code was all-rights-reserved by default and nobody could legally use it. `LICENSE` was also the last file missing from `REPO_POLICIES.md`'s required minimum. MIT, by standing org policy rather than a per-repo call: any public repo lacking a licence gets MIT, and a private one with no licence is already all-rights-reserved. `sneak/dnswatcher` is public (`private: false` on the Gitea repo record). `README.md`'s first line now names the licence, per the Description requirement, and the License section states MIT and points at the file instead of recording the decision as pending. `TODO.md` updated in the same commit. ### Verification `LICENSE` is the canonical MIT text byte-for-byte with only the copyright line filled in (`Copyright (c) 2026 sneak`) — no clauses added, removed, reworded, or reflowed. It was not typed from memory: the file was copied from an existing verbatim MIT template on disk and only the copyright line edited (`diff` against that template shows that one line and nothing else), then the result was word-diffed against SPDX `MIT.txt` fetched from `spdx/license-list-data`, ignoring only line wrapping and the placeholder — identical. `make fmt` did **not** touch `LICENSE`, and cannot: `script/fmt` runs `gofmt -s -w .` and `goimports -w .` only, with no prettier or markdown step in the repo, so no exclusion was needed. `make check` green, exit 0. Tests executed rather than replayed (zero `(cached)` lines, `internal/resolver 3.098s`), and the Docker lint stage ran rather than cached: ``` #10 [deps 4/4] RUN go mod download #10 CACHED #12 [lint 2/2] RUN golangci-lint run --config .golangci.yml ./... #12 36.44 0 issues. #12 DONE 36.7s ``` - **Comment-only corrections in `script/bootstrap`, `script/cibuild`, and `Dockerfile.lint`** — #137 Follow-up to this PR's own review. Nothing executable changed: the diff touches comment lines, one warning string, and `TODO.md`. - `script/bootstrap`'s header justified the pinned `goimports` install by claiming `script/fmt-check` runs it on the host. Verified against the script: `script/fmt-check` runs `gofmt -l .` and nothing else. The header now credits `script/fmt` alone. That `fmt-check` does not verify goimports at all is #119 and was deliberately left alone. - `script/cibuild`'s header still said the `Dockerfile` runs `make check`. It now describes the current file: lint stage runs `make fmt-check` and `golangci-lint`, builder stage runs `make test` and `make build`. - The `docker`-missing warning was three fragments, each re-prefixed with `bootstrap:` mid-clause. Now one sentence: `bootstrap: WARNING: docker not found; install it to run make lint and make docker.` - `Dockerfile.lint`'s comment explained why `golangci-lint config verify` is omitted but read as though the omission were free. It now states the residual risk: unknown top-level keys in `.golangci.yml` are silently ignored, so a mistyped or wrong-schema key lints clean while applying nothing. `config verify` was **not** added — the network-dependence reasoning stands. ### Verification `make check` green, exit 0. Tests executed rather than replayed (zero `(cached)` lines, `internal/resolver 2.820s`), Docker lint stage executed rather than cached: ``` #8 [deps 4/4] RUN go mod download #8 CACHED #10 [lint 2/2] RUN golangci-lint run --config .golangci.yml ./... #10 13.30 0 issues. ``` Comment-only confirmed by reading the whole diff: no statement, flag, or command changed anywhere. --- ## Issues closed by this merge The commits on `next` each carry a bare `(closes #N)` in their subject, but the references in the prose above are full URLs, which Gitea's auto-close parser does not act on. Listing them here in bare form so the merge to `main` definitively closes them rather than leaving them open to be re-picked up as idle work: Closes #93 Closes #102 Closes #134 Closes #137 Closes #139 Co-authored-by: sneak <sneak@sneak.berlin> Reviewed-on: #136 Co-authored-by: clawbot <clawbot@noreply.example.org> Co-committed-by: clawbot <clawbot@noreply.example.org>
2026-09-09 14:57:55 +02:00
parent 9347a2838b
commit e77e206fc4
19 changed files with 1905 additions and 322 deletions
@@ -0,0 +1,284 @@
+package resolver_test
+
+import (
+	"context"
+	"sync"
+	"testing"
+	"time"
+
+	"github.com/stretchr/testify/assert"
+
+	"sneak.berlin/go/dnswatcher/internal/resolver"
+)
+
+// Tests for the live-DNS harness in livedns_test.go itself. These
+// exercise pure logic and the retry/concurrency plumbing; they
+// perform no DNS resolution of any kind, so they neither mock DNS
+// nor depend on it.
+
+// Names for the synthetic status maps below. Nothing is ever queried
+// at them: they are map keys handed to the package's pure counting
+// helpers, not a stand-in for a nameserver.
+const (
+	nsExample1 = "ns1.example."
+	nsExample2 = "ns2.example."
+	nsExample3 = "ns3.example."
+	nsExample4 = "ns4.example."
+)
+
+func TestLiveQuorumIsStrictMajority(t *testing.T) {
+	t.Parallel()
+
+	cases := map[int]int{
+		0:  1,
+		1:  1,
+		2:  2,
+		3:  2,
+		4:  3,
+		5:  3,
+		13: 7,
+	}
+
+	for total, want := range cases {
+		assert.Equal(
+			t, want, liveQuorum(total),
+			"liveQuorum(%d)", total,
+		)
+	}
+}
+
+func TestStatusCountingIgnoresSilentNameservers(t *testing.T) {
+	t.Parallel()
+
+	results := map[string]*resolver.NameserverResponse{
+		nsExample1: {
+			Nameserver: nsExample1,
+			Status:     resolver.StatusOK,
+		},
+		nsExample2: {
+			Nameserver: nsExample2,
+			Status:     resolver.StatusOK,
+		},
+		nsExample3: {
+			Nameserver: nsExample3,
+			Status:     resolver.StatusTimeout,
+		},
+		nsExample4: {
+			Nameserver: nsExample4,
+			Status:     resolver.StatusError,
+		},
+	}
+
+	assert.Equal(
+		t, 2, countStatus(results, resolver.StatusOK),
+	)
+	assert.Equal(
+		t, 0, countStatus(results, resolver.StatusNXDomain),
+	)
+
+	// Two of four answered, which is short of the quorum of
+	// three: this is the state that triggers a retry rather
+	// than an assertion failure.
+	assert.Equal(t, 2, answeredCount(results))
+	assert.Less(t, answeredCount(results), liveQuorum(len(results)))
+
+	assert.Equal(
+		t,
+		"ns1.example.=ok ns2.example.=ok "+
+			"ns3.example.=timeout ns4.example.=error",
+		describeStatuses(results),
+	)
+}
+
+func TestRetryLiveRecoversFromTransientFailure(t *testing.T) {
+	t.Parallel()
+
+	const wantAttempts = 2
+
+	attempts := 0
+
+	retryLive(t, "transient", func(_ context.Context) error {
+		attempts++
+
+		if attempts < wantAttempts {
+			return errLiveNoAnswer
+		}
+
+		return nil
+	})
+
+	assert.Equal(t, wantAttempts, attempts)
+}
+
+func TestRetryLiveGivesEachAttemptADeadline(t *testing.T) {
+	t.Parallel()
+
+	retryLive(t, "deadline", func(ctx context.Context) error {
+		deadline, ok := ctx.Deadline()
+		assert.True(t, ok, "attempt should carry a deadline")
+
+		remaining := time.Until(deadline)
+
+		assert.LessOrEqual(t, remaining, liveAttemptTimeout)
+
+		// Lower bound too: without one this passes for a
+		// deadline far shorter than intended, which would
+		// silently turn every live attempt into an instant
+		// timeout.
+		assert.Greater(t, remaining, liveAttemptTimeout/2)
+
+		return nil
+	})
+}
+
+// TestUnsanctionedStatusesRejectsWrongAnswers is the regression test
+// for the defect this allowlist exists to prevent: a minority of
+// nameservers answering WRONGLY while quorum keeps the suite green.
+// nodata is the case that motivated it — it is a wrong answer, not
+// silence, and it was previously banned by neither test.
+func TestUnsanctionedStatusesRejectsWrongAnswers(t *testing.T) {
+	t.Parallel()
+
+	// Four nameservers, three OK and one answering nodata: a
+	// quorum of three is satisfied and no NXDOMAIN is present, so
+	// the old blocklist assertions both passed on this input.
+	results := map[string]*resolver.NameserverResponse{
+		nsExample1: {
+			Nameserver: nsExample1,
+			Status:     resolver.StatusOK,
+		},
+		nsExample2: {
+			Nameserver: nsExample2,
+			Status:     resolver.StatusOK,
+		},
+		nsExample3: {
+			Nameserver: nsExample3,
+			Status:     resolver.StatusOK,
+		},
+		nsExample4: {
+			Nameserver: nsExample4,
+			Status:     resolver.StatusNoData,
+		},
+	}
+
+	assert.GreaterOrEqual(
+		t,
+		countStatus(results, resolver.StatusOK),
+		liveQuorum(len(results)),
+	)
+	assert.Zero(t, countStatus(results, resolver.StatusNXDomain))
+
+	// nodata is an ANSWER, so it never triggers a retry: nothing
+	// but the allowlist stands between it and a false green.
+	assert.Equal(t, len(results), answeredCount(results))
+
+	assert.Equal(
+		t,
+		[]string{nsExample4 + "=nodata"},
+		unsanctionedStatuses(
+			results,
+			resolver.StatusOK,
+			resolver.StatusTimeout,
+			resolver.StatusError,
+		),
+		"nodata must be reported as an unsanctioned status",
+	)
+}
+
+func TestUnsanctionedStatusesToleratesSilenceOnly(t *testing.T) {
+	t.Parallel()
+
+	results := map[string]*resolver.NameserverResponse{
+		nsExample1: {
+			Nameserver: nsExample1,
+			Status:     resolver.StatusNXDomain,
+		},
+		nsExample2: {
+			Nameserver: nsExample2,
+			Status:     resolver.StatusTimeout,
+		},
+		nsExample3: {
+			Nameserver: nsExample3,
+			Status:     resolver.StatusError,
+		},
+	}
+
+	allowed := []string{
+		resolver.StatusNXDomain,
+		resolver.StatusTimeout,
+		resolver.StatusError,
+	}
+
+	assert.Empty(
+		t,
+		unsanctionedStatuses(results, allowed...),
+		"timeout and error are non-answers and are tolerated",
+	)
+
+	// The same silent nameservers do not count towards a quorum.
+	assert.Equal(t, 1, answeredCount(results))
+
+	// An unknown status is treated as silence by answeredCount —
+	// so it retries and fails loudly — and is unsanctioned by the
+	// allowlist rather than quietly permitted.
+	const laterStatus = "some-status-added-later"
+
+	results[nsExample4] = &resolver.NameserverResponse{
+		Nameserver: nsExample4,
+		Status:     laterStatus,
+	}
+
+	assert.Equal(t, 1, answeredCount(results))
+	assert.Equal(
+		t,
+		[]string{nsExample4 + "=" + laterStatus},
+		unsanctionedStatuses(results, allowed...),
+	)
+}
+
+func TestRunLiveBoundsConcurrency(t *testing.T) {
+	t.Parallel()
+
+	const workers = 24
+
+	var (
+		mu       sync.Mutex
+		wg       sync.WaitGroup
+		inFlight int
+		maxSeen  int
+	)
+
+	wg.Add(workers)
+
+	for range workers {
+		go func() {
+			defer wg.Done()
+
+			_ = runLive(func(_ context.Context) error {
+				mu.Lock()
+				inFlight++
+
+				if inFlight > maxSeen {
+					maxSeen = inFlight
+				}
+				mu.Unlock()
+
+				time.Sleep(time.Millisecond)
+
+				mu.Lock()
+				inFlight--
+				mu.Unlock()
+
+				return nil
+			})
+		}()
+	}
+
+	wg.Wait()
+
+	assert.Positive(t, maxSeen)
+	assert.LessOrEqual(
+		t, maxSeen, liveConcurrency,
+		"live queries must stay under the package-wide gate",
+	)
+}
@@ -0,0 +1,495 @@
+package resolver_test
+
+import (
+	"context"
+	"errors"
+	"fmt"
+	"slices"
+	"sort"
+	"strings"
+	"testing"
+	"time"
+
+	"sneak.berlin/go/dnswatcher/internal/resolver"
+)
+
+// ----------------------------------------------------------------
+// Live DNS test support
+// ----------------------------------------------------------------
+//
+// Every test in this package resolves against the real, live DNS —
+// see TESTING.md. Nothing here mocks, fakes, stubs, records or
+// replays DNS, and nothing here skips or gates a test: the helpers
+// below only change *how* the live queries are issued, so that a
+// single dropped UDP packet or one slow authoritative server does
+// not turn a correct resolver into a red build.
+//
+// Three mechanisms, all test-side:
+//
+//  1. Bounded concurrency. The package's tests are parallel and the
+//     build hosts have many cores, so without a limit every test
+//     starts its own iterative resolution at the same instant and
+//     they all hit the first root server in rootServerList() within
+//     a few milliseconds of each other. Root servers rate-limit
+//     that, which shows up as a different arbitrary subset of tests
+//     failing on each run. liveGate caps how many resolutions are
+//     in flight at once.
+//
+//  2. Retry with exponential backoff. Each live operation gets
+//     several attempts with its own timeout. The retry predicate is
+//     strictly transport-level — "did a nameserver answer at all" —
+//     never the assertion the test is making. A resolver that
+//     answers incorrectly still fails on the first attempt.
+//
+//  3. Quorum. Where an assertion spans several independent
+//     nameservers, a strict majority answering as expected is
+//     enough; a server that fails to answer is tolerated, while a
+//     server that answers *wrongly* still fails the test.
+//
+// The tolerance in (3) is expressed as an ALLOWLIST of sanctioned
+// statuses, never as a blocklist of known-bad ones. A blocklist bans
+// the one wrong answer its author thought of and silently admits
+// every other status, including any added to the resolver later; an
+// allowlist fails on anything nobody explicitly sanctioned. Silence
+// (timeout, error) is the only thing quorum exists to tolerate. A
+// *wrong answer* — nxdomain for a name that exists, ok for one that
+// does not, nodata for either — is never tolerated at any count.
+
+const (
+	// liveAttempts is how many times a live DNS operation is
+	// attempted before the test fails.
+	liveAttempts = 3
+
+	// liveAttemptTimeout bounds one attempt. Worst case for an
+	// operation is liveAttempts * liveAttemptTimeout plus the
+	// backoff — about 26 seconds, well inside the 90-second
+	// `go test -timeout` backstop even when several operations
+	// exhaust their attempts.
+	liveAttemptTimeout = 8 * time.Second
+
+	// liveBackoffBase is the delay after the first failed
+	// attempt; it is multiplied by liveBackoffFactor each time.
+	liveBackoffBase = 500 * time.Millisecond
+
+	// liveBackoffFactor is the exponential backoff multiplier.
+	liveBackoffFactor = 2
+
+	// liveConcurrency caps how many live resolutions may be in
+	// flight across the whole package at once.
+	liveConcurrency = 6
+
+	// minNameservers is the smallest nameserver count a
+	// well-run zone is expected to publish.
+	minNameservers = 2
+)
+
+// liveGate bounds concurrent live resolutions package-wide. It has
+// to be package scoped: the whole point is that it is shared by
+// every parallel test in the package.
+//
+//nolint:gochecknoglobals // package-wide live query rate limit
+var liveGate = make(chan struct{}, liveConcurrency)
+
+var (
+	// errLiveNoAnswer reports that a live operation produced no
+	// usable answer, which is retried rather than asserted on.
+	errLiveNoAnswer = errors.New("no answer from live DNS")
+
+	// errLiveNoQuorum reports that too few of a domain's
+	// nameservers answered for a quorum assertion to be made.
+	errLiveNoQuorum = errors.New("no nameserver quorum")
+)
+
+// runLive executes one attempt of a live operation, holding a slot
+// in liveGate for its duration and bounding it with its own
+// timeout.
+func runLive(op func(ctx context.Context) error) error {
+	liveGate <- struct{}{}
+	defer func() { <-liveGate }()
+
+	ctx, cancel := context.WithTimeout(
+		context.Background(), liveAttemptTimeout,
+	)
+	defer cancel()
+
+	return op(ctx)
+}
+
+// retryLive runs op until it reports success, retrying transport
+// failures with exponential backoff, and fails the test if every
+// attempt fails. op returns an error only for a failure to obtain
+// an answer — never for an answer the test disagrees with, which
+// belongs in an assertion so that it fails immediately. op stores
+// whatever it obtained where its caller can find it.
+func retryLive(
+	t *testing.T,
+	what string,
+	op func(ctx context.Context) error,
+) {
+	t.Helper()
+
+	var last error
+
+	backoff := liveBackoffBase
+
+	for attempt := range liveAttempts {
+		if attempt > 0 {
+			t.Logf(
+				"%s: attempt %d of %d failed (%v), "+
+					"retrying in %s",
+				what, attempt, liveAttempts, last, backoff,
+			)
+			time.Sleep(backoff)
+
+			backoff *= liveBackoffFactor
+		}
+
+		last = runLive(op)
+		if last == nil {
+			return
+		}
+	}
+
+	t.Fatalf(
+		"%s: no answer after %d live attempts: %v",
+		what, liveAttempts, last,
+	)
+}
+
+// liveQuorum is how many of total nameservers must agree for a
+// multi-nameserver assertion to hold: a strict majority.
+func liveQuorum(total int) int {
+	if total < 1 {
+		return 1
+	}
+
+	return total/2 + 1
+}
+
+// countStatus counts the responses carrying the given status.
+func countStatus(
+	results map[string]*resolver.NameserverResponse,
+	status string,
+) int {
+	n := 0
+
+	for _, resp := range results {
+		if resp.Status == status {
+			n++
+		}
+	}
+
+	return n
+}
+
+// liveAnswerStatuses is the closed set of statuses that count as a
+// nameserver having ANSWERED at all, whether or not the test agrees
+// with the answer. It is deliberately an allowlist: a status added
+// to the resolver later is treated as silence, so it can only ever
+// cause a retry and then a loud failure, never a quiet pass.
+func liveAnswerStatuses() []string {
+	return []string{
+		resolver.StatusOK,
+		resolver.StatusNXDomain,
+		resolver.StatusNoData,
+	}
+}
+
+// answeredCount counts the nameservers that produced an answer of
+// any kind, as opposed to failing or timing out.
+func answeredCount(
+	results map[string]*resolver.NameserverResponse,
+) int {
+	answers := liveAnswerStatuses()
+
+	n := 0
+
+	for _, resp := range results {
+		if slices.Contains(answers, resp.Status) {
+			n++
+		}
+	}
+
+	return n
+}
+
+// unsanctionedStatuses returns "nameserver=status" for every result
+// whose status the caller did not explicitly sanction, sorted for a
+// stable failure message. Callers pass the full closed set they will
+// accept — the expected answer plus whichever non-answers (timeout,
+// error) quorum is allowed to tolerate — so that any status outside
+// it fails the test by name.
+func unsanctionedStatuses(
+	results map[string]*resolver.NameserverResponse,
+	allowed ...string,
+) []string {
+	offenders := make([]string, 0, len(results))
+
+	for ns, resp := range results {
+		if slices.Contains(allowed, resp.Status) {
+			continue
+		}
+
+		offenders = append(
+			offenders, fmt.Sprintf("%s=%s", ns, resp.Status),
+		)
+	}
+
+	sort.Strings(offenders)
+
+	return offenders
+}
+
+// describeStatuses renders per-nameserver statuses for use in
+// assertion failure messages.
+func describeStatuses(
+	results map[string]*resolver.NameserverResponse,
+) string {
+	parts := make([]string, 0, len(results))
+	for ns, resp := range results {
+		parts = append(
+			parts, fmt.Sprintf("%s=%s", ns, resp.Status),
+		)
+	}
+
+	sort.Strings(parts)
+
+	return strings.Join(parts, " ")
+}
+
+// ----------------------------------------------------------------
+// Live operation wrappers
+// ----------------------------------------------------------------
+
+// liveFindAuthoritative resolves a domain's authoritative
+// nameservers, retrying until the delegation chain can be walked.
+func liveFindAuthoritative(
+	t *testing.T,
+	r *resolver.Resolver,
+	domain string,
+) []string {
+	t.Helper()
+
+	var out []string
+
+	retryLive(
+		t,
+		"FindAuthoritativeNameservers("+domain+")",
+		func(ctx context.Context) error {
+			ns, err := r.FindAuthoritativeNameservers(ctx, domain)
+			if err != nil {
+				return err
+			}
+
+			if len(ns) == 0 {
+				return fmt.Errorf(
+					"%w: %s has no nameservers",
+					errLiveNoAnswer, domain,
+				)
+			}
+
+			out = ns
+
+			return nil
+		},
+	)
+
+	return out
+}
+
+// liveLookupNS is liveFindAuthoritative through the LookupNS entry
+// point, so that both entry points stay independently exercised.
+func liveLookupNS(
+	t *testing.T,
+	r *resolver.Resolver,
+	domain string,
+) []string {
+	t.Helper()
+
+	var out []string
+
+	retryLive(
+		t,
+		"LookupNS("+domain+")",
+		func(ctx context.Context) error {
+			ns, err := r.LookupNS(ctx, domain)
+			if err != nil {
+				return err
+			}
+
+			if len(ns) == 0 {
+				return fmt.Errorf(
+					"%w: %s has no nameservers",
+					errLiveNoAnswer, domain,
+				)
+			}
+
+			out = ns
+
+			return nil
+		},
+	)
+
+	return out
+}
+
+// liveQueryNameserver queries one nameserver, retrying while that
+// nameserver fails to answer. NXDOMAIN and NODATA are answers and
+// are returned to the caller to assert on.
+func liveQueryNameserver(
+	t *testing.T,
+	r *resolver.Resolver,
+	nameserver string,
+	hostname string,
+) *resolver.NameserverResponse {
+	t.Helper()
+
+	what := fmt.Sprintf(
+		"QueryNameserver(%s, %s)", nameserver, hostname,
+	)
+
+	var out *resolver.NameserverResponse
+
+	retryLive(
+		t,
+		what,
+		func(ctx context.Context) error {
+			resp, err := r.QueryNameserver(
+				ctx, nameserver, hostname,
+			)
+			if err != nil {
+				return err
+			}
+
+			if resp.Status == resolver.StatusTimeout ||
+				resp.Status == resolver.StatusError {
+				return fmt.Errorf(
+					"%w: %s returned %s: %s",
+					errLiveNoAnswer, nameserver,
+					resp.Status, resp.Error,
+				)
+			}
+
+			out = resp
+
+			return nil
+		},
+	)
+
+	return out
+}
+
+// liveQueryAllNameservers queries every authoritative nameserver
+// for a hostname, retrying until a quorum of them has answered.
+// Individual nameservers that stay silent are left in the result
+// for the caller to account for.
+func liveQueryAllNameservers(
+	t *testing.T,
+	r *resolver.Resolver,
+	hostname string,
+) map[string]*resolver.NameserverResponse {
+	t.Helper()
+
+	var out map[string]*resolver.NameserverResponse
+
+	retryLive(
+		t,
+		"QueryAllNameservers("+hostname+")",
+		func(ctx context.Context) error {
+			results, err := r.QueryAllNameservers(ctx, hostname)
+			if err != nil {
+				return err
+			}
+
+			if len(results) == 0 {
+				return fmt.Errorf(
+					"%w: no nameservers queried for %s",
+					errLiveNoAnswer, hostname,
+				)
+			}
+
+			answered := answeredCount(results)
+			if answered < liveQuorum(len(results)) {
+				return fmt.Errorf(
+					"%w: %d of %d answered: %s",
+					errLiveNoQuorum, answered,
+					len(results), describeStatuses(results),
+				)
+			}
+
+			out = results
+
+			return nil
+		},
+	)
+
+	return out
+}
+
+// liveResolveIPs resolves a hostname that is expected to have
+// addresses, retrying until at least one is returned.
+func liveResolveIPs(
+	t *testing.T,
+	r *resolver.Resolver,
+	hostname string,
+) []string {
+	t.Helper()
+
+	var out []string
+
+	retryLive(
+		t,
+		"ResolveIPAddresses("+hostname+")",
+		func(ctx context.Context) error {
+			ips, err := r.ResolveIPAddresses(ctx, hostname)
+			if err != nil {
+				return err
+			}
+
+			if len(ips) == 0 {
+				return fmt.Errorf(
+					"%w: no addresses for %s",
+					errLiveNoAnswer, hostname,
+				)
+			}
+
+			out = ips
+
+			return nil
+		},
+	)
+
+	return out
+}
+
+// liveResolveIPsAllowingEmpty resolves a hostname that may legitimately
+// have no addresses, so the empty result is returned rather than
+// retried. Used for names that must not exist; the corresponding
+// QueryAllNameservers test is what proves the nameservers actively
+// said NXDOMAIN rather than merely staying silent.
+func liveResolveIPsAllowingEmpty(
+	t *testing.T,
+	r *resolver.Resolver,
+	hostname string,
+) []string {
+	t.Helper()
+
+	var out []string
+
+	retryLive(
+		t,
+		"ResolveIPAddresses("+hostname+")",
+		func(ctx context.Context) error {
+			ips, err := r.ResolveIPAddresses(ctx, hostname)
+			if err != nil {
+				return err
+			}
+
+			out = ips
+
+			return nil
+		},
+	)
+
+	return out
+}
@@ -32,32 +32,17 @@ func newTestResolver(t *testing.T) *resolver.Resolver {
 	return resolver.NewFromLogger(log)
 }

-func testContext(t *testing.T) context.Context {
-	t.Helper()
-
-	ctx, cancel := context.WithTimeout(
-		context.Background(), 60*time.Second,
-	)
-	t.Cleanup(cancel)
-
-	return ctx
-}
-
+// findOneNSForDomain picks one authoritative nameserver to aim a
+// test at. Live-DNS retry, concurrency and quorum handling live in
+// livedns_test.go.
 func findOneNSForDomain(
 	t *testing.T,
 	r *resolver.Resolver,
-	ctx context.Context, //nolint:revive // test helper
 	domain string,
 ) string {
 	t.Helper()

-	nameservers, err := r.FindAuthoritativeNameservers(
-		ctx, domain,
-	)
-	require.NoError(t, err)
-	require.NotEmpty(t, nameservers)
-
-	return nameservers[0]
+	return liveFindAuthoritative(t, r, domain)[0]
 }

 // ----------------------------------------------------------------
@@ -70,13 +55,7 @@ func TestFindAuthoritativeNameservers_ValidDomain(
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	nameservers, err := r.FindAuthoritativeNameservers(
-		ctx, "google.com",
-	)
-	require.NoError(t, err)
-	require.NotEmpty(t, nameservers)
+	nameservers := liveFindAuthoritative(t, r, "google.com")

 	hasGoogleNS := false

@@ -99,13 +78,9 @@ func TestFindAuthoritativeNameservers_Subdomain(
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
+	nameservers := liveFindAuthoritative(t, r, "www.google.com")

-	nameservers, err := r.FindAuthoritativeNameservers(
-		ctx, "www.google.com",
-	)
-	require.NoError(t, err)
-	require.NotEmpty(t, nameservers)
+	assert.NotEmpty(t, nameservers)
 }

 func TestFindAuthoritativeNameservers_ReturnsSorted(
@@ -114,12 +89,7 @@ func TestFindAuthoritativeNameservers_ReturnsSorted(
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	nameservers, err := r.FindAuthoritativeNameservers(
-		ctx, "google.com",
-	)
-	require.NoError(t, err)
+	nameservers := liveFindAuthoritative(t, r, "google.com")

 	assert.True(
 		t,
@@ -134,17 +104,8 @@ func TestFindAuthoritativeNameservers_Deterministic(
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	first, err := r.FindAuthoritativeNameservers(
-		ctx, "google.com",
-	)
-	require.NoError(t, err)
-
-	second, err := r.FindAuthoritativeNameservers(
-		ctx, "google.com",
-	)
-	require.NoError(t, err)
+	first := liveFindAuthoritative(t, r, "google.com")
+	second := liveFindAuthoritative(t, r, "google.com")

 	assert.Equal(t, first, second)
 }
@@ -155,17 +116,8 @@ func TestFindAuthoritativeNameservers_TrailingDot(
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	ns1, err := r.FindAuthoritativeNameservers(
-		ctx, "google.com",
-	)
-	require.NoError(t, err)
-
-	ns2, err := r.FindAuthoritativeNameservers(
-		ctx, "google.com.",
-	)
-	require.NoError(t, err)
+	ns1 := liveFindAuthoritative(t, r, "google.com")
+	ns2 := liveFindAuthoritative(t, r, "google.com.")

 	assert.Equal(t, ns1, ns2)
 }
@@ -176,13 +128,7 @@ func TestFindAuthoritativeNameservers_CloudflareDomain(
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	nameservers, err := r.FindAuthoritativeNameservers(
-		ctx, "cloudflare.com",
-	)
-	require.NoError(t, err)
-	require.NotEmpty(t, nameservers)
+	nameservers := liveFindAuthoritative(t, r, "cloudflare.com")

 	for _, ns := range nameservers {
 		assert.True(t, strings.HasSuffix(ns, "."),
@@ -199,13 +145,9 @@ func TestQueryNameserver_BasicA(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-	ns := findOneNSForDomain(t, r, ctx, "google.com")
+	ns := findOneNSForDomain(t, r, "google.com")
+	resp := liveQueryNameserver(t, r, ns, "www.google.com")

-	resp, err := r.QueryNameserver(
-		ctx, ns, "www.google.com",
-	)
-	require.NoError(t, err)
 	require.NotNil(t, resp)

 	assert.Equal(t, resolver.StatusOK, resp.Status)
@@ -222,13 +164,8 @@ func TestQueryNameserver_AAAA(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-	ns := findOneNSForDomain(t, r, ctx, "cloudflare.com")
-
-	resp, err := r.QueryNameserver(
-		ctx, ns, "cloudflare.com",
-	)
-	require.NoError(t, err)
+	ns := findOneNSForDomain(t, r, "cloudflare.com")
+	resp := liveQueryNameserver(t, r, ns, "cloudflare.com")

 	aaaaRecords := resp.Records["AAAA"]
 	require.NotEmpty(t, aaaaRecords,
@@ -247,13 +184,8 @@ func TestQueryNameserver_MX(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-	ns := findOneNSForDomain(t, r, ctx, "google.com")
-
-	resp, err := r.QueryNameserver(
-		ctx, ns, "google.com",
-	)
-	require.NoError(t, err)
+	ns := findOneNSForDomain(t, r, "google.com")
+	resp := liveQueryNameserver(t, r, ns, "google.com")

 	mxRecords := resp.Records["MX"]
 	require.NotEmpty(t, mxRecords,
@@ -265,13 +197,8 @@ func TestQueryNameserver_TXT(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-	ns := findOneNSForDomain(t, r, ctx, "google.com")
-
-	resp, err := r.QueryNameserver(
-		ctx, ns, "google.com",
-	)
-	require.NoError(t, err)
+	ns := findOneNSForDomain(t, r, "google.com")
+	resp := liveQueryNameserver(t, r, ns, "google.com")

 	txtRecords := resp.Records["TXT"]
 	require.NotEmpty(t, txtRecords,
@@ -297,14 +224,10 @@ func TestQueryNameserver_NXDomain(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-	ns := findOneNSForDomain(t, r, ctx, "google.com")
-
-	resp, err := r.QueryNameserver(
-		ctx, ns,
-		"this-surely-does-not-exist-xyz.google.com",
+	ns := findOneNSForDomain(t, r, "google.com")
+	resp := liveQueryNameserver(
+		t, r, ns, "this-surely-does-not-exist-xyz.google.com",
 	)
-	require.NoError(t, err)

 	assert.Equal(t, resolver.StatusNXDomain, resp.Status)
 }
@@ -313,13 +236,8 @@ func TestQueryNameserver_RecordsSorted(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-	ns := findOneNSForDomain(t, r, ctx, "google.com")
-
-	resp, err := r.QueryNameserver(
-		ctx, ns, "google.com",
-	)
-	require.NoError(t, err)
+	ns := findOneNSForDomain(t, r, "google.com")
+	resp := liveQueryNameserver(t, r, ns, "google.com")

 	for recordType, values := range resp.Records {
 		assert.True(
@@ -336,13 +254,8 @@ func TestQueryNameserver_ResponseIncludesNameserver(
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-	ns := findOneNSForDomain(t, r, ctx, "cloudflare.com")
-
-	resp, err := r.QueryNameserver(
-		ctx, ns, "cloudflare.com",
-	)
-	require.NoError(t, err)
+	ns := findOneNSForDomain(t, r, "cloudflare.com")
+	resp := liveQueryNameserver(t, r, ns, "cloudflare.com")

 	assert.Equal(t, ns, resp.Nameserver)
 }
@@ -353,14 +266,10 @@ func TestQueryNameserver_EmptyRecordsOnNXDomain(
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-	ns := findOneNSForDomain(t, r, ctx, "google.com")
-
-	resp, err := r.QueryNameserver(
-		ctx, ns,
-		"this-surely-does-not-exist-xyz.google.com",
+	ns := findOneNSForDomain(t, r, "google.com")
+	resp := liveQueryNameserver(
+		t, r, ns, "this-surely-does-not-exist-xyz.google.com",
 	)
-	require.NoError(t, err)

 	totalRecords := 0
 	for _, values := range resp.Records {
@@ -374,18 +283,9 @@ func TestQueryNameserver_TrailingDotHandling(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-	ns := findOneNSForDomain(t, r, ctx, "google.com")
-
-	resp1, err := r.QueryNameserver(
-		ctx, ns, "google.com",
-	)
-	require.NoError(t, err)
-
-	resp2, err := r.QueryNameserver(
-		ctx, ns, "google.com.",
-	)
-	require.NoError(t, err)
+	ns := findOneNSForDomain(t, r, "google.com")
+	resp1 := liveQueryNameserver(t, r, ns, "google.com")
+	resp2 := liveQueryNameserver(t, r, ns, "google.com.")

 	assert.Equal(t, resp1.Status, resp2.Status)
 }
@@ -398,15 +298,9 @@ func TestQueryAllNameservers_ReturnsAllNS(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
+	results := liveQueryAllNameservers(t, r, "google.com")

-	results, err := r.QueryAllNameservers(
-		ctx, "google.com",
-	)
-	require.NoError(t, err)
-	require.NotEmpty(t, results)
-
-	assert.GreaterOrEqual(t, len(results), 2)
+	assert.GreaterOrEqual(t, len(results), minNameservers)

 	for ns, resp := range results {
 		assert.Equal(t, ns, resp.Nameserver)
@@ -417,19 +311,36 @@ func TestQueryAllNameservers_AllReturnOK(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
+	results := liveQueryAllNameservers(t, r, "google.com")

-	results, err := r.QueryAllNameservers(
-		ctx, "google.com",
+	// A quorum, not unanimity: one authoritative server being
+	// slow or rate-limiting us is a property of the live
+	// internet, not a resolver defect.
+	assert.GreaterOrEqual(
+		t,
+		countStatus(results, resolver.StatusOK),
+		liveQuorum(len(results)),
+		"a quorum of nameservers should answer OK: %s",
+		describeStatuses(results),
 	)
-	require.NoError(t, err)

-	for ns, resp := range results {
-		assert.Equal(
-			t, resolver.StatusOK, resp.Status,
-			"NS %s should return OK", ns,
-		)
-	}
+	// Quorum tolerates SILENCE only. Every individual result must
+	// be either the expected answer or a non-answer: ok, timeout
+	// or error, and nothing else. Stated as a closed allowlist so
+	// that a wrong answer no one thought to ban — nxdomain and
+	// nodata today, any status added later — fails here rather
+	// than sliding through under the quorum.
+	assert.Empty(
+		t,
+		unsanctionedStatuses(
+			results,
+			resolver.StatusOK,
+			resolver.StatusTimeout,
+			resolver.StatusError,
+		),
+		"every nameserver must answer OK or not answer at all: %s",
+		describeStatuses(results),
+	)
 }

 func TestQueryAllNameservers_NXDomainFromAllNS(
@@ -438,20 +349,34 @@ func TestQueryAllNameservers_NXDomainFromAllNS(
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	results, err := r.QueryAllNameservers(
-		ctx,
-		"this-surely-does-not-exist-xyz.google.com",
+	results := liveQueryAllNameservers(
+		t, r, "this-surely-does-not-exist-xyz.google.com",
 	)
-	require.NoError(t, err)

-	for ns, resp := range results {
-		assert.Equal(
-			t, resolver.StatusNXDomain, resp.Status,
-			"NS %s should return nxdomain", ns,
-		)
-	}
+	assert.GreaterOrEqual(
+		t,
+		countStatus(results, resolver.StatusNXDomain),
+		liveQuorum(len(results)),
+		"a quorum of nameservers should report NXDOMAIN: %s",
+		describeStatuses(results),
+	)
+
+	// Silence is tolerated; any actual answer other than NXDOMAIN
+	// is not. Closed allowlist for the same reason as above: a
+	// server answering `ok` or `nodata` for a name that must not
+	// exist is a wrong answer, not a slow one.
+	assert.Empty(
+		t,
+		unsanctionedStatuses(
+			results,
+			resolver.StatusNXDomain,
+			resolver.StatusTimeout,
+			resolver.StatusError,
+		),
+		"every nameserver must report NXDOMAIN or not answer "+
+			"at all: %s",
+		describeStatuses(results),
+	)
 }

 // ----------------------------------------------------------------
@@ -462,11 +387,7 @@ func TestLookupNS_ValidDomain(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	nameservers, err := r.LookupNS(ctx, "google.com")
-	require.NoError(t, err)
-	require.NotEmpty(t, nameservers)
+	nameservers := liveLookupNS(t, r, "google.com")

 	for _, ns := range nameservers {
 		assert.True(t, strings.HasSuffix(ns, "."),
@@ -479,10 +400,7 @@ func TestLookupNS_Sorted(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	nameservers, err := r.LookupNS(ctx, "google.com")
-	require.NoError(t, err)
+	nameservers := liveLookupNS(t, r, "google.com")

 	assert.True(t, sort.StringsAreSorted(nameservers))
 }
@@ -491,15 +409,8 @@ func TestLookupNS_MatchesFindAuthoritative(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	fromLookup, err := r.LookupNS(ctx, "google.com")
-	require.NoError(t, err)
-
-	fromFind, err := r.FindAuthoritativeNameservers(
-		ctx, "google.com",
-	)
-	require.NoError(t, err)
+	fromLookup := liveLookupNS(t, r, "google.com")
+	fromFind := liveFindAuthoritative(t, r, "google.com")

 	assert.Equal(t, fromFind, fromLookup)
 }
@@ -512,11 +423,7 @@ func TestResolveIPAddresses_ReturnsIPs(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	ips, err := r.ResolveIPAddresses(ctx, "google.com")
-	require.NoError(t, err)
-	require.NotEmpty(t, ips)
+	ips := liveResolveIPs(t, r, "google.com")

 	for _, ip := range ips {
 		parsed := net.ParseIP(ip)
@@ -530,10 +437,7 @@ func TestResolveIPAddresses_Deduplicated(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	ips, err := r.ResolveIPAddresses(ctx, "google.com")
-	require.NoError(t, err)
+	ips := liveResolveIPs(t, r, "google.com")

 	seen := make(map[string]bool)

@@ -547,10 +451,7 @@ func TestResolveIPAddresses_Sorted(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	ips, err := r.ResolveIPAddresses(ctx, "google.com")
-	require.NoError(t, err)
+	ips := liveResolveIPs(t, r, "google.com")

 	assert.True(t, sort.StringsAreSorted(ips))
 }
@@ -561,13 +462,10 @@ func TestResolveIPAddresses_NXDomainReturnsEmpty(
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
-
-	ips, err := r.ResolveIPAddresses(
-		ctx,
-		"this-surely-does-not-exist-xyz.google.com",
+	ips := liveResolveIPsAllowingEmpty(
+		t, r, "this-surely-does-not-exist-xyz.google.com",
 	)
-	require.NoError(t, err)
+
 	assert.Empty(t, ips)
 }

@@ -575,11 +473,9 @@ func TestResolveIPAddresses_CloudflareDomain(t *testing.T) {
 	t.Parallel()

 	r := newTestResolver(t)
-	ctx := testContext(t)
+	ips := liveResolveIPs(t, r, "cloudflare.com")

-	ips, err := r.ResolveIPAddresses(ctx, "cloudflare.com")
-	require.NoError(t, err)
-	require.NotEmpty(t, ips)
+	assert.NotEmpty(t, ips)
 }

 // ----------------------------------------------------------------