clawbot
aa3da062a2
check / check (push) Failing after 1m58s
The DNS-mock removal deleted TestQueryNameserverIP_Timeout and left a comment in its place, so the resolver's StatusTimeout / StatusError classification branch went untested. The stated obstacle was that a query to a black-holed RFC 5737 address comes back StatusOK, because the build environment transparently intercepts UDP/53 and answers it locally. That is a property of that environment, not of the resolver, and it only rules out choosing a remote address. internal/resolver/transport_test.go binds real nameservers on 127.0.0.1 instead and aims the query at them: one silent on A queries and answering every other type (StatusTimeout), one answering SERVFAIL (StatusError), and one address with nothing listening, which is refused rather than dropped and so classifies as NoData. This is not a mock — no DNSClient is substituted. The resolver dials a real socket, writes a real query with the real miekg/dns client, and applies its real deadline and real classification logic to what comes back. Substituting the client is what TESTING.md bans; choosing the server is not, and the resolver is aimed at a caller-chosen nameserver in production too. queryDNS now dials a nameserver address that already carries a port as written, defaulting to 53 only for a bare address. That is what makes a nameserver on any other port reachable, on loopback or otherwise. Silence on one record type rather than all eight keeps the timeout test to two query timeouts (4s) instead of sixteen (32s), and it is asserted: the test fails if it ever costs more than 8s. The watcher's assertStatePopulated and TestDomainPortAndTLSChecks asserted only that hostname, port and certificate state were non-empty, plus non-zero checker call counts. Neither was vacuous, but neither would have caught the watcher resolving the wrong addresses. The port and TLS test doubles now record their arguments, and both tests assert that the state keys and the arguments the checkers were actually called with match the addresses live DNS returned — exactly those, no more and no fewer. Verified by mutation: making the watcher drop all but one resolved address fails both tests, and it passed both of them before. TESTING.md records why a loopback nameserver is not a mock, so the new tests are not mistaken for a violation of the rule they respect.
245 lines
13 KiB
Markdown
245 lines
13 KiB
Markdown
# Workflow
|
|
|
|
* branch (from `main`)
|
|
* do the work in Next Step
|
|
* move Next Step to the top of Completed Steps
|
|
* move the top item of Future Steps into Next Step
|
|
* commit (`TODO.md` changes in the same commit as the work)
|
|
* merge to `main` if the branch is not protected, otherwise open a PR
|
|
* push
|
|
|
|
# Status
|
|
|
|
pre-1.0. No git tags. Core resolver work in flight on feature/resolver
|
|
(dirty: internal/resolver/resolver_test.go). Local checkout has diverged
|
|
from origin: origin/main is 8 commits ahead (watcher orchestrator,
|
|
unified TARGETS) and origin/feature/resolver already contains the full
|
|
iterative resolver implementation. DNS mocking is banned in this repo
|
|
(see `TESTING.md`): all tests use live DNS only. The hermetic mocked
|
|
tests previously noted on `feature/resolver` are gone from its current
|
|
tip, which carries a live-DNS suite against `*.dns.sneak.cloud`.
|
|
|
|
# Next Step
|
|
|
|
Add the README sections required by policy (Description, Getting Started,
|
|
Rationale, Design, TODO, License, Author) if any are still missing.
|
|
|
|
# Completed Steps
|
|
|
|
- 2026-09-03: restored the transport-failure classification coverage
|
|
the DNS-mock removal had dropped, and tightened two over-tolerant
|
|
watcher assertions. `internal/resolver/transport_test.go` covers
|
|
`StatusTimeout`, `StatusError` and the connection-refused path by
|
|
binding real nameservers on loopback rather than by mocking
|
|
`DNSClient` or by aiming a query at a remote address and hoping the
|
|
network black-holes it; `queryDNS` now honours a port already
|
|
present in a nameserver address, which is what lets a query be
|
|
aimed at one. The watcher's `assertStatePopulated` and
|
|
`TestDomainPortAndTLSChecks` now assert that port and certificate
|
|
state, and the arguments the port and TLS checkers were called
|
|
with, match the addresses live DNS returned — previously they
|
|
asserted only that those sets were non-empty, which would not have
|
|
caught resolving the wrong addresses. `TESTING.md` records why a
|
|
loopback nameserver is not a mock.
|
|
|
|
- 2026-08-10: comment-only corrections to `script/bootstrap`,
|
|
`script/cibuild`, and `Dockerfile.lint`. The `goimports` pin in
|
|
`script/bootstrap` was justified by a claim that `script/fmt-check`
|
|
runs it on the host; it does not (it runs `gofmt -l .` only), so the
|
|
header now credits `script/fmt` alone. `script/cibuild` still claimed
|
|
the `Dockerfile` runs `make check`, which stopped being true when
|
|
linting moved to its own stage; it now describes the lint stage
|
|
(`make fmt-check` plus `golangci-lint`) and the builder stage
|
|
(`make test`, `make build`). The `docker`-missing warning in
|
|
`script/bootstrap` reads as one sentence instead of three fragments
|
|
each re-prefixed with `bootstrap:`. `Dockerfile.lint` now records the
|
|
residual risk of omitting `golangci-lint config verify`: unknown
|
|
top-level keys in `.golangci.yml` are silently ignored, so a mistyped
|
|
key lints clean while applying nothing. No behaviour changed
|
|
- 2026-08-10: MIT `LICENSE` added at the repository root, closing the
|
|
last gap in `REPO_POLICIES.md`'s required-minimum file list and
|
|
removing the all-rights-reserved default that would otherwise have
|
|
shipped with a 1.0 tag. The licence choice is the standing org policy
|
|
(any public repo lacking a licence gets MIT; a private repo with no
|
|
licence is already all-rights-reserved), and this repo is public. The
|
|
file holds the canonical MIT text byte-for-byte with only the
|
|
copyright line filled in (`Copyright (c) 2026 sneak`); no clauses were
|
|
added, removed, or reflowed. `README.md`'s first line now names the
|
|
licence, as the Description requirement demands, and the License
|
|
section states MIT and points at the file instead of saying the choice
|
|
is pending. `make fmt` covers only Go sources (`gofmt -s`,
|
|
`goimports`), so it cannot reflow `LICENSE`
|
|
- 2026-08-10: the policy scaffold (`REPO_POLICIES.md`, `.editorconfig`,
|
|
`.dockerignore`, `.gitea/workflows/check.yml`, and the `fmt-check`,
|
|
`docker`, and hooks Makefile targets) is present; it landed piecemeal
|
|
across the scripts-to-rule-them-all and policy commits rather than as
|
|
the single commit this file once planned
|
|
- 2026-08-10: Go's test cache disabled for `script/test` via `-count=1`,
|
|
so every invocation actually executes. A cached pass replays an
|
|
earlier run's output without querying DNS at all, which in this repo
|
|
means the suite's entire premise goes unexercised while the run
|
|
reports green in under a second. The conditional verbose rerun that
|
|
`REPO_POLICIES.md` mandates was added at the same time (the primary
|
|
run had been unconditionally `-v`): quiet first, `-v` only on
|
|
failure, `-count=1` on both, and exit 1 forced regardless of the
|
|
rerun's result so a flake passing the second time cannot turn the
|
|
build green. `-timeout 90s` left alone as the deliberate backstop
|
|
above the 60s hard cap. Uncached suite runs ~4s, well inside the 20s
|
|
target
|
|
- 2026-08-10: live-DNS test flakiness addressed by robustness rather
|
|
than gating, per the owner's ruling on #93: new
|
|
`internal/resolver/livedns_test.go` adds a package-wide concurrency
|
|
gate (so parallel tests stop bursting at the first root server),
|
|
retry with exponential backoff on transport failures only, and
|
|
quorum instead of unanimity for multi-nameserver assertions. Quorum
|
|
tolerates silence only: every per-nameserver status must be in a
|
|
closed allowlist (`ok`/`timeout`/`error`, or
|
|
`nxdomain`/`timeout`/`error`), so a wrong answer from a minority —
|
|
`nodata` today, any status added later — fails the test instead of
|
|
sliding through under the majority. The
|
|
`make test` cap moved to the new org-wide 60s hard cap / 20s target
|
|
with a 90s `-timeout` backstop; `REPO_POLICIES.md` re-vendored
|
|
byte-identical from `sneak/prompts`. No mocks, no `-short`, no build
|
|
tags, no skips, and no change to production resolver behaviour
|
|
- 2026-08-10: all linting moved into Docker: new root `Dockerfile.lint`
|
|
on the digest-pinned `golangci/golangci-lint:v2.12.2` image,
|
|
`script/lint` reduced to a thin wrapper that builds it with
|
|
`--no-cache-filter=lint` so the linter actually executes every run,
|
|
golangci-lint install dropped from `script/bootstrap` (goimports
|
|
stays, `script/fmt` needs it on the host), and the root `Dockerfile`
|
|
given its own lint stage so its build no longer recurses through
|
|
`make check` into `script/lint`. `golangci-lint config verify` is
|
|
deliberately omitted: it fetches its schema over an unpinned live
|
|
HTTPS call
|
|
- 2026-08-07: DNS mocking removed from the entire test suite; watcher
|
|
tests now drive the real iterative resolver against live DNS and
|
|
`TESTING.md` bans DNS mocks in every package (`remove-dns-mocking`
|
|
branch)
|
|
- 2026-08-07: golangci-lint bumped to v2.12.2 (commit-pinned installs
|
|
in `Dockerfile` and `script/bootstrap`); `.golangci.yml` set to the
|
|
org-standard v2-schema config used across the org's repos
|
|
(owner-authorized; same file is being landed as canonical via prompts
|
|
PR #24), with settings under `linters.settings` so the
|
|
lll/funlen/cyclop/dupl thresholds apply; fixed the resulting
|
|
`goconst`, `dupl`, and `lll` findings; the informational `gomodguard`
|
|
deprecation warning under this config is accepted
|
|
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
|
|
Makefile shims, README Entrypoints section
|
|
- 2026-02-20: iterative DNS resolver implemented; tests made hermetic
|
|
with mocked DNS (origin/feature/resolver, unmerged; superseded — DNS
|
|
mocking is banned, see `TESTING.md`)
|
|
- 2026-02-20: CI actions and go install refs pinned to commit SHAs;
|
|
Gitea Actions workflow for make check (origin/ci/make-check, unmerged)
|
|
- 2026-02-20: watcher monitoring orchestrator merged to main (#8)
|
|
- 2026-02-20: DOMAINS/HOSTNAMES unified into single TARGETS config (#11)
|
|
- 2026-02-19: TCP port connectivity checker, made concurrent with port
|
|
validation; gosec G704 SSRF findings fixed without suppression
|
|
(feature branches, unmerged)
|
|
- 2026-02-19: TLS certificate inspector with no-peer-certificates error
|
|
path and IP SANs (feature branch, unmerged)
|
|
- 2026-02-19: gosec SSRF and formatting fixes on main
|
|
- 2026-02-19: initial scaffold with per-nameserver DNS monitoring model
|
|
|
|
# Future Steps
|
|
|
|
Compliance:
|
|
|
|
- Pin Dockerfile base images by sha256 and ensure the Docker build runs
|
|
make check
|
|
|
|
Branch reconciliation:
|
|
|
|
- Sync local checkout with origin: local main is 8 commits behind
|
|
origin/main; local feature/resolver has diverged from
|
|
origin/feature/resolver, which already implements the resolver
|
|
- Merge in-flight branches to main once green: feature/resolver
|
|
(confirm its tests remain live-DNS — DNS mocking is banned, see
|
|
`TESTING.md`), ci/make-check, feature/portcheck-implementation,
|
|
feature/tlscheck-implementation
|
|
|
|
Resolver (plan from untracked TODO.md; largely implemented on
|
|
origin/feature/resolver, verify each item before closing):
|
|
|
|
- Add github.com/miekg/dns dependency
|
|
- roots.go: hardcoded IANA root server list (a through m, IPv4/IPv6),
|
|
rootServers() returning ip:53 strings
|
|
- query.go: low-level query(ctx, server, name, qtype): UDP with TCP
|
|
fallback on truncation, RD=0, context respected, 5s per-query timeout,
|
|
returns raw *dns.Msg
|
|
- trace.go: iterative delegation chasing from roots: referral detection
|
|
(NOERROR, empty answer, NS in authority), glue extraction with
|
|
bailiwick check, out-of-bailiwick NS resolved with recursion guard,
|
|
delegation depth limit (20), retry across nameservers on failure, do
|
|
not chase CNAMEs inside trace
|
|
- FindAuthoritativeNameservers: NS set via trace, sorted, FQDN
|
|
normalized, trailing dot handled; must pass its 9 tests
|
|
- QueryNameserver: resolve NS host to IPs, query A/AAAA/CNAME/MX/TXT/
|
|
SRV/CAA/NS, build NameserverResponse with status mapping (OK,
|
|
NXDomain, NoData, Error), documented record formatting, sorted values,
|
|
lame delegation detection; must pass its 16 tests
|
|
- QueryAllNameservers: find NS set for parent domain (public suffix
|
|
list), query all NS in parallel with bounded concurrency, return map
|
|
even when all fail, context cancellation; must pass its 4 tests
|
|
- LookupNS: thin wrapper over FindAuthoritativeNameservers, sorted,
|
|
identical results; must pass its 3 tests
|
|
- ResolveIPAddresses: collect A/AAAA from all NS, follow CNAME chains
|
|
with MaxCNAMEDepth, dedupe, sort, NXDOMAIN returns empty slice with
|
|
nil error; must pass its 9 tests
|
|
- All 39 resolver tests pass, make check green, merge to main
|
|
|
|
Watcher (internal/watcher/watcher.go):
|
|
|
|
- Scheduling loop in Run(ctx): initial check on startup, separate
|
|
tickers for DNS/port and TLS intervals, persist state via state.Save()
|
|
after each cycle, clean shutdown on context cancel
|
|
- Domain check: LookupNS, compare to stored state, store silently on
|
|
first run, notify with old/new NS lists on change
|
|
- Hostname check: QueryAllNameservers, compare per-NS records; notify on
|
|
record changes, NS failure, NS recovery, inconsistency detected,
|
|
inconsistency resolved, empty response; store silently on first run
|
|
- Port check: ResolveIPAddresses, check ports 80 and 443 per IP, notify
|
|
on open/closed transitions, handle new and disappeared IPs
|
|
- TLS check: for each open IP:443, CheckCertificate; notify on expiry
|
|
warning, certificate change (CN/issuer/SANs), TLS failure/recovery
|
|
|
|
Port checker (internal/portcheck/portcheck.go):
|
|
|
|
- Tests against known-open ports and RFC documentation IPs
|
|
- CheckPort: net.DialTimeout (5s), context respected; (true, nil) open,
|
|
(false, nil) closed/timeout/refused, error only for unexpected
|
|
failures
|
|
|
|
TLS checker (internal/tlscheck/tlscheck.go):
|
|
|
|
- Tests against known public HTTPS servers, verify fields populated
|
|
- CheckCertificate: tls.Dial to specific IP:443 with hostname as SNI;
|
|
extract subject CN, issuer CN and org, NotAfter, SANs; error on
|
|
handshake failure
|
|
|
|
Notification service (internal/notify/notify.go, Slack/Mattermost/ntfy
|
|
backends exist):
|
|
|
|
- Structured notification types: DNS change, port change, TLS expiry,
|
|
TLS change, NS failure, NS recovery, NS inconsistency
|
|
- Per-backend formatting: Slack/Mattermost attachment colors (red
|
|
failures/expiry, yellow warnings, green recoveries, blue info); ntfy
|
|
priorities (urgent failures, high warnings, default changes, low
|
|
recoveries); include hostname, nameserver, old/new values, timestamps
|
|
|
|
HTTP API handlers:
|
|
|
|
- Wire *state.State and *watcher.Watcher into handler params
|
|
- GET /api/v1/status: full state snapshot as JSON
|
|
- GET /api/v1/domains: domain states with NS records and last-checked
|
|
- GET /api/v1/hostnames: hostname states with per-NS record data
|
|
|
|
Infrastructure notes (from untracked TODO.md):
|
|
|
|
- Module path sneak.berlin/go/dnswatcher differs from the git.eeqj.de
|
|
remote intentionally; do not "fix" it
|
|
- Dependencies: github.com/miekg/dns, golang.org/x/net/publicsuffix
|
|
- Resolver tests originally used live DNS against `*.dns.sneak.cloud`
|
|
(required records documented in the test file header); `main` now
|
|
tests against live public DNS. DNS mocking is banned (see
|
|
`TESTING.md`); never reintroduce hermetic mocked DNS tests
|