dnswatcher/TODO.md

# Workflow

* branch (from `main`)
* do the work in Next Step
* move Next Step to the top of Completed Steps
* move the top item of Future Steps into Next Step
* commit (`TODO.md` changes in the same commit as the work)
* merge to `main` if the branch is not protected, otherwise open a PR
* push

# Status

pre-1.0. No git tags.

# Next Step

Add the README sections required by policy (Description, Getting Started,
Rationale, Design, TODO, License, Author) if any are still missing.

# Completed Steps

- 2026-09-29: `.golangci.yml` re-fetched unchanged from `sneak/prompts`. It
  replaces the deprecated `gomodguard` with `gomodguard_v2`, so `make lint` no
  longer warns about it, and turns on `depguard` with the org rule that keeps
  `net/http/httptest` out of files that are not tests. This repo had no `deny`
  entries of its own to carry forward (closes #123).
- 2026-09-29: nothing stands in for DNS any more. Watcher tests that look
  something up in DNS use the real resolver against live DNS servers and test
  record and nameserver changes by preparing the saved state a check starts
  from; the resolver timeout test queries an address that never answers, and
  `NewFromLoggerWithClient`, used only by its stand-in client, is gone. The
  live-DNS retry and concurrency limit moved to `internal/livedns`, which both
  test packages use. `TESTING.md` states the README's rule (closes #159).
- 2026-09-28: the inconsistency alert is sent once, on the check where two
  nameservers start to disagree or where a nameserver that disagrees first
  appears, instead of on every check while they disagree, and not again after
  a restart. Every pair of nameservers is compared, not only neighbours in
  sorted order of name (closes #158).
- 2026-09-28: DNS names in record values (CNAME, MX, SRV and NS targets) are
  lower-cased, so nameservers that answer in different letter case no longer
  count as inconsistent or as a record change (closes #157).
- 2026-09-28: `script/cibuild` and `script/docker` now pass
  `--no-cache-filter=lint,builder` so lint and tests run every build (closes
  #115).
- 2026-09-28: the server timeout test now drives `Run` and checks the
  `http.Server` it serves carries the timeouts; corrected the `ReadTimeout`
  note in that test (closes #120).
- 2026-09-28: upaas deploy readiness — runtime image runs as unprivileged
  `dnswatcher`, Docker `HEALTHCHECK`, startup fails when the data directory is
  not writable, README "Running under upaas" (closes #147).
- 2026-09-21: added behavioural tests for `internal/globals`,
  `internal/healthcheck`, and `internal/logger` (closes #110).
- 2026-09-21: `go mod tidy` dropped the redundant `golang.org/x/sync`
  `// indirect` line so `script/bootstrap` leaves a clean tree (#132)
- 2026-08-10: comment-only corrections to `script/bootstrap`,
  `script/cibuild`, and `Dockerfile.lint`. The `goimports` pin in
  `script/bootstrap` was justified by a claim that `script/fmt-check`
  runs it on the host; it does not (it runs `gofmt -l .` only), so the
  header now credits `script/fmt` alone. `script/cibuild` still claimed
  the `Dockerfile` runs `make check`, which stopped being true when
  linting moved to its own stage; it now describes the lint stage
  (`make fmt-check` plus `golangci-lint`) and the builder stage
  (`make test`, `make build`). The `docker`-missing warning in
  `script/bootstrap` reads as one sentence instead of three fragments
  each re-prefixed with `bootstrap:`. `Dockerfile.lint` now records the
  residual risk of omitting `golangci-lint config verify`: unknown
  top-level keys in `.golangci.yml` are silently ignored, so a mistyped
  key lints clean while applying nothing. No behaviour changed
- 2026-08-10: MIT `LICENSE` added at the repository root, closing the
  last gap in `REPO_POLICIES.md`'s required-minimum file list and
  removing the all-rights-reserved default that would otherwise have
  shipped with a 1.0 tag. The licence choice is the standing org policy
  (any public repo lacking a licence gets MIT; a private repo with no
  licence is already all-rights-reserved), and this repo is public. The
  file holds the canonical MIT text byte-for-byte with only the
  copyright line filled in (`Copyright (c) 2026 sneak`); no clauses were
  added, removed, or reflowed. `README.md`'s first line now names the
  licence, as the Description requirement demands, and the License
  section states MIT and points at the file instead of saying the choice
  is pending. `make fmt` covers only Go sources (`gofmt -s`,
  `goimports`), so it cannot reflow `LICENSE`
- 2026-08-10: the policy scaffold (`REPO_POLICIES.md`, `.editorconfig`,
  `.dockerignore`, `.gitea/workflows/check.yml`, and the `fmt-check`,
  `docker`, and hooks Makefile targets) is present; it landed piecemeal
  across the scripts-to-rule-them-all and policy commits rather than as
  the single commit this file once planned
- 2026-08-10: Go's test cache disabled for `script/test` via `-count=1`,
  so every invocation actually executes. A cached pass replays an
  earlier run's output without querying DNS at all, which in this repo
  means the suite's entire premise goes unexercised while the run
  reports green in under a second. The conditional verbose rerun that
  `REPO_POLICIES.md` mandates was added at the same time (the primary
  run had been unconditionally `-v`): quiet first, `-v` only on
  failure, `-count=1` on both, and exit 1 forced regardless of the
  rerun's result so a flake passing the second time cannot turn the
  build green. `-timeout 90s` left alone as the deliberate backstop
  above the 60s hard cap. Uncached suite runs ~4s, well inside the 20s
  target
- 2026-08-10: live-DNS test flakiness addressed by robustness rather
  than gating, per the owner's ruling on #93: new
  `internal/resolver/livedns_test.go` adds a package-wide concurrency
  gate (so parallel tests stop bursting at the first root server),
  retry with exponential backoff on transport failures only, and
  quorum instead of unanimity for multi-nameserver assertions. Quorum
  tolerates silence only: every per-nameserver status must be in a
  closed allowlist (`ok`/`timeout`/`error`, or
  `nxdomain`/`timeout`/`error`), so a wrong answer from a minority —
  `nodata` today, any status added later — fails the test instead of
  sliding through under the majority. The
  `make test` cap moved to the new org-wide 60s hard cap / 20s target
  with a 90s `-timeout` backstop; `REPO_POLICIES.md` re-vendored
  byte-identical from `sneak/prompts`. No mocks, no `-short`, no build
  tags, no skips, and no change to production resolver behaviour
- 2026-08-10: all linting moved into Docker: new root `Dockerfile.lint`
  on the digest-pinned `golangci/golangci-lint:v2.12.2` image,
  `script/lint` reduced to a thin wrapper that builds it with
  `--no-cache-filter=lint` so the linter actually executes every run,
  golangci-lint install dropped from `script/bootstrap` (goimports
  stays, `script/fmt` needs it on the host), and the root `Dockerfile`
  given its own lint stage so its build no longer recurses through
  `make check` into `script/lint`. `golangci-lint config verify` is
  deliberately omitted: it fetches its schema over an unpinned live
  HTTPS call
- 2026-08-09: in-flight notification deliveries are now drained at
  shutdown (#106): `notify.New` registers an fx `OnStop` hook that waits
  on a `sync.WaitGroup` of tracked delivery goroutines, bounded by the
  `OnStop` context; on expiry the outstanding count is logged at warn
  level and parked retry backoffs are released instead of being dropped
  silently, and deliveries submitted after the drain begins are refused
  so shutdown cannot be extended indefinitely; an `OnStop` context that
  is already expired on entry with nothing outstanding drains quietly
  rather than warning about deliveries that were never abandoned
- 2026-08-09: `http.Server` now sets all four socket-level timeouts
  (`ReadTimeout` 15s, `ReadHeaderTimeout` 10s, `WriteTimeout` 75s,
  `IdleTimeout` 120s) as named constants in `internal/server/server.go`,
  closing the slowloris / unreaped-keep-alive exposure required by
  `REPO_POLICIES.md` before 1.0; `WriteTimeout` is deliberately greater
  than the 60s `chimw.Timeout` handler budget so that budget stays
  reachable, and tests in `internal/server` pin both the non-zero
  values and that relationship (#99)
- 2026-08-09: security response headers middleware
  (`SecurityHeaders()` in `internal/middleware/middleware.go`)
  registered globally in `internal/server/routes.go`, so HSTS, CSP,
  `X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy`, and
  `Permissions-Policy` are set on every response including `/s/...` and
  `/metrics`; the CSP needs no `unsafe-inline`/`unsafe-eval` because the
  dashboard ships no JavaScript and no inline styles; HSTS is emitted
  unconditionally per policy (TLS-terminating proxy in front). Remaining
  1.0 hardening items — `http.Server` timeouts, request body limits,
  rate limiting, CORS scoping — are tracked separately
- 2026-08-07: golangci-lint bumped to v2.12.2 (commit-pinned installs
  in `Dockerfile` and `script/bootstrap`); `.golangci.yml` set to the
  org-standard v2-schema config used across the org's repos
  (owner-authorized; same file is being landed as canonical via prompts
  PR #24), with settings under `linters.settings` so the
  lll/funlen/cyclop/dupl thresholds apply; fixed the resulting
  `goconst`, `dupl`, and `lll` findings; the informational `gomodguard`
  deprecation warning under this config is accepted
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
  Makefile shims, README Entrypoints section
- 2026-02-20: iterative DNS resolver implemented; tests made hermetic
  with mocked DNS (origin/feature/resolver, unmerged)
- 2026-02-20: CI actions and go install refs pinned to commit SHAs;
  Gitea Actions workflow for make check (origin/ci/make-check, unmerged)
- 2026-02-20: watcher monitoring orchestrator merged to main (#8)
- 2026-02-20: DOMAINS/HOSTNAMES unified into single TARGETS config (#11)
- 2026-02-19: TCP port connectivity checker, made concurrent with port
  validation; gosec G704 SSRF findings fixed without suppression
  (feature branches, unmerged)
- 2026-02-19: TLS certificate inspector with no-peer-certificates error
  path and IP SANs (feature branch, unmerged)
- 2026-02-19: gosec SSRF and formatting fixes on main
- 2026-02-19: initial scaffold with per-nameserver DNS monitoring model

# Future Steps

Compliance:

- Pin Dockerfile base images by sha256 and ensure the Docker build runs
  make check

Branch reconciliation:

- Sync local checkout with origin: local main is 8 commits behind
  origin/main; local feature/resolver has diverged from
  origin/feature/resolver, which already implements the resolver
- Merge in-flight branches to main once green: feature/resolver,
  ci/make-check, feature/portcheck-implementation,
  feature/tlscheck-implementation

Resolver (plan from untracked TODO.md; largely implemented on
origin/feature/resolver, verify each item before closing):

- Add github.com/miekg/dns dependency
- roots.go: hardcoded IANA root server list (a through m, IPv4/IPv6),
  rootServers() returning ip:53 strings
- query.go: low-level query(ctx, server, name, qtype): UDP with TCP
  fallback on truncation, RD=0, context respected, 5s per-query timeout,
  returns raw *dns.Msg
- trace.go: iterative delegation chasing from roots: referral detection
  (NOERROR, empty answer, NS in authority), glue extraction with
  bailiwick check, out-of-bailiwick NS resolved with recursion guard,
  delegation depth limit (20), retry across nameservers on failure, do
  not chase CNAMEs inside trace
- FindAuthoritativeNameservers: NS set via trace, sorted, FQDN
  normalized, trailing dot handled; must pass its 9 tests
- QueryNameserver: resolve NS host to IPs, query A/AAAA/CNAME/MX/TXT/
  SRV/CAA/NS, build NameserverResponse with status mapping (OK,
  NXDomain, NoData, Error), documented record formatting, sorted values,
  lame delegation detection; must pass its 16 tests
- QueryAllNameservers: find NS set for parent domain (public suffix
  list), query all NS in parallel with bounded concurrency, return map
  even when all fail, context cancellation; must pass its 4 tests
- LookupNS: thin wrapper over FindAuthoritativeNameservers, sorted,
  identical results; must pass its 3 tests
- ResolveIPAddresses: collect A/AAAA from all NS, follow CNAME chains
  with MaxCNAMEDepth, dedupe, sort, NXDOMAIN returns empty slice with
  nil error; must pass its 9 tests
- All 39 resolver tests pass, make check green, merge to main

Watcher (internal/watcher/watcher.go):

- Scheduling loop in Run(ctx): initial check on startup, separate
  tickers for DNS/port and TLS intervals, persist state via state.Save()
  after each cycle, clean shutdown on context cancel
- Domain check: LookupNS, compare to stored state, store silently on
  first run, notify with old/new NS lists on change
- Hostname check: QueryAllNameservers, compare per-NS records; notify on
  record changes, NS failure, NS recovery, inconsistency detected,
  inconsistency resolved, empty response; store silently on first run
- Port check: ResolveIPAddresses, check ports 80 and 443 per IP, notify
  on open/closed transitions, handle new and disappeared IPs
- TLS check: for each open IP:443, CheckCertificate; notify on expiry
  warning, certificate change (CN/issuer/SANs), TLS failure/recovery

Port checker (internal/portcheck/portcheck.go):

- Tests against known-open ports and RFC documentation IPs
- CheckPort: net.DialTimeout (5s), context respected; (true, nil) open,
  (false, nil) closed/timeout/refused, error only for unexpected
  failures

TLS checker (internal/tlscheck/tlscheck.go):

- Tests against known public HTTPS servers, verify fields populated
- CheckCertificate: tls.Dial to specific IP:443 with hostname as SNI;
  extract subject CN, issuer CN and org, NotAfter, SANs; error on
  handshake failure

Notification service (internal/notify/notify.go, Slack/Mattermost/ntfy
backends exist):

- Structured notification types: DNS change, port change, TLS expiry,
  TLS change, NS failure, NS recovery, NS inconsistency
- Per-backend formatting: Slack/Mattermost attachment colors (red
  failures/expiry, yellow warnings, green recoveries, blue info); ntfy
  priorities (urgent failures, high warnings, default changes, low
  recoveries); include hostname, nameserver, old/new values, timestamps

HTTP API handlers:

- Wire *state.State and *watcher.Watcher into handler params
- GET /api/v1/status: full state snapshot as JSON
- GET /api/v1/domains: domain states with NS records and last-checked
- GET /api/v1/hostnames: hostname states with per-NS record data

Infrastructure notes (from untracked TODO.md):

- Module path sneak.berlin/go/dnswatcher differs from the git.eeqj.de
  remote intentionally; do not "fix" it
- Dependencies: github.com/miekg/dns, golang.org/x/net/publicsuffix
- DNS is never mocked; tests that look something up in DNS query live DNS
  servers (README, "No DNS mocking. Ever.")