sneak/dnswatcher
1
0
Fork 0
Code Issues 4 Pull Requests 1 Actions Packages Projects Releases Wiki Activity

docs: bring TODO.md up to date (closes #146)
check / check (push) Successful in 1m8s
Details

Browse Source 
Next Step and Future Steps list the open issues by full URL, in the order
of the review on #144; issues
the review does not name sit next to the entries they relate to, and
#144 itself comes last. Left
out: #146, which this change
closes; #59, DNSSEC, ruled
post-1.0; issues assigned to sneak, which wait on the owner. Shipped work,
the dropped domains and hostnames endpoints and the line about mocked
resolver tests are gone. Every Completed Steps entry is at most two lines;
none was dropped. Workflow branches from `next` and opens PRs against it.
Wrapped by hand at 80 columns: `make fmt` does not format Markdown yet
(#119). The old Next Step is now
#173.

Model: opus-5-5
This commit is contained in:
sneak
2026-10-01 19:20:26 +00:00
parent 3390d7065e
commit d38b9dc957
1 changed files with 74 additions and 247 deletions
Download Patch File Download Diff File Expand all files Collapse all files
TODO.md
+74 -247
View File
@@ -1,24 +1,27 @@
# Workflow # Workflow
* branch (from `main`) * branch (from `next`)
* do the work in Next Step * do the work in Next Step
* move Next Step to the top of Completed Steps * move Next Step to the top of Completed Steps
* move the top item of Future Steps into Next Step * move the top item of Future Steps into Next Step
* commit (`TODO.md` changes in the same commit as the work) * commit (`TODO.md` changes in the same commit as the work)
* merge to `main` if the branch is not protected, otherwise open a PR
* push * push
* open a PR against `next`
# Status # Status
pre-1.0. No git tags. pre-1.0. No git tags. Work lands on `next` by PR. Open work for 1.0 is tracked
on the 1.0 milestone: https://git.eeqj.de/sneak/dnswatcher/milestone/7
# Next Step # Next Step
Add the README sections required by policy (Description, Getting Started, NS failure and NS recovery notifications:
Rationale, Design, TODO, License, Author) if any are still missing. https://git.eeqj.de/sneak/dnswatcher/issues/104
# Completed Steps # Completed Steps
- 2026-10-01: `TODO.md` brought up to date: open issues listed by URL, every
Completed Steps entry cut to at most two lines (closes #146).
- 2026-10-01: wildcard CORS now applies only to the public routes, not to - 2026-10-01: wildcard CORS now applies only to the public routes, not to
`/metrics`, and allows only the methods they serve (closes #100). `/metrics`, and allows only the methods they serve (closes #100).
- 2026-10-01: `internal/state` and `internal/watcher` no longer export test-only - 2026-10-01: `internal/state` and `internal/watcher` no longer export test-only
@@ -27,259 +30,83 @@ Rationale, Design, TODO, License, Author) if any are still missing.
the bound they check, and require the drain's debug line (closes #116). the bound they check, and require the drain's debug line (closes #116).
- 2026-09-29: the entrypoint chowns the data directory to `dnswatcher` and runs - 2026-09-29: the entrypoint chowns the data directory to `dnswatcher` and runs
dnswatcher as that user, so a host bind mount needs no chown (closes #166). dnswatcher as that user, so a host bind mount needs no chown (closes #166).
- 2026-09-29: the live-DNS test package is renamed `internal/livednstest` and - 2026-09-29: the live-DNS test package is renamed `internal/livednstest`;
added to the `test-support` `deny` list in `.golangci.yml`, so `make lint` `make lint` fails when program code imports it (closes #164).
fails when program code imports it (closes #164). - 2026-09-29: `.golangci.yml` re-fetched from `sneak/prompts`, with
- 2026-09-29: `.golangci.yml` re-fetched unchanged from `sneak/prompts`. It `gomodguard_v2` and the org `depguard` `test-support` rule (closes #123).
replaces the deprecated `gomodguard` with `gomodguard_v2`, so `make lint` no - 2026-09-29: watcher and resolver tests that look something up in DNS use the
longer warns about it, and turns on `depguard` with the org `test-support` real resolver against live DNS servers (closes #159).
rule, which rejects `net/http/httptest` except in test files and in files - 2026-09-28: the inconsistency alert is sent once, when two nameservers start
under a directory whose name ends in `test`. This repo had no `deny` entries to disagree; every pair of nameservers is compared (closes #158).
of its own to carry forward (closes #123).
- 2026-09-29: nothing stands in for DNS any more. Watcher tests that look
something up in DNS use the real resolver against live DNS servers and test
record and nameserver changes by preparing the saved state a check starts
from; the resolver timeout test queries an address that never answers, and
`NewFromLoggerWithClient`, used only by its stand-in client, is gone. The
live-DNS retry and concurrency limit moved to `internal/livednstest`, which
both test packages use. `TESTING.md` states the README's rule (closes #159).
- 2026-09-28: the inconsistency alert is sent once, on the check where two
nameservers start to disagree or where a nameserver that disagrees first
appears, instead of on every check while they disagree, and not again after
a restart. Every pair of nameservers is compared, not only neighbours in
sorted order of name (closes #158).
- 2026-09-28: DNS names in record values (CNAME, MX, SRV and NS targets) are - 2026-09-28: DNS names in record values (CNAME, MX, SRV and NS targets) are
lower-cased, so nameservers that answer in different letter case no longer lower-cased, so letter case alone is not a change (closes #157).
count as inconsistent or as a record change (closes #157). - 2026-09-28: lint and tests run on every build: `script/cibuild` and
- 2026-09-28: `script/cibuild` and `script/docker` now pass `script/docker` pass `--no-cache-filter=lint,builder` (closes #115).
`--no-cache-filter=lint,builder` so lint and tests run every build (closes - 2026-09-28: the server timeout test drives `Run` and checks the timeouts on
#115). the `http.Server` it serves (closes #120).
- 2026-09-28: the server timeout test now drives `Run` and checks the - 2026-09-28: upaas deploy readiness: the image runs as user `dnswatcher` with a
`http.Server` it serves carries the timeouts; corrected the `ReadTimeout` `HEALTHCHECK`; README "Running under upaas" (closes #147).
note in that test (closes #120).
- 2026-09-28: upaas deploy readiness — runtime image runs as unprivileged
`dnswatcher`, Docker `HEALTHCHECK`, startup fails when the data directory is
not writable, README "Running under upaas" (closes #147).
- 2026-09-21: added behavioural tests for `internal/globals`, - 2026-09-21: added behavioural tests for `internal/globals`,
`internal/healthcheck`, and `internal/logger` (closes #110). `internal/healthcheck`, and `internal/logger` (closes #110).
- 2026-09-21: `go mod tidy` dropped the redundant `golang.org/x/sync` - 2026-09-21: `go mod tidy` dropped the redundant `golang.org/x/sync`
`// indirect` line so `script/bootstrap` leaves a clean tree (#132) `// indirect` line so `script/bootstrap` leaves a clean tree (#132)
- 2026-08-10: comment-only corrections to `script/bootstrap`, - 2026-08-10: comment-only corrections to `script/bootstrap`, `script/cibuild`
`script/cibuild`, and `Dockerfile.lint`. The `goimports` pin in and `Dockerfile.lint`; no behaviour changed.
`script/bootstrap` was justified by a claim that `script/fmt-check` - 2026-08-10: MIT `LICENSE` added at the repository root; the README's first
runs it on the host; it does not (it runs `gofmt -l .` only), so the line and License section name the licence.
header now credits `script/fmt` alone. `script/cibuild` still claimed - 2026-08-10: policy scaffold present: `REPO_POLICIES.md`, `.editorconfig`,
the `Dockerfile` runs `make check`, which stopped being true when `.dockerignore`, CI workflow, `make fmt-check`, `make docker`, `make hooks`.
linting moved to its own stage; it now describes the lint stage - 2026-08-10: Go's test cache disabled in `script/test` (`-count=1`), so every
(`make fmt-check` plus `golangci-lint`) and the builder stage run queries live DNS; a failed run is rerun with `-v`.
(`make test`, `make build`). The `docker`-missing warning in - 2026-08-10: live-DNS tests made robust rather than gated (#93): a limit on
`script/bootstrap` reads as one sentence instead of three fragments concurrent lookups, retries, and a quorum across nameservers.
each re-prefixed with `bootstrap:`. `Dockerfile.lint` now records the - 2026-08-10: all linting moved into Docker: `script/lint` builds
residual risk of omitting `golangci-lint config verify`: unknown `Dockerfile.lint`, and the root `Dockerfile` has its own lint stage.
top-level keys in `.golangci.yml` are silently ignored, so a mistyped - 2026-08-09: in-flight notification deliveries are drained at shutdown, bounded
key lints clean while applying nothing. No behaviour changed by the shutdown deadline (#106).
- 2026-08-10: MIT `LICENSE` added at the repository root, closing the - 2026-08-09: `http.Server` sets all four socket timeouts; `WriteTimeout` stays
last gap in `REPO_POLICIES.md`'s required-minimum file list and above the 60s handler timeout (#99).
removing the all-rights-reserved default that would otherwise have - 2026-08-09: `SecurityHeaders()` middleware sets HSTS, CSP and the other
shipped with a 1.0 tag. The licence choice is the standing org policy security headers `REPO_POLICIES.md` requires on every response.
(any public repo lacking a licence gets MIT; a private repo with no - 2026-08-07: golangci-lint bumped to v2.12.2 and `.golangci.yml` set to the org
licence is already all-rights-reserved), and this repo is public. The config; fixed the resulting `goconst`, `dupl` and `lll` findings.
file holds the canonical MIT text byte-for-byte with only the - 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints, Makefile
copyright line filled in (`Copyright (c) 2026 sneak`); no clauses were shims, README Entrypoints section
added, removed, or reflowed. `README.md`'s first line now names the - 2026-02-20: iterative DNS resolver implemented
licence, as the Description requirement demands, and the License - 2026-02-20: CI actions and go install refs pinned to commit SHAs; Gitea
section states MIT and points at the file instead of saying the choice Actions workflow added
is pending. `make fmt` covers only Go sources (`gofmt -s`,
`goimports`), so it cannot reflow `LICENSE`
- 2026-08-10: the policy scaffold (`REPO_POLICIES.md`, `.editorconfig`,
`.dockerignore`, `.gitea/workflows/check.yml`, and the `fmt-check`,
`docker`, and hooks Makefile targets) is present; it landed piecemeal
across the scripts-to-rule-them-all and policy commits rather than as
the single commit this file once planned
- 2026-08-10: Go's test cache disabled for `script/test` via `-count=1`,
so every invocation actually executes. A cached pass replays an
earlier run's output without querying DNS at all, which in this repo
means the suite's entire premise goes unexercised while the run
reports green in under a second. The conditional verbose rerun that
`REPO_POLICIES.md` mandates was added at the same time (the primary
run had been unconditionally `-v`): quiet first, `-v` only on
failure, `-count=1` on both, and exit 1 forced regardless of the
rerun's result so a flake passing the second time cannot turn the
build green. `-timeout 90s` left alone as the deliberate backstop
above the 60s hard cap. Uncached suite runs ~4s, well inside the 20s
target
- 2026-08-10: live-DNS test flakiness addressed by robustness rather
than gating, per the owner's ruling on #93: new
`internal/resolver/livedns_test.go` adds a package-wide concurrency
gate (so parallel tests stop bursting at the first root server),
retry with exponential backoff on transport failures only, and
quorum instead of unanimity for multi-nameserver assertions. Quorum
tolerates silence only: every per-nameserver status must be in a
closed allowlist (`ok`/`timeout`/`error`, or
`nxdomain`/`timeout`/`error`), so a wrong answer from a minority —
`nodata` today, any status added later — fails the test instead of
sliding through under the majority. The
`make test` cap moved to the new org-wide 60s hard cap / 20s target
with a 90s `-timeout` backstop; `REPO_POLICIES.md` re-vendored
byte-identical from `sneak/prompts`. No mocks, no `-short`, no build
tags, no skips, and no change to production resolver behaviour
- 2026-08-10: all linting moved into Docker: new root `Dockerfile.lint`
on the digest-pinned `golangci/golangci-lint:v2.12.2` image,
`script/lint` reduced to a thin wrapper that builds it with
`--no-cache-filter=lint` so the linter actually executes every run,
golangci-lint install dropped from `script/bootstrap` (goimports
stays, `script/fmt` needs it on the host), and the root `Dockerfile`
given its own lint stage so its build no longer recurses through
`make check` into `script/lint`. `golangci-lint config verify` is
deliberately omitted: it fetches its schema over an unpinned live
HTTPS call
- 2026-08-09: in-flight notification deliveries are now drained at
shutdown (#106): `notify.New` registers an fx `OnStop` hook that waits
on a `sync.WaitGroup` of tracked delivery goroutines, bounded by the
`OnStop` context; on expiry the outstanding count is logged at warn
level and parked retry backoffs are released instead of being dropped
silently, and deliveries submitted after the drain begins are refused
so shutdown cannot be extended indefinitely; an `OnStop` context that
is already expired on entry with nothing outstanding drains quietly
rather than warning about deliveries that were never abandoned
- 2026-08-09: `http.Server` now sets all four socket-level timeouts
(`ReadTimeout` 15s, `ReadHeaderTimeout` 10s, `WriteTimeout` 75s,
`IdleTimeout` 120s) as named constants in `internal/server/server.go`,
closing the slowloris / unreaped-keep-alive exposure required by
`REPO_POLICIES.md` before 1.0; `WriteTimeout` is deliberately greater
than the 60s `chimw.Timeout` handler budget so that budget stays
reachable, and tests in `internal/server` pin both the non-zero
values and that relationship (#99)
- 2026-08-09: security response headers middleware
(`SecurityHeaders()` in `internal/middleware/middleware.go`)
registered globally in `internal/server/routes.go`, so HSTS, CSP,
`X-Frame-Options`, `X-Content-Type-Options`, `Referrer-Policy`, and
`Permissions-Policy` are set on every response including `/s/...` and
`/metrics`; the CSP needs no `unsafe-inline`/`unsafe-eval` because the
dashboard ships no JavaScript and no inline styles; HSTS is emitted
unconditionally per policy (TLS-terminating proxy in front). Remaining
1.0 hardening items — `http.Server` timeouts, request body limits,
rate limiting, CORS scoping — are tracked separately
- 2026-08-07: golangci-lint bumped to v2.12.2 (commit-pinned installs
in `Dockerfile` and `script/bootstrap`); `.golangci.yml` set to the
org-standard v2-schema config used across the org's repos
(owner-authorized; same file is being landed as canonical via prompts
PR #24), with settings under `linters.settings` so the
lll/funlen/cyclop/dupl thresholds apply; fixed the resulting
`goconst`, `dupl`, and `lll` findings; the informational `gomodguard`
deprecation warning under this config is accepted
- 2026-07-07 Adopted scripts-to-rule-them-all: `script/` entrypoints,
Makefile shims, README Entrypoints section
- 2026-02-20: iterative DNS resolver implemented; tests made hermetic
with mocked DNS (origin/feature/resolver, unmerged)
- 2026-02-20: CI actions and go install refs pinned to commit SHAs;
Gitea Actions workflow for make check (origin/ci/make-check, unmerged)
- 2026-02-20: watcher monitoring orchestrator merged to main (#8) - 2026-02-20: watcher monitoring orchestrator merged to main (#8)
- 2026-02-20: DOMAINS/HOSTNAMES unified into single TARGETS config (#11) - 2026-02-20: DOMAINS/HOSTNAMES unified into single TARGETS config (#11)
- 2026-02-19: TCP port connectivity checker, made concurrent with port - 2026-02-19: TCP port connectivity checker, made concurrent with port
validation; gosec G704 SSRF findings fixed without suppression validation; gosec G704 SSRF findings fixed without suppression
(feature branches, unmerged) - 2026-02-19: TLS certificate inspector with no-peer-certificates error path and
- 2026-02-19: TLS certificate inspector with no-peer-certificates error IP SANs
path and IP SANs (feature branch, unmerged)
- 2026-02-19: gosec SSRF and formatting fixes on main - 2026-02-19: gosec SSRF and formatting fixes on main
- 2026-02-19: initial scaffold with per-nameserver DNS monitoring model - 2026-02-19: initial scaffold with per-nameserver DNS monitoring model
# Future Steps # Future Steps
Compliance: - nameserver IP address changes: https://git.eeqj.de/sneak/dnswatcher/issues/105
- `DNSWATCHER_SENTRY_DSN` does nothing:
- Pin Dockerfile base images by sha256 and ensure the Docker build runs https://git.eeqj.de/sneak/dnswatcher/issues/107
make check - invalid DNS or TLS interval silently replaced by the default:
https://git.eeqj.de/sneak/dnswatcher/issues/177
Branch reconciliation: - rate limit on `/metrics` Basic Auth:
https://git.eeqj.de/sneak/dnswatcher/issues/101
- Sync local checkout with origin: local main is 8 commits behind - images report version `dev`: https://git.eeqj.de/sneak/dnswatcher/issues/109
origin/main; local feature/resolver has diverged from - trial run of the finished image:
origin/feature/resolver, which already implements the resolver https://git.eeqj.de/sneak/dnswatcher/issues/149
- Merge in-flight branches to main once green: feature/resolver, - 1.0 readiness: run it with a real config and read the logs:
ci/make-check, feature/portcheck-implementation, https://git.eeqj.de/sneak/dnswatcher/issues/66
feature/tlscheck-implementation - `goimports` in `make fmt-check`, Markdown formatting:
https://git.eeqj.de/sneak/dnswatcher/issues/119
Resolver (plan from untracked TODO.md; largely implemented on - final state save at shutdown: https://git.eeqj.de/sneak/dnswatcher/issues/114
origin/feature/resolver, verify each item before closing): - `internal/notify` shutdown tests hang when a drain returns early:
https://git.eeqj.de/sneak/dnswatcher/issues/176
- Add github.com/miekg/dns dependency - README accuracy sweep: https://git.eeqj.de/sneak/dnswatcher/issues/108
- roots.go: hardcoded IANA root server list (a through m, IPv4/IPv6), - README sections required by policy:
rootServers() returning ip:53 strings https://git.eeqj.de/sneak/dnswatcher/issues/173
- query.go: low-level query(ctx, server, name, qtype): UDP with TCP - `script/install-precommit` in a linked worktree:
fallback on truncation, RD=0, context respected, 5s per-query timeout, https://git.eeqj.de/sneak/dnswatcher/issues/129
returns raw *dns.Msg - fixed root server order: https://git.eeqj.de/sneak/dnswatcher/issues/138
- trace.go: iterative delegation chasing from roots: referral detection - review toward 1.0: https://git.eeqj.de/sneak/dnswatcher/issues/144
(NOERROR, empty answer, NS in authority), glue extraction with
bailiwick check, out-of-bailiwick NS resolved with recursion guard,
delegation depth limit (20), retry across nameservers on failure, do
not chase CNAMEs inside trace
- FindAuthoritativeNameservers: NS set via trace, sorted, FQDN
normalized, trailing dot handled; must pass its 9 tests
- QueryNameserver: resolve NS host to IPs, query A/AAAA/CNAME/MX/TXT/
SRV/CAA/NS, build NameserverResponse with status mapping (OK,
NXDomain, NoData, Error), documented record formatting, sorted values,
lame delegation detection; must pass its 16 tests
- QueryAllNameservers: find NS set for parent domain (public suffix
list), query all NS in parallel with bounded concurrency, return map
even when all fail, context cancellation; must pass its 4 tests
- LookupNS: thin wrapper over FindAuthoritativeNameservers, sorted,
identical results; must pass its 3 tests
- ResolveIPAddresses: collect A/AAAA from all NS, follow CNAME chains
with MaxCNAMEDepth, dedupe, sort, NXDOMAIN returns empty slice with
nil error; must pass its 9 tests
- All 39 resolver tests pass, make check green, merge to main
Watcher (internal/watcher/watcher.go):
- Scheduling loop in Run(ctx): initial check on startup, separate
tickers for DNS/port and TLS intervals, persist state via state.Save()
after each cycle, clean shutdown on context cancel
- Domain check: LookupNS, compare to stored state, store silently on
first run, notify with old/new NS lists on change
- Hostname check: QueryAllNameservers, compare per-NS records; notify on
record changes, NS failure, NS recovery, inconsistency detected,
inconsistency resolved, empty response; store silently on first run
- Port check: ResolveIPAddresses, check ports 80 and 443 per IP, notify
on open/closed transitions, handle new and disappeared IPs
- TLS check: for each open IP:443, CheckCertificate; notify on expiry
warning, certificate change (CN/issuer/SANs), TLS failure/recovery
Port checker (internal/portcheck/portcheck.go):
- Tests against known-open ports and RFC documentation IPs
- CheckPort: net.DialTimeout (5s), context respected; (true, nil) open,
(false, nil) closed/timeout/refused, error only for unexpected
failures
TLS checker (internal/tlscheck/tlscheck.go):
- Tests against known public HTTPS servers, verify fields populated
- CheckCertificate: tls.Dial to specific IP:443 with hostname as SNI;
extract subject CN, issuer CN and org, NotAfter, SANs; error on
handshake failure
Notification service (internal/notify/notify.go, Slack/Mattermost/ntfy
backends exist):
- Structured notification types: DNS change, port change, TLS expiry,
TLS change, NS failure, NS recovery, NS inconsistency
- Per-backend formatting: Slack/Mattermost attachment colors (red
failures/expiry, yellow warnings, green recoveries, blue info); ntfy
priorities (urgent failures, high warnings, default changes, low
recoveries); include hostname, nameserver, old/new values, timestamps
HTTP API handlers:
- Wire *state.State and *watcher.Watcher into handler params
- GET /api/v1/status: full state snapshot as JSON
- GET /api/v1/domains: domain states with NS records and last-checked
- GET /api/v1/hostnames: hostname states with per-NS record data
Infrastructure notes (from untracked TODO.md):
- Module path sneak.berlin/go/dnswatcher differs from the git.eeqj.de
remote intentionally; do not "fix" it
- Dependencies: github.com/miekg/dns, golang.org/x/net/publicsuffix
- DNS is never mocked; tests that look something up in DNS query live DNS
servers (README, "No DNS mocking. Ever.")