Compare commits
11
Commits
prod
..
df80bc1fb6
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
df80bc1fb6 | ||
|
|
b351a2350c | ||
|
|
1ffe303a6e | ||
|
|
fc43f893a5 | ||
|
|
ae06f7e3a1 | ||
|
|
b8662b8a9c | ||
|
|
168281ad60 | ||
|
|
6f6bf3a65b | ||
|
|
87bce43f8d | ||
|
|
9cb2c2b7e0 | ||
|
|
cc86473410 |
+22
-6
@@ -41,15 +41,31 @@ FROM alpine@sha256:c3f8e73fdb79deaebaa2037150150191b9dcbfba68b4a46d70103204c53f4
|
|||||||
|
|
||||||
RUN apk add --no-cache ca-certificates tzdata
|
RUN apk add --no-cache ca-certificates tzdata
|
||||||
|
|
||||||
WORKDIR /app
|
# The binary lives in /usr/local/bin, not the working directory: config
|
||||||
|
# loading searches the working directory, and with a YAML config type Viper
|
||||||
|
# also matches an extension-less file named "dnswatcher" there, so a binary
|
||||||
|
# named "dnswatcher" in that directory would be parsed as a config file.
|
||||||
|
COPY --from=builder /src/bin/dnswatcher /usr/local/bin/dnswatcher
|
||||||
|
|
||||||
COPY --from=builder /src/bin/dnswatcher /app/dnswatcher
|
# Run as an unprivileged user. The data directory is owned by that user so
|
||||||
|
# writes succeed both on a bind mount and when Docker seeds a fresh named
|
||||||
# Create data directory
|
# volume from the image (a fresh volume inherits this directory's ownership).
|
||||||
RUN mkdir -p /var/lib/dnswatcher
|
RUN addgroup -S dnswatcher \
|
||||||
|
&& adduser -S -G dnswatcher -u 10001 dnswatcher \
|
||||||
|
&& mkdir -p /var/lib/dnswatcher \
|
||||||
|
&& chown dnswatcher:dnswatcher /var/lib/dnswatcher
|
||||||
|
|
||||||
ENV DNSWATCHER_DATA_DIR=/var/lib/dnswatcher
|
ENV DNSWATCHER_DATA_DIR=/var/lib/dnswatcher
|
||||||
|
|
||||||
|
WORKDIR /var/lib/dnswatcher
|
||||||
|
|
||||||
|
USER dnswatcher
|
||||||
|
|
||||||
EXPOSE 8080
|
EXPOSE 8080
|
||||||
|
|
||||||
ENTRYPOINT ["/app/dnswatcher"]
|
# busybox wget (already in alpine) probes the health endpoint. PORT defaults
|
||||||
|
# to 8080 and is honoured if the operator overrides it.
|
||||||
|
HEALTHCHECK --interval=30s --timeout=5s --start-period=10s --retries=3 \
|
||||||
|
CMD wget -q -O /dev/null "http://127.0.0.1:${PORT:-8080}/.well-known/healthcheck" || exit 1
|
||||||
|
|
||||||
|
ENTRYPOINT ["/usr/local/bin/dnswatcher"]
|
||||||
|
|||||||
@@ -182,6 +182,27 @@ dnswatcher exposes a lightweight HTTP API for operational visibility:
|
|||||||
| `GET /api/v1/status` | Current monitoring state |
|
| `GET /api/v1/status` | Current monitoring state |
|
||||||
| `GET /metrics` | Prometheus metrics (optional) |
|
| `GET /metrics` | Prometheus metrics (optional) |
|
||||||
|
|
||||||
|
#### Server timeouts
|
||||||
|
|
||||||
|
The HTTP server sets all four socket-level timeouts. These are compile-time
|
||||||
|
constants in `internal/server/server.go`, not configurable via environment
|
||||||
|
variables.
|
||||||
|
|
||||||
|
| Timeout | Value | Purpose |
|
||||||
|
|---------------------|-------|-----------------------------------------------|
|
||||||
|
| `ReadHeaderTimeout` | 10s | Bounds the request header read (slowloris) |
|
||||||
|
| `ReadTimeout` | 15s | Bounds the whole request read, headers + body |
|
||||||
|
| `WriteTimeout` | 75s | Bounds handler execution plus response flush |
|
||||||
|
| `IdleTimeout` | 120s | Reaps idle keep-alive connections |
|
||||||
|
|
||||||
|
These are distinct from the 60s per-request handler budget applied by
|
||||||
|
`chimw.Timeout` in `internal/server/routes.go`, which cancels the request
|
||||||
|
context but does not touch the socket. `WriteTimeout` is deliberately
|
||||||
|
larger than that budget: the write deadline is armed once request headers
|
||||||
|
are read, so a smaller value would sever the connection before a handler
|
||||||
|
using its full budget could respond. `IdleTimeout` exceeds common
|
||||||
|
Prometheus scrape intervals so the scraper reuses its connection.
|
||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
## Architecture
|
## Architecture
|
||||||
@@ -446,6 +467,35 @@ docker run -d \
|
|||||||
|
|
||||||
---
|
---
|
||||||
|
|
||||||
|
## Deploying with upaas
|
||||||
|
|
||||||
|
upaas builds this repo's `Dockerfile` and runs the resulting image with the
|
||||||
|
volume, port, and environment variables the operator enters. Configure:
|
||||||
|
|
||||||
|
- **Branch:** `main`. This is the recommended branch on the deployment
|
||||||
|
decision issue and is not yet confirmed by the owner.
|
||||||
|
- **Volume:** mount persistent storage at `/var/lib/dnswatcher`. The image runs
|
||||||
|
as an unprivileged user (`dnswatcher`, uid 10001) that owns this directory; a
|
||||||
|
fresh named volume inherits that ownership, and a bind-mounted host directory
|
||||||
|
must be writable by uid 10001.
|
||||||
|
- **Port:** the container listens on `8080`. Override with `PORT` if needed;
|
||||||
|
the healthcheck honours it.
|
||||||
|
- **Required environment:** `DNSWATCHER_TARGETS`, a comma-separated list of the
|
||||||
|
domains and hostnames to watch. The process refuses to start without it.
|
||||||
|
- **Recommended environment:** the notification endpoints
|
||||||
|
`DNSWATCHER_SLACK_WEBHOOK`, `DNSWATCHER_MATTERMOST_WEBHOOK`, and
|
||||||
|
`DNSWATCHER_NTFY_TOPIC` (without at least one, changes are only visible on the
|
||||||
|
dashboard); and `DNSWATCHER_METRICS_USERNAME` / `DNSWATCHER_METRICS_PASSWORD`
|
||||||
|
to enable the basic-auth-protected `/metrics` endpoint.
|
||||||
|
- **Healthcheck:** the image already declares a Docker `HEALTHCHECK` against
|
||||||
|
`/.well-known/healthcheck`; no operator configuration is needed. The same
|
||||||
|
path is available for an external probe.
|
||||||
|
|
||||||
|
The dashboard is unauthenticated and shows every watched name and recent alert;
|
||||||
|
decide deliberately whether to expose it publicly.
|
||||||
|
|
||||||
|
---
|
||||||
|
|
||||||
## Monitoring Lifecycle
|
## Monitoring Lifecycle
|
||||||
|
|
||||||
1. **Startup**: Load state from disk. If no state file exists, start
|
1. **Startup**: Load state from disk. If no state file exists, start
|
||||||
|
|||||||
@@ -23,6 +23,10 @@ Rationale, Design, TODO, License, Author) if any are still missing.
|
|||||||
|
|
||||||
# Completed Steps
|
# Completed Steps
|
||||||
|
|
||||||
|
- 2026-09-21: upaas deploy readiness — runtime image runs as unprivileged
|
||||||
|
`dnswatcher`, Docker `HEALTHCHECK`, README "Deploying with upaas" (closes #147).
|
||||||
|
- 2026-09-21: `go mod tidy` dropped the redundant `golang.org/x/sync`
|
||||||
|
`// indirect` line so `script/bootstrap` leaves a clean tree (#132)
|
||||||
- 2026-08-10: comment-only corrections to `script/bootstrap`,
|
- 2026-08-10: comment-only corrections to `script/bootstrap`,
|
||||||
`script/cibuild`, and `Dockerfile.lint`. The `goimports` pin in
|
`script/cibuild`, and `Dockerfile.lint`. The `goimports` pin in
|
||||||
`script/bootstrap` was justified by a claim that `script/fmt-check`
|
`script/bootstrap` was justified by a claim that `script/fmt-check`
|
||||||
@@ -101,6 +105,14 @@ Rationale, Design, TODO, License, Author) if any are still missing.
|
|||||||
so shutdown cannot be extended indefinitely; an `OnStop` context that
|
so shutdown cannot be extended indefinitely; an `OnStop` context that
|
||||||
is already expired on entry with nothing outstanding drains quietly
|
is already expired on entry with nothing outstanding drains quietly
|
||||||
rather than warning about deliveries that were never abandoned
|
rather than warning about deliveries that were never abandoned
|
||||||
|
- 2026-08-09: `http.Server` now sets all four socket-level timeouts
|
||||||
|
(`ReadTimeout` 15s, `ReadHeaderTimeout` 10s, `WriteTimeout` 75s,
|
||||||
|
`IdleTimeout` 120s) as named constants in `internal/server/server.go`,
|
||||||
|
closing the slowloris / unreaped-keep-alive exposure required by
|
||||||
|
`REPO_POLICIES.md` before 1.0; `WriteTimeout` is deliberately greater
|
||||||
|
than the 60s `chimw.Timeout` handler budget so that budget stays
|
||||||
|
reachable, and tests in `internal/server` pin both the non-zero
|
||||||
|
values and that relationship (#99)
|
||||||
- 2026-08-07: golangci-lint bumped to v2.12.2 (commit-pinned installs
|
- 2026-08-07: golangci-lint bumped to v2.12.2 (commit-pinned installs
|
||||||
in `Dockerfile` and `script/bootstrap`); `.golangci.yml` set to the
|
in `Dockerfile` and `script/bootstrap`); `.golangci.yml` set to the
|
||||||
org-standard v2-schema config used across the org's repos
|
org-standard v2-schema config used across the org's repos
|
||||||
|
|||||||
@@ -40,7 +40,6 @@ require (
|
|||||||
go.yaml.in/yaml/v2 v2.4.2 // indirect
|
go.yaml.in/yaml/v2 v2.4.2 // indirect
|
||||||
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
go.yaml.in/yaml/v3 v3.0.4 // indirect
|
||||||
golang.org/x/mod v0.32.0 // indirect
|
golang.org/x/mod v0.32.0 // indirect
|
||||||
golang.org/x/sync v0.19.0 // indirect
|
|
||||||
golang.org/x/sys v0.41.0 // indirect
|
golang.org/x/sys v0.41.0 // indirect
|
||||||
golang.org/x/text v0.34.0 // indirect
|
golang.org/x/text v0.34.0 // indirect
|
||||||
golang.org/x/tools v0.41.0 // indirect
|
golang.org/x/tools v0.41.0 // indirect
|
||||||
|
|||||||
@@ -0,0 +1,19 @@
|
|||||||
|
package server
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"time"
|
||||||
|
)
|
||||||
|
|
||||||
|
// NewHTTPServer exports newHTTPServer for testing.
|
||||||
|
func NewHTTPServer(
|
||||||
|
listenAddr string,
|
||||||
|
handler http.Handler,
|
||||||
|
) *http.Server {
|
||||||
|
return newHTTPServer(listenAddr, handler)
|
||||||
|
}
|
||||||
|
|
||||||
|
// RequestTimeout exports the handler execution budget applied by
|
||||||
|
// chimw.Timeout in SetupRoutes, so tests can assert the relationship
|
||||||
|
// between it and the server's WriteTimeout.
|
||||||
|
const RequestTimeout time.Duration = requestTimeout
|
||||||
@@ -33,8 +33,52 @@ type Params struct {
|
|||||||
// shutdownTimeout is how long to wait for graceful shutdown.
|
// shutdownTimeout is how long to wait for graceful shutdown.
|
||||||
const shutdownTimeout = 30 * time.Second
|
const shutdownTimeout = 30 * time.Second
|
||||||
|
|
||||||
// readHeaderTimeout is the max duration for reading request headers.
|
// Socket-level timeouts for the HTTP server.
|
||||||
const readHeaderTimeout = 10 * time.Second
|
//
|
||||||
|
// These bound time spent on the connection itself and are a distinct
|
||||||
|
// control from the per-request handler budget enforced by
|
||||||
|
// chimw.Timeout(requestTimeout) in routes.go: that one cancels the
|
||||||
|
// request context after requestTimeout but never touches the socket,
|
||||||
|
// so without the values below a peer can hold a connection open
|
||||||
|
// forever (slowloris, unreaped keep-alives).
|
||||||
|
//
|
||||||
|
// The one hard constraint between the two controls is
|
||||||
|
// writeTimeout > requestTimeout. net/http arms the write deadline
|
||||||
|
// once the request headers have been read, so on a plaintext
|
||||||
|
// connection it covers handler execution AND the response flush. If
|
||||||
|
// writeTimeout were <= requestTimeout the server would sever the
|
||||||
|
// connection before a handler that legitimately consumed its full
|
||||||
|
// budget could emit anything, making the 60s budget unreachable in
|
||||||
|
// practice. The margin between them is the response-flush allowance.
|
||||||
|
//
|
||||||
|
// The only clients of this service are browsers loading the dashboard
|
||||||
|
// and a Prometheus scraper; the values are sized for those.
|
||||||
|
const (
|
||||||
|
// readHeaderTimeout is the max duration for reading request
|
||||||
|
// headers.
|
||||||
|
readHeaderTimeout = 10 * time.Second
|
||||||
|
|
||||||
|
// readTimeout bounds reading the entire request, headers plus
|
||||||
|
// body. Every route here is a GET with no body, so this only
|
||||||
|
// ever needs to cover headers; the extra 5s over
|
||||||
|
// readHeaderTimeout is slack, not a real allowance, and keeps a
|
||||||
|
// body dribbled one byte at a time from holding the read side
|
||||||
|
// open indefinitely.
|
||||||
|
readTimeout = 15 * time.Second
|
||||||
|
|
||||||
|
// writeTimeout must exceed the requestTimeout handler budget
|
||||||
|
// (60s) per the note above. The 15s difference is the allowance
|
||||||
|
// for flushing a completed response to a slow client.
|
||||||
|
writeTimeout = 75 * time.Second
|
||||||
|
|
||||||
|
// idleTimeout reaps keep-alive connections between requests. It
|
||||||
|
// is deliberately longer than the common Prometheus scrape
|
||||||
|
// intervals (15s/30s/60s) so the scraper reuses its connection
|
||||||
|
// rather than reconnecting every cycle, while a browser tab
|
||||||
|
// left open on the dashboard stops occupying a connection
|
||||||
|
// within two minutes of going quiet.
|
||||||
|
idleTimeout = 120 * time.Second
|
||||||
|
)
|
||||||
|
|
||||||
// Server is the HTTP server.
|
// Server is the HTTP server.
|
||||||
type Server struct {
|
type Server struct {
|
||||||
@@ -76,16 +120,29 @@ func New(
|
|||||||
return srv, nil
|
return srv, nil
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// newHTTPServer builds the listening http.Server with every
|
||||||
|
// socket-level timeout set. All four are set deliberately: a zero
|
||||||
|
// value in net/http means "no limit", not "some default".
|
||||||
|
func newHTTPServer(
|
||||||
|
listenAddr string,
|
||||||
|
handler http.Handler,
|
||||||
|
) *http.Server {
|
||||||
|
return &http.Server{
|
||||||
|
Addr: listenAddr,
|
||||||
|
Handler: handler,
|
||||||
|
ReadTimeout: readTimeout,
|
||||||
|
ReadHeaderTimeout: readHeaderTimeout,
|
||||||
|
WriteTimeout: writeTimeout,
|
||||||
|
IdleTimeout: idleTimeout,
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
// Run starts the HTTP server.
|
// Run starts the HTTP server.
|
||||||
func (s *Server) Run() {
|
func (s *Server) Run() {
|
||||||
s.SetupRoutes()
|
s.SetupRoutes()
|
||||||
|
|
||||||
listenAddr := fmt.Sprintf(":%d", s.port)
|
listenAddr := fmt.Sprintf(":%d", s.port)
|
||||||
s.httpServer = &http.Server{
|
s.httpServer = newHTTPServer(listenAddr, s)
|
||||||
Addr: listenAddr,
|
|
||||||
Handler: s,
|
|
||||||
ReadHeaderTimeout: readHeaderTimeout,
|
|
||||||
}
|
|
||||||
|
|
||||||
s.log.Info("http server starting", "addr", listenAddr)
|
s.log.Info("http server starting", "addr", listenAddr)
|
||||||
|
|
||||||
|
|||||||
@@ -0,0 +1,112 @@
|
|||||||
|
package server_test
|
||||||
|
|
||||||
|
import (
|
||||||
|
"net/http"
|
||||||
|
"testing"
|
||||||
|
|
||||||
|
"sneak.berlin/go/dnswatcher/internal/server"
|
||||||
|
)
|
||||||
|
|
||||||
|
// noopHandler stands in for the router; newHTTPServer only stores it.
|
||||||
|
func noopHandler() http.Handler {
|
||||||
|
return http.HandlerFunc(
|
||||||
|
func(w http.ResponseWriter, _ *http.Request) {
|
||||||
|
w.WriteHeader(http.StatusOK)
|
||||||
|
},
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHTTPServerTimeoutsAreSet asserts that every socket-level
|
||||||
|
// timeout is configured. A zero value in net/http means "no limit",
|
||||||
|
// so a refactor that silently drops one of these reintroduces the
|
||||||
|
// slowloris / unreaped-keep-alive exposure this guards against.
|
||||||
|
//
|
||||||
|
// The assertions are on the configured field values only; nothing
|
||||||
|
// here measures elapsed time, so the test cannot flake on timing.
|
||||||
|
func TestHTTPServerTimeoutsAreSet(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
srv := server.NewHTTPServer(":8080", noopHandler())
|
||||||
|
|
||||||
|
if srv.ReadTimeout <= 0 {
|
||||||
|
t.Errorf(
|
||||||
|
"ReadTimeout must be non-zero, got %v",
|
||||||
|
srv.ReadTimeout,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if srv.ReadHeaderTimeout <= 0 {
|
||||||
|
t.Errorf(
|
||||||
|
"ReadHeaderTimeout must be non-zero, got %v",
|
||||||
|
srv.ReadHeaderTimeout,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if srv.WriteTimeout <= 0 {
|
||||||
|
t.Errorf(
|
||||||
|
"WriteTimeout must be non-zero, got %v",
|
||||||
|
srv.WriteTimeout,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
|
||||||
|
if srv.IdleTimeout <= 0 {
|
||||||
|
t.Errorf(
|
||||||
|
"IdleTimeout must be non-zero, got %v",
|
||||||
|
srv.IdleTimeout,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestWriteTimeoutExceedsHandlerBudget pins the one relationship the
|
||||||
|
// values must satisfy. net/http arms the write deadline once request
|
||||||
|
// headers are read, so it covers handler execution plus the response
|
||||||
|
// flush. If WriteTimeout were not greater than the chimw.Timeout
|
||||||
|
// handler budget, the connection would be severed before a handler
|
||||||
|
// that used its full budget could respond, making that budget
|
||||||
|
// unreachable.
|
||||||
|
func TestWriteTimeoutExceedsHandlerBudget(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
srv := server.NewHTTPServer(":8080", noopHandler())
|
||||||
|
|
||||||
|
if srv.WriteTimeout <= server.RequestTimeout {
|
||||||
|
t.Errorf(
|
||||||
|
"WriteTimeout (%v) must exceed handler budget (%v)",
|
||||||
|
srv.WriteTimeout,
|
||||||
|
server.RequestTimeout,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestReadTimeoutCoversHeaderTimeout asserts the read deadline for
|
||||||
|
// the whole request is at least as long as the header-only deadline;
|
||||||
|
// a smaller ReadTimeout would make ReadHeaderTimeout unreachable.
|
||||||
|
func TestReadTimeoutCoversHeaderTimeout(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
srv := server.NewHTTPServer(":8080", noopHandler())
|
||||||
|
|
||||||
|
if srv.ReadTimeout < srv.ReadHeaderTimeout {
|
||||||
|
t.Errorf(
|
||||||
|
"ReadTimeout (%v) must be >= ReadHeaderTimeout (%v)",
|
||||||
|
srv.ReadTimeout,
|
||||||
|
srv.ReadHeaderTimeout,
|
||||||
|
)
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// TestHTTPServerAddrAndHandler covers the rest of the constructor so
|
||||||
|
// a future edit cannot drop the listen address or the handler.
|
||||||
|
func TestHTTPServerAddrAndHandler(t *testing.T) {
|
||||||
|
t.Parallel()
|
||||||
|
|
||||||
|
srv := server.NewHTTPServer(":9999", noopHandler())
|
||||||
|
|
||||||
|
if srv.Addr != ":9999" {
|
||||||
|
t.Errorf("Addr = %q, want %q", srv.Addr, ":9999")
|
||||||
|
}
|
||||||
|
|
||||||
|
if srv.Handler == nil {
|
||||||
|
t.Error("Handler must not be nil")
|
||||||
|
}
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user